REMOTE SENSING AND GIS ACCURACY ASSESSMENT
CHAPTER 1 PUTTING THE MAP BACK IN MAP ACCURACY ASSESSMENT 1 CHAPTER 2 SAMPLING DESIGN FOR ACCURACY ASSESSMENT OF LARGE AREA LAND COVER MAPS CHALLENGES AND FUTURE DIRECTIONS 13 CHAPTER 3 VALIDATION OF GLOBAL LAND COVER PRODUCTS BY THE COMMITTEE ON EARTH OBSERVING SATELLITES 31 CHAPTER 4 IN SITU ESTIMATES OF FOREST LAI FOR MODIS DATA VALIDATION 41 CHAPTER 5 LIGHT ATTENUATION PRO LING AS AN INDICATOR OF STRUCTURAL CHANGES IN COASTAL MARSHES 59 CHAPTER 6 PARTICIPATORY REFERENCE DATA COLLECTION METHODS FOR ACCURACY ASSESSMENT OF LAND COVER CHANGE MAPS 75 CHAPTER 7 THEMATIC ACCURACY ASSESSMENT OF REGIONAL SCALE LAND COVER DATA 91 CHAPTER 8 AN INDEPENDENT RELIABILITY ASSESSMENT FOR THE AUSTRALIAN AGRICULTURAL LAND COVER CHANGE PROJECT 1990 91 1995 105 CHAPTER 9 ASSESSING THE ACCURACY OF SATELLITE DERIVED LAND COVER CLASSI CATION USING HISTORICAL AERIAL PHOTOGRAPHY DIGITAL ORTHOPHOTO QUADRANGLES AND AIRBORNE VIDEO DATA 115 CHAPTER 10 USING CLASSI CATION CONSISTENCY IN INTERSCENE OVERLAP AREAS TO MODEL SPATIAL VARIATIONS IN LAND COVER ACCURACY OVER LARGE GEOGRAPHIC REGIONS 132 CHAPTER 11 GEOSTATISTICAL MAPPING OF THEMATIC CLASSI CATION UNCERTAINTY 144 CONTENTS CHAPTER 12 AN ERROR MATRIX APPROACH TO FUZZY ACCURACY ASSESSMENT THE NIMA GEOCOVER PROJECT 162 CHAPTER 13 MAPPING SPATIAL ACCURACY AND ESTIMATING LANDSCAPE INDICATORS FROM THEMATIC LAND COVER MAPS USING FUZZY SET THEORY 172 CHAPTER 14 FUZZY SET AND SPATIAL ANALYSIS TECHNIQUES FOR EVALUATING THEMATIC ACCURACY OF A LAND COVER MAP 188 CHAPTER 15 THE EFFECTS OF CLASSI CATION ACCURACY ON LANDSCAPE INDICES 208 CHAPTER 1 PUTTING THE MAP BACK IN MAP ACCURACY ASSESSMENT 1 1 INTRODUCTION THE NEED FOR ASSESSING THE ACCURACY OF A MAP GENERATED FROM ANY REMOTELY SENSED DATA HAS BECOME UNIVERSALLY RECOGNIZED AS AN INTEGRAL PROJECT COMPONENT IN THE LAST FEW YEARS MOST PROJECTS HAVE REQUIRED THAT A CERTAIN LEVEL OF ACCURACY BE ACHIEVED FOR THE PROJECT AND MAP TO BE DEEMED A SUCCESS WITH THE WIDESPREAD APPLICATION OF GEOGRAPHIC INFORMATION SYSTEMS GIS EMPLOYING REMOTELY SENSED DATA AS LAYERS THE NEED FOR SUCH AN ASSESSMENT HAS BECOME EVEN MORE CRITICAL THERE ARE A NUMBER OF REASONS WHY THIS ASSESSMENT IS SO IMPORTANT INCLUDING 1 THE NEED TO PERFORM A SELF EVALUATION AND TO LEARN FROM YOUR MISTAKES THE ABILITY TO COMPARE METHOD ALGORITHMS ANALYSTS QUANTITATIVELY THE DESIRE TO USE THE RESULTING MAPS SPATIAL INFORMATION IN SOME DECISION MAKING PROCESS THERE ARE MANY EXAMPLES IN THE LITERATURE AS WELL AS AN OVERWHELMING SELECTION OF ANECDOTAL EVIDENCE TO DEMONSTRATE THE NEED FOR ACCURACY ASSESSMENT MANY DIFFERENT GROUPS HAVE MAPPED AND OR QUANTI ED THE AMOUNT OF TROPICAL DEFORESTATION OCCURRING IN SOUTH AMERICA OR SOUTHEAST ASIA ESTIMATES HAVE RANGED BY ALMOST AN ORDER OF MAGNITUDE WHICH ESTIMATE IS CORRECT WITHOUT A VALID ACCURACY ASSESSMENT WE MAY NEVER KNOW SEVERAL FEDERAL STATE AND LOCAL AGENCIES HAVE CREATED MAPS OF WETLANDS IN A COUNTY ON THE EASTERN SHORE OF MARYLAND TECHNIQUES USED TO MAKE THESE MAPS INCLUDED SATELLITE IMAGERY AERIAL PHOTOGRAPHY VARIOUS SCALES AND LM TYPES AND GROUND SAMPLING COMPARING THE VARIOUS MAPS YIELDED LITTLE AGREEMENT ABOUT WHERE WETLANDS ACTUALLY EXISTED WITHOUT A VALID ACCURACY ASSESSMENT WE MAY NEVER KNOW WHICH OF THESE MAPS TO USE IT IS NO LONGER SUF CIENT JUST TO MAKE A MAP USING REMOTELY SENSED OR OTHER SPATIAL DATA IT IS ABSOLUTELY NECESSARY TO TAKE SOME STEPS TOWARD ASSESSING THE ACCURACY OR VALIDITY OF THAT MAP THERE ARE A NUMBER OF WAYS TO INVESTIGATE THE ACCURACY ERROR IN SPATIAL DATA INCLUDING BUT NOT LIMITED TO VISUAL INSPECTION NONSITE SPECI C ANALYSIS GENERATING DIFFERENCE IMAGES ERROR BUDGET ANALYSIS AND QUANTITATIVE ACCURACY ASSESSMENT THE GOAL OF THIS CHAPTER IS TO REVIEW THE CURRENT KNOWLEDGE OF ACCURACY ASSESSMENT METHODS AND STIMULATE THE READER TO FURTHER THE PROGRESSION OF DIAGNOSTIC TECHNIQUES AND INFORMATION TO SUPPORT THE APPROPRIATE APPLICATION OF SPATIAL DATA THE ULTIMATE OBJECTIVE IS TO MOTIVATE EVERYONE TO CONDUCT OR DEMAND AN APPROPRIATE ACCURACY ASSESSMENT OR VALIDATION AND MAKE CERTAIN IT IS INCLUDED AS AN ESSENTIAL METADATA ELEMENT 1 2 ACCURACY ASSESSMENT OVERVIEW 1 2 1 HISTORICAL REVIEW THE HISTORY OF ACCURACY ASSESSMENT OF DIGITAL REMOTELY SENSED DATA IS RELATIVELY SHORT BEGINNING IN ABOUT 1975 BEFORE 1975 MAPS DERIVED FROM ANALOG REMOTELY SENSED DATA I E PHOTO INTERPRE TATION WERE RARELY SUBJECTED TO ANY KIND OF QUANTITATIVE ACCURACY ASSESSMENT FIELD CHECKING WAS TYPICALLY PERFORMED AS PART OF THE INTERPRETATION PROCESS BUT NO OVERALL MAP ACCURACY OR OTHER QUANTITATIVE MEASURES OF QUALITY WERE TYPICALLY INCORPORATED INTO THE ANALYSIS ONLY AFTER PHOTO INTERPRETATION BEGAN BEING USED AS REFERENCE DATA TO COMPARE MAPS DERIVED FROM DIGITAL REMOTE SENSOR DATA DID ISSUES CONCERNING THE ACCURACY OF THE PHOTO INTERPRETATION ARISE ALL THE ACCURACY ASSESSMENT TECHNIQUES MENTIONED IN THIS CHAPTER CAN BE APPLIED TO ASSESSING THE ACCURACY OF BOTH ANALOG AND DIGITAL REMOTELY SENSED DATA CONGALTON AND MEAD 1983 CONGALTON ET AL 1983 THE HISTORY OF ACCURACY ASSESSMENT CAN BE EFFECTIVELY DIVIDED INTO FOUR DEVELOPMENTAL EPOCHS OR AGES IN THE BEGINNING NO REAL ACCURACY ASSESSMENT WAS PERFORMED RATHER AN IT LOOKS GOOD MENTALITY PREVAILED THIS APPROACH IS TYPICAL OF MANY NEW EMERGING TECHNOLOGIES DESPITE THE MATURING OF THE TECHNOLOGY OVER THE LAST 25 YEARS SOME REMOTE SENSING ANALYSTS ARE STILL STUCK IN THIS MENTALITY OF COURSE THE MAP MUST LOOK GOOD BEFORE ANY FURTHER ANALYSIS SHOULD BE PER FORMED WHY ASSESS A MAP THAT IS OBVIOUSLY POOR HOWEVER WHILE LOOKING GOOD IS A REQUIRED CHARACTERISTIC IT IS NOT SUF CIENT FOR A VALID ASSESSMENT THE SECOND AGE OF ACCURACY ASSESSMENT COULD BE CALLED THE EPOCH OF NONSITE SPECI C ASSESSMENT DURING THIS PERIOD OVERALL ACREAGES WERE COMPARED BETWEEN GROUND ESTIMATES AND THE MAP WITHOUT REGARD FOR LOCATION MEYER ET AL 1975 IN SOME INSTANCES SUCH AS IMAGERY WITH VERY LARGE PIXELS E G AVHRR IMAGERY A NONSITE SPECI C ASSESSMENT MAY BE THE BEST AND OR ONLY CHOICE FOR VALIDATION FOR MOST IMAGERY THE AGE OF NONSITE SPECI C ASSESSMENT QUICKLY GAVE WAY TO THE AGE OF THE SITE SPECI C ASSESSMENT THIRD AGE IN A SITE SPECI C ASSESSMENT ACTUAL PLACES ON THE GROUND REMOTE SENSING AND GIS ACCURACY ASSESSMENT 2 I E LOCATIONS WERE COMPARED TO THE SAME PLACE ON THE MAP AND A MEASURE OF OVERALL ACCURACY I E PERCENTAGE CORRECT WAS COMPUTED FINALLY THE FOURTH AND CURRENT AGE OF ACCURACY ASSESSMENT COULD BE CALLED THE AGE OF THE ERROR MATRIX THIS EPOCH INCLUDES A SIGNI CANT NUMBER OF ANALYSIS TECHNIQUES MOST IMPORTANTLY THE KAPPA ANALYSIS A BRIEF REVIEW OF THE TECHNIQUES AND CONSIDERATIONS OF THE ERROR MATRIX AGE CAN BE FOUND BELOW AND IS DESCRIBED IN MORE DETAIL IN CONGALTON AND GREEN 1999 1 2 2 ESTABLISHED TECHNIQUES AND CONSIDERATIONS SINCE THE MID 1980S THE ERROR MATRIX HAS BEEN ACCEPTED AS THE STANDARD DESCRIPTIVE REPORTING TOOL FOR ACCURACY ASSESSMENT OF REMOTELY SENSED DATA THE USE OF THE ERROR MATRIX HAS SIGNI CANTLY IMPROVED OUR ABILITY TO CONDUCT ACCURACY ASSESSMENTS IN ADDITION ANALYSIS TOOLS INCLUDING DISCRETE MULTIVARIATE TECHNIQUES HAVE FACILITATED THE COMPARISON AND DEVELOPMENT OF VARIOUS METHODOLOGIES ALGORITHMS AND APPROACHES MANY FACTORS AFFECT THE COMPILATION OF THE ERROR MATRIX AND MUST BE CONSIDERED WHEN DESIGNING ANY ACCURACY ASSESSMENT THE CURRENT STATE OF KNOWLEDGE CONCERNING THE ERROR MATRIX ANALYSIS TECHNIQUES AND SOME CONSIDERATIONS ARE BRIE Y REVIEWED HERE 1 2 3 THE ERROR MATRIX AN ERROR MATRIX IS A SQUARE ARRAY OF NUMBERS ORGANIZED IN ROWS AND COLUMNS THAT EXPRESS THE NUMBER OF SAMPLE UNITS I E PIXELS CLUSTERS OF PIXELS OR POLYGONS ASSIGNED TO A PARTICULAR CATEGORY RELATIVE TO THE ACTUAL CATEGORY AS INDICATED BY THE REFERENCE DATA TABLE 1 1 THE COLUMNS TYPICALLY REPRESENT THE REFERENCE DATA AND THE ROWS INDICATE THE MAP GENERATED FROM THE REMOTELY SENSED DATA REFERENCE DATA ARE ASSUMED CORRECT AND CAN BE COLLECTED FROM A VARIETY OF SOURCES INCLUDING PHOTOGRAPHIC INTERPRETATION GROUND OR ELD OBSERVATION AND GROUND OR ELD MEASUREMENT THE ERROR MATRIX ONCE CORRECTLY GENERATED CAN BE USED AS A STARTING POINT FOR A SERIES OF DESCRIPTIVE AND ANALYTICAL STATISTICAL TECHNIQUES THE MOST COMMON AND SIMPLEST DESCRIPTIVE STATISTIC IS OVERALL ACCURACY WHICH IS COMPUTED BY DIVIDING THE TOTAL CORRECT I E THE SUM OF THE MAJOR DIAGONAL BY THE TOTAL NUMBER OF SAMPLE UNITS IN THE ERROR MATRIX IN ADDITION INDIVIDUAL CATEGORY ACCURACIES CAN BE COMPUTED IN A SIMILAR MANNER TRADITIONALLY THE TOTAL NUMBER OF CORRECT SAMPLE TABLE 1 1 EXAMPLE ERROR MATRIX LAND COVER CATEGORIES D DECIDUOUS C CONIFER AG AGRICULTURE SB SHRUB 63 6 0 11 85 8 11 89 4 22 24 79 8 4 7 3 113 101 107 103 73 118 132 424 D C AG SB D C AG SB ROW TOTAL COLUMN TOTAL REFERENCE DATA CLASSIFIED DATA PRODUCER S ACCURACY D 63 73 86 C 79 101 78 AG 85 118 72 SB 89 132 67 USER S ACCURACY D 63 113 56 C 79 101 78 AG 85 107 79 SB 89 103 86 OVERALL ACCURACY 63 79 85 89 424 316 424 75 101 REMOTE SENSING AND GIS ACCURACY ASSESSMENT 3