Abstract
This paper examines the limitations of current evaluation metrics in GeoAI. Through two case studies on deep learning models—a building detection classification problem and a remote sensing image fusion regression problem—this paper demonstrates how traditional statistical evaluation matrices alone can be misleading in geospatial problems. The findings indicate that traditional metrics (e.g., RMSE, MAE) used in current GeoAI models can have difficulty capturing the spatial dimensions inherent to geospatial problems. This paper suggests that the model evaluation process in GeoAI should move beyond traditional evaluation matrices by integrating spatial thinking throughout the modeling pipeline—not only incorporating spatial accuracy in model evaluation but also embedding it within optimization functions in model structure and model training.
Keywords
Geospatial AI, Evaluation Matrices, GIS
Document Type
Paper
Start Date
19-6-2025 10:50 AM
End Date
19-6-2025 11:50 AM
DOI
10.5703/1288284317905
Recommended Citation
Lyu, Fangzheng, "Evaluating the Evaluation Matrices: Integrating Spatial Assessment in Geospatial AI Model Training and Evaluation" (2025). I-GUIDE Forum. 5.
https://docs.lib.purdue.edu/iguide/2025/presentations/5
Evaluating the Evaluation Matrices: Integrating Spatial Assessment in Geospatial AI Model Training and Evaluation
This paper examines the limitations of current evaluation metrics in GeoAI. Through two case studies on deep learning models—a building detection classification problem and a remote sensing image fusion regression problem—this paper demonstrates how traditional statistical evaluation matrices alone can be misleading in geospatial problems. The findings indicate that traditional metrics (e.g., RMSE, MAE) used in current GeoAI models can have difficulty capturing the spatial dimensions inherent to geospatial problems. This paper suggests that the model evaluation process in GeoAI should move beyond traditional evaluation matrices by integrating spatial thinking throughout the modeling pipeline—not only incorporating spatial accuracy in model evaluation but also embedding it within optimization functions in model structure and model training.