EarthScape: A Multimodal Dataset for Surficial Geologic Mapping and Earth Surface Analysis

Surficial geologic (SG) maps are essential for understanding surface processes and supporting infrastructure planning, but current workflows are labor-intensive and difficult to scale. We introduce EarthScape, an AI-ready multimodal dataset for SG mapping that integrates digital elevation models, aerial imagery, multi-scale terrain features, and hydrologic and infrastructure vector data within a unified, reproducible pipeline. We report baseline benchmarks across single-modality, multi-scale, and multimodal configurations. Our experiments show that terrain features provide the most reliable predictive signal, while raw spectral and elevation inputs degrade substantially under cross-region evaluation. EarthScape offers a geographically compact, but modality-rich benchmark for multimodal fusion, domain adaptation, and surface modeling. EarthScape is available for direct download at https://uknowledge.uky.edu/kgs_data/16/, and code is available at https://github.com/masseygeo/earthscape.

💡 Research Summary

EarthScape is a newly released, AI‑ready multimodal dataset specifically designed for surficial geologic (SG) mapping and broader surface‑aware geospatial analysis. The authors combine publicly available high‑resolution aerial RGB + NIR imagery (0.15 m ground sampling distance), airborne LiDAR‑derived digital elevation models (DEM, 1.52 m GSD), a suite of five DEM‑derived terrain descriptors (slope, profile curvature, planform curvature, elevation percentile, and slope standard deviation) computed at five spatial scales (1.52 m to 60.96 m), and vector layers for hydrography (USGS NHD) and infrastructure (OpenStreetMap). All modalities are re‑projected, resampled, and co‑registered onto a common 1.52 m grid, then sliced into 256 × 256 pixel patches (≈390 m side length) with 50 % overlap, yielding 31 018 patches covering two disjoint regions in Kentucky, USA.

Each patch contains 38 aligned channels: 4 spectral bands (RGB + NIR), 1 DEM band, 25 terrain‑feature bands (5 descriptors × 5 scales), and 2 binary layers (hydrography, infrastructure). The target SG map is rasterized from vector polygons into the same grid and provides a multi‑label mask with seven mutually exclusive classes: Qr (residual), Qal (alluvium), Qc (colluvium), af1 (artificial fill), Qca (colluvial apron), Qat (terrace deposits), and Qaf (alluvial fan). Class frequencies are heavily long‑tailed: Qr appears in 94 % of patches, while Qaf appears in only 0.9 %, leading to an effective number of samples (ENS) ranging from 9 464 for Qr down to 266 for Qaf and an imbalance ratio per label (IRLbl) spanning more than two orders of magnitude.

The paper details a reproducible preprocessing pipeline, including geometry validation, rasterization, multi‑scale terrain computation (using 5 × 5 kernels for curvature‑type features and variable‑size kernels for percentile and roughness), Gaussian smoothing, and systematic patch extraction. This pipeline is openly released, enabling other researchers to extend the dataset to new regions or add additional modalities.

Baseline experiments cover three configurations: (1) unimodal convolutional neural networks (CNNs) on RGB + NIR or DEM alone, (2) multi‑scale UNet architectures that ingest terrain features at multiple resolutions, and (3) multimodal transformer‑style models that fuse all 38 channels. Evaluation uses mean Intersection‑over‑Union (mIoU) and per‑class F1 scores, both within‑region (train‑test split from the same Kentucky area) and cross‑region (training on one area, testing on the other). Results show that raw spectral or elevation inputs degrade sharply under cross‑region testing (mIoU ≈ 0.35), whereas models that incorporate multi‑scale terrain descriptors retain higher performance (mIoU ≈ 0.48) and exhibit better generalization across geographic domains. Adding hydrography and infrastructure layers modestly improves discrimination for classes associated with fluvial processes (e.g., Qal, Qca).

The authors emphasize that terrain‑derived features provide the most robust predictive signal because they encode physical processes (slope, curvature, roughness) that are less sensitive to illumination, sensor differences, or local land‑cover variations. They also discuss the challenges posed by severe class imbalance, suggesting that future work should explore cost‑sensitive loss functions, focal loss, oversampling, or meta‑learning strategies. Domain shift between the two Kentucky regions, though limited in geographic extent, already reveals the need for domain adaptation techniques such as adversarial feature alignment or domain‑specific batch normalization.

EarthScape fills a critical gap in geospatial AI benchmarks: existing remote‑sensing datasets (e.g., SpaceNet, BigEarthNet) focus on anthropogenic objects or coarse land‑cover categories, while prior geologic datasets target discrete hazards (landslides, floods) rather than continuous surficial material mapping. By providing a rich, multimodal, multi‑scale, and well‑documented dataset, EarthScape enables systematic research on multimodal fusion, multi‑scale representation learning, long‑tail classification, and cross‑domain generalization in the context of earth‑surface science.

In conclusion, the paper contributes (1) a comprehensive, publicly available multimodal dataset with detailed documentation and code, (2) thorough analysis of class distribution, multi‑scale terrain features, and domain shift, and (3) baseline performance benchmarks that highlight terrain features as the key driver of successful SG mapping. The authors invite the community to extend the dataset to other physiographic settings, incorporate additional sensors (e.g., SAR, hyperspectral), and develop advanced learning strategies to tackle the remaining challenges of imbalance and domain adaptation.