Scaling Remote Sensing Foundation Models: Data-Driven Insights

This paper addresses the critical challenge of scaling foundation models for remote sensing, a domain with limited data compared to natural images. It investigates the scaling behavior of vision transformers using a massive dataset of commercial satellite imagery. The findings provide valuable insights into data-collection strategies and compute budgets for future development of large-scale remote sensing models, particularly highlighting the data-limited regime.

• •Explores scaling behaviors of vision transformers on petascale remote sensing data.
• •Identifies a data-limited regime, suggesting data collection is more critical than model size.
• •Provides practical insights for data collection, compute budgets, and optimization schedules for future RS foundation models.