Weakly Supervised Deep Learning Models for Detecting and Monitoring Spatio-Temporal Earth Surface Anomalies in Optical and Radar Satellite Time Series (WeMonitor)
Goals and focus
With recent satellite missions and new concepts such as the Copernicus program and the Planet Labs fleet a golden era for Earth observation (EO) has begun. We have access to very large and ever-growing data volumes of satellite image time series (SITS) on a global scale, which enables tackling global applications at an unprecedented spatial resolution and revisit time. It now allows the observation of small-scale and rapid change dynamics at the Earth’s surface. Such spatio-temporal Earth surface anomalies are often caused by natural or man-made hazards including but not limited to solid waste dump sites, landslides, deforestation, dam breaks, fires, and volcanic eruptions, which clearly differentiate due to their rapid nature from long-term and seasonal dynamics.
Starting from two examplary applications (landslides and dump sites) of spatio-temporal anomalies in SITS we aim at developing a generic deep learning architecture that can be trained and applied interdisciplinarily across the HGF research fields “Earth and Environment” (GFZ) and “Aeronautics, Space and Transport” (DLR), dealing with rapid spatio-temporal anomalies in SITS.
However, algorithms must be efficient enough and sufficiently transferable to be applied for large-scale and multi-object monitoring. Despite the success of existing deep learning models for anomaly detection, the lack of relevant and large labeled datasets currently limits their applicability in many real-world applications. Further, most models can only deal with multi-temporal remote sensing data from a single data source and do not fully excavate the potential value of multi-source (e.g. optical and SAR) remote sensing time series. In addition, there exists the problem of “one data set, one model”, which means that an anomaly detection model needs to be retrained for different regions, landscapes or even image scenes lacking transferability and generalization capability. To overcome these shortcomings, our proposal focuses on three main goals:
A) Developing interactive data annotation methods within an active learning strategy to improve the training of weakly supervised deep learning models;
B) Implementing transfer learning algorithms to benefit from the similarities of spatio-temporal anomaly detection across research fields;
C) Developing efficient multi-modal learning methods to leverage the complementary information content of multi-modal data sources for spatio-temporal anomaly detection in SITS.
The following key deliverables are expected:
● Code: open source availability on gitlab/github,
● Data publications: training data for landslides and dump sites,
● Data set: annotated SITS,
● Data set: resulting landslide inventories and dump site detections,
● Scientific publications.
