As part of the Amazon Sustainability Data Initiative, Amazon Web Services (AWS) is supporting Digital Earth Africa (DE Africa). DE Africa is enabling African nations to track changes across the continent in unprecedented detail by making Earth observation (EO) data more easily accessible. This will provide valuable insights for better decision-making around prevention and planning in areas including flooding, droughts, soil and coastal erosion, agriculture, forest-cover, land use and land cover change, water availability and quality, and changes to human settlements.
AWS is collaborating with DE Africa as the technology enabler for global observations – helping data users across the continent gain fast access to petabytes of decision-ready satellite data. Users will be able to analyze critical data within minutes of it being available and use it to develop preventative and early warning systems to address sustainability challenges, such as land degradation over time, climate change patterns affecting agricultural production, and severe weather trends that lead to community displacement.
The DE Africa initiative aligns with the Amazon Sustainability Data Initiative (ASDI), which was created to help researchers and innovators access the data and information they need to innovate and problem solve for sustainability. DE Africa will leverage the expertise and platform maturity from Digital Earth Australia (DEA). DEA is a platform that uses spatial data and images recorded by satellites orbiting the Earth to detect physical changes across Australia in unprecedented detail. DEA and AWS have a long-standing collaborative relationship and DEA is largely built on AWS services. And DE Africa will be among the first customers to move mission-critical workloads to the AWS Africa (Cape Town) Region. This will enable DE Africa to leverage AWS advanced technologies to drive innovation from local data centers with the assurance that their content will not leave the country.
Earth observations to support sustainability decision-making
The sustainable development data revolution allows for the fusion of different technologies from satellites, mobile phones, cloud computing, and machine learning – making available unprecedented volumes of data to support decision-making. Despite these advances, data availability, accessibility, quality, usability, disaggregation, timeliness, and capacity are still challenges for many countries. Several factors are helping unlock the vast potential for insights and information to address national development priorities. Operational satellites are providing routine global coverage; free and open data policies remove legal and cost barriers to data access; analysis ready data (ARD) is removing pre-processing burdens from users; computing and storage costs are decreasing and cloud architectures are becoming more mainstream. In addition, the Open Data Cube (ODC) enables continental-scale analyses and web-service-driven visualizations over complete time-series of data.
DE Africa is using openly accessible and freely available EO data and the ODC’s technology to produce and make accessible decision-ready products. Working closely with the AfriGEO community and stakeholders and partner organizations across the continent, DE Africa will be responsive to the information needs, challenges, and priorities of its users.
The project will leverage technology and services developed in Australia (implemented as Digital Earth Australia) and build upon the learnings from the Africa Regional Data Cube (ARDC). The ARDC, launched in May 2018, is a data management technology developed on AWS that builds capacity in African communities in support of their local and national needs framed around the United Nations Sustainable Development Goals (SDGs) and broader development priorities. The project focused on five different countries in Africa – Ghana, Kenya, Senegal, Sierra Leone, and Tanzania – and aimed to understand their data-related challenges and respond to these needs with improved access and use of geospatial data. DE Africa expands this approach and its goals to the entire African continent by providing an operational infrastructure and program that is institutionalized in Africa.
A snapshot of the Water Observations from Space (WOfS) continental-wide data product from DE Africa for the Bintang Bolong River in The Gambia. Derived from USGS Landsat data, blue and purple colors indicate persistent observations, while red and orange colors indicate more sporadic observations.
Leveraging the AWS Cloud to extract insights
DE Africa will deliver a continental-scale platform and program that democratizes the capacity to process and analyze satellite data. The Amazon Sustainability Data Initiative is committing to support DE Africa by providing storage of large datasets including Copernicus Sentinel-2 and USGS Landsat satellite imagery in the new AWS Africa (Cape Town) Region.
The DE Africa infrastructure will store all data in Amazon Simple Storage Service (Amazon S3) in Cloud Optimized GeoTIFF (COG) format accompanied by Spatiotemporal Asset Catalog (STAC) metadata. The use of COG and STAC enables interoperability through the use of modern standards, so that individuals, governments, and businesses can use the data without the requirement of bespoke solutions.
DE Africa applications will run on a cluster of servers orchestrated by Amazon Elastic Kubernetes Service (Amazon EKS), enabling more advanced computing practices while simplifying the effort required to create and deploy new applications. A key component in the DE Africa application suite is the ODC. The ODC uses an RDS hosted PostgreSQL database as an index to the EO data that is being stored, which enables a number of applications to be driven off the index. For example, there are OGC Web Services that enable visualization of scene-based or continental-scale derived products in real time. Data processing pipelines are driven off the ODC index and run on Amazon Elastic Compute Cloud (Amazon EC2) spot instances managed by Amazon EKS and scheduled using an Amazon Simple Queue Service (Amazon SQS) queue, which allows large volumes of data to be processed and analyzed efficiently. A JupyterHub-powered Sandbox enables data scientists to explore the 30-year archive of data, anywhere in Africa. Preview some of the data visualizations now in the Digital Earth Africa Map, a web-mapping interface to access spatial data.
Empowering countries with timely and meaningful data and information
Through DE Africa, individuals, governments, and businesses in countries will be empowered through an operational infrastructure providing continental-scale data and decision-ready information products to make informed decisions and choices. Development activities can be more effective through access and use of information that improves the understanding of issues and solutions. DE Africa also hopes to catalyze a continental digital transformation by engaging the private sector to innovate by leveraging its products and services. Access to data for commercial product and services development can also help increase economic development and job creation.
This article was originally published by the AWS Public Sector Blog Team.