Introduction

Nowadays, we observe a rise in publicly accessible Earth Observation (EO) data. Together with it, there is more and more EO data providers, each potentially having a different data access policy. This difference is visible at various levels: in the data discovery (CSW, OpenSearch more or less custom, etc.), in the product access (object storage, downloads, direct file system access, etc.), in the storage structure and in the authentication mechanisms (OAUTH, JWT, basic auth,…). All these different technologies add a knowledge overhead on a user (end-user or application developer) wishing to take advantage of these data. EODAG was designed to solve this problem.

EODAG (Earth Observation Data Access Gateway) is a command line tool and a plugin-oriented Python framework for searching, aggregating results and downloading remote sensed images while offering a unified API for data access regardless of the data provider. The EODAG SDK is structured around three functions:

  • List product types: list of supported products and their description
  • Search products (by product type) : searches products according to the search criteria provided
  • Download products : download product “as is”

EODAG is developed in Python. It is structured according to a modular plugin architecture, easily extensible and able to integrate new data providers. Three types of plugins compose the tool:

  • Catalog search plugins, responsible for searching data (OpenSearch, CSW, …), building paths, retrieving quicklook, combining results
  • Download plugins, allowing to download and retrieve data locally (via FTP, HTTP, ..), always with the same directory organization
  • Authentication plugins, which are used to authenticate the user on the external services used (JSON Token, Basic Auth, OAUTH, …).

Available providers

There are currently 9 available providers implemented on eodag:

  • sobloo: Airbus DIAS
  • usgs: U.S geological survey catalog for Landsat products
  • aws_eos: EOS search for Amazon public datasets
  • theia: French National Space Agency (CNES) catalog for Sentinel 2 products, Pleiades and Landsat products
  • peps: French National Space Agency (CNES) catalog for Copernicus (Sentinel 1, 2, 3) products
  • creodias: CloudFerro DIAS
  • mundi: Atos DIAS
  • onda: Serco DIAS

Note

For developers, there are 2 ways for adding support for a new provider:

  • By configuring existing plugins: a provider is an instance of already implemented plugins (search, download) => this only involves knowing how to write yaml documents.
  • By developing new plugins (most of the time it will be search plugins) and configuring instances of these plugins.

See Creating EODAG plugins for more details on how to extend eodag.

How to configure authentication for available providers

Create a configuration file containing your credentials for each provider. You can download this template, which has the following layout:

peps:
    priority: # Lower value means lower priority (Default: 1)
    search:   # Search parameters configuration
    download:
        extract:  # whether to extract the downloaded products (true or false).
        outputs_prefix: # where to store downloaded products.
        dl_url_params:  # additional parameters to pass over to the download url as an url parameter
    auth:
        credentials:
            username:
            password:
theia:
    priority: # Lower value means lower priority (Default: 0)
    search:   # Search parameters configuration
    download:
        extract:
        outputs_prefix:
        dl_url_params:
    auth:
        credentials:
            ident:
            pass:
usgs:
    priority: # Lower value means lower priority (Default: 0)
    api:
        extract:
        outputs_prefix:
        dl_url_params:
        product_location_scheme:
        credentials:
            username:
            password:
aws_eos:
    priority: # Lower value means lower priority (Default: 0)
    search:   # Search parameters configuration
    auth:
        credentials:
            apikey:
            aws_access_key_id:
            aws_secret_access_key:
    download:
        outputs_prefix:
sobloo:
    priority: # Lower value means lower priority (Default: 0)
    search:   # Search parameters configuration
    download:
        extract:
        outputs_prefix:
        dl_url_params:
    auth:
        credentials:
            apikey:
creodias:
    priority: # Lower value means lower priority (Default: 0)
    search:   # Search parameters configuration
    download:
        extract:
        outputs_prefix:
    auth:
        credentials:
            username:
            password:
mundi:
    priority: # Lower value means lower priority (Default: 0)
    search:   # Search parameters configuration
    download:
        extract:
        outputs_prefix:
    auth:
        credentials:
            apikey:
onda:
    priority: # Lower value means lower priority (Default: 0)
    search:   # Search parameters configuration
    download:
        extract:
        outputs_prefix:
    auth:
        credentials:
            username:
            password:

Warning

This file contains login information in clear text. Make sure you correctly configure access rules to it. It should be read/write-able only by the current user of eodag. However, note that you can alternatively configure your credentials through environment variables instead

Note

The above template is used to create a default user configuration file at ~/.config/eodag/eodag.yml which you can directly modify with your preferences. If you don’t want to use the file created in this location, you can still tell eodag which file to use with the environment variable: EODAG_CFG_FILE.

Fill this configuration file with the credentials you obtained from each provider.

For usgs, create an account here: https://ers.cr.usgs.gov/register/

For theia, you only need to register once here: https://sso.theia-land.fr/theia/register/register.xhtml

For peps, create an account here: https://peps.cnes.fr/rocket/#/register

For aws_eos, you need credentials for both EOS (search) and AWS (download):

For sobloo, create an account here for an api key:

For creodias, create an account here: https://portal.creodias.eu/register.php

For mundi, create an account here: https://mundiwebservices.com (click on “login” and then go in the “register” tab). Then use as apikey the Web Token provided in https://mundiwebservices.com/account/profile

For onda, create an account here: https://www.onda-dias.eu/crm/

Storage status handle

The storage status has been standardized for all providers in the parameter storageStatus, and its different values mapped to these 3 unique status:

  • ONLINE: the product is available for download (immediately);
  • STAGING: the product has been ordered and will be ONLINE soon;
  • OFFLINE: the product is not available for download, but can eventually be ordered.

If product status is OFFLINE, the download method will request an order of the product (if available) and retry to download it every 2 minutes (or specified wait parameter) during 20 minutes (or specified timeout parameter).

If product status is STAGING, the download method will retry to download it every 2 minutes (or specified wait parameter) during 20 minutes (or specified timeout parameter).

See download() documentation for more details.

Parameters mapping

The list of parameters mapped for available providers can be found in this CSV file.