Plugins#

EODAG uses a two-level plugin system. Plugin topics are abstract interfaces for a specific functionality of EODAG like Search or Download. EODAG providers are implementations of at least one plugin topic. The more plugin topics are implemented by a provider, the more functionality of eodag are available for this provider.

Plugin Management#

The plugin machinery is managed by one instance of PluginManager. The instance manager knows how to discover plugins at runtime, using setuptools entry points mechanism.

Plugins Available#

EODAG currently advertises 5 types of plugins: Search, Download, Crunch, Authentication and Api.

The providers are implemented with a triplet of Search/Authentication/Download plugins or with an Api plugin:

Provider

Search

Authentication

Download

aws_eos

PostJsonSearch

AwsAuth

AwsDownload

theia

QueryStringSearch

TokenAuth

HTTPDownload

peps

QueryStringSearch

GenericAuth

HTTPDownload

creodias

QueryStringSearch

KeycloakOIDCPasswordAuth

HTTPDownload

onda

ODataV4Search

GenericAuth

HTTPDownload

astraea_eod

StacSearch

AwsAuth

AwsDownload

usgs_satapi_aws

StacSearch

AwsAuth

AwsDownload

earth_search

StacSearch

AwsAuth

AwsDownload

earth_search_cog

StacSearch

None

HTTPDownload

earth_search_gcs

StacSearch

AwsAuth

AwsDownload

usgs

UsgsApi

UsgsApi

UsgsApi

ecmwf

EcmwfApi

EcmwfApi

EcmwfApi

cop_ads

CdsApi

CdsApi

CdsApi

cop_cds

CdsApi

CdsApi

CdsApi

meteoblue

BuildPostSearchResult

HttpQueryStringAuth

HTTPDownload

cop_dataspace

QueryStringSearch

KeycloakOIDCPasswordAuth

HTTPDownload

planetary_computer

StacSearch

SASAuth

HTTPDownload

hydroweb_next

StacSearch

HTTPHeaderAuth

HTTPDownload

wekeo

DataRequestSearch

TokenAuth

HTTPDownload

Creating Plugins#

EODAG plugins can be separated from the core eodag package. They live in your Python site-packages as standalone Python packages or modules, with eodag’s specific entry points that will be looked at by the plugin manager to load plugins at runtime. These entry points are: eodag.plugins.{topic}, with possible values for topic being api, search, crunch, auth and download.

All you have to do to create eodag plugins is start a new Python project, add eodag as a dependancy to your project, and then create the plugin type you want in Python module(s) by importing its base class from eodag.plugins.{topic}.base and subclassing it. Here is an example for an Api plugin class located in a module named your_package.your_module:

from eodag.plugins.apis.base import Api

class SampleApiPlugin(Api):

   def query(self):
      pass

   def download(self):
      pass

Then, in the setup.py of your Python project, add this:

setup(
   ...
   entry_points={
      ...
      'eodag.plugins.api': [
         'SampleApiPlugin = your_package.your_module:SampleApiPlugin'
      ],
      ...
   },
   ...
)

See what the PyPa explains to better understand this concept. In EODAG, the name you give to your plugin in the setup.py script’s entry point doesn’t matter, but we prefer it to be the same as the class name of the plugin. What matters is that the entry point must be a class deriving from one of the 5 plugin topics supported. Be particularly careful with consistency between the entry point name and the super class of you plugin class. Here is a list of entry point names and the plugin topic to which they map:

  • ‘eodag.plugins.api’ : Api

  • ‘eodag.plugins.auth’ : Authentication

  • ‘eodag.plugins.crunch’ : Crunch

  • ‘eodag.plugins.download’ : Download

  • ‘eodag.plugins.search’ : Search

As an example of external plugin, you can have a look to eodag-sentinelsat.

Provider configuration#

The plugin structure is reflected in the internal providers configuration file. Here is a sample:

provider_name:
   priority: 1
   products:
      # List of supported product types
      # This is a mapping containing all the information required by the search plugin class to perform its job.
      # The mapping is available in the config attribute of the search plugin as config['products']
      S2_MSI_L1C:
         a-config-key-needed-by-search-plugin-to-search-this-product-type: value
         another-config-key: another-value
         # Whether this product type is partially supported by this provider (the provider does not contain all the
         # products of this type)
         partial: True
      ...
   search:
      plugin: CustomSearchPluginClass
      api_endpoint: https://mandatory.config.key/
      a-key-conf-used-by-the-plugin-class-init-method: value
      another-random-key: random-value
      # A mapping between the search result of the provider and the eodag way of describing EO products (the keys are
      # the same as in the OpenSearch specification)
      metadata_mapping:
         ...
      ...
   download:
      plugin: CustomDownloadPlugin
      # Same as with search for random config keys as needed by the plugin class
      ...
   auth:
      plugin: CustomAuthPlugin
      # Same as with search for random config keys as needed by the plugin class
      ...

Note however, that for a provider which already has a Python library for accessing its products, the configuration varies a little bit. It does not have the ‘search’ and ‘download’ keys. Instead, there is a single ‘api’ key like this:

provider_name:
   ...
   api:
      plugin: ApiPluginClassName
      ...

An EOProduct has a properties attribute which is built based on how its metadata are set in the provider configuration. The following converters can be used to transform the values collected from the provider:

metadata_mapping.format_metadata(*args: Tuple[Any], **kwargs: Any) str#

Format a string of form {<field_name>#<conversion_function>}

The currently understood converters are:
  • datetime_to_timestamp_milliseconds: converts a utc date string to a timestamp in milliseconds

  • to_rounded_wkt: simplify the WKT of a geometry

  • to_bounds_lists: convert to list(s) of bounds

  • to_nwse_bounds: convert to North,West,South,East bounds

  • to_nwse_bounds_str: convert to North,West,South,East bounds string with given separator

  • to_geojson: convert to a GeoJSON (via __geo_interface__ if exists)

  • from_ewkt: convert EWKT to shapely geometry / WKT in DEFAULT_PROJ

  • to_ewkt: convert to EWKT (Extended Well-Known text)

  • from_georss: convert GeoRSS to shapely geometry / WKT in DEFAULT_PROJ

  • csv_list: convert to a comma separated list

  • to_iso_utc_datetime_from_milliseconds: convert a utc timestamp in given milliseconds to a utc iso datetime

  • to_iso_utc_datetime: convert a UTC datetime string to ISO UTC datetime string

  • to_iso_date: remove the time part of a iso datetime string

  • remove_extension: on a string that contains dots, only take the first part of the list obtained by splitting the string on dots

  • get_group_name: get the matching regex group name

  • replace_str: execute “string”.replace(old, new)

  • recursive_sub_str: recursively substitue in the structure (e.g. dict) values matching a regex

  • slice_str: slice a string (equivalent to s[start, end, step])

  • fake_l2a_title_from_l1c: used to generate SAFE format metadata for data from AWS

  • s2msil2a_title_to_aws_productinfo: used to generate SAFE format metadata for data from AWS

  • split_cop_dem_id: get the bbox by splitting the product id

  • split_corine_id: get the product type by splitting the product id

  • to_datetime_dict: convert a datetime string to a dictionary where values are either a string or a list

  • get_ecmwf_time: get the time of a datetime string in the ECMWF format

Parameters
  • search_param (str) – The string to be formatted

  • args (tuple) – (optional) Additional arguments to use in the formatting process

  • kwargs (Any) – (optional) Additional named-arguments to use when formatting

Returns

The formatted string

Return type

str

For example:

search:
   ...
   metadata_mapping:
      publicationDate: '{$.data.timestamp#to_iso_utc_datetime_from_milliseconds}'
      ...