Plugins

EODAG uses a two-level plugin system. Plugin topics are abstract interfaces for a specific functionality of EODAG like Search or Download. EODAG providers are implementations of at least one plugin topic. The more plugin topics are implemented by a provider, the more functionality of eodag are available for this provider.

Plugin Management

The plugin machinery is managed by one instance of PluginManager. The instance manager knows how to discover plugins at runtime, using setuptools entry points mechanism.

Plugins Available

EODAG currently advertises 5 types of plugins: Search, Download, Crunch, Authentication and Api.

• Search Plugins
• Crunch Plugins
• Authentication Plugins
• Download Plugins
• API Plugins

The providers are implemented with a triplet of Search/Authentication/Download plugins or with an Api plugin:

Provider	Search	Authentication	Download
aws_eos	PostJsonSearch	AwsAuth	AwsDownload
theia	QueryStringSearch	TokenAuth	HTTPDownload
peps	QueryStringSearch	GenericAuth	HTTPDownload
creodias	QueryStringSearch	KeycloakOIDCPasswordAuth	HTTPDownload
onda	ODataV4Search	GenericAuth	HTTPDownload
astraea_eod	StacSearch	AwsAuth	AwsDownload
usgs_satapi_aws	StacSearch	AwsAuth	AwsDownload
earth_search	StacSearch	AwsAuth	AwsDownload
earth_search_cog	StacSearch	None	HTTPDownload
earth_search_gcs	StacSearch	AwsAuth	AwsDownload
usgs	UsgsApi	UsgsApi	UsgsApi
ecmwf	EcmwfApi	EcmwfApi	EcmwfApi
cop_ads	CdsApi	CdsApi	CdsApi
cop_cds	CdsApi	CdsApi	CdsApi
meteoblue	BuildPostSearchResult	HttpQueryStringAuth	HTTPDownload
cop_dataspace	QueryStringSearch	KeycloakOIDCPasswordAuth	HTTPDownload
planetary_computer	StacSearch	SASAuth	HTTPDownload
hydroweb_next	StacSearch	HTTPHeaderAuth	HTTPDownload
wekeo	DataRequestSearch	TokenAuth	HTTPDownload

Creating Plugins

EODAG plugins can be separated from the core eodag package. They live in your Python site-packages as standalone Python packages or modules, with eodag’s specific entry points that will be looked at by the plugin manager to load plugins at runtime. These entry points are: eodag.plugins.{topic}, with possible values for topic being api, search, crunch, auth and download.

All you have to do to create eodag plugins is start a new Python project, add eodag as a dependancy to your project, and then create the plugin type you want in Python module(s) by importing its base class from eodag.plugins.{topic}.base and subclassing it. Here is an example for an Api plugin class located in a module named your_package.your_module:

from eodag.plugins.apis.base import Api

class SampleApiPlugin(Api):

   def query(self):
      pass

   def download(self):
      pass

Then, in the setup.py of your Python project, add this:

setup(
   ...
   entry_points={
      ...
      'eodag.plugins.api': [
         'SampleApiPlugin = your_package.your_module:SampleApiPlugin'
      ],
      ...
   },
   ...
)

See what the PyPa explains to better understand this concept. In EODAG, the name you give to your plugin in the setup.py script’s entry point doesn’t matter, but we prefer it to be the same as the class name of the plugin. What matters is that the entry point must be a class deriving from one of the 5 plugin topics supported. Be particularly careful with consistency between the entry point name and the super class of you plugin class. Here is a list of entry point names and the plugin topic to which they map:

• ‘eodag.plugins.api’ : Api

• ‘eodag.plugins.auth’ : Authentication

• ‘eodag.plugins.crunch’ : Crunch

• ‘eodag.plugins.download’ : Download

• ‘eodag.plugins.search’ : Search

As an example of external plugin, you can have a look to eodag-sentinelsat.

Provider configuration

The plugin structure is reflected in the internal providers configuration file. Here is a sample:

provider_name:
   priority: 1
   products:
      # List of supported product types
      # This is a mapping containing all the information required by the search plugin class to perform its job.
      # The mapping is available in the config attribute of the search plugin as config['products']
      S2_MSI_L1C:
         a-config-key-needed-by-search-plugin-to-search-this-product-type: value
         another-config-key: another-value
         # Whether this product type is partially supported by this provider (the provider does not contain all the
         # products of this type)
         partial: True
      ...
   search:
      plugin: CustomSearchPluginClass
      api_endpoint: https://mandatory.config.key/
      a-key-conf-used-by-the-plugin-class-init-method: value
      another-random-key: random-value
      # A mapping between the search result of the provider and the eodag way of describing EO products (the keys are
      # the same as in the OpenSearch specification)
      metadata_mapping:
         ...
      ...
   download:
      plugin: CustomDownloadPlugin
      # Same as with search for random config keys as needed by the plugin class
      ...
   auth:
      plugin: CustomAuthPlugin
      # Same as with search for random config keys as needed by the plugin class
      ...

Note however, that for a provider which already has a Python library for accessing its products, the configuration varies a little bit. It does not have the ‘search’ and ‘download’ keys. Instead, there is a single ‘api’ key like this:

provider_name:
   ...
   api:
      plugin: ApiPluginClassName
      ...

An EOProduct has a properties attribute which is built based on how its metadata are set in the provider configuration. The following converters can be used to transform the values collected from the provider:

metadata_mapping.format_metadata(*args, **kwargs)

Format a string of form {<field_name>#<conversion_function>}

The currently understood converters are:

• datetime_to_timestamp_milliseconds: converts a utc date string to a timestamp in milliseconds

• to_rounded_wkt: simplify the WKT of a geometry

• to_bounds_lists: convert to list(s) of bounds

• to_nwse_bounds: convert to North,West,South,East bounds

• to_nwse_bounds_str: convert to North,West,South,East bounds string with given separator

• to_geojson: convert to a GeoJSON (via __geo_interface__ if exists)

• from_ewkt: convert EWKT to shapely geometry / WKT in DEFAULT_PROJ

• to_ewkt: convert to EWKT (Extended Well-Known text)

• from_georss: convert GeoRSS to shapely geometry / WKT in DEFAULT_PROJ

• csv_list: convert to a comma separated list

• to_iso_utc_datetime_from_milliseconds: convert a utc timestamp in given milliseconds to a utc iso datetime

• to_iso_utc_datetime: convert a UTC datetime string to ISO UTC datetime string

• to_iso_date: remove the time part of a iso datetime string

• remove_extension: on a string that contains dots, only take the first part of the list obtained by splitting the string on dots

• get_group_name: get the matching regex group name

• replace_str: execute “string”.replace(old, new)

• recursive_sub_str: recursively substitue in the structure (e.g. dict) values matching a regex

• slice_str: slice a string (equivalent to s[start, end, step])

• fake_l2a_title_from_l1c: used to generate SAFE format metadata for data from AWS

• s2msil2a_title_to_aws_productinfo: used to generate SAFE format metadata for data from AWS

• split_cop_dem_id: get the bbox by splitting the product id

• split_corine_id: get the product type by splitting the product id

• to_datetime_dict: convert a datetime string to a dictionary where values are either a string or a list

• get_ecmwf_time: get the time of a datetime string in the ECMWF format

Parameters

• search_param (str) – The string to be formatted

• args (tuple) – (optional) Additional arguments to use in the formatting process

• kwargs (Any) – (optional) Additional named-arguments to use when formatting

Returns

The formatted string

Return type

str

For example:

search:
   ...
   metadata_mapping:
      publicationDate: '{$.data.timestamp#to_iso_utc_datetime_from_milliseconds}'
      ...