Parameters mapping

Parameters mapping#

eodag interacts with various data providers, each having its own way of naming and structuring metadata parameters. To provide a unified interface for users, eodag maps these provider metadata parameters to a common model.

Parameters mapping#

EODAG maps each provider’s specific metadata parameters to a common model using OGC OpenSearch Extension for Earth Observation. Extra parameters having no equivalent in this model are mapped as is.

Depending on the provider, some parameters are queryable or not. This is configured in each configuration file located in the eodag/resources/providers/ directory.

  • If a parameter metadata-mapping is a list, the first element will help constructing the query (using format()), and the 2nd will help extracting its values from the query result (using jsonpath)

  • If a parameter metadata-mapping is a string, it will not be queryable and this string will help extracting its values from the query result (using jsonpath).

some_provider:
   search:
      metadata_mapping:
         queryableParameter:
            - 'this_is_query_string={queryableParameter}'
            - '$.jsonpath.in.result.to.parameter'
         nonQueryableParameter: '$.jsonpath.in.result.to.another_parameter'

Formatters#

An EOProduct has a properties attribute which is built based on how its metadata are set in the provider configuration. For example:

search:
   ...
   metadata_mapping:
      publicationDate: '{$.data.timestamp#to_iso_utc_datetime_from_milliseconds}'
      ...

The following converters can be used to transform the values collected from the provider:

eodag.api.product.metadata_mapping.format_metadata(search_param, *args, **kwargs)[source]#

Format a string of form {<field_name>#<conversion_function>}

The currently understood converters are:

  • assets_list_to_dict: convert a list of asset objects into a dictionary keyed by asset name

  • ceda_collection_name: generate a CEDA collection name from a string

  • wekeo_to_cop_collection: converts the name of a collection from the WEkEO format to the Copernicus format

  • csv_list: convert to a comma separated list

  • dates_from_cmems_id: extract min/max UTC datetimes from a CMEMS product identifier

  • datetime_to_timestamp_milliseconds: converts a utc date string to a timestamp in milliseconds

  • dict_filter: filter dict items using a jsonpath predicate

  • dict_filter_and_sub: filter dict items using jsonpath and then apply recursive_sub_str

  • dict_update: add/update nested dictionary items from a list of key/value pairs

  • dict_with_roles: keep only dict items with given roles in their “roles” list

  • fake_l2a_title_from_l1c: used to generate SAFE format metadata for data from AWS

  • from_alternate: update assets using given alternate

  • from_ewkt: convert EWKT to shapely geometry / WKT in DEFAULT_PROJ

  • from_georss: convert GeoRSS to shapely geometry / WKT in DEFAULT_PROJ

  • get_ecmwf_time: get the time of a datetime string in the ECMWF format

  • get_dates_from_string: extract start/end UTC datetimes from a date range embedded in text

  • get_group_name: get the matching regex group name

  • get_hydrological_year: build hydrological year string(s) from an input date

  • get_variables_from_path: extract variables listed in the query part of a path

  • interval_to_datetime_dict: convert a date interval string to a dictionary of year/month/day lists

  • literalize_unicode: convert a string to its raw Unicode literal form

  • not_available: replace value with “Not Available”

  • recursive_sub_str: recursively substitue in the structure (e.g. dict) values matching a regex

  • remove_extension: on a string that contains dots, only take the first part of the list obtained by splitting the string on dots

  • replace_str: execute “string”.replace(old, new)

  • replace_str_tuple: apply multiple replacements on a string (parts or complete)

  • replace_tuple: apply multiple replacements matching whole value

  • s2msil2a_title_to_aws_productinfo: used to generate SAFE format metadata for data from AWS

  • sanitize: sanitize string

  • slice_str: slice a string (equivalent to s[start, end, step])

  • split: split a string using given separator

  • split_cop_dem_id: get the bbox by splitting the product id

  • split_corine_id: get the collection by splitting the product id

  • split_id_into_s3_params: parse a Sentinel-3 product id into S3 query parameter values

  • to_bounds: convert an input geometry to [min_lon, min_lat, max_lon, max_lat]

  • to_bounds_lists: convert to list(s) of bounds

  • to_datetime_dict: convert a datetime string to a dictionary where values are either a string or a list

  • to_ewkt: convert to EWKT (Extended Well-Known text)

  • to_geojson: convert to a GeoJSON (via __geo_interface__ if exists)

  • to_geojson_polytope: convert shapely Point/LineString/Polygon to ECMWF polytope feature dicts

  • to_iso_date: remove the time part of a iso datetime string

  • to_iso_utc_datetime_from_milliseconds: convert a utc timestamp in given milliseconds to a utc iso datetime

  • to_iso_utc_datetime: convert a UTC datetime string to ISO UTC datetime string

  • to_longitude_latitude: compute geometry center as a {"lon": ..., "lat": ...} dictionary

  • to_lower: Convert a string to lowercase

  • to_non_separated_date: convert an ISO datetime/date string to YYYYMMDD

  • to_nwse_bounds_str: convert to North,West,South,East bounds string with given separator

  • to_nwse_bounds: convert to North,West,South,East bounds

  • to_rounded_wkt: simplify the WKT of a geometry

  • to_title: Convert a string to title case

  • to_upper: Convert a string to uppercase

Parameters:

  • search_param (str) – The string to be formatted

  • args (Any) – (optional) Additional arguments to use in the formatting process

  • kwargs (Any) – (optional) Additional named-arguments to use when formatting

Return type:

str

Returns:

The formatted string

>>> format_metadata("{date#to_iso_utc_datetime}", date="2021-04-21")
'2021-04-21T00:00:00.000Z'
>>> format_metadata("{values#csv_list}", values=["a", "b", "c"])
'a,b,c'

Queryables#

The list_queryables() method will help you to dynamically check which parameters are queryable for a given provider or collection. See Python API User Guide / Queryables for more information and examples.