Add a Provider#
eodag provides a set of plugins which don’t know anything about the
providers themselves, they just implement generic methods required to talk to different kinds of data catalog. For instance:
QueryStringSearch: Search plugin that implements a search protocol that relies on query stringsHTTPHeaderAuth: Authentication plugin that implements HTTP authentication using headersHTTPDownload: Download plugin that implements download over HTTP protocol
Dynamically add a new provider#
You can dynamically add a new provider, from your python code using add_provider()
or update_providers_config() methods.
Check Python API User Guide / Add-or-update-a-provider for guidelines.
Configure a new provider#
The simplest way to add a new provider is to use existing plugins. This approach requires to
provide the new provider’s configuration in a YAML format. The following example shows how to add a new STAC provider:
another_earth_search:
search:
type: StacSearch
api_endpoint: https://earth-search.aws.element84.com/v1/search
need_auth: false
products:
S2_MSI_L1C:
_collection: sentinel-2-l1c
GENERIC_COLLECTION:
_collection: '{collection}'
download:
type: AwsDownload
auth:
type: AwsAuth
credentials:
aws_access_key_id: PLEASE_CHANGE_ME
aws_secret_access_key: PLEASE_CHANGE_ME
It configures the following existing plugins: StacSearch (search),
AwsAuth (authentication) and AwsDownload (download).
Each plugin configuration is inserted following the appropriate plugin topic key:
searchfor search pluginsdownloadfor download pluginsauth,search_auth, ordownload_authfor authentication pluginsapifor api plugins
Of course, it is also necessary to know how to configure these plugins (which parameters they take, what values they can have, etc.).
You can get some inspiration from the Providers pre-configuration section by analysing how eodag configures the providers it comes installed with.
Add more plugins#
eodag is a plugin-oriented framework which means it can be easily extended. If the plugins it offers are not sufficient for your
own needs (i.e. getting data from a provider not supported by eodag), you should then write your own plugins (possibly by extending one of provided by eodag)
and configure them. What you are the most likely to be willing to do is either to develop a new Search plugin or an
Api plugin (e.g. to create an interface with another program).
eodag-sentinelsat (archived) was a good example of an Api plugin. It created an interface with the sentinalsat library to search and download products from SciHub.
See more details about how to create a new plugin in this dedicated section.
Providers pre-configuration#
All the providers are pre-configured in eodag in a YAML file.
Click on the link below to display its full content.
providers.yml
---
!provider # MARK: usgs
name: usgs
priority: 0
description: U.S geological survey catalog for Landsat products
roles:
- host
url: https://earthexplorer.usgs.gov/
api: !plugin
type: UsgsApi
need_auth: true
pagination:
max_limit: 5000
total_items_nb_key_path: '$.totalHits'
metadata_mapping:
id:
- '_ID'
- '$.displayId'
geometry:
- '{{"scene_filter": {{"spatialFilter": {{"filterType": "geojson", "geoJson": {geometry#to_geojson} }} }} }}'
- '$.spatialBounds'
title: '$.displayId'
description: '$.summary'
eo:cloud_cover:
- '{{"scene_filter": {{"cloudCoverFilter":{{"max":{eo:cloud_cover}, "includeUnknown": false }} }} }}'
- '$.cloudCover'
start_datetime:
- '{{"start_date": "{start_datetime#to_iso_utc_datetime}" }}'
- '{$.temporalCoverage.startDate#to_iso_utc_datetime}'
end_datetime:
- '{{"end_date": "{end_datetime#to_iso_utc_datetime}" }}'
- '{$.temporalCoverage.endDate#to_iso_utc_datetime}'
published: '{$.publishDate#to_iso_utc_datetime}'
ingest_after:
- '{{"scene_filter": {{"ingestFilter":{{"start":"{ingest_after#to_iso_utc_datetime}" }} }} }}'
- $.null
ingest_before:
- '{{"scene_filter": {{"ingestFilter":{{"end":"{ingest_before#to_iso_utc_datetime}" }} }} }}'
- $.null
scene_filter:
- '{{"scene_filter": {scene_filter#to_geojson} }}'
- $.null
eodag:thumbnail: '$.browse[0].thumbnailPath'
eodag:quicklook: '$.browse[0].browsePath'
order:status: '{$.available#get_group_name((?P<succeeded>True)|(?P<orderable>False))}'
eodag:download_link: 'https://earthexplorer.usgs.gov/download/external/options/{_collection}/{entityId}/M2M/'
# metadata needed for download
usgs:entityId: '$.entityId'
usgs:productId: '$.id'
extract: True
order_enabled: true
max_workers: 2
products:
# datasets list http://kapadia.github.io/usgs/_sources/reference/catalog/ee.txt may be outdated
# see also https://dds.cr.usgs.gov/ee-data/coveragemaps/shp/ee/
LANDSAT_C2L1:
_collection: landsat_ot_c2_l1
LANDSAT_C2L2:
_collection: landsat_ot_c2_l2
LANDSAT_TM_C1:
_collection: landsat_tm_c1
LANDSAT_TM_C2L1:
_collection: landsat_tm_c2_l1
LANDSAT_TM_C2L2:
_collection: landsat_tm_c2_l2
LANDSAT_ETM_C1:
_collection: landsat_etm_c1
LANDSAT_ETM_C2L1:
_collection: landsat_etm_c2_l1
LANDSAT_ETM_C2L2:
_collection: landsat_etm_c2_l2
S2_MSI_L1C:
_collection: SENTINEL_2A
GENERIC_COLLECTION:
_collection: '{collection}'
---
!provider # MARK: aws_eos
name: aws_eos
priority: 0
description: EOS search for Amazon public datasets
roles:
- host
url: https://developers.eos.com/datasets_description.html
search: !plugin
type: PostJsonSearch
api_endpoint: 'https://gate.eos.com/api/lms/search/v2/{_collection}'
need_auth: true
auth_error_code:
- 402
- 403
results_entry: 'results'
ssl_verify: true
pagination:
next_page_query_obj: '{{"limit":{limit},"page":{next_page_token}}}'
total_items_nb_key_path: '$.meta.found'
# 2021/04/28: aws_eos doesn't specify a limit in its docs. It says that the default
# value is 500 (https://doc.eos.com/search.api/#single-dataset-search).
# Let's set it to this value for now
max_limit: 500
discover_metadata:
auto_discovery: true
metadata_pattern: '^[a-zA-Z0-9_]+$'
search_param: '{{{{"search":{{{{"{metadata}":"{{{metadata}}}" }}}} }}}}'
metadata_path: '$.*'
metadata_mapping:
_collection:
- '{{"collection": "{_collection}"}}'
- '$.null'
geometry:
- '{{"search":{{"shape": {geometry#to_geojson} }} }}'
- '$.dataGeometry'
# order:status set to succeeded for consistency between providers
order:status: '{$.null#replace_str("Not Available","succeeded")}'
products:
CBERS4_PAN10M_L2:
instruments: PAN10M
_collection: cbers4
processing:level: 2
metadata_mapping:
# OpenSearch Parameters for Collection Search (Table 3)
product:type: '$.null'
platform:
- '{{"search":{{"satelliteName":"{platform}" }} }}'
- '$.satelliteName'
instruments:
- '{{"search":{{"sensor":"{instruments#csv_list}" }} }}'
- '{$.sensor#split( )}'
processing:level:
- '{{"search":{{"processingLevel":"{processing:level}" }} }}'
- '$.processingLevel'
# INSPIRE obligated OpenSearch Parameters for Collection Search (Table 4)
title:
- '{{"search":{{"sceneID":"{title}" }} }}'
- '$.sceneID'
# OpenSearch Parameters for Acquistion Parameters Search (Table 6)
start_datetime:
- '{{"search":{{"date":{{"from":"{start_datetime}"}} }} }}'
- '$.date'
end_datetime:
- '{{"search":{{"date":{{"to":"{end_datetime}"}} }} }}'
- '$.date'
view:sun_azimuth:
- '{{"search":{{"sunAzimuth":"{view:sun_azimuth}" }} }}'
- '$.sunAzimuth'
view:sun_elevation:
- '{{"search":{{"sunElevation":"{view:sun_elevation}" }} }}'
- '$.sunElevation'
# Custom parameters (not defined in the base document referenced above)
_aws_path: '$.downloadUrl'
eodag:download_link: 's3://cbers-pds/{_aws_path}'
eodag:mtd_download_link: 's3://cbers-meta-pds/{_aws_path}'
_preview_basename: '{$.sceneID#replace_str("_L2","")}'
eodag:thumbnail: 'https://s3.amazonaws.com/cbers-meta-pds/{_aws_path}/{_preview_basename}_small.jpeg'
eodag:quicklook: 'https://s3.amazonaws.com/cbers-meta-pds/{_aws_path}/{_preview_basename}.jpg'
id:
- '{{"search":{{"sceneID":"{id}" }} }}'
- '{title}'
CBERS4_PAN10M_L4:
instruments: PAN10M
_collection: cbers4
processing:level: 4
metadata_mapping_from_product: CBERS4_PAN10M_L2
metadata_mapping:
# Custom parameters (not defined in the base document referenced above)
_preview_basename: '{$.sceneID#replace_str("_L4","")}'
eodag:thumbnail: 'https://s3.amazonaws.com/cbers-meta-pds/{_aws_path}/{_preview_basename}_small.jpeg'
eodag:quicklook: 'https://s3.amazonaws.com/cbers-meta-pds/{_aws_path}/{_preview_basename}.jpg'
CBERS4_PAN5M_L2:
instruments: PAN5M
_collection: cbers4
processing:level: 2
metadata_mapping_from_product: CBERS4_PAN10M_L2
CBERS4_PAN5M_L4:
instruments: PAN5M
_collection: cbers4
processing:level: 4
metadata_mapping_from_product: CBERS4_PAN10M_L2
metadata_mapping:
# Custom parameters (not defined in the base document referenced above)
_preview_basename: '{$.sceneID#replace_str("_L4","")}'
eodag:thumbnail: 'https://s3.amazonaws.com/cbers-meta-pds/{_aws_path}/{_preview_basename}_small.jpeg'
eodag:quicklook: 'https://s3.amazonaws.com/cbers-meta-pds/{_aws_path}/{_preview_basename}.jpg'
CBERS4_MUX_L2:
instruments: MUX
_collection: cbers4
processing:level: 2
metadata_mapping_from_product: CBERS4_PAN10M_L2
CBERS4_MUX_L4:
instruments: MUX
_collection: cbers4
processing:level: 4
metadata_mapping_from_product: CBERS4_PAN10M_L2
metadata_mapping:
# Custom parameters (not defined in the base document referenced above)
_preview_basename: '{$.sceneID#replace_str("_L4","")}'
eodag:thumbnail: 'https://s3.amazonaws.com/cbers-meta-pds/{_aws_path}/{_preview_basename}_small.jpeg'
eodag:quicklook: 'https://s3.amazonaws.com/cbers-meta-pds/{_aws_path}/{_preview_basename}.jpg'
CBERS4_AWFI_L2:
instruments: AWFI
_collection: cbers4
processing:level: 2
metadata_mapping_from_product: CBERS4_PAN10M_L2
CBERS4_AWFI_L4:
instruments: AWFI
_collection: cbers4
processing:level: 4
metadata_mapping_from_product: CBERS4_PAN10M_L2
metadata_mapping:
# Custom parameters (not defined in the base document referenced above)
_preview_basename: '{$.sceneID#replace_str("_L4","")}'
eodag:thumbnail: 'https://s3.amazonaws.com/cbers-meta-pds/{_aws_path}/{_preview_basename}_small.jpeg'
eodag:quicklook: 'https://s3.amazonaws.com/cbers-meta-pds/{_aws_path}/{_preview_basename}.jpg'
L8_OLI_TIRS_C1L1:
_collection: landsat8
_on_amazon: true
metadata_mapping:
# OpenSearch Parameters for Collection Search (Table 3)
product:type: '$.null'
constellation:
- '{{"search":{{"satelliteName":"{constellation}" }} }}'
- '$.satelliteName'
instruments:
- '{{"search":{{"sensor":"{instruments#csv_list}" }} }}'
- '{$.sensor#split( )}'
# INSPIRE obligated OpenSearch Parameters for Collection Search (Table 4)
title:
- '{{"search":{{"productID":"{title}" }} }}'
- '$.productID'
# OpenSearch Parameters for Product Search (Table 5)
eo:cloud_cover:
- '{{"search":{{"cloudCoverage":"{eo:cloud_cover}" }} }}'
- '$.cloudCoverage'
start_datetime:
- '{{"search":{{"date":{{"from":"{start_datetime}"}} }} }}'
- '$.sceneStartTime'
end_datetime:
- '{{"search":{{"date":{{"to":"{end_datetime}"}} }} }}'
- '$.sceneStopTime'
view:sun_azimuth:
- '{{"search":{{"sunAzimuth":"{view:sun_azimuth}" }} }}'
- '$.sunAzimuth'
view:sun_elevation:
- '{{"search":{{"sunElevation":"{view:sun_elevation}" }} }}'
- '$.sunElevation'
# Custom parameters (not defined in the base document referenced above)
_on_amazon:
- '{{"search":{{"onAmazon":"{_on_amazon}" }} }}'
- '$.onAmazon'
_path: '$.path'
_row: '$.row'
eodag:download_link: 's3://landsat-pds/c1/L8/{_path:03.0f}/{_row:03.0f}/{title}/'
eodag:thumbnail: '$.thumbnail'
eodag:quicklook: 'https://landsat-pds.s3.amazonaws.com/c1/L8/{_path:03.0f}/{_row:03.0f}/{title}/{title}_thumb_large.jpg'
id:
- '{{"search":{{"productID":"{id}" }} }}'
- '{title}'
MODIS_MCD43A4:
_collection: modis
metadata_mapping:
# OpenSearch Parameters for Collection Search (Table 3)
product:type: '$.null'
instruments:
- '{{"search":{{"satelliteName":"{instruments#csv_list}" }} }}'
- '{$.satelliteName#split( )}'
# INSPIRE obligated OpenSearch Parameters for Collection Search (Table 4)
title:
- '{{"search":{{"sceneID":"{title}" }} }}'
- '$.sceneID'
# OpenSearch Parameters for Product Search (Table 5)
eo:cloud_cover:
- '{{"search":{{"cloudCoverage":"{eo:cloud_cover}" }} }}'
- '$.cloudCoverage'
# OpenSearch Parameters for Acquistion Parameters Search (Table 6)
start_datetime:
- '{{"search":{{"date":{{"from":"{start_datetime}"}} }} }}'
- '$.BeginningDateTime'
end_datetime:
- '{{"search":{{"date":{{"to":"{end_datetime}"}} }} }}'
- '$.EndingDateTime'
# Custom parameters (not defined in the base document referenced above)
_vertical_tile_nb: '$.verticalTileNumber'
_horizontal_tile_nb: '$.horizontalTileNumber'
_doy_date: '{$.sceneID#slice_str(9,16,1)}'
eodag:download_link: 's3://modis-pds/MCD43A4.006/{_horizontal_tile_nb:02.0f}/{_vertical_tile_nb:02.0f}/{_doy_date}/'
eodag:thumbnail: '$.thumbnail'
eodag:quicklook: '$.thumbnail'
id:
- '{{"search":{{"sceneID":"{id}" }} }}'
- '{title}'
NAIP:
_collection: naip
metadata_mapping:
# OpenSearch Parameters for Collection Search (Table 3)
product:type: '$.null'
platform:
- '{{"search":{{"satelliteName":"{platform}" }} }}'
- '$.satelliteName'
# INSPIRE obligated OpenSearch Parameters for Collection Search (Table 4)
title:
- '{{"search":{{"sceneID":"{title}" }} }}'
- '$.sceneID'
# OpenSearch Parameters for Acquistion Parameters Search (Table 6)
start_datetime:
- '{{"search":{{"date":{{"from":"{start_datetime}"}} }} }}'
- '$.date'
end_datetime:
- '{{"search":{{"date":{{"to":"{end_datetime}"}} }} }}'
- '$.date'
# Custom parameters (not defined in the base document referenced above)
_aws_path: '$.awsPath'
eodag:download_link: 's3://naip-analytic/{_aws_path}'
id:
- '{{"search":{{"sceneID":"{id}" }} }}'
- '{title}'
S1_SAR_GRD:
product:type: GRD
_collection: sentinel1
metadata_mapping:
eo:cloud_cover: '{$.cloudCoverage#not_available}'
# OpenSearch Parameters for Collection Search (Table 3)
platform:
- '{{"search":{{"missionId":"{platform}" }} }}'
- '$.missionId'
# INSPIRE obligated OpenSearch Parameters for Collection Search (Table 4)
title:
- '{{"search":{{"sceneID":"{title}" }} }}'
- '$.sceneID'
# OpenSearch Parameters for Product Search (Table 5)
sat:absolute_orbit:
- '{{"search":{{"absoluteOrbitNumber":"{sat:absolute_orbit}" }} }}'
- '$.absoluteOrbitNumber'
sat:orbit_state:
- '{{"search":{{"passDirection":"{sat:orbit_state}" }} }}'
- '{$.passDirection#to_lower}'
sar:instrument_mode:
- '{{"search":{{"mode":"{sar:instrument_mode}" }} }}'
- '$.mode'
# OpenSearch Parameters for Acquistion Parameters Search (Table 6)
start_datetime:
- '{{"search":{{"date":{{"from":"{start_datetime}"}} }} }}'
- '$.date'
end_datetime:
- '{{"search":{{"date":{{"to":"{end_datetime}"}} }} }}'
- '$.date'
sar:polarizations:
- '{{"search":{{"polarization":"{sar:polarizations#replace_tuple(((["HH"],"SH"),(["VV"],"SV"),(["HH","HV"], "DH"),(["VV","VH"],"DV")))}" }} }}'
- '{$.polarization#replace_tuple((("SH",["HH"]),("SV",["VV"]),("DH",["HH","HV"]),("DV",["VV","VH"])))}'
# Custom parameters (not defined in the base document referenced above)
_aws_path: '$.awsPath'
eodag:download_link: 's3://sentinel-s1-l1c/{_aws_path}'
eodag:thumbnail: 'https://render.eosda.com/S1/thumb/{title}.png'
eodag:quicklook: 'https://render.eosda.com/S1/thumb/{title}.png'
id:
- '{{"search":{{"sceneID":"{id}" }} }}'
- '{title}'
S2_MSI_L1C:
_collection: sentinel2
metadata_mapping:
# OpenSearch Parameters for Collection Search (Table 3)
product:type: '$.null'
platform:
- '{{"search":{{"satelliteName":"{platform}" }} }}'
- '$.satelliteName'
# INSPIRE obligated OpenSearch Parameters for Collection Search (Table 4)
title:
- '{{"search":{{"productName":"{title}" }} }}'
- '$.productName'
# OpenSearch Parameters for Product Search (Table 5)
eo:cloud_cover:
- '{{"search":{{"cloudCoverage":"{eo:cloud_cover}" }} }}'
- '$.cloudCoverage'
# OpenSearch Parameters for Acquistion Parameters Search (Table 6)
start_datetime:
- '{{"search":{{"date":{{"from":"{start_datetime}"}} }} }}'
- '$.timestamp'
end_datetime:
- '{{"search":{{"date":{{"to":"{end_datetime}"}} }} }}'
- '$.timestamp'
view:sun_azimuth:
- '{{"search":{{"azimuthAngle":"{view:sun_azimuth}" }} }}'
- '$.azimuthAngle'
view:incidence_angle:
- '{{"search":{{"zenithAngle":"{view:incidence_angle}" }} }}'
- '$.zenithAngle'
# Custom parameters (not defined in the base document referenced above)
eodag:thumbnail: '{$.thumbnail#replace_str("sentinel-s2-l1c.s3.amazonaws.com","roda.sentinel-hub.com/sentinel-s2-l1c")}'
_aws_path: '$.awsPath'
eodag:download_link: 's3://sentinel-s2-l1c/{_aws_path}'
eodag:product_path: '$.productPath'
id:
- '{{"search":{{"productName":"{id}" }} }}'
- '{title}'
_processed_l2a: '$.null'
_aws_path_l2a: '$.null'
S2_MSI_L2A:
_collection: sentinel2
_processed_l2a: true
# specific QueryStringSearch usage for these parameters (replaces current query)
specific_qssearch:
parameters:
- title
- id
results_entry: '$'
_collection:
- tileInfo
- productInfo
merge_responses: true
metadata_mapping:
title:
- 'title'
- '$.name'
id:
- 'title'
- '{title}'
_aws_path_l2a: '$.tiles[0].path'
eodag:download_link: 's3://sentinel-s2-l2a/{_aws_path_l2a}'
eodag:product_path: '$.path'
start_datetime: '$.timestamp'
end_datetime: '$.timestamp'
geometry: '$.tileDataGeometry'
eodag:product_info: 'https://roda.sentinel-hub.com/sentinel-s2-l2a/{_aws_path_l2a}/productInfo.json'
originalSceneID: '$.tiles[0].datastrip.id'
metadata_mapping:
# OpenSearch Parameters for Collection Search (Table 3)
product:type: '$.null'
platform:
- '{{"search":{{"satelliteName":"{platform}" }} }}'
- '$.satelliteName'
# INSPIRE obligated OpenSearch Parameters for Collection Search (Table 4)
title: '{$.productName#fake_l2a_title_from_l1c}'
# OpenSearch Parameters for Product Search (Table 5)
eo:cloud_cover:
- '{{"search":{{"cloudCoverage":"{eo:cloud_cover}" }} }}'
- '$.cloudCoverage'
# OpenSearch Parameters for Acquistion Parameters Search (Table 6)
start_datetime:
- '{{"search":{{"date":{{"from":"{start_datetime}"}} }} }}'
- '$.timestamp'
end_datetime:
- '{{"search":{{"date":{{"to":"{end_datetime}"}} }} }}'
- '$.timestamp'
view:sun_azimuth:
- '{{"search":{{"azimuthAngle":"{view:sun_azimuth}" }} }}'
- '$.azimuthAngle'
view:incidence_angle:
- '{{"search":{{"zenithAngle":"{view:incidence_angle}" }} }}'
- '$.zenithAngle'
# Custom parameters (not defined in the base document referenced above)
eodag:thumbnail: '{$.thumbnail#replace_str("sentinel-s2-l1c.s3.amazonaws.com","roda.sentinel-hub.com/sentinel-s2-l1c")}'
eodag:quicklook: '{eodag:thumbnail}'
eodag:download_link: 's3://sentinel-s2-l2a/{_aws_path_l2a}'
_aws_path: '$.null'
eodag:product_path: '$.null'
eodag:product_info: 'https://roda.sentinel-hub.com/sentinel-s2-l2a/{_aws_path_l2a}/productInfo.json'
id:
- '{id#s2msil2a_title_to_aws_productinfo}'
- '{title}'
_processed_l2a:
- '{{"search":{{"processedL2A":"{_processed_l2a}" }} }}'
- '$.processedL2A'
_aws_path_l2a: '$.awsPathL2A'
download: !plugin
type: AwsDownload
ssl_verify: true
products:
CBERS4_MUX_L2:
default_bucket: 'cbers-pds'
complementary_url_key:
- eodag:mtd_download_link
CBERS4_AWFI_L2:
complementary_url_key:
- eodag:mtd_download_link
CBERS4_PAN5M_L2:
complementary_url_key:
- eodag:mtd_download_link
CBERS4_PAN10M_L2:
complementary_url_key:
- eodag:mtd_download_link
S1_SAR_GRD:
default_bucket: 'sentinel-s1-l1c'
build_safe: true
S2_MSI_L1C:
default_bucket: 'sentinel-s2-l1c'
build_safe: true
complementary_url_key:
- eodag:product_path
S2_MSI_L2A:
default_bucket: 'sentinel-s2-l2a'
build_safe: true
fetch_metadata:
fetch_url: '{eodag:product_info}'
fetch_format: json
update_metadata:
title: '$.name'
id: '{title}'
eodag:product_path: '$.path'
complementary_url_key:
- eodag:product_path
download_auth: !plugin
type: AwsAuth
matching_url: s3://
requester_pays: True
search_auth: !plugin
type: HttpQueryStringAuth
matching_url: https://gate.eos.com
ssl_verify: true
---
!provider # MARK: creodias
name: creodias
priority: 0
description: CloudFerro DIAS
roles:
- host
url: https://creodias.eu/
search: !plugin
type: ODataV4Search
api_endpoint: 'https://datahub.creodias.eu/odata/v1/Products'
need_auth: false
timeout: 120
ssl_verify: true
dont_quote:
- '['
- ']'
- '$'
- '='
- '&'
- ':'
- '%'
pagination:
next_page_url_tpl: '{url}?{search}&$top={limit}&$skip={skip}&$expand=Attributes&$expand=Assets'
next_page_url_key_path: '$.["@odata.nextLink"]'
next_page_token_key: skip
parse_url_key: $skip
count_tpl: '&$count=True'
total_items_nb_key_path: '$."@odata.count"'
max_limit: 1_000
sort:
sort_by_default:
- !!python/tuple [start_datetime, ASC]
sort_by_tpl: '&$orderby={sort_param} {sort_order}'
sort_param_mapping:
start_datetime: ContentDate/Start
end_datetime: ContentDate/End
published: PublicationDate
updated: ModificationDate
sort_order_mapping:
ascending: asc
descending: desc
max_sort_params: 1
results_entry: 'value'
free_text_search_operations:
$filter:
union: ' or '
wrapper: '{}'
operations:
and:
- "Collection/Name eq '{_collection}'"
- "OData.CSC.Intersects(area=geography'{geometry#to_ewkt}')"
- "Attributes/OData.CSC.StringAttribute/any(att:att/Name eq 'productType' and att/OData.CSC.StringAttribute/Value eq '{product:type}')"
- "Attributes/OData.CSC.StringAttribute/any(att:att/Name eq 'platformShortName' and att/OData.CSC.StringAttribute/Value eq '{constellation}')"
- "Attributes/OData.CSC.StringAttribute/any(att:att/Name eq 'platformSerialIdentifier' and att/OData.CSC.StringAttribute/Value eq '{platform#replace_str(\"^S[1-3]\", \"\")}')"
- "Attributes/OData.CSC.IntegerAttribute/any(att:att/Name eq 'spatialResolution' and att/OData.CSC.StringAttribute/Value eq '{gsd}')"
- "Attributes/OData.CSC.StringAttribute/any(att:att/Name eq 'authority' and att/OData.CSC.StringAttribute/Value eq '{providers}')"
- "Attributes/OData.CSC.IntegerAttribute/any(att:att/Name eq 'orbitNumber' and att/OData.CSC.StringAttribute/Value eq {sat:absolute_orbit})"
- "Attributes/OData.CSC.IntegerAttribute/any(att:att/Name eq 'relativeOrbitNumber' and att/OData.CSC.StringAttribute/Value eq {sat:relative_orbit})"
- "Attributes/OData.CSC.StringAttribute/any(att:att/Name eq 'orbitDirection' and att/OData.CSC.StringAttribute/Value eq '{sat:orbit_state#to_upper}')"
- "Attributes/OData.CSC.DoubleAttribute/any(att:att/Name eq 'cloudCover' and att/OData.CSC.DoubleAttribute/Value le {eo:cloud_cover})"
- "Attributes/OData.CSC.StringAttribute/any(att:att/Name eq 'instrumentShortName' and att/OData.CSC.StringAttribute/Value eq '{instruments#csv_list}')"
- "Attributes/OData.CSC.StringAttribute/any(att:att/Name eq 'tileId' and att/OData.CSC.StringAttribute/Value eq '{grid:code#replace_str(\"MGRS-\",\"\")}')"
- "Attributes/OData.CSC.StringAttribute/any(att:att/Name eq 'processingLevel' and att/OData.CSC.StringAttribute/Value eq '{processing:level}')"
- "Attributes/OData.CSC.StringAttribute/any(att:att/Name eq 'processingDate' and att/OData.CSC.StringAttribute/Value eq '{processing:datetime}')"
- "Attributes/OData.CSC.StringAttribute/any(att:att/Name eq 'processingCenter' and att/OData.CSC.StringAttribute/Value eq '{processing:facility}')"
- "Attributes/OData.CSC.StringAttribute/any(att:att/Name eq 'processorVersion' and att/OData.CSC.StringAttribute/Value eq '{processing:version}')"
- "Attributes/OData.CSC.StringAttribute/any(att:att/Name eq 'swathIdentifier' and att/OData.CSC.StringAttribute/Value eq '{sar:instrument_mode}')"
- "Attributes/OData.CSC.StringAttribute/any(att:att/Name eq 'datatakeID' and att/OData.CSC.StringAttribute/Value eq '{s1:datatake_id}')"
- "Attributes/OData.CSC.StringAttribute/any(att:att/Name eq 'instrumentConfigurationID' and att/OData.CSC.StringAttribute/Value eq '{s1:instrument_configuration_ID}')"
- "Attributes/OData.CSC.StringAttribute/any(att:att/Name eq 'sliceNumber' and att/OData.CSC.StringAttribute/Value eq '{s1:slice_number}')"
- "Attributes/OData.CSC.StringAttribute/any(att:att/Name eq 'totalSlices' and att/OData.CSC.StringAttribute/Value eq '{s1:total_slices}')"
- "Attributes/OData.CSC.StringAttribute/any(att:att/Name eq 'granuleIdentifier' and att/OData.CSC.StringAttribute/Value eq '{s2:tile_id}')"
- "Attributes/OData.CSC.StringAttribute/any(att:att/Name eq 'productGroupId' and att/OData.CSC.StringAttribute/Value eq '{s2:datatake_id}')"
- "Attributes/OData.CSC.StringAttribute/any(att:att/Name eq 'Name' and att/OData.CSC.StringAttribute/Value eq '{s2:product_uri}')"
- "Attributes/OData.CSC.StringAttribute/any(att:att/Name eq 'datastripId' and att/OData.CSC.StringAttribute/Value eq '{s2:datastrip_id}')"
- "Attributes/OData.CSC.StringAttribute/any(att:att/Name eq 'operationalMode' and att/OData.CSC.StringAttribute/Value eq '{s2:datatake_type}')"
- "ModificationDate gt {updated_after#to_iso_utc_datetime}"
- "ModificationDate lt {updated_before#to_iso_utc_datetime}"
- "PublicationDate gt {published_after#to_iso_utc_datetime}"
- "PublicationDate lt {published_before#to_iso_utc_datetime}"
- "ContentDate/Start lt {end_datetime#to_iso_utc_datetime}"
- "ContentDate/End gt {start_datetime#to_iso_utc_datetime}"
- "Attributes/OData.CSC.StringAttribute/any(att:att/Name eq 'polarisationChannels' and att/OData.CSC.StringAttribute/Value eq '{sar:polarizations#csv_list(%26)}')"
- contains(Name,'{id}')
discover_metadata:
auto_discovery: true
metadata_pattern: '^(?!collection)[a-zA-Z0-9]+$'
search_param:
free_text_search_operations:
$filter:
operations:
and:
- "Attributes/OData.CSC.StringAttribute/any(att:att/Name eq '{metadata}' and att/OData.CSC.StringAttribute/Value eq '{{{metadata}}}')"
metadata_path: '$.Attributes.*'
discover_collections:
fetch_url: null
per_product_metadata_query: false
metadata_pre_mapping:
metadata_path: '$.Attributes'
metadata_path_id: 'Name'
metadata_path_value: 'Value'
metadata_mapping:
_collection:
- null
- '$.null'
# hide duplicated metadata
beginningDateTime: '$.null'
endingDateTime: '$.null'
platformSerialIdentifier: '$.null'
# Opensearch resource identifier within the search engine context (in our case
# within the context of the data provider)
# Queryable parameters are set with null as 1st configuration list value to mark them as queryable,
# but `free_text_search_operations.$filter.operations.and` entries are then used instead.
uid: '$.Id'
# OpenSearch Parameters for Collection Search (Table 3)
product:type:
- null
- '$.Attributes.productType'
constellation:
- null
- '$.Attributes.platformShortName'
platform:
- null
- '$.Attributes.platformSerialIdentifier'
instruments:
- null
- '{$.Attributes.instrumentShortName#split( )}'
processing:level:
- null
- '$.Attributes.processingLevel'
processing:datetime:
- null
- '$.Attributes.processingDate'
processing:facility:
- null
- '$.Attributes.processingCenter'
processing:version:
- null
- '{$.Attributes.processorVersion#to_geojson}'
_processor_name:
- null
- '$.Attributes.processorName'
processing:software: '{{"{_processor_name}":"{processing:version}"}}'
# INSPIRE obligated OpenSearch Parameters for Collection Search (Table 4)
title: '{$.Name#remove_extension}'
gsd:
- null
- '$.Attributes.spatialResolution'
_provider: '$.Attributes.origin'
providers:
- null
- '[{{"name":"{_provider}","roles":["producer"]}}]'
published_after:
- null
- '$.null'
published_before:
- null
- '$.null'
published: '$.PublicationDate'
# OpenSearch Parameters for Product Search (Table 5)
sat:absolute_orbit:
- null
- '$.Attributes.orbitNumber'
sat:relative_orbit:
- null
- '$.Attributes.relativeOrbitNumber'
sat:orbit_state:
- null
- '{$.Attributes.orbitDirection#to_lower}'
eo:cloud_cover:
- null
- '$.Attributes.cloudCover'
updated_after:
- null
- '$.null'
updated_before:
- null
- '$.null'
updated: '$.ModificationDate'
start_datetime:
- null
- '$.ContentDate.Start'
end_datetime:
- null
- '$.ContentDate.End'
product:timeliness:
- null
- '$.Attributes.timeliness'
sar:instrument_mode:
- null
- '$.Attributes.swathIdentifier'
sar:polarizations:
- null
- '{$.Attributes.polarisationChannels#split(&)}'
s1:datatake_id:
- null
- '$.Attributes.datatakeID'
s1:instrument_configuration_ID:
- null
- '$.Attributes.instrumentConfigurationID'
s1:slice_number:
- null
- '$.Attributes.sliceNumber'
s1:total_slices:
- null
- '$.Attributes.totalSlices'
s2:tile_id:
- null
- '$.Attributes.granuleIdentifier'
s2:datatake_id:
- null
- '$.Attributes.productGroupId'
s2:product_uri:
- null
- '$.Attributes.Name'
s2:datastrip_id:
- null
- '$.Attributes.datastripId'
s2:datatake_type:
- null
- '$.Attributes.operationalMode'
# Custom parameters (not defined in the base document referenced above)
id:
- null
- '{$.Name#remove_extension}'
grid:code:
- null
- '{$.Attributes.tileId#replace_str(r"^","MGRS-")}'
# The geographic extent of the product
geometry:
- null
- '{$.Footprint#from_ewkt}'
type: "$.ContentType"
file:size: "$.ContentLength"
file:checksum: '$.Checksum[?Algorithm="MD5"].Value'
# The url to download the product "as is" (literal or as a template to be completed either after the search result
# is obtained from the provider or during the eodag download phase)
eodag:download_link: 'https://zipper.creodias.eu/odata/v1/Products({uid})/$value'
# order:status: must be one of succeeded, ordered, orderable
order:status: '{$.Online#get_group_name((?P<succeeded>True)|(?P<orderable>False))}'
collection:
- null
- $.null
eodag:quicklook: '$.Assets[?(@.Type="QUICKLOOK")].DownloadLink'
eodag:thumbnail: '$.Assets[?(@.Type="QUICKLOOK")].DownloadLink'
download: !plugin
type: HTTPDownload
extract: true
order_enabled: false
archive_depth: 2
ssl_verify: true
auth: !plugin
type: KeycloakOIDCPasswordAuth
matching_url: https://[-\w]+.creodias.eu
oidc_config_url: https://identity.cloudferro.com/auth/realms/Creodias-new/.well-known/openid-configuration
client_id: 'CLOUDFERRO_PUBLIC'
client_secret: 'dc0aca03-2dc6-4798-a5de-fc5aeb6c8ee1'
token_provision: qs
token_qs_key: 'token'
auth_error_code: 401
ssl_verify: true
allowed_audiences: ["CLOUDFERRO_PUBLIC"]
products:
# S1
S1_AUX_GNSSRD:
product:type: AUX_GNSSRD
_collection: SENTINEL-1
metadata_mapping:
eo:cloud_cover: '{$.Attributes.cloudCover#not_available}'
platform:
- null
- '$.Attributes.platformSerialIdentifier.`sub(/^/, S1)`'
S1_AUX_MOEORB:
product:type: AUX_MOEORB
_collection: SENTINEL-1
metadata_mapping:
eo:cloud_cover: '{$.Attributes.cloudCover#not_available}'
platform:
- null
- '$.Attributes.platformSerialIdentifier.`sub(/^/, S1)`'
S1_AUX_POEORB:
product:type: AUX_POEORB
_collection: SENTINEL-1
metadata_mapping:
eo:cloud_cover: '{$.Attributes.cloudCover#not_available}'
platform:
- null
- '$.Attributes.platformSerialIdentifier.`sub(/^/, S1)`'
S1_AUX_PREORB:
product:type: AUX_PREORB
_collection: SENTINEL-1
metadata_mapping:
eo:cloud_cover: '{$.Attributes.cloudCover#not_available}'
platform:
- null
- '$.Attributes.platformSerialIdentifier.`sub(/^/, S1)`'
S1_AUX_PROQUA:
product:type: AUX_PROQUA
_collection: SENTINEL-1
metadata_mapping:
eo:cloud_cover: '{$.Attributes.cloudCover#not_available}'
platform:
- null
- '$.Attributes.platformSerialIdentifier.`sub(/^/, S1)`'
S1_AUX_RESORB:
product:type: AUX_RESORB
_collection: SENTINEL-1
metadata_mapping:
eo:cloud_cover: '{$.Attributes.cloudCover#not_available}'
platform:
- null
- '$.Attributes.platformSerialIdentifier.`sub(/^/, S1)`'
S1_SAR_RAW:
product:type: RAW
_collection: SENTINEL-1
metadata_mapping:
eo:cloud_cover: '{$.Attributes.cloudCover#not_available}'
platform:
- null
- '$.Attributes.platformSerialIdentifier.`sub(/^/, S1)`'
S1_SAR_GRD:
product:type: GRD
_collection: SENTINEL-1
metadata_mapping:
eo:cloud_cover: '{$.Attributes.cloudCover#not_available}'
platform:
- null
- '$.Attributes.platformSerialIdentifier.`sub(/^/, S1)`'
S1_SAR_SLC:
product:type: SLC
_collection: SENTINEL-1
metadata_mapping:
eo:cloud_cover: '{$.Attributes.cloudCover#not_available}'
platform:
- null
- '$.Attributes.platformSerialIdentifier.`sub(/^/, S1)`'
S1_SAR_OCN:
product:type: OCN
_collection: SENTINEL-1
metadata_mapping:
eo:cloud_cover: '{$.Attributes.cloudCover#not_available}'
platform:
- null
- '$.Attributes.platformSerialIdentifier.`sub(/^/, S1)`'
S1_SAR_L3_IW_MCM:
product:type: S1SAR_L3_IW_MCM
_collection: GLOBAL-MOSAICS
metadata_mapping:
eo:cloud_cover: '{$.Attributes.cloudCover#not_available}'
platform:
- null
- '$.Attributes.platformSerialIdentifier.`sub(/^/, S1)`'
S1_SAR_L3_DH_MCM:
product:type: S1SAR_L3_DH_MCM
_collection: GLOBAL-MOSAICS
metadata_mapping:
eo:cloud_cover: '{$.Attributes.cloudCover#not_available}'
platform:
- null
- '$.Attributes.platformSerialIdentifier.`sub(/^/, S1)`'
# S2
S2_MSI_L1C:
_collection: SENTINEL-2
product:type: S2MSI1C
metadata_mapping:
platform:
- null
- '$.Attributes.platformSerialIdentifier.`sub(/^/, S2)`'
S2_MSI_L2A:
_collection: SENTINEL-2
product:type: S2MSI2A
metadata_mapping:
platform:
- null
- '$.Attributes.platformSerialIdentifier.`sub(/^/, S2)`'
# S3 SRAL
S3_SRA:
product:type: SR_1_SRA___
_collection: SENTINEL-3
metadata_mapping:
eo:cloud_cover: '{$.Attributes.cloudCover#not_available}'
platform:
- null
- '$.Attributes.platformSerialIdentifier.`sub(/^/, S3)`'
S3_SRA_A:
product:type: SR_1_SRA_A_
_collection: SENTINEL-3
metadata_mapping:
eo:cloud_cover: '{$.Attributes.cloudCover#not_available}'
platform:
- null
- '$.Attributes.platformSerialIdentifier.`sub(/^/, S3)`'
S3_SRA_BS:
product:type: SR_1_SRA_BS
_collection: SENTINEL-3
metadata_mapping:
eo:cloud_cover: '{$.Attributes.cloudCover#not_available}'
platform:
- null
- '$.Attributes.platformSerialIdentifier.`sub(/^/, S3)`'
S3_LAN:
product:type: SR_2_LAN___
_collection: SENTINEL-3
metadata_mapping:
eo:cloud_cover: '{$.Attributes.cloudCover#not_available}'
platform:
- null
- '$.Attributes.platformSerialIdentifier.`sub(/^/, S3)`'
S3_LAN_HY:
product:type: SR_2_LAN_HY
_collection: SENTINEL-3
metadata_mapping:
eo:cloud_cover: '{$.Attributes.cloudCover#not_available}'
platform:
- null
- '$.Attributes.platformSerialIdentifier.`sub(/^/, S3)`'
S3_LAN_SI:
product:type: SR_2_LAN_SI
_collection: SENTINEL-3
metadata_mapping:
eo:cloud_cover: '{$.Attributes.cloudCover#not_available}'
platform:
- null
- '$.Attributes.platformSerialIdentifier.`sub(/^/, S3)`'
S3_LAN_LI:
product:type: SR_2_LAN_LI
_collection: SENTINEL-3
metadata_mapping:
eo:cloud_cover: '{$.Attributes.cloudCover#not_available}'
platform:
- null
- '$.Attributes.platformSerialIdentifier.`sub(/^/, S3)`'
S3_WAT:
product:type: SR_2_WAT___
_collection: SENTINEL-3
metadata_mapping:
eo:cloud_cover: '{$.Attributes.cloudCover#not_available}'
platform:
- null
- '$.Attributes.platformSerialIdentifier.`sub(/^/, S3)`'
# S3 OLCI
S3_EFR:
product:type: OL_1_EFR___
_collection: SENTINEL-3
metadata_mapping:
platform:
- null
- '$.Attributes.platformSerialIdentifier.`sub(/^/, S3)`'
S3_ERR:
product:type: OL_1_ERR___
_collection: SENTINEL-3
metadata_mapping:
platform:
- null
- '$.Attributes.platformSerialIdentifier.`sub(/^/, S3)`'
S3_OLCI_L2LRR:
product:type: OL_2_LRR___
_collection: SENTINEL-3
metadata_mapping:
platform:
- null
- '$.Attributes.platformSerialIdentifier.`sub(/^/, S3)`'
S3_OLCI_L2LFR:
product:type: OL_2_LFR___
_collection: SENTINEL-3
metadata_mapping:
platform:
- null
- '$.Attributes.platformSerialIdentifier.`sub(/^/, S3)`'
S3_OLCI_L2WRR:
product:type: OL_2_WRR___
_collection: SENTINEL-3
metadata_mapping:
platform:
- null
- '$.Attributes.platformSerialIdentifier.`sub(/^/, S3)`'
S3_OLCI_L2WFR:
product:type: OL_2_WFR___
_collection: SENTINEL-3
metadata_mapping:
platform:
- null
- '$.Attributes.platformSerialIdentifier.`sub(/^/, S3)`'
# S3 SLSTR
S3_SLSTR_L1RBT:
product:type: SL_1_RBT___
_collection: SENTINEL-3
metadata_mapping:
platform:
- null
- '$.Attributes.platformSerialIdentifier.`sub(/^/, S3)`'
S3_SLSTR_L2LST:
product:type: SL_2_LST___
_collection: SENTINEL-3
metadata_mapping:
platform:
- null
- '$.Attributes.platformSerialIdentifier.`sub(/^/, S3)`'
S3_SLSTR_L2WST:
product:type: SL_2_WST___
_collection: SENTINEL-3
metadata_mapping:
platform:
- null
- '$.Attributes.platformSerialIdentifier.`sub(/^/, S3)`'
S3_SLSTR_L2AOD:
product:type: SL_2_AOD___
_collection: SENTINEL-3
metadata_mapping:
platform:
- null
- '$.Attributes.platformSerialIdentifier.`sub(/^/, S3)`'
S3_SLSTR_L2FRP:
product:type: SL_2_FRP___
_collection: SENTINEL-3
metadata_mapping:
platform:
- null
- '$.Attributes.platformSerialIdentifier.`sub(/^/, S3)`'
# S3 SY
S3_SY_AOD:
product:type: SY_2_AOD___
_collection: SENTINEL-3
metadata_mapping:
platform:
- null
- '$.Attributes.platformSerialIdentifier.`sub(/^/, S3)`'
S3_SY_SYN:
product:type: SY_2_SYN___
_collection: SENTINEL-3
metadata_mapping:
platform:
- null
- '$.Attributes.platformSerialIdentifier.`sub(/^/, S3)`'
S3_SY_V10:
product:type: SY_2_V10___
_collection: SENTINEL-3
metadata_mapping:
platform:
- null
- '$.Attributes.platformSerialIdentifier.`sub(/^/, S3)`'
S3_SY_VG1:
product:type: SY_2_VG1___
_collection: SENTINEL-3
metadata_mapping:
platform:
- null
- '$.Attributes.platformSerialIdentifier.`sub(/^/, S3)`'
S3_SY_VGP:
product:type: SY_2_VGP___
_collection: SENTINEL-3
metadata_mapping:
platform:
- null
- '$.Attributes.platformSerialIdentifier.`sub(/^/, S3)`'
# S5P L1
S5P_L1B_IR_SIR:
product:type: L1B_IR_SIR
_collection: SENTINEL-5P
S5P_L1B_IR_UVN:
product:type: L1B_IR_UVN
_collection: SENTINEL-5P
S5P_L1B_RA_BD1:
product:type: L1B_RA_BD1
_collection: SENTINEL-5P
S5P_L1B_RA_BD2:
product:type: L1B_RA_BD2
_collection: SENTINEL-5P
S5P_L1B_RA_BD3:
product:type: L1B_RA_BD3
_collection: SENTINEL-5P
S5P_L1B_RA_BD4:
product:type: L1B_RA_BD4
_collection: SENTINEL-5P
S5P_L1B_RA_BD5:
product:type: L1B_RA_BD5
_collection: SENTINEL-5P
S5P_L1B_RA_BD6:
product:type: L1B_RA_BD6
_collection: SENTINEL-5P
S5P_L1B_RA_BD7:
product:type: L1B_RA_BD7
_collection: SENTINEL-5P
S5P_L1B_RA_BD8:
product:type: L1B_RA_BD8
_collection: SENTINEL-5P
# S5P L2
S5P_L2_NO2:
product:type: L2__NO2___
_collection: SENTINEL-5P
S5P_L2_CLOUD:
product:type: L2__CLOUD_
_collection: SENTINEL-5P
S5P_L2_O3:
product:type: L2__O3____
_collection: SENTINEL-5P
S5P_L2_CO:
product:type: L2__CO____
_collection: SENTINEL-5P
S5P_L2_AER_AI:
product:type: L2__AER_AI
_collection: SENTINEL-5P
S5P_L2_O3_PR:
product:type: L2__O3__PR
_collection: SENTINEL-5P
S5P_L2_O3_TCL:
product:type: L2__O3_TCL
_collection: SENTINEL-5P
S5P_L2_AER_LH:
product:type: L2__AER_LH
_collection: SENTINEL-5P
S5P_L2_HCHO:
product:type: L2__HCHO__
_collection: SENTINEL-5P
S5P_L2_CH4:
product:type: L2__CH4___
_collection: SENTINEL-5P
S5P_L2_NP_BD3:
product:type: L2__NP_BD3
_collection: SENTINEL-5P
S5P_L2_NP_BD6:
product:type: L2__NP_BD6
_collection: SENTINEL-5P
S5P_L2_NP_BD7:
product:type: L2__NP_BD7
_collection: SENTINEL-5P
S5P_L2_SO2:
product:type: L2__SO2___
_collection: SENTINEL-5P
# COP DEM
COP_DEM_GLO30_DGED:
product:type: DGE_30
_collection: COP-DEM
metadata_mapping:
eo:cloud_cover: '{$.Attributes.cloudCover#not_available}'
_provider: '$.Attributes.authority'
COP_DEM_GLO30_DTED:
product:type: DTE_30
_collection: COP-DEM
metadata_mapping:
eo:cloud_cover: '{$.Attributes.cloudCover#not_available}'
_provider: '$.Attributes.authority'
COP_DEM_GLO90_DGED:
product:type: DGE_90
_collection: COP-DEM
metadata_mapping:
eo:cloud_cover: '{$.Attributes.cloudCover#not_available}'
_provider: '$.Attributes.authority'
COP_DEM_GLO90_DTED:
product:type: DTE_90
_collection: COP-DEM
metadata_mapping:
eo:cloud_cover: '{$.Attributes.cloudCover#not_available}'
_provider: '$.Attributes.authority'
GENERIC_COLLECTION:
_collection: '{collection}'
---
!provider # MARK: usgs_satapi_aws
name: usgs_satapi_aws
priority: 0
roles:
- host
description: USGS Landsatlook SAT API
url: https://landsatlook.usgs.gov/stac-server
search: !plugin
type: StacSearch
api_endpoint: https://landsatlook.usgs.gov/stac-server/search
need_auth: false
ssl_verify: true
discover_queryables:
fetch_url: null
collection_fetch_url: null
pagination:
# 2021/03/19: no more than 10_000 (if greater, returns a 500 error code)
# but in practive if an Internal Server Error is returned for more than
# about 500 products.
max_limit: 500
next_page_token_key: next
# Note: Sorting is intentionally disabled for this provider, as enabling it causes pagination to malfunction.
metadata_mapping:
assets: '{$.assets#from_alternate(s3)}'
products:
LANDSAT_C2L1:
_collection: landsat-c2l1
LANDSAT_C2L2_SR:
_collection: landsat-c2l2-sr
LANDSAT_C2L2_ST:
_collection: landsat-c2l2-st
LANDSAT_C2L2ALB_BT:
_collection: landsat-c2l2alb-bt
LANDSAT_C2L2ALB_SR:
_collection: landsat-c2l2alb-sr
LANDSAT_C2L2ALB_ST:
_collection: landsat-c2l2alb-st
LANDSAT_C2L2ALB_TA:
_collection: landsat-c2l2alb-ta
GENERIC_COLLECTION:
_collection: '{collection}'
download: !plugin
type: AwsDownload
ssl_verify: true
auth: !plugin
type: AwsAuth
matching_url: s3://
requester_pays: True
---
!provider # MARK: earth_search
name: earth_search
priority: 0
roles:
- host
description: Earth Search
url: https://www.element84.com/earth-search/
anchor_mgrs: &mgrs
mgrs:utm_zone:
- '{{"query":{{"mgrs:utm_zone":{{"eq":"{mgrs:utm_zone}"}}}}}}'
- '$.properties."mgrs:utm_zone"'
mgrs:latitude_band:
- '{{"query":{{"mgrs:latitude_band":{{"eq":"{mgrs:latitude_band}"}}}}}}'
- '$.properties."mgrs:latitude_band"'
mgrs:grid_square:
- '{{"query":{{"mgrs:grid_square":{{"eq":"{mgrs:grid_square}"}}}}}}'
- '$.properties."mgrs:grid_square"'
grid:code:
- '{{"query":{{"mgrs:utm_zone":{{"eq":"{grid:code#slice_str(5,7,1)}"}},"mgrs:latitude_band":{{"eq":"{grid:code#slice_str(7,8,1)}"}},"mgrs:grid_square":{{"eq":"{grid:code#slice_str(8,10,1)}"}}}}}}'
- 'MGRS-{mgrs:utm_zone}{mgrs:latitude_band}{mgrs:grid_square}'
search: !plugin
type: StacSearch
api_endpoint: https://earth-search.aws.element84.com/v1/search
need_auth: false
ssl_verify: true
discover_collections:
results_entry: '$.collections[?id!="sentinel-s2-l2a-cogs"]'
discover_queryables:
fetch_url: null
collection_fetch_url: null
pagination:
# Override the default next page url key path of StacSearch because the next link returned
# by Earth Search is invalid (as of 2021/04/29). Entry set to null (None) to avoid using the
# next page retrieval mechanism, `next_page_url_tpl` will be used instead (inherited from StacSearch)
# Remove that entry if Earth Search updates that and returns a valid link.
next_page_url_key_path: null
next_page_query_obj_key_path: null
next_page_token_key: page
# 2021/04/28: Earth-Search relies on Sat-API whose docs (http://sat-utils.github.io/sat-api/#search-stac-items-by-simple-filtering-)
# say the max is 10_000. In practice a too high number (e.g. 5_000) returns a 502 error ({"message": "Internal server error"}).
# Let's set it to a more robust number: 500
max_limit: 500
sort:
sort_by_default:
- !!python/tuple [start_datetime, ASC]
sort_param_mapping:
id: id
start_datetime: properties.datetime
created: properties.created
updated: properties.updated
platform: properties.platform
gsd: properties.gsd
eo:cloud_cover: properties.eo:cloud_cover
metadata_mapping:
grid:code:
- '{{"query":{{"grid:code":{{"eq":"{grid:code}"}}}}}}'
- '$.properties.grid:code'
assets: '{$.assets#dict_filter($[?(href=~"^s3.*")])}'
products:
S1_SAR_GRD:
_collection: sentinel-1-grd
S2_MSI_L1C:
_collection: sentinel-2-l1c
metadata_mapping:
id:
- '{{"query":{{"s2:product_uri":{{"eq":"{id}.SAFE"}}}}}}'
- '{$.properties."s2:product_uri"#remove_extension}'
title: '{$.properties."s2:product_uri"#remove_extension}'
platform: '$.id.`split(_, 0, -1)`'
eodag:product_path: |
$.properties."s2:product_uri".`sub(/([S2AB]{3})_MSIL1C_([0-9]{4})([0-9]{2})([0-9]{2})(T.*).SAFE/, products!\\2!\\3!\\4!\\1_MSIL1C_\\2\\3\\4\\5)`.`sub(/!0*/, /)`
eodag:tile_path: |
$.assets.tileinfo_metadata.href.`sub(/.*/sentinel-s2-l1c\/(tiles\/.*)\/tileInfo\.json/, \\1)`
<<: *mgrs
S2_MSI_L2A_COG:
_collection: sentinel-2-l2a
metadata_mapping:
<<: *mgrs
assets: '{$.assets#dict_filter_and_sub("$[?(@.href =~ \"^https://sentinel-cogs.*/\")]","https\u003A//[a-z0-9-\.]+/","s3\u003A//sentinel-cogs/")}'
LANDSAT_C2L2:
_collection: landsat-c2-l2
NAIP:
_collection: naip
COP_DEM_GLO30_DGED:
_collection: cop-dem-glo-30
COP_DEM_GLO90_DGED:
_collection: cop-dem-glo-90
GENERIC_COLLECTION:
_collection: '{collection}'
download: !plugin
type: AwsDownload
ssl_verify: true
products:
S2_MSI_L1C:
default_bucket: 'sentinel-s2-l1c'
build_safe: true
complementary_url_key:
- eodag:product_path
- eodag:tile_path
auth: !plugin
type: AwsAuth
matching_url: s3://
requester_pays: True
---
!provider # MARK: earth_search_gcs
name: earth_search_gcs
priority: 0
roles:
- host
description: Google Cloud Storage through Earth Search
url: https://www.element84.com/earth-search/
search: !plugin
type: StacSearch
api_endpoint: https://earth-search.aws.element84.com/v0/search
need_auth: false
ssl_verify: true
discover_collections:
fetch_url: null
discover_queryables:
fetch_url: null
collection_fetch_url: null
pagination:
# Override the default next page url key path of StacSearch because the next link returned
# by Earth Search is invalid (as of 2021/04/29). Entry set to null (None) to avoid using the
# next page retrieval mechanism, `next_page_url_tpl` will be used instead (inherited from StacSearch)
# Remove that entry if Earth Search updates that and returns a valid link.
next_page_url_key_path: null
next_page_query_obj_key_path: null
next_page_token_key: page
# 2021/04/28: Earth-Search relies on Sat-API whose docs (http://sat-utils.github.io/sat-api/#search-stac-items-by-simple-filtering-)
# say the max is 10_000. In practice a too high number (e.g. 5_000) returns a 502 error ({"message": "Internal server error"}).
# Let's set it to a more robust number: 500
max_limit: 500
sort:
sort_by_default:
- !!python/tuple [start_datetime, ASC]
sort_param_mapping:
id: id
start_datetime: properties.datetime
created: properties.created
updated: properties.updated
platform: properties.platform
gsd: properties.gsd
eo:cloud_cover: properties.eo:cloud_cover
products:
S2_MSI_L1C:
_collection: sentinel-s2-l1c
metadata_mapping:
title: '$.properties."sentinel:product_id"'
platform: '$.id.`split(_, 0, -1)`'
mgrs:utm_zone: '$.properties."sentinel:utm_zone"'
mgrs:latitude_band: '$.properties."sentinel:latitude_band"'
mgrs:grid_square: '$.properties."sentinel:grid_square"'
eodag:download_link: 's3://gcp-public-data-sentinel-2/tiles/{mgrs:utm_zone}/{mgrs:latitude_band}/{mgrs:grid_square}/{title}.SAFE'
L8_OLI_TIRS_C1L1:
_collection: landsat-8-l1-c1
metadata_mapping:
wrsPath: '$.properties."landsat:wrs_path"'
wrsRow: '$.properties."landsat:wrs_row"'
eodag:download_link: 's3://gcp-public-data-landsat/LC08/01/{wrsPath:03d}/{wrsRow:03d}/{title}'
GENERIC_COLLECTION:
_collection: '{collection}'
download: !plugin
type: AwsDownload
s3_endpoint: https://storage.googleapis.com
ignore_assets: True
ssl_verify: true
products:
S2_MSI_L1C:
default_bucket: 'gcp-public-data-sentinel-2'
auth: !plugin
type: AwsAuth
matching_url: s3://
s3_endpoint: https://storage.googleapis.com
support_presign_url: False
matching_conf:
s3_endpoint: https://storage.googleapis.com
---
!provider # MARK: ecmwf
name: ecmwf
priority: 0
description: ECMWF archive products
roles:
- host
url: https://www.ecmwf.int
api: !plugin
type: EcmwfApi
auth_endpoint: https://api.ecmwf.int/v1
metadata_mapping:
product:type: $.productType
start_datetime: '{$.start_datetime#to_iso_utc_datetime}'
end_datetime:
- 'date={start_datetime#to_iso_date}/to/{end_datetime#to_iso_date(-1,)}'
- '{$.end_datetime#to_iso_utc_datetime}'
# The geographic extent of the product
geometry:
- 'area={geometry#to_nwse_bounds_str(/)}'
- '$.geometry'
eodag:default_geometry: 'POLYGON((180 -90, 180 90, -180 90, -180 -90, 180 -90))'
# order:status set to succeeded for consistency between providers
order:status: '{$.null#replace_str("Not Available","succeeded")}'
qs: $.qs
eodag:download_link: 'https://apps.ecmwf.int/datasets/data/{dataset}?{qs#to_geojson}'
products:
# See Archive Catalog in https://apps.ecmwf.int/archive-catalogue/
# See available Public Datasets in https://apps.ecmwf.int/datasets/
TIGGE_CF_SFC:
dataset: tigge
GENERIC_COLLECTION:
dataset: '{collection}'
---
!provider # MARK: cop_ads
name: cop_ads
priority: 0
description: Copernicus Atmosphere Data Store
roles:
- host
url: https://ads.atmosphere.copernicus.eu/
anchor_month_year: &month_year
end_datetime:
- |
{{
"year": {_date#interval_to_datetime_dict}["year"],
"month": {_date#interval_to_datetime_dict}["month"]
}}
- '{$.end_datetime#to_iso_date}'
auth: !plugin
type: HTTPHeaderAuth
ssl_verify: true
headers:
PRIVATE-TOKEN: "{apikey}"
download: !plugin
type: HTTPDownload
timeout: 30
ssl_verify: true
auth_error_code: 401
order_enabled: true
order_method: POST
order_on_response:
metadata_mapping:
eodag:order_id: $.json.jobID
eodag:status_link: https://ads.atmosphere.copernicus.eu/api/retrieve/v1/jobs/{eodag:order_id}?request=true
eodag:search_link: https://ads.atmosphere.copernicus.eu/api/retrieve/v1/jobs/{eodag:order_id}/results
order_status:
request:
method: GET
metadata_mapping:
eodag:order_status: $.json.status
order:date: $.json.created
created: $.json.created
published: $.json.finished
updated: $.json.updated
ecmwf:dataset: $.json.processID
eodag:request_params: $.json.metadata.request.ids
error:
eodag:order_status: failed
success:
eodag:order_status: successful
on_success:
need_search: true
metadata_mapping:
eodag:download_link: $.json.asset.value.href
search: !plugin
type: ECMWFSearch
need_auth: true
ssl_verify: true
timeout: 30
end_date_excluded: false
remove_from_query:
- dataset
discover_queryables:
fetch_url: null
collection_fetch_url: null
constraints_url: "https://ads.atmosphere.copernicus.eu/api/catalogue/v1/collections/{dataset}/constraints.json"
form_url: https://ads.atmosphere.copernicus.eu/api/catalogue/v1/collections/{dataset}/form.json
metadata_mapping:
product:type: $.productType
start_datetime: '{$.start_datetime#to_iso_utc_datetime}'
end_datetime:
- 'date={start_datetime#to_iso_date}/{end_datetime#to_iso_date}'
- '{$.end_datetime#to_iso_utc_datetime}'
# The geographic extent of the product
geometry:
- '{{"area": {geometry#to_nwse_bounds}}}'
- $.geometry
qs: $.qs
eodag:order_link: 'https://ads.atmosphere.copernicus.eu/api/retrieve/v1/processes/{dataset}/execution?{{"inputs": {qs#to_geojson}}}'
products:
# See available Public Datasets in https://ads.atmosphere.copernicus.eu/cdsapp#!/search?type=dataset
CAMS_GAC_FORECAST:
dataset: cams-global-atmospheric-composition-forecasts
CAMS_GFE_GFAS:
dataset: cams-global-fire-emissions-gfas
CAMS_EU_AIR_QUALITY_FORECAST:
dataset: cams-europe-air-quality-forecasts
CAMS_EU_AIR_QUALITY_RE:
dataset: cams-europe-air-quality-reanalyses
metadata_mapping:
<<: *month_year
CAMS_GRF:
dataset: cams-global-radiative-forcings
metadata_mapping:
<<: *month_year
CAMS_GRF_AUX:
dataset: cams-global-radiative-forcing-auxilliary-variables
metadata_mapping:
<<: *month_year
CAMS_SOLAR_RADIATION:
dataset: cams-solar-radiation-timeseries
altitude: "-999"
metadata_mapping:
geometry:
- '{{"location": {{"longitude": {geometry#to_longitude_latitude}["lon"], "latitude": {geometry#to_longitude_latitude}["lat"]}}}}'
- $.geometry
CAMS_GREENHOUSE_EGG4_MONTHLY:
dataset: cams-global-ghg-reanalysis-egg4-monthly
metadata_mapping:
<<: *month_year
CAMS_GREENHOUSE_EGG4:
dataset: cams-global-ghg-reanalysis-egg4
CAMS_GREENHOUSE_INVERSION:
dataset: cams-global-greenhouse-gas-inversion
metadata_mapping:
<<: *month_year
CAMS_GLOBAL_EMISSIONS:
dataset: cams-global-emission-inventories
metadata_mapping:
end_datetime:
- |
{{
"year": {_date#interval_to_datetime_dict}["year"]
}}
- '{$.end_datetime#to_iso_date}'
CAMS_EAC4:
dataset: cams-global-reanalysis-eac4
CAMS_EAC4_MONTHLY:
dataset: cams-global-reanalysis-eac4-monthly
metadata_mapping:
<<: *month_year
GENERIC_COLLECTION:
dataset: '{collection}'
---
!provider # MARK: cop_cds
name: cop_cds
priority: 0
description: Copernicus Climate Data Store
roles:
- host
url: https://cds.climate.copernicus.eu
# anchors to avoid duplications
anchor_nday_month_year: &nday_month_year
end_datetime:
- |
{{
"year": {_date#interval_to_datetime_dict}["year"],
"month": {_date#interval_to_datetime_dict}["month"],
"nominal_day": {_date#interval_to_datetime_dict}["day"]
}}
- '{$.end_datetime#to_iso_date}'
anchor_day_month_year: &day_month_year
end_datetime:
- |
{{
"year": {_date#interval_to_datetime_dict}["year"],
"month": {_date#interval_to_datetime_dict}["month"],
"day": {_date#interval_to_datetime_dict}["day"]
}}
- '{$.end_datetime#to_iso_date}'
anchor_time_day_month_year: &time_day_month_year
end_datetime:
- |
{{
"year": {_date#interval_to_datetime_dict}["year"],
"month": {_date#interval_to_datetime_dict}["month"],
"day": {_date#interval_to_datetime_dict}["day"],
"time": {start_datetime#get_ecmwf_time}
}}
- '{$.end_datetime#to_iso_date}'
anchor_month_year: &month_year
end_datetime:
- |
{{
"year": {_date#interval_to_datetime_dict}["year"],
"month": {_date#interval_to_datetime_dict}["month"]
}}
- '{$.end_datetime#to_iso_date}'
anchor_time_month_year: &time_month_year
end_datetime:
- |
{{
"year": {_date#interval_to_datetime_dict}["year"],
"month": {_date#interval_to_datetime_dict}["month"],
"time": {start_datetime#get_ecmwf_time}
}}
- '{$.end_datetime#to_iso_date}'
auth: !plugin
type: HTTPHeaderAuth
ssl_verify: true
headers:
PRIVATE-TOKEN: "{apikey}"
download: !plugin
type: HTTPDownload
timeout: 30
ssl_verify: true
order_enabled: true
auth_error_code: 401
order_method: POST
order_on_response:
metadata_mapping:
eodag:order_id: $.json.jobID
eodag:status_link: https://cds.climate.copernicus.eu/api/retrieve/v1/jobs/{eodag:order_id}?request=true
eodag:search_link: https://cds.climate.copernicus.eu/api/retrieve/v1/jobs/{eodag:order_id}/results
order_status:
request:
method: GET
metadata_mapping:
eodag:order_status: $.json.status
order:date: $.json.created
created: $.json.created
published: $.json.finished
updated: $.json.updated
ecmwf:dataset: $.json.processID
eodag:request_params: $.json.metadata.request.ids
error:
eodag:order_status: failed
success:
eodag:order_status: successful
on_success:
need_search: true
metadata_mapping:
eodag:download_link: $.json.asset.value.href
search: !plugin
type: ECMWFSearch
need_auth: true
ssl_verify: true
timeout: 30
end_date_excluded: false
remove_from_query:
- dataset
- date
discover_queryables:
fetch_url: null
collection_fetch_url: null
constraints_url: https://cds.climate.copernicus.eu/api/catalogue/v1/collections/{dataset}/constraints.json
form_url: https://cds.climate.copernicus.eu/api/catalogue/v1/collections/{dataset}/form.json
metadata_mapping:
product:type: $.productType
start_datetime: '{$.start_datetime#to_iso_utc_datetime}'
end_datetime:
- date={start_datetime#to_iso_date}/{end_datetime#to_iso_date}
- '{$.end_datetime#to_iso_utc_datetime}'
# The geographic extent of the product
geometry:
- '{{"area": {geometry#to_nwse_bounds}}}'
- $.geometry
qs: $.qs
eodag:order_link: 'https://cds.climate.copernicus.eu/api/retrieve/v1/processes/{dataset}/execution?{{"inputs": {qs#to_geojson}}}'
products:
# See available Public Datasets in https://cds.climate.copernicus.eu/cdsapp#!/search?type=dataset
AG_ERA5:
dataset: sis-agrometeorological-indicators
metadata_mapping:
<<: *day_month_year
ERA5_SL:
dataset: reanalysis-era5-single-levels
metadata_mapping:
<<: *time_day_month_year
ERA5_PL:
dataset: reanalysis-era5-pressure-levels
metadata_mapping:
<<: *time_day_month_year
ERA5_PL_MONTHLY:
dataset: reanalysis-era5-pressure-levels-monthly-means
metadata_mapping:
<<: *time_month_year
ERA5_LAND:
dataset: reanalysis-era5-land
metadata_mapping:
<<: *time_day_month_year
ERA5_LAND_MONTHLY:
dataset: reanalysis-era5-land-monthly-means
metadata_mapping:
<<: *time_month_year
ERA5_SL_MONTHLY:
dataset: reanalysis-era5-single-levels-monthly-means
metadata_mapping:
<<: *time_month_year
UERRA_EUROPE_SL:
dataset: reanalysis-uerra-europe-single-levels
metadata_mapping:
<<: *time_day_month_year
GLACIERS_DIST_RANDOLPH:
dataset: insitu-glaciers-extent
GRIDDED_GLACIERS_MASS_CHANGE:
dataset: derived-gridded-glacier-mass-change
metadata_mapping:
end_datetime:
- |
{{
"hydrological_year": {end_datetime#get_hydrological_year}
}}
- '{$.end_datetime#get_hydrological_year}'
SATELLITE_CARBON_DIOXIDE:
dataset: satellite-carbon-dioxide
metadata_mapping:
<<: *day_month_year
SATELLITE_FIRE_BURNED_AREA:
dataset: satellite-fire-burned-area
metadata_mapping:
<<: *nday_month_year
SATELLITE_METHANE:
dataset: satellite-methane
metadata_mapping:
<<: *day_month_year
SATELLITE_SEA_ICE_EDGE_TYPE:
dataset: satellite-sea-ice-edge-type
metadata_mapping:
<<: *day_month_year
SATELLITE_SEA_ICE_THICKNESS:
dataset: satellite-sea-ice-thickness
metadata_mapping:
<<: *month_year
SATELLITE_SEA_ICE_CONCENTRATION:
dataset: satellite-sea-ice-concentration
metadata_mapping:
<<: *day_month_year
SATELLITE_SEA_LEVEL_GLOBAL:
dataset: satellite-sea-level-global
metadata_mapping:
<<: *day_month_year
SEASONAL_POSTPROCESSED_PL:
dataset: seasonal-postprocessed-pressure-levels
metadata_mapping:
<<: *month_year
SEASONAL_POSTPROCESSED_SL:
dataset: seasonal-postprocessed-single-levels
metadata_mapping:
<<: *month_year
SEASONAL_ORIGINAL_SL:
dataset: seasonal-original-single-levels
metadata_mapping:
<<: *day_month_year
SEASONAL_ORIGINAL_PL:
dataset: seasonal-original-pressure-levels
metadata_mapping:
<<: *day_month_year
SEASONAL_MONTHLY_PL:
dataset: seasonal-monthly-pressure-levels
metadata_mapping:
<<: *month_year
SEASONAL_MONTHLY_SL:
dataset: seasonal-monthly-single-levels
metadata_mapping:
<<: *month_year
SIS_HYDRO_MET_PROJ:
dataset: sis-hydrology-meteorology-derived-projections
data_format: zip
CMIP6_CLIMATE_PROJECTIONS:
dataset: projections-cmip6
metadata_mapping:
<<: *month_year
GENERIC_COLLECTION:
dataset: '{collection}'
---
!provider # MARK: sara
name: sara
priority: 0
description: Sentinel Australasia Regional Access
roles:
- host
url: https://www.copernicus.gov.au/
search: !plugin
type: QueryStringSearch
# The endpoint is based off of the collection. There is a generic endpoint,
# but can be very slow if not enough metdata is provided.
api_endpoint: 'https://copernicus.nci.org.au/sara.server/1.0/api/collections/{_collection}/search.json'
need_auth: false
ssl_verify: true
pagination:
next_page_url_tpl: '{url}?{search}&maxRecords={limit}&page={next_page_token}'
total_items_nb_key_path: '$.properties.totalResults'
next_page_token_key: page
# 2021/03/19: 500 is the max, no error if greater
max_limit: 500
sort:
sort_by_default:
- !!python/tuple [start_datetime, ASC]
sort_by_tpl: '&sortParam={sort_param}&sortOrder={sort_order}'
sort_param_mapping:
start_datetime: startDate
end_datetime: completionDate
sar:instrument_mode: sensorMode
sort_order_mapping:
ascending: asc
descending: desc
max_sort_params: 1
discover_metadata:
auto_discovery: true
metadata_pattern: '^(?!collection)[a-zA-Z0-9_]+$'
search_param: '{metadata}={{{metadata}}}'
metadata_path: '$.properties.*'
metadata_mapping:
# Opensearch resource identifier within the search engine context (in our case
# within the context of the data provider)
uid: '$.id'
# OpenSearch Parameters for Collection Search (Table 3)
product:type:
- productType
- '$.properties.productType'
constellation: '$.properties.collection'
platform:
- platform
- '$.properties.platform'
instruments:
- 'instrument={instruments#csv_list}'
- '{$.properties.instrument#split( )}'
processing:level:
- processingLevel
- '$.properties.processingLevel'
# INSPIRE obligated OpenSearch Parameters for Collection Search (Table 4)
title: '$.properties.title'
keywords: '$.properties.keywords[*].name'
description: '$.properties.description'
gsd:
- 'resolution'
- '{$.properties.resolution#replace_tuple(((0,"Not Available"),))}'
_provider: '$.properties.organisationName'
providers:
- 'organisationName'
- '[{{"name":"{_provider}","roles":["producer"]}}]'
published: '$.properties.published'
license: '$.properties.license.licenseId'
# OpenSearch Parameters for Product Search (Table 5)
product:acquisition_type: '$.properties.acquisitionType'
sat:relative_orbit:
- 'orbitNumber'
- '$.properties.orbitNumber'
sat:absolute_orbit:
- 'absoluteOrbitNumber'
- '$.properties.absoluteOrbitNumber'
sat:orbit_state:
- 'orbitDirection={sat:orbit_state#to_title}'
- '{$.properties.orbitDirection#to_lower}'
sar:instrument_mode:
- 'swath'
- '$.properties.swath'
eo:cloud_cover:
- 'cloudCover=[0,{eo:cloud_cover}]'
- '$.properties.cloudCover'
eo:snow_cover:
- 'snowCover=[0,{eo:snow_cover}]'
- '$.properties.snowCover'
version: '$.properties.version'
created: '$.properties.dhusIngestDate'
updated: '$.properties.updated'
sar:instrument_mode:
- 'sensorMode'
- '$.properties.sensorMode'
# OpenSearch Parameters for Acquistion Parameters Search (Table 6)
start_datetime:
- startDate
- '$.properties.startDate'
end_datetime:
- completionDate
- '$.properties.completionDate'
sar:polarizations:
- 'polarisation={sar:polarizations#csv_list(,)}'
- '{$.properties.polarisation#split(,)}'
# Custom parameters (not defined in the base document referenced above)
id:
- 'productIdentifier={id#remove_extension}'
- '$.properties.productIdentifier'
# The geographic extent of the product
geometry:
- 'geometry={geometry#to_rounded_wkt}'
- '$.geometry'
# The url of the quicklook
eodag:quicklook: '$.properties.quicklook'
# The url to download the product "as is" (literal or as a template to be completed either after the search result
# is obtained from the provider or during the eodag download phase)
eodag:download_link: '$.properties.services.download.url'
type: '{$.properties.services.download.mimeType#replace_str("application/unknown","application/octet-stream")}'
file:size: '$.properties.services.download.size'
file:checksum: '{$.properties.services.download.checksum#replace_str("md5=","")}'
# order:status set to succeeded for consistency between providers
order:status: '{$.null#replace_str("Not Available","succeeded")}'
# Additional metadata provided by the providers but that don't appear in the reference spec
eodag:thumbnail: '$.properties.thumbnail'
products:
# Sentinel 1
S1_SAR_OCN:
product:type: OCN
_collection: S1
instruments: C-SAR
S1_SAR_GRD:
product:type: GRD
_collection: S1
instruments: C-SAR
S1_SAR_SLC:
product:type: SLC
_collection: S1
instruments: C-SAR
# Sentinel 2
S2_MSI_L1C:
_collection: S2
product:type: S2MSIL1C
instruments: MSI
processing:level: L1C
S2_MSI_L2A:
_collection: S2
product:type: S2MSIL2A
instruments: MSI
processing:level: L2A
# OLCI products
# L1
S3_EFR:
product:type: OL_1_EFR___
_collection: S3
instruments: OLCI
processing:level: LEVEL-1
S3_ERR:
product:type: OL_1_ERR___
_collection: S3
instruments: OLCI
processing:level: LEVEL-1
# output during radiometric calibration mode
S3_RAC:
product:type: OL_1_RAC___
_collection: S3
instruments: OLCI
processing:level: LEVEL-1
# L2
S3_OLCI_L2LRR:
product:type: OL_2_LRR___
_collection: S3
instruments: OLCI
processing:level: LEVEL-2
S3_OLCI_L2LFR:
product:type: OL_2_LFR___
_collection: S3
instruments: OLCI
processing:level: LEVEL-2
S3_OLCI_L2WRR:
product:type: OL_2_WRR___
_collection: S3
instruments: OLCI
processing:level: LEVEL-2
S3_OLCI_L2WFR:
product:type: OL_2_WFR___
_collection: S3
instruments: OLCI
processing:level: LEVEL-2
# SLSTR products
S3_SLSTR_L1RBT:
product:type: SL_1_RBT___
_collection: S3
instruments: SLSTR
processing:level: LEVEL-1
S3_SLSTR_L2LST:
product:type: SL_2_LST___
_collection: S3
instruments: SLSTR
processing:level: LEVEL-2
S3_SLSTR_L2WST:
product:type: SL_2_WST___
_collection: S3
instruments: SLSTR
processing:level: LEVEL-2
S3_SLSTR_L2AOD:
product:type: SL_2_AOD___
_collection: S3
instruments: SLSTR
processing:level: LEVEL-2
S3_SLSTR_L2FRP:
product:type: SL_2_FRP___
_collection: S3
instruments: SLSTR
processing:level: LEVEL-2
# SRAL
# L1
S3_SRA:
product:type: SR_1_SRA___
_collection: S3
instruments: SRAL
processing:level: LEVEL-1
S3_SRA_A:
product:type: SR_1_SRA_A_
_collection: S3
instruments: SRAL
processing:level: LEVEL-1
S3_SRA_BS:
product:type: SR_1_SRA_BS
_collection: S3
instruments: SRAL
processing:level: LEVEL-1
# L2
S3_LAN:
product:type: SR_2_LAN___
_collection: S3
instruments: SRAL
processing:level: LEVEL-2
S3_WAT:
product:type: SR_2_WAT___
_collection: S3
instruments: SRAL
processing:level: LEVEL-2
# Synergy products
S3_SY_AOD:
product:type: SY_2_AOD___
_collection: S3
instruments: SYNERGY
processing:level: LEVEL-2
S3_SY_SYN:
product:type: SY_2_SYN___
_collection: S3
instruments: SYNERGY
processing:level: LEVEL-2
S3_SY_V10:
product:type: SY_2_V10___
_collection: S3
instruments: SYNERGY
processing:level: LEVEL-2W
S3_SY_VG1:
product:type: SY_2_VG1___
_collection: S3
instruments: SYNERGY
processing:level: LEVEL-2
S3_SY_VGP:
product:type: SY_2_VGP___
_collection: S3
instruments: SYNERGY
processing:level: LEVEL-2
GENERIC_COLLECTION:
_collection: '{collection}'
download: !plugin
type: HTTPDownload
extract: true
archive_depth: 2
order_enabled: true
auth_error_code: 403
ssl_verify: true
auth: !plugin
type: GenericAuth
matching_url: https://copernicus.nci.org.au
method: basic
---
!provider # MARK: meteoblue
name: meteoblue
priority: 0
roles:
- host
description: Meteoblue
url: https://www.meteoblue.com
search: !plugin
type: MeteoblueSearch
api_endpoint: 'https://my.meteoblue.com/dataset/query'
need_auth: true
ssl_verify: true
pagination:
next_page_query_obj: '{{"checkOnly":true}}'
discover_metadata:
auto_discovery: true
metadata_pattern: '^[a-zA-Z0-9_]+$'
search_param: '{{{{"{metadata}":{{{metadata}#to_geojson}} }}}}'
metadata_path: '$.*'
metadata_mapping:
start_datetime: '{$.start_datetime#to_iso_utc_datetime}'
end_datetime:
- '{{"timeIntervals": [ "{start_datetime#to_iso_date}/{end_datetime#to_iso_date(-1,)}" ] }}'
- '{$.end_datetime#to_iso_utc_datetime}'
geometry:
- '{{"geometry": {geometry#to_geojson} }}'
- '$.geometry'
eodag:default_geometry: 'POLYGON((180 -90, 180 90, -180 90, -180 -90, 180 -90))'
collection: '$.queries[0].domain'
order:status: '{$.requiresJobQueue#get_group_name((?P<succeeded>False)|(?P<orderable>True))}'
qs: $.qs
eodag:download_link: https://my.meteoblue.com/dataset/query?{qs#to_geojson}
# Meteoblue specific parameters
datapoints: '$.datapoints'
requiresJobQueue: '$.requiresJobQueue'
requiresComplexJobQueue: '$.requiresComplexJobQueue'
units:
- '{{"units": {units#to_geojson} }}'
- '$.units'
queries:
- '{{"queries": {queries#to_geojson} }}'
- '$.queries'
format:
- '{{"format": {format#to_geojson} }}'
- '$.format'
timeIntervalsAlignment:
- '{{"timeIntervalsAlignment": {timeIntervalsAlignment#to_geojson} }}'
- '$.timeIntervalsAlignment'
eodag:order_link: https://my.meteoblue.com/dataset/query?{qs#replace_str(r"^(.*)(\")(queries\")(.)",r"\1\2runOnJobQueue\2\4 true, \2\3\4")}
products:
NEMSGLOBAL_TCDC:
product:type: NEMSGLOBAL
queries: [{'domain':'NEMSGLOBAL','gapFillDomain':null,'timeResolution':'daily','codes':[{'code':71,'level':'sfc','aggregation':'mean'}]}]
format: netCDF
units: {'temperature':'C','velocity':'km/h','length':'metric','energy':'watts'}
timeIntervalsAlignment:
geometry: {"type": "Polygon", "coordinates": [[[180, -90], [180, 90], [-180, 90], [-180, -90], [180, -90]]]}
NEMSAUTO_TCDC:
product:type: NEMSAUTO
queries: [{'domain':'NEMSAUTO','gapFillDomain':null,'timeResolution':'daily','codes':[{'code':71,'level':'sfc','aggregation':'mean'}]}]
format: netCDF
units: {'temperature':'C','velocity':'km/h','length':'metric','energy':'watts'}
timeIntervalsAlignment:
geometry: {"type": "Polygon", "coordinates": [[[180, -90], [180, 90], [-180, 90], [-180, -90], [180, -90]]]}
GENERIC_COLLECTION:
_collection: '{collection}'
geometry: {"type": "Polygon", "coordinates": [[[180, -90], [180, 90], [-180, 90], [-180, -90], [180, -90]]]}
download: !plugin
type: HTTPDownload
ssl_verify: true
method: POST
extract: False
order_enabled: true
order_method: POST
order_on_response:
metadata_mapping:
eodag:order_id: '$.json.id'
eodag:status_link: 'http://my.meteoblue.com/queue/status/{eodag:order_id}'
order_status:
request:
method: GET
metadata_mapping:
eodag:order_percent: $.json.percentCompleted
eodag:order_status: $.json.status
error:
eodag:order_status: error
on_success:
metadata_mapping:
eodag:order_id: '$.json.id'
eodag:download_link: 'http://queueresults.meteoblue.com/{eodag:order_id}'
eodag:download_method: '{$.null#replace_str("Not Available","GET")}'
auth: !plugin
type: HttpQueryStringAuth
matching_url: https?://[a-z]+.meteoblue.com
auth_uri: 'http://my.meteoblue.com/dataset/meta?dataset=NEMSAUTO'
ssl_verify: true
---
!provider # MARK: cop_dataspace
name: cop_dataspace
priority: 0
description: Copernicus Data Space Ecosystem
roles:
- host
url: https://dataspace.copernicus.eu/
search: !plugin
type: ODataV4Search
api_endpoint: 'https://catalogue.dataspace.copernicus.eu/odata/v1/Products'
need_auth: false
timeout: 120
ssl_verify: true
dont_quote:
- '['
- ']'
- '$'
- '='
- '&'
- ':'
- '%'
pagination:
next_page_url_tpl: '{url}?{search}&$top={limit}&$skip={skip}&$expand=Attributes&$expand=Assets'
next_page_url_key_path: '$.["@odata.nextLink"]'
next_page_token_key: skip
parse_url_key: $skip
count_tpl: '&$count=True'
total_items_nb_key_path: '$."@odata.count"'
max_limit: 1_000
sort:
sort_by_default:
- !!python/tuple [start_datetime, ASC]
sort_by_tpl: '&$orderby={sort_param} {sort_order}'
sort_param_mapping:
start_datetime: ContentDate/Start
end_datetime: ContentDate/End
published: PublicationDate
updated: ModificationDate
sort_order_mapping:
ascending: asc
descending: desc
max_sort_params: 1
results_entry: 'value'
free_text_search_operations:
$filter:
union: ' or '
wrapper: '{}'
operations:
and:
- "Collection/Name eq '{_collection}'"
- "OData.CSC.Intersects(area=geography'{geometry#to_ewkt}')"
- "Attributes/OData.CSC.StringAttribute/any(att:att/Name eq 'productType' and att/OData.CSC.StringAttribute/Value eq '{product:type}')"
- "Attributes/OData.CSC.StringAttribute/any(att:att/Name eq 'platformShortName' and att/OData.CSC.StringAttribute/Value eq '{constellation}')"
- "Attributes/OData.CSC.StringAttribute/any(att:att/Name eq 'platformSerialIdentifier' and att/OData.CSC.StringAttribute/Value eq '{platform#replace_str(\"^S[1-3]\", \"\")}')"
- "Attributes/OData.CSC.IntegerAttribute/any(att:att/Name eq 'spatialResolution' and att/OData.CSC.StringAttribute/Value eq '{gsd}')"
- "Attributes/OData.CSC.StringAttribute/any(att:att/Name eq 'authority' and att/OData.CSC.StringAttribute/Value eq '{providers}')"
- "Attributes/OData.CSC.IntegerAttribute/any(att:att/Name eq 'orbitNumber' and att/OData.CSC.StringAttribute/Value eq {sat:absolute_orbit})"
- "Attributes/OData.CSC.IntegerAttribute/any(att:att/Name eq 'relativeOrbitNumber' and att/OData.CSC.StringAttribute/Value eq {sat:relative_orbit})"
- "Attributes/OData.CSC.StringAttribute/any(att:att/Name eq 'orbitDirection' and att/OData.CSC.StringAttribute/Value eq '{sat:orbit_state#to_upper}')"
- "Attributes/OData.CSC.DoubleAttribute/any(att:att/Name eq 'cloudCover' and att/OData.CSC.DoubleAttribute/Value le {eo:cloud_cover})"
- "Attributes/OData.CSC.StringAttribute/any(att:att/Name eq 'instrumentShortName' and att/OData.CSC.StringAttribute/Value eq '{instruments#csv_list}')"
- "Attributes/OData.CSC.StringAttribute/any(att:att/Name eq 'tileId' and att/OData.CSC.StringAttribute/Value eq '{grid:code#replace_str(\"MGRS-\",\"\")}')"
- "Attributes/OData.CSC.StringAttribute/any(att:att/Name eq 'processingLevel' and att/OData.CSC.StringAttribute/Value eq '{processing:level}')"
- "Attributes/OData.CSC.StringAttribute/any(att:att/Name eq 'processingDate' and att/OData.CSC.StringAttribute/Value eq '{processing:datetime}')"
- "Attributes/OData.CSC.StringAttribute/any(att:att/Name eq 'processingCenter' and att/OData.CSC.StringAttribute/Value eq '{processing:facility}')"
- "Attributes/OData.CSC.StringAttribute/any(att:att/Name eq 'processorVersion' and att/OData.CSC.StringAttribute/Value eq '{processing:version}')"
- "Attributes/OData.CSC.StringAttribute/any(att:att/Name eq 'swathIdentifier' and att/OData.CSC.StringAttribute/Value eq '{sar:instrument_mode}')"
- "Attributes/OData.CSC.StringAttribute/any(att:att/Name eq 'datatakeID' and att/OData.CSC.StringAttribute/Value eq '{s1:datatake_id}')"
- "Attributes/OData.CSC.StringAttribute/any(att:att/Name eq 'instrumentConfigurationID' and att/OData.CSC.StringAttribute/Value eq '{s1:instrument_configuration_ID}')"
- "Attributes/OData.CSC.StringAttribute/any(att:att/Name eq 'sliceNumber' and att/OData.CSC.StringAttribute/Value eq '{s1:slice_number}')"
- "Attributes/OData.CSC.StringAttribute/any(att:att/Name eq 'totalSlices' and att/OData.CSC.StringAttribute/Value eq '{s1:total_slices}')"
- "Attributes/OData.CSC.StringAttribute/any(att:att/Name eq 'granuleIdentifier' and att/OData.CSC.StringAttribute/Value eq '{s2:tile_id}')"
- "Attributes/OData.CSC.StringAttribute/any(att:att/Name eq 'productGroupId' and att/OData.CSC.StringAttribute/Value eq '{s2:datatake_id}')"
- "Attributes/OData.CSC.StringAttribute/any(att:att/Name eq 'Name' and att/OData.CSC.StringAttribute/Value eq '{s2:product_uri}')"
- "Attributes/OData.CSC.StringAttribute/any(att:att/Name eq 'datastripId' and att/OData.CSC.StringAttribute/Value eq '{s2:datastrip_id}')"
- "Attributes/OData.CSC.StringAttribute/any(att:att/Name eq 'operationalMode' and att/OData.CSC.StringAttribute/Value eq '{s2:datatake_type}')"
- "ModificationDate gt {updated_after#to_iso_utc_datetime}"
- "ModificationDate lt {updated_before#to_iso_utc_datetime}"
- "PublicationDate gt {published_after#to_iso_utc_datetime}"
- "PublicationDate lt {published_before#to_iso_utc_datetime}"
- "ContentDate/Start lt {end_datetime#to_iso_utc_datetime}"
- "ContentDate/End gt {start_datetime#to_iso_utc_datetime}"
- "Attributes/OData.CSC.StringAttribute/any(att:att/Name eq 'polarisationChannels' and att/OData.CSC.StringAttribute/Value eq '{sar:polarizations#csv_list(%26)}')"
- contains(Name,'{id}')
discover_metadata:
auto_discovery: true
metadata_pattern: '^(?!collection)[a-zA-Z0-9]+$'
search_param:
free_text_search_operations:
$filter:
operations:
and:
- "Attributes/OData.CSC.StringAttribute/any(att:att/Name eq '{metadata}' and att/OData.CSC.StringAttribute/Value eq '{{{metadata}}}')"
metadata_path: '$.Attributes.*'
per_product_metadata_query: false
metadata_pre_mapping:
metadata_path: '$.Attributes'
metadata_path_id: 'Name'
metadata_path_value: 'Value'
metadata_mapping:
_collection:
- null
- '$.null'
# hide duplicated metadata
beginningDateTime: '$.null'
endingDateTime: '$.null'
platformSerialIdentifier: '$.null'
# Opensearch resource identifier within the search engine context (in our case
# within the context of the data provider)
# Queryable parameters are set with null as 1st configuration list value to mark them as queryable,
# but `free_text_search_operations.$filter.operations.and` entries are then used instead.
uid: '$.Id'
# OpenSearch Parameters for Collection Search (Table 3)
product:type:
- null
- '$.Attributes.productType'
constellation:
- null
- '$.Attributes.platformShortName'
platform:
- null
- '$.Attributes.platformSerialIdentifier'
instruments:
- null
- '{$.Attributes.instrumentShortName#split( )}'
processing:level:
- null
- '$.Attributes.processingLevel'
processing:datetime:
- null
- '$.Attributes.processingDate'
processing:facility:
- null
- '$.Attributes.processingCenter'
processing:version:
- null
- '{$.Attributes.processorVersion#to_geojson}'
_processor_name:
- null
- '$.Attributes.processorName'
processing:software: '{{"{_processor_name}":"{processing:version}"}}'
# INSPIRE obligated OpenSearch Parameters for Collection Search (Table 4)
title: '{$.Name#remove_extension}'
gsd:
- null
- '$.Attributes.spatialResolution'
_provider: '$.Attributes.origin'
providers:
- null
- '[{{"name":"{_provider}","roles":["producer"]}}]'
published_after:
- null
- '$.null'
published_before:
- null
- '$.null'
published: '$.PublicationDate'
# OpenSearch Parameters for Product Search (Table 5)
sat:absolute_orbit:
- null
- '$.Attributes.orbitNumber'
sat:relative_orbit:
- null
- '$.Attributes.relativeOrbitNumber'
sat:orbit_state:
- null
- '{$.Attributes.orbitDirection#to_lower}'
eo:cloud_cover:
- null
- '$.Attributes.cloudCover'
updated_after:
- null
- '$.null'
updated_before:
- null
- '$.null'
updated:
- null
- '$.ModificationDate'
sat:relative_orbit:
- null
- '$.Attributes.relativeOrbitNumber'
# OpenSearch Parameters for Acquistion Parameters Search (Table 6)
start_datetime:
- null
- '$.ContentDate.Start'
end_datetime:
- null
- '$.ContentDate.End'
product:timeliness:
- null
- '$.Attributes.timeliness'
sar:instrument_mode:
- null
- '$.Attributes.swathIdentifier'
sar:polarizations:
- null
- '{$.Attributes.polarisationChannels#split(&)}'
s1:datatake_id:
- null
- '$.Attributes.datatakeID'
s1:instrument_configuration_ID:
- null
- '$.Attributes.instrumentConfigurationID'
s1:slice_number:
- null
- '$.Attributes.sliceNumber'
s1:total_slices:
- null
- '$.Attributes.totalSlices'
s2:tile_id:
- null
- '$.Attributes.granuleIdentifier'
s2:datatake_id:
- null
- '$.Attributes.productGroupId'
s2:product_uri:
- null
- '$.Attributes.Name'
s2:datastrip_id:
- null
- '$.Attributes.datastripId'
s2:datatake_type:
- null
- '$.Attributes.operationalMode'
# Custom parameters (not defined in the base document referenced above)
id:
- null
- '{$.Name#remove_extension}'
grid:code:
- null
- '{$.Attributes.tileId#replace_str(r"^","MGRS-")}'
file:size: '$.ContentLength'
file:checksum: '$.Checksum[?Algorithm="MD5"].Value'
type: '$.ContentType'
# The geographic extent of the product
geometry:
- null
- '{$.Footprint#from_ewkt}'
# The url to download the product "as is" (literal or as a template to be completed either after the search result
# is obtained from the provider or during the eodag download phase)
eodag:download_link: 'https://catalogue.dataspace.copernicus.eu/odata/v1/Products({uid})/$value'
# order:status: must be one of succeeded, ordered, orderable
order:status: '{$.Online#get_group_name((?P<succeeded>True)|(?P<orderable>False))}'
collection:
- null
- $.null
eodag:quicklook: '$.Assets[?(@.Type="QUICKLOOK")].DownloadLink'
eodag:thumbnail: '$.Assets[?(@.Type="QUICKLOOK")].DownloadLink'
download: !plugin
type: HTTPDownload
extract: true
order_enabled: false
archive_depth: 2
max_workers: 4
ssl_verify: true
auth: !plugin
type: KeycloakOIDCPasswordAuth
matching_url: https://catalogue.dataspace.copernicus.eu
oidc_config_url: https://identity.dataspace.copernicus.eu/auth/realms/CDSE/.well-known/openid-configuration
client_id: 'cdse-public'
client_secret: null
token_provision: qs
token_qs_key: 'token'
auth_error_code: 401
ssl_verify: true
allowed_audiences: ["CLOUDFERRO_PUBLIC"]
products:
# S2
S2_MSI_L1C:
_collection: SENTINEL-2
product:type: S2MSI1C
metadata_mapping:
platform:
- null
- '$.Attributes.platformSerialIdentifier.`sub(/^/, S2)`'
S2_MSI_L2A:
_collection: SENTINEL-2
product:type: S2MSI2A
metadata_mapping:
platform:
- null
- '$.Attributes.platformSerialIdentifier.`sub(/^/, S2)`'
# S1
S1_AUX_GNSSRD:
product:type: AUX_GNSSRD
_collection: SENTINEL-1
metadata_mapping:
eo:cloud_cover: '{$.Attributes.cloudCover#not_available}'
platform:
- null
- '$.Attributes.platformSerialIdentifier.`sub(/^/, S1)`'
S1_AUX_MOEORB:
product:type: AUX_MOEORB
_collection: SENTINEL-1
metadata_mapping:
eo:cloud_cover: '{$.Attributes.cloudCover#not_available}'
platform:
- null
- '$.Attributes.platformSerialIdentifier.`sub(/^/, S1)`'
S1_AUX_POEORB:
product:type: AUX_POEORB
_collection: SENTINEL-1
metadata_mapping:
eo:cloud_cover: '{$.Attributes.cloudCover#not_available}'
platform:
- null
- '$.Attributes.platformSerialIdentifier.`sub(/^/, S1)`'
S1_AUX_PREORB:
product:type: AUX_PREORB
_collection: SENTINEL-1
metadata_mapping:
eo:cloud_cover: '{$.Attributes.cloudCover#not_available}'
platform:
- null
- '$.Attributes.platformSerialIdentifier.`sub(/^/, S1)`'
S1_AUX_PROQUA:
product:type: AUX_PROQUA
_collection: SENTINEL-1
metadata_mapping:
eo:cloud_cover: '{$.Attributes.cloudCover#not_available}'
platform:
- null
- '$.Attributes.platformSerialIdentifier.`sub(/^/, S1)`'
S1_AUX_RESORB:
product:type: AUX_RESORB
_collection: SENTINEL-1
metadata_mapping:
eo:cloud_cover: '{$.Attributes.cloudCover#not_available}'
platform:
- null
- '$.Attributes.platformSerialIdentifier.`sub(/^/, S1)`'
S1_SAR_RAW:
product:type: RAW
_collection: SENTINEL-1
metadata_mapping:
eo:cloud_cover: '{$.Attributes.cloudCover#not_available}'
platform:
- null
- '$.Attributes.platformSerialIdentifier.`sub(/^/, S1)`'
S1_SAR_GRD:
product:type: GRD
_collection: SENTINEL-1
metadata_mapping:
eo:cloud_cover: '{$.Attributes.cloudCover#not_available}'
platform:
- null
- '$.Attributes.platformSerialIdentifier.`sub(/^/, S1)`'
S1_SAR_GRD_COG:
product:type: GRD-COG
_collection: SENTINEL-1
metadata_mapping:
eo:cloud_cover: '{$.Attributes.cloudCover#not_available}'
platform:
- null
- '$.Attributes.platformSerialIdentifier.`sub(/^/, S1)`'
S1_SAR_SLC:
product:type: SLC
_collection: SENTINEL-1
metadata_mapping:
eo:cloud_cover: '{$.Attributes.cloudCover#not_available}'
platform:
- null
- '$.Attributes.platformSerialIdentifier.`sub(/^/, S1)`'
S1_SAR_OCN:
product:type: OCN
_collection: SENTINEL-1
metadata_mapping:
eo:cloud_cover: '{$.Attributes.cloudCover#not_available}'
platform:
- null
- '$.Attributes.platformSerialIdentifier.`sub(/^/, S1)`'
S1_SAR_L3_IW_MCM:
product:type: S1SAR_L3_IW_MCM
_collection: GLOBAL-MOSAICS
metadata_mapping:
eo:cloud_cover: '{$.Attributes.cloudCover#not_available}'
platform:
- null
- '$.Attributes.platformSerialIdentifier.`sub(/^/, S1)`'
S1_SAR_L3_DH_MCM:
product:type: S1SAR_L3_DH_MCM
_collection: GLOBAL-MOSAICS
metadata_mapping:
eo:cloud_cover: '{$.Attributes.cloudCover#not_available}'
platform:
- null
- '$.Attributes.platformSerialIdentifier.`sub(/^/, S1)`'
# S3 SRAL
S3_SRA:
product:type: SR_1_SRA___
_collection: SENTINEL-3
metadata_mapping:
eo:cloud_cover: '{$.Attributes.cloudCover#not_available}'
platform:
- null
- '$.Attributes.platformSerialIdentifier.`sub(/^/, S3)`'
S3_SRA_A:
product:type: SR_1_SRA_A_
_collection: SENTINEL-3
metadata_mapping:
eo:cloud_cover: '{$.Attributes.cloudCover#not_available}'
platform:
- null
- '$.Attributes.platformSerialIdentifier.`sub(/^/, S3)`'
S3_SRA_BS:
product:type: SR_1_SRA_BS
_collection: SENTINEL-3
metadata_mapping:
eo:cloud_cover: '{$.Attributes.cloudCover#not_available}'
platform:
- null
- '$.Attributes.platformSerialIdentifier.`sub(/^/, S3)`'
S3_LAN:
product:type: SR_2_LAN___
_collection: SENTINEL-3
metadata_mapping:
eo:cloud_cover: '{$.Attributes.cloudCover#not_available}'
platform:
- null
- '$.Attributes.platformSerialIdentifier.`sub(/^/, S3)`'
S3_LAN_HY:
product:type: SR_2_LAN_HY
_collection: SENTINEL-3
metadata_mapping:
eo:cloud_cover: '{$.Attributes.cloudCover#not_available}'
platform:
- null
- '$.Attributes.platformSerialIdentifier.`sub(/^/, S3)`'
S3_LAN_SI:
product:type: SR_2_LAN_SI
_collection: SENTINEL-3
metadata_mapping:
eo:cloud_cover: '{$.Attributes.cloudCover#not_available}'
platform:
- null
- '$.Attributes.platformSerialIdentifier.`sub(/^/, S3)`'
S3_LAN_LI:
product:type: SR_2_LAN_LI
_collection: SENTINEL-3
metadata_mapping:
eo:cloud_cover: '{$.Attributes.cloudCover#not_available}'
platform:
- null
- '$.Attributes.platformSerialIdentifier.`sub(/^/, S3)`'
S3_WAT:
product:type: SR_2_WAT___
_collection: SENTINEL-3
metadata_mapping:
eo:cloud_cover: '{$.Attributes.cloudCover#not_available}'
platform:
- null
- '$.Attributes.platformSerialIdentifier.`sub(/^/, S3)`'
# S3 OLCI
S3_EFR:
product:type: OL_1_EFR___
_collection: SENTINEL-3
metadata_mapping:
platform:
- null
- '$.Attributes.platformSerialIdentifier.`sub(/^/, S3)`'
S3_ERR:
product:type: OL_1_ERR___
_collection: SENTINEL-3
metadata_mapping:
platform:
- null
- '$.Attributes.platformSerialIdentifier.`sub(/^/, S3)`'
S3_OLCI_L2LRR:
product:type: OL_2_LRR___
_collection: SENTINEL-3
metadata_mapping:
platform:
- null
- '$.Attributes.platformSerialIdentifier.`sub(/^/, S3)`'
S3_OLCI_L2LFR:
product:type: OL_2_LFR___
_collection: SENTINEL-3
metadata_mapping:
platform:
- null
- '$.Attributes.platformSerialIdentifier.`sub(/^/, S3)`'
S3_OLCI_L2WRR:
product:type: OL_2_WRR___
_collection: SENTINEL-3
metadata_mapping:
platform:
- null
- '$.Attributes.platformSerialIdentifier.`sub(/^/, S3)`'
S3_OLCI_L2WFR:
product:type: OL_2_WFR___
_collection: SENTINEL-3
metadata_mapping:
platform:
- null
- '$.Attributes.platformSerialIdentifier.`sub(/^/, S3)`'
# S3 SLSTR
S3_SLSTR_L1RBT:
product:type: SL_1_RBT___
_collection: SENTINEL-3
metadata_mapping:
platform:
- null
- '$.Attributes.platformSerialIdentifier.`sub(/^/, S3)`'
S3_SLSTR_L2LST:
product:type: SL_2_LST___
_collection: SENTINEL-3
metadata_mapping:
platform:
- null
- '$.Attributes.platformSerialIdentifier.`sub(/^/, S3)`'
S3_SLSTR_L2WST:
product:type: SL_2_WST___
_collection: SENTINEL-3
metadata_mapping:
platform:
- null
- '$.Attributes.platformSerialIdentifier.`sub(/^/, S3)`'
S3_SLSTR_L2AOD:
product:type: SL_2_AOD___
_collection: SENTINEL-3
metadata_mapping:
platform:
- null
- '$.Attributes.platformSerialIdentifier.`sub(/^/, S3)`'
S3_SLSTR_L2FRP:
product:type: SL_2_FRP___
_collection: SENTINEL-3
metadata_mapping:
platform:
- null
- '$.Attributes.platformSerialIdentifier.`sub(/^/, S3)`'
# S3 SY
S3_SY_AOD:
product:type: SY_2_AOD___
_collection: SENTINEL-3
metadata_mapping:
platform:
- null
- '$.Attributes.platformSerialIdentifier.`sub(/^/, S3)`'
S3_SY_SYN:
product:type: SY_2_SYN___
_collection: SENTINEL-3
metadata_mapping:
platform:
- null
- '$.Attributes.platformSerialIdentifier.`sub(/^/, S3)`'
S3_SY_V10:
product:type: SY_2_V10___
_collection: SENTINEL-3
metadata_mapping:
platform:
- null
- '$.Attributes.platformSerialIdentifier.`sub(/^/, S3)`'
S3_SY_VG1:
product:type: SY_2_VG1___
_collection: SENTINEL-3
metadata_mapping:
platform:
- null
- '$.Attributes.platformSerialIdentifier.`sub(/^/, S3)`'
S3_SY_VGP:
product:type: SY_2_VGP___
_collection: SENTINEL-3
metadata_mapping:
platform:
- null
- '$.Attributes.platformSerialIdentifier.`sub(/^/, S3)`'
# S5P L1
S5P_L1B_IR_SIR:
product:type: L1B_IR_SIR
_collection: Sentinel-5P
S5P_L1B_IR_UVN:
product:type: L1B_IR_UVN
_collection: Sentinel-5P
S5P_L1B_RA_BD1:
product:type: L1B_RA_BD1
_collection: SENTINEL-5P
S5P_L1B_RA_BD2:
product:type: L1B_RA_BD2
_collection: SENTINEL-5P
S5P_L1B_RA_BD3:
product:type: L1B_RA_BD3
_collection: SENTINEL-5P
S5P_L1B_RA_BD4:
product:type: L1B_RA_BD4
_collection: SENTINEL-5P
S5P_L1B_RA_BD5:
product:type: L1B_RA_BD5
_collection: SENTINEL-5P
S5P_L1B_RA_BD6:
product:type: L1B_RA_BD6
_collection: SENTINEL-5P
S5P_L1B_RA_BD7:
product:type: L1B_RA_BD7
_collection: SENTINEL-5P
S5P_L1B_RA_BD8:
product:type: L1B_RA_BD8
_collection: SENTINEL-5P
# S5P L2
S5P_L2_NO2:
product:type: L2__NO2___
_collection: SENTINEL-5P
S5P_L2_CLOUD:
product:type: L2__CLOUD_
_collection: SENTINEL-5P
S5P_L2_O3:
product:type: L2__O3____
_collection: SENTINEL-5P
S5P_L2_CO:
product:type: L2__CO____
_collection: SENTINEL-5P
S5P_L2_AER_AI:
product:type: L2__AER_AI
_collection: SENTINEL-5P
S5P_L2_O3_PR:
product:type: L2__O3__PR
_collection: SENTINEL-5P
S5P_L2_O3_TCL:
product:type: L2__O3_TCL
_collection: Sentinel-5P
S5P_L2_AER_LH:
product:type: L2__AER_LH
_collection: SENTINEL-5P
S5P_L2_HCHO:
product:type: L2__HCHO__
_collection: SENTINEL-5P
S5P_L2_CH4:
product:type: L2__CH4___
_collection: SENTINEL-5P
S5P_L2_NP_BD3:
product:type: L2__NP_BD3
_collection: SENTINEL-5P
S5P_L2_NP_BD6:
product:type: L2__NP_BD6
_collection: SENTINEL-5P
S5P_L2_NP_BD7:
product:type: L2__NP_BD7
_collection: SENTINEL-5P
S5P_L2_SO2:
product:type: L2__SO2___
_collection: SENTINEL-5P
GENERIC_COLLECTION:
_collection: '{collection}'
---
!provider # MARK: planetary_computer
name: planetary_computer
priority: 0
roles:
- host
description: Microsoft Planetary Computer
url: https://planetarycomputer.microsoft.com
search: !plugin
type: StacSearch
api_endpoint: https://planetarycomputer.microsoft.com/api/stac/v1/search
need_auth: false
ssl_verify: true
pagination:
max_limit: 1000
next_page_token_key: token
sort:
sort_param_mapping:
id: id
start_datetime: properties.datetime
platform: properties.platform
metadata_mapping:
created: '$.properties.created'
description: '$.properties.description'
grid:code:
- '{{"query":{{"s2:mgrs_tile":{{"eq":"{grid:code#replace_str("MGRS-","")}"}}}}}}'
- '{$.properties."s2:mgrs_tile"#replace_str(r"^T?(.*)$",r"MGRS-\1")}'
products:
S1_SAR_GRD:
_collection: sentinel-1-grd
metadata_mapping:
processing:level:
- '{{"query":{{"s1:processing_level":{{"eq":"{processing:level}"}}}}}}'
- '{$.properties."s1:processing_level"#replace_str(r"^(.+)$",r"L\1")}'
S2_MSI_L2A:
_collection: sentinel-2-l2a
LANDSAT_C2L1:
_collection: landsat-c2-l1
LANDSAT_C2L2:
_collection: landsat-c2-l2
MODIS_MCD43A4:
_collection: modis-43A4-061
NAIP:
_collection: naip
GENERIC_COLLECTION:
_collection: '{collection}'
download: !plugin
type: HTTPDownload
auth_error_code: 403
ssl_verify: true
auth: !plugin
type: SASAuth
matching_url: https://[-\w\.]+.blob.core.windows.net
auth_uri: 'https://planetarycomputer.microsoft.com/api/sas/v1/sign?href={url}'
signed_url_key: href
ssl_verify: true
headers:
Ocp-Apim-Subscription-Key: "{apikey}"
---
!provider # MARK: hydroweb_next
name: hydroweb_next
priority: 0
roles:
- host
description: hydroweb.next thematic hub for hydrology data access
url: https://hydroweb.next.theia-land.fr
search: !plugin
type: StacSearch
api_endpoint: https://hydroweb.next.theia-land.fr/api/v1/rs-catalog/stac/search
need_auth: true
auth_error_code: 401
ssl_verify: true
asset_key_from_href: false
discover_queryables:
fetch_url: null
collection_fetch_url: null
pagination:
max_limit: 10_000
next_page_url_key_path: null
next_page_query_obj_key_path: null
next_page_token_key: page
sort:
sort_by_default:
- !!python/tuple [start_datetime, ASC]
sort_param_mapping:
id: id
start_datetime: properties.start_datetime
end_datetime: properties.end_datetime
version: properties.version
processing:level: processing:level
metadata_mapping:
start_datetime:
- '{{"query":{{"end_datetime":{{"gte":"{start_datetime#to_iso_utc_datetime}"}}}}}}'
- '$.properties.start_datetime'
end_datetime:
- '{{"query":{{"start_datetime":{{"lte":"{end_datetime#to_iso_utc_datetime}"}}}}}}'
- '$.properties.end_datetime'
processing:software:
- '{{"query":{{"processing:software":{{"eq":{processing:software}}}}}}}'
- '$.properties.processing:software'
spatial:cycle_id:
- '{{"query":{{"spatial:cycle_id":{{"eq":{spatial:cycle_id}}}}}}}'
- '$.properties.spatial:cycle_id'
spatial:pass_id:
- '{{"query":{{"spatial:pass_id":{{"eq":{spatial:pass_id}}}}}}}'
- '$.properties.spatial:pass_id'
spatial:scene_id:
- '{{"query":{{"spatial:scene_id":{{"eq":{spatial:scene_id}}}}}}}'
- '$.properties.spatial:scene_id'
products:
GENERIC_COLLECTION:
_collection: '{collection}'
download: !plugin
type: HTTPDownload
auth_error_code: 401
ssl_verify: true
auth: !plugin
type: HTTPHeaderAuth
matching_url: https://hydroweb.next.theia-land.fr
headers:
X-API-Key: "{apikey}"
---
!provider # MARK: wekeo_main
name: wekeo_main # wekeo_main
group: wekeo
priority: 0
roles:
- host
description: WEkEO - Sentinel and some various Copernicus data
url: https://www.wekeo.eu/
# anchors to avoid duplications
anchor_s1_sar: &s1_sar_params
processing:level:
- '{{"processingLevel": "{processing:level}"}}'
- '$.id.`sub(/^[^_]([^_]+)_([^_]+)_([^_]+)_([0-4]+).*/, LEVEL\\4)`'
sar:instrument_mode:
- '{{"sensorMode": "{sar:instrument_mode}"}}'
- '$.id.`sub(/^[^_]([^_]+)_([^_]+)_.*/, \\2)`'
swath:
- '{{"swath": "{swath}"}}'
- '$.null'
polarisation:
- '{{"polarisation": "{polarisation}"}}'
- '$.null'
sat:relative_orbit:
- '{{"relativeOrbitNumber": "{sat:relative_orbit}"}}'
- '$.null'
missionTakeId:
- '{{"missionTakeId": "{missionTakeId}"}}'
- '$.null'
anchor_orbit_cycle: &orbit_cycle
sat:relative_orbit:
- '{{"relativeOrbitNumber": "{sat:relative_orbit}"}}'
- '$.null'
sat:absolute_orbit:
- '{{"orbit": "{sat:absolute_orbit}"}}'
- '$.null'
sat:orbit_cycle:
- '{{"cycle": "{sat:orbit_cycle}"}}'
- '$.null'
anchor_id_from_date: &id_from_date
id:
- |
{{
"startdate": "{id#replace_str(r'^.*_([0-9][0-9][0-9][0-9])([0-9][0-9])([0-9][0-9])([0-9][0-9])([0-9][0-9])_.*$',r'\1-\2-\3T\4%3A\5%3A00Z')}",
"enddate": "{id#replace_str(r'^.*_([0-9][0-9][0-9][0-9])([0-9][0-9])([0-9][0-9])([0-9][0-9])([0-9][0-9])_.*$',r'\1-\2-\3T\4%3A\5%3A00Z')}"
}}
- '$.id'
search: !plugin
type: WekeoSearch
api_endpoint: https://gateway.prod.wekeo2.eu/hda-broker/api/v1/dataaccess/search
timeout: 60
need_auth: true
auth_error_code: 401
results_entry: 'features'
pagination:
total_items_nb_key_path: '$.properties.totalResults'
next_page_query_obj: '{{"itemsPerPage":{limit},"startIndex":{next_page_token}}}'
max_limit: 200
next_page_token_key: skip
discover_collections:
fetch_url: null
discover_queryables:
fetch_url: null
collection_fetch_url: 'https://gateway.prod.wekeo2.eu/hda-broker/api/v1/dataaccess/queryable/{provider_collection}'
result_type: json
results_entry: '$.properties[*]'
metadata_mapping:
_collection:
- '{{"dataset_id": "{_collection}"}}'
- '$.null'
geometry:
- '{{"bbox": {geometry#to_bounds}}}'
- '$.geometry'
eodag:default_geometry: 'POLYGON((180 -90, 180 90, -180 90, -180 -90, 180 -90))'
id:
- '{{"productIdentifier": "{id}"}}'
- '{$.id#remove_extension}'
start_datetime:
- '{{"startdate": "{start_datetime}"}}'
- '$.properties.startdate'
end_datetime:
- '{{"enddate": "{end_datetime}"}}'
- '$.properties.enddate'
eodag:download_link: '$.properties.location'
file:size: '$.properties.size'
file:local_path: '$.properties.localpath'
eodag:quicklook: '$.properties.thumbnail'
eodag:thumbnail: '$.properties.thumbnail'
title: '$.id'
order:status: 'orderable'
processing:level:
- '{{"processingLevel": "{processing:level}"}}'
- '$.null'
product:type:
- '{{"productType": "{product:type}"}}'
- '$.null'
product:timeliness:
- '{{"timeliness": "{product:timeliness}"}}'
- '$.null'
sat:orbit_state:
- '{{"orbitDirection": "{sat:orbit_state}"}}'
- '$.null'
variable:
- '{{"variable": "{variable}"}}'
- '$.null'
system:
- '{{"system": "{system}"}}'
- '$.null'
version:
- '{{"version": {version}}}'
- '$.null'
region:
- '{{"region": {region}}}'
- '$.null'
type:
- '{{"type": "{type}"}}'
- '$.null'
source:
- '{{"source": {source}}}'
- '$.null'
model:
- '{{"model": {model}}}'
- '$.null'
level:
- '{{"level": {level}}}'
- '$.null'
step:
- '{{"step": {step}}}'
- '$.null'
satellite:
- '{{"satellite": {satellite}}}'
- '$.null'
products:
S1_SAR_GRD:
_collection: EO:ESA:DAT:SENTINEL-1
product:type: GRD
metadata_mapping:
<<: *s1_sar_params
eodag:order_link: 'https://gateway.prod.wekeo2.eu/hda-broker/api/v1/dataaccess/download?{{"location": "{eodag:download_link}","product_id":"{id}.SAFE", "dataset_id": "EO:ESA:DAT:SENTINEL-1"}}'
S1_SAR_RAW:
_collection: EO:ESA:DAT:SENTINEL-1
product:type: RAW
metadata_mapping_from_product: S1_SAR_GRD
S1_SAR_OCN:
_collection: EO:ESA:DAT:SENTINEL-1
product:type: OCN
metadata_mapping_from_product: S1_SAR_GRD
S1_SAR_SLC:
_collection: EO:ESA:DAT:SENTINEL-1
product:type: SLC
metadata_mapping_from_product: S1_SAR_GRD
S2_MSI_L1C:
_collection: EO:ESA:DAT:SENTINEL-2
product:type: S2MSI1C
processing:level: S2MSI1C
metadata_mapping:
processing:level:
- '{{"processingLevel": "{processing:level}"}}'
- '$.id.`sub(/^[^_]([^_]+)_([^_]+)_.*/, S2\\2)`'
eo:cloud_cover:
- '{{"cloudCover": "{eo:cloud_cover}"}}'
- '$.null'
eodag:order_link: 'https://gateway.prod.wekeo2.eu/hda-broker/api/v1/dataaccess/download?{{"location": "{eodag:download_link}","product_id":"{id}.SAFE", "dataset_id": "EO:ESA:DAT:SENTINEL-2"}}'
S2_MSI_L2A:
_collection: EO:ESA:DAT:SENTINEL-2
product:type: S2MSI2A
processing:level: S2MSI2A
metadata_mapping_from_product: S2_MSI_L1C
S3_LAN_HY:
_collection: EO:ESA:DAT:SENTINEL-3
product:type: SR_2_LAN_HY
processing:level: 2
metadata_mapping:
processing:level:
- '{{"processingLevel": "{processing:level}"}}'
- '2'
eodag:order_link: 'https://gateway.prod.wekeo2.eu/hda-broker/api/v1/dataaccess/download?{{"location": "{eodag:download_link}","product_id":"{id}.SEN3", "dataset_id": "EO:ESA:DAT:SENTINEL-3"}}'
S3_LAN_SI:
_collection: EO:ESA:DAT:SENTINEL-3
product:type: SR_2_LAN_SI
processing:level: 2
metadata_mapping_from_product: S3_LAN_HY
S3_LAN_LI:
_collection: EO:ESA:DAT:SENTINEL-3
product:type: SR_2_LAN_LI
processing:level: 2
metadata_mapping_from_product: S3_LAN_HY
S3_OLCI_L2LFR:
_collection: EO:ESA:DAT:SENTINEL-3
product:type: OL_2_LFR___
processing:level: 2
metadata_mapping_from_product: S3_LAN_HY
S3_OLCI_L2LRR:
_collection: EO:ESA:DAT:SENTINEL-3
product:type: OL_2_LRR___
processing:level: 2
metadata_mapping_from_product: S3_LAN_HY
S3_SLSTR_L2:
_collection: EO:ESA:DAT:SENTINEL-3
product:type: SL_2_LST___
processing:level: 2
metadata_mapping_from_product: S3_LAN_HY
S3_EFR:
_collection: EO:EUM:DAT:SENTINEL-3:OL_1_EFR___
metadata_mapping:
id:
- '{{"timeliness": {id#split_id_into_s3_params}["timeliness"], "sat": {id#split_id_into_s3_params}["sat"], "startdate": {id#split_id_into_s3_params}["startDate"], "enddate": {id#split_id_into_s3_params}["endDate"]}}'
- '{$.id#remove_extension}'
constellation:
- '{{"sat": "{constellation}"}}'
- '$.id.`sub(/^[^_]([^_]+)_.*/, Sentinel-\\1)`'
platform: '$.id.`sub(/^[^_]([^_]+)_.*/, S\\1)`'
<<: *orbit_cycle
eodag:order_link: 'https://gateway.prod.wekeo2.eu/hda-broker/api/v1/dataaccess/download?{{"location": "{eodag:download_link}","product_id":"{id}.SEN3", "dataset_id": "EO:EUM:DAT:SENTINEL-3:OL_1_EFR___"}}'
S3_ERR:
_collection: EO:EUM:DAT:SENTINEL-3:OL_1_ERR___
metadata_mapping_from_product: S3_EFR
metadata_mapping:
eodag:order_link: 'https://gateway.prod.wekeo2.eu/hda-broker/api/v1/dataaccess/download?{{"location": "{eodag:download_link}","product_id":"{id}.SEN3", "dataset_id": "EO:EUM:DAT:SENTINEL-3:OL_1_ERR___"}}'
S3_OLCI_L2WFR:
_collection: EO:EUM:DAT:SENTINEL-3:OL_2_WFR___
metadata_mapping_from_product: S3_EFR
metadata_mapping:
eodag:order_link: 'https://gateway.prod.wekeo2.eu/hda-broker/api/v1/dataaccess/download?{{"location": "{eodag:download_link}","product_id":"{id}.SEN3", "dataset_id": "EO:EUM:DAT:SENTINEL-3:OL_2_WFR___"}}'
S3_OLCI_L2WRR:
_collection: EO:EUM:DAT:SENTINEL-3:OL_2_WRR___
metadata_mapping_from_product: S3_EFR
metadata_mapping:
eodag:order_link: 'https://gateway.prod.wekeo2.eu/hda-broker/api/v1/dataaccess/download?{{"location": "{eodag:download_link}","product_id":"{id}.SEN3", "dataset_id": "EO:EUM:DAT:SENTINEL-3:OL_2_WRR___"}}'
S3_SRA:
_collection: EO:EUM:DAT:SENTINEL-3:SR_1_SRA___
metadata_mapping_from_product: S3_EFR
metadata_mapping:
eodag:order_link: 'https://gateway.prod.wekeo2.eu/hda-broker/api/v1/dataaccess/download?{{"location": "{eodag:download_link}","product_id":"{id}.SEN3", "dataset_id": "EO:EUM:DAT:SENTINEL-3:SR_1_SRA___"}}'
S3_SRA_A:
_collection: EO:EUM:DAT:SENTINEL-3:SR_1_SRA_A_
metadata_mapping_from_product: S3_EFR
metadata_mapping:
eodag:order_link: 'https://gateway.prod.wekeo2.eu/hda-broker/api/v1/dataaccess/download?{{"location": "{eodag:download_link}","product_id":"{id}.SEN3", "dataset_id": "EO:EUM:DAT:SENTINEL-3:SR_1_SRA_A_"}}'
S3_SRA_BS:
_collection: EO:EUM:DAT:SENTINEL-3:SR_1_SRA_BS
metadata_mapping_from_product: S3_EFR
metadata_mapping:
eodag:order_link: 'https://gateway.prod.wekeo2.eu/hda-broker/api/v1/dataaccess/download?{{"location": "{eodag:download_link}","product_id":"{id}.SEN3", "dataset_id": "EO:EUM:DAT:SENTINEL-3:SR_1_SRA_BS"}}'
S3_SLSTR_L1RBT:
_collection: EO:EUM:DAT:SENTINEL-3:SL_1_RBT___
product:type: SL_1_RBT___
metadata_mapping:
id:
- '{{"productType": "SL_1_RBT___", "timeliness": {id#split_id_into_s3_params}["timeliness"], "sat": {id#split_id_into_s3_params}["sat"], "startdate": {id#split_id_into_s3_params}["startDate"], "enddate": {id#split_id_into_s3_params}["endDate"]}}'
- '{$.id#remove_extension}'
constellation:
- '{{"sat": "{constellation}"}}'
- '$.id.`sub(/^[^_]([^_]+)_.*/, Sentinel-\\1)`'
platform: '$.id.`sub(/^[^_]([^_]+)_.*/, S\\1)`'
<<: *orbit_cycle
eodag:order_link: 'https://gateway.prod.wekeo2.eu/hda-broker/api/v1/dataaccess/download?{{"location": "{eodag:download_link}","product_id":"{id}.SEN3", "dataset_id": "EO:EUM:DAT:SENTINEL-3:SL_1_RBT___"}}'
S3_WAT:
_collection: EO:EUM:DAT:SENTINEL-3:SR_2_WAT___
metadata_mapping:
id:
- '{{"type": "SR_2_WAT___", "timeliness": {id#split_id_into_s3_params}["timeliness"], "sat": {id#split_id_into_s3_params}["sat"], "startdate": {id#split_id_into_s3_params}["startDate"], "enddate": {id#split_id_into_s3_params}["endDate"]}}'
- '{$.id#remove_extension}'
eodag:order_link: 'https://gateway.prod.wekeo2.eu/hda-broker/api/v1/dataaccess/download?{{"location": "{eodag:download_link}","product_id":"{id}.SEN3", "dataset_id": "EO:EUM:DAT:SENTINEL-3:SR_2_WAT___"}}'
S5P_L1B_IR_ALL:
_collection: EO:ESA:DAT:SENTINEL-5P
processing:level: L1B
metadata_mapping:
processing:level:
- '{{"processingLevel": "{processing:level}"}}'
- '$.id.`sub(/^[^_]([^_]+)_([^_]+)_([^_]+)_.*/, \\3)`'
processingMode:
- '{{"processingMode": "{processingMode}"}}'
- '$.null'
eodag:order_link: 'https://gateway.prod.wekeo2.eu/hda-broker/api/v1/dataaccess/download?{{"location": "{eodag:download_link}","product_id":"{id}.nc", "dataset_id": "EO:ESA:DAT:SENTINEL-5P"}}'
S5P_L2_IR_ALL:
_collection: EO:ESA:DAT:SENTINEL-5P
processing:level: L2
metadata_mapping_from_product: S5P_L1B_IR_ALL
EEA_HRL_TCF:
_collection: EO:EEA:DAT:HRL:TCF
metadata_mapping:
start_datetime:
- '{{"year": {start_datetime#to_datetime_dict("string")}["year"]}}'
- $.properties.startdate
end_datetime: $.properties.enddate
resolution:
- '{{"resolution": "{resolution}"}}'
- '{$.id#replace_str(r"^.*_R([0-9]+m)_.*$",r"\1")}'
eodag:order_link: 'https://gateway.prod.wekeo2.eu/hda-broker/api/v1/dataaccess/download?{{"location": "{eodag:download_link}","product_id":"{id}", "dataset_id": "EO:EEA:DAT:HRL:TCF"}}'
COP_DEM_GLO30_DGED:
_collection: EO:ESA:DAT:COP-DEM
product:type: DGE_30
metadata_mapping:
eodag:order_link: 'https://gateway.prod.wekeo2.eu/hda-broker/api/v1/dataaccess/download?{{"location": "{eodag:download_link}","product_id":"{id}", "dataset_id": "EO:ESA:DAT:COP-DEM"}}'
COP_DEM_GLO30_DTED:
_collection: EO:ESA:DAT:COP-DEM
product:type: DTE_30
metadata_mapping_from_product: COP_DEM_GLO30_DGED
COP_DEM_GLO90_DGED:
_collection: EO:ESA:DAT:COP-DEM
product:type: DGE_90
metadata_mapping_from_product: COP_DEM_GLO30_DGED
COP_DEM_GLO90_DTED:
_collection: EO:ESA:DAT:COP-DEM
product:type: DTE_90
metadata_mapping_from_product: COP_DEM_GLO30_DGED
CLMS_GLO_NDVI_333M:
_collection: EO:JRC:DAT:CLMS
product:type: vegetation_indices
_product_identifier: ndvi_global_300m_10daily_v1
metadata_mapping:
_product_identifier:
- '{{"productIdentifier": "{_product_identifier}"}}'
- '$.null'
id: '$.id'
eodag:order_link: 'https://gateway.prod.wekeo2.eu/hda-broker/api/v1/dataaccess/download?{{"location": "{eodag:download_link}","product_id":"{id}", "dataset_id": "EO:JRC:DAT:CLMS"}}'
CLMS_GLO_NDVI_1KM_LTS:
_collection: EO:JRC:DAT:CLMS
product:type: vegetation_indices
_product_identifier: ndvi-lts_global_1km_10daily_v2
metadata_mapping_from_product: CLMS_GLO_NDVI_333M
CLMS_CORINE:
_collection: EO:EEA:DAT:CORINE
product:type: Corine Land Cover 2018
discover_metadata:
raise_mtd_discovery_error: true
metadata_mapping:
id:
- '{{"format": "{id#get_group_name((?P<GeoPackage>geoPackage)|(?P<ESRI fgdb>fgdb)|(?P<GeoTiff100mt>raster100m))}"}}'
- '$.id'
start_datetime: '$.properties.startdate'
end_datetime: '$.properties.enddate'
format:
- '{{"format": "{format}"}}'
- '{$.id#get_group_name((?P<GeoPackage>geoPackage)|(?P<ESRI fgdb>fgdb)|(?P<GeoTiff100mt>raster100m))}'
eodag:default_geometry: 'POLYGON((180 -90, 180 90, -180 90, -180 -90, 180 -90))'
eodag:order_link: 'https://gateway.prod.wekeo2.eu/hda-broker/api/v1/dataaccess/download?{{"location": "{eodag:download_link}","product_id":"{id}", "dataset_id": "EO:EEA:DAT:CORINE"}}'
CLMS_GLO_FCOVER_333M:
_collection: EO:JRC:DAT:CLMS
product:type: vegetation_properties
_product_identifier: fcover_global_300m_10daily_v1
metadata_mapping_from_product: CLMS_GLO_NDVI_333M
CLMS_GLO_DMP_333M:
_collection: EO:JRC:DAT:CLMS
product:type: dry-gross_dry_matter_productivity
_product_identifier: dry-gross_dry_matter_productivity/dmp_global_300m_10daily_v1
metadata_mapping_from_product: CLMS_GLO_NDVI_333M
CLMS_GLO_GDMP_333M:
_collection: EO:JRC:DAT:CLMS
product:type: dry-gross_dry_matter_productivity
_product_identifier: gdmp_global_300m_10daily_v1
metadata_mapping_from_product: CLMS_GLO_NDVI_333M
CLMS_GLO_FAPAR_333M:
_collection: EO:JRC:DAT:CLMS
product:type: vegetation_properties
_product_identifier: fapar_global_300m_10daily_v1
metadata_mapping_from_product: CLMS_GLO_NDVI_333M
CLMS_GLO_LAI_333M:
_collection: EO:JRC:DAT:CLMS
product:type: vegetation_properties
_product_identifier: lai_global_300m_10daily_v1
metadata_mapping_from_product: CLMS_GLO_NDVI_333M
CLMS_HRVPP_ST:
_collection: EO:EEA:DAT:CLMS_HRVPP_ST
metadata_mapping:
id:
- '{{"uid": "{id}"}}'
- '$.id'
eodag:order_link: 'https://gateway.prod.wekeo2.eu/hda-broker/api/v1/dataaccess/download?{{"location": "{eodag:download_link}","product_id":"{id}", "dataset_id": "EO:EEA:DAT:CLMS_HRVPP_ST"}}'
CLMS_HRVPP_ST_LAEA:
_collection: EO:EEA:DAT:CLMS_HRVPP_ST-LAEA
metadata_mapping:
id:
- '{{"uid": "{id}"}}'
- '$.id'
eodag:order_link: 'https://gateway.prod.wekeo2.eu/hda-broker/api/v1/dataaccess/download?{{"location": "{eodag:download_link}","product_id":"{id}", "dataset_id": "EO:EEA:DAT:CLMS_HRVPP_ST-LAEA"}}'
CLMS_HRVPP_VPP:
_collection: EO:EEA:DAT:CLMS_HRVPP_VPP
metadata_mapping:
id:
- '{{"uid": "{id}"}}'
- '$.id'
eodag:order_link: 'https://gateway.prod.wekeo2.eu/hda-broker/api/v1/dataaccess/download?{{"location": "{eodag:download_link}","product_id":"{id}", "dataset_id": "EO:EEA:DAT:CLMS_HRVPP_VPP"}}'
CLMS_HRVPP_VPP_LAEA:
_collection: EO:EEA:DAT:CLMS_HRVPP_VPP-LAEA
metadata_mapping:
id:
- '{{"uid": "{id}"}}'
- '$.id'
eodag:order_link: 'https://gateway.prod.wekeo2.eu/hda-broker/api/v1/dataaccess/download?{{"location": "{eodag:download_link}","product_id":"{id}", "dataset_id": "EO:EEA:DAT:CLMS_HRVPP_VPP-LAEA"}}'
auth: !plugin
type: TokenAuth
matching_url: https://[-\w\.]+.wekeo2.eu
auth_uri: 'https://gateway.prod.wekeo2.eu/hda-broker/gettoken'
refresh_uri: 'https://gateway.prod.wekeo2.eu/hda-broker/refreshtoken'
token_type: json
token_key: access_token
refresh_token_key: refresh_token
token_expiration_key: expires_in
download: !plugin
type: HTTPDownload
base_uri: https://gateway.prod.wekeo2.eu/hda-broker/api/v1/dataaccess
flatten_top_dirs: true
auth_error_code: 401
order_enabled: true
order_method: 'POST'
order_on_response:
metadata_mapping:
eodag:order_id: '$.json.download_id'
eodag:status_link: 'https://gateway.prod.wekeo2.eu/hda-broker/api/v1/dataaccess/downloads?download_id={eodag:order_id}'
eodag:download_link: 'https://gateway.prod.wekeo2.eu/hda-broker/api/v1/dataaccess/download/{eodag:order_id}'
order_status:
request:
method: GET
metadata_mapping:
eodag:order_status: $.json.features[0].status
eodag:order_message: $.json.features[0].message
error:
eodag:order_status: Error
success:
eodag:order_status: Done
ordered:
http_code: 202
on_success:
metadata_mapping:
eodag:download_id: $.json.features[0]._id
eodag:download_link: https://gateway.prod.wekeo2.eu/hda-broker/api/v1/dataaccess/download/{eodag:download_id}
---
!provider # MARK: wekeo_ecmwf
name: wekeo_ecmwf
group: wekeo
priority: 0
roles:
- host
description: WEkEO - ECMWF data
url: https://www.wekeo.eu/
# anchors to avoid duplications
anchor_time_day_month_year: &time_day_month_year
start_datetime:
- |
{{
"year": {_date#interval_to_datetime_dict}["year"],
"month": {_date#interval_to_datetime_dict}["month"],
"day": {_date#interval_to_datetime_dict}["day"],
"time": {start_datetime#get_ecmwf_time}
}}
- $.properties.startdate
end_datetime: $.properties.enddate
anchor_day_month_year: &day_month_year
start_datetime:
- |
{{
"year": {_date#interval_to_datetime_dict}["year"],
"month": {_date#interval_to_datetime_dict}["month"],
"day": {_date#interval_to_datetime_dict}["day"]
}}
- $.properties.startdate
end_datetime: $.properties.enddate
anchor_month_year: &month_year
start_datetime:
- |
{{
"year": {_date#interval_to_datetime_dict}["year"],
"month": {_date#interval_to_datetime_dict}["month"]
}}
- $.properties.startdate
end_datetime: $.properties.enddate
search: !plugin
type: WekeoECMWFSearch
api_endpoint: https://gateway.prod.wekeo2.eu/hda-broker/api/v1/dataaccess/search
need_auth: true
timeout: 60
need_auth: true
auth_error_code: 401
results_entry: 'features'
pagination:
total_items_nb_key_path: '$.properties.totalResults'
next_page_query_obj: '{{"itemsPerPage":{limit},"startIndex":{next_page_token}}}'
next_page_token_key: skip
max_limit: 200
discover_collections:
fetch_url: null
dynamic_discover_queryables:
- collection_selector: # cop_ads
- field: dataset
prefix: EO:ECMWF:DAT:CAMS
discover_queryables:
fetch_url: null
collection_fetch_url: null
constraints_url: https://ads.atmosphere.copernicus.eu/api/catalogue/v1/collections/{dataset#wekeo_to_cop_collection(EO:ECMWF:DAT:)}/constraints.json
form_url: https://ads.atmosphere.copernicus.eu/api/catalogue/v1/collections/{dataset#wekeo_to_cop_collection(EO:ECMWF:DAT:)}/form.json
- collection_selector: # cop_cds
- field: dataset
prefix: EO:ECMWF:DAT:SATELLITE
- field: dataset
prefix: EO:ECMWF:DAT:SEASONAL
- field: dataset
prefix: EO:ECMWF:DAT:INSITU
- field: dataset
prefix: EO:ECMWF:DAT:DERIVED
- field: dataset
prefix: EO:ECMWF:DAT:REANALYSIS
- field: dataset
prefix: EO:ECMWF:DAT:SIS
discover_queryables:
fetch_url: null
collection_fetch_url: null
constraints_url: https://cds.climate.copernicus.eu/api/catalogue/v1/collections/{dataset#wekeo_to_cop_collection(EO:ECMWF:DAT:)}/constraints.json
form_url: https://cds.climate.copernicus.eu/api/catalogue/v1/collections/{dataset#wekeo_to_cop_collection(EO:ECMWF:DAT:)}/form.json
- collection_selector: # cop_ewds
- field: dataset
prefix: EO:ECMWF:DAT:CEMS
discover_queryables:
fetch_url: null
collection_fetch_url: null
constraints_url: https://ewds.climate.copernicus.eu/api/catalogue/v1/collections/{dataset#wekeo_to_cop_collection(EO:ECMWF:DAT:)}/constraints.json
form_url: https://ewds.climate.copernicus.eu/api/catalogue/v1/collections/{dataset#wekeo_to_cop_collection(EO:ECMWF:DAT:)}/form.json
metadata_mapping:
geometry:
- '{{"bbox": {geometry#to_bounds}}}'
- $.geometry
start_datetime:
- '{{"startdate": "{start_datetime#to_iso_utc_datetime}"}}'
- $.properties.startdate
end_datetime:
# map 'date' to validate request against copernicus queryables
- '{{"enddate": "{end_datetime#to_iso_utc_datetime}", "date": "{start_datetime#to_iso_date}/{end_datetime#to_iso_date}"}}'
- $.properties.enddate
product:type:
- dataset_id
- $.dataset
eodag:download_link: $.properties.location
file:size: '$.properties.size'
dataset:
- dataset_id
- $.dataset
eodag:order_link: 'https://gateway.prod.wekeo2.eu/hda-broker/api/v1/dataaccess/download?{{"location": "{eodag:download_link}","product_id":"{id}", "dataset_id": "{dataset}"}}'
products:
SATELLITE_CARBON_DIOXIDE:
dataset: EO:ECMWF:DAT:SATELLITE_CARBON_DIOXIDE
metadata_mapping:
<<: *day_month_year
SATELLITE_FIRE_BURNED_AREA:
dataset: EO:ECMWF:DAT:SATELLITE_FIRE_BURNED_AREA
metadata_mapping:
start_datetime:
- |
{{
"year": {start_datetime#to_datetime_dict(list)}["year"],
"month": {start_datetime#to_datetime_dict(list)}["month"],
"nominal_day": {start_datetime#to_datetime_dict(list)}["day"]
}}
- $.properties.startdate
end_datetime: $.properties.enddate
SATELLITE_METHANE:
dataset: EO:ECMWF:DAT:SATELLITE_METHANE
metadata_mapping:
<<: *day_month_year
SATELLITE_SEA_ICE_EDGE_TYPE:
dataset: EO:ECMWF:DAT:SATELLITE_SEA_ICE_EDGE_TYPE
metadata_mapping:
<<: *day_month_year
SATELLITE_SEA_ICE_THICKNESS:
dataset: EO:ECMWF:DAT:SATELLITE_SEA_ICE_THICKNESS
metadata_mapping:
<<: *month_year
SATELLITE_SEA_ICE_CONCENTRATION:
dataset: EO:ECMWF:DAT:SATELLITE_SEA_ICE_CONCENTRATION
metadata_mapping:
<<: *day_month_year
SATELLITE_SEA_LEVEL_GLOBAL:
dataset: EO:ECMWF:DAT:SATELLITE_SEA_LEVEL_GLOBAL
metadata_mapping:
<<: *day_month_year
SEASONAL_ORIGINAL_SL:
dataset: EO:ECMWF:DAT:SEASONAL_ORIGINAL_SINGLE_LEVELS
metadata_mapping:
<<: *day_month_year
SEASONAL_ORIGINAL_PL:
dataset: EO:ECMWF:DAT:SEASONAL_ORIGINAL_PRESSURE_LEVELS
metadata_mapping:
<<: *day_month_year
SEASONAL_POSTPROCESSED_SL:
dataset: EO:ECMWF:DAT:SEASONAL_POSTPROCESSED_SINGLE_LEVELS
metadata_mapping:
<<: *month_year
SEASONAL_POSTPROCESSED_PL:
dataset: EO:ECMWF:DAT:SEASONAL_POSTPROCESSED_PRESSURE_LEVELS
metadata_mapping:
<<: *month_year
SEASONAL_MONTHLY_SL:
dataset: EO:ECMWF:DAT:SEASONAL_MONTHLY_SINGLE_LEVELS
metadata_mapping:
<<: *month_year
SEASONAL_MONTHLY_PL:
dataset: EO:ECMWF:DAT:SEASONAL_MONTHLY_PRESSURE_LEVELS
metadata_mapping:
<<: *month_year
GLACIERS_DIST_RANDOLPH:
dataset: EO:ECMWF:DAT:INSITU_GLACIERS_EXTENT
metadata_mapping:
<<: *day_month_year
FIRE_HISTORICAL:
dataset: EO:ECMWF:DAT:CEMS_FIRE_HISTORICAL_V1
metadata_mapping:
<<: *day_month_year
GRIDDED_GLACIERS_MASS_CHANGE:
dataset: EO:ECMWF:DAT:DERIVED_GRIDDED_GLACIER_MASS_CHANGE
metadata_mapping:
start_datetime:
- |
{{
"hydrological_year": {start_datetime#get_hydrological_year}
}}
- $.properties.startdate
end_datetime: $.properties.enddate
UERRA_EUROPE_SL:
dataset: EO:ECMWF:DAT:REANALYSIS_UERRA_EUROPE_SINGLE_LEVELS
metadata_mapping:
<<: *time_day_month_year
AG_ERA5:
dataset: EO:ECMWF:DAT:SIS_AGROMETEOROLOGICAL_INDICATORS
metadata_mapping:
<<: *day_month_year
ERA5_SL:
dataset: EO:ECMWF:DAT:REANALYSIS_ERA5_SINGLE_LEVELS
metadata_mapping:
<<: *time_day_month_year
ERA5_PL:
dataset: EO:ECMWF:DAT:REANALYSIS_ERA5_PRESSURE_LEVELS
metadata_mapping:
<<: *time_day_month_year
ERA5_SL_MONTHLY:
dataset: EO:ECMWF:DAT:REANALYSIS_ERA5_SINGLE_LEVELS_MONTHLY_MEANS
metadata_mapping:
start_datetime:
- |
{{
"year": {start_datetime#to_datetime_dict(list)}["year"],
"month": {start_datetime#to_datetime_dict(list)}["month"],
"time": {start_datetime#get_ecmwf_time}
}}
- $.properties.startdate
end_datetime: $.properties.enddate
ERA5_PL_MONTHLY:
dataset: EO:ECMWF:DAT:REANALYSIS_ERA5_PRESSURE_LEVELS_MONTHLY_MEANS
metadata_mapping_from_product: ERA5_SL_MONTHLY
ERA5_LAND:
dataset: EO:ECMWF:DAT:REANALYSIS_ERA5_LAND
metadata_mapping:
<<: *time_day_month_year
ERA5_LAND_MONTHLY:
dataset: EO:ECMWF:DAT:REANALYSIS_ERA5_LAND_MONTHLY_MEANS
metadata_mapping_from_product: ERA5_SL_MONTHLY
CAMS_EAC4:
dataset: EO:ECMWF:DAT:CAMS_GLOBAL_REANALYSIS_EAC4
CAMS_GLOBAL_EMISSIONS:
dataset: EO:ECMWF:DAT:CAMS_GLOBAL_EMISSION_INVENTORIES
metadata_mapping:
start_datetime:
- |
{{
"year": {start_datetime#to_datetime_dict(list)}["year"]
}}
- $.properties.startdate
end_datetime: $.properties.enddate
CAMS_EAC4_MONTHLY:
dataset: EO:ECMWF:DAT:CAMS_GLOBAL_REANALYSIS_EAC4_MONTHLY
metadata_mapping:
<<: *month_year
CAMS_GREENHOUSE_INVERSION:
dataset: EO:ECMWF:DAT:CAMS_GLOBAL_GREENHOUSE_GAS_INVERSION
metadata_mapping:
<<: *month_year
CAMS_EU_AIR_QUALITY_RE:
dataset: EO:ECMWF:DAT:CAMS_EUROPE_AIR_QUALITY_REANALYSES
metadata_mapping:
<<: *month_year
CAMS_GRF:
dataset: EO:ECMWF:DAT:CAMS_GLOBAL_RADIATIVE_FORCINGS
metadata_mapping:
<<: *month_year
CAMS_GRF_AUX:
dataset: EO:ECMWF:DAT:CAMS_GLOBAL_RADIATIVE_FORCING_AUXILLIARY_VARIABLES
metadata_mapping:
<<: *month_year
CAMS_GREENHOUSE_EGG4:
dataset: EO:ECMWF:DAT:CAMS_GLOBAL_GHG_REANALYSIS_EGG4
CAMS_GREENHOUSE_EGG4_MONTHLY:
dataset: EO:ECMWF:DAT:CAMS_GLOBAL_GHG_REANALYSIS_EGG4_MONTHLY
metadata_mapping:
<<: *month_year
CAMS_EU_AIR_QUALITY_FORECAST:
dataset: EO:ECMWF:DAT:CAMS_EUROPE_AIR_QUALITY_FORECASTS
CAMS_GAC_FORECAST:
dataset: EO:ECMWF:DAT:CAMS_GLOBAL_ATMOSPHERIC_COMPOSITION_FORECASTS
CAMS_GFE_GFAS:
dataset: EO:ECMWF:DAT:CAMS_GLOBAL_FIRE_EMISSIONS_GFAS
CAMS_SOLAR_RADIATION:
dataset: EO:ECMWF:DAT:CAMS_SOLAR_RADIATION_TIMESERIES
altitude: "-999"
metadata_mapping:
geometry:
# longitude/latitude to order from wekeo_ecmwf, location to validate against cop_ads constraints
- '{{"longitude": {geometry#to_longitude_latitude}["lon"], "latitude": {geometry#to_longitude_latitude}["lat"], "location": {{"longitude": {geometry#to_longitude_latitude}["lon"], "latitude": {geometry#to_longitude_latitude}["lat"]}}}}'
- '$.null'
auth: !plugin
type: TokenAuth
auth_uri: 'https://gateway.prod.wekeo2.eu/hda-broker/gettoken'
refresh_uri: 'https://gateway.prod.wekeo2.eu/hda-broker/refreshtoken'
token_type: json
token_key: access_token
refresh_token_key: refresh_token
token_expiration_key: expires_in
download: !plugin
type: HTTPDownload
auth_error_code: 401
order_enabled: true
order_method: 'POST'
order_on_response:
metadata_mapping:
eodag:order_id: '$.json.download_id'
eodag:status_link: 'https://gateway.prod.wekeo2.eu/hda-broker/api/v1/dataaccess/downloads?download_id={eodag:order_id}'
order_status:
request:
method: GET
metadata_mapping:
eodag:order_status: $.json.features[0].status
order:date: $.json.features[0].started_at
created: $.json.features[0].started_at
eodag:order_message: $.json.features[0].message
eodag:order_id: $.json.features[0]._id
ecmwf:dataset: $.json.features[0].dataset_id
error:
eodag:order_status: Error
success:
eodag:order_status: Done
ordered:
http_code: 202
on_success:
metadata_mapping:
eodag:download_id: $.json.features[0]._id
eodag:download_link: https://gateway.prod.wekeo2.eu/hda-broker/api/v1/dataaccess/download/{eodag:download_id}
products:
FIRE_HISTORICAL:
output_extension: .grib
---
!provider # MARK: wekeo_cmems
name: wekeo_cmems
group: wekeo
priority: 0
roles:
- host
description: WEkEO - Copernicus Marine Service
url: https://www.wekeo.eu/
search: !plugin
type: PostJsonSearch
api_endpoint: 'https://gateway.prod.wekeo2.eu/hda-broker/api/v1/dataaccess/search'
need_auth: true
timeout: 60
auth_error_code: 401
results_entry: 'features'
pagination:
total_items_nb_key_path: '$.properties.totalResults'
next_page_query_obj: '{{"itemsPerPage":{limit},"startIndex":{next_page_token}}}'
next_page_token_key: skip
discover_collections:
fetch_url: https://gateway.prod.wekeo2.eu/hda-broker/api/v1/datasets?itemsPerPage=1400&q=EO:MO
single_collection_fetch_qs: q={_collection}
single_collection_fetch_url: https://gateway.prod.wekeo2.eu/hda-broker/api/v1/datasets/{_collection}
result_type: json
results_entry: 'features'
generic_collection_id: '$.dataset_id'
generic_collection_parsable_properties:
collection: '$.dataset_id'
generic_collection_parsable_metadata:
description: '$.metadata.description'
license: '$.terms'
instruments: '$.null'
constellation: '$.null'
platform: '$.null'
single_collection_parsable_metadata:
title: '$.metadata._source.datasetTitle'
_mission_start_date: '$.metadata._source.tempextent_begin'
_mission_end_date: '$.metadata._source.tempextent_end'
extent: '{{"spatial": {{"bbox": [[[[-180.0, -90.0, 180.0, 90.0]]]]}}, "temporal": {{"interval": [[{_mission_start_date}, {_mission_end_date}]]}}}}'
processing:level: '$.null'
keywords: '$.metadata._source.keywords'
discover_queryables:
fetch_url: null
collection_fetch_url: null
constraints_url: 'https://gateway.prod.wekeo2.eu/hda-broker/api/v1/dataaccess/queryable/{dataset}'
constraints_entry: constraints
metadata_mapping:
product:type:
- '{{"dataset_id": "{product:type}"}}'
- '$.null'
id:
- '{{"min_date": {id#dates_from_cmems_id}["min_date"], "max_date": {id#dates_from_cmems_id}["max_date"]}}'
- '$.id'
geometry:
- '{{"bbox": {geometry#to_bounds}}}'
- '$.geometry'
start_datetime:
- '{{"min_date": "{start_datetime#to_iso_utc_datetime}"}}'
- '$.properties.startdate'
end_datetime:
- '{{"max_date": "{end_datetime#to_iso_utc_datetime}"}}'
- '$.properties.enddate'
variable:
- '{{"variable": "{variable}"}}'
- '{$.properties.location#get_variables_from_path}'
eodag:download_link: '$.properties.location'
title: '$.id'
order:status: 'orderable'
eodag:order_link: 'https://gateway.prod.wekeo2.eu/hda-broker/api/v1/dataaccess/download?{{"location": "{eodag:download_link}","product_id":"{id}", "cacheable": "true", "dataset_id": "productType"}}'
products:
GENERIC_COLLECTION:
_collection: '{collection}'
auth: !plugin
type: TokenAuth
matching_url: https://[-\w\.]+.wekeo2.eu
auth_uri: 'https://gateway.prod.wekeo2.eu/hda-broker/gettoken'
refresh_uri: 'https://gateway.prod.wekeo2.eu/hda-broker/refreshtoken'
token_type: json
token_key: access_token
refresh_token_key: refresh_token
download: !plugin
type: HTTPDownload
auth_error_code: 401
order_enabled: true
order_method: 'POST'
order_on_response:
metadata_mapping:
eodag:order_id: '$.json.download_id'
eodag:status_link: 'https://gateway.prod.wekeo2.eu/hda-broker/api/v1/dataaccess/downloads?download_id={eodag:order_id}'
order_status:
request:
method: GET
metadata_mapping:
eodag:order_status: $.json.features[0].status
eodag:order_message: $.json.features[0].message
error:
eodag:order_status: Error
success:
eodag:order_status: Done
ordered:
http_code: 202
on_success:
metadata_mapping:
eodag:download_id: $.json.features[0]._id
eodag:download_link: https://gateway.prod.wekeo2.eu/hda-broker/api/v1/dataaccess/download/{eodag:download_id}
---
!provider # MARK: creodias_s3
name: creodias_s3
priority: 0
description: CloudFerro DIAS data through S3 protocol
roles:
- host
url: https://creodias.eu/
search: !plugin
type: CreodiasS3Search
api_endpoint: 'https://datahub.creodias.eu/odata/v1/Products'
s3_endpoint: 'https://eodata.cloudferro.com'
need_auth: true
timeout: 120
ssl_verify: true
dont_quote:
- '['
- ']'
- '$'
- '='
- '&'
- ':'
- '%'
pagination:
next_page_url_tpl: '{url}?{search}&$top={limit}&$skip={skip}&$expand=Attributes&$expand=Assets'
count_tpl: '&$count=True'
total_items_nb_key_path: '$."@odata.count"'
next_page_url_key_path: '$.["@odata.nextLink"]'
next_page_token_key: skip
parse_url_key: $skip
max_limit: 1_000
sort:
sort_by_default:
- !!python/tuple [start_datetime, ASC]
sort_by_tpl: '&$orderby={sort_param} {sort_order}'
sort_param_mapping:
start_datetime: ContentDate/Start
end_datetime: ContentDate/End
published: PublicationDate
updated: ModificationDate
sort_order_mapping:
ascending: asc
descending: desc
max_sort_params: 1
results_entry: 'value'
free_text_search_operations:
$filter:
union: ' or '
wrapper: '{}'
operations:
and:
- "Collection/Name eq '{_collection}'"
- "OData.CSC.Intersects(area=geography'{geometry#to_ewkt}')"
- "Attributes/OData.CSC.StringAttribute/any(att:att/Name eq 'productType' and att/OData.CSC.StringAttribute/Value eq '{product:type}')"
- "Attributes/OData.CSC.StringAttribute/any(att:att/Name eq 'platformShortName' and att/OData.CSC.StringAttribute/Value eq '{constellation}')"
- "Attributes/OData.CSC.StringAttribute/any(att:att/Name eq 'platformSerialIdentifier' and att/OData.CSC.StringAttribute/Value eq '{platform#replace_str(\"^S[1-3]\", \"\")}')"
- "Attributes/OData.CSC.IntegerAttribute/any(att:att/Name eq 'spatialResolution' and att/OData.CSC.StringAttribute/Value eq '{gsd}')"
- "Attributes/OData.CSC.StringAttribute/any(att:att/Name eq 'authority' and att/OData.CSC.StringAttribute/Value eq '{providers}')"
- "Attributes/OData.CSC.IntegerAttribute/any(att:att/Name eq 'orbitNumber' and att/OData.CSC.StringAttribute/Value eq {sat:absolute_orbit})"
- "Attributes/OData.CSC.IntegerAttribute/any(att:att/Name eq 'relativeOrbitNumber' and att/OData.CSC.StringAttribute/Value eq {sat:relative_orbit})"
- "Attributes/OData.CSC.StringAttribute/any(att:att/Name eq 'orbitDirection' and att/OData.CSC.StringAttribute/Value eq '{sat:orbit_state#to_upper}')"
- "Attributes/OData.CSC.DoubleAttribute/any(att:att/Name eq 'cloudCover' and att/OData.CSC.DoubleAttribute/Value le {eo:cloud_cover})"
- "Attributes/OData.CSC.StringAttribute/any(att:att/Name eq 'instrumentShortName' and att/OData.CSC.StringAttribute/Value eq '{instruments#csv_list}')"
- "Attributes/OData.CSC.StringAttribute/any(att:att/Name eq 'tileId' and att/OData.CSC.StringAttribute/Value eq '{grid:code#replace_str(\"MGRS-\",\"\")}')"
- "Attributes/OData.CSC.StringAttribute/any(att:att/Name eq 'processingLevel' and att/OData.CSC.StringAttribute/Value eq '{processing:level}')"
- "Attributes/OData.CSC.StringAttribute/any(att:att/Name eq 'processingDate' and att/OData.CSC.StringAttribute/Value eq '{processing:datetime}')"
- "Attributes/OData.CSC.StringAttribute/any(att:att/Name eq 'processingCenter' and att/OData.CSC.StringAttribute/Value eq '{processing:facility}')"
- "Attributes/OData.CSC.StringAttribute/any(att:att/Name eq 'processorVersion' and att/OData.CSC.StringAttribute/Value eq '{processing:version}')"
- "Attributes/OData.CSC.StringAttribute/any(att:att/Name eq 'swathIdentifier' and att/OData.CSC.StringAttribute/Value eq '{sar:instrument_mode}')"
- "Attributes/OData.CSC.StringAttribute/any(att:att/Name eq 'datatakeID' and att/OData.CSC.StringAttribute/Value eq '{s1:datatake_id}')"
- "Attributes/OData.CSC.StringAttribute/any(att:att/Name eq 'instrumentConfigurationID' and att/OData.CSC.StringAttribute/Value eq '{s1:instrument_configuration_ID}')"
- "Attributes/OData.CSC.StringAttribute/any(att:att/Name eq 'sliceNumber' and att/OData.CSC.StringAttribute/Value eq '{s1:slice_number}')"
- "Attributes/OData.CSC.StringAttribute/any(att:att/Name eq 'totalSlices' and att/OData.CSC.StringAttribute/Value eq '{s1:total_slices}')"
- "Attributes/OData.CSC.StringAttribute/any(att:att/Name eq 'granuleIdentifier' and att/OData.CSC.StringAttribute/Value eq '{s2:tile_id}')"
- "Attributes/OData.CSC.StringAttribute/any(att:att/Name eq 'productGroupId' and att/OData.CSC.StringAttribute/Value eq '{s2:datatake_id}')"
- "Attributes/OData.CSC.StringAttribute/any(att:att/Name eq 'Name' and att/OData.CSC.StringAttribute/Value eq '{s2:product_uri}')"
- "Attributes/OData.CSC.StringAttribute/any(att:att/Name eq 'datastripId' and att/OData.CSC.StringAttribute/Value eq '{s2:datastrip_id}')"
- "Attributes/OData.CSC.StringAttribute/any(att:att/Name eq 'operationalMode' and att/OData.CSC.StringAttribute/Value eq '{s2:datatake_type}')"
- "ModificationDate gt {updated_after#to_iso_utc_datetime}"
- "ModificationDate lt {updated_before#to_iso_utc_datetime}"
- "PublicationDate gt {published_after#to_iso_utc_datetime}"
- "PublicationDate lt {published_before#to_iso_utc_datetime}"
- "ContentDate/Start lt {end_datetime#to_iso_utc_datetime}"
- "ContentDate/End gt {start_datetime#to_iso_utc_datetime}"
- "Attributes/OData.CSC.StringAttribute/any(att:att/Name eq 'polarisationChannels' and att/OData.CSC.StringAttribute/Value eq '{sar:polarizations#csv_list(%26)}')"
- contains(Name,'{id}')
discover_metadata:
auto_discovery: true
metadata_pattern: '^(?!collection)[a-zA-Z0-9]+$'
search_param:
free_text_search_operations:
$filter:
operations:
and:
- "Attributes/OData.CSC.StringAttribute/any(att:att/Name eq '{metadata}' and att/OData.CSC.StringAttribute/Value eq '{{{metadata}}}')"
metadata_path: '$.Attributes.*'
discover_collections:
fetch_url: null
per_product_metadata_query: false
metadata_pre_mapping:
metadata_path: '$.Attributes'
metadata_path_id: 'Name'
metadata_path_value: 'Value'
metadata_mapping:
_collection:
- null
- '$.null'
# hide duplicated metadata
beginningDateTime: '$.null'
endingDateTime: '$.null'
platformSerialIdentifier: '$.null'
# Opensearch resource identifier within the search engine context (in our case
# within the context of the data provider)
# Queryable parameters are set with null as 1st configuration list value to mark them as queryable,
# but `free_text_search_operations.$filter.operations.and` entries are then used instead.
uid: '$.Id'
# OpenSearch Parameters for Collection Search (Table 3)
product:type:
- null
- '$.Attributes.productType'
constellation:
- null
- '$.Attributes.platformShortName'
platform:
- null
- '$.Attributes.platformSerialIdentifier'
instruments:
- null
- '{$.Attributes.instrumentShortName#split( )}'
processing:level:
- null
- '$.Attributes.processingLevel'
processing:datetime:
- null
- '$.Attributes.processingDate'
processing:facility:
- null
- '$.Attributes.processingCenter'
processing:version:
- null
- '{$.Attributes.processorVersion#to_geojson}'
_processor_name:
- null
- '$.Attributes.processorName'
processing:software: '{{"{_processor_name}":"{processing:version}"}}'
# INSPIRE obligated OpenSearch Parameters for Collection Search (Table 4)
title: '{$.Name#remove_extension}'
gsd:
- null
- '$.Attributes.spatialResolution'
_provider: '$.Attributes.origin'
providers:
- null
- '[{{"name":"{_provider}","roles":["producer"]}}]'
published_after:
- null
- '$.null'
published_before:
- null
- '$.null'
published: '$.PublicationDate'
# OpenSearch Parameters for Product Search (Table 5)
sat:absolute_orbit:
- null
- '$.Attributes.orbitNumber'
sat:relative_orbit:
- null
- '$.Attributes.relativeOrbitNumber'
sat:orbit_state:
- null
- '{$.Attributes.orbitDirection#to_lower}'
eo:cloud_cover:
- null
- '$.Attributes.cloudCover'
updated_after:
- null
- '$.null'
updated_before:
- null
- '$.null'
updated: '$.ModificationDate'
# OpenSearch Parameters for Acquistion Parameters Search (Table 6)
start_datetime:
- null
- '$.ContentDate.Start'
end_datetime:
- null
- '$.ContentDate.End'
product:timeliness:
- null
- '$.Attributes.timeliness'
sar:instrument_mode:
- null
- '$.Attributes.swathIdentifier'
sar:polarizations:
- null
- '{$.Attributes.polarisationChannels#split(&)}'
s1:datatake_id:
- null
- '$.Attributes.datatakeID'
s1:instrument_configuration_ID:
- null
- '$.Attributes.instrumentConfigurationID'
s1:slice_number:
- null
- '$.Attributes.sliceNumber'
s1:total_slices:
- null
- '$.Attributes.totalSlices'
s2:tile_id:
- null
- '$.Attributes.granuleIdentifier'
s2:datatake_id:
- null
- '$.Attributes.productGroupId'
s2:product_uri:
- null
- '$.Attributes.Name'
s2:datastrip_id:
- null
- '$.Attributes.datastripId'
s2:datatake_type:
- null
- '$.Attributes.operationalMode'
# Custom parameters (not defined in the base document referenced above)
id:
- null
- '{$.Name#remove_extension}'
grid:code:
- null
- '{$.Attributes.tileId#replace_str(r"^","MGRS-")}'
# The geographic extent of the product
geometry:
- null
- '{$.Footprint#from_ewkt}'
# The url to download the product "as is" (literal or as a template to be completed either after the search result
# is obtained from the provider or during the eodag download phase)
eodag:download_link: '$.S3Path.`sub(/^(.*)$/, s3:/\\1)`'
# order:status: must be one of succeeded, ordered, orderable
order:status: '{$.Online#get_group_name((?P<succeeded>True)|(?P<orderable>False))}'
collection:
- null
- $.null
eodag:quicklook: '$.Assets[?(@.Type="QUICKLOOK")].DownloadLink'
eodag:thumbnail: '$.Assets[?(@.Type="QUICKLOOK")].DownloadLink'
download: !plugin
type: AwsDownload
s3_endpoint: 'https://eodata.cloudferro.com'
s3_bucket: 'eodata'
ssl_verify: true
auth: !plugin
type: AwsAuth
auth_error_code: 403
s3_endpoint: 'https://eodata.cloudferro.com'
support_presign_url: False
matching_conf:
s3_endpoint: 'https://eodata.cloudferro.com'
products:
# S1
S1_AUX_GNSSRD:
product:type: AUX_GNSSRD
_collection: SENTINEL-1
metadata_mapping:
eo:cloud_cover: '{$.Attributes.cloudCover#not_available}'
platform:
- null
- '$.Attributes.platformSerialIdentifier.`sub(/^/, S1)`'
S1_AUX_MOEORB:
product:type: AUX_MOEORB
_collection: SENTINEL-1
metadata_mapping:
eo:cloud_cover: '{$.Attributes.cloudCover#not_available}'
platform:
- null
- '$.Attributes.platformSerialIdentifier.`sub(/^/, S1)`'
S1_AUX_POEORB:
product:type: AUX_POEORB
_collection: SENTINEL-1
metadata_mapping:
eo:cloud_cover: '{$.Attributes.cloudCover#not_available}'
platform:
- null
- '$.Attributes.platformSerialIdentifier.`sub(/^/, S1)`'
S1_AUX_PREORB:
product:type: AUX_PREORB
_collection: SENTINEL-1
metadata_mapping:
eo:cloud_cover: '{$.Attributes.cloudCover#not_available}'
platform:
- null
- '$.Attributes.platformSerialIdentifier.`sub(/^/, S1)`'
S1_AUX_PROQUA:
product:type: AUX_PROQUA
_collection: SENTINEL-1
metadata_mapping:
eo:cloud_cover: '{$.Attributes.cloudCover#not_available}'
platform:
- null
- '$.Attributes.platformSerialIdentifier.`sub(/^/, S1)`'
S1_AUX_RESORB:
product:type: AUX_RESORB
_collection: SENTINEL-1
metadata_mapping:
eo:cloud_cover: '{$.Attributes.cloudCover#not_available}'
platform:
- null
- '$.Attributes.platformSerialIdentifier.`sub(/^/, S1)`'
S1_SAR_RAW:
product:type: RAW
_collection: SENTINEL-1
metadata_mapping:
eo:cloud_cover: '{$.Attributes.cloudCover#not_available}'
platform:
- null
- '$.Attributes.platformSerialIdentifier.`sub(/^/, S1)`'
S1_SAR_GRD:
product:type: GRD
_collection: SENTINEL-1
metadata_mapping:
eo:cloud_cover: '{$.Attributes.cloudCover#not_available}'
platform:
- null
- '$.Attributes.platformSerialIdentifier.`sub(/^/, S1)`'
S1_SAR_SLC:
product:type: SLC
_collection: SENTINEL-1
metadata_mapping:
eo:cloud_cover: '{$.Attributes.cloudCover#not_available}'
platform:
- null
- '$.Attributes.platformSerialIdentifier.`sub(/^/, S1)`'
S1_SAR_OCN:
product:type: OCN
_collection: SENTINEL-1
metadata_mapping:
eo:cloud_cover: '{$.Attributes.cloudCover#not_available}'
platform:
- null
- '$.Attributes.platformSerialIdentifier.`sub(/^/, S1)`'
S1_SAR_L3_IW_MCM:
product:type: S1SAR_L3_IW_MCM
_collection: GLOBAL-MOSAICS
metadata_mapping:
eo:cloud_cover: '{$.Attributes.cloudCover#not_available}'
platform:
- null
- '$.Attributes.platformSerialIdentifier.`sub(/^/, S1)`'
S1_SAR_L3_DH_MCM:
product:type: S1SAR_L3_DH_MCM
_collection: GLOBAL-MOSAICS
metadata_mapping:
eo:cloud_cover: '{$.Attributes.cloudCover#not_available}'
platform:
- null
- '$.Attributes.platformSerialIdentifier.`sub(/^/, S1)`'
# S2
S2_MSI_L1C:
_collection: SENTINEL-2
product:type: S2MSI1C
metadata_mapping:
platform:
- null
- '$.Attributes.platformSerialIdentifier.`sub(/^/, S2)`'
S2_MSI_L2A:
_collection: SENTINEL-2
product:type: S2MSI2A
metadata_mapping:
platform:
- null
- '$.Attributes.platformSerialIdentifier.`sub(/^/, S2)`'
# S3 SRAL
S3_SRA:
product:type: SR_1_SRA___
_collection: SENTINEL-3
metadata_mapping:
eo:cloud_cover: '{$.Attributes.cloudCover#not_available}'
platform:
- null
- '$.Attributes.platformSerialIdentifier.`sub(/^/, S3)`'
S3_SRA_A:
product:type: SR_1_SRA_A_
_collection: SENTINEL-3
metadata_mapping:
eo:cloud_cover: '{$.Attributes.cloudCover#not_available}'
platform:
- null
- '$.Attributes.platformSerialIdentifier.`sub(/^/, S3)`'
S3_SRA_BS:
product:type: SR_1_SRA_BS
_collection: SENTINEL-3
metadata_mapping:
eo:cloud_cover: '{$.Attributes.cloudCover#not_available}'
platform:
- null
- '$.Attributes.platformSerialIdentifier.`sub(/^/, S3)`'
S3_LAN:
product:type: SR_2_LAN___
_collection: SENTINEL-3
metadata_mapping:
eo:cloud_cover: '{$.Attributes.cloudCover#not_available}'
platform:
- null
- '$.Attributes.platformSerialIdentifier.`sub(/^/, S3)`'
S3_LAN_HY:
product:type: SR_2_LAN_HY
_collection: SENTINEL-3
metadata_mapping:
eo:cloud_cover: '{$.Attributes.cloudCover#not_available}'
platform:
- null
- '$.Attributes.platformSerialIdentifier.`sub(/^/, S3)`'
S3_LAN_SI:
product:type: SR_2_LAN_SI
_collection: SENTINEL-3
metadata_mapping:
eo:cloud_cover: '{$.Attributes.cloudCover#not_available}'
platform:
- null
- '$.Attributes.platformSerialIdentifier.`sub(/^/, S3)`'
S3_LAN_LI:
product:type: SR_2_LAN_LI
_collection: SENTINEL-3
metadata_mapping:
eo:cloud_cover: '{$.Attributes.cloudCover#not_available}'
platform:
- null
- '$.Attributes.platformSerialIdentifier.`sub(/^/, S3)`'
S3_WAT:
product:type: SR_2_WAT___
_collection: SENTINEL-3
metadata_mapping:
eo:cloud_cover: '{$.Attributes.cloudCover#not_available}'
platform:
- null
- '$.Attributes.platformSerialIdentifier.`sub(/^/, S3)`'
# S3 OLCI
S3_EFR:
product:type: OL_1_EFR___
_collection: SENTINEL-3
metadata_mapping:
platform:
- null
- '$.Attributes.platformSerialIdentifier.`sub(/^/, S3)`'
S3_ERR:
product:type: OL_1_ERR___
_collection: SENTINEL-3
metadata_mapping:
platform:
- null
- '$.Attributes.platformSerialIdentifier.`sub(/^/, S3)`'
S3_OLCI_L2LRR:
product:type: OL_2_LRR___
_collection: SENTINEL-3
metadata_mapping:
platform:
- null
- '$.Attributes.platformSerialIdentifier.`sub(/^/, S3)`'
S3_OLCI_L2LFR:
product:type: OL_2_LFR___
_collection: SENTINEL-3
metadata_mapping:
platform:
- null
- '$.Attributes.platformSerialIdentifier.`sub(/^/, S3)`'
S3_OLCI_L2WRR:
product:type: OL_2_WRR___
_collection: SENTINEL-3
metadata_mapping:
platform:
- null
- '$.Attributes.platformSerialIdentifier.`sub(/^/, S3)`'
S3_OLCI_L2WFR:
product:type: OL_2_WFR___
_collection: SENTINEL-3
metadata_mapping:
platform:
- null
- '$.Attributes.platformSerialIdentifier.`sub(/^/, S3)`'
# S3 SLSTR
S3_SLSTR_L1RBT:
product:type: SL_1_RBT___
_collection: SENTINEL-3
metadata_mapping:
platform:
- null
- '$.Attributes.platformSerialIdentifier.`sub(/^/, S3)`'
S3_SLSTR_L2LST:
product:type: SL_2_LST___
_collection: SENTINEL-3
metadata_mapping:
platform:
- null
- '$.Attributes.platformSerialIdentifier.`sub(/^/, S3)`'
S3_SLSTR_L2WST:
product:type: SL_2_WST___
_collection: SENTINEL-3
metadata_mapping:
platform:
- null
- '$.Attributes.platformSerialIdentifier.`sub(/^/, S3)`'
S3_SLSTR_L2AOD:
product:type: SL_2_AOD___
_collection: SENTINEL-3
metadata_mapping:
platform:
- null
- '$.Attributes.platformSerialIdentifier.`sub(/^/, S3)`'
S3_SLSTR_L2FRP:
product:type: SL_2_FRP___
_collection: SENTINEL-3
metadata_mapping:
platform:
- null
- '$.Attributes.platformSerialIdentifier.`sub(/^/, S3)`'
# S3 SY
S3_SY_AOD:
product:type: SY_2_AOD___
_collection: SENTINEL-3
metadata_mapping:
platform:
- null
- '$.Attributes.platformSerialIdentifier.`sub(/^/, S3)`'
S3_SY_SYN:
product:type: SY_2_SYN___
_collection: SENTINEL-3
metadata_mapping:
platform:
- null
- '$.Attributes.platformSerialIdentifier.`sub(/^/, S3)`'
S3_SY_V10:
product:type: SY_2_V10___
_collection: SENTINEL-3
metadata_mapping:
platform:
- null
- '$.Attributes.platformSerialIdentifier.`sub(/^/, S3)`'
S3_SY_VG1:
product:type: SY_2_VG1___
_collection: SENTINEL-3
metadata_mapping:
platform:
- null
- '$.Attributes.platformSerialIdentifier.`sub(/^/, S3)`'
S3_SY_VGP:
product:type: SY_2_VGP___
_collection: SENTINEL-3
metadata_mapping:
platform:
- null
- '$.Attributes.platformSerialIdentifier.`sub(/^/, S3)`'
# S5P L1
S5P_L1B_IR_SIR:
product:type: L1B_IR_SIR
_collection: SENTINEL-5P
S5P_L1B_IR_UVN:
product:type: L1B_IR_UVN
_collection: SENTINEL-5P
S5P_L1B_RA_BD1:
product:type: L1B_RA_BD1
_collection: SENTINEL-5P
S5P_L1B_RA_BD2:
product:type: L1B_RA_BD2
_collection: SENTINEL-5P
S5P_L1B_RA_BD3:
product:type: L1B_RA_BD3
_collection: SENTINEL-5P
S5P_L1B_RA_BD4:
product:type: L1B_RA_BD4
_collection: SENTINEL-5P
S5P_L1B_RA_BD5:
product:type: L1B_RA_BD5
_collection: SENTINEL-5P
S5P_L1B_RA_BD6:
product:type: L1B_RA_BD6
_collection: SENTINEL-5P
S5P_L1B_RA_BD7:
product:type: L1B_RA_BD7
_collection: SENTINEL-5P
S5P_L1B_RA_BD8:
product:type: L1B_RA_BD8
_collection: SENTINEL-5P
# S5P L2
S5P_L2_NO2:
product:type: L2__NO2___
_collection: SENTINEL-5P
S5P_L2_CLOUD:
product:type: L2__CLOUD_
_collection: SENTINEL-5P
S5P_L2_O3:
product:type: L2__O3____
_collection: SENTINEL-5P
S5P_L2_CO:
product:type: L2__CO____
_collection: SENTINEL-5P
S5P_L2_AER_AI:
product:type: L2__AER_AI
_collection: SENTINEL-5P
S5P_L2_O3_PR:
product:type: L2__O3__PR
_collection: SENTINEL-5P
S5P_L2_O3_TCL:
product:type: L2__O3_TCL
_collection: SENTINEL-5P
S5P_L2_AER_LH:
product:type: L2__AER_LH
_collection: SENTINEL-5P
S5P_L2_HCHO:
product:type: L2__HCHO__
_collection: SENTINEL-5P
S5P_L2_CH4:
product:type: L2__CH4___
_collection: SENTINEL-5P
S5P_L2_NP_BD3:
product:type: L2__NP_BD3
_collection: SENTINEL-5P
S5P_L2_NP_BD6:
product:type: L2__NP_BD6
_collection: SENTINEL-5P
S5P_L2_NP_BD7:
product:type: L2__NP_BD7
_collection: SENTINEL-5P
S5P_L2_SO2:
product:type: L2__SO2___
_collection: SENTINEL-5P
# COP DEM
COP_DEM_GLO30_DGED:
product:type: DGE_30
_collection: COP-DEM
metadata_mapping:
eo:cloud_cover: '{$.Attributes.cloudCover#not_available}'
COP_DEM_GLO30_DTED:
product:type: DTE_30
_collection: COP-DEM
metadata_mapping:
eo:cloud_cover: '{$.Attributes.cloudCover#not_available}'
COP_DEM_GLO90_DGED:
product:type: DGE_90
_collection: COP-DEM
metadata_mapping:
eo:cloud_cover: '{$.Attributes.cloudCover#not_available}'
COP_DEM_GLO90_DTED:
product:type: DTE_90
_collection: COP-DEM
metadata_mapping:
eo:cloud_cover: '{$.Attributes.cloudCover#not_available}'
GENERIC_COLLECTION:
_collection: '{collection}'
---
!provider # MARK: dedt_lumi
name: dedt_lumi
priority: 0
roles:
- host
description: Destination Earth Digital Twin Outputs from LUMI through Polytope API
url: https://polytope.lumi.apps.dte.destination-earth.eu/openapi
search: !plugin
type: ECMWFSearch
need_auth: true
ssl_verify: true
discover_queryables:
fetch_url: null
collection_fetch_url: null
constraints_url: "https://s3.central.data.destination-earth.eu/swift/v1/constraints/dedt_lumi/{dataset}_{activity}_{experiment}_{model}.json"
form_url: "https://s3.central.data.destination-earth.eu/swift/v1/constraints/dedt_lumi/form_{dataset}.json"
discover_metadata:
auto_discovery: true
search_param: '{metadata}'
metadata_pattern: '^(feature|interpolation|grid)$'
metadata_mapping:
geometry:
- '{{"feature": {geometry#to_geojson_polytope}}}'
- "$.geometry"
start_datetime: '{$.start_datetime#to_iso_utc_datetime}'
end_datetime:
- '{{"date": "{start_datetime#to_non_separated_date}/to/{end_datetime#to_non_separated_date}"}}'
- '{$.end_datetime#to_iso_utc_datetime}'
product:type:
- dataset
- $.dataset
qs: $.qs
eodag:order_link: 'https://polytope.lumi.apps.dte.destination-earth.eu/api/v1/requests/destination-earth?{{"verb": "retrieve", "request": {qs#to_geojson} }}'
products:
DT_CLIMATE_ADAPTATION:
discover_queryables:
constraints_url: "https://s3.central.data.destination-earth.eu/swift/v1/dedt-lumi-constraints/{dataset}.json"
dataset: climate-dt
DT_EXTREMES:
discover_queryables:
constraints_url: "https://s3.central.data.destination-earth.eu/swift/v1/dedt-lumi-constraints/{dataset}.json"
dataset: extremes-dt
class: d1
expver: "0001"
type: fc
time: "0000"
DT_CLIMATE_G1_CMIP6_HIST_ICON_R1:
dataset: climate-dt
class: d1
generation: "1"
expver: "0001"
stream: clte
type: fc
activity: CMIP6
realization: "1"
model: ICON
resolution: high
experiment: hist
DT_CLIMATE_G1_CMIP6_HIST_IFS_NEMO_R1:
dataset: climate-dt
class: d1
generation: "1"
expver: "0001"
stream: clte
type: fc
activity: CMIP6
experiment: hist
realization: "1"
model: IFS-NEMO
DT_CLIMATE_G1_HIGHRESMIP_CONT_IFS_NEMO_R1:
dataset: climate-dt
class: d1
generation: "1"
expver: "0001"
stream: clte
type: fc
activity: HighResMIP
experiment: cont
realization: "1"
model: IFS-NEMO
resolution: high
DT_CLIMATE_G1_SCENARIOMIP_SSP3_7_0_ICON_R1:
dataset: climate-dt
class: d1
generation: "1"
expver: "0001"
stream: clte
type: fc
activity: ScenarioMIP
experiment: SSP3-7.0
realization: "1"
model: ICON
resolution: high
DT_CLIMATE_G1_SCENARIOMIP_SSP3_7_0_IFS_NEMO_R1:
dataset: climate-dt
class: d1
generation: "1"
expver: "0001"
stream: clte
type: fc
activity: ScenarioMIP
experiment: SSP3-7.0
realization: "1"
model: IFS-NEMO
DT_CLIMATE_G1_STORY_NUDGING_CONT_IFS_FESOM_R1:
dataset: climate-dt
class: d1
generation: "1"
expver: "0001"
stream: clte
type: fc
activity: story-nudging
experiment: cont
realization: "1"
model: IFS-FESOM
DT_CLIMATE_G1_STORY_NUDGING_HIST_IFS_FESOM_R1:
dataset: climate-dt
class: d1
generation: "1"
expver: "0001"
stream: clte
type: fc
activity: story-nudging
experiment: hist
realization: "1"
model: IFS-FESOM
DT_CLIMATE_G1_STORY_NUDGING_TPLUS2_0K_IFS_FESOM_R1:
dataset: climate-dt
class: d1
generation: "1"
expver: "0001"
stream: clte
type: fc
activity: story-nudging
experiment: Tplus2.0K
realization: "1"
model: IFS-FESOM
DT_CLIMATE_G1_CMIP6_HIST_IFS_FESOM_R1:
discover_queryables:
form_url: "https://s3.central.data.destination-earth.eu/swift/v1/constraints/dedt_lumi/form_DT_CLIMATE_G1_CMIP6_HIST_IFS_FESOM_R1.json"
constraints_url: "https://s3.central.data.destination-earth.eu/swift/v1/constraints/dedt_lumi/climate-dt_{activity}_{experiment}_{model}.json"
class: ng
generation: "1"
expver: "0001"
stream: clte
type: fc
activity: CMIP6
experiment: hist
realization: "1"
model: IFS-FESOM
DT_CLIMATE_G1_SCENARIOMIP_SSP3_7_0_IFS_FESOM_R2:
dataset: climate-dt
class: d1
generation: "1"
expver: "0001"
stream: clte
type: fc
activity: ScenarioMIP
experiment: SSP3-7.0
realization: "2"
model: IFS-FESOM
DT_CLIMATE_G2_BASELINE_CONT_ICON_R1:
discover_queryables:
form_url: "https://s3.central.data.destination-earth.eu/swift/v1/constraints/dedt_lumi/form_climate-dt-phase2.json"
dataset: climate-dt
class: d1
generation: "2"
expver: "0001"
type: fc
activity: baseline
experiment: cont
realization: "1"
model: ICON
DT_CLIMATE_G2_BASELINE_CONT_IFS_FESOM_R1:
discover_queryables:
form_url: "https://s3.central.data.destination-earth.eu/swift/v1/constraints/dedt_lumi/form_climate-dt-phase2.json"
dataset: climate-dt
class: d1
generation: "2"
expver: "0001"
type: fc
activity: baseline
experiment: cont
realization: "1"
model: IFS-FESOM
DT_CLIMATE_G2_BASELINE_HIST_ICON_R1:
discover_queryables:
form_url: "https://s3.central.data.destination-earth.eu/swift/v1/constraints/dedt_lumi/form_climate-dt-phase2.json"
dataset: climate-dt
class: d1
generation: "2"
expver: "0001"
type: fc
activity: baseline
experiment: hist
realization: "1"
model: ICON
DT_CLIMATE_G2_BASELINE_HIST_IFS_FESOM_R1:
discover_queryables:
form_url: "https://s3.central.data.destination-earth.eu/swift/v1/constraints/dedt_lumi/form_climate-dt-phase2.json"
dataset: climate-dt
class: d1
generation: "2"
expver: "0001"
type: fc
activity: baseline
experiment: hist
realization: "1"
model: IFS-FESOM
DT_CLIMATE_G2_PROJECTIONS_SSP3_7_0_ICON_R1:
discover_queryables:
form_url: "https://s3.central.data.destination-earth.eu/swift/v1/constraints/dedt_lumi/form_climate-dt-phase2.json"
dataset: climate-dt
class: d1
generation: "2"
expver: "0001"
type: fc
activity: projections
experiment: SSP3-7.0
realization: "1"
model: ICON
DT_CLIMATE_G2_PROJECTIONS_SSP3_7_0_IFS_FESOM_R1:
discover_queryables:
form_url: "https://s3.central.data.destination-earth.eu/swift/v1/constraints/dedt_lumi/form_climate-dt-phase2.json"
dataset: climate-dt
class: d1
generation: "2"
expver: "0001"
type: fc
activity: projections
experiment: SSP3-7.0
realization: "1"
model: IFS-FESOM
download: !plugin
type: HTTPDownload
ssl_verify: true
auth_error_code: 401
order_enabled: True
order_method: POST
order_on_response:
metadata_mapping:
eodag:order_id: '{$.headers.Location#slice_str(-36,,1)}'
_previous_order_id: '{$.json.eodag:order_id#replace_str("Not Available","")}'
eodag:combined_order_id: '{eodag:order_id#replace_str("Not Available","")}{_previous_order_id}'
eodag:status_link: "https://polytope.lumi.apps.dte.destination-earth.eu/api/v1/requests/{eodag:combined_order_id#replace_str(r'^$','Not Available')}"
order_status:
request:
method: GET
headers:
metadata_mapping:
eodag:order_status: $.json.status
eodag:order_message: $.json.message
error_message: $.null
success:
http_code: 303
error:
eodag:order_status: failed
on_success:
result_type: json
metadata_mapping:
eodag:download_link: $.headers.Location
no_auth_download: True
auth: !plugin
type: OIDCAuthorizationCodeFlowAuth
oidc_config_url: https://auth.destine.eu/realms/desp/.well-known/openid-configuration
redirect_uri: https://polytope.lumi.apps.dte.destination-earth.eu/
client_id: polytope-api-public
user_consent_needed: false
token_exchange_post_data_method: data
token_provision: header
login_form_xpath: //form[@id='kc-form-login']
authentication_uri_source: login-form
---
!provider # MARK: dedt_mn5
name: dedt_mn5
priority: 0
roles:
- host
description: Destination Earth Digital Twin Outputs from marenostrum through Polytope API
url: https://polytope.mn5.apps.dte.destination-earth.eu/openapi
search: !plugin
type: ECMWFSearch
need_auth: true
ssl_verify: true
discover_queryables:
fetch_url: null
collection_fetch_url: null
constraints_url: "https://s3.central.data.destination-earth.eu/swift/v1/constraints/dedt_mn5/{dataset}_{activity}_{experiment}_{model}.json"
form_url: "https://s3.central.data.destination-earth.eu/swift/v1/constraints/dedt_mn5/form_{dataset}.json"
discover_metadata:
auto_discovery: true
search_param: '{metadata}'
metadata_pattern: '^(feature|interpolation|grid)$'
metadata_mapping:
geometry:
- '{{"feature": {geometry#to_geojson_polytope}}}'
- "$.geometry"
start_datetime: '{$.start_datetime#to_iso_utc_datetime}'
end_datetime:
- '{{"date": "{start_datetime#to_non_separated_date}/to/{end_datetime#to_non_separated_date}"}}'
- '{$.end_datetime#to_iso_utc_datetime}'
product:type:
- dataset
- $.dataset
qs: $.qs
eodag:order_link: 'https://polytope.mn5.apps.dte.destination-earth.eu/api/v1/requests/destination-earth?{{"verb": "retrieve", "request": {qs#to_geojson} }}'
products:
DT_CLIMATE_G1_HIGHRESMIP_CONT_IFS_FESOM_R1:
dataset: climate-dt
class: d1
generation: "1"
expver: "0001"
stream: clte
type: fc
activity: HighResMIP
experiment: cont
realization: "1"
model: IFS-FESOM
DT_CLIMATE_G1_SCENARIOMIP_SSP3_7_0_IFS_FESOM_R1:
class: d1
dataset: climate-dt
generation: "1"
expver: "0001"
stream: clte
type: fc
activity: ScenarioMIP
experiment: SSP3-7.0
realization: "1"
model: IFS-FESOM
DT_CLIMATE_G2_BASELINE_CONT_IFS_NEMO_R1:
discover_queryables:
form_url: "https://s3.central.data.destination-earth.eu/swift/v1/constraints/dedt_mn5/form_climate-dt-phase2.json"
dataset: climate-dt
class: d1
generation: "2"
expver: "0001"
type: fc
activity: baseline
experiment: cont
realization: "1"
model: IFS-NEMO
DT_CLIMATE_G2_BASELINE_HIST_IFS_NEMO_R1:
discover_queryables:
form_url: "https://s3.central.data.destination-earth.eu/swift/v1/constraints/dedt_mn5/form_climate-dt-phase2.json"
dataset: climate-dt
class: d1
generation: "2"
expver: "0001"
type: fc
activity: baseline
experiment: hist
realization: "1"
model: IFS-NEMO
DT_CLIMATE_G2_PROJECTIONS_SSP3_7_0_IFS_NEMO_R1:
discover_queryables:
form_url: "https://s3.central.data.destination-earth.eu/swift/v1/constraints/dedt_mn5/form_climate-dt-phase2.json"
dataset: climate-dt
class: d1
generation: "2"
expver: "0001"
type: fc
activity: projections
experiment: SSP3-7.0
realization: "1"
model: IFS-NEMO
DT_CLIMATE_G2_STORY_NUDGING_CONT_IFS_FESOM_R1:
discover_queryables:
constraints_url: "https://s3.central.data.destination-earth.eu/swift/v1/constraints/dedt_mn5/climate-dt_story-nudging_cont_IFS-FESOM_2_1.json"
form_url: "https://s3.central.data.destination-earth.eu/swift/v1/constraints/dedt_mn5/form_climate-dt-phase2.json"
dataset: climate-dt
class: d1
generation: "2"
expver: "0001"
type: fc
activity: story-nudging
experiment: cont
realization: "1"
model: IFS-FESOM
DT_CLIMATE_G2_STORY_NUDGING_CONT_IFS_FESOM_R2:
discover_queryables:
constraints_url: "https://s3.central.data.destination-earth.eu/swift/v1/constraints/dedt_mn5/climate-dt_story-nudging_cont_IFS-FESOM_2_2.json"
form_url: "https://s3.central.data.destination-earth.eu/swift/v1/constraints/dedt_mn5/form_climate-dt-phase2.json"
dataset: climate-dt
class: d1
generation: "2"
expver: "0001"
type: fc
activity: story-nudging
experiment: cont
realization: "2"
model: IFS-FESOM
DT_CLIMATE_G2_STORY_NUDGING_CONT_IFS_FESOM_R3:
discover_queryables:
constraints_url: "https://s3.central.data.destination-earth.eu/swift/v1/constraints/dedt_mn5/climate-dt_story-nudging_cont_IFS-FESOM_2_3.json"
form_url: "https://s3.central.data.destination-earth.eu/swift/v1/constraints/dedt_mn5/form_climate-dt-phase2.json"
dataset: climate-dt
class: d1
generation: "2"
expver: "0001"
type: fc
activity: story-nudging
experiment: cont
realization: "3"
model: IFS-FESOM
DT_CLIMATE_G2_STORY_NUDGING_CONT_IFS_FESOM_R4:
discover_queryables:
constraints_url: "https://s3.central.data.destination-earth.eu/swift/v1/constraints/dedt_mn5/climate-dt_story-nudging_cont_IFS-FESOM_2_4.json"
form_url: "https://s3.central.data.destination-earth.eu/swift/v1/constraints/dedt_mn5/form_climate-dt-phase2.json"
dataset: climate-dt
class: d1
generation: "2"
expver: "0001"
type: fc
activity: story-nudging
experiment: cont
realization: "4"
model: IFS-FESOM
DT_CLIMATE_G2_STORY_NUDGING_CONT_IFS_FESOM_R5:
discover_queryables:
constraints_url: "https://s3.central.data.destination-earth.eu/swift/v1/constraints/dedt_mn5/climate-dt_story-nudging_cont_IFS-FESOM_2_5.json"
form_url: "https://s3.central.data.destination-earth.eu/swift/v1/constraints/dedt_mn5/form_climate-dt-phase2.json"
dataset: climate-dt
class: d1
generation: "2"
expver: "0001"
type: fc
activity: story-nudging
experiment: cont
realization: "5"
model: IFS-FESOM
DT_CLIMATE_G2_STORY_NUDGING_HIST_IFS_FESOM_R1:
discover_queryables:
constraints_url: "https://s3.central.data.destination-earth.eu/swift/v1/constraints/dedt_mn5/climate-dt_story-nudging_hist_IFS-FESOM_2_1.json"
form_url: "https://s3.central.data.destination-earth.eu/swift/v1/constraints/dedt_mn5/form_climate-dt-phase2.json"
dataset: climate-dt
class: d1
generation: "2"
expver: "0001"
type: fc
activity: story-nudging
experiment: hist
realization: "1"
model: IFS-FESOM
DT_CLIMATE_G2_STORY_NUDGING_HIST_IFS_FESOM_R2:
discover_queryables:
constraints_url: "https://s3.central.data.destination-earth.eu/swift/v1/constraints/dedt_mn5/climate-dt_story-nudging_hist_IFS-FESOM_2_2.json"
form_url: "https://s3.central.data.destination-earth.eu/swift/v1/constraints/dedt_mn5/form_climate-dt-phase2.json"
dataset: climate-dt
class: d1
generation: "2"
expver: "0001"
type: fc
activity: story-nudging
experiment: hist
realization: "2"
model: IFS-FESOM
DT_CLIMATE_G2_STORY_NUDGING_HIST_IFS_FESOM_R3:
discover_queryables:
constraints_url: "https://s3.central.data.destination-earth.eu/swift/v1/constraints/dedt_mn5/climate-dt_story-nudging_hist_IFS-FESOM_2_3.json"
form_url: "https://s3.central.data.destination-earth.eu/swift/v1/constraints/dedt_mn5/form_climate-dt-phase2.json"
dataset: climate-dt
class: d1
generation: "2"
expver: "0001"
type: fc
activity: story-nudging
experiment: hist
realization: "3"
model: IFS-FESOM
DT_CLIMATE_G2_STORY_NUDGING_HIST_IFS_FESOM_R4:
discover_queryables:
constraints_url: "https://s3.central.data.destination-earth.eu/swift/v1/constraints/dedt_mn5/climate-dt_story-nudging_hist_IFS-FESOM_2_4.json"
form_url: "https://s3.central.data.destination-earth.eu/swift/v1/constraints/dedt_mn5/form_climate-dt-phase2.json"
dataset: climate-dt
class: d1
generation: "2"
expver: "0001"
type: fc
activity: story-nudging
experiment: hist
realization: "4"
model: IFS-FESOM
DT_CLIMATE_G2_STORY_NUDGING_HIST_IFS_FESOM_R5:
discover_queryables:
constraints_url: "https://s3.central.data.destination-earth.eu/swift/v1/constraints/dedt_mn5/climate-dt_story-nudging_hist_IFS-FESOM_2_5.json"
form_url: "https://s3.central.data.destination-earth.eu/swift/v1/constraints/dedt_mn5/form_climate-dt-phase2.json"
dataset: climate-dt
class: d1
generation: "2"
expver: "0001"
type: fc
activity: story-nudging
experiment: hist
realization: "5"
model: IFS-FESOM
DT_CLIMATE_G2_STORY_NUDGING_TPLUS2_0K_IFS_FESOM_R1:
discover_queryables:
constraints_url: "https://s3.central.data.destination-earth.eu/swift/v1/constraints/dedt_mn5/climate-dt_story-nudging_Tplus2.0K_IFS-FESOM_2_1.json"
form_url: "https://s3.central.data.destination-earth.eu/swift/v1/constraints/dedt_mn5/form_climate-dt-phase2.json"
dataset: climate-dt
class: d1
generation: "2"
expver: "0001"
type: fc
activity: story-nudging
experiment: Tplus2.0K
realization: "1"
model: IFS-FESOM
DT_CLIMATE_G2_STORY_NUDGING_TPLUS2_0K_IFS_FESOM_R2:
discover_queryables:
constraints_url: "https://s3.central.data.destination-earth.eu/swift/v1/constraints/dedt_mn5/climate-dt_story-nudging_Tplus2.0K_IFS-FESOM_2_2.json"
form_url: "https://s3.central.data.destination-earth.eu/swift/v1/constraints/dedt_mn5/form_climate-dt-phase2.json"
dataset: climate-dt
class: d1
generation: "2"
expver: "0001"
type: fc
activity: story-nudging
experiment: Tplus2.0K
realization: "2"
model: IFS-FESOM
DT_CLIMATE_G2_STORY_NUDGING_TPLUS2_0K_IFS_FESOM_R3:
discover_queryables:
constraints_url: "https://s3.central.data.destination-earth.eu/swift/v1/constraints/dedt_mn5/climate-dt_story-nudging_Tplus2.0K_IFS-FESOM_2_3.json"
form_url: "https://s3.central.data.destination-earth.eu/swift/v1/constraints/dedt_mn5/form_climate-dt-phase2.json"
dataset: climate-dt
class: d1
generation: "2"
expver: "0001"
type: fc
activity: story-nudging
experiment: Tplus2.0K
realization: "3"
model: IFS-FESOM
DT_CLIMATE_G2_STORY_NUDGING_TPLUS2_0K_IFS_FESOM_R4:
discover_queryables:
constraints_url: "https://s3.central.data.destination-earth.eu/swift/v1/constraints/dedt_mn5/climate-dt_story-nudging_Tplus2.0K_IFS-FESOM_2_4.json"
form_url: "https://s3.central.data.destination-earth.eu/swift/v1/constraints/dedt_mn5/form_climate-dt-phase2.json"
dataset: climate-dt
class: d1
generation: "2"
expver: "0001"
type: fc
activity: story-nudging
experiment: Tplus2.0K
realization: "4"
model: IFS-FESOM
DT_CLIMATE_G2_STORY_NUDGING_TPLUS2_0K_IFS_FESOM_R5:
discover_queryables:
constraints_url: "https://s3.central.data.destination-earth.eu/swift/v1/constraints/dedt_mn5/climate-dt_story-nudging_Tplus2.0K_IFS-FESOM_2_5.json"
form_url: "https://s3.central.data.destination-earth.eu/swift/v1/constraints/dedt_mn5/form_climate-dt-phase2.json"
dataset: climate-dt
class: d1
generation: "2"
expver: "0001"
type: fc
activity: story-nudging
experiment: Tplus2.0K
realization: "5"
model: IFS-FESOM
download: !plugin
type: HTTPDownload
ssl_verify: true
auth_error_code: 401
order_enabled: True
order_method: POST
order_on_response:
metadata_mapping:
eodag:order_id: '{$.headers.Location#slice_str(-36,,1)}'
_previous_order_id: '{$.json.eodag:order_id#replace_str("Not Available","")}'
eodag:combined_order_id: '{eodag:order_id#replace_str("Not Available","")}{_previous_order_id}'
eodag:status_link: "https://polytope.mn5.apps.dte.destination-earth.eu/api/v1/requests/{eodag:combined_order_id#replace_str(r'^$','Not Available')}"
order_status:
request:
method: GET
headers:
metadata_mapping:
eodag:order_status: $.json.status
eodag:order_message: $.json.message
error_message: $.null
success:
http_code: 303
error:
eodag:order_status: failed
on_success:
result_type: json
metadata_mapping:
eodag:download_link: $.headers.Location
no_auth_download: True
auth: !plugin
type: OIDCAuthorizationCodeFlowAuth
oidc_config_url: https://auth.destine.eu/realms/desp/.well-known/openid-configuration
redirect_uri: https://polytope.lumi.apps.dte.destination-earth.eu/
client_id: polytope-api-public
user_consent_needed: false
token_exchange_post_data_method: data
token_provision: header
login_form_xpath: //form[@id='kc-form-login']
authentication_uri_source: login-form
---
!provider # MARK: dedl
name: dedl
priority: 0
roles:
- host
description: DEDL STAC
url: https://hda.data.destination-earth.eu/stac/v2/
# anchors to avoid duplications
anchor_orderable_mm: &orderable_mm
_order_href: '$.links[?rel=="retrieve"].href'
_order_body: '$.links[?rel=="retrieve"].body'
eodag:order_link: "{_order_href}?{{{_order_body#replace_str(\"'\", '\"')}}}"
eodag:download_link: '$.null'
assets: '$.null'
search: !plugin
type: StacSearch
api_endpoint: https://hda.data.destination-earth.eu/stac/v2/search
need_auth: true
timeout: 60
asset_key_from_href: false
metadata_mapping:
eodag:default_geometry: 'POLYGON((180 -90, 180 90, -180 90, -180 -90, 180 -90))'
eodag:quicklook: '{eodag:thumbnail}'
order:status: '{$.properties."order:status"#get_group_name((?P<succeeded>succeeded)|(?P<ordered>shipping)|(?P<orderable>orderable))}'
eodag:download_link: '$.assets.downloadLink.href'
federation:backends: '$.null'
storage:schemes: '$.null'
storage:tier: '$.null'
assets: '$.null'
discover_collections:
fetch_url: 'https://hda.data.destination-earth.eu/stac/v2/collections'
result_type: json
results_entry: '$.collections[*]'
generic_collection_id: '$.id'
generic_collection_parsable_metadata:
description: '$.description'
keywords: '{$.keywords#csv_list}'
license: '$.license'
title: '$.title'
extent: '$.extent'
pagination:
max_limit: 100
next_page_token_key: token
sort:
sort_param_mapping:
id: id
start_datetime: datetime
created: created
updated: updated
platform: platform
gsd: gsd
eo:cloud_cover: eo:cloud_cover
download: !plugin
type: HTTPDownload
auth_error_code: 403
timeout: 20
ssl_verify: true
extract: true
archive_depth: 2
order_enabled: True
order_method: POST
order_on_response:
metadata_mapping:
eodag:status_link: '$.json.links[?rel=="self"].href'
eodag:order_status: $.json.properties."order:status"
order_status:
request:
method: GET
metadata_mapping:
order:status: $.json.properties."order:status"
success:
http_code: 200
on_success:
metadata_mapping:
eodag:download_link: '$.json.assets.downloadLink.href'
order:status: $.json.properties."order:status"
auth: !plugin
type: OIDCTokenExchangeAuth
matching_url: https://[-\w\.]+.data.destination-earth.eu
subject:
oidc_config_url: https://auth.destine.eu/realms/desp/.well-known/openid-configuration
redirect_uri: https://hda.data.destination-earth.eu/stac/v2
client_id: dedl-hda
user_consent_needed: false
exchange_url_error_pattern:
TERMS_AND_CONDITIONS: Terms and conditions are not accepted
token_exchange_post_data_method: data
token_key: access_token
token_provision: header
login_form_xpath: //form[@id='kc-form-login']
authentication_uri_source: login-form
subject_issuer: desp-oidc
token_uri: https://identity.data.destination-earth.eu/auth/realms/dedl/protocol/openid-connect/token
client_id: hda-public
audience: hda-public
token_key: access_token
products:
# Sentinel 1
S1_SAR_GRD:
_collection: EO.ESA.DAT.SENTINEL-1.L1_GRD
S1_SAR_SLC:
_collection: EO.ESA.DAT.SENTINEL-1.L1_SLC
# Sentinel 2
S2_MSI_L1C:
_collection: EO.ESA.DAT.SENTINEL-2.MSI.L1C
S2_MSI_L2A:
_collection: EO.ESA.DAT.SENTINEL-2.MSI.L2A
# Sentinel 3 - S3 OLCI L1
S3_EFR:
_collection: EO.EUM.DAT.SENTINEL-3.OL_1_EFR___
S3_ERR:
_collection: EO.EUM.DAT.SENTINEL-3.OL_1_ERR___
# Sentinel 3 - S3 OLCI L2
S3_OLCI_L2LRR:
_collection: EO.ESA.DAT.SENTINEL-3.OL_2_LRR___
S3_OLCI_L2LFR:
_collection: EO.ESA.DAT.SENTINEL-3.OL_2_LFR___
S3_OLCI_L2WRR:
_collection: EO.EUM.DAT.SENTINEL-3.OL_2_WRR___
S3_OLCI_L2WFR:
_collection: EO.EUM.DAT.SENTINEL-3.OL_2_WFR___
# Sentinel 3 - S3 SLSTR
S3_SLSTR_L1RBT:
_collection: EO.EUM.DAT.SENTINEL-3.SL_1_RBT___
S3_SLSTR_L2LST:
_collection: EO.ESA.DAT.SENTINEL-3.SL_2_LST___
S3_SLSTR_L2WST:
_collection: EO.EUM.DAT.SENTINEL-3.SL_2_WST___
S3_SLSTR_L2AOD:
_collection: EO.EUM.DAT.SENTINEL-3.AOD
S3_SLSTR_L2FRP:
_collection: EO.EUM.DAT.SENTINEL-3.FRP
# Sentinel 3 - S3 SRAL
S3_SRA:
_collection: EO.EUM.DAT.SENTINEL-3.SR_1_SRA___
S3_SRA_A:
_collection: EO.EUM.DAT.SENTINEL-3.SR_1_SRA_A_
S3_SRA_BS:
_collection: EO.EUM.DAT.SENTINEL-3.SR_1_SRA_BS
S3_LAN:
_collection: EO.ESA.DAT.SENTINEL-3.SR_2_LAN___
S3_WAT:
_collection: EO.EUM.DAT.SENTINEL-3.SR_2_WAT___
# S5
S5P_L1B_IR_ALL:
_collection: EO.ESA.DAT.SENTINEL-5P.TROPOMI.L1
S5P_L2_IR_ALL:
_collection: EO.ESA.DAT.SENTINEL-5P.TROPOMI.L2
# ECMWF - COPERNICUS CDS
ERA5_SL:
_collection: EO.ECMWF.DAT.REANALYSIS_ERA5_SINGLE_LEVELS
metadata_mapping:
<<: *orderable_mm
ERA5_SL_MONTHLY:
_collection: EO.ECMWF.DAT.REANALYSIS_ERA5_SINGLE_LEVELS_MONTHLY_MEANS
metadata_mapping:
<<: *orderable_mm
ERA5_PL:
_collection: EO.ECMWF.DAT.ERA5_HOURLY_VARIABLES_ON_PRESSURE_LEVELS
metadata_mapping:
<<: *orderable_mm
ERA5_PL_MONTHLY:
_collection: EO.ECMWF.DAT.ERA5_MONTHLY_MEANS_VARIABLES_ON_PRESSURE_LEVELS
metadata_mapping:
<<: *orderable_mm
ERA5_LAND:
_collection: EO.ECMWF.DAT.ERA5_LAND_HOURLY
metadata_mapping:
<<: *orderable_mm
ERA5_LAND_MONTHLY:
_collection: EO.ECMWF.DAT.ERA5_LAND_MONTHLY
metadata_mapping:
<<: *orderable_mm
UERRA_EUROPE_SL:
_collection: EO.ECMWF.DAT.REANALYSIS_UERRA_EUROPE_SINGLE_LEVELS
metadata_mapping:
<<: *orderable_mm
GRIDDED_GLACIERS_MASS_CHANGE:
_collection: EO.ECMWF.DAT.DERIVED_GRIDDED_GLACIER_MASS_CHANGE
metadata_mapping:
<<: *orderable_mm
GLACIERS_DIST_RANDOLPH:
_collection: EO.ECMWF.DAT.GLACIERS_DISTRIBUTION_DATA_FROM_RANDOLPH_GLACIER_INVENTORY_2000
metadata_mapping:
<<: *orderable_mm
SATELLITE_CARBON_DIOXIDE:
_collection: EO.ECMWF.DAT.CO2_DATA_FROM_SATELLITE_SENSORS_2002_PRESENT
metadata_mapping:
<<: *orderable_mm
SATELLITE_METHANE:
_collection: EO.ECMWF.DAT.METHANE_DATA_SATELLITE_SENSORS_2002_PRESENT
metadata_mapping:
<<: *orderable_mm
SEASONAL_POSTPROCESSED_PL:
_collection: EO.ECMWF.DAT.SEASONAL_FORECAST_ANOMALIES_ON_PRESSURE_LEVELS_2017_PRESENT
metadata_mapping:
<<: *orderable_mm
SATELLITE_SEA_LEVEL_GLOBAL:
_collection: EO.ECMWF.DAT.SEA_LEVEL_DAILY_GRIDDED_DATA_FOR_GLOBAL_OCEAN_1993_PRESENT
metadata_mapping:
<<: *orderable_mm
SATELLITE_SEA_ICE_EDGE_TYPE:
_collection: EO.ECMWF.DAT.SATELLITE_SEA_ICE_EDGE_TYPE
metadata_mapping:
<<: *orderable_mm
SATELLITE_SEA_ICE_THICKNESS:
_collection: EO.ECMWF.DAT.SATELLITE_SEA_ICE_THICKNESS
metadata_mapping:
<<: *orderable_mm
SATELLITE_SEA_ICE_CONCENTRATION:
_collection: EO.ECMWF.DAT.SATELLITE_SEA_ICE_CONCENTRATION
metadata_mapping:
<<: *orderable_mm
SEASONAL_POSTPROCESSED_SL:
_collection: EO.ECMWF.DAT.SEASONAL_FORECAST_ANOMALIES_ON_SINGLE_LEVELS_2017_PRESENT
metadata_mapping:
<<: *orderable_mm
SEASONAL_ORIGINAL_SL:
_collection: EO.ECMWF.DAT.SEASONAL_FORECAST_DAILY_DATA_ON_SINGLE_LEVELS_2017_PRESENT
metadata_mapping:
<<: *orderable_mm
SEASONAL_ORIGINAL_PL:
_collection: EO.ECMWF.DAT.SEASONAL_FORECAST_DAILY_DATA_ON_PRESSURE_LEVELS_2017_PRESENT
metadata_mapping:
<<: *orderable_mm
SEASONAL_MONTHLY_PL:
_collection: EO.ECMWF.DAT.SEASONAL_FORECAST_MONTHLY_STATISTICS_ON_PRESSURE_LEVELS_2017_PRESENT
metadata_mapping:
<<: *orderable_mm
SEASONAL_MONTHLY_SL:
_collection: EO.ECMWF.DAT.SEASONAL_FORECAST_MONTHLY_STATISTICS_ON_SINGLE_LEVELS_2017_PRESENT
metadata_mapping:
<<: *orderable_mm
SIS_HYDRO_MET_PROJ:
_collection: EO.ECMWF.DAT.SIS_HYDROLOGY_METEOROLOGY_DERIVED_PROJECTIONS
metadata_mapping:
<<: *orderable_mm
# ECMWF - CEMS
FIRE_HISTORICAL:
_collection: EO.ECMWF.DAT.CEMS_FIRE_HISTORICAL
GLOFAS_FORECAST:
_collection: EO.ECMWF.DAT.CEMS_GLOFAS_FORECAST
GLOFAS_HISTORICAL:
_collection: EO.ECMWF.DAT.CEMS_GLOFAS_HISTORICAL
GLOFAS_REFORECAST:
_collection: EO.ECMWF.DAT.CEMS_GLOFAS_REFORECAST
GLOFAS_SEASONAL:
_collection: EO.ECMWF.DAT.CEMS_GLOFAS_SEASONAL
GLOFAS_SEASONAL_REFORECAST:
_collection: EO.ECMWF.DAT.CEMS_GLOFAS_SEASONAL_REFORECAST
EFAS_FORECAST:
_collection: EO.ECMWF.DAT.EFAS_FORECAST
EFAS_HISTORICAL:
_collection: EO.ECMWF.DAT.EFAS_HISTORICAL
EFAS_REFORECAST:
_collection: EO.ECMWF.DAT.EFAS_REFORECAST
EFAS_SEASONAL:
_collection: EO.ECMWF.DAT.EFAS_SEASONAL
EFAS_SEASONAL_REFORECAST:
_collection: EO.ECMWF.DAT.EFAS_SEASONAL_REFORECAST
# COPERNICUS ADS
CAMS_GAC_FORECAST:
_collection: EO.ECMWF.DAT.CAMS_GLOBAL_ATMOSHERIC_COMPO_FORECAST
metadata_mapping:
<<: *orderable_mm
CAMS_EU_AIR_QUALITY_FORECAST:
_collection: EO.ECMWF.DAT.CAMS_EUROPE_AIR_QUALITY_FORECASTS
metadata_mapping:
<<: *orderable_mm
CAMS_GFE_GFAS:
_collection: EO.ECMWF.DAT.CAMS_GLOBAL_FIRE_EMISSIONS_GFAS
metadata_mapping:
<<: *orderable_mm
CAMS_SOLAR_RADIATION:
_collection: EO.ECMWF.DAT.CAMS_SOLAR_RADIATION_TIMESERIES
metadata_mapping:
<<: *orderable_mm
CAMS_GREENHOUSE_INVERSION:
_collection: EO.ECMWF.DAT.CAMS_GREENHOUSE_GAS_FLUXES
metadata_mapping:
<<: *orderable_mm
CAMS_EAC4_MONTHLY:
_collection: EO.ECMWF.DAT.CAMS_GLOBAL_REANALYSIS_EAC4_MONTHLY_AV_FIELDS
metadata_mapping:
<<: *orderable_mm
CAMS_EU_AIR_QUALITY_RE:
_collection: EO.ECMWF.DAT.CAMS_EUROPE_AIR_QUALITY_REANALYSES
metadata_mapping:
<<: *orderable_mm
CAMS_EAC4:
_collection: EO.ECMWF.DAT.CAMS_GLOBAL_REANALYSIS_EAC4
metadata_mapping:
<<: *orderable_mm
CAMS_GRF_AUX:
_collection: EO.ECMWF.DAT.CAMS_GLOBAL_RADIATIVE_FORCING_AUX
metadata_mapping:
<<: *orderable_mm
CAMS_GRF:
_collection: EO.ECMWF.DAT.CAMS_GLOBAL_RADIATIVE_FORCING
metadata_mapping:
<<: *orderable_mm
CAMS_GREENHOUSE_EGG4_MONTHLY:
_collection: EO.ECMWF.DAT.CAMS_GLOBAL_GREENHOUSE_GAS_REANALYSIS_MONTHLY_AV_FIELDS
metadata_mapping:
<<: *orderable_mm
CAMS_GREENHOUSE_EGG4:
_collection: EO.ECMWF.DAT.CAMS_GLOBAL_GREENHOUSE_GAS_REANALYSIS
metadata_mapping:
<<: *orderable_mm
CAMS_GLOBAL_EMISSIONS:
_collection: EO.ECMWF.DAT.CAMS_GLOBAL_EMISSION_INVENTORIES
metadata_mapping:
<<: *orderable_mm
# COPERNICUS ADS - Digital Elevation Model
COP_DEM_GLO30_DGED:
_collection: EO.DEM.DAT.COP-DEM_GLO-30-DGED
COP_DEM_GLO30_DTED:
_collection: EO.DEM.DAT.COP-DEM_GLO-30-DTED
COP_DEM_GLO90_DGED:
title: Copernicus DEM GLO-90 DGED
_collection: EO.DEM.DAT.COP-DEM_GLO-90-DGED
COP_DEM_GLO90_DTED:
_collection: EO.DEM.DAT.COP-DEM_GLO-90-DTED
# COPERNICUS Marine
MO_GLOBAL_ANALYSISFORECAST_PHY_001_024:
_collection: EO.MO.DAT.GLOBAL_ANALYSISFORECAST_PHY_001_024
MO_GLOBAL_ANALYSISFORECAST_WAV_001_027:
_collection: EO.MO.DAT.GLOBAL_ANALYSISFORECAST_WAV_001_027
MO_GLOBAL_ANALYSISFORECAST_BGC_001_028:
_collection: EO.MO.DAT.GLOBAL_ANALYSISFORECAST_BGC_001_028
MO_GLOBAL_MULTIYEAR_PHY_ENS_001_031:
_collection: EO.MO.DAT.GLOBAL_MULTIYEAR_PHY_ENS_001_031
MO_GLOBAL_MULTIYEAR_WAV_001_032:
_collection: EO.MO.DAT.GLOBAL_MULTIYEAR_WAV_001_032
MO_GLOBAL_MULTIYEAR_BGC_001_033:
_collection: EO.MO.DAT.GLOBAL_MULTIYEAR_BGC_001_033
MO_INSITU_GLO_PHY_TS_OA_NRT_013_002:
_collection: EO.MO.DAT.INSITU_GLO_PHY_TS_OA_NRT_013_002
MO_INSITU_GLO_PHY_TS_OA_MY_013_052:
_collection: EO.MO.DAT.INSITU_GLO_PHY_TS_OA_MY_013_052
MO_INSITU_GLO_PHY_UV_DISCRETE_NRT_013_048:
_collection: EO.MO.DAT.INSITU_GLO_PHY_UV_DISCRETE_NRT_013_048
MO_MULTIOBS_GLO_BIO_BGC_3D_REP_015_010:
_collection: EO.MO.DAT.MULTIOBS_GLO_BIO_BGC_3D_REP_015_010
MO_MULTIOBS_GLO_BIO_CARBON_SURFACE_MYNRT_015_008:
_collection: EO.MO.DAT.MULTIOBS_GLO_BIO_CARBON_SURFACE_MYNRT_015_008
MO_MULTIOBS_GLO_BGC_NUTRIENTS_CARBON_PROFILES_MYNRT_015_009:
_collection: EO.MO.DAT.MULTIOBS_GLO_BGC_NUTRIENTS_CARBON_PROFILES_MYNRT_015_009
MO_MULTIOBS_GLO_PHY_MYNRT_015_003:
_collection: EO.MO.DAT.MULTIOBS_GLO_PHY_MYNRT_015_003
MO_MULTIOBS_GLO_PHY_S_SURFACE_MYNRT_015_013:
_collection: EO.MO.DAT.MULTIOBS_GLO_PHY_S_SURFACE_MYNRT_015_013
MO_MULTIOBS_GLO_PHY_TSUV_3D_MYNRT_015_012:
_collection: EO.MO.DAT.MULTIOBS_GLO_PHY_TSUV_3D_MYNRT_015_012
MO_MULTIOBS_GLO_PHY_UVW_3D_MYNRT_015_007:
_collection: EO.MO.DAT.MULTIOBS_GLO_PHY_UVW_3D_MYNRT_015_007
MO_OCEANCOLOUR_GLO_BGC_L3_MY_009_103:
_collection: EO.MO.DAT.OCEANCOLOUR_GLO_BGC_L3_MY_009_103
MO_OCEANCOLOUR_GLO_BGC_L3_MY_009_107:
_collection: EO.MO.DAT.OCEANCOLOUR_GLO_BGC_L3_MY_009_107
MO_OCEANCOLOUR_GLO_BGC_L3_NRT_009_101:
_collection: EO.MO.DAT.OCEANCOLOUR_GLO_BGC_L3_NRT_009_101
MO_OCEANCOLOUR_GLO_BGC_L4_MY_009_104:
_collection: EO.MO.DAT.OCEANCOLOUR_GLO_BGC_L4_MY_009_104
MO_OCEANCOLOUR_GLO_BGC_L4_MY_009_108:
_collection: EO.MO.DAT.OCEANCOLOUR_GLO_BGC_L4_MY_009_108
MO_OCEANCOLOUR_GLO_BGC_L4_NRT_009_102:
_collection: EO.MO.DAT.OCEANCOLOUR_GLO_BGC_L4_NRT_009_102
MO_SEAICE_GLO_SEAICE_L4_NRT_OBSERVATIONS_011_001:
_collection: EO.MO.DAT.SEAICE_GLO_SEAICE_L4_NRT_OBSERVATIONS_011_001
MO_SEAICE_GLO_SEAICE_L4_NRT_OBSERVATIONS_011_006:
_collection: EO.MO.DAT.SEAICE_GLO_SEAICE_L4_NRT_OBSERVATIONS_011_006
MO_SEAICE_GLO_SEAICE_L4_REP_OBSERVATIONS_011_009:
_collection: EO.MO.DAT.SEAICE_GLO_SEAICE_L4_REP_OBSERVATIONS_011_009
MO_SEALEVEL_GLO_PHY_L4_NRT_008_046:
_collection: EO.MO.DAT.SEALEVEL_GLO_PHY_L4_NRT_008_046
MO_SEALEVEL_GLO_PHY_MDT_008_063:
_collection: EO.MO.DAT.SEALEVEL_GLO_PHY_MDT_008_063
MO_SST_GLO_SST_L3S_NRT_OBSERVATIONS_010_010:
_collection: EO.MO.DAT.SST_GLO_SST_L3S_NRT_OBSERVATIONS_010_010
MO_SST_GLO_SST_L4_NRT_OBSERVATIONS_010_001:
_collection: EO.MO.DAT.SST_GLO_SST_L4_NRT_OBSERVATIONS_010_001
MO_SST_GLO_SST_L4_REP_OBSERVATIONS_010_011:
_collection: EO.MO.DAT.SST_GLO_SST_L4_REP_OBSERVATIONS_010_011
MO_SST_GLO_SST_L4_REP_OBSERVATIONS_010_024:
_collection: EO.MO.DAT.SST_GLO_SST_L4_REP_OBSERVATIONS_010_024
MO_WIND_GLO_PHY_CLIMATE_L4_MY_012_003:
_collection: EO.MO.DAT.WIND_GLO_PHY_CLIMATE_L4_MY_012_003
MO_WIND_GLO_PHY_L3_MY_012_005:
_collection: EO.MO.DAT.WIND_GLO_PHY_L3_MY_012_005
MO_WIND_GLO_PHY_L3_NRT_012_002:
_collection: EO.MO.DAT.WIND_GLO_PHY_L3_NRT_012_002
MO_WIND_GLO_PHY_L4_MY_012_006:
_collection: EO.MO.DAT.WIND_GLO_PHY_L4_MY_012_006
MO_WIND_GLO_PHY_L4_NRT_012_004:
_collection: EO.MO.DAT.WIND_GLO_PHY_L4_NRT_012_004
MO_WAVE_GLO_PHY_SWH_L3_NRT_014_001:
_collection: EO.MO.DAT.WAVE_GLO_PHY_SWH_L3_NRT_014_001
MO_WAVE_GLO_PHY_SWH_L4_NRT_014_003:
_collection: EO.MO.DAT.WAVE_GLO_PHY_SWH_L4_NRT_014_003
MO_WAVE_GLO_PHY_SPC_FWK_L3_NRT_014_002:
_collection: EO.MO.DAT.WAVE_GLO_PHY_SPC_FWK_L3_NRT_014_002
# CLMS
CLMS_CORINE:
_collection: EO.CLMS.DAT.CORINE
CLMS_GLO_DMP_333M:
_collection: EO.CLMS.DAT.GLO.DMP300_V1
CLMS_GLO_FAPAR_333M:
_collection: EO.CLMS.DAT.GLO.FAPAR300_V1
CLMS_GLO_FCOVER_333M:
_collection: EO.CLMS.DAT.GLO.FCOVER300_V1
CLMS_GLO_GDMP_333M:
_collection: EO.CLMS.DAT.GLO.GDMP300_V1
CLMS_GLO_LAI_333M:
_collection: EO.CLMS.DAT.GLO.LAI300_V1
CLMS_GLO_NDVI_1KM_LTS:
_collection: EO.CLMS.DAT.GLO.NDVI_1KM_V2
CLMS_GLO_NDVI_333M:
_collection: EO.CLMS.DAT.GLO.NDVI300_V1
# Landsat data
LANDSAT_C2L1:
_collection: EO.NASA.DAT.LANDSAT.C2_L1
LANDSAT_C2L2:
_collection: EO.NASA.DAT.LANDSAT.C2_L2
# DT Output
DT_EXTREMES:
_collection: EO.ECMWF.DAT.DT_EXTREMES
metadata_mapping:
order:status: '{$.null#replace_str("Not Available","orderable")}'
<<: *orderable_mm
DT_CLIMATE_ADAPTATION:
_collection: EO.ECMWF.DAT.DT_CLIMATE_ADAPTATION
metadata_mapping:
order:status: '{$.null#replace_str("Not Available","orderable")}'
<<: *orderable_mm
# AERIS
AERIS_IAGOS:
_collection: EO.AERIS.DAT.IAGOS
# METOP
METOP_ASCSZFR02:
_collection: EO.EUM.CM.METOP.ASCSZFR02
METOP_ASCSZOR02:
_collection: EO.EUM.CM.METOP.ASCSZOR02
METOP_ASCSZRR02:
_collection: EO.EUM.CM.METOP.ASCSZRR02
METOP_AMSU_L1:
_collection: EO.EUM.DAT.METOP.AMSUL1
METOP_ASCSZF1B:
_collection: EO.EUM.DAT.METOP.ASCSZF1B
METOP_ASCSZR1B:
_collection: EO.EUM.DAT.METOP.ASCSZR1B
METOP_ASCSZO1B:
_collection: EO.EUM.DAT.METOP.ASCSZO1B
METOP_AVHRRGACR02:
_collection: EO.EUM.DAT.METOP.AVHRRGACR02
METOP_AVHRRL1:
_collection: EO.EUM.DAT.METOP.AVHRRL1
METOP_GLB_SST_NC:
_collection: EO.EUM.DAT.METOP.GLB-SST-NC
METOP_GOMEL1:
_collection: EO.EUM.DAT.METOP.GOMEL1
METOP_GOMEL1R03:
_collection: EO.EUM.DAT.METOP.GOMEL1R03
METOP_HIRSL1:
_collection: EO.EUM.DAT.MULT.HIRSL1
METOP_IASIL1C_ALL:
_collection: EO.EUM.DAT.METOP.IASIL1C-ALL
METOP_IASSND02:
_collection: EO.EUM.DAT.METOP.IASSND02
METOP_IASTHR011:
_collection: EO.EUM.DAT.METOP.IASTHR011
METOP_LSA_002:
_collection: EO.EUM.DAT.METOP.LSA-002
METOP_MHSL1:
_collection: EO.EUM.DAT.METOP.MHSL1
METOP_OSI_104:
_collection: EO.EUM.DAT.METOP.OSI-104
METOP_OSI_150A:
_collection: EO.EUM.DAT.METOP.OSI-150-A
METOP_OSI_150B:
_collection: EO.EUM.DAT.METOP.OSI-150-B
METOP_SOMO12:
_collection: EO.EUM.DAT.METOP.SOMO12
METOP_SOMO25:
_collection: EO.EUM.DAT.METOP.SOMO25
# MSG
MSG_AMVR02:
_collection: EO.EUM.DAT.AMVR02
MSG_GSAL2R02:
_collection: EO.EUM.DAT.GSAL2R02
MSG_CLM:
_collection: EO.EUM.DAT.MSG.CLM
MSG_CLM_IODC:
_collection: EO.EUM.DAT.MSG.CLM-IODC
MSG_HRSEVIRI:
_collection: EO.EUM.DAT.MSG.HRSEVIRI
MSG_HRSEVIRI_IODC:
_collection: EO.EUM.DAT.MSG.HRSEVIRI-IODC
MSG_LSA_FRM:
_collection: EO.EUM.DAT.MSG.LSA-FRM
MSG_LSA_LST_CDR:
_collection: EO.EUM.DAT.MSG.LSA-LST-CDR
MSG_LSA_LSTDE:
_collection: EO.EUM.DAT.MSG.LSA-LSTDE
MSG_RSS_CLM:
_collection: EO.EUM.DAT.MSG.RSS-CLM
MSG_MSG15_RSS:
_collection: EO.EUM.DAT.MSG.MSG15-RSS
# GSW
GSW_CHANGE:
_collection: EO.GSW.DAT.CHANGE
GSW_EXTENT:
_collection: EO.GSW.DAT.EXTENT
GSW_OCCURRENCE:
_collection: EO.GSW.DAT.OCCURRENCE
GSW_RECURRENCE:
_collection: EO.GSW.DAT.RECURRENCE
GSW_SEASONALITY:
_collection: EO.GSW.DAT.SEASONALITY
GSW_TRANSITIONS:
_collection: EO.GSW.DAT.TRANSITIONS
# Eurostat
EUSTAT_GREENHOUSE_GAS_EMISSION_AGRICULTURE:
_collection: STAT.EUSTAT.DAT.GREENHOUSE_GAS_EMISSION_AGRICULTURE
EUSTAT_POP_AGE_GROUP_SEX_NUTS3:
_collection: STAT.EUSTAT.DAT.POP_AGE_GROUP_SEX_NUTS3
EUSTAT_POP_AGE_SEX_NUTS2:
_collection: STAT.EUSTAT.DAT.POP_AGE_SEX_NUTS2
EUSTAT_POP_CHANGE_DEMO_BALANCE_CRUDE_RATES_NUTS3:
_collection: STAT.EUSTAT.DAT.POP_CHANGE_DEMO_BALANCE_CRUDE_RATES_NUTS3
EUSTAT_SHARE_ENERGY_FROM_RENEWABLE:
_collection: STAT.EUSTAT.DAT.SHARE_ENERGY_FROM_RENEWABLE
EUSTAT_AVAILABLE_BEDS_HOSPITALS_NUTS2:
_collection: STAT.EUSTAT.DAT.AVAILABLE_BEDS_HOSPITALS_NUTS2
EUSTAT_BATHING_SITES_WATER_QUALITY:
_collection: STAT.EUSTAT.DAT.BATHING_SITES_WATER_QUALITY
EUSTAT_SOIL_SEALING_INDEX:
_collection: STAT.EUSTAT.DAT.SOIL_SEALING_INDEX
EUSTAT_SURFACE_TERRESTRIAL_PROTECTED_AREAS:
_collection: STAT.EUSTAT.DAT.SURFACE_TERRESTRIAL_PROTECTED_AREAS
EUSTAT_POP_DENSITY_NUTS3:
_collection: STAT.EUSTAT.DAT.POP_DENSITY_NUTS3
# ISIMIP
ISIMIP_CLIMATE_FORCING_ISIMIP3B:
_collection: EO.ISIMIP.DAT.CLIMATE-FORCING_ISIMIP3b
ISIMIP_SOCIO_ECONOMIC_FORCING_ISIMIP3B:
_collection: EO.ISIMIP.DAT.SOCIO-ECONOMIC-FORCING_ISIMIP3b
GENERIC_COLLECTION:
_collection: '{collection}'
# Spacenet
SPACENET_BUILDINGS_DETECTION_V1:
_collection: EO.SPACENET.DAT.BUILDINGS_DETECTION_V1
metadata_mapping:
label:properties:
- '{{"query":{{"label:properties":{{"eq":{label:properties}}}}}}}'
- '{$.properties.label:properties#replace_str("Not Available","None")}'
SPACENET_BUILDINGS_DETECTION_V2:
_collection: EO.SPACENET.DAT.BUILDINGS_DETECTION_V2
metadata_mapping_from_product: SPACENET_BUILDINGS_DETECTION_V1
SPACENET_ROADS_NETWORK_DETECTION:
_collection: EO.SPACENET.DAT.ROADS_NETWORK_DETECTION
metadata_mapping_from_product: SPACENET_BUILDINGS_DETECTION_V1
SPACENET_OFF_NADIR_BUILDING:
_collection: EO.SPACENET.DAT.OFF_NADIR_BUILDING
metadata_mapping_from_product: SPACENET_BUILDINGS_DETECTION_V1
SPACENET_ROADS_NETWORK_ROUTE_TRAVEL:
_collection: EO.SPACENET.DAT.ROADS_NETWORK_ROUTE_TRAVEL
metadata_mapping_from_product: SPACENET_BUILDINGS_DETECTION_V1
SPACENET_ALL_WEATHER_MAPPING:
_collection: EO.SPACENET.DAT.ALL_WEATHER_MAPPING
metadata_mapping_from_product: SPACENET_BUILDINGS_DETECTION_V1
---
!provider # MARK: eumetsat_ds
name: eumetsat_ds
priority: 0
description: EUMETSAT Data Store
roles:
- host
url: https://data.eumetsat.int
anchor_sentinel: &sentinel_params
sat:orbit_state: '{$.properties.acquisitionInformation[0].acquisitionParameters.orbitDirection#to_lower}'
sat:relative_orbit: '$.properties.acquisitionInformation[0].acquisitionParameters.relativeOrbitNumber'
product:timeliness: '$.properties.productInformation.timeliness'
sat:orbit_cycle: '$.properties.acquisitionInformation[0].acquisitionParameters.cycleNumber'
anchor_orbit_zone_tile: &orbit_zone_tile
sat:absolute_orbit:
- orbit
- '$.properties.acquisitionInformation[0].acquisitionParameters.orbitNumber'
search: !plugin
type: EumetsatDsSearch
api_endpoint: 'https://api.eumetsat.int/data/search-products/1.0.0/os'
need_auth: false
ssl_verify: true
dont_quote:
- '='
- '&'
asset_key_from_href: false
pagination:
next_page_url_tpl: '{url}?{search}&c={limit}&pw={next_page_token}'
start_page: 0
total_items_nb_key_path: '$.totalResults'
# 2024/02/01: 500 is the max, no error if greater
max_limit: 500
sort:
sort_by_default:
- !!python/tuple [start_datetime, ASC]
sort_by_tpl: '&sort={sort_param},{sort_order}'
sort_param_mapping:
start_datetime: start,time
published: publicationDate,
sort_order_mapping:
ascending: '1'
descending: '0'
max_sort_params: 1
literal_search_params:
format: json
discover_metadata:
auto_discovery: true
metadata_pattern: '^[a-zA-Z0-9_]+$'
search_param: '{metadata}={{{metadata}}}'
metadata_path: '$.properties.*'
discover_collections:
fetch_url: https://api.eumetsat.int/data/browse/1.0.0/collections?format=json
result_type: json
results_entry: '$.links[*]'
generic_collection_id: '$.title'
generic_collection_parsable_properties:
_collection: '$.title'
generic_collection_parsable_metadata:
description: '$.null'
instruments: '$.null'
constellation: '$.null'
platform: '$.null'
processing:level: '$.null'
keywords: '$.null'
license: '$.null'
title: '$.title'
extent: '$.null'
metadata_mapping:
# Opensearch resource identifier within the search engine context (in our case
# within the context of the data provider)
uid: '$.id'
# OpenSearch Parameters for Collection Search (Table 3)
product:type:
- type
- '$.properties.productInformation.productType'
platform: '$.properties.acquisitionInformation[0].platform.platformShortName'
instruments: '{$.properties.acquisitionInformation[0].instrument.instrumentShortName#split( )}'
# INSPIRE obligated OpenSearch Parameters for Collection Search (Table 4)
title: '{$.properties.title#sanitize}'
# OpenSearch Parameters for Product Search (Table 5)
_collection:
- pi
- '$.properties.parentIdentifier'
updated: '$.properties.updated'
# OpenSearch Parameters for Acquistion Parameters Search (Table 6)
start_datetime:
- dtstart
- '{$.properties.date#replace_str(r"\/.*","")}'
end_datetime:
- dtend
- '{$.properties.date#replace_str(r".*\/","")}'
# Custom parameters (not defined in the base document referenced above)
id:
- id
- $.properties.identifier
# The geographic extent of the product
geometry:
- 'geo={geometry#to_rounded_wkt}'
- '$.geometry'
eodag:default_geometry: 'POLYGON((180 -90, 180 90, -180 90, -180 -90, 180 -90))'
# The url of the quicklook
eodag:quicklook: '$.properties.links.previews[?(@.title="Quicklook")].href'
# The url to download the product "as is" (literal or as a template to be completed either after the search result
# is obtained from the provider or during the eodag download phase)
eodag:download_link: '$.properties.links.data[?(@.title="Product download")].href'
# order:status set to succeeded for consistency between providers
order:status: '{$.null#replace_str("Not Available","succeeded")}'
assets: '{$.properties.links.sip-entries#assets_list_to_dict}'
# Additional metadata provided by the providers but that don't appear in the reference spec
size: '$.properties.productInformation.size'
type: '$.null'
# set duplicate metadata due to metadata discovery to null
acquisitionInformation: '$.null'
productInformation: '$.null'
extraInformation: '$.null'
products:
# S3 SRAL
S3_SRA:
product:type: SR_1_SRA___
_collection: EO:EUM:DAT:0406
metadata_mapping:
<<: *orbit_zone_tile
<<: *sentinel_params
S3_SRA_A:
product:type: SR_1_SRA_A_
_collection: EO:EUM:DAT:0413
metadata_mapping:
<<: *orbit_zone_tile
<<: *sentinel_params
S3_SRA_1A_BC004:
product:type: SR_1_SRA_A_
_collection: EO:EUM:DAT:0583
metadata_mapping:
<<: *orbit_zone_tile
<<: *sentinel_params
S3_SRA_1A_BC005:
product:type: SR_1_SRA_A_
_collection: EO:EUM:DAT:0836
metadata_mapping:
<<: *orbit_zone_tile
<<: *sentinel_params
S3_SRA_1B_BC004:
product:type: SR_1_SRA___
_collection: EO:EUM:DAT:0584
metadata_mapping:
<<: *orbit_zone_tile
<<: *sentinel_params
S3_SRA_1B_BC005:
product:type: SR_1_SRA___
_collection: EO:EUM:DAT:0833
metadata_mapping:
<<: *orbit_zone_tile
<<: *sentinel_params
S3_SRA_BS:
product:type: SR_1_SRA_BS
_collection: EO:EUM:DAT:0414
metadata_mapping:
<<: *orbit_zone_tile
<<: *sentinel_params
S3_SRA_BS_BC004:
product:type: SR_1_SRA_BS
_collection: EO:EUM:DAT:0585
metadata_mapping:
<<: *orbit_zone_tile
<<: *sentinel_params
S3_SRA_BS_BC005:
product:type: SR_1_SRA_BS
_collection: EO:EUM:DAT:0835
metadata_mapping:
<<: *orbit_zone_tile
<<: *sentinel_params
S3_WAT:
product:type: SR_2_WAT___
_collection: EO:EUM:DAT:0415
metadata_mapping:
<<: *orbit_zone_tile
<<: *sentinel_params
S3_WAT_BC004:
product:type: SR_2_WAT___
_collection: EO:EUM:DAT:0586
metadata_mapping:
<<: *orbit_zone_tile
<<: *sentinel_params
S3_WAT_BC005:
product:type: SR_2_WAT___
_collection: EO:EUM:DAT:0834
metadata_mapping:
<<: *orbit_zone_tile
<<: *sentinel_params
# S3 OLCI
S3_EFR:
product:type: OL_1_EFR___
_collection: EO:EUM:DAT:0409
metadata_mapping:
<<: *orbit_zone_tile
<<: *sentinel_params
S3_EFR_BC002:
product:type: OL_1_EFR___
_collection: EO:EUM:DAT:0577
metadata_mapping:
<<: *orbit_zone_tile
<<: *sentinel_params
S3_ERR:
product:type: OL_1_ERR___
_collection: EO:EUM:DAT:0410
metadata_mapping:
<<: *orbit_zone_tile
<<: *sentinel_params
S3_ERR_BC002:
product:type: OL_1_ERR___
_collection: EO:EUM:DAT:0578
metadata_mapping:
<<: *orbit_zone_tile
<<: *sentinel_params
S3_OLCI_L2WRR:
product:type: OL_2_WRR___
_collection: EO:EUM:DAT:0408
metadata_mapping:
<<: *orbit_zone_tile
<<: *sentinel_params
S3_OLCI_L2WRR_BC003:
product:type: OL_2_WRR___
_collection: EO:EUM:DAT:0557
metadata_mapping:
<<: *orbit_zone_tile
<<: *sentinel_params
S3_OLCI_L2WFR:
product:type: OL_2_WFR___
_collection: EO:EUM:DAT:0407
metadata_mapping:
<<: *orbit_zone_tile
<<: *sentinel_params
S3_OLCI_L2WFR_BC003:
product:type: OL_2_WFR___
_collection: EO:EUM:DAT:0556
metadata_mapping:
<<: *orbit_zone_tile
<<: *sentinel_params
S3_OL_2_WFRBC003:
product:type: OL_2_WFR___
_collection: EO:EUM:DAT:0556
metadata_mapping:
<<: *orbit_zone_tile
<<: *sentinel_params
platform:
- sat
- '$.properties.acquisitionInformation[0].platform.platformShortName'
# S3 SLSTR
S3_SLSTR_L1RBT:
product:type: SL_1_RBT___
_collection: EO:EUM:DAT:0411
metadata_mapping:
<<: *orbit_zone_tile
<<: *sentinel_params
S3_SLSTR_L1RBT_BC003:
product:type: SL_1_RBT___
_collection: EO:EUM:DAT:0581
metadata_mapping:
<<: *orbit_zone_tile
<<: *sentinel_params
S3_SLSTR_L1RBT_BC004:
product:type: SL_1_RBT___
_collection: EO:EUM:DAT:0615
metadata_mapping:
<<: *orbit_zone_tile
<<: *sentinel_params
S3_SLSTR_L2WST:
product:type: SL_2_WST___
_collection: EO:EUM:DAT:0412
metadata_mapping:
<<: *orbit_zone_tile
<<: *sentinel_params
S3_SLSTR_L2WST_BC003:
product:type: SL_2_WST___
_collection: EO:EUM:DAT:0582
metadata_mapping:
<<: *orbit_zone_tile
<<: *sentinel_params
S3_SLSTR_L2AOD:
product:type: SL_2_AOD___
_collection: EO:EUM:DAT:0416
metadata_mapping:
<<: *orbit_zone_tile
<<: *sentinel_params
S3_SLSTR_L2FRP:
product:type: SL_2_FRP___
_collection: EO:EUM:DAT:0417
metadata_mapping:
<<: *orbit_zone_tile
<<: *sentinel_params
fire:
- fire
- '$.properties.extraInformation.fireDetected'
# METOP
METOP_AMSU_L1:
_collection: EO:EUM:DAT:METOP:AMSUL1
metadata_mapping:
<<: *orbit_zone_tile
METOP_OSI_104:
_collection: EO:EUM:DAT:METOP:OSI-104
metadata_mapping:
<<: *orbit_zone_tile
METOP_OSI_150A:
_collection: EO:EUM:DAT:METOP:OSI-150-A
metadata_mapping:
<<: *orbit_zone_tile
METOP_OSI_150B:
_collection: EO:EUM:DAT:METOP:OSI-150-B
metadata_mapping:
<<: *orbit_zone_tile
METOP_ASCSZF1B:
_collection: EO:EUM:DAT:METOP:ASCSZF1B
metadata_mapping:
<<: *orbit_zone_tile
METOP_ASCSZR1B:
_collection: EO:EUM:DAT:METOP:ASCSZR1B
metadata_mapping:
<<: *orbit_zone_tile
METOP_ASCSZO1B:
_collection: EO:EUM:DAT:METOP:ASCSZO1B
metadata_mapping:
<<: *orbit_zone_tile
METOP_ASCSZFR02:
_collection: EO:EUM:CM:METOP:ASCSZFR02
metadata_mapping:
<<: *orbit_zone_tile
METOP_ASCSZOR02:
_collection: EO:EUM:CM:METOP:ASCSZOR02
metadata_mapping:
<<: *orbit_zone_tile
METOP_ASCSZRR02:
_collection: EO:EUM:CM:METOP:ASCSZRR02
metadata_mapping:
<<: *orbit_zone_tile
METOP_AVHRRL1:
_collection: EO:EUM:DAT:METOP:AVHRRL1
metadata_mapping:
<<: *orbit_zone_tile
METOP_SOMO12:
_collection: EO:EUM:DAT:METOP:SOMO12
metadata_mapping:
<<: *orbit_zone_tile
METOP_SOMO25:
_collection: EO:EUM:DAT:METOP:SOMO25
metadata_mapping:
<<: *orbit_zone_tile
METOP_AVHRRGACR02:
_collection: EO:EUM:DAT:0558
METOP_LSA_002:
_collection: EO:EUM:DAT:METOP:LSA-002
METOP_GLB_SST_NC:
_collection: EO:EUM:DAT:METOP:GLB-SST-NC
METOP_GOMEL1:
_collection: EO:EUM:DAT:METOP:GOMEL1
metadata_mapping:
<<: *orbit_zone_tile
METOP_GOMEL1R03:
_collection: EO:EUM:DAT:0533
metadata_mapping:
<<: *orbit_zone_tile
METOP_IASTHR011:
_collection: EO:EUM:DAT:0576
metadata_mapping:
<<: *orbit_zone_tile
METOP_IASSND02:
_collection: EO:EUM:DAT:METOP:IASSND02
metadata_mapping:
<<: *orbit_zone_tile
METOP_IASIL1C_ALL:
_collection: EO:EUM:DAT:METOP:IASIL1C-ALL
metadata_mapping:
<<: *orbit_zone_tile
METOP_MHSL1:
_collection: EO:EUM:DAT:METOP:MHSL1
metadata_mapping:
<<: *orbit_zone_tile
METOP_HIRSL1:
_collection: EO:EUM:DAT:MULT:HIRSL1
metadata_mapping:
<<: *orbit_zone_tile
# MSG
MSG_CLM:
_collection: EO:EUM:DAT:MSG:CLM
MSG_CLM_IODC:
_collection: EO:EUM:DAT:MSG:CLM-IODC
MSG_GSAL2R02:
_collection: EO:EUM:DAT:0300
MSG_HRSEVIRI:
_collection: EO:EUM:DAT:MSG:HRSEVIRI
MSG_HRSEVIRI_IODC:
_collection: EO:EUM:DAT:MSG:HRSEVIRI-IODC
MSG_RSS_CLM:
_collection: EO:EUM:DAT:MSG:RSS-CLM
MSG_MSG15_RSS:
_collection: EO:EUM:DAT:MSG:MSG15-RSS
MSG_LSA_FRM:
_collection: EO:EUM:DAT:0398
MSG_LSA_LST_CDR:
_collection: EO:EUM:DAT:0088
MSG_LSA_LSTDE:
_collection: EO:EUM:DAT:0394
MSG_AMVR02:
_collection: EO:EUM:DAT:0405
MFG_GSA_57:
_collection: EO:EUM:DAT:0301
MFG_GSA_63:
_collection: EO:EUM:DAT:0302
MSG_MFG_GSA_0:
_collection: EO:EUM:DAT:0300
MSG_CTH:
_collection: EO:EUM:DAT:MSG:CTH
MSG_CTH_IODC:
_collection: EO:EUM:DAT:MSG:CTH-IODC
HIRS_FDR_1_MULTI:
_collection: EO:EUM:DAT:0961
MSG_OCA_CDR:
_collection: EO:EUM:DAT:0617
S6_RADIO_OCCULTATION:
_collection: EO:EUM:DAT:0853
MTG_LI_AF:
_collection: EO:EUM:DAT:0686
MTG_LI_LFL:
_collection: EO:EUM:DAT:0691
MTG_LI_LGR:
_collection: EO:EUM:DAT:0782
MTG_LI_AFA:
_collection: EO:EUM:DAT:0687
MTG_LI_AFR:
_collection: EO:EUM:DAT:0688
MTG_LI_LEF:
_collection: EO:EUM:DAT:0690
MTG_FCI_FDHSI:
_collection: EO:EUM:DAT:0662
MTG_FCI_HRFI:
_collection: EO:EUM:DAT:0665
MTG_FCI_ASR_BUFR:
_collection: EO:EUM:DAT:0799
MTG_FCI_ASR_NETCDF:
_collection: EO:EUM:DAT:0677
MTG_FCI_AMV_BUFR:
_collection: EO:EUM:DAT:0998
MTG_FCI_AMV_NETCDF:
_collection: EO:EUM:DAT:0676
MTG_FCI_CLM:
_collection: EO:EUM:DAT:0678
MTG_FCI_GII:
_collection: EO:EUM:DAT:0683
MTG_FCI_OCA:
_collection: EO:EUM:DAT:0684
MTG_FCI_OLR:
_collection: EO:EUM:DAT:0685
MSG_SEVIRI_RSS_AMV_CDR_V1:
_collection: EO:EUM:DAT:1083
MSG_SEVIRI_RSS_HR_IMG_L1_5_V1:
_collection: EO:EUM:DAT:0962
MSG_SEVIRI_SARAH_CDR_V003:
_collection: EO:EUM:DAT:0863
MTG_FCI_ACTIVE_FIRE_L2_V1:
_collection: EO:EUM:DAT:0682
MULT_PMW_IR_GIRAFE_PRECIP_CDR_V001:
_collection: EO:EUM:DAT:0921
GENERIC_COLLECTION:
_collection: '{collection}'
download: !plugin
type: HTTPDownload
extract: true
ignore_assets: True
ssl_verify: true
auth: !plugin
type: TokenAuth
matching_url: https://api.eumetsat.int
auth_uri: 'https://api.eumetsat.int/token'
auth_tuple: [username, password]
auth_error_code: 401
req_data:
grant_type: client_credentials
token_type: json
token_key: access_token
token_expiration_key: expires_in
ssl_verify: true
---
!provider # MARK: cop_marine
name: cop_marine
priority: 0
description: Copernicus Marine Data Store
roles:
- host
url: https://marine.copernicus.eu/
search: !plugin
type: CopMarineSearch
api_endpoint: 'https://stac.marine.copernicus.eu/metadata/{_collection}/product.stac.json'
need_auth: false
ssl_verify: true
asset_key_from_href: false
timeout: 20
results_entry: links
discover_collections:
fetch_url: 'https://stac.marine.copernicus.eu/metadata/catalog.stac.json'
result_type: json
results_entry: '$.collections[*]'
generic_collection_id: '$.id'
generic_collection_parsable_properties:
collection: '$.id'
generic_collection_parsable_metadata:
instruments: '$.null'
constellation: '$.null'
platform: '$.null'
processing:level: '$.properties.processingLevel'
keywords: '$.keywords'
license: '$.license'
title: '$.title'
extent: '$.extent'
description: '$.description'
providers: '$.providers'
sci:doi: '$.sci:doi'
discover_metadata:
auto_discovery: false
discover_queryables:
fetch_url: null
metadata_mapping:
eodag:default_geometry: 'POLYGON((180 -90, 180 90, -180 90, -180 -90, 180 -90))'
products:
MO_GLOBAL_ANALYSISFORECAST_PHY_001_024:
_collection: GLOBAL_ANALYSISFORECAST_PHY_001_024
MO_GLOBAL_ANALYSISFORECAST_BGC_001_028:
_collection: GLOBAL_ANALYSISFORECAST_BGC_001_028
MO_GLOBAL_ANALYSISFORECAST_WAV_001_027:
_collection: GLOBAL_ANALYSISFORECAST_WAV_001_027
MO_GLOBAL_MULTIYEAR_BGC_001_033:
_collection: GLOBAL_MULTIYEAR_BGC_001_033
MO_GLOBAL_MULTIYEAR_WAV_001_032:
_collection: GLOBAL_MULTIYEAR_WAV_001_032
MO_GLOBAL_MULTIYEAR_PHY_ENS_001_031:
_collection: GLOBAL_MULTIYEAR_PHY_ENS_001_031
MO_INSITU_GLO_PHY_UV_DISCRETE_NRT_013_048:
_collection: INSITU_GLO_PHY_UV_DISCRETE_NRT_013_048
code_mapping:
param: platform
index: 3
MO_INSITU_GLO_PHY_TS_OA_NRT_013_002:
_collection: INSITU_GLO_PHY_TS_OA_NRT_013_002
code_mapping:
param: cop_marine:analysis_name
index: 1
MO_INSITU_GLO_PHY_TS_OA_MY_013_052:
_collection: INSITU_GLO_PHY_TS_OA_MY_013_052
code_mapping:
param: cop_marine:analysis_name
index: 1
MO_MULTIOBS_GLO_BIO_BGC_3D_REP_015_010:
_collection: MULTIOBS_GLO_BIO_BGC_3D_REP_015_010
MO_MULTIOBS_GLO_BIO_CARBON_SURFACE_MYNRT_015_008:
_collection: MULTIOBS_GLO_BIO_CARBON_SURFACE_MYNRT_015_008
MO_MULTIOBS_GLO_BGC_NUTRIENTS_CARBON_PROFILES_MYNRT_015_009:
_collection: MULTIOBS_GLO_BGC_NUTRIENTS_CARBON_PROFILES_MYNRT_015_009
MO_MULTIOBS_GLO_PHY_MYNRT_015_003:
_collection: MULTIOBS_GLO_PHY_MYNRT_015_003
MO_MULTIOBS_GLO_PHY_S_SURFACE_MYNRT_015_013:
_collection: MULTIOBS_GLO_PHY_S_SURFACE_MYNRT_015_013
MO_MULTIOBS_GLO_PHY_TSUV_3D_MYNRT_015_012:
_collection: MULTIOBS_GLO_PHY_TSUV_3D_MYNRT_015_012
MO_MULTIOBS_GLO_PHY_UVW_3D_MYNRT_015_007:
_collection: MULTIOBS_GLO_PHY_UVW_3D_MYNRT_015_007
MO_SEAICE_GLO_SEAICE_L4_NRT_OBSERVATIONS_011_001:
_collection: SEAICE_GLO_SEAICE_L4_NRT_OBSERVATIONS_011_001
MO_SEAICE_GLO_SEAICE_L4_REP_OBSERVATIONS_011_009:
_collection: SEAICE_GLO_SEAICE_L4_REP_OBSERVATIONS_011_009
MO_SEAICE_GLO_SEAICE_L4_NRT_OBSERVATIONS_011_006:
_collection: SEAICE_GLO_SEAICE_L4_NRT_OBSERVATIONS_011_006
MO_SEALEVEL_GLO_PHY_L4_NRT_008_046:
_collection: SEALEVEL_GLO_PHY_L4_NRT_008_046
MO_SEALEVEL_GLO_PHY_MDT_008_063:
_collection: SEALEVEL_GLO_PHY_MDT_008_063
MO_SST_GLO_SST_L3S_NRT_OBSERVATIONS_010_010:
_collection: SST_GLO_SST_L3S_NRT_OBSERVATIONS_010_010
code_mapping:
param: eodag:sensor_type
index: 3
pattern: '[A-Z]{3}'
MO_SST_GLO_SST_L4_NRT_OBSERVATIONS_010_001:
_collection: SST_GLO_SST_L4_NRT_OBSERVATIONS_010_001
MO_SST_GLO_SST_L4_REP_OBSERVATIONS_010_011:
_collection: SST_GLO_SST_L4_REP_OBSERVATIONS_010_011
MO_SST_GLO_SST_L4_REP_OBSERVATIONS_010_024:
_collection: SST_GLO_SST_L4_REP_OBSERVATIONS_010_024
MO_WAVE_GLO_PHY_SPC_FWK_L3_NRT_014_002:
_collection: WAVE_GLO_PHY_SPC-FWK_L3_NRT_014_002
code_mapping:
param: platform
index: 0
pattern: 's1a|s1b|cfo'
MO_WAVE_GLO_PHY_SWH_L3_NRT_014_001:
_collection: WAVE_GLO_PHY_SWH_L3_NRT_014_001
code_mapping:
param: platform
index: 4
MO_WAVE_GLO_PHY_SWH_L4_NRT_014_003:
_collection: WAVE_GLO_PHY_SWH_L4_NRT_014_003
MO_WIND_GLO_PHY_CLIMATE_L4_MY_012_003:
_collection: WIND_GLO_PHY_CLIMATE_L4_MY_012_003
MO_WIND_GLO_PHY_L3_NRT_012_002:
_collection: WIND_GLO_PHY_L3_NRT_012_002
MO_WIND_GLO_PHY_L3_MY_012_005:
_collection: WIND_GLO_PHY_L3_MY_012_005
MO_WIND_GLO_PHY_L4_NRT_012_004:
_collection: WIND_GLO_PHY_L4_NRT_012_004
MO_WIND_GLO_PHY_L4_MY_012_006:
_collection: WIND_GLO_PHY_L4_MY_012_006
MO_OCEANCOLOUR_GLO_BGC_L3_MY_009_107:
_collection: OCEANCOLOUR_GLO_BGC_L3_MY_009_107
MO_OCEANCOLOUR_GLO_BGC_L3_NRT_009_101:
_collection: OCEANCOLOUR_GLO_BGC_L3_NRT_009_101
code_mapping:
param: eodag:sensor_type
index: 6
pattern: 'olci|gapfree-multi|multi-climatology|multi'
MO_OCEANCOLOUR_GLO_BGC_L3_MY_009_103:
_collection: OCEANCOLOUR_GLO_BGC_L3_MY_009_103
code_mapping:
param: eodag:sensor_type
index: 6
pattern: 'olci|gapfree-multi|multi-climatology|multi'
MO_OCEANCOLOUR_GLO_BGC_L4_NRT_009_102:
_collection: OCEANCOLOUR_GLO_BGC_L4_NRT_009_102
code_mapping:
param: eodag:sensor_type
index: 6
pattern: 'olci|gapfree-multi|multi-climatology|multi'
MO_OCEANCOLOUR_GLO_BGC_L4_MY_009_104:
_collection: OCEANCOLOUR_GLO_BGC_L4_MY_009_104
code_mapping:
param: eodag:sensor_type
index: 6
pattern: 'olci|gapfree-multi|multi-climatology|multi'
MO_OCEANCOLOUR_GLO_BGC_L4_MY_009_108:
_collection: OCEANCOLOUR_GLO_BGC_L4_MY_009_108
GENERIC_COLLECTION:
_collection: '{collection}'
download: !plugin
type: AwsDownload
s3_endpoint: https://s3.waw3-1.cloudferro.com
bucket_path_level: 0
auth: !plugin
type: AwsAuth
matching_url: s3://
s3_endpoint: https://s3.waw3-1.cloudferro.com
matching_conf:
s3_endpoint: https://s3.waw3-1.cloudferro.com
---
!provider # MARK: geodes
name: geodes
priority: 0
roles:
- host
description: French National Space Agency (CNES) Earth Observation portal
url: https://geodes.cnes.fr
search: !plugin
type: GeodesSearch
api_endpoint: https://geodes-portal.cnes.fr/api/stac/search
need_auth: false
asset_key_from_href: false
discover_queryables:
fetch_url: null
collection_fetch_url: null
discover_collections:
fetch_url: https://geodes-portal.cnes.fr/api/stac/collections
fetch_method: POST
fetch_body:
limit: 10000
pagination:
total_items_nb_key_path: '$.context.matched'
next_page_url_key_path: null
next_page_query_obj_key_path: null
next_page_token_key: page
# As of 2025/11/21 the geodes API documentation (https://geodes.cnes.fr/support/api/) states that pagination must be limited to 80.
max_limit: 80
sort:
sort_by_tpl: '{{"sortBy": [ {{"field": "{sort_param}", "direction": "{sort_order}" }} ] }}'
sort_by_default:
- !!python/tuple [start_datetime, ASC]
sort_param_mapping:
id: identifier
start_datetime: start_datetime
end_datetime: end_datetime
platform: platform
eo:cloud_cover: eo:cloud_cover
sort_order_mapping:
ascending: asc
descending: dsc
metadata_mapping:
uid: '$.id'
# OpenSearch Parameters for Collection Search (Table 3)
product:type:
- '{{"query":{{"product:type":{{"eq":"{product:type}"}}}}}}'
- '$.properties.product:type'
instruments:
- '{{"query":{{"instrument":{{"eq":"{instruments#csv_list( )}"}}}}}}'
- '{$.properties.instrument#split( )}'
# INSPIRE obligated OpenSearch Parameters for Collection Search (Table 4)
title: '$.properties.identifier'
keywords: '$.properties.keywords'
# OpenSearch Parameters for Product Search (Table 5)
sat:absolute_orbit:
- '{{"query":{{"sat:absolute_orbit":{{"eq":{sat:absolute_orbit}}}}}}}'
- '$.properties."sat:absolute_orbit"'
sat:orbit_state:
- '{{"query":{{"sat:orbit_state":{{"eq":{sat:orbit_state}}}}}}}'
- '{$.properties.sat:orbit_state#to_lower}'
sat:relative_orbit:
- '{{"query":{{"sat:relative_orbit":{{"eq":{sat:relative_orbit}}}}}}}'
- '$.properties."sat:relative_orbit"'
eo:cloud_cover:
- '{{"query":{{"eo:cloud_cover":{{"lte":{eo:cloud_cover}}}}}}}'
- '$.properties."eo:cloud_cover"'
# OpenSearch Parameters for Acquistion Parameters Search (Table 6)
start_datetime:
- '{{"query":{{"end_datetime":{{"gte":"{start_datetime#to_iso_utc_datetime}"}}}}}}'
- '$.properties.start_datetime'
end_datetime:
- '{{"query":{{"start_datetime":{{"lte":"{end_datetime#to_iso_utc_datetime}"}}}}}}'
- '$.properties.end_datetime'
sar:polarizations:
- '{{"query":{{"sar:polarizations":{{"eq":"{sar:polarizations#csv_list( )}"}}}}}}'
- '{$.properties.sar:polarizations#split( )}'
sar:instrument_mode: '$.null'
sar:beam_ids:
- '{{"query":{{"swath":{{"eq":"{sar:beam_ids#csv_list( )}"}}}}}}'
- '{$.properties.swath#split( )}'
# Custom parameters (not defined in the base document referenced above)
id:
- '{{"query":{{"identifier":{{"eq":"{id}"}}}}}}'
- '$.properties.identifier'
grid:code:
- '{{"query":{{"grid:code":{{"contains":"{grid:code#replace_str("MGRS-","T")}"}}}}}}'
- '{$.properties."grid:code"#replace_str(r"^T?(.*)$",r"MGRS-\1")}'
sci:doi: '{$.properties.sci:doi#replace_str(r"^\[\]$","Not Available")}'
published: '$.properties.datetime'
eodag:download_link: '$.assets[?(@.roles[0] == "data") & (@.type != "application/xml")].href'
eodag:quicklook: '$.assets[?(@.roles[0] == "overview")].href.`sub(/^(.*)$/, \\1?scope=gdh)`'
eodag:thumbnail: '$.assets[?(@.roles[0] == "overview")].href.`sub(/^(.*)$/, \\1?scope=gdh)`'
# order:status set to orderable by default and then updated from plugin
order:status: '{$.null#replace_str("Not Available","orderable")}'
products:
S1_SAR_OCN:
product:type: OCN
_collection: PEPS_S1_L2
metadata_mapping:
eo:cloud_cover: '{$.properties."eo:cloud_cover"#not_available}'
S1_SAR_GRD:
product:type: GRD
_collection: PEPS_S1_L1
metadata_mapping:
eo:cloud_cover: '{$.properties."eo:cloud_cover"#not_available}'
S1_SAR_SLC:
product:type: SLC
_collection: PEPS_S1_L1
metadata_mapping:
eo:cloud_cover: '{$.properties."eo:cloud_cover"#not_available}'
S2_MSI_L1C:
_collection: PEPS_S2_L1C
S2_MSI_L2A_MAJA:
_collection: THEIA_REFLECTANCE_SENTINEL2_L2A
S2_MSI_L2B_MAJA_SNOW:
_collection: THEIA_SNOW_SENTINEL2_L2B
S2_MSI_L2B_MAJA_WATER:
_collection: THEIA_WATERQUAL_SENTINEL2_L2B
GENERIC_COLLECTION:
_collection: '{collection}'
download: !plugin
type: HTTPDownload
ignore_assets: true
archive_depth: 2
auth_error_code:
- 401
- 403
auth: !plugin
type: HTTPHeaderAuth
matching_url: https://geodes-portal.cnes.fr
headers:
X-API-Key: "{apikey}"
---
!provider # MARK: geodes_s3
name: geodes_s3
priority: 0
roles:
- host
description: French National Space Agency (CNES) Earth Observation portal with internal s3 Datalake
url: https://geodes.cnes.fr
search: !plugin
type: StacListAssets
api_endpoint: https://geodes-portal.cnes.fr/api/stac/search
s3_endpoint: https://s3.datalake.cnes.fr
need_auth: true
asset_key_from_href: false
discover_queryables:
fetch_url: null
collection_fetch_url: null
discover_collections:
fetch_url: https://geodes-portal.cnes.fr/api/stac/collections
fetch_method: POST
fetch_body:
limit: 10000
pagination:
total_items_nb_key_path: '$.context.matched'
next_page_url_key_path: null
next_page_query_obj_key_path: null
next_page_token_key: page
# As of 2025/11/21 the geodes API documentation (https://geodes.cnes.fr/support/api/) states that pagination must be limited to 80.
max_limit: 80
sort:
sort_by_tpl: '{{"sortBy": [ {{"field": "{sort_param}", "direction": "{sort_order}" }} ] }}'
sort_by_default:
- !!python/tuple [start_datetime, ASC]
sort_param_mapping:
id: identifier
start_datetime: start_datetime
end_datetime: end_datetime
platform: platform
eo:cloud_cover: eo:cloud_cover
sort_order_mapping:
ascending: asc
descending: dsc
metadata_mapping:
uid: '$.id'
# OpenSearch Parameters for Collection Search (Table 3)
product:type:
- '{{"query":{{"product:type":{{"eq":"{product:type}"}}}}}}'
- '$.properties.product:type'
instruments:
- '{{"query":{{"instrument":{{"eq":"{instruments#csv_list( )}"}}}}}}'
- '{$.properties.instrument#split( )}'
# INSPIRE obligated OpenSearch Parameters for Collection Search (Table 4)
title: '$.properties.identifier'
keywords: '$.properties.keywords'
# OpenSearch Parameters for Product Search (Table 5)
sat:absolute_orbit:
- '{{"query":{{"sat:absolute_orbit":{{"eq":{sat:absolute_orbit}}}}}}}'
- '$.properties."sat:absolute_orbit"'
sat:orbit_state:
- '{{"query":{{"sat:orbit_state":{{"eq":{sat:orbit_state}}}}}}}'
- '{$.properties.sat:orbit_state#to_lower}'
sat:relative_orbit:
- '{{"query":{{"sat:relative_orbit":{{"eq":{sat:relative_orbit}}}}}}}'
- '$.properties."sat:relative_orbit"'
eo:cloud_cover:
- '{{"query":{{"eo:cloud_cover":{{"lte":{eo:cloud_cover}}}}}}}'
- '$.properties."eo:cloud_cover"'
# OpenSearch Parameters for Acquistion Parameters Search (Table 6)
start_datetime:
- '{{"query":{{"end_datetime":{{"gte":"{start_datetime#to_iso_utc_datetime}"}}}}}}'
- '$.properties.start_datetime'
end_datetime:
- '{{"query":{{"start_datetime":{{"lte":"{end_datetime#to_iso_utc_datetime}"}}}}}}'
- '$.properties.end_datetime'
sar:polarizations:
- '{{"query":{{"sar:polarizations":{{"eq":"{sar:polarizations#csv_list( )}"}}}}}}'
- '{$.properties.sar:polarizations#split( )}'
sar:instrument_mode: '$.null'
sar:beam_ids:
- '{{"query":{{"swath":{{"eq":"{sar:beam_ids#csv_list( )}"}}}}}}'
- '{$.properties.swath#split( )}'
# Custom parameters (not defined in the base document referenced above)
id:
- '{{"query":{{"identifier":{{"eq":"{id}"}}}}}}'
- '$.properties.identifier'
grid:code:
- '{{"query":{{"grid:code":{{"contains":"{grid:code#replace_str("MGRS-","T")}"}}}}}}'
- '{$.properties."grid:code"#replace_str(r"^T?(.*)$",r"MGRS-\1")}'
sci:doi: '{$.properties.sci:doi#replace_str(r"^\[\]$","Not Available")}'
published: '$.properties.datetime'
eodag:download_link: '$.properties.endpoint_url'
eodag:quicklook: '$.assets[?(@.roles[0] == "overview")].href.`sub(/^(.*)$/, \\1?scope=gdh)`'
eodag:thumbnail: '$.assets[?(@.roles[0] == "overview")].href.`sub(/^(.*)$/, \\1?scope=gdh)`'
# order:status set to succeeded for consistency between providers
order:status: '{$.null#replace_str("Not Available","succeeded")}'
products:
S1_SAR_OCN:
product:type: OCN
_collection: PEPS_S1_L2
metadata_mapping:
eo:cloud_cover: '{$.properties."eo:cloud_cover"#not_available}'
S1_SAR_GRD:
product:type: GRD
_collection: PEPS_S1_L1
metadata_mapping:
eo:cloud_cover: '{$.properties."eo:cloud_cover"#not_available}'
S1_SAR_SLC:
product:type: SLC
_collection: PEPS_S1_L1
metadata_mapping:
eo:cloud_cover: '{$.properties."eo:cloud_cover"#not_available}'
S2_MSI_L1C:
_collection: PEPS_S2_L1C
S2_MSI_L2A_MAJA:
_collection: THEIA_REFLECTANCE_SENTINEL2_L2A
S2_MSI_L2B_MAJA_SNOW:
_collection: THEIA_SNOW_SENTINEL2_L2B
S2_MSI_L2B_MAJA_WATER:
_collection: THEIA_WATERQUAL_SENTINEL2_L2B
GENERIC_COLLECTION:
_collection: '{collection}'
download: !plugin
type: AwsDownload
s3_endpoint: https://s3.datalake.cnes.fr
ignore_assets: True
auth: !plugin
type: AwsAuth
auth_error_code: 403
s3_endpoint: https://s3.datalake.cnes.fr
matching_conf:
s3_endpoint: https://s3.datalake.cnes.fr
---
!provider # MARK: cop_ewds
name: cop_ewds
priority: 0
description: CEMS Early Warning Data Store
roles:
- host
url: https://ewds.climate.copernicus.eu
anchor_time_day_month_year: &time_day_month_year
end_datetime:
- |
{{
"year": {_date#interval_to_datetime_dict}["year"],
"month": {_date#interval_to_datetime_dict}["month"],
"day": {_date#interval_to_datetime_dict}["day"],
"time": {start_datetime#get_ecmwf_time}
}}
- '{$.end_datetime#to_iso_date}'
anchor_day_month_year: &day_month_year
end_datetime:
- |
{{
"year": {_date#interval_to_datetime_dict}["year"],
"month": {_date#interval_to_datetime_dict}["month"],
"day": {_date#interval_to_datetime_dict}["day"]
}}
- '{$.end_datetime#to_iso_date}'
anchor_month_year: &month_year
end_datetime:
- |
{{
"year": {_date#interval_to_datetime_dict}["year"],
"month": {_date#interval_to_datetime_dict}["month"]
}}
- '{$.end_datetime#to_iso_date}'
anchor_time_hday_hmonth_hyear: &time_hday_hmonth_hyear
end_datetime:
- |
{{
"hyear": {_date#interval_to_datetime_dict}["year"],
"hmonth": {_date#interval_to_datetime_dict}["month"],
"hday": {_date#interval_to_datetime_dict}["day"],
"time": {start_datetime#get_ecmwf_time}
}}
- '{$.end_datetime#to_iso_date}'
anchor_hday_hmonth_hyear: &hday_hmonth_hyear
end_datetime:
- |
{{
"hyear": {_date#interval_to_datetime_dict}["year"],
"hmonth": {_date#interval_to_datetime_dict}["month"],
"hday": {_date#interval_to_datetime_dict}["day"]
}}
- '{$.end_datetime#to_iso_date}'
anchor_hmonth_hyear: &hmonth_hyear
end_datetime:
- |
{{
"hyear": {_date#interval_to_datetime_dict}["year"],
"hmonth": {_date#interval_to_datetime_dict}["month"]
}}
- '{$.end_datetime#to_iso_date}'
auth: !plugin
type: HTTPHeaderAuth
ssl_verify: true
headers:
PRIVATE-TOKEN: "{apikey}"
download: !plugin
type: HTTPDownload
timeout: 30
ssl_verify: true
extract: false
auth_error_code: 401
order_enabled: true
order_method: POST
order_on_response:
metadata_mapping:
eodag:order_id: $.json.jobID
eodag:status_link: https://ewds.climate.copernicus.eu/api/retrieve/v1/jobs/{eodag:order_id}?request=true
eodag:search_link: https://ewds.climate.copernicus.eu/api/retrieve/v1/jobs/{eodag:order_id}/results
order_status:
request:
method: GET
metadata_mapping:
eodag:order_status: $.json.status
order:date: $.json.created
created: $.json.created
published: $.json.finished
updated: $.json.updated
ecmwf:dataset: $.json.processID
eodag:request_params: $.json.metadata.request.ids
error:
eodag:order_status: failed
success:
eodag:order_status: successful
on_success:
need_search: true
metadata_mapping:
eodag:download_link: $.json.asset.value.href
search: !plugin
type: ECMWFSearch
need_auth: true
ssl_verify: true
end_date_excluded: false
remove_from_query:
- dataset
- date
discover_queryables:
fetch_url: null
collection_fetch_url: null
constraints_url: https://ewds.climate.copernicus.eu/api/catalogue/v1/collections/{dataset}/constraints.json
form_url: https://ewds.climate.copernicus.eu/api/catalogue/v1/collections/{dataset}/form.json
metadata_mapping:
product:type: $.productType
start_datetime: '{$.start_datetime#to_iso_utc_datetime}'
end_datetime:
- date={start_datetime#to_iso_date}/{end_datetime#to_iso_date}
- '{$.end_datetime#to_iso_utc_datetime}'
# The geographic extent of the product
geometry:
- '{{"area": {geometry#to_nwse_bounds}}}'
- $.geometry
qs: $.qs
eodag:order_link: 'https://ewds.climate.copernicus.eu/api/retrieve/v1/processes/{dataset}/execution?{{"inputs": {qs#to_geojson}}}'
products:
EFAS_HISTORICAL:
dataset: efas-historical
metadata_mapping:
<<: *time_hday_hmonth_hyear
EFAS_FORECAST:
dataset: efas-forecast
metadata_mapping:
<<: *time_day_month_year
EFAS_SEASONAL:
dataset: efas-seasonal
metadata_mapping:
<<: *month_year
EFAS_REFORECAST:
dataset: efas-reforecast
metadata_mapping:
<<: *hday_hmonth_hyear
EFAS_SEASONAL_REFORECAST:
dataset: efas-seasonal-reforecast
metadata_mapping:
<<: *hmonth_hyear
GLOFAS_HISTORICAL:
dataset: cems-glofas-historical
metadata_mapping:
<<: *hday_hmonth_hyear
GLOFAS_FORECAST:
dataset: cems-glofas-forecast
metadata_mapping:
<<: *day_month_year
GLOFAS_SEASONAL:
dataset: cems-glofas-seasonal
metadata_mapping:
<<: *month_year
GLOFAS_SEASONAL_REFORECAST:
dataset: cems-glofas-seasonal-reforecast
metadata_mapping:
<<: *hmonth_hyear
GLOFAS_REFORECAST:
dataset: cems-glofas-reforecast
metadata_mapping:
<<: *hday_hmonth_hyear
FIRE_HISTORICAL:
dataset: cems-fire-historical-v1
metadata_mapping:
<<: *day_month_year
FIRE_SEASONAL:
dataset: cems-fire-seasonal
metadata_mapping:
<<: *day_month_year
GENERIC_COLLECTION:
dataset: '{collection}'
---
!provider # MARK: fedeo_ceda
name: fedeo_ceda
priority: 0
description: CEDA datasets through FedEO Catalog
roles:
- host
url: https://fedeo.ceos.org/
search: !plugin
type: StacSearch
api_endpoint: 'https://fedeo.ceos.org/search'
ssl_verify: true
timeout: 60
pagination:
next_page_url_tpl: '{url}?startRecord={next_page_token}'
next_page_query_obj: '{{"limit":{limit}}}'
next_page_token_key: skip
# startRecord is equivalent to skip + 1, so make skip start at page 2
start_page: 2
next_page_query_obj_key_path: null
next_page_url_key_path: null
discover_queryables:
fetch_url: null
collection_fetch_url: null
discover_collections:
fetch_url: 'https://fedeo.ceos.org/series/eo:organisationName/CEDA/'
result_type: json
results_entry: '$.links[?(@.rel=="child")]'
generic_collection_id: '{$.href#replace_str(r".*/", "")}'
generic_collection_parsable_properties:
_collection: '{$.href#replace_str(r".*/", "")}'
title: '$.title'
single_collection_fetch_url: 'https://fedeo.ceos.org/series/eo:organisationName/CEDA/{_collection}'
single_collection_parsable_metadata:
id: '{$.assets.enclosure.href#ceda_collection_name}'
description: '{$.description#literalize_unicode}'
instruments: '$.summaries.instruments'
keywords: '$.keywords'
license: '$.license'
platform: '{$.summaries.platform#csv_list}'
title: '$.title'
metadata_mapping:
title: '{$.properties.title#remove_extension}'
geometry: '$.geometry'
assets: '{$.assets#dict_filter($[?(@.href =~ ".*\/thredds\/fileServer\/.*")])}'
products:
GENERIC_COLLECTION:
_collection: '{collection}'
download: !plugin
type: HTTPDownload
---
!provider # MARK: DLR EOC Geoservice
name: dlr_eoc_geoservice
priority: 0
description: DLR Earth Observation Center (EOC) Geoservice
roles:
- host
url: https://geoservice.dlr.de
search: !plugin
type: StacSearch
api_endpoint: https://geoservice.dlr.de/eoc/ogc/stac/v1/search
sort:
sort_by_default:
- !!python/tuple [start_datetime, ASC]
sort_param_mapping:
id: id
start_datetime: properties.datetime
created: properties.created
updated: properties.updated
eo:cloud_cover: properties.eo:cloud_cover
products:
S2_MSI_L2A_MAJA:
_collection: S2_L2A_MAJA
metadata_mapping:
eodag:download_link: '$.assets.data.href'
SUPERSITES:
_collection: SUPERSITES
metadata_mapping:
eodag:download_link: '$.assets.data.href'
GENERIC_COLLECTION:
_collection: '{collection}'
download: !plugin
type: HTTPDownload
auth_error_code:
- 401
- 403
products:
S2_MSI_L2A_MAJA:
ignore_assets: true
SUPERSITES:
ignore_assets: true
auth: !plugin
type: GenericAuth
matching_url: https://download.geoservice.dlr.de
---
!provider # MARK: eocat
name: eocat
priority: 0
description: ESA Catalog provides interoperable access, following ISO/OGC interface guidelines, to Earth Observation metadata
roles:
- host
url: https://eocat.esa.int/eo-catalogue
search: !plugin
type: StacSearch
api_endpoint: 'https://eocat.esa.int/eo-catalogue/search'
ssl_verify: false
timeout: 90
discover_queryables:
fetch_url: null
collection_fetch_url: null
discover_collections:
fetch_url: 'https://eocat.esa.int/eo-catalogue/collections?limit=500'
result_type: json
results_entry: '$.collections[?(@.links[*].rel=="items" & @.id!="datasets" & @.id!="services")]'
single_collection_fetch_url: 'https://eocat.esa.int/eo-catalogue/collections/{_collection}'
single_collection_parsable_metadata:
platform: '{$.summaries.platform#csv_list}'
metadata_mapping:
assets: '{$.assets#dict_with_roles(["data", "thumbnail", "overview"])}'
eodag:download_link: '$.assets.enclosure.href'
title: '{$.properties.title#remove_extension}'
products:
GENERIC_COLLECTION:
_collection: '{collection}'
download: !plugin
type: HTTPDownload
ignore_assets: true
auth: !plugin
type: EOIAMAuth
matching_url: https://[-\w\.]+.eo.esa.int
auth_uri: 'https://eoiam-idp.eo.esa.int'
---
!provider # MARK: cop_ghsl
name: cop_ghsl
priority: 0
description: Copernicus Global Human Settlement Layer
roles:
- host
url: https://human-settlement.emergency.copernicus.eu/index.php
search: !plugin
type: CopGhslSearch
api_endpoint: https://human-settlement.emergency.copernicus.eu/data/tilesDLD
need_auth: false
ssl_verify: true
metadata_mapping:
eodag:default_geometry: 'POLYGON((180 -90, 180 90, -180 90, -180 -90, 180 -90))'
pagination:
max_items_per_page: 10000
discover_queryables:
fetch_url: null
collection_fetch_url: null
constraints_url: "https://s3.central.data.destination-earth.eu/swift/v1/constraints/cop_ghsl_dev/{collection}.json"
products:
# product types with tiles
GHS_BUILT_S:
_collection: BUILT
metadata_mapping:
tile_size: '{$.tile_size#replace_str_tuple((("m", ""), ("k", "000")))}'
dataset: GHS_BUILT_S_{add_filter}_E{year}_GLOBE_R2023A_{proj_code}_{tile_size}
eodag:download_link: 'https://jeodpp.jrc.ec.europa.eu/ftp/jrc-opendata/GHSL/GHS_BUILT_S_GLOBE_R2023A/{dataset}/V1-0/tiles/{dataset}_V1_0_{tile_id}.zip'
GHS_BUILT_H:
_collection: builtH
year: '2018'
metadata_mapping:
tile_size: '{$.tile_size#replace_str("m", "")}'
dataset: GHS_BUILT_H_{add_filter}_E{year}_GLOBE_R2023A_{proj_code}_{tile_size}
eodag:download_link: 'https://jeodpp.jrc.ec.europa.eu/ftp/jrc-opendata/GHSL/GHS_BUILT_H_GLOBE_R2023A/{dataset}/V1-0/tiles/{dataset}_V1_0_{tile_id}.zip'
GHS_BUILT_V:
_collection: builtV
metadata_mapping:
tile_size: '{$.tile_size#replace_str_tuple((("m", ""), ("k", "000")))}'
dataset: GHS_BUILT_V_{add_filter}_E{year}_GLOBE_R2023A_{proj_code}_{tile_size}
eodag:download_link: 'https://jeodpp.jrc.ec.europa.eu/ftp/jrc-opendata/GHSL/GHS_BUILT_V_GLOBE_R2023A/{dataset}/V1-0/tiles/{dataset}_V1_0_{tile_id}.zip'
GHS_BUILT_C:
_collection: builtC
year: '2018'
tile_size: 10m
proj:code: 'EPSG:54009'
metadata_mapping:
tile_size: '{$.tile_size#replace_str("m", "")}'
dataset: GHS_BUILT_C_{add_filter}_E{year}_GLOBE_R2023A_{proj_code}_{tile_size}
eodag:download_link: 'https://jeodpp.jrc.ec.europa.eu/ftp/jrc-opendata/GHSL/GHS_BUILT_C_GLOBE_R2023A/{dataset}/V1-0/tiles/{dataset}_V1_0_{tile_id}.zip'
GHS_LAND:
_collection: land
year: '2018'
proj:code: 'EPSG:54009'
metadata_mapping:
tile_size: '{$.tile_size#replace_str_tuple((("m", ""), ("k", "000")))}'
dataset: GHS_LAND_E2018_GLOBE_R2022A_{proj_code}_{tile_size}
eodag:download_link: 'https://jeodpp.jrc.ec.europa.eu/ftp/jrc-opendata/GHSL/GHS_LAND_GLOBE_R2022A/{dataset}/V1-0/tiles/{dataset}_V1_0_{tile_id}.zip'
GHS_POP:
_collection: POP
metadata_mapping:
tile_size: '{$.tile_size#replace_str_tuple((("m", ""), ("k", "000")))}'
dataset: GHS_POP_E{year}_GLOBE_R2023A_{proj_code}_{tile_size}
eodag:download_link: 'https://jeodpp.jrc.ec.europa.eu/ftp/jrc-opendata/GHSL/GHS_POP_GLOBE_R2023A/{dataset}/V1-0/tiles/{dataset}_V1_0_{tile_id}.zip'
GHS_SMOD:
_collection: SMOD
tile_size: 1k
proj:code: 'EPSG:54009'
metadata_mapping:
tile_size: '{$.tile_size#replace_str("k", "000")}'
dataset: GHS_SMOD_E{year}_GLOBE_R2023A_{proj_code}_{tile_size}
eodag:download_link: 'https://jeodpp.jrc.ec.europa.eu/ftp/jrc-opendata/GHSL/GHS_SMOD_GLOBE_R2023A/{dataset}/V2-0/tiles/{dataset}_V2_0_{tile_id}.zip'
GHS_ESM:
_collection: ESM
year: '2015'
proj:code: 'EPSG:3035'
metadata_mapping:
tile_size: '{$.tile_size#replace_str("2m", "02")}'
dataset: ESM_BUILT_VHR2015_EUROPE_R2019_3035_{tile_size}
eodag:download_link: 'https://jeodpp.jrc.ec.europa.eu/ftp/jrc-opendata/GHSL/ESM_BUILT_VHR2015_Europe_R2019/{dataset}/V1-0/tiles/{tile_id}.zip'
# produc types with one file
GHS_DUC:
_collection: DUC
metadata_mapping:
eodag:download_link: 'https://jeodpp.jrc.ec.europa.eu/ftp/jrc-opendata/GHSL//GHS_DUC_GLOBE_R2023A/V2-0/GHS_DUC_MT_GLOBE_R2023A_V2_0.zip'
GHS_FUA:
_collection: FUA
metadata_mapping:
eodag:download_link: 'https://jeodpp.jrc.ec.europa.eu/ftp/jrc-opendata/GHSL//GHS_FUA_UCDB2015_GLOBE_R2019A/V1-0/GHS_FUA_UCDB2015_GLOBE_R2019A_54009_1K_V1_0.zip'
GHS_BUILT_LAUSTAT:
_collection: BUILT_LAUSTAT
metadata_mapping:
eodag:download_link: 'https://jeodpp.jrc.ec.europa.eu/ftp/jrc-opendata/GHSL//GHS_BUILT_LAUSTAT_EUROPE_R2023A/V1-0/GHS-BUILT-LAUSTAT_EUROPE_R2023A.zip'
# products with several files
GHS_ENACT_POP:
_collection: ENACT_POP
grouped_by: month
year: '2011'
metadata_mapping:
eodag:download_link: https://jeodpp.jrc.ec.europa.eu
assets_mapping:
day:
href: 'https://jeodpp.jrc.ec.europa.eu/ftp/jrc-opendata/ENACT/ENACT_POP_2011_EU28_R2020A/ENACT_POP_D{month}2011_EU28_R2020A_{proj_code}_{tile_size}/V1-0/ENACT_POP_D{month}2011_EU28_R2020A_{proj_code}_{tile_size}_V1_0.zip'
type: "application/zip"
title: Daytime grid
night:
href: 'https://jeodpp.jrc.ec.europa.eu/ftp/jrc-opendata/ENACT/ENACT_POP_2011_EU28_R2020A/ENACT_POP_N{month}2011_EU28_R2020A_{proj_code}_{tile_size}/V1-0/ENACT_POP_N{month}2011_EU28_R2020A_{proj_code}_{tile_size}_V1_0.zip'
type: "application/zip"
title: Nightime grid
GHS_UCDB_DOMAIN:
_collection: UCDB
grouped_by: thematic_domain
metadata_mapping:
eodag:download_link: https://jeodpp.jrc.ec.europa.eu/ftp/jrc-opendata/GHSL/GHS_UCDB_GLOBE_R2024A/GHS_UCDB_THEME_GLOBE_R2024A/GHS_UCDB_THEME_{thematic_domain}_GLOBE_R2024A/V1-1/GHS_UCDB_THEME_{thematic_domain}_GLOBE_R2024A_V1_1.zip
GHS_UCDB_REGION:
_collection: UCDB
grouped_by: region
metadata_mapping:
eodag:download_link: https://jeodpp.jrc.ec.europa.eu/ftp/jrc-opendata/GHSL/GHS_UCDB_GLOBE_R2024A/GHS_UCDB_REGION_GLOBE_R2024A/GHS_UCDB_REGION_{region}_R2024A/V1-1/GHS_UCDB_REGION_{region}_R2024A_V1_1.zip
download: !plugin
type: HTTPDownload
ssl_verify: true
---
!provider # MARK: cop_dataspace_s3
name: cop_dataspace_s3
priority: 0
description: Copernicus Data Space Ecosystem through S3 protocol
roles:
- host
url: https://dataspace.copernicus.eu/
search: !plugin
type: CreodiasS3Search
api_endpoint: 'https://catalogue.dataspace.copernicus.eu/odata/v1/Products'
s3_endpoint: 'https://eodata.dataspace.copernicus.eu'
need_auth: true
timeout: 120
ssl_verify: true
dont_quote:
- '['
- ']'
- '$'
- '='
- '&'
- ':'
- '%'
pagination:
next_page_url_tpl: '{url}?{search}&$top={limit}&$skip={skip}&$expand=Attributes&$expand=Assets'
next_page_url_key_path: '$.["@odata.nextLink"]'
next_page_token_key: skip
parse_url_key: $skip
count_tpl: '&$count=True'
total_items_nb_key_path: '$."@odata.count"'
max_limit: 1_000
sort:
sort_by_default:
- !!python/tuple [start_datetime, ASC]
sort_by_tpl: '&$orderby={sort_param} {sort_order}'
sort_param_mapping:
start_datetime: ContentDate/Start
end_datetime: ContentDate/End
published: PublicationDate
updated: ModificationDate
sort_order_mapping:
ascending: asc
descending: desc
max_sort_params: 1
results_entry: 'value'
free_text_search_operations:
$filter:
union: ' or '
wrapper: '{}'
operations:
and:
- "Collection/Name eq '{_collection}'"
- "OData.CSC.Intersects(area=geography'{geometry#to_ewkt}')"
- "Attributes/OData.CSC.StringAttribute/any(att:att/Name eq 'productType' and att/OData.CSC.StringAttribute/Value eq '{product:type}')"
- "Attributes/OData.CSC.StringAttribute/any(att:att/Name eq 'platformShortName' and att/OData.CSC.StringAttribute/Value eq '{constellation}')"
- "Attributes/OData.CSC.StringAttribute/any(att:att/Name eq 'platformSerialIdentifier' and att/OData.CSC.StringAttribute/Value eq '{platform#replace_str(\"^S[1-3]\", \"\")}')"
- "Attributes/OData.CSC.IntegerAttribute/any(att:att/Name eq 'spatialResolution' and att/OData.CSC.StringAttribute/Value eq '{gsd}')"
- "Attributes/OData.CSC.StringAttribute/any(att:att/Name eq 'authority' and att/OData.CSC.StringAttribute/Value eq '{providers}')"
- "Attributes/OData.CSC.IntegerAttribute/any(att:att/Name eq 'orbitNumber' and att/OData.CSC.StringAttribute/Value eq {sat:absolute_orbit})"
- "Attributes/OData.CSC.IntegerAttribute/any(att:att/Name eq 'relativeOrbitNumber' and att/OData.CSC.StringAttribute/Value eq {sat:relative_orbit})"
- "Attributes/OData.CSC.StringAttribute/any(att:att/Name eq 'orbitDirection' and att/OData.CSC.StringAttribute/Value eq '{sat:orbit_state#to_upper}')"
- "Attributes/OData.CSC.DoubleAttribute/any(att:att/Name eq 'cloudCover' and att/OData.CSC.DoubleAttribute/Value le {eo:cloud_cover})"
- "Attributes/OData.CSC.StringAttribute/any(att:att/Name eq 'instrumentShortName' and att/OData.CSC.StringAttribute/Value eq '{instruments#csv_list}')"
- "Attributes/OData.CSC.StringAttribute/any(att:att/Name eq 'tileId' and att/OData.CSC.StringAttribute/Value eq '{grid:code#replace_str(\"MGRS-\",\"\")}')"
- "Attributes/OData.CSC.StringAttribute/any(att:att/Name eq 'processingLevel' and att/OData.CSC.StringAttribute/Value eq '{processing:level}')"
- "Attributes/OData.CSC.StringAttribute/any(att:att/Name eq 'processingDate' and att/OData.CSC.StringAttribute/Value eq '{processing:datetime}')"
- "Attributes/OData.CSC.StringAttribute/any(att:att/Name eq 'processingCenter' and att/OData.CSC.StringAttribute/Value eq '{processing:facility}')"
- "Attributes/OData.CSC.StringAttribute/any(att:att/Name eq 'processorVersion' and att/OData.CSC.StringAttribute/Value eq '{processing:version}')"
- "Attributes/OData.CSC.StringAttribute/any(att:att/Name eq 'swathIdentifier' and att/OData.CSC.StringAttribute/Value eq '{sar:instrument_mode}')"
- "Attributes/OData.CSC.StringAttribute/any(att:att/Name eq 'datatakeID' and att/OData.CSC.StringAttribute/Value eq '{s1:datatake_id}')"
- "Attributes/OData.CSC.StringAttribute/any(att:att/Name eq 'instrumentConfigurationID' and att/OData.CSC.StringAttribute/Value eq '{s1:instrument_configuration_ID}')"
- "Attributes/OData.CSC.StringAttribute/any(att:att/Name eq 'sliceNumber' and att/OData.CSC.StringAttribute/Value eq '{s1:slice_number}')"
- "Attributes/OData.CSC.StringAttribute/any(att:att/Name eq 'totalSlices' and att/OData.CSC.StringAttribute/Value eq '{s1:total_slices}')"
- "Attributes/OData.CSC.StringAttribute/any(att:att/Name eq 'granuleIdentifier' and att/OData.CSC.StringAttribute/Value eq '{s2:tile_id}')"
- "Attributes/OData.CSC.StringAttribute/any(att:att/Name eq 'productGroupId' and att/OData.CSC.StringAttribute/Value eq '{s2:datatake_id}')"
- "Attributes/OData.CSC.StringAttribute/any(att:att/Name eq 'Name' and att/OData.CSC.StringAttribute/Value eq '{s2:product_uri}')"
- "Attributes/OData.CSC.StringAttribute/any(att:att/Name eq 'datastripId' and att/OData.CSC.StringAttribute/Value eq '{s2:datastrip_id}')"
- "Attributes/OData.CSC.StringAttribute/any(att:att/Name eq 'operationalMode' and att/OData.CSC.StringAttribute/Value eq '{s2:datatake_type}')"
- "ModificationDate gt {updated_after#to_iso_utc_datetime}"
- "ModificationDate lt {updated_before#to_iso_utc_datetime}"
- "PublicationDate gt {published_after#to_iso_utc_datetime}"
- "PublicationDate lt {published_before#to_iso_utc_datetime}"
- "ContentDate/Start lt {end_datetime#to_iso_utc_datetime}"
- "ContentDate/End gt {start_datetime#to_iso_utc_datetime}"
- "Attributes/OData.CSC.StringAttribute/any(att:att/Name eq 'polarisationChannels' and att/OData.CSC.StringAttribute/Value eq '{sar:polarizations#csv_list(%26)}')"
- contains(Name,'{id}')
discover_metadata:
auto_discovery: true
metadata_pattern: '^(?!collection)[a-zA-Z0-9]+$'
search_param:
free_text_search_operations:
$filter:
operations:
and:
- "Attributes/OData.CSC.StringAttribute/any(att:att/Name eq '{metadata}' and att/OData.CSC.StringAttribute/Value eq '{{{metadata}}}')"
metadata_path: '$.Attributes.*'
per_product_metadata_query: false
metadata_pre_mapping:
metadata_path: '$.Attributes'
metadata_path_id: 'Name'
metadata_path_value: 'Value'
metadata_mapping:
_collection:
- null
- '$.null'
# hide duplicated metadata
beginningDateTime: '$.null'
endingDateTime: '$.null'
platformSerialIdentifier: '$.null'
# Opensearch resource identifier within the search engine context (in our case
# within the context of the data provider)
# Queryable parameters are set with null as 1st configuration list value to mark them as queryable,
# but `free_text_search_operations.$filter.operations.and` entries are then used instead.
uid: '$.Id'
# OpenSearch Parameters for Collection Search (Table 3)
product:type:
- null
- '$.Attributes.productType'
constellation:
- null
- '$.Attributes.platformShortName'
platform:
- null
- '$.Attributes.platformSerialIdentifier'
instruments:
- null
- '{$.Attributes.instrumentShortName#split( )}'
processing:level:
- null
- '$.Attributes.processingLevel'
processing:datetime:
- null
- '$.Attributes.processingDate'
processing:facility:
- null
- '$.Attributes.processingCenter'
processing:version:
- null
- '{$.Attributes.processorVersion#to_geojson}'
_processor_name:
- null
- '$.Attributes.processorName'
processing:software: '{{"{_processor_name}":"{processing:version}"}}'
# INSPIRE obligated OpenSearch Parameters for Collection Search (Table 4)
title: '{$.Name#remove_extension}'
gsd:
- null
- '$.Attributes.spatialResolution'
_provider: '$.Attributes.origin'
providers:
- null
- '[{{"name":"{_provider}","roles":["producer"]}}]'
published_after:
- null
- '$.null'
published_before:
- null
- '$.null'
published: '$.PublicationDate'
# OpenSearch Parameters for Product Search (Table 5)
sat:absolute_orbit:
- null
- '$.Attributes.orbitNumber'
sat:relative_orbit:
- null
- '$.Attributes.relativeOrbitNumber'
sat:orbit_state:
- null
- '{$.Attributes.orbitDirection#to_lower}'
eo:cloud_cover:
- null
- '$.Attributes.cloudCover'
updated_after:
- null
- '$.null'
updated_before:
- null
- '$.null'
updated:
- null
- '$.ModificationDate'
sat:relative_orbit:
- null
- '$.Attributes.relativeOrbitNumber'
# OpenSearch Parameters for Acquistion Parameters Search (Table 6)
start_datetime:
- null
- '$.ContentDate.Start'
end_datetime:
- null
- '$.ContentDate.End'
product:timeliness:
- null
- '$.Attributes.timeliness'
sar:instrument_mode:
- null
- '$.Attributes.swathIdentifier'
sar:polarizations:
- null
- '{$.Attributes.polarisationChannels#split(&)}'
s1:datatake_id:
- null
- '$.Attributes.datatakeID'
s1:instrument_configuration_ID:
- null
- '$.Attributes.instrumentConfigurationID'
s1:slice_number:
- null
- '$.Attributes.sliceNumber'
s1:total_slices:
- null
- '$.Attributes.totalSlices'
s2:tile_id:
- null
- '$.Attributes.granuleIdentifier'
s2:datatake_id:
- null
- '$.Attributes.productGroupId'
s2:product_uri:
- null
- '$.Attributes.Name'
s2:datastrip_id:
- null
- '$.Attributes.datastripId'
s2:datatake_type:
- null
- '$.Attributes.operationalMode'
# Custom parameters (not defined in the base document referenced above)
id:
- null
- '{$.Name#remove_extension}'
grid:code:
- null
- '{$.Attributes.tileId#replace_str(r"^","MGRS-")}'
# The geographic extent of the product
geometry:
- null
- '{$.Footprint#from_ewkt}'
# The url to download the product "as is" (literal or as a template to be completed either after the search result
# is obtained from the provider or during the eodag download phase)
eodag:download_link: '$.S3Path.`sub(/^(.*)$/, s3:/\\1)`'
# order:status: must be one of succeeded, ordered, orderable
order:status: '{$.Online#get_group_name((?P<succeeded>True)|(?P<orderable>False))}'
collection:
- null
- $.null
eodag:quicklook: '$.Assets[?(@.Type="QUICKLOOK")].DownloadLink'
eodag:thumbnail: '$.Assets[?(@.Type="QUICKLOOK")].DownloadLink'
download: !plugin
type: AwsDownload
s3_endpoint: 'https://eodata.dataspace.copernicus.eu'
s3_bucket: 'eodata.dataspace.copernicus.eu'
ssl_verify: true
auth: !plugin
type: AwsAuth
auth_error_code: 403
s3_endpoint: 'https://eodata.dataspace.copernicus.eu'
support_presign_url: False
matching_conf:
s3_endpoint: 'https://eodata.dataspace.copernicus.eu'
products:
# S2
S2_MSI_L1C:
_collection: SENTINEL-2
product:type: S2MSI1C
metadata_mapping:
platform:
- null
- '$.Attributes.platformSerialIdentifier.`sub(/^/, S2)`'
S2_MSI_L2A:
_collection: SENTINEL-2
product:type: S2MSI2A
metadata_mapping:
platform:
- null
- '$.Attributes.platformSerialIdentifier.`sub(/^/, S2)`'
# S1
S1_AUX_GNSSRD:
product:type: AUX_GNSSRD
_collection: SENTINEL-1
metadata_mapping:
eo:cloud_cover: '{$.Attributes.cloudCover#not_available}'
platform:
- null
- '$.Attributes.platformSerialIdentifier.`sub(/^/, S1)`'
S1_AUX_MOEORB:
product:type: AUX_MOEORB
_collection: SENTINEL-1
metadata_mapping:
eo:cloud_cover: '{$.Attributes.cloudCover#not_available}'
platform:
- null
- '$.Attributes.platformSerialIdentifier.`sub(/^/, S1)`'
S1_AUX_POEORB:
product:type: AUX_POEORB
_collection: SENTINEL-1
metadata_mapping:
eo:cloud_cover: '{$.Attributes.cloudCover#not_available}'
platform:
- null
- '$.Attributes.platformSerialIdentifier.`sub(/^/, S1)`'
S1_AUX_PREORB:
product:type: AUX_PREORB
_collection: SENTINEL-1
metadata_mapping:
eo:cloud_cover: '{$.Attributes.cloudCover#not_available}'
platform:
- null
- '$.Attributes.platformSerialIdentifier.`sub(/^/, S1)`'
S1_AUX_PROQUA:
product:type: AUX_PROQUA
_collection: SENTINEL-1
metadata_mapping:
eo:cloud_cover: '{$.Attributes.cloudCover#not_available}'
platform:
- null
- '$.Attributes.platformSerialIdentifier.`sub(/^/, S1)`'
S1_AUX_RESORB:
product:type: AUX_RESORB
_collection: SENTINEL-1
metadata_mapping:
eo:cloud_cover: '{$.Attributes.cloudCover#not_available}'
platform:
- null
- '$.Attributes.platformSerialIdentifier.`sub(/^/, S1)`'
S1_SAR_RAW:
product:type: RAW
_collection: SENTINEL-1
metadata_mapping:
eo:cloud_cover: '{$.Attributes.cloudCover#not_available}'
platform:
- null
- '$.Attributes.platformSerialIdentifier.`sub(/^/, S1)`'
S1_SAR_GRD:
product:type: GRD
_collection: SENTINEL-1
metadata_mapping:
eo:cloud_cover: '{$.Attributes.cloudCover#not_available}'
platform:
- null
- '$.Attributes.platformSerialIdentifier.`sub(/^/, S1)`'
S1_SAR_GRD_COG:
product:type: GRD-COG
_collection: SENTINEL-1
metadata_mapping:
eo:cloud_cover: '{$.Attributes.cloudCover#not_available}'
platform:
- null
- '$.Attributes.platformSerialIdentifier.`sub(/^/, S1)`'
S1_SAR_SLC:
product:type: SLC
_collection: SENTINEL-1
metadata_mapping:
eo:cloud_cover: '{$.Attributes.cloudCover#not_available}'
platform:
- null
- '$.Attributes.platformSerialIdentifier.`sub(/^/, S1)`'
S1_SAR_OCN:
product:type: OCN
_collection: SENTINEL-1
metadata_mapping:
eo:cloud_cover: '{$.Attributes.cloudCover#not_available}'
platform:
- null
- '$.Attributes.platformSerialIdentifier.`sub(/^/, S1)`'
S1_SAR_L3_IW_MCM:
product:type: S1SAR_L3_IW_MCM
_collection: GLOBAL-MOSAICS
metadata_mapping:
eo:cloud_cover: '{$.Attributes.cloudCover#not_available}'
platform:
- null
- '$.Attributes.platformSerialIdentifier.`sub(/^/, S1)`'
S1_SAR_L3_DH_MCM:
product:type: S1SAR_L3_DH_MCM
_collection: GLOBAL-MOSAICS
metadata_mapping:
eo:cloud_cover: '{$.Attributes.cloudCover#not_available}'
platform:
- null
- '$.Attributes.platformSerialIdentifier.`sub(/^/, S1)`'
# S3 SRAL
S3_SRA:
product:type: SR_1_SRA___
_collection: SENTINEL-3
metadata_mapping:
eo:cloud_cover: '{$.Attributes.cloudCover#not_available}'
platform:
- null
- '$.Attributes.platformSerialIdentifier.`sub(/^/, S3)`'
S3_SRA_A:
product:type: SR_1_SRA_A_
_collection: SENTINEL-3
metadata_mapping:
eo:cloud_cover: '{$.Attributes.cloudCover#not_available}'
platform:
- null
- '$.Attributes.platformSerialIdentifier.`sub(/^/, S3)`'
S3_SRA_BS:
product:type: SR_1_SRA_BS
_collection: SENTINEL-3
metadata_mapping:
eo:cloud_cover: '{$.Attributes.cloudCover#not_available}'
platform:
- null
- '$.Attributes.platformSerialIdentifier.`sub(/^/, S3)`'
S3_LAN:
product:type: SR_2_LAN___
_collection: SENTINEL-3
metadata_mapping:
eo:cloud_cover: '{$.Attributes.cloudCover#not_available}'
platform:
- null
- '$.Attributes.platformSerialIdentifier.`sub(/^/, S3)`'
S3_LAN_HY:
product:type: SR_2_LAN_HY
_collection: SENTINEL-3
metadata_mapping:
eo:cloud_cover: '{$.Attributes.cloudCover#not_available}'
platform:
- null
- '$.Attributes.platformSerialIdentifier.`sub(/^/, S3)`'
S3_LAN_SI:
product:type: SR_2_LAN_SI
_collection: SENTINEL-3
metadata_mapping:
eo:cloud_cover: '{$.Attributes.cloudCover#not_available}'
platform:
- null
- '$.Attributes.platformSerialIdentifier.`sub(/^/, S3)`'
S3_LAN_LI:
product:type: SR_2_LAN_LI
_collection: SENTINEL-3
metadata_mapping:
eo:cloud_cover: '{$.Attributes.cloudCover#not_available}'
platform:
- null
- '$.Attributes.platformSerialIdentifier.`sub(/^/, S3)`'
S3_WAT:
product:type: SR_2_WAT___
_collection: SENTINEL-3
metadata_mapping:
eo:cloud_cover: '{$.Attributes.cloudCover#not_available}'
platform:
- null
- '$.Attributes.platformSerialIdentifier.`sub(/^/, S3)`'
# S3 OLCI
S3_EFR:
product:type: OL_1_EFR___
_collection: SENTINEL-3
metadata_mapping:
platform:
- null
- '$.Attributes.platformSerialIdentifier.`sub(/^/, S3)`'
S3_ERR:
product:type: OL_1_ERR___
_collection: SENTINEL-3
metadata_mapping:
platform:
- null
- '$.Attributes.platformSerialIdentifier.`sub(/^/, S3)`'
S3_OLCI_L2LRR:
product:type: OL_2_LRR___
_collection: SENTINEL-3
metadata_mapping:
platform:
- null
- '$.Attributes.platformSerialIdentifier.`sub(/^/, S3)`'
S3_OLCI_L2LFR:
product:type: OL_2_LFR___
_collection: SENTINEL-3
metadata_mapping:
platform:
- null
- '$.Attributes.platformSerialIdentifier.`sub(/^/, S3)`'
S3_OLCI_L2WRR:
product:type: OL_2_WRR___
_collection: SENTINEL-3
metadata_mapping:
platform:
- null
- '$.Attributes.platformSerialIdentifier.`sub(/^/, S3)`'
S3_OLCI_L2WFR:
product:type: OL_2_WFR___
_collection: SENTINEL-3
metadata_mapping:
platform:
- null
- '$.Attributes.platformSerialIdentifier.`sub(/^/, S3)`'
# S3 SLSTR
S3_SLSTR_L1RBT:
product:type: SL_1_RBT___
_collection: SENTINEL-3
metadata_mapping:
platform:
- null
- '$.Attributes.platformSerialIdentifier.`sub(/^/, S3)`'
S3_SLSTR_L2LST:
product:type: SL_2_LST___
_collection: SENTINEL-3
metadata_mapping:
platform:
- null
- '$.Attributes.platformSerialIdentifier.`sub(/^/, S3)`'
S3_SLSTR_L2WST:
product:type: SL_2_WST___
_collection: SENTINEL-3
metadata_mapping:
platform:
- null
- '$.Attributes.platformSerialIdentifier.`sub(/^/, S3)`'
S3_SLSTR_L2AOD:
product:type: SL_2_AOD___
_collection: SENTINEL-3
metadata_mapping:
platform:
- null
- '$.Attributes.platformSerialIdentifier.`sub(/^/, S3)`'
S3_SLSTR_L2FRP:
product:type: SL_2_FRP___
_collection: SENTINEL-3
metadata_mapping:
platform:
- null
- '$.Attributes.platformSerialIdentifier.`sub(/^/, S3)`'
# S3 SY
S3_SY_AOD:
product:type: SY_2_AOD___
_collection: SENTINEL-3
metadata_mapping:
platform:
- null
- '$.Attributes.platformSerialIdentifier.`sub(/^/, S3)`'
S3_SY_SYN:
product:type: SY_2_SYN___
_collection: SENTINEL-3
metadata_mapping:
platform:
- null
- '$.Attributes.platformSerialIdentifier.`sub(/^/, S3)`'
S3_SY_V10:
product:type: SY_2_V10___
_collection: SENTINEL-3
metadata_mapping:
platform:
- null
- '$.Attributes.platformSerialIdentifier.`sub(/^/, S3)`'
S3_SY_VG1:
product:type: SY_2_VG1___
_collection: SENTINEL-3
metadata_mapping:
platform:
- null
- '$.Attributes.platformSerialIdentifier.`sub(/^/, S3)`'
S3_SY_VGP:
product:type: SY_2_VGP___
_collection: SENTINEL-3
metadata_mapping:
platform:
- null
- '$.Attributes.platformSerialIdentifier.`sub(/^/, S3)`'
# S5P L1
S5P_L1B_IR_SIR:
product:type: L1B_IR_SIR
_collection: Sentinel-5P
S5P_L1B_IR_UVN:
product:type: L1B_IR_UVN
_collection: Sentinel-5P
S5P_L1B_RA_BD1:
product:type: L1B_RA_BD1
_collection: SENTINEL-5P
S5P_L1B_RA_BD2:
product:type: L1B_RA_BD2
_collection: SENTINEL-5P
S5P_L1B_RA_BD3:
product:type: L1B_RA_BD3
_collection: SENTINEL-5P
S5P_L1B_RA_BD4:
product:type: L1B_RA_BD4
_collection: SENTINEL-5P
S5P_L1B_RA_BD5:
product:type: L1B_RA_BD5
_collection: SENTINEL-5P
S5P_L1B_RA_BD6:
product:type: L1B_RA_BD6
_collection: SENTINEL-5P
S5P_L1B_RA_BD7:
product:type: L1B_RA_BD7
_collection: SENTINEL-5P
S5P_L1B_RA_BD8:
product:type: L1B_RA_BD8
_collection: SENTINEL-5P
# S5P L2
S5P_L2_NO2:
product:type: L2__NO2___
_collection: SENTINEL-5P
S5P_L2_CLOUD:
product:type: L2__CLOUD_
_collection: SENTINEL-5P
S5P_L2_O3:
product:type: L2__O3____
_collection: SENTINEL-5P
S5P_L2_CO:
product:type: L2__CO____
_collection: SENTINEL-5P
S5P_L2_AER_AI:
product:type: L2__AER_AI
_collection: SENTINEL-5P
S5P_L2_O3_PR:
product:type: L2__O3__PR
_collection: SENTINEL-5P
S5P_L2_O3_TCL:
product:type: L2__O3_TCL
_collection: Sentinel-5P
S5P_L2_AER_LH:
product:type: L2__AER_LH
_collection: SENTINEL-5P
S5P_L2_HCHO:
product:type: L2__HCHO__
_collection: SENTINEL-5P
S5P_L2_CH4:
product:type: L2__CH4___
_collection: SENTINEL-5P
S5P_L2_NP_BD3:
product:type: L2__NP_BD3
_collection: SENTINEL-5P
S5P_L2_NP_BD6:
product:type: L2__NP_BD6
_collection: SENTINEL-5P
S5P_L2_NP_BD7:
product:type: L2__NP_BD7
_collection: SENTINEL-5P
S5P_L2_SO2:
product:type: L2__SO2___
_collection: SENTINEL-5P
GENERIC_COLLECTION:
_collection: '{collection}'
---
!provider # MARK: theia
name: theia
priority: 0
description: Data Terra Theia, environmental thematic hub for land data access.
roles:
- host
url: https://api.datastore-mtd.theia.data-terra.org/
search: !plugin
type: StacSearch
api_endpoint: 'https://api.datastore-mtd.theia.data-terra.org/search'
timeout: 90
discover_collections:
fetch_url: 'https://api.datastore-mtd.theia.data-terra.org/collections?limit=50'
result_type: json
results_entry: '$.collections[?(@.links[*].rel=="items")]'
single_collection_fetch_url: 'https://api.datastore-mtd.theia.data-terra.org/collections/{_collection}'
single_collection_parsable_metadata:
platform: '{$.summaries.platform#csv_list}'
sort:
sort_by_default:
- !!python/tuple [start_datetime, ASC]
sort_param_mapping:
id: id
start_datetime: properties.datetime
created: properties.created
updated: properties.updated
eo:cloud_cover: properties.eo:cloud_cover
products:
GENERIC_COLLECTION:
_collection: '{collection}'
download: !plugin
type: HTTPDownload
auth: !plugin
type: SASAuth
matching_url: https://s3-data.meso.umontpellier.fr
auth_uri: 'https://signing.stac.teledetection.fr/sign?url={url}'
signed_url_key: href
ssl_verify: true
headers:
access-key: "{access_key}"
secret-key: "{secret_key}"
Provider configuration#
The plugin structure is reflected in the internal providers configuration file. Here is a sample:
provider_name:
priority: 1
products:
# List of supported collections
# This is a mapping containing all the information required by the search plugin class to perform its job.
# The mapping is available in the config attribute of the search plugin as config['products']
S2_MSI_L1C:
a-config-key-needed-by-search-plugin-to-search-this-collection: value
another-config-key: another-value
# Whether this collection is partially supported by this provider (the provider does not contain all the
# products of this type)
partial: True
...
search:
plugin: CustomSearchPluginClass
api_endpoint: https://mandatory.config.key/
a-key-conf-used-by-the-plugin-class-init-method: value
another-random-key: random-value
# A mapping between the search result of the provider and the eodag way of describing EO products (the keys are
# the same as in the OpenSearch specification)
metadata_mapping:
...
...
download:
plugin: CustomDownloadPlugin
# Same as with search for random config keys as needed by the plugin class
...
auth:
plugin: CustomAuthPlugin
# Same as with search for random config keys as needed by the plugin class
...
Note however, that for a provider which already has a Python library for accessing its products, the configuration varies a little bit. It does not have the ‘search’ and ‘download’ keys. Instead, there is a single ‘api’ key like this:
provider_name:
...
api:
plugin: ApiPluginClassName
...