Add a Provider#
eodag provides a set of plugins which don’t know anything about the
providers themselves, they just implement generic methods required to talk to different kinds of data catalog. For instance:
QueryStringSearch: Search plugin that implements a search protocol that relies on query stringsHTTPHeaderAuth: Authentication plugin that implements HTTP authentication using headersHTTPDownload: Download plugin that implements download over HTTP protocol
Configure a new provider#
The simplest way to add a new provider is to configure existing plugins. This approach requires to
provide the new provider’s configuration in a YAML format. The following example, extracted from
the STAC client page, shows how to add a new STAC provider:
tamn:
search:
type: StacSearch
api_endpoint: https://tamn.snapplanet.io/search
need_auth: false
products:
S2_MSI_L1C:
productType: S2
GENERIC_PRODUCT_TYPE:
productType: '{productType}'
download:
type: AwsDownload
base_uri: https://tamn.snapplanet.io
flatten_top_dirs: True
auth:
type: AwsAuth
credentials:
aws_access_key_id: PLEASE_CHANGE_ME
aws_secret_access_key: PLEASE_CHANGE_ME
It configures the following existing plugins: StacSearch (search),
AwsAuth (authentication) and AwsDownload (download).
Of course, it is also necessary to know how to configure these plugins (which parameters they take, what values they can have, etc.).
You can get some inspiration from the Providers pre-configuration section by analysing how eodag configures the providers it comes installed with.
Add more plugins#
eodag is a plugin-oriented framework which means it can be easily extended. If the plugins it offers are not sufficient for your
own needs (i.e. getting data from a provider not supported by eodag), you should then write your own plugins (possibly by extending one of provided by eodag)
and configure them. What you are the most likely to be willing to do is either to develop a new Search plugin or an
Api plugin (e.g. to create an interface with another program).
eodag-sentinelsat is a good example of an Api plugin. It creates an interface with the sentinalsat library to search and download products from SciHub.
See more details about how to create a new plugin in this dedicated section.
Providers pre-configuration#
All the providers are pre-configured in eodag in a YAML file.
Click on the link below to display its full content.
providers.yml
---
!provider # MARK: usgs
name: usgs
priority: 0
description: U.S geological survey catalog for Landsat products
roles:
- host
url: https://earthexplorer.usgs.gov/
api: !plugin
type: UsgsApi
need_auth: true
google_base_url: 'http://storage.googleapis.com/earthengine-public/landsat/'
pagination:
max_items_per_page: 5000
total_items_nb_key_path: '$.totalHits'
metadata_mapping:
id: '$.displayId'
geometry: '$.spatialBounds'
productType: '$.productType'
title: '$.displayId'
abstract: '$.summary'
cloudCover: '$.cloudCover'
startTimeFromAscendingNode: '$.temporalCoverage.startDate'
completionTimeFromAscendingNode: '$.temporalCoverage.endDate'
publicationDate: '$.publishDate'
thumbnail: '$.browse[0].thumbnailPath'
quicklook: '$.browse[0].browsePath'
storageStatus: '{$.available#get_group_name((?P<ONLINE>True)|(?P<OFFLINE>False))}'
downloadLink: 'https://earthexplorer.usgs.gov/download/external/options/{productType}/{entityId}/M2M/'
# metadata needed for download
entityId: '$.entityId'
productId: '$.id'
extract: True
order_enabled: true
products:
# datasets list http://kapadia.github.io/usgs/_sources/reference/catalog/ee.txt may be outdated
# see also https://dds.cr.usgs.gov/ee-data/coveragemaps/shp/ee/
LANDSAT_C2L1:
dataset: landsat_ot_c2_l1
outputs_extension: .tar.gz
LANDSAT_C2L2:
dataset: landsat_ot_c2_l2
outputs_extension: .tar.gz
LANDSAT_TM_C1:
dataset: landsat_tm_c1
outputs_extension: .tar.gz
LANDSAT_TM_C2L1:
dataset: landsat_tm_c2_l1
outputs_extension: .tar.gz
LANDSAT_TM_C2L2:
dataset: landsat_tm_c2_l2
outputs_extension: .tar.gz
LANDSAT_ETM_C1:
dataset: landsat_etm_c1
outputs_extension: .tar.gz
LANDSAT_ETM_C2L1:
dataset: landsat_etm_c2_l1
outputs_extension: .tar.gz
LANDSAT_ETM_C2L2:
dataset: landsat_etm_c2_l2
outputs_extension: .tar.gz
S2_MSI_L1C:
dataset: SENTINEL_2A
outputs_extension: .zip
GENERIC_PRODUCT_TYPE:
dataset: '{productType}'
---
!provider # MARK: aws_eos
name: aws_eos
priority: 0
description: EOS search for Amazon public datasets
roles:
- host
url: https://developers.eos.com/datasets_description.html
search: !plugin
type: PostJsonSearch
api_endpoint: 'https://gate.eos.com/api/lms/search/v2/{collection}?api_key={apikey}'
need_auth: true
auth_error_code: 403
results_entry: 'results'
ssl_verify: true
pagination:
next_page_query_obj: '{{"limit":{items_per_page},"page":{page}}}'
total_items_nb_key_path: '$.meta.found'
# 2021/04/28: aws_eos doesn't specify a limit in its docs. It says that the default
# value is 500 (https://doc.eos.com/search.api/#single-dataset-search).
# Let's set it to this value for now
max_items_per_page: 500
query_params_key: 'search'
discover_metadata:
auto_discovery: true
metadata_pattern: '^[a-zA-Z0-9_]+$'
search_param: '{{{{"search":{{{{"{metadata}":"{{{metadata}}}" }}}} }}}}'
metadata_path: '$.*'
metadata_mapping:
# landsat8_downloadLink : 's3://landsat-pds/c{storedInCollection}/L8/{path}/{row}/{productID}'
geometry:
- '{{"search":{{"shape": {geometry#to_geojson} }} }}'
- '$.dataGeometry'
# storageStatus set to ONLINE for consistency between providers
storageStatus: '{$.null#replace_str("Not Available","ONLINE")}'
products:
CBERS4_PAN10M_L2:
instrument: PAN10M
collection: cbers4
processingLevel: 2
metadata_mapping:
# OpenSearch Parameters for Collection Search (Table 3)
productType: '$.null'
platformSerialIdentifier:
- '{{"search":{{"satelliteName":"{platformSerialIdentifier}" }} }}'
- '$.satelliteName'
instrument:
- '{{"search":{{"sensor":"{instrument}" }} }}'
- '$.sensor'
processingLevel:
- '{{"search":{{"processingLevel":"{processingLevel}" }} }}'
- '$.processingLevel'
# INSPIRE obligated OpenSearch Parameters for Collection Search (Table 4)
title:
- '{{"search":{{"sceneID":"{title}" }} }}'
- '$.sceneID'
# OpenSearch Parameters for Acquistion Parameters Search (Table 6)
startTimeFromAscendingNode:
- '{{"search":{{"date":{{"from":"{startTimeFromAscendingNode}"}} }} }}'
- '$.date'
completionTimeFromAscendingNode:
- '{{"search":{{"date":{{"to":"{completionTimeFromAscendingNode}"}} }} }}'
- '$.date'
illuminationAzimuthAngle:
- '{{"search":{{"sunAzimuth":"{illuminationAzimuthAngle}" }} }}'
- '$.sunAzimuth'
illuminationElevationAngle:
- '{{"search":{{"sunElevation":"{illuminationElevationAngle}" }} }}'
- '$.sunElevation'
# Custom parameters (not defined in the base document referenced above)
awsPath:
- '{{"search":{{"downloadUrl":"{awsPath}" }} }}'
- '$.downloadUrl'
downloadLink: 's3://cbers-pds/{awsPath}'
metaDownloadLink: 's3://cbers-meta-pds/{awsPath}'
previewBaseName: '{$.sceneID#replace_str("_L2","")}'
thumbnail: 'https://s3.amazonaws.com/cbers-meta-pds/{awsPath}/{previewBaseName}_small.jpeg'
quicklook: 'https://s3.amazonaws.com/cbers-meta-pds/{awsPath}/{previewBaseName}.jpg'
id:
- '{{"search":{{"sceneID":"{id}" }} }}'
- '{title}'
CBERS4_PAN10M_L4:
instrument: PAN10M
collection: cbers4
processingLevel: 4
metadata_mapping_from_product: CBERS4_PAN10M_L2
metadata_mapping:
# Custom parameters (not defined in the base document referenced above)
previewBaseName: '{$.sceneID#replace_str("_L4","")}'
thumbnail: 'https://s3.amazonaws.com/cbers-meta-pds/{awsPath}/{previewBaseName}_small.jpeg'
quicklook: 'https://s3.amazonaws.com/cbers-meta-pds/{awsPath}/{previewBaseName}.jpg'
CBERS4_PAN5M_L2:
instrument: PAN5M
collection: cbers4
processingLevel: 2
metadata_mapping_from_product: CBERS4_PAN10M_L2
CBERS4_PAN5M_L4:
instrument: PAN5M
collection: cbers4
processingLevel: 4
metadata_mapping_from_product: CBERS4_PAN10M_L2
metadata_mapping:
# Custom parameters (not defined in the base document referenced above)
previewBaseName: '{$.sceneID#replace_str("_L4","")}'
thumbnail: 'https://s3.amazonaws.com/cbers-meta-pds/{awsPath}/{previewBaseName}_small.jpeg'
quicklook: 'https://s3.amazonaws.com/cbers-meta-pds/{awsPath}/{previewBaseName}.jpg'
CBERS4_MUX_L2:
instrument: MUX
collection: cbers4
processingLevel: 2
metadata_mapping_from_product: CBERS4_PAN10M_L2
CBERS4_MUX_L4:
instrument: MUX
collection: cbers4
processingLevel: 4
metadata_mapping_from_product: CBERS4_PAN10M_L2
metadata_mapping:
# Custom parameters (not defined in the base document referenced above)
previewBaseName: '{$.sceneID#replace_str("_L4","")}'
thumbnail: 'https://s3.amazonaws.com/cbers-meta-pds/{awsPath}/{previewBaseName}_small.jpeg'
quicklook: 'https://s3.amazonaws.com/cbers-meta-pds/{awsPath}/{previewBaseName}.jpg'
CBERS4_AWFI_L2:
instrument: AWFI
collection: cbers4
processingLevel: 2
metadata_mapping_from_product: CBERS4_PAN10M_L2
CBERS4_AWFI_L4:
instrument: AWFI
collection: cbers4
processingLevel: 4
metadata_mapping_from_product: CBERS4_PAN10M_L2
metadata_mapping:
# Custom parameters (not defined in the base document referenced above)
previewBaseName: '{$.sceneID#replace_str("_L4","")}'
thumbnail: 'https://s3.amazonaws.com/cbers-meta-pds/{awsPath}/{previewBaseName}_small.jpeg'
quicklook: 'https://s3.amazonaws.com/cbers-meta-pds/{awsPath}/{previewBaseName}.jpg'
L8_OLI_TIRS_C1L1:
collection: landsat8
onAmazon: true
metadata_mapping:
# OpenSearch Parameters for Collection Search (Table 3)
productType: '$.null'
platform:
- '{{"search":{{"satelliteName":"{platform}" }} }}'
- '$.satelliteName'
instrument:
- '{{"search":{{"sensor":"{instrument}" }} }}'
- '$.sensor'
# INSPIRE obligated OpenSearch Parameters for Collection Search (Table 4)
title:
- '{{"search":{{"productID":"{title}" }} }}'
- '$.productID'
# OpenSearch Parameters for Product Search (Table 5)
cloudCover:
- '{{"search":{{"cloudCoverage":"{cloudCover}" }} }}'
- '$.cloudCoverage'
# OpenSearch Parameters for Acquistion Parameters Search (Table 6)
availabilityTime:
- '{{"search":{{"IndexingTime":{{"from":"{availabilityTime}"}} }} }}'
- '$.IndexingTime'
acquisitionStation:
- '{{"search":{{"receivingStation":{{"from":"{acquisitionStation}"}} }} }}'
- '$.receivingStation'
startTimeFromAscendingNode:
- '{{"search":{{"date":{{"from":"{startTimeFromAscendingNode}"}} }} }}'
- '$.sceneStartTime'
completionTimeFromAscendingNode:
- '{{"search":{{"date":{{"to":"{completionTimeFromAscendingNode}"}} }} }}'
- '$.sceneStopTime'
illuminationAzimuthAngle:
- '{{"search":{{"sunAzimuth":"{illuminationAzimuthAngle}" }} }}'
- '$.sunAzimuth'
illuminationElevationAngle:
- '{{"search":{{"sunElevation":"{illuminationElevationAngle}" }} }}'
- '$.sunElevation'
# Custom parameters (not defined in the base document referenced above)
onAmazon:
- '{{"search":{{"onAmazon":"{onAmazon}" }} }}'
- '$.onAmazon'
path:
- '{{"search":{{"path":"{path}" }} }}'
- '$.path'
row:
- '{{"search":{{"row":"{row}" }} }}'
- '$.row'
downloadLink: 's3://landsat-pds/c1/L8/{path:03.0f}/{row:03.0f}/{title}/'
thumbnail: '$.thumbnail'
quicklook: 'https://landsat-pds.s3.amazonaws.com/c1/L8/{path:03.0f}/{row:03.0f}/{title}/{title}_thumb_large.jpg'
id:
- '{{"search":{{"productID":"{id}" }} }}'
- '{title}'
MODIS_MCD43A4:
collection: modis
metadata_mapping:
# OpenSearch Parameters for Collection Search (Table 3)
productType: '$.null'
instrument:
- '{{"search":{{"satelliteName":"{instrument}" }} }}'
- '$.satelliteName'
# INSPIRE obligated OpenSearch Parameters for Collection Search (Table 4)
title:
- '{{"search":{{"sceneID":"{title}" }} }}'
- '$.sceneID'
# OpenSearch Parameters for Product Search (Table 5)
cloudCover:
- '{{"search":{{"cloudCoverage":"{cloudCover}" }} }}'
- '$.cloudCoverage'
# OpenSearch Parameters for Acquistion Parameters Search (Table 6)
startTimeFromAscendingNode:
- '{{"search":{{"date":{{"from":"{startTimeFromAscendingNode}"}} }} }}'
- '$.BeginningDateTime'
completionTimeFromAscendingNode:
- '{{"search":{{"date":{{"to":"{completionTimeFromAscendingNode}"}} }} }}'
- '$.EndingDateTime'
# Custom parameters (not defined in the base document referenced above)
verticalTileNumber:
- '{{"search":{{"verticalTileNumber":"{verticalTileNumber}" }} }}'
- '$.verticalTileNumber'
horizontalTileNumber:
- '{{"search":{{"horizontalTileNumber":"{horizontalTileNumber}" }} }}'
- '$.horizontalTileNumber'
doyDate: '{$.sceneID#slice_str(9,16,1)}'
downloadLink: 's3://modis-pds/MCD43A4.006/{horizontalTileNumber:02.0f}/{verticalTileNumber:02.0f}/{doyDate}/'
thumbnail: '$.thumbnail'
quicklook: '$.thumbnail'
id:
- '{{"search":{{"sceneID":"{id}" }} }}'
- '{title}'
NAIP:
collection: naip
metadata_mapping:
# OpenSearch Parameters for Collection Search (Table 3)
productType: '$.null'
platformSerialIdentifier:
- '{{"search":{{"satelliteName":"{platformSerialIdentifier}" }} }}'
- '$.satelliteName'
# INSPIRE obligated OpenSearch Parameters for Collection Search (Table 4)
title:
- '{{"search":{{"sceneID":"{title}" }} }}'
- '$.sceneID'
# OpenSearch Parameters for Acquistion Parameters Search (Table 6)
startTimeFromAscendingNode:
- '{{"search":{{"date":{{"from":"{startTimeFromAscendingNode}"}} }} }}'
- '$.date'
completionTimeFromAscendingNode:
- '{{"search":{{"date":{{"to":"{completionTimeFromAscendingNode}"}} }} }}'
- '$.date'
# Custom parameters (not defined in the base document referenced above)
awsPath:
- '{{"search":{{"awsPath":"{awsPath}" }} }}'
- '$.awsPath'
downloadLink: 's3://naip-analytic/{awsPath}'
id:
- '{{"search":{{"sceneID":"{id}" }} }}'
- '{title}'
S1_SAR_GRD:
productType: GRD
collection: sentinel1
metadata_mapping:
cloudCover: '$.null'
# OpenSearch Parameters for Collection Search (Table 3)
platformSerialIdentifier:
- '{{"search":{{"missionId":"{platformSerialIdentifier}" }} }}'
- '$.missionId'
# INSPIRE obligated OpenSearch Parameters for Collection Search (Table 4)
title:
- '{{"search":{{"sceneID":"{title}" }} }}'
- '$.sceneID'
# OpenSearch Parameters for Product Search (Table 5)
orbitNumber:
- '{{"search":{{"absoluteOrbitNumber":"{orbitNumber}" }} }}'
- '$.absoluteOrbitNumber'
orbitDirection:
- '{{"search":{{"passDirection":"{orbitDirection}" }} }}'
- '$.passDirection'
sensorMode:
- '{{"search":{{"mode":"{sensorMode}" }} }}'
- '$.mode'
# OpenSearch Parameters for Acquistion Parameters Search (Table 6)
startTimeFromAscendingNode:
- '{{"search":{{"date":{{"from":"{startTimeFromAscendingNode}"}} }} }}'
- '$.date'
completionTimeFromAscendingNode:
- '{{"search":{{"date":{{"to":"{completionTimeFromAscendingNode}"}} }} }}'
- '$.date'
polarizationMode:
- '{{"search":{{"polarization":"{polarizationMode}" }} }}'
- '$.polarization'
# Custom parameters (not defined in the base document referenced above)
awsPath:
- '{{"search":{{"awsPath":"{awsPath}" }} }}'
- '$.awsPath'
downloadLink: 's3://sentinel-s1-l1c/{awsPath}'
thumbnail: 'https://render.eosda.com/S1/thumb/{title}.png'
quicklook: 'https://render.eosda.com/S1/thumb/{title}.png'
id:
- '{{"search":{{"sceneID":"{id}" }} }}'
- '{title}'
S2_MSI_L1C:
collection: sentinel2
metadata_mapping:
# OpenSearch Parameters for Collection Search (Table 3)
productType: '$.null'
platformSerialIdentifier:
- '{{"search":{{"satelliteName":"{platformSerialIdentifier}" }} }}'
- '$.satelliteName'
# INSPIRE obligated OpenSearch Parameters for Collection Search (Table 4)
title:
- '{{"search":{{"productName":"{title}" }} }}'
- '$.productName'
# OpenSearch Parameters for Product Search (Table 5)
cloudCover:
- '{{"search":{{"cloudCoverage":"{cloudCover}" }} }}'
- '$.cloudCoverage'
# OpenSearch Parameters for Acquistion Parameters Search (Table 6)
startTimeFromAscendingNode:
- '{{"search":{{"date":{{"from":"{startTimeFromAscendingNode}"}} }} }}'
- '$.timestamp'
completionTimeFromAscendingNode:
- '{{"search":{{"date":{{"to":"{completionTimeFromAscendingNode}"}} }} }}'
- '$.timestamp'
illuminationAzimuthAngle:
- '{{"search":{{"azimuthAngle":"{illuminationAzimuthAngle}" }} }}'
- '$.azimuthAngle'
illuminationZenithAngle:
- '{{"search":{{"zenithAngle":"{illuminationZenithAngle}" }} }}'
- '$.zenithAngle'
# Custom parameters (not defined in the base document referenced above)
originalTileMeta: '{$.originalTileMeta#replace_str("sentinel-s2-l1c.s3.amazonaws.com","roda.sentinel-hub.com/sentinel-s2-l1c")}'
thumbnail: '{$.thumbnail#replace_str("sentinel-s2-l1c.s3.amazonaws.com","roda.sentinel-hub.com/sentinel-s2-l1c")}'
awsPath:
- '{{"search":{{"awsPath":"{awsPath}" }} }}'
- '$.awsPath'
downloadLink: 's3://sentinel-s2-l1c/{awsPath}'
productMetaLink: '{$.productMetaLink#replace_str("sentinel-s2-l1c.s3.amazonaws.com","roda.sentinel-hub.com/sentinel-s2-l1c")}'
productPath:
- '{{"search":{{"productPath":"{productPath}" }} }}'
- '$.productPath'
id:
- '{{"search":{{"productName":"{id}" }} }}'
- '{title}'
processedL2A: '$.null'
awsPathL2A: '$.null'
S2_MSI_L2A:
collection: sentinel2
processedL2A: true
# specific QueryStringSearch usage for these parameters (replaces current query)
specific_qssearch:
parameters:
- title
- id
results_entry: '$'
collection:
- tileInfo
- productInfo
merge_responses: true
metadata_mapping:
title:
- 'title'
- '$.name'
id:
- 'title'
- '{title}'
awsPathL2A: '$.tiles[0].path'
downloadLink: 's3://sentinel-s2-l2a/{awsPathL2A}'
productPath: '$.path'
startTimeFromAscendingNode: '$.timestamp'
completionTimeFromAscendingNode: '$.timestamp'
geometry: '$.tileDataGeometry'
productInfo: 'https://roda.sentinel-hub.com/sentinel-s2-l2a/{awsPathL2A}/productInfo.json'
originalSceneID: '$.tiles[0].datastrip.id'
metadata_mapping:
# OpenSearch Parameters for Collection Search (Table 3)
productType: '$.null'
platformSerialIdentifier:
- '{{"search":{{"satelliteName":"{platformSerialIdentifier}" }} }}'
- '$.satelliteName'
# INSPIRE obligated OpenSearch Parameters for Collection Search (Table 4)
title: '{$.productName#fake_l2a_title_from_l1c}'
# OpenSearch Parameters for Product Search (Table 5)
cloudCover:
- '{{"search":{{"cloudCoverage":"{cloudCover}" }} }}'
- '$.cloudCoverage'
# OpenSearch Parameters for Acquistion Parameters Search (Table 6)
startTimeFromAscendingNode:
- '{{"search":{{"date":{{"from":"{startTimeFromAscendingNode}"}} }} }}'
- '$.timestamp'
completionTimeFromAscendingNode:
- '{{"search":{{"date":{{"to":"{completionTimeFromAscendingNode}"}} }} }}'
- '$.timestamp'
illuminationAzimuthAngle:
- '{{"search":{{"azimuthAngle":"{illuminationAzimuthAngle}" }} }}'
- '$.azimuthAngle'
illuminationZenithAngle:
- '{{"search":{{"zenithAngle":"{illuminationZenithAngle}" }} }}'
- '$.zenithAngle'
# Custom parameters (not defined in the base document referenced above)
originalTileMeta: '{$.originalTileMeta#replace_str("sentinel-s2-l1c.s3.amazonaws.com","roda.sentinel-hub.com/sentinel-s2-l2a")}'
thumbnail: '{$.thumbnail#replace_str("sentinel-s2-l1c.s3.amazonaws.com","roda.sentinel-hub.com/sentinel-s2-l1c")}'
quicklook: '{thumbnail}'
downloadLink: 's3://sentinel-s2-l2a/{awsPathL2A}'
awsPath: '$.null'
productMetaLink: '$.null'
productPath: '$.null'
productInfo: 'https://roda.sentinel-hub.com/sentinel-s2-l2a/{awsPathL2A}/productInfo.json'
id:
- '{id#s2msil2a_title_to_aws_productinfo}'
- '{title}'
processedL2A:
- '{{"search":{{"processedL2A":"{processedL2A}" }} }}'
- '$.processedL2A'
awsPathL2A:
- '{{"search":{{"awsPathL2A":"{awsPathL2A}" }} }}'
- '$.awsPathL2A'
download: !plugin
type: AwsDownload
requester_pays: True
ssl_verify: true
products:
CBERS4_MUX_L2:
default_bucket: 'cbers-pds'
complementary_url_key:
- metaDownloadLink
CBERS4_AWFI_L2:
complementary_url_key:
- metaDownloadLink
CBERS4_PAN5M_L2:
complementary_url_key:
- metaDownloadLink
CBERS4_PAN10M_L2:
complementary_url_key:
- metaDownloadLink
S1_SAR_GRD:
default_bucket: 'sentinel-s1-l1c'
build_safe: true
S2_MSI_L1C:
default_bucket: 'sentinel-s2-l1c'
build_safe: true
complementary_url_key:
- productPath
S2_MSI_L2A:
default_bucket: 'sentinel-s2-l2a'
build_safe: true
fetch_metadata:
fetch_url: '{productInfo}'
fetch_format: json
update_metadata:
title: '$.name'
id: '{title}'
productPath: '$.path'
complementary_url_key:
- productPath
auth: !plugin
type: AwsAuth
ssl_verify: true
---
!provider # MARK: theia
name: theia
priority: 0
description: French National Space Agency (CNES) catalog for Sentinel 2 products, Pleiades and Landsat products
roles:
- host
url: https://theia.cnes.fr/atdistrib/rocket/
search: !plugin
type: QueryStringSearch
api_endpoint: 'https://theia.cnes.fr/atdistrib/resto2/api/collections/{collection}/search.json'
need_auth: false
ssl_verify: true
pagination:
next_page_url_tpl: '{url}?{search}&maxRecords={items_per_page}&page={page}'
total_items_nb_key_path: '$.properties.totalResults'
# 2021/03/19: Returns a 400 error code if greater than 500.
max_items_per_page: 500
discover_metadata:
auto_discovery: true
metadata_pattern: '^(?!collection)[a-zA-Z0-9_]+$'
search_param: '{metadata}={{{metadata}}}'
metadata_path: '$.properties.*'
metadata_mapping:
# Opensearch resource identifier within the search engine context (in our case
# within the context of the data provider)
uid: '$.id'
# OpenSearch Parameters for Collection Search (Table 3)
productType:
- productType
- '$.properties.productType'
platform:
- platform
- '$.properties.platform'
platformSerialIdentifier:
- 'platform={platformSerialIdentifier}'
- '$.properties.platform'
instrument:
- instrument
- '$.properties.instrument'
processingLevel:
- processingLevel
- '$.properties.processingLevel'
# INSPIRE obligated OpenSearch Parameters for Collection Search (Table 4)
title: '$.properties.title'
keyword: '$.properties.keywords'
resolution: '$.properties.resolution'
organisationName: '$.properties.organisationName'
publicationDate: '$.properties.published'
accessConstraint: '$.properties.license'
# OpenSearch Parameters for Product Search (Table 5)
parentIdentifier: '$.properties.parentIdentifier'
orbitNumber: '$.properties.orbitNumber'
cloudCover:
- 'cloudCover=[0,{cloudCover}]'
- '$.properties.cloudCover'
snowCover: '$.properties.snowCover'
productVersion: '$.properties.version'
creationDate: '$.properties.productionDate'
modificationDate: '$.properties.modificationDate'
processingDate: '$.properties.processingDate'
sensorMode: '$.properties.sensorMode'
archivingCenter: '$.properties.archivingCenter'
processingMode: '$.properties.processingMode'
# OpenSearch Parameters for Acquistion Parameters Search (Table 6)
availabilityTime: '$.properties.availabilityTime'
acquisitionStation: '$.properties.acquisitionStation'
acquisitionSubType: '$.properties.acquisitionSubType'
startTimeFromAscendingNode:
- 'startDate={startTimeFromAscendingNode#to_iso_date}'
- '$.properties.startDate'
completionTimeFromAscendingNode:
- 'completionDate={completionTimeFromAscendingNode#to_iso_date}'
- '$.properties.completionDate'
illuminationAzimuthAngle: '$.properties.illuminationAzimuthAngle'
illuminationZenithAngle: '$.properties.illuminationZenithAngle'
illuminationElevationAngle: '$.properties.illuminationElevationAngle'
polarizationMode: '$.properties.polarizationMode'
polarizationChannels: '$.properties.polarisationChannels'
antennaLookDirection: '$.properties.antennaLookDirection'
minimumIncidenceAngle: '$.properties.minimumIncidenceAngle'
maximumIncidenceAngle: '$.properties.maximumIncidenceAngle'
dopplerFrequency: '$.properties.dopplerFrequency'
incidenceAngleVariation: '$.properties.incidenceAngleVariation'
# Custom parameters (not defined in the base document referenced above)
id: '$.properties.productIdentifier'
tileIdentifier:
- 'location=T{tileIdentifier#replace_str(r"^T(.*)$",r"\1")}'
- '{$.properties.location#replace_str(r"^T(.*)$",r"\1")}'
# The geographic extent of the product
geometry:
- 'geometry={geometry#to_rounded_wkt}'
- '$.geometry'
# The url of the quicklook
quicklook: '$.properties.quicklook'
# The url to download the product "as is" (literal or as a template to be completed either after the search result
# is obtained from the provider or during the eodag download phase)
downloadLink: '$.properties.services.download.url'
# Additional metadata provided by the providers but that don't appear in the reference spec
# Or has a different signification for the provider
thumbnail: '$.properties.thumbnail'
# storageStatus set to ONLINE for consistency between providers
storageStatus: '{$.null#replace_str("Not Available","ONLINE")}'
products:
S2_MSI_L2A_MAJA:
processingLevel: LEVEL2A
collection: SENTINEL2
S2_MSI_L2B_MAJA_SNOW:
processingLevel: L2B-SNOW
collection: Snow
S2_MSI_L2B_MAJA_WATER:
processingLevel: L2B-WATER
collection: WaterQual
S2_MSI_L3A_WASP:
processingLevel: LEVEL3A
collection: SENTINEL2
SPOT_SWH:
collection: SPOTWORLDHERITAGE
processingLevel: LEVEL1C
SPOT_SWH_OLD:
collection: SWH1
processingLevel: LEVEL1C
SPOT5_SPIRIT:
collection: Spirit
processingLevel: 1A
VENUS_L1C:
collection: VENUS
processingLevel: LEVEL1C
VENUS_L2A_MAJA:
collection: VENUS
processingLevel: LEVEL2A
VENUS_L3A_MAJA:
collection: VENUS
processingLevel: LEVEL3A
L8_REFLECTANCE:
collection: LANDSAT
productType: REFLECTANCE
L57_REFLECTANCE:
collection: Landsat57
productType: REFLECTANCE
PLD_PAN:
collection: Pleiades
productType: Panchromatique+(Pan)
PLD_XS:
collection: Pleiades
productType: Multispectral+(XS)
PLD_BUNDLE:
collection: Pleiades
productType: Bundle+(Pan,+XS)
PLD_PANSHARPENED:
collection: Pleiades
productType: Pansharpened+(Pan%2BXS)
OSO:
collection: OSO
GENERIC_PRODUCT_TYPE:
productType: '{productType}'
collection: '{collection}'
processingLevel: '{processingLevel}'
download: !plugin
type: HTTPDownload
base_uri: 'https://theia.cnes.fr/atdistrib/resto2'
extract: true
archive_depth: 2
order_enabled: true
auth_error_code: 403
ssl_verify: true
dl_url_params:
issuerId: theia
auth: !plugin
type: TokenAuth
auth_uri: 'https://theia.cnes.fr/atdistrib/services/authenticate/'
ssl_verify: true
---
!provider # MARK: peps
name: peps
priority: 1
description: |
The PEPS platform, the French "mirror site", redistributes the products of Sentinel satellites,
S1A, S1B, S2A and S2B, S3A and S3B from COPERNICUS, the European system for the Earth monitoring.
roles:
- host
url: https://peps.cnes.fr
search: !plugin
type: QueryStringSearch
api_endpoint: 'https://peps.cnes.fr/resto/api/collections/{collection}/search.json'
need_auth: false
ssl_verify: true
pagination:
next_page_url_tpl: '{url}?{search}&maxRecords={items_per_page}&page={page}'
total_items_nb_key_path: '$.properties.totalResults'
# 2021/03/19: 500 is the max, no error if greater
max_items_per_page: 500
discover_metadata:
auto_discovery: true
metadata_pattern: '^(?!collection)[a-zA-Z0-9_]+$'
search_param: '{metadata}={{{metadata}}}'
metadata_path: '$.properties.*'
metadata_mapping:
# Opensearch resource identifier within the search engine context (in our case
# within the context of the data provider)
uid: '$.id'
# OpenSearch Parameters for Collection Search (Table 3)
productType:
- productType
- '$.properties.productType'
platform: '$.properties.collection'
platformSerialIdentifier:
- platform
- '$.properties.platform'
instrument:
- instrument
- '$.properties.instrument'
processingLevel:
- processingLevel
- '$.properties.processingLevel'
# INSPIRE obligated OpenSearch Parameters for Collection Search (Table 4)
title: '$.properties.title'
topicCategory: '$.properties.topicCategory'
keyword: '$.properties.keywords'
abstract: '$.properties.description'
resolution:
- 'resolution'
- '$.properties.resolution'
organisationName:
- 'organisationName'
- '$.properties.organisationName'
publicationDate: '$.properties.published'
accessConstraint: '$.properties.license'
# OpenSearch Parameters for Product Search (Table 5)
parentIdentifier:
- 'parentIdentifier'
- '$.properties.parentIdentifier'
acquisitionType: '$.properties.acquisitionType'
orbitNumber:
- 'orbitNumber'
- '$.properties.orbitNumber'
orbitDirection:
- 'orbitDirection'
- '$.properties.orbitDirection'
swathIdentifier:
- 'swath'
- '$.properties.swath'
cloudCover:
- 'cloudCover=[0,{cloudCover}]'
- '$.properties.cloudCover'
snowCover:
- 'snowCover=[0,{snowCover}]'
- '$.properties.snowCover'
productVersion: '$.properties.version'
productQualityStatus: '$.properties.onlineQualityCheck'
processorName: '$.properties.processingName'
processingCenter: '$.properties.processingCenter'
creationDate: '$.properties.dhusIngestDate'
modificationDate: '$.properties.updated'
sensorMode:
- 'sensorMode'
- '$.properties.sensorMode'
# OpenSearch Parameters for Acquistion Parameters Search (Table 6)
startTimeFromAscendingNode:
- startDate
- '$.properties.startDate'
completionTimeFromAscendingNode:
- completionDate
- '$.properties.completionDate'
polarizationMode:
- 'polarisation'
- '$.properties.polarisation'
# Custom parameters (not defined in the base document referenced above)
id:
- 'productIdentifier={id#remove_extension}'
- '$.properties.productIdentifier'
tileIdentifier:
- 'tileid'
- '$.properties.mgrs'
# The geographic extent of the product
geometry:
- 'geometry={geometry#to_rounded_wkt}'
- '$.geometry'
# The url of the quicklook
quicklook: '$.properties.quicklook'
# The url to download the product "as is" (literal or as a template to be completed either after the search result
# is obtained from the provider or during the eodag download phase)
downloadLink: '$.properties.services.download.url'
# storageStatus: must be one of ONLINE, STAGING, OFFLINE
storageStatus: '{$.properties.storage.mode#get_group_name((?P<ONLINE>disk|tier2)|(?P<STAGING>staging)|(?P<OFFLINE>unknown|tape|tier3))}'
# Additional metadata provided by the providers but that don't appear in the reference spec
thumbnail: '$.properties.thumbnail'
products:
S1_SAR_OCN:
productType: OCN
collection: S1
metadata_mapping:
cloudCover: '$.null'
S1_SAR_GRD:
productType: GRD
collection: S1
metadata_mapping:
cloudCover: '$.null'
S1_SAR_SLC:
productType: SLC
collection: S1
metadata_mapping:
cloudCover: '$.null'
S2_MSI_L1C:
collection: S2ST
productType: S2MSI1C
GENERIC_PRODUCT_TYPE:
productType: '{productType}'
collection: '{collection}'
download: !plugin
type: HTTPDownload
base_uri: 'https://peps.cnes.fr/resto'
extract: true
archive_depth: 2
order_enabled: true
auth_error_code: 401
ssl_verify: true
dl_url_params:
issuerId: peps
auth: !plugin
type: GenericAuth
auth_uri: 'https://peps.cnes.fr/resto/api/users/connect'
ssl_verify: true
---
!provider # MARK: creodias
name: creodias
priority: 0
description: CloudFerro DIAS
roles:
- host
url: https://creodias.eu/
search: !plugin
type: QueryStringSearch
api_endpoint: 'http://datahub.creodias.eu/resto/api/collections/{collection}/search.json'
need_auth: false
timeout: 60
ssl_verify: true
pagination:
next_page_url_tpl: '{url}?{search}&maxRecords={items_per_page}&page={page}&exactCount=1'
total_items_nb_key_path: '$.properties.totalResults'
max_items_per_page: 1_000
sort:
sort_by_default:
- !!python/tuple [startTimeFromAscendingNode, ASC]
sort_by_tpl: '&sortParam={sort_param}&sortOrder={sort_order}'
sort_param_mapping:
startTimeFromAscendingNode: startDate
completionTimeFromAscendingNode: completionDate
publicationDate: published
sort_order_mapping:
ascending: asc
descending: desc
max_sort_params: 1
discover_metadata:
auto_discovery: true
metadata_pattern: '^(?!collection)[a-zA-Z0-9]+$'
search_param: '{metadata}={{{metadata}}}'
metadata_path: '$.properties.*'
discover_product_types:
fetch_url: null
metadata_mapping:
uid: '$.id'
productType:
- productType
- '$.properties.productType'
platform: '$.properties.collection'
platformSerialIdentifier:
- platform
- '$.properties.platform'
instrument:
- instrument
- '$.properties.instrument'
processingLevel:
- processingLevel
- '$.properties.processingLevel'
# INSPIRE obligated OpenSearch Parameters for Collection Search (Table 4)
title: '{$.properties.title#remove_extension}'
topicCategory: '$.properties.topicCategory'
keyword: '$.properties.keywords'
abstract: '$.properties.description'
resolution:
- 'resolution'
- '$.properties.resolution'
organisationName:
- 'organisationName'
- '$.properties.organisationName'
publicationDate: '$.properties.published'
accessConstraint: '$.properties.license'
# OpenSearch Parameters for Product Search (Table 5)
parentIdentifier:
- 'parentIdentifier'
- '$.properties.parentIdentifier'
acquisitionType: '$.properties.acquisitionType'
orbitNumber:
- 'orbitNumber'
- '$.properties.orbitNumber'
orbitDirection:
- 'orbitDirection'
- '$.properties.orbitDirection'
swathIdentifier:
- 'swath'
- '$.properties.swath'
cloudCover:
- 'cloudCover=[0,{cloudCover}]'
- '$.properties.cloudCover'
snowCover:
- 'snowCover=[0,{snowCover}]'
- '$.properties.snowCover'
productVersion: '$.properties.version'
productQualityStatus: '$.properties.onlineQualityCheck'
processorName: '$.properties.processingName'
processingCenter: '$.properties.processingCenter'
creationDate: '$.properties.dhusIngestDate'
modificationDate: '$.properties.updated'
sensorMode:
- 'sensorMode'
- '$.properties.sensorMode'
# OpenSearch Parameters for Acquistion Parameters Search (Table 6)
startTimeFromAscendingNode:
- startDate
- '$.properties.startDate'
completionTimeFromAscendingNode:
- completionDate
- '$.properties.completionDate'
polarizationMode:
- 'polarisation'
- '$.properties.polarisation'
# Custom parameters (not defined in the base document referenced above)
id:
- 'productIdentifier=%{id#remove_extension}%'
- '{$.properties.title#remove_extension}'
tileIdentifier:
- tileId
- '$.null'
# The geographic extent of the product
geometry:
- 'geometry={geometry#to_rounded_wkt}'
# - '$.geometry'
- '($.geometry.`str()`.`sub(/^\\[\\]$/, POLYGON((180 -90, 180 90, -180 90, -180 -90, 180 -90)))`)|($.geometry[*])'
# The url of the quicklook
quicklook: '$.properties.thumbnail'
# The url to download the product "as is" (literal or as a template to be completed either after the search result
# is obtained from the provider or during the eodag download phase)
downloadLink: 'https://zipper.creodias.eu/download/{uid}'
# storageStatus must be one of ONLINE, STAGING, OFFLINE
storageStatus: '$.properties.status'
# Additional metadata provided by the providers but that don't appear in the reference spec
thumbnail: '$.properties.thumbnail'
download: !plugin
type: HTTPDownload
base_uri: 'https://zipper.creodias.eu/download/'
extract: true
order_enabled: false
archive_depth: 2
ssl_verify: true
auth: !plugin
type: KeycloakOIDCPasswordAuth
auth_base_uri: 'https://identity.cloudferro.com/auth'
realm: 'Creodias-new'
client_id: 'CLOUDFERRO_PUBLIC'
client_secret: 'dc0aca03-2dc6-4798-a5de-fc5aeb6c8ee1'
token_provision: qs
token_qs_key: 'token'
auth_error_code: 401
ssl_verify: true
products:
# S1
S1_SAR_RAW:
productType: RAW
collection: Sentinel1
metadata_mapping:
cloudCover: '$.null'
S1_SAR_GRD:
productType: GRD
collection: Sentinel1
metadata_mapping:
cloudCover: '$.null'
S1_SAR_SLC:
productType: SLC
collection: Sentinel1
metadata_mapping:
cloudCover: '$.null'
S1_SAR_OCN:
productType: OCN
collection: Sentinel1
metadata_mapping:
cloudCover: '$.null'
# S2
S2_MSI_L1C:
collection: SENTINEL-2
productType: S2MSI1C
S2_MSI_L2A:
collection: SENTINEL-2
productType: S2MSI2A
# S3 SRAL
S3_SRA:
productType: SRA
collection: Sentinel3
metadata_mapping:
cloudCover: '$.null'
S3_SRA_A:
productType: SRA_A
collection: Sentinel3
metadata_mapping:
cloudCover: '$.null'
S3_SRA_BS:
productType: SRA_BS
collection: Sentinel3
metadata_mapping:
cloudCover: '$.null'
S3_LAN:
productType: LAN
collection: Sentinel3
metadata_mapping:
cloudCover: '$.null'
S3_WAT:
productType: WAT
collection: Sentinel3
metadata_mapping:
cloudCover: '$.null'
# S3 OLCI
S3_EFR:
productType: EFR
collection: Sentinel3
S3_ERR:
productType: ERR
collection: Sentinel3
S3_OLCI_L2LRR:
productType: LRR
collection: Sentinel3
S3_OLCI_L2LFR:
productType: LFR
collection: Sentinel3
S3_OLCI_L2WRR:
productType: WRR
collection: Sentinel3
S3_OLCI_L2WFR:
productType: WFR
collection: Sentinel3
# S3 SLSTR
S3_SLSTR_L1RBT:
productType: RBT
collection: Sentinel3
S3_SLSTR_L2LST:
productType: LST
collection: Sentinel3
S3_SLSTR_L2WST:
productType: WST
collection: Sentinel3
S3_SLSTR_L2AOD:
productType: AOD___
collection: Sentinel3
S3_SLSTR_L2FRP:
productType: FRP___
collection: Sentinel3
# S3 SY
S3_SY_AOD:
productType: SY_2_AOD___
collection: Sentinel3
S3_SY_SYN:
productType: SY_2_SYN___
collection: Sentinel3
S3_SY_V10:
productType: SY_2_V10___
collection: Sentinel3
S3_SY_VG1:
productType: SY_2_VG1___
collection: Sentinel3
S3_SY_VGP:
productType: SY_2_VGP___
collection: Sentinel3
# S5P L1
S5P_L1B_IR_SIR:
productType: L1B_IR_SIR
collection: Sentinel5P
S5P_L1B_IR_UVN:
productType: L1B_IR_UVN
collection: Sentinel5P
S5P_L1B_RA_BD1:
productType: L1B_RA_BD1
collection: Sentinel5P
S5P_L1B_RA_BD2:
productType: L1B_RA_BD2
collection: Sentinel5P
S5P_L1B_RA_BD3:
productType: L1B_RA_BD3
collection: Sentinel5P
S5P_L1B_RA_BD4:
productType: L1B_RA_BD4
collection: Sentinel5P
S5P_L1B_RA_BD5:
productType: L1B_RA_BD5
collection: Sentinel5P
S5P_L1B_RA_BD6:
productType: L1B_RA_BD6
collection: Sentinel5P
S5P_L1B_RA_BD7:
productType: L1B_RA_BD7
collection: Sentinel5P
S5P_L1B_RA_BD8:
productType: L1B_RA_BD8
collection: Sentinel5P
# S5P L2
S5P_L2_NO2:
productType: L2__NO2___
collection: Sentinel5P
S5P_L2_CLOUD:
productType: L2__CLOUD_
collection: Sentinel5P
S5P_L2_O3:
productType: L2__O3____
collection: Sentinel5P
S5P_L2_CO:
productType: L2__CO____
collection: Sentinel5P
S5P_L2_AER_AI:
productType: L2__AER_AI
collection: Sentinel5P
S5P_L2_O3_PR:
productType: L2__O3__PR
collection: Sentinel5P
S5P_L2_O3_TCL:
productType: L2__O3_TCL
collection: Sentinel5P
S5P_L2_AER_LH:
productType: L2__AER_LH
collection: Sentinel5P
S5P_L2_HCHO:
productType: L2__HCHO__
collection: Sentinel5P
S5P_L2_CH4:
productType: L2__CH4___
collection: Sentinel5P
S5P_L2_NP_BD3:
productType: L2__NP_BD3
collection: Sentinel5P
S5P_L2_NP_BD6:
productType: L2__NP_BD6
collection: Sentinel5P
S5P_L2_NP_BD7:
productType: L2__NP_BD7
collection: Sentinel5P
S5P_L2_SO2:
productType: L2__SO2___
collection: Sentinel5P
# COP DEM
COP_DEM_GLO30_DGED:
productType: DGE_30
collection: CopDem
metadata_mapping:
cloudCover: '$.null'
COP_DEM_GLO30_DTED:
productType: DTE_30
collection: CopDem
metadata_mapping:
cloudCover: '$.null'
COP_DEM_GLO90_DGED:
productType: DGE_90
collection: CopDem
metadata_mapping:
cloudCover: '$.null'
COP_DEM_GLO90_DTED:
productType: DTE_90
collection: CopDem
metadata_mapping:
cloudCover: '$.null'
GENERIC_PRODUCT_TYPE:
productType: '{productType}'
collection: '{collection}'
---
!provider # MARK: onda
name: onda
priority: 0
description: Serco DIAS
roles:
- host
url: https://www.onda-dias.eu/cms/
search: !plugin
type: ODataV4Search
api_endpoint: 'https://catalogue.onda-dias.eu/dias-catalogue/Products'
timeout: 60
need_auth: false
ssl_verify: true
dont_quote:
- '['
- ']'
- '$'
- '='
- '&'
- ':'
pagination:
count_endpoint: 'https://catalogue.onda-dias.eu/dias-catalogue/Products/$count'
next_page_url_tpl: '{url}?{search}&$top={items_per_page}&$skip={skip}&$expand=Metadata'
# 2021/03/19: 2000 is the max, if greater 200 response but contains an error message
max_items_per_page: 2_000
sort:
sort_by_default:
- !!python/tuple [startTimeFromAscendingNode, ASC]
sort_by_tpl: '&$orderby={sort_param} {sort_order}'
sort_param_mapping:
startTimeFromAscendingNode: beginPosition
uid: id
storageStatus: offline
sort_order_mapping:
ascending: asc
descending: desc
max_sort_params: 1
results_entry: 'value'
literal_search_params:
$format: json
free_text_search_operations:
$search:
union: ' OR '
wrapper: '"{}"'
operations:
AND:
- 'footprint:"Intersects({geometry#to_rounded_wkt})"'
- 'productType:{productType}'
- 'platformName:{platform}'
- 'cloudCoverPercentage:[0 TO {cloudCover}]'
- 'beginPosition:[{startTimeFromAscendingNode#to_iso_utc_datetime} TO *]'
- 'endPosition:[* TO {completionTimeFromAscendingNode#to_iso_utc_datetime}]'
- '{id#remove_extension}'
- 'platformSerialIdentifier:{platformSerialIdentifier}'
- 'instrumentShortName:{instrument}'
- 'processingLevel:{processingLevel}'
- 'sensorType:{sensorType}'
- 'topicCategory:{topicCategory}'
- 'lineage:{lineage}'
- 'orbitNumber:{orbitNumber}'
- 'orbitDirection:{orbitDirection}'
- 'processingBaseline:{productVersion}'
- 'generalQualityFlag:{productQualityStatus}'
- 'sensorOperationalMode:{sensorMode}'
- 'tileIdentifier:{tileIdentifier}'
discover_metadata:
auto_discovery: true
metadata_pattern: '^[a-zA-Z0-9]+$'
search_param:
free_text_search_operations:
$search:
operations:
AND:
- '{metadata}:{{{metadata}}}'
metadata_path: '$.Metadata.*'
per_product_metadata_query: false
metadata_pre_mapping:
metadata_path: '$.Metadata'
metadata_path_id: 'id'
metadata_path_value: 'value'
metadata_mapping:
# Opensearch resource identifier within the search engine context (in our case
# within the context of the data provider)
# Queryable parameters are set with null as 1st configuration list value to mark them as queryable,
# but `free_text_search_operations.$search.operations.AND` entries are then used instead.
uid: '$.id'
# OpenSearch Parameters for Collection Search (Table 3)
productType:
- null
- '$.Metadata.productType'
platform:
- null
- '$.Metadata.platformName'
platformSerialIdentifier:
- null
- '$.Metadata.platformSerialIdentifier'
instrument:
- null
- '$.Metadata.instrumentShortName'
processingLevel:
- null
- '$.Metadata.processingLevel'
sensorType:
- null
- '$.Metadata.sensorType'
# INSPIRE obligated OpenSearch Parameters for Collection Search (Table 4)
title: '{$.Metadata.filename#remove_extension}'
topicCategory:
- null
- '$.Metadata.topicCategory'
lineage:
- null
- '$.Metadata.lineage'
# OpenSearch Parameters for Product Search (Table 5)
orbitNumber:
- null
- '$.Metadata.orbitNumber'
orbitDirection:
- null
- '$.Metadata.orbitDirection'
cloudCover:
- null
- '$.Metadata.cloudCoverPercentage'
productVersion:
- null
- '$.Metadata.processingBaseline'
productQualityStatus:
- null
- '$.Metadata.generalQualityFlag'
creationDate: '$.Metadata.creationDate'
processingDate: '$.Metadata.processingDate'
sensorMode:
- null
- '$.Metadata.sensorOperationalMode'
# OpenSearch Parameters for Acquistion Parameters Search (Table 6)
startTimeFromAscendingNode:
- null
- '$.beginPosition'
completionTimeFromAscendingNode:
- null
- '$.endPosition'
polarizationChannels: '{$.Metadata.polarisationChannels#replace_str(","," ")}'
# Custom parameters (not defined in the base document referenced above)
id:
- null
- '{$.Metadata.filename#remove_extension}'
tileIdentifier:
- null
- '$.Metadata.tileIdentifier'
# The geographic extent of the product
geometry:
- null
- '$.footprint'
# The url of the quicklook
quicklook: '$.quicklook'
# The url to download the product "as is" (literal or as a template to be completed either after the search result
# is obtained from the provider or during the eodag download phase)
downloadLink: '%(base_uri)s({uid})/$value'
# storageStatus: must be one of ONLINE, STAGING, OFFLINE
storageStatus: '{$.offline#get_group_name((?P<ONLINE>False)|(?P<OFFLINE>True))}'
# Url used for ordering product if it is offline/archived
orderLink: 'https://catalogue.onda-dias.eu/dias-catalogue/Products({uid})/Ens.Order'
products:
# S1
S1_SAR_OCN:
productType: '*OCN*'
metadata_mapping:
cloudCover: '$.null'
S1_SAR_GRD:
productType: '*GRD*'
metadata_mapping:
cloudCover: '$.null'
S1_SAR_SLC:
productType: '*SLC*'
metadata_mapping:
cloudCover: '$.null'
S1_SAR_RAW:
productType: '*RAW*'
metadata_mapping:
cloudCover: '$.null'
# S2
S2_MSI_L1C:
productType: S2MSI1C
S2_MSI_L2A:
productType: S2MSI2A
# S3 SRAL
S3_SRA:
productType: SR_1_SRA___
metadata_mapping:
cloudCover: '$.null'
S3_SRA_A:
productType: SR_1_SRA_A_
metadata_mapping:
cloudCover: '$.null'
S3_SRA_BS:
productType: SR_1_SRA_BS
metadata_mapping:
cloudCover: '$.null'
S3_LAN:
productType: SR_2_LAN___
metadata_mapping:
cloudCover: '$.null'
S3_WAT:
productType: SR_2_WAT___
metadata_mapping:
cloudCover: '$.null'
# S3 OLCI
S3_EFR:
productType: OL_1_EFR___
S3_ERR:
productType: OL_1_ERR___
S3_OLCI_L2LFR:
productType: OL_2_LFR___
S3_OLCI_L2LRR:
productType: OL_2_LRR___
S3_OLCI_L2WRR:
productType: OL_2_WRR___
S3_OLCI_L2WFR:
productType: OL_2_WFR___
# S3 SLSTR
S3_SLSTR_L1RBT:
productType: SL_1_RBT___
S3_SLSTR_L2LST:
productType: SL_2_LST___
S3_SLSTR_L2WST:
productType: SL_2_WST___
S3_SLSTR_L2FRP:
productType: SL_2_FRP___
# S3 SY
S3_SY_AOD:
productType: SY_2_AOD___
S3_SY_SYN:
productType: SY_2_SYN___
S3_SY_V10:
productType: SY_2_V10___
S3_SY_VG1:
productType: SY_2_VG1___
S3_SY_VGP:
productType: SY_2_VGP___
# S5P L1
S5P_L1B_RA_BD1:
productType: L1B_RA_BD1
S5P_L1B_RA_BD2:
productType: L1B_RA_BD2
S5P_L1B_RA_BD3:
productType: L1B_RA_BD3
S5P_L1B_RA_BD4:
productType: L1B_RA_BD4
S5P_L1B_RA_BD5:
productType: L1B_RA_BD5
S5P_L1B_RA_BD6:
productType: L1B_RA_BD6
S5P_L1B_RA_BD7:
productType: L1B_RA_BD7
S5P_L1B_RA_BD8:
productType: L1B_RA_BD8
# S5P L2
S5P_L2_NO2:
productType: L2__NO2___
S5P_L2_CLOUD:
productType: L2__CLOUD_
S5P_L2_O3:
productType: L2__O3____
S5P_L2_CO:
productType: L2__CO____
S5P_L2_AER_AI:
productType: L2__AER_AI
S5P_L2_O3_PR:
productType: L2__O3__PR
S5P_L2_AER_LH:
productType: L2__AER_LH
S5P_L2_HCHO:
productType: L2__HCHO__
S5P_L2_CH4:
productType: L2__CH4___
S5P_L2_NP_BD3:
productType: L2__NP_BD3
S5P_L2_NP_BD6:
productType: L2__NP_BD6
S5P_L2_NP_BD7:
productType: L2__NP_BD7
S5P_L2_SO2:
productType: L2__SO2___
# Landsat
L8_OLI_TIRS_C1L1:
platform: 'Landsat-*'
GENERIC_PRODUCT_TYPE:
productType: '{productType}'
platform: '{platform}'
download: !plugin
type: HTTPDownload
base_uri: 'https://catalogue.onda-dias.eu/dias-catalogue/Products'
extract: true
archive_depth: 2
auth_error_code: 401
order_enabled: true
order_method: 'POST'
ssl_verify: true
order_headers:
Content-Type: application/json
auth: !plugin
type: GenericAuth
ssl_verify: true
---
!provider # MARK: astraea_eod
name: astraea_eod
priority: 0
roles:
- host
description: Astraea Earth OnDemand
url: https://earthondemand.astraea.earth
search: !plugin
type: StacSearch
api_endpoint: https://eod-catalog-svc-prod.astraea.earth/search
need_auth: false
ssl_verify: true
discover_queryables:
fetch_url: null
product_type_fetch_url: null
pagination:
# 2021/03/19: The docs (https://eod-catalog-svc-prod.astraea.earth/api.html#operation/getSearchSTAC)
# say the max is 10_000. In practice 1_000 products are returned if more are asked (even greater
# than 10_000), without any error.
# This provider doesn't implement any pagination, let's just try to get the maximum number of
# products available at once then, so we stick to 10_000.
max_items_per_page: 10_000
sort:
sort_by_default:
- !!python/tuple [startTimeFromAscendingNode, ASC]
sort_param_mapping:
id: id
startTimeFromAscendingNode: properties.datetime
creationDate: properties.created
metadata_mapping:
# redefine the following mapppings as the provider does not support advanced queries/filtering,
# these parameters will not be queryable
doi: '$.properties."sci:doi"'
processingLevel: '$.properties."processing:level"'
platform: '$.properties.constellation'
platformSerialIdentifier: '$.properties.platform'
instrument: '$.properties.instruments'
# INSPIRE obligated OpenSearch Parameters for Collection Search (Table 4)
title: '$.id'
abstract: '$.properties.description'
resolution: '$.properties.gsd'
publicationDate: '$.properties.published'
# OpenSearch Parameters for Product Search (Table 5)
orbitNumber: '$.properties."sat:relative_orbit"'
orbitDirection: '$.properties."sat:orbit_state"'
cloudCover: '$.properties."eo:cloud_cover"'
sensorMode: '$.properties."sar:instrument_mode"'
creationDate: '$.properties.created'
modificationDate: '$.properties.updated'
productVersion: '$.properties.version'
# OpenSearch Parameters for Acquistion Parameters Search (Table 6)
availabilityTime: '$.properties.availabilityTime'
acquisitionStation: '$.properties.acquisitionStation'
acquisitionSubType: '$.properties.acquisitionSubType'
illuminationAzimuthAngle: '$.properties."view:sun_azimuth"'
illuminationElevationAngle: '$.properties."view:sun_elevation"'
polarizationChannels: '$.properties."sar:polarizations"'
dopplerFrequency: '$.properties."sar:frequency_band"'
products:
S1_SAR_GRD:
productType: sentinel1_l1c_grd
metadata_mapping:
cloudCover: '$.null'
platformSerialIdentifier: '$.id.`split(_, 0, -1)`'
polarizationMode: '$.id.`sub(/.{14}([A-Z]{2}).*/, \\1)`'
awsPath: '$.assets.productInfo.href.`sub(/(.*)\/productInfo\.json/, \\1)`'
assets: |
{$.assets#dict_update([["manifest.safe",[["title","manifest.safe"],["href","{awsPath}/manifest.safe"],["roles",["metadata"]],["type","application/xml"]]]])}
S2_MSI_L1C:
productType: sentinel2_l1c
metadata_mapping:
platformSerialIdentifier: '$.id.`split(_, 0, -1)`'
tileInfo: '{$.assets.tileInfo.href#replace_str(r"s3(.*)sentinel-s2-l1c/",r"https\1roda.sentinel-hub.com/sentinel-s2-l1c/")}'
tilePath: |
$.assets.tileInfo.href.`sub(/.*\/sentinel-s2-l1c\/(tiles\/.*)\/tileInfo\.json/, \\1)`
S2_MSI_L2A:
productType: sentinel2_l2a
metadata_mapping:
platformSerialIdentifier: '$.id.`split(_, 0, -1)`'
tileInfo: '{$.assets.tileInfo.href#replace_str(r"s3(.*)sentinel-s2-l2a/",r"https\1roda.sentinel-hub.com/sentinel-s2-l2a/")}'
tilePath: |
$.assets.tileInfo.href.`sub(/.*\/sentinel-s2-l2a\/(tiles\/.*)\/tileInfo\.json/, \\1)`
LANDSAT_C2L1:
productType: landsat8_c2l1t1
MODIS_MCD43A4:
productType: mcd43a4
NAIP:
productType: naip
GENERIC_PRODUCT_TYPE:
productType: '{productType}'
download: !plugin
type: AwsDownload
requester_pays: True
flatten_top_dirs: True
ssl_verify: true
products:
S1_SAR_GRD:
default_bucket: 'sentinel-s1-l1c'
build_safe: true
complementary_url_key:
- awsPath
S2_MSI_L1C:
default_bucket: 'sentinel-s2-l1c'
build_safe: true
fetch_metadata:
fetch_url: '{tileInfo}'
fetch_format: json
update_metadata:
title: '$.productName'
productPath: '$.productPath'
complementary_url_key:
- productPath
- tilePath
S2_MSI_L2A:
default_bucket: 'sentinel-s2-l2a'
build_safe: true
fetch_metadata:
fetch_url: '{tileInfo}'
fetch_format: json
update_metadata:
title: '$.productName'
productPath: '$.productPath'
complementary_url_key:
- productPath
- tilePath
auth: !plugin
type: AwsAuth
ssl_verify: true
---
!provider # MARK: usgs_satapi_aws
name: usgs_satapi_aws
priority: 0
roles:
- host
description: USGS Landsatlook SAT API
url: https://landsatlook.usgs.gov/stac-server
search: !plugin
type: StacSearch
api_endpoint: https://landsatlook.usgs.gov/stac-server/search
need_auth: false
ssl_verify: true
discover_queryables:
fetch_url: null
product_type_fetch_url: null
pagination:
# 2021/03/19: no more than 10_000 (if greater, returns a 500 error code)
# but in practive if an Internal Server Error is returned for more than
# about 500 products.
max_items_per_page: 500
sort:
sort_by_default:
- !!python/tuple [startTimeFromAscendingNode, ASC]
sort_param_mapping:
id: id
startTimeFromAscendingNode: properties.datetime
creationDate: properties.created
modificationDate: properties.updated
platformSerialIdentifier: properties.platform
illuminationElevationAngle: properties.view:sun_elevation
illuminationAzimuthAngle: properties.view:sun_azimuth
cloudCover: properties.eo:cloud_cover
metadata_mapping:
assets: '{$.assets#recursive_sub_str(r"https?(.*)landsatlook.usgs.gov/data/",r"s3\1usgs-landsat/")}'
awsProductId: '{$.assets.thumbnail.href#replace_str(r".+/([A-Z0-9_]+)/[\w.]+$",r"\1")}'
products:
LANDSAT_C2L1:
productType: landsat-c2l1
LANDSAT_C2L2_SR:
productType: landsat-c2l2-sr
LANDSAT_C2L2_ST:
productType: landsat-c2l2-st
LANDSAT_C2L2ALB_BT:
productType: landsat-c2l2alb-bt
LANDSAT_C2L2ALB_SR:
productType: landsat-c2l2alb-sr
LANDSAT_C2L2ALB_ST:
productType: landsat-c2l2alb-st
LANDSAT_C2L2ALB_TA:
productType: landsat-c2l2alb-ta
GENERIC_PRODUCT_TYPE:
productType: '{productType}'
download: !plugin
type: AwsDownload
requester_pays: True
flatten_top_dirs: True
ssl_verify: true
auth: !plugin
type: AwsAuth
ssl_verify: true
---
!provider # MARK: earth_search
name: earth_search
priority: 0
roles:
- host
description: Earth Search
url: https://www.element84.com/earth-search/
search: !plugin
type: StacSearch
api_endpoint: https://earth-search.aws.element84.com/v1/search
need_auth: false
ssl_verify: true
discover_product_types:
results_entry: '$.collections[?id!="sentinel-s2-l2a-cogs"]'
discover_queryables:
fetch_url: null
product_type_fetch_url: null
pagination:
# Override the default next page url key path of StacSearch because the next link returned
# by Earth Search is invalid (as of 2021/04/29). Entry set to null (None) to avoid using the
# next page retrieval mechanism, `next_page_url_tpl` will be used instead (inherited from StacSearch)
# Remove that entry if Earth Search updates that and returns a valid link.
next_page_url_key_path: null
# 2021/04/28: Earth-Search relies on Sat-API whose docs (http://sat-utils.github.io/sat-api/#search-stac-items-by-simple-filtering-)
# say the max is 10_000. In practice a too high number (e.g. 5_000) returns a 502 error ({"message": "Internal server error"}).
# Let's set it to a more robust number: 500
max_items_per_page: 500
sort:
sort_by_default:
- !!python/tuple [startTimeFromAscendingNode, ASC]
sort_param_mapping:
id: id
startTimeFromAscendingNode: properties.datetime
creationDate: properties.created
modificationDate: properties.updated
platformSerialIdentifier: properties.platform
resolution: properties.gsd
cloudCover: properties.eo:cloud_cover
metadata_mapping:
utmZone:
- '{{"query":{{"mgrs:utm_zone":{{"eq":"{utmZone}"}}}}}}'
- '$.properties."mgrs:utm_zone"'
latitudeBand:
- '{{"query":{{"mgrs:latitude_band":{{"eq":"{latitudeBand}"}}}}}}'
- '$.properties."mgrs:latitude_band"'
gridSquare:
- '{{"query":{{"mgrs:grid_square":{{"eq":"{gridSquare}"}}}}}}'
- '$.properties."mgrs:grid_square"'
tileIdentifier:
- '{{"query":{{"mgrs:utm_zone":{{"eq":"{tileIdentifier#slice_str(0,2,1)}"}},"mgrs:latitude_band":{{"eq":"{tileIdentifier#slice_str(2,3,1)}"}},"mgrs:grid_square":{{"eq":"{tileIdentifier#slice_str(3,5,1)}"}}}}}}'
- '{utmZone}{latitudeBand}{gridSquare}'
assets: '{$.assets#dict_filter($[?(href=~"^s3.*")])}'
products:
S1_SAR_GRD:
productType: sentinel-1-grd
S2_MSI_L1C:
productType: sentinel-2-l1c
metadata_mapping:
title: '{$.properties."s2:product_uri"#remove_extension}'
platformSerialIdentifier: '$.id.`split(_, 0, -1)`'
polarizationMode: '$.id.`sub(/.{14}([A-Z]{2}).*/, \\1)`'
productPath: |
$.properties."s2:product_uri".`sub(/([S2AB]{3})_MSIL1C_([0-9]{4})([0-9]{2})([0-9]{2})(T.*).SAFE/, products!\\2!\\3!\\4!\\1_MSIL1C_\\2\\3\\4\\5)`.`sub(/!0*/, /)`
tilePath: |
$.assets.tileinfo_metadata.href.`sub(/.*/sentinel-s2-l1c\/(tiles\/.*)\/tileInfo\.json/, \\1)`
utmZone:
- '{{"query":{{"mgrs:utm_zone":{{"eq":"{utmZone}"}}}}}}'
- '$.properties."mgrs:utm_zone"'
latitudeBand:
- '{{"query":{{"mgrs:latitude_band":{{"eq":"{latitudeBand}"}}}}}}'
- '$.properties."mgrs:latitude_band"'
gridSquare:
- '{{"query":{{"mgrs:grid_square":{{"eq":"{gridSquare}"}}}}}}'
- '$.properties."mgrs:grid_square"'
tileIdentifier:
- '{{"query":{{"mgrs:utm_zone":{{"eq":"{tileIdentifier#slice_str(0,2,1)}"}},"mgrs:latitude_band":{{"eq":"{tileIdentifier#slice_str(2,3,1)}"}},"mgrs:grid_square":{{"eq":"{tileIdentifier#slice_str(3,5,1)}"}}}}}}'
- '{utmZone}{latitudeBand}{gridSquare}'
LANDSAT_C2L2:
productType: landsat-c2-l2
NAIP:
productType: naip
COP_DEM_GLO30_DGED:
productType: cop-dem-glo-30
COP_DEM_GLO90_DGED:
productType: cop-dem-glo-90
GENERIC_PRODUCT_TYPE:
productType: '{productType}'
download: !plugin
type: AwsDownload
requester_pays: True
flatten_top_dirs: True
ssl_verify: true
products:
S2_MSI_L1C:
default_bucket: 'sentinel-s2-l1c'
build_safe: true
complementary_url_key:
- productPath
- tilePath
auth: !plugin
type: AwsAuth
ssl_verify: true
---
!provider # MARK: earth_search_cog
name: earth_search_cog
priority: 0
roles:
- host
description: Earth Search with Cloud Optimized GeoTIFF (COG) formatted assets
url: https://www.element84.com/earth-search/
search: !plugin
type: StacSearch
api_endpoint: https://earth-search.aws.element84.com/v1/search
need_auth: false
ssl_verify: true
discover_product_types:
fetch_url: null
discover_queryables:
fetch_url: null
product_type_fetch_url: null
pagination:
# Override the default next page url key path of StacSearch because the next link returned
# by Earth Search is invalid (as of 2021/04/29). Entry set to null (None) to avoid using the
# next page retrieval mechanism, `next_page_url_tpl` will be used instead (inherited from StacSearch)
# Remove that entry if Earth Search updates that and returns a valid link.
next_page_url_key_path: null
# 2021/04/28: Earth-Search relies on Sat-API whose docs (http://sat-utils.github.io/sat-api/#search-stac-items-by-simple-filtering-)
# say the max is 10_000. In practice a too high number (e.g. 5_000) returns a 502 error ({"message": "Internal server error"}).
# Let's set it to a more robust number: 500
max_items_per_page: 500
sort:
sort_by_default:
- !!python/tuple [startTimeFromAscendingNode, ASC]
sort_param_mapping:
id: id
startTimeFromAscendingNode: properties.datetime
creationDate: properties.created
modificationDate: properties.updated
platform: properties.constellation
platformSerialIdentifier: properties.platform
resolution: properties.gsd
cloudCover: properties.eo:cloud_cover
metadata_mapping:
platformSerialIdentifier: '$.id.`split(_, 0, -1)`'
polarizationMode: '$.id.`sub(/.{14}([A-Z]{2}).*/, \\1)`'
title: '{$.properties."s2:product_uri"#remove_extension}'
assets: '{$.assets#dict_filter($[?(href=~"^http.*")])}'
products:
S2_MSI_L2A_COG:
productType: sentinel-2-l2a
GENERIC_PRODUCT_TYPE:
productType: '{productType}'
download: !plugin
type: HTTPDownload
base_uri: 'https://sentinel-cogs.s3.us-west-2.amazonaws.com'
ssl_verify: true
---
!provider # MARK: earth_search_gcs
name: earth_search_gcs
priority: 0
roles:
- host
description: Google Cloud Storage through Earth Search
url: https://www.element84.com/earth-search/
search: !plugin
type: StacSearch
api_endpoint: https://earth-search.aws.element84.com/v0/search
need_auth: false
ssl_verify: true
discover_product_types:
fetch_url: null
discover_queryables:
fetch_url: null
product_type_fetch_url: null
pagination:
# Override the default next page url key path of StacSearch because the next link returned
# by Earth Search is invalid (as of 2021/04/29). Entry set to null (None) to avoid using the
# next page retrieval mechanism, `next_page_url_tpl` will be used instead (inherited from StacSearch)
# Remove that entry if Earth Search updates that and returns a valid link.
next_page_url_key_path: null
# 2021/04/28: Earth-Search relies on Sat-API whose docs (http://sat-utils.github.io/sat-api/#search-stac-items-by-simple-filtering-)
# say the max is 10_000. In practice a too high number (e.g. 5_000) returns a 502 error ({"message": "Internal server error"}).
# Let's set it to a more robust number: 500
max_items_per_page: 500
sort:
sort_by_default:
- !!python/tuple [startTimeFromAscendingNode, ASC]
sort_param_mapping:
id: id
startTimeFromAscendingNode: properties.datetime
creationDate: properties.created
modificationDate: properties.updated
platformSerialIdentifier: properties.platform
resolution: properties.gsd
cloudCover: properties.eo:cloud_cover
products:
S2_MSI_L1C:
productType: sentinel-s2-l1c
metadata_mapping:
title: '$.properties."sentinel:product_id"'
platformSerialIdentifier: '$.id.`split(_, 0, -1)`'
polarizationMode: '$.id.`sub(/.{14}([A-Z]{2}).*/, \\1)`'
utmZone: '$.properties."sentinel:utm_zone"'
latitudeBand: '$.properties."sentinel:latitude_band"'
gridSquare: '$.properties."sentinel:grid_square"'
downloadLink: 's3://gcp-public-data-sentinel-2/tiles/{utmZone}/{latitudeBand}/{gridSquare}/{title}.SAFE'
L8_OLI_TIRS_C1L1:
productType: landsat-8-l1-c1
metadata_mapping:
wrsPath: '$.properties."landsat:wrs_path"'
wrsRow: '$.properties."landsat:wrs_row"'
downloadLink: 's3://gcp-public-data-landsat/LC08/01/{wrsPath:03d}/{wrsRow:03d}/{title}'
GENERIC_PRODUCT_TYPE:
productType: '{productType}'
download: !plugin
type: AwsDownload
base_uri: https://storage.googleapis.com
ignore_assets: True
ssl_verify: true
products:
S2_MSI_L1C:
default_bucket: 'gcp-public-data-sentinel-2'
auth: !plugin
type: AwsAuth
ssl_verify: true
---
!provider # MARK: ecmwf
name: ecmwf
priority: 0
description: ECMWF archive products
roles:
- host
url: https://www.ecmwf.int
api: !plugin
type: EcmwfApi
api_endpoint: https://api.ecmwf.int/v1
extract: false
ssl_verify: true
metadata_mapping:
productType: '$.productType'
title: '$.id'
startTimeFromAscendingNode: '{$.startTimeFromAscendingNode#to_iso_date}'
completionTimeFromAscendingNode:
- 'date={startTimeFromAscendingNode#to_iso_date}/to/{completionTimeFromAscendingNode#to_iso_date(-1,)}'
- '{$.completionTimeFromAscendingNode#to_iso_date}'
id: '$.id'
# The geographic extent of the product
geometry:
- 'area={geometry#to_nwse_bounds_str(/)}'
- '$.geometry'
defaultGeometry: 'POLYGON((180 -90, 180 90, -180 90, -180 -90, 180 -90))'
# storageStatus set to ONLINE for consistency between providers
storageStatus: '{$.null#replace_str("Not Available","ONLINE")}'
downloadLink: 'https://apps.ecmwf.int/datasets/data/{dataset}'
# ECMWF/MARS specific parameters
# Full list here https://confluence.ecmwf.int/display/UDOC/Keywords+in+MARS+and+Dissemination+requests
accuracy:
- accuracy
- '$.accuracy'
anoffset:
- anoffset
- '$.anoffset'
bitmap:
- bitmap
- '$.bitmap'
block:
- block
- '$.block'
channel:
- channel
- '$.channel'
class:
- class
- '$.class'
database:
- database
- '$.database'
dataset:
- dataset
- '$.dataset'
diagnostic:
- diagnostic
- '$.diagnostic'
direction:
- direction
- '$.direction'
domain:
- domain
- '$.domain'
duplicates:
- duplicates
- '$.duplicates'
expect:
- expect
- '$.expect'
expver:
- expver
- '$.expver'
fcmonth:
- fcmonth
- '$.fcmonth'
fcperiod:
- fcperiod
- '$.fcperiod'
fieldset:
- fieldset
- '$.fieldset'
filter:
- filter
- '$.filter'
format:
- format
- '$.format'
frame:
- frame
- '$.frame'
frequency:
- frequency
- '$.frequency'
grid:
- grid
- '$.grid'
hdate:
- hdate
- '$.hdate'
ident:
- ident
- '$.ident'
interpolation:
- interpolation
- '$.interpolation'
intgrid:
- intgrid
- '$.intgrid'
iteration:
- iteration
- '$.iteration'
latitude:
- latitude
- '$.latitude'
levelist:
- levelist
- '$.levelist'
levtype:
- levtype
- '$.levtype'
longitude:
- longitude
- '$.longitude'
lsm:
- lsm
- '$.lsm'
method:
- method
- '$.method'
number:
- number
- '$.number'
obsgroup:
- obsgroup
- '$.obsgroup'
obstype:
- obstype
- '$.obstype'
origin:
- origin
- '$.origin'
packing:
- packing
- '$.packing'
padding:
- padding
- '$.padding'
param:
- param
- '$.param'
priority:
- priority
- '$.priority'
product:
- product
- '$.product'
range:
- range
- '$.range'
refdate:
- refdate
- '$.refdate'
reference:
- reference
- '$.reference'
reportype:
- reportype
- '$.reportype'
repres:
- repres
- '$.repres'
resol:
- resol
- '$.resol'
rotation:
- rotation
- '$.rotation'
section:
- section
- '$.section'
source:
- source
- '$.source'
step:
- step
- '$.step'
stream:
- stream
- '$.stream'
system:
- system
- '$.system'
target:
- target
- '$.target'
time:
- time
- '$.time'
truncation:
- truncation
- '$.truncation'
type:
- type
- '$.type'
use:
- use
- '$.use'
products:
# See Archive Catalog in https://apps.ecmwf.int/archive-catalogue/
# See available Public Datasets in https://apps.ecmwf.int/datasets/
TIGGE_CF_SFC:
class: ti
dataset: tigge
expver: prod
type: cf
levtype: sfc
origin: ecmf
grid: 0.5/0.5
param: 59/134/136/146/147/151/165/166/167/168/172/176/177/179/189/235/228001/228002/228039/228139/228141/228144/228164/228228
step: 0
time: 00:00
GENERIC_PRODUCT_TYPE:
dataset: '{productType}'
---
!provider # MARK: cop_ads
name: cop_ads
priority: 0
description: Copernicus Atmosphere Data Store
roles:
- host
url: https://ads.atmosphere.copernicus.eu/
auth: !plugin
type: GenericAuth
method: basic
ssl_verify: true
download: !plugin
type: HTTPDownload
base_uri: https://ads.atmosphere.copernicus.eu/api/v2
ssl_verify: true
extract: false
auth_error_code: 401
order_enabled: true
order_method: 'POST'
order_status_error:
state: failed
order_status_success:
state: completed
order_on_response:
metadata_mapping:
order_id: '$.request_id'
orderStatusLink: 'https://ads.atmosphere.copernicus.eu/api/v2/tasks/{order_id}'
downloadLink: '$.location'
order_status_method: 'GET'
products:
CAMS_GAC_FORECAST:
outputs_extension: .grib
CAMS_GFE_GFAS:
outputs_extension: .grib
CAMS_EU_AIR_QUALITY_FORECAST:
outputs_extension: .grib
CAMS_EU_AIR_QUALITY_RE:
outputs_extension: .zip
extract: true
CAMS_GRF:
outputs_extension: .zip
extract: true
CAMS_GRF_AUX:
outputs_extension: .zip
extract: true
CAMS_SOLAR_RADIATION:
outputs_extension: .csv
CAMS_GREENHOUSE_EGG4_MONTHLY:
outputs_extension: .grib
CAMS_GREENHOUSE_EGG4:
outputs_extension: .grib
CAMS_GREENHOUSE_INVERSION:
outputs_extension: .grib
CAMS_GLOBAL_EMISSIONS:
outputs_extension: .zip
extract: true
CAMS_EAC4:
outputs_extension: .grib
CAMS_EAC4_MONTHLY:
outputs_extension: .grib
search: !plugin
type: BuildSearchResult
ssl_verify: true
end_date_excluded: false
remove_from_query:
- dataset
discover_queryables:
fetch_url: null
product_type_fetch_url: null
constraints_file_url: "https://datastore.copernicus-climate.eu/cams/published-forms/camsprod/{dataset}/constraints.json"
metadata_mapping:
productType: '$.productType'
title: '$.id'
startTimeFromAscendingNode: '{$.startTimeFromAscendingNode#to_iso_date}'
completionTimeFromAscendingNode:
- 'date={startTimeFromAscendingNode#to_iso_date}/{completionTimeFromAscendingNode#to_iso_date}'
- '{$.completionTimeFromAscendingNode#to_iso_date}'
_date: '{startTimeFromAscendingNode}/{completionTimeFromAscendingNode}'
id: '$.id'
# The geographic extent of the product
geometry:
- 'area={geometry#to_nwse_bounds_str(/)}'
- '$.geometry'
defaultGeometry: 'POLYGON((180 -90, 180 90, -180 90, -180 -90, 180 -90))'
storageStatus: '{$.null#replace_str("Not Available","OFFLINE")}'
downloadLink: '$.null'
orderLink: 'https://ads.atmosphere.copernicus.eu/api/v2/resources/{dataset}'
# Copernicus ADS specific parameters
accuracy:
- accuracy
- '$.accuracy'
aerosol_type:
- aerosol_type
- '$.aerosol_type'
altitude:
- altitude
- '$.altitude'
anoffset:
- anoffset
- '$.anoffset'
api_product_type:
- product_type
- '$.api_product_type'
band:
- band
- '$.band'
bitmap:
- bitmap
- '$.bitmap'
block:
- block
- '$.block'
channel:
- channel
- '$.channel'
class:
- class
- '$.class'
database:
- database
- '$.database'
dataset:
- dataset
- '$.dataset'
date_range:
- date_range
- '$.date_range'
day:
- day
- '$.day'
diagnostic:
- diagnostic
- '$.diagnostic'
direction:
- direction
- '$.direction'
domain:
- domain
- '$.domain'
duplicates:
- duplicates
- '$.duplicates'
expect:
- expect
- '$.expect'
expver:
- expver
- '$.expver'
fcperiod:
- fcperiod
- '$.fcperiod'
fieldset:
- fieldset
- '$.fieldset'
filter:
- filter
- '$.filter'
forcing_type:
- forcing_type
- '$.forcing_type'
format:
- format
- '$.format'
frequency:
- frequency
- '$.frequency'
grid:
- grid
- '$.grid'
hdate:
- hdate
- '$.hdate'
ident:
- ident
- '$.ident'
input_observations:
- input_observations
- '$.input_observations'
interpolation:
- interpolation
- '$.interpolation'
intgrid:
- intgrid
- '$.intgrid'
iteration:
- iteration
- '$.iteration'
latitude:
- latitude
- '$.latitude'
level:
- level
- '$.level'
levelist:
- levelist
- '$.levelist'
leadtime_hour:
- leadtime_hour
- '$.leadtime_hour'
levtype:
- levtype
- '$.levtype'
location:
- location
- '$.location'
longitude:
- longitude
- '$.longitude'
lsm:
- lsm
- '$.lsm'
method:
- method
- '$.method'
model:
- model
- '$.model'
model_level:
- model_level
- '$.model_level'
month:
- month
- '$.month'
number:
- number
- '$.number'
obsgroup:
- obsgroup
- '$.obsgroup'
obstype:
- obstype
- '$.obstype'
origin:
- origin
- '$.origin'
packing:
- packing
- '$.packing'
padding:
- padding
- '$.padding'
param:
- param
- '$.param'
pressure_level:
- pressure_level
- '$.pressure_level'
priority:
- priority
- '$.priority'
product:
- product
- '$.product'
range:
- range
- '$.range'
refdate:
- refdate
- '$.refdate'
reference:
- reference
- '$.reference'
reportype:
- reportype
- '$.reportype'
repres:
- repres
- '$.repres'
resol:
- resol
- '$.resol'
rotation:
- rotation
- '$.rotation'
section:
- section
- '$.section'
sky_type:
- sky_type
- '$.sky_type'
source:
- source
- '$.source'
step:
- step
- '$.step'
stream:
- stream
- '$.stream'
system:
- system
- '$.system'
target:
- target
- '$.target'
time:
- time
- '$.time'
time_aggregation:
- time_aggregation
- '$.time_aggregation'
time_reference:
- time_reference
- '$.time_reference'
time_step:
- time_step
- '$.time_step'
truncation:
- truncation
- '$.truncation'
type:
- type
- '$.type'
use:
- use
- '$.use'
variable:
- variable
- '$.variable'
version:
- version
- '$.version'
quantity:
- quantity
- '$.quantity'
year:
- year
- '$.year'
products:
# See available Public Datasets in https://ads.atmosphere.copernicus.eu/cdsapp#!/search?type=dataset
CAMS_GAC_FORECAST:
dataset: cams-global-atmospheric-composition-forecasts
type: forecast
format: grib
variable: ammonium_aerosol_optical_depth_550nm
time: '00:00'
leadtime_hour: '0'
_default_end_date: '2019-07-12T00:00Z'
CAMS_GFE_GFAS:
dataset: cams-global-fire-emissions-gfas
format: grib
variable: altitude_of_plume_bottom
_default_end_date: '2018-07-03T00:00Z'
CAMS_EU_AIR_QUALITY_FORECAST:
dataset: cams-europe-air-quality-forecasts
model: ensemble
format: grib
variable: alder_pollen
type: forecast
time: '00:00'
level: '0'
leadtime_hour: '0'
metadata_mapping:
geometry:
- 'area={geometry#to_nwse_bounds}'
- '$.geometry'
CAMS_EU_AIR_QUALITY_RE:
dataset: cams-europe-air-quality-reanalyses
type: validated_reanalysis
format: zip
variable: nitrogen_dioxide
model: ensemble
level: '0'
metadata_mapping:
completionTimeFromAscendingNode:
- |
{{
"year": {_date#interval_to_datetime_dict}["year"],
"month": {_date#interval_to_datetime_dict}["month"],
"day": {_date#interval_to_datetime_dict}["day"]
}}
- '{$.completionTimeFromAscendingNode#to_iso_date}'
CAMS_GRF:
dataset: cams-global-radiative-forcings
format: zip
variable: radiative_forcing_of_carbon_dioxide
forcing_type: instantaneous
band: long_wave
sky_type: all_sky
level: surface
version: '2'
metadata_mapping:
completionTimeFromAscendingNode:
- |
{{
"year": {_date#interval_to_datetime_dict}["year"],
"month": {_date#interval_to_datetime_dict}["month"],
"day": {_date#interval_to_datetime_dict}["day"]
}}
- '{$.completionTimeFromAscendingNode#to_iso_date}'
CAMS_GRF_AUX:
dataset: cams-global-radiative-forcing-auxilliary-variables
band: short_wave
sky_type: clear_sky
version: '1.5'
format: zip
variable: aerosol_radiation_effect
aerosol_type: marine
level: surface
metadata_mapping:
completionTimeFromAscendingNode:
- |
{{
"year": {_date#interval_to_datetime_dict}["year"],
"month": {_date#interval_to_datetime_dict}["month"],
"day": {_date#interval_to_datetime_dict}["day"]
}}
- '{$.completionTimeFromAscendingNode#to_iso_date}'
CAMS_SOLAR_RADIATION:
dataset: cams-solar-radiation-timeseries
sky_type: clear
time_step: 1minute
time_reference: true_solar_time
location:
latitude: 0
longitude: 0
altitude: -999
format: csv
CAMS_GREENHOUSE_EGG4_MONTHLY:
dataset: cams-global-ghg-reanalysis-egg4-monthly
format: grib
variable: snow_albedo
api_product_type: monthly_mean
metadata_mapping:
completionTimeFromAscendingNode:
- |
{{
"year": {_date#interval_to_datetime_dict}["year"],
"month": {_date#interval_to_datetime_dict}["month"],
"day": {_date#interval_to_datetime_dict}["day"]
}}
- '{$.completionTimeFromAscendingNode#to_iso_date}'
CAMS_GREENHOUSE_EGG4:
dataset: cams-global-ghg-reanalysis-egg4
format: grib
variable: snow_albedo
step: '0'
CAMS_GREENHOUSE_INVERSION:
dataset: cams-global-greenhouse-gas-inversion
version: latest
variable: carbon_dioxide
quantity: mean_column
input_observations: surface
time_aggregation: instantaneous
metadata_mapping:
completionTimeFromAscendingNode:
- |
{{
"year": {_date#interval_to_datetime_dict}["year"],
"month": {_date#interval_to_datetime_dict}["month"],
"day": {_date#interval_to_datetime_dict}["day"]
}}
- '{$.completionTimeFromAscendingNode#to_iso_date}'
CAMS_GLOBAL_EMISSIONS:
dataset: cams-global-emission-inventories
version: latest
format: zip
variable: acids
source: anthropogenic
metadata_mapping:
completionTimeFromAscendingNode:
- |
{{
"year": {_date#interval_to_datetime_dict}["year"],
"month": {_date#interval_to_datetime_dict}["month"],
"day": {_date#interval_to_datetime_dict}["day"]
}}
- '{$.completionTimeFromAscendingNode#to_iso_date}'
CAMS_EAC4:
dataset: cams-global-reanalysis-eac4
format: grib
variable: '2m_dewpoint_temperature'
time: '00:00'
CAMS_EAC4_MONTHLY:
dataset: cams-global-reanalysis-eac4-monthly
format: grib
variable: 2m_dewpoint_temperature
api_product_type: monthly_mean
metadata_mapping:
completionTimeFromAscendingNode:
- |
{{
"year": {_date#interval_to_datetime_dict}["year"],
"month": {_date#interval_to_datetime_dict}["month"],
"day": {_date#interval_to_datetime_dict}["day"]
}}
- '{$.completionTimeFromAscendingNode#to_iso_date}'
GENERIC_PRODUCT_TYPE:
dataset: '{productType}'
---
!provider # MARK: cop_cds
name: cop_cds
priority: 0
description: Copernicus Climate Data Store
roles:
- host
url: https://cds.climate.copernicus.eu/
auth: !plugin
type: GenericAuth
method: basic
ssl_verify: true
download: !plugin
type: HTTPDownload
base_uri: https://cds.climate.copernicus.eu/api/v2
ssl_verify: true
extract: false
auth_error_code: 401
order_enabled: true
order_method: 'POST'
order_status_error:
state: failed
order_status_success:
state: completed
order_on_response:
metadata_mapping:
order_id: '$.request_id'
orderStatusLink: 'https://cds.climate.copernicus.eu/api/v2/tasks/{order_id}'
downloadLink: '$.location'
order_status_method: 'GET'
products:
ERA5_SL:
outputs_extension: .grib
ERA5_PL:
outputs_extension: .grib
ERA5_PL_MONTHLY:
outputs_extension: .grib
ERA5_LAND:
outputs_extension: .grib
ERA5_LAND_MONTHLY:
outputs_extension: .grib
ERA5_SL_MONTHLY:
outputs_extension: .grib
UERRA_EUROPE_SL:
outputs_extension: .grib
SATELLITE_SEA_LEVEL_BLACK_SEA:
outputs_extension: .zip
extract: true
GLACIERS_DIST_RANDOLPH:
outputs_extension: .zip
extract: true
FIRE_HISTORICAL:
outputs_extension: .grib
GLOFAS_FORECAST:
outputs_extension: .grib
GLOFAS_REFORECAST:
outputs_extension: .grib
GLOFAS_HISTORICAL:
outputs_extension: .grib
GLOFAS_SEASONAL:
outputs_extension: .grib
GLOFAS_SEASONAL_REFORECAST:
outputs_extension: .grib
EFAS_FORECAST:
outputs_extension: .grib.zip
extract: true
EFAS_HISTORICAL:
outputs_extension: .grib.zip
extract: true
EFAS_REFORECAST:
outputs_extension: .grib.zip
extract: true
EFAS_SEASONAL:
outputs_extension: .grib.zip
extract: true
EFAS_SEASONAL_REFORECAST:
outputs_extension: .grib.zip
extract: true
SATELLITE_CARBON_DIOXIDE:
outputs_extension: .zip
extract: true
SATELLITE_METHANE:
outputs_extension: .zip
extract: true
SEASONAL_POSTPROCESSED_PL:
outputs_extension: .grib
SEASONAL_POSTPROCESSED_SL:
outputs_extension: .grib
SEASONAL_ORIGINAL_SL:
outputs_extension: .grib
SEASONAL_ORIGINAL_PL:
outputs_extension: .grib
SEASONAL_MONTHLY_PL:
outputs_extension: .grib
SEASONAL_MONTHLY_SL:
outputs_extension: .grib
SIS_HYDRO_MET_PROJ:
outputs_extension: .zip
extract: true
search: !plugin
type: BuildSearchResult
ssl_verify: true
end_date_excluded: false
remove_from_query:
- dataset
discover_queryables:
fetch_url: null
product_type_fetch_url: null
constraints_file_url: "http://datastore.copernicus-climate.eu/c3s/published-forms/c3sprod/{dataset}/constraints.json"
metadata_mapping:
productType: '$.productType'
title: '$.id'
startTimeFromAscendingNode: '{$.startTimeFromAscendingNode#to_iso_date}'
completionTimeFromAscendingNode:
- 'date={startTimeFromAscendingNode#to_iso_date}/{completionTimeFromAscendingNode#to_iso_date}'
- '{$.completionTimeFromAscendingNode#to_iso_date}'
_date: '{startTimeFromAscendingNode}/{completionTimeFromAscendingNode}'
id: '$.id'
# The geographic extent of the product
geometry:
- 'area={geometry#to_nwse_bounds_str(/)}'
- '$.geometry'
defaultGeometry: 'POLYGON((180 -90, 180 90, -180 90, -180 -90, 180 -90))'
storageStatus: '{$.null#replace_str("Not Available","OFFLINE")}'
downloadLink: '$.null'
orderLink: 'https://cds.climate.copernicus.eu/api/v2/resources/{dataset}'
# Copernicus CDS specific parameters
accuracy:
- accuracy
- '$.accuracy'
anoffset:
- anoffset
- '$.anoffset'
api_product_type:
- product_type
- '$.api_product_type'
bitmap:
- bitmap
- '$.bitmap'
block:
- block
- '$.block'
channel:
- channel
- '$.channel'
class:
- class
- '$.class'
database:
- database
- '$.database'
dataset:
- dataset
- '$.dataset'
dataset_type:
- dataset_type
- '$.dataset_type'
date_range:
- date_range
- '$.date_range'
day:
- day
- '$.day'
diagnostic:
- diagnostic
- '$.diagnostic'
direction:
- direction
- '$.direction'
domain:
- domain
- '$.domain'
duplicates:
- duplicates
- '$.duplicates'
ensemble_member:
- ensemble_member
- '$.ensemble_member'
expect:
- expect
- '$.expect'
experiment:
- experiment
- '$.experiment'
expver:
- expver
- '$.expver'
fcperiod:
- fcperiod
- '$.fcperiod'
fieldset:
- fieldset
- '$.fieldset'
filter:
- filter
- '$.filter'
format:
- format
- '$.format'
frequency:
- frequency
- '$.frequency'
gcm:
- gcm
- '$.gcm'
grid:
- grid
- '$.grid'
hdate:
- hdate
- '$.hdate'
horizontal_resolution:
- horizontal_resolution
- '$.horizontal_resolution'
hydrological_model:
- hydrological_model
- '$.hydrological_model'
ident:
- ident
- '$.ident'
interpolation:
- interpolation
- '$.interpolation'
intgrid:
- intgrid
- '$.intgrid'
iteration:
- iteration
- '$.iteration'
latitude:
- latitude
- '$.latitude'
levelist:
- levelist
- '$.levelist'
leadtime_hour:
- leadtime_hour
- '$.leadtime_hour'
leadtime_month:
- leadtime_month
- '$.leadtime_month'
levtype:
- levtype
- '$.levtype'
longitude:
- longitude
- '$.longitude'
lsm:
- lsm
- '$.lsm'
method:
- method
- '$.method'
model_levels:
- model_levels
- '$.model_levels'
month:
- month
- '$.month'
number:
- number
- '$.number'
obsgroup:
- obsgroup
- '$.obsgroup'
obstype:
- obstype
- '$.obstype'
origin:
- origin
- '$.origin'
originating_centre:
- originating_centre
- '$.originating_centre'
packing:
- packing
- '$.packing'
padding:
- padding
- '$.padding'
param:
- param
- '$.param'
period:
- period
- '$.period'
pressure_level:
- pressure_level
- '$.pressure_level'
priority:
- priority
- '$.priority'
processing_level:
- processing_level
- '$.processing_level'
processing_type:
- processing_type
- '$.processing_type'
product:
- product
- '$.product'
range:
- range
- '$.range'
rcm:
- rcm
- '$.rcm'
refdate:
- refdate
- '$.refdate'
reference:
- reference
- '$.reference'
reportype:
- reportype
- '$.reportype'
repres:
- repres
- '$.repres'
resol:
- resol
- '$.resol'
rotation:
- rotation
- '$.rotation'
section:
- section
- '$.section'
sensor_and_algorithm:
- sensor_and_algorithm
- '$.sensor_and_algorithm'
source:
- source
- '$.source'
step:
- step
- '$.step'
stream:
- stream
- '$.stream'
system:
- system
- '$.system'
target:
- target
- '$.target'
time:
- time
- '$.time'
time_aggregation:
- time_aggregation
- '$.time_aggregation'
truncation:
- truncation
- '$.truncation'
type:
- type
- '$.type'
use:
- use
- '$.use'
variable:
- variable
- '$.variable'
variable_type:
- variable_type
- '$.variable_type'
version:
- version
- '$.version'
system_version:
- system_version
- '$.system_version'
year:
- year
- '$.year'
products:
# See available Public Datasets in https://cds.climate.copernicus.eu/cdsapp#!/search?type=dataset
ERA5_SL:
dataset: reanalysis-era5-single-levels
api_product_type: reanalysis
time: '00:00'
format: grib
ERA5_PL:
dataset: reanalysis-era5-pressure-levels
api_product_type: reanalysis
time: '00:00'
format: grib
ERA5_PL_MONTHLY:
dataset: reanalysis-era5-pressure-levels-monthly-means
api_product_type: monthly_averaged_reanalysis
time: '00:00'
format: grib
variable: divergence
pressure_level: '1'
metadata_mapping:
completionTimeFromAscendingNode:
- |
{{
"year": {_date#interval_to_datetime_dict}["year"],
"month": {_date#interval_to_datetime_dict}["month"],
"day": {_date#interval_to_datetime_dict}["day"]
}}
- '{$.completionTimeFromAscendingNode#to_iso_date}'
ERA5_LAND:
dataset: reanalysis-era5-land
variable: 2m_dewpoint_temperature
time: '01:00'
format: grib
ERA5_LAND_MONTHLY:
dataset: reanalysis-era5-land-monthly-means
api_product_type: monthly_averaged_reanalysis
variable: 2m_dewpoint_temperature
time: '00:00'
format: grib
metadata_mapping:
completionTimeFromAscendingNode:
- |
{{
"year": {_date#interval_to_datetime_dict}["year"],
"month": {_date#interval_to_datetime_dict}["month"],
"day": {_date#interval_to_datetime_dict}["day"]
}}
- '{$.completionTimeFromAscendingNode#to_iso_date}'
ERA5_SL_MONTHLY:
dataset: reanalysis-era5-single-levels-monthly-means
api_product_type: monthly_averaged_reanalysis
time: '00:00'
format: grib
metadata_mapping:
completionTimeFromAscendingNode:
- |
{{
"year": {_date#interval_to_datetime_dict}["year"],
"month": {_date#interval_to_datetime_dict}["month"],
"day": {_date#interval_to_datetime_dict}["day"]
}}
- '{$.completionTimeFromAscendingNode#to_iso_date}'
UERRA_EUROPE_SL:
dataset: reanalysis-uerra-europe-single-levels
origin: mescan_surfex
variable: 10m_wind_direction
time: '00:00'
format: grib
SATELLITE_SEA_LEVEL_BLACK_SEA:
dataset: satellite-sea-level-black-sea
format: zip
metadata_mapping:
completionTimeFromAscendingNode:
- |
{{
"year": {_date#interval_to_datetime_dict}["year"],
"month": {_date#interval_to_datetime_dict}["month"],
"day": {_date#interval_to_datetime_dict}["day"]
}}
- '{$.completionTimeFromAscendingNode#to_iso_date}'
GLACIERS_DIST_RANDOLPH:
dataset: insitu-glaciers-extent
variable: glacier_area
api_product_type: gridded
version: '6_0'
format: zip
FIRE_HISTORICAL:
dataset: cems-fire-historical-v1
grid: original_grid
dataset_type: consolidated_dataset
api_product_type: reanalysis
variable: build_up_index
system_version: '4_1'
format: grib
GLOFAS_FORECAST:
dataset: cems-glofas-forecast
system_version: operational
variable: river_discharge_in_the_last_24_hours
format: grib
hydrological_model: lisflood
api_product_type: control_forecast
leadtime_hour: '24'
GLOFAS_REFORECAST:
dataset: cems-glofas-reforecast
variable: river_discharge_in_the_last_24_hours
format: grib
system_version: version_4_0
hydrological_model: lisflood
api_product_type: control_reforecast
leadtime_hour: '24'
GLOFAS_HISTORICAL:
dataset: cems-glofas-historical
system_version: version_4_0
variable: river_discharge_in_the_last_24_hours
format: grib
hydrological_model: lisflood
api_product_type: consolidated
metadata_mapping:
completionTimeFromAscendingNode:
- 'hdate={startTimeFromAscendingNode#to_iso_date}/{completionTimeFromAscendingNode#to_iso_date(-1,)}'
- '{$.completionTimeFromAscendingNode#to_iso_date}'
GLOFAS_SEASONAL:
dataset: cems-glofas-seasonal
variable: river_discharge_in_the_last_24_hours
format: grib
system_version: operational
hydrological_model: lisflood
leadtime_hour: '24'
metadata_mapping:
completionTimeFromAscendingNode:
- |
{{
"year": {_date#interval_to_datetime_dict}["year"],
"month": {_date#interval_to_datetime_dict}["month"],
"day": {_date#interval_to_datetime_dict}["day"]
}}
- '{$.completionTimeFromAscendingNode#to_iso_date}'
GLOFAS_SEASONAL_REFORECAST:
dataset: cems-glofas-seasonal-reforecast
format: grib
variable: river_discharge_in_the_last_24_hours
system_version: version_4_0
hydrological_model: lisflood
leadtime_hour: '24'
metadata_mapping:
completionTimeFromAscendingNode:
- |
{{
"hyear": {_date#interval_to_datetime_dict}["year"],
"hmonth": {_date#interval_to_datetime_dict}["month"],
"hday": {_date#interval_to_datetime_dict}["day"]
}}
- '{$.completionTimeFromAscendingNode#to_iso_date}'
EFAS_FORECAST:
dataset: efas-forecast
format: grib.zip
system_version: operational
originating_centre: ecmwf
api_product_type: control_forecast
variable: river_discharge_in_the_last_24_hours
model_levels: surface_level
time: '00:00'
leadtime_hour: '24'
EFAS_HISTORICAL:
dataset: efas-historical
format: grib.zip
model_levels: surface_level
variable: snow_depth_water_equivalent
system_version: version_5_0
time: '00:00'
metadata_mapping:
completionTimeFromAscendingNode:
- 'hdate={startTimeFromAscendingNode#to_iso_date}/{completionTimeFromAscendingNode#to_iso_date(-1,)}'
- '{$.completionTimeFromAscendingNode#to_iso_date}'
EFAS_REFORECAST:
dataset: efas-reforecast
system_version: version_5_0
format: grib.zip
api_product_type: control_forecast
variable: river_discharge_in_the_last_6_hours
model_levels: surface_level
leadtime_hour: '6'
metadata_mapping:
completionTimeFromAscendingNode:
- 'hdate={startTimeFromAscendingNode#to_iso_date}/{completionTimeFromAscendingNode#to_iso_date(-1,)}'
- '{$.completionTimeFromAscendingNode#to_iso_date}'
EFAS_SEASONAL:
dataset: efas-seasonal
system_version: operational
format: grib.zip
variable: river_discharge_in_the_last_24_hours
model_levels: surface_level
leadtime_hour: '24'
metadata_mapping:
completionTimeFromAscendingNode:
- |
{{
"year": {_date#interval_to_datetime_dict}["year"],
"month": {_date#interval_to_datetime_dict}["month"],
"day": {_date#interval_to_datetime_dict}["day"]
}}
- '{$.completionTimeFromAscendingNode#to_iso_date}'
EFAS_SEASONAL_REFORECAST:
dataset: efas-seasonal-reforecast
system_version: version_5_0
format: grib.zip
variable: river_discharge_in_the_last_24_hours
model_levels: surface_level
leadtime_hour: '24'
metadata_mapping:
completionTimeFromAscendingNode:
- |
{{
"hyear": {_date#interval_to_datetime_dict}["year"],
"hmonth": {_date#interval_to_datetime_dict}["month"],
"hday": {_date#interval_to_datetime_dict}["day"]
}}
- '{$.completionTimeFromAscendingNode#to_iso_date}'
SATELLITE_CARBON_DIOXIDE:
dataset: satellite-carbon-dioxide
format: zip
processing_level: level_2
variable: xco2
sensor_and_algorithm: sciamachy_wfmd
version: '4.0'
metadata_mapping:
completionTimeFromAscendingNode:
- |
{{
"year": {_date#interval_to_datetime_dict}["year"],
"month": {_date#interval_to_datetime_dict}["month"],
"day": {_date#interval_to_datetime_dict}["day"]
}}
- '{$.completionTimeFromAscendingNode#to_iso_date}'
SATELLITE_METHANE:
dataset: satellite-methane
format: zip
processing_level: level_2
variable: xch4
sensor_and_algorithm: sciamachy_wfmd
version: '4.0'
metadata_mapping:
completionTimeFromAscendingNode:
- |
{{
"year": {_date#interval_to_datetime_dict}["year"],
"month": {_date#interval_to_datetime_dict}["month"],
"day": {_date#interval_to_datetime_dict}["day"]
}}
- '{$.completionTimeFromAscendingNode#to_iso_date}'
SEASONAL_POSTPROCESSED_PL:
dataset: seasonal-postprocessed-pressure-levels
format: grib
originating_centre: ecmwf
system: '4'
variable: geopotential_anomaly
pressure_level: '10'
api_product_type: 'ensemble_mean'
leadtime_month: '1'
metadata_mapping:
completionTimeFromAscendingNode:
- |
{{
"year": {_date#interval_to_datetime_dict}["year"],
"month": {_date#interval_to_datetime_dict}["month"],
"day": {_date#interval_to_datetime_dict}["day"]
}}
- '{$.completionTimeFromAscendingNode#to_iso_date}'
SEASONAL_POSTPROCESSED_SL:
dataset: seasonal-postprocessed-single-levels
format: grib
originating_centre: ecmwf
system: '4'
variable: 2m_dewpoint_temperature_anomaly
api_product_type: 'ensemble_mean'
leadtime_month: '1'
metadata_mapping:
completionTimeFromAscendingNode:
- |
{{
"year": {_date#interval_to_datetime_dict}["year"],
"month": {_date#interval_to_datetime_dict}["month"],
"day": {_date#interval_to_datetime_dict}["day"]
}}
- '{$.completionTimeFromAscendingNode#to_iso_date}'
SEASONAL_ORIGINAL_SL:
dataset: seasonal-original-single-levels
format: grib
originating_centre: ecmwf
system: '5'
variable: 10m_u_component_of_wind
leadtime_hour: '6'
SEASONAL_ORIGINAL_PL:
dataset: seasonal-original-pressure-levels
format: grib
originating_centre: ecmwf
system: '5'
variable: geopotential
pressure_level: '10'
leadtime_hour: '12'
SEASONAL_MONTHLY_PL:
dataset: seasonal-monthly-pressure-levels
format: grib
originating_centre: ecmwf
system: '5'
variable: geopotential
pressure_level: '10'
api_product_type: monthly_mean
leadtime_month: '1'
metadata_mapping:
completionTimeFromAscendingNode:
- |
{{
"year": {_date#interval_to_datetime_dict}["year"],
"month": {_date#interval_to_datetime_dict}["month"],
"day": {_date#interval_to_datetime_dict}["day"]
}}
- '{$.completionTimeFromAscendingNode#to_iso_date}'
SEASONAL_MONTHLY_SL:
dataset: seasonal-monthly-single-levels
format: grib
originating_centre: ecmwf
system: '5'
variable: 10m_u_component_of_wind
api_product_type: monthly_mean
leadtime_month: '1'
metadata_mapping:
completionTimeFromAscendingNode:
- |
{{
"year": {_date#interval_to_datetime_dict}["year"],
"month": {_date#interval_to_datetime_dict}["month"],
"day": {_date#interval_to_datetime_dict}["day"]
}}
- '{$.completionTimeFromAscendingNode#to_iso_date}'
SIS_HYDRO_MET_PROJ:
dataset: sis-hydrology-meteorology-derived-projections
format: zip
api_product_type: climate_impact_indicators
variable: 2m_air_temperature
variable_type: absolute_change_from_reference_period
processing_type: original
time_aggregation: monthly_mean
horizontal_resolution: 5_km
experiment: degree_scenario
rcm: cclm4_8_17
gcm: ec_earth
ensemble_member: r12i1p1
period: 1_5_c
GENERIC_PRODUCT_TYPE:
dataset: '{productType}'
---
!provider # MARK: sara
name: sara
priority: 0
description: Sentinel Australasia Regional Access
roles:
- host
url: https://www.copernicus.gov.au/
search: !plugin
type: QueryStringSearch
# The endpoint is based off of the collection. There is a generic endpoint,
# but can be very slow if not enough metdata is provided.
api_endpoint: 'https://copernicus.nci.org.au/sara.server/1.0/api/collections/{collection}/search.json'
need_auth: false
ssl_verify: true
pagination:
next_page_url_tpl: '{url}?{search}&maxRecords={items_per_page}&page={page}'
total_items_nb_key_path: '$.properties.totalResults'
# 2021/03/19: 500 is the max, no error if greater
max_items_per_page: 500
sort:
sort_by_default:
- !!python/tuple [startTimeFromAscendingNode, ASC]
sort_by_tpl: '&sortParam={sort_param}&sortOrder={sort_order}'
sort_param_mapping:
startTimeFromAscendingNode: startDate
completionTimeFromAscendingNode: completionDate
sensorMode: sensorMode
sort_order_mapping:
ascending: asc
descending: desc
max_sort_params: 1
discover_metadata:
auto_discovery: true
metadata_pattern: '^(?!collection)[a-zA-Z0-9_]+$'
search_param: '{metadata}={{{metadata}}}'
metadata_path: '$.properties.*'
metadata_mapping:
# Opensearch resource identifier within the search engine context (in our case
# within the context of the data provider)
uid: '$.id'
# OpenSearch Parameters for Collection Search (Table 3)
productType:
- productType
- '$.properties.productType'
platform: '$.properties.collection'
platformSerialIdentifier:
- platform
- '$.properties.platform'
instrument:
- instrument
- '$.properties.instrument'
processingLevel:
- processingLevel
- '$.properties.processingLevel'
# INSPIRE obligated OpenSearch Parameters for Collection Search (Table 4)
title: '$.properties.title'
topicCategory: '$.properties.topicCategory'
keyword: '$.properties.keywords'
abstract: '$.properties.description'
resolution:
- 'resolution'
- '$.properties.resolution'
organisationName:
- 'organisationName'
- '$.properties.organisationName'
publicationDate: '$.properties.published'
accessConstraint: '$.properties.license'
# OpenSearch Parameters for Product Search (Table 5)
parentIdentifier:
- 'parentIdentifier'
- '$.properties.parentIdentifier'
acquisitionType: '$.properties.acquisitionType'
orbitNumber:
- 'orbitNumber'
- '$.properties.orbitNumber'
orbitDirection:
- 'orbitDirection'
- '$.properties.orbitDirection'
swathIdentifier:
- 'swath'
- '$.properties.swath'
cloudCover:
- 'cloudCover=[0,{cloudCover}]'
- '$.properties.cloudCover'
snowCover:
- 'snowCover=[0,{snowCover}]'
- '$.properties.snowCover'
productVersion: '$.properties.version'
productQualityStatus: '$.properties.onlineQualityCheck'
processorName: '$.properties.processingName'
processingCenter: '$.properties.processingCenter'
creationDate: '$.properties.dhusIngestDate'
modificationDate: '$.properties.updated'
sensorMode:
- 'sensorMode'
- '$.properties.sensorMode'
# OpenSearch Parameters for Acquistion Parameters Search (Table 6)
startTimeFromAscendingNode:
- startDate
- '$.properties.startDate'
completionTimeFromAscendingNode:
- completionDate
- '$.properties.completionDate'
polarizationMode:
- 'polarisation'
- '$.properties.polarisation'
# Custom parameters (not defined in the base document referenced above)
id:
- 'productIdentifier={id#remove_extension}'
- '$.properties.productIdentifier'
# The geographic extent of the product
geometry:
- 'geometry={geometry#to_rounded_wkt}'
- '$.geometry'
# The url of the quicklook
quicklook: '$.properties.quicklook'
# The url to download the product "as is" (literal or as a template to be completed either after the search result
# is obtained from the provider or during the eodag download phase)
downloadLink: '$.properties.services.download.url'
# storageStatus set to ONLINE for consistency between providers
storageStatus: '{$.null#replace_str("Not Available","ONLINE")}'
# Additional metadata provided by the providers but that don't appear in the reference spec
thumbnail: '$.properties.thumbnail'
products:
# Sentinel 1
S1_SAR_OCN:
productType: OCN
collection: S1
instrument: C-SAR
S1_SAR_GRD:
productType: GRD
collection: S1
instrument: C-SAR
S1_SAR_SLC:
productType: SLC
collection: S1
instrument: C-SAR
# Sentinel 2
S2_MSI_L1C:
collection: S2
productType: S2MSIL1C
instrument: MSI
processingLevel: L1C
S2_MSI_L2A:
collection: S2
productType: S2MSIL2A
instrument: MSI
processingLevel: L2A
# OLCI products
# L1
S3_EFR:
productType: OL_1_EFR___
collection: S3
instrument: OLCI
processingLevel: LEVEL-1
S3_ERR:
productType: OL_1_ERR___
collection: S3
instrument: OLCI
processingLevel: LEVEL-1
# output during radiometric calibration mode
S3_RAC:
productType: OL_1_RAC___
collection: S3
instrument: OLCI
processingLevel: LEVEL-1
# L2
S3_OLCI_L2LRR:
productType: OL_2_LRR___
collection: S3
instrument: OLCI
processingLevel: LEVEL-2
S3_OLCI_L2LFR:
productType: OL_2_LFR___
collection: S3
instrument: OLCI
processingLevel: LEVEL-2
S3_OLCI_L2WRR:
productType: OL_2_WRR___
collection: S3
instrument: OLCI
processingLevel: LEVEL-2
S3_OLCI_L2WFR:
productType: OL_2_WFR___
collection: S3
instrument: OLCI
processingLevel: LEVEL-2
# SLSTR products
S3_SLSTR_L1RBT:
productType: SL_1_RBT___
collection: S3
instrument: SLSTR
processingLevel: LEVEL-1
S3_SLSTR_L2LST:
productType: SL_2_LST___
collection: S3
instrument: SLSTR
processingLevel: LEVEL-2
S3_SLSTR_L2WST:
productType: SL_2_WST___
collection: S3
instrument: SLSTR
processingLevel: LEVEL-2
S3_SLSTR_L2AOD:
productType: SL_2_AOD___
collection: S3
instrument: SLSTR
processingLevel: LEVEL-2
S3_SLSTR_L2FRP:
productType: SL_2_FRP___
collection: S3
instrument: SLSTR
processingLevel: LEVEL-2
# SRAL
# L1
S3_SRA:
productType: SR_1_SRA___
collection: S3
instrument: SRAL
processingLevel: LEVEL-1
S3_SRA_A:
productType: SR_1_SRA_A_
collection: S3
instrument: SRAL
processingLevel: LEVEL-1
S3_SRA_BS:
productType: SR_1_SRA_BS
collection: S3
instrument: SRAL
processingLevel: LEVEL-1
# L2
S3_LAN:
productType: SR_2_LAN___
collection: S3
instrument: SRAL
processingLevel: LEVEL-2
S3_WAT:
productType: SR_2_WAT___
collection: S3
instrument: SRAL
processingLevel: LEVEL-2
# Synergy products
S3_SY_AOD:
productType: SY_2_AOD___
collection: S3
instrument: SYNERGY
processingLevel: LEVEL-2
S3_SY_SYN:
productType: SY_2_SYN___
collection: S3
instrument: SYNERGY
processingLevel: LEVEL-2
S3_SY_V10:
productType: SY_2_V10___
collection: S3
instrument: SYNERGY
processingLevel: LEVEL-2W
S3_SY_VG1:
productType: SY_2_VG1___
collection: S3
instrument: SYNERGY
processingLevel: LEVEL-2
S3_SY_VGP:
productType: SY_2_VGP___
collection: S3
instrument: SYNERGY
processingLevel: LEVEL-2
GENERIC_PRODUCT_TYPE:
productType: '{productType}'
collection: '{collection}'
download: !plugin
type: HTTPDownload
base_uri: 'https://copernicus.nci.org.au/'
extract: true
archive_depth: 2
order_enabled: true
auth_error_code: 403
ssl_verify: true
auth: !plugin
type: GenericAuth
method: basic
ssl_verify: true
---
!provider # MARK: meteoblue
name: meteoblue
priority: 0
roles:
- host
description: Meteoblue
url: https://www.meteoblue.com
search: !plugin
type: BuildPostSearchResult
api_endpoint: 'https://my.meteoblue.com/dataset/query'
need_auth: true
ssl_verify: true
pagination:
next_page_query_obj: '{{"checkOnly":true}}'
discover_metadata:
auto_discovery: true
metadata_pattern: '^[a-zA-Z0-9_]+$'
search_param: '{{{{"{metadata}":{{{metadata}#to_geojson}} }}}}'
metadata_path: '$.*'
metadata_mapping:
startTimeFromAscendingNode: '{$.startTimeFromAscendingNode#to_iso_utc_datetime}'
completionTimeFromAscendingNode:
- '{{"timeIntervals": [ "{startTimeFromAscendingNode#to_iso_date}/{completionTimeFromAscendingNode#to_iso_date(-1,)}" ] }}'
- '{$.completionTimeFromAscendingNode#to_iso_utc_datetime}'
geometry:
- '{{"geometry": {geometry#to_geojson} }}'
- '$.geometry'
defaultGeometry: 'POLYGON((180 -90, 180 90, -180 90, -180 -90, 180 -90))'
productType: '$.queries[0].domain'
storageStatus: '{$.requiresJobQueue#get_group_name((?P<ONLINE>False)|(?P<OFFLINE>True))}'
downloadLink: 'https://my.meteoblue.com/dataset/query'
# Meteoblue specific parameters
datapoints: '$.datapoints'
requiresJobQueue: '$.requiresJobQueue'
requiresComplexJobQueue: '$.requiresComplexJobQueue'
units:
- '{{"units": {units#to_geojson} }}'
- '$.units'
queries:
- '{{"queries": {queries#to_geojson} }}'
- '$.queries'
format:
- '{{"format": {format#to_geojson} }}'
- '$.format'
timeIntervalsAlignment:
- '{{"timeIntervalsAlignment": {timeIntervalsAlignment#to_geojson} }}'
- '$.timeIntervalsAlignment'
orderLink: '{$.downloadLink#replace_str(r"^(.*)(\")(queries\")(.)",r"\1\2runOnJobQueue\2\4 true, \2\3\4")}'
products:
NEMSGLOBAL_TCDC:
queries: [{'domain':'NEMSGLOBAL','gapFillDomain':null,'timeResolution':'daily','codes':[{'code':71,'level':'sfc','aggregation':'mean'}]}]
format: netCDF
units: {'temperature':'C','velocity':'km/h','length':'metric','energy':'watts'}
timeIntervalsAlignment:
geometry: {"type": "Polygon", "coordinates": [[[180, -90], [180, 90], [-180, 90], [-180, -90], [180, -90]]]}
NEMSAUTO_TCDC:
queries: [{'domain':'NEMSAUTO','gapFillDomain':null,'timeResolution':'daily','codes':[{'code':71,'level':'sfc','aggregation':'mean'}]}]
format: netCDF
units: {'temperature':'C','velocity':'km/h','length':'metric','energy':'watts'}
timeIntervalsAlignment:
geometry: {"type": "Polygon", "coordinates": [[[180, -90], [180, 90], [-180, 90], [-180, -90], [180, -90]]]}
GENERIC_PRODUCT_TYPE:
productType: '{productType}'
geometry: {"type": "Polygon", "coordinates": [[[180, -90], [180, 90], [-180, 90], [-180, -90], [180, -90]]]}
download: !plugin
type: HTTPDownload
base_uri: 'https://my.meteoblue.com/dataset/query'
method: POST
extract: False
order_enabled: true
order_method: 'POST'
ssl_verify: true
order_on_response:
metadata_mapping:
order_id: '$.id'
downloadLink: 'http://queueresults.meteoblue.com/{order_id}'
downloadMethod: '{$.null#replace_str("Not Available","GET")}'
orderStatusLink: 'http://my.meteoblue.com/queue/status/{order_id}'
order_status_method: 'GET'
order_status_percent: percentCompleted
order_status_error:
status: error
outputs_extension: .nc
auth: !plugin
type: HttpQueryStringAuth
auth_uri: 'http://my.meteoblue.com/dataset/meta?dataset=NEMSAUTO'
ssl_verify: true
---
!provider # MARK: cop_dataspace
name: cop_dataspace
priority: 0
description: Copernicus Data Space Ecosystem
roles:
- host
url: https://dataspace.copernicus.eu/
search: !plugin
type: QueryStringSearch
api_endpoint: 'http://catalogue.dataspace.copernicus.eu/resto/api/collections/{collection}/search.json'
need_auth: false
timeout: 60
ssl_verify: true
pagination:
next_page_url_tpl: '{url}?{search}&maxRecords={items_per_page}&page={page}&exactCount=1'
total_items_nb_key_path: '$.properties.totalResults'
max_items_per_page: 1_000
sort:
sort_by_default:
- !!python/tuple [startTimeFromAscendingNode, ASC]
sort_by_tpl: '&sortParam={sort_param}&sortOrder={sort_order}'
sort_param_mapping:
startTimeFromAscendingNode: startDate
completionTimeFromAscendingNode: completionDate
publicationDate: published
modificationDate: updated
sort_order_mapping:
ascending: asc
descending: desc
max_sort_params: 1
discover_metadata:
auto_discovery: true
metadata_pattern: '^(?!collection)[a-zA-Z0-9]+$'
search_param: '{metadata}={{{metadata}}}'
metadata_path: '$.properties.*'
metadata_mapping:
# Opensearch resource identifier within the search engine context (in our case
# within the context of the data provider)
uid: '$.id'
# OpenSearch Parameters for Collection Search (Table 3)
productType:
- productType
- '$.properties.productType'
platform: '$.properties.collection'
platformSerialIdentifier:
- platform
- '$.properties.platform'
instrument:
- instrument
- '$.properties.instrument'
processingLevel:
- processingLevel
- '$.properties.processingLevel'
# INSPIRE obligated OpenSearch Parameters for Collection Search (Table 4)
title: '{$.properties.title#remove_extension}'
resolution:
- 'resolution'
- '$.properties.resolution'
organisationName:
- 'organisationName'
- '$.properties.organisationName'
publicationDate: '$.properties.published'
# OpenSearch Parameters for Product Search (Table 5)
orbitNumber:
- 'orbitNumber'
- '$.properties.orbitNumber'
orbitDirection:
- 'orbitDirection'
- '$.properties.orbitDirection'
cloudCover:
- 'cloudCover=[0,{cloudCover}]'
- '$.properties.cloudCover'
modificationDate: '$.properties.updated'
sensorMode:
- 'sensorMode'
- '$.properties.sensorMode'
# OpenSearch Parameters for Acquistion Parameters Search (Table 6)
startTimeFromAscendingNode:
- startDate
- '$.properties.startDate'
completionTimeFromAscendingNode:
- completionDate
- '$.properties.completionDate'
polarizationChannels:
- 'polarisation'
- '$.properties.polarisation'
# Custom parameters (not defined in the base document referenced above)
id:
- 'productIdentifier={id#remove_extension}'
- '{$.properties.title#remove_extension}'
tileIdentifier:
- tileId
- '$.null'
# The geographic extent of the product
geometry:
- 'geometry={geometry#to_rounded_wkt}'
- '($.geometry.`str()`.`sub(/^\\[\\]$/, POLYGON((180 -90, 180 90, -180 90, -180 -90, 180 -90)))`)|($.geometry[*])'
# The url of the quicklook
quicklook: '$.properties.thumbnail'
# The url to download the product "as is" (literal or as a template to be completed either after the search result
# is obtained from the provider or during the eodag download phase)
downloadLink: 'https://catalogue.dataspace.copernicus.eu/odata/v1/Products({uid})/$value'
# storageStatus: must be one of ONLINE, STAGING, OFFLINE
storageStatus: '$.properties.status'
# Additional metadata provided by the providers but that don't appear in the reference spec
thumbnail: '$.properties.thumbnail'
download: !plugin
type: HTTPDownload
base_uri: 'https://catalogue.dataspace.copernicus.eu/odata/v1/Products'
extract: true
order_enabled: false
archive_depth: 2
ssl_verify: true
auth: !plugin
type: KeycloakOIDCPasswordAuth
auth_base_uri: 'https://identity.dataspace.copernicus.eu/auth'
realm: 'CDSE'
client_id: 'cdse-public'
client_secret: null
token_provision: qs
token_qs_key: 'token'
auth_error_code: 401
ssl_verify: true
products:
# S2
S2_MSI_L1C:
collection: Sentinel2
productType: S2MSI1C
S2_MSI_L2A:
collection: Sentinel2
productType: S2MSI2A
# S1
S1_SAR_RAW:
productType: RAW
collection: Sentinel1
metadata_mapping:
cloudCover: '$.null'
S1_SAR_GRD:
productType: GRD
collection: Sentinel1
metadata_mapping:
cloudCover: '$.null'
S1_SAR_SLC:
productType: SLC
collection: Sentinel1
metadata_mapping:
cloudCover: '$.null'
S1_SAR_OCN:
productType: OCN
collection: Sentinel1
metadata_mapping:
cloudCover: '$.null'
# S3 SRAL
S3_SRA:
productType: SR_1_SRA___
collection: Sentinel3
metadata_mapping:
cloudCover: '$.null'
S3_SRA_A:
productType: SR_1_SRA_A_
collection: Sentinel3
metadata_mapping:
cloudCover: '$.null'
S3_SRA_BS:
productType: SR_1_SRA_BS
collection: Sentinel3
metadata_mapping:
cloudCover: '$.null'
S3_LAN:
productType: SR_2_LAN___
collection: Sentinel3
metadata_mapping:
cloudCover: '$.null'
S3_WAT:
productType: SR_2_WAT___
collection: Sentinel3
metadata_mapping:
cloudCover: '$.null'
# S3 OLCI
S3_EFR:
productType: OL_1_EFR___
collection: Sentinel3
S3_ERR:
productType: OL_1_ERR___
collection: Sentinel3
S3_OLCI_L2LRR:
productType: OL_2_LRR___
collection: Sentinel3
S3_OLCI_L2LFR:
productType: OL_2_LFR___
collection: Sentinel3
S3_OLCI_L2WRR:
productType: OL_2_WRR___
collection: Sentinel3
S3_OLCI_L2WFR:
productType: OL_2_WFR___
collection: Sentinel3
# S3 SLSTR
S3_SLSTR_L1RBT:
productType: SL_1_RBT___
collection: Sentinel3
S3_SLSTR_L2LST:
productType: SL_2_LST___
collection: Sentinel3
S3_SLSTR_L2WST:
productType: SL_2_WST___
collection: Sentinel3
S3_SLSTR_L2AOD:
productType: SL_2_AOD___
collection: Sentinel3
S3_SLSTR_L2FRP:
productType: SL_2_FRP___
collection: Sentinel3
# S3 SY
S3_SY_AOD:
productType: SY_2_AOD___
collection: Sentinel3
S3_SY_SYN:
productType: SY_2_SYN___
collection: Sentinel3
S3_SY_V10:
productType: SY_2_V10___
collection: Sentinel3
S3_SY_VG1:
productType: SY_2_VG1___
collection: Sentinel3
S3_SY_VGP:
productType: SY_2_VGP___
collection: Sentinel3
# S5P L1
S5P_L1B_IR_SIR:
productType: L1B_IR_SIR
collection: Sentinel5P
S5P_L1B_IR_UVN:
productType: L1B_IR_UVN
collection: Sentinel5P
S5P_L1B_RA_BD1:
productType: L1B_RA_BD1
collection: Sentinel5P
S5P_L1B_RA_BD2:
productType: L1B_RA_BD2
collection: Sentinel5P
S5P_L1B_RA_BD3:
productType: L1B_RA_BD3
collection: Sentinel5P
S5P_L1B_RA_BD4:
productType: L1B_RA_BD4
collection: Sentinel5P
S5P_L1B_RA_BD5:
productType: L1B_RA_BD5
collection: Sentinel5P
S5P_L1B_RA_BD6:
productType: L1B_RA_BD6
collection: Sentinel5P
S5P_L1B_RA_BD7:
productType: L1B_RA_BD7
collection: Sentinel5P
S5P_L1B_RA_BD8:
productType: L1B_RA_BD8
collection: Sentinel5P
# S5P L2
S5P_L2_NO2:
productType: L2__NO2___
collection: Sentinel5P
S5P_L2_CLOUD:
productType: L2__CLOUD_
collection: Sentinel5P
S5P_L2_O3:
productType: L2__O3____
collection: Sentinel5P
S5P_L2_CO:
productType: L2__CO____
collection: Sentinel5P
S5P_L2_AER_AI:
productType: L2__AER_AI
collection: Sentinel5P
S5P_L2_O3_PR:
productType: L2__O3__PR
collection: Sentinel5P
S5P_L2_O3_TCL:
productType: L2__O3_TCL
collection: Sentinel5P
S5P_L2_AER_LH:
productType: L2__AER_LH
collection: Sentinel5P
S5P_L2_HCHO:
productType: L2__HCHO__
collection: Sentinel5P
S5P_L2_CH4:
productType: L2__CH4___
collection: Sentinel5P
S5P_L2_NP_BD3:
productType: L2__NP_BD3
collection: Sentinel5P
S5P_L2_NP_BD6:
productType: L2__NP_BD6
collection: Sentinel5P
S5P_L2_NP_BD7:
productType: L2__NP_BD7
collection: Sentinel5P
S5P_L2_SO2:
productType: L2__SO2___
collection: Sentinel5P
GENERIC_PRODUCT_TYPE:
productType: '{productType}'
collection: '{collection}'
---
!provider # MARK: planetary_computer
name: planetary_computer
priority: 0
roles:
- host
description: Microsoft Planetary Computer
url: https://planetarycomputer.microsoft.com
search: !plugin
type: StacSearch
api_endpoint: https://planetarycomputer.microsoft.com/api/stac/v1/search
need_auth: false
ssl_verify: true
pagination:
max_items_per_page: 1000
sort:
sort_param_mapping:
id: id
startTimeFromAscendingNode: properties.datetime
platformSerialIdentifier: properties.platform
metadata_mapping:
tileIdentifier:
- '{{"query":{{"s2:mgrs_tile":{{"eq":"{tileIdentifier}"}}}}}}'
- '$.properties."s2:mgrs_tile"'
products:
S1_SAR_GRD:
productType: sentinel-1-grd
metadata_mapping:
processingLevel:
- '{{"query":{{"s1:processing_level":{{"eq":"{processingLevel}"}}}}}}'
- '$.properties."s1:processing_level"'
S2_MSI_L2A:
productType: sentinel-2-l2a
LANDSAT_C2L1:
productType: landsat-c2-l1
LANDSAT_C2L2:
productType: landsat-c2-l2
MODIS_MCD43A4:
productType: modis-43A4-061
NAIP:
productType: naip
GENERIC_PRODUCT_TYPE:
productType: '{productType}'
download: !plugin
type: HTTPDownload
base_uri: 'https://planetarycomputer.microsoft.com/api/stac/download'
flatten_top_dirs: True
auth_error_code: 403
ssl_verify: true
auth: !plugin
type: SASAuth
auth_uri: 'https://planetarycomputer.microsoft.com/api/sas/v1/sign?href={url}'
signed_url_key: href
ssl_verify: true
headers:
Ocp-Apim-Subscription-Key: "{apikey}"
---
!provider # MARK: hydroweb_next
name: hydroweb_next
priority: 0
roles:
- host
description: hydroweb.next thematic hub for hydrology data access
url: https://hydroweb.next.theia-land.fr
search: !plugin
type: StacSearch
api_endpoint: https://hydroweb.next.theia-land.fr/api/v1/rs-catalog/stac/search
need_auth: true
auth_error_code: 401
ssl_verify: true
discover_queryables:
fetch_url: null
product_type_fetch_url: null
pagination:
max_items_per_page: 10_000
sort:
sort_by_default:
- !!python/tuple [startTimeFromAscendingNode, ASC]
sort_param_mapping:
id: id
startTimeFromAscendingNode: properties.start_datetime
completionTimeFromAscendingNode: properties.end_datetime
productVersion: properties.version
processingLevel: processing:level
metadata_mapping:
startTimeFromAscendingNode:
- '{{"query":{{"end_datetime":{{"gte":"{startTimeFromAscendingNode#to_iso_utc_datetime}"}}}}}}'
- '$.properties.start_datetime'
completionTimeFromAscendingNode:
- '{{"query":{{"start_datetime":{{"lte":"{completionTimeFromAscendingNode#to_iso_utc_datetime}"}}}}}}'
- '$.properties.end_datetime'
products:
GENERIC_PRODUCT_TYPE:
productType: '{productType}'
download: !plugin
type: HTTPDownload
base_uri: https://hydroweb.next.theia-land.fr
flatten_top_dirs: true
auth_error_code: 401
ssl_verify: true
auth: !plugin
type: HTTPHeaderAuth
ssl_verify: true
headers:
X-API-Key: "{apikey}"
---
!provider # MARK: wekeo
name: wekeo
priority: 0
roles:
- host
description: WEkEO - Copernicus and Sentinel data
url: https://www.wekeo.eu/
# anchors to avoid duplications
anchor_day_month_year: &day_month_year
startTimeFromAscendingNode:
- |
{{
"year": {startTimeFromAscendingNode#to_datetime_dict(list)}["year"],
"month": {startTimeFromAscendingNode#to_datetime_dict(list)}["month"],
"day": {startTimeFromAscendingNode#to_datetime_dict(list)}["day"]
}}
- $.properties.startDate
completionTimeFromAscendingNode: $.properties.endDate
anchor_hist_day_month_year: &hist_day_month_year
startTimeFromAscendingNode:
- |
{{
"hyear": {startTimeFromAscendingNode#to_datetime_dict(list)}["year"],
"hmonth": {startTimeFromAscendingNode#to_datetime_dict(list)}["month"],
"hday": {startTimeFromAscendingNode#to_datetime_dict(list)}["day"]
}}
- $.properties.startDate
completionTimeFromAscendingNode: $.properties.endDate
anchor_year_month: &month_year
startTimeFromAscendingNode:
- |
{{
"year": {startTimeFromAscendingNode#to_datetime_dict(list)}["year"],
"month": {startTimeFromAscendingNode#to_datetime_dict(list)}["month"]
}}
- $.properties.startDate
completionTimeFromAscendingNode: $.properties.endDate
anchor_hist_year_month: &hist_month_year
startTimeFromAscendingNode:
- |
{{
"hyear": {startTimeFromAscendingNode#to_datetime_dict(list)}["year"],
"hmonth": {startTimeFromAscendingNode#to_datetime_dict(list)}["month"]
}}
- $.properties.startDate
completionTimeFromAscendingNode: $.properties.endDate
anchor_dates_eum: &eo_eum_dates
startTimeFromAscendingNode:
- '{{"dtstart": "{startTimeFromAscendingNode}"}}'
- '$.properties.startdate'
completionTimeFromAscendingNode:
- '{{"dtend": "{completionTimeFromAscendingNode}"}}'
- '$.properties.enddate'
anchor_dates_clms: &clms_dates
startTimeFromAscendingNode:
- '{{"start": "{startTimeFromAscendingNode}"}}'
- '$.properties.startdate'
completionTimeFromAscendingNode:
- '{{"end": "{completionTimeFromAscendingNode}"}}'
- '$.properties.enddate'
anchor_s1_sar: &s1_sar_params
processingLevel:
- '{{"processingLevel": "{processingLevel}"}}'
- '$.id.`sub(/^[^_]([^_]+)_([^_]+)_([^_]+)_([0-4]+).*/, LEVEL\\4)`'
sensorMode:
- '{{"sensorMode": "{sensorMode}"}}'
- '$.id.`sub(/^[^_]([^_]+)_([^_]+)_.*/, \\2)`'
swath:
- '{{"swath": "{swath}"}}'
- '$.null'
polarisation:
- '{{"polarisation": "{polarisation}"}}'
- '$.null'
relativeOrbitNumber:
- '{{"relativeOrbitNumber": "{relativeOrbitNumber}"}}'
- '$.null'
missionTakeId:
- '{{"missionTakeId": "{missionTakeId}"}}'
- '$.null'
anchor_orbit_cycle: &orbit_cycle
relativeOrbitNumber:
- '{{"relativeOrbitNumber": "{relativeOrbitNumber}"}}'
- '$.null'
orbitNumber:
- '{{"orbit": "{orbitNumber}"}}'
- '$.null'
cycleNumber:
- '{{"cycle": "{cycleNumber}"}}'
- '$.null'
andchor_id_from_date: &id_from_date
id:
- |
{{
"start": "{id#replace_str(r'^.*_([0-9][0-9][0-9][0-9])([0-9][0-9])([0-9][0-9])([0-9][0-9])([0-9][0-9])_.*$',r'\1-\2-\3T\4%3A\5%3A00Z')}",
"end": "{id#replace_str(r'^.*_([0-9][0-9][0-9][0-9])([0-9][0-9])([0-9][0-9])([0-9][0-9])([0-9][0-9])_.*$',r'\1-\2-\3T\4%3A\5%3A00Z')}"
}}
- '$.id'
search: !plugin
type: PostJsonSearch
api_endpoint: 'https://gateway.prod.wekeo2.eu/hda-broker/api/v1/dataaccess/search'
need_auth: true
auth_error_code: 401
results_entry: 'features'
two_passes_id_search: true
pagination:
total_items_nb_key_path: '$.properties.totalResults'
next_page_query_obj: '{{"itemsPerPage":{items_per_page},"startIndex":{skip}}}'
max_items_per_page: 200
query_params_key: 'search'
discover_product_types:
fetch_url: null
metadata_mapping:
productType:
- '{{"dataset_id": "{productType}"}}'
- '$.null'
geometry:
- '{{"bbox": {geometry#to_bounds}}}'
- '$.geometry'
id:
- '{{"productIdentifier": "{id}"}}'
- '{$.id#remove_extension}'
startTimeFromAscendingNode:
- '{{"startDate": "{startTimeFromAscendingNode}"}}'
- '$.properties.startdate'
completionTimeFromAscendingNode:
- '{{"completionDate": "{completionTimeFromAscendingNode}"}}'
- '$.properties.enddate'
downloadLink: '$.properties.location'
title: '$.id'
storageStatus: 'OFFLINE'
processingLevel:
- '{{"processingLevel": "{processingLevel}"}}'
- '$.null'
providerProductType:
- '{{"productType": "{providerProductType}"}}'
- '$.null'
timeliness:
- '{{"timeliness": "{timeliness}"}}'
- '$.null'
orbitDirection:
- '{{"orbitDirection": "{orbitDirection}"}}'
- '$.null'
variable:
- '{{"variable": {variable}}}'
- '$.null'
leadtime_hour:
- '{{"leadtime_hour": {leadtime_hour}}}'
- '$.null'
leadtime_month:
- '{{"leadtime_month": {leadtime_month}}}'
- '$.null'
origin:
- '{{"origin": "{origin}"}}'
- '$.null'
system:
- '{{"system": "{system}"}}'
- '$.null'
format:
- '{{"format": "{format}"}}'
- '$.null'
pressure_level:
- '{{"pressure_level": {pressure_level}}}'
- '$.null'
model_level:
- '{{"model_level": {model_level}}}'
- '$.null'
sensor_and_algorithm:
- '{{"sensor_and_algorithm": "{sensor_and_algorithm}"}}'
- '$.null'
version:
- '{{"version": {version}}}'
- '$.null'
time:
- '{{"time": {time}}}'
- '$.null'
region:
- '{{"region": "{region}"}}'
- '$.null'
type:
- '{{"type": "{type}"}}'
- '$.null'
source:
- '{{"source": {source}}}'
- '$.null'
quantity:
- '{{"quantity": "{quantity}"}}'
- '$.null'
input_observations:
- '{{"input_observations": "{input_observations}"}}'
- '$.null'
aggregation:
- '{{"aggregation": "{aggregation}"}}'
- '$.null'
model:
- '{{"model": {model}}}'
- '$.null'
level:
- '{{"level": {level}}}'
- '$.null'
forcing_type:
- '{{"forcing_type": "{forcing_type}"}}'
- '$.null'
sky_type:
- '{{"sky_type": {sky_type}}}'
- '$.null'
band:
- '{{"band": {band}}}'
- '$.null'
aerosol_type:
- '{{"aerosol_type": {aerosol_type}}}'
- '$.null'
step:
- '{{"step": {step}}}'
- '$.null'
longitude:
- '{{"longitude": "{longitude}"}}'
- '$.null'
latitude:
- '{{"latitude": "{latitude}"}}'
- '$.null'
altitude:
- '{{"altitude": "{altitude}"}}'
- '$.null'
time_reference:
- '{{"time_reference": "{time_reference}"}}'
- '$.null'
grid:
- '{{"grid": "{grid}"}}'
- '$.null'
soil_level:
- '{{"soil_level": {soil_level}}}'
- '$.null'
products:
S1_SAR_GRD:
productType: EO:ESA:DAT:SENTINEL-1
providerProductType: GRD
metadata_mapping:
<<: *s1_sar_params
orderLink: 'https://gateway.prod.wekeo2.eu/hda-broker/api/v1/dataaccess/download?{{"location": "{downloadLink}","product_id":"{id}", "cacheable": "true", "dataset_id": "EO:ESA:DAT:SENTINEL-1"}}'
S1_SAR_RAW:
productType: EO:ESA:DAT:SENTINEL-1
providerProductType: RAW
metadata_mapping:
<<: *s1_sar_params
orderLink: 'https://gateway.prod.wekeo2.eu/hda-broker/api/v1/dataaccess/download?{{"location": "{downloadLink}","product_id":"{id}", "cacheable": "true", "dataset_id": "EO:ESA:DAT:SENTINEL-1"}}'
S1_SAR_OCN:
productType: EO:ESA:DAT:SENTINEL-1
providerProductType: OCN
metadata_mapping:
<<: *s1_sar_params
orderLink: 'https://gateway.prod.wekeo2.eu/hda-broker/api/v1/dataaccess/download?{{"location": "{downloadLink}","product_id":"{id}", "cacheable": "true", "dataset_id": "EO:ESA:DAT:SENTINEL-1"}}'
S1_SAR_SLC:
productType: EO:ESA:DAT:SENTINEL-1
providerProductType: SLC
metadata_mapping:
<<: *s1_sar_params
orderLink: 'https://gateway.prod.wekeo2.eu/hda-broker/api/v1/dataaccess/download?{{"location": "{downloadLink}","product_id":"{id}", "cacheable": "true", "dataset_id": "EO:ESA:DAT:SENTINEL-1"}}'
S2_MSI_L1C:
productType: EO:ESA:DAT:SENTINEL-2
processingLevel: S2MSI1C
metadata_mapping:
processingLevel:
- '{{"processingLevel": "{processingLevel}"}}'
- '$.id.`sub(/^[^_]([^_]+)_([^_]+)_.*/, S2\\2)`'
cloudCover:
- '{{"cloudCover": "{cloudCover}"}}'
- '$.null'
orderLink: 'https://gateway.prod.wekeo2.eu/hda-broker/api/v1/dataaccess/download?{{"location": "{downloadLink}","product_id":"{id}", "cacheable": "true", "dataset_id": "EO:ESA:DAT:SENTINEL-2"}}'
S2_MSI_L2A:
productType: EO:ESA:DAT:SENTINEL-2
processingLevel: S2MSI2A
metadata_mapping:
processingLevel:
- '{{"processingLevel": "{processingLevel}"}}'
- '$.id.`sub(/^[^_]([^_]+)_([^_]+)_.*/, S2\\2)`'
cloudCover:
- '{{"cloudCover": "{cloudCover}"}}'
- '$.null'
orderLink: 'https://gateway.prod.wekeo2.eu/hda-broker/api/v1/dataaccess/download?{{"location": "{downloadLink}","product_id":"{id}", "cacheable": "true", "dataset_id": "EO:ESA:DAT:SENTINEL-2"}}'
S2_MSI_L2AP:
productType: EO:ESA:DAT:SENTINEL-2
processingLevel: S2MSI2AP
metadata_mapping:
processingLevel:
- '{{"processingLevel": "{processingLevel}"}}'
- '$.id.`sub(/^[^_]([^_]+)_([^_]+)_.*/, S2\\2)`'
cloudCover:
- '{{"cloudCover": "{cloudCover}"}}'
- '$.null'
orderLink: 'https://gateway.prod.wekeo2.eu/hda-broker/api/v1/dataaccess/download?{{"location": "{downloadLink}","product_id":"{id}", "cacheable": "true", "dataset_id": "EO:ESA:DAT:SENTINEL-2"}}'
S3_LAN_HY:
productType: EO:ESA:DAT:SENTINEL-3
providerProductType: SR_2_LAN_HY
processingLevel: 2
metadata_mapping:
processingLevel:
- '{{"processingLevel": "{processingLevel}"}}'
- '2'
orderLink: 'https://gateway.prod.wekeo2.eu/hda-broker/api/v1/dataaccess/download?{{"location": "{downloadLink}","product_id":"{id}", "cacheable": "true", "dataset_id": "EO:ESA:DAT:SENTINEL-3"}}'
S3_LAN_SI:
productType: EO:ESA:DAT:SENTINEL-3
providerProductType: SR_2_LAN_SI
processingLevel: 2
metadata_mapping:
processingLevel:
- '{{"processingLevel": "{processingLevel}"}}'
- '2'
orderLink: 'https://gateway.prod.wekeo2.eu/hda-broker/api/v1/dataaccess/download?{{"location": "{downloadLink}","product_id":"{id}", "cacheable": "true", "dataset_id": "EO:ESA:DAT:SENTINEL-3"}}'
S3_LAN_LI:
productType: EO:ESA:DAT:SENTINEL-3
providerProductType: SR_2_LAN_LI
processingLevel: 2
metadata_mapping:
processingLevel:
- '{{"processingLevel": "{processingLevel}"}}'
- '2'
orderLink: 'https://gateway.prod.wekeo2.eu/hda-broker/api/v1/dataaccess/download?{{"location": "{downloadLink}","product_id":"{id}", "cacheable": "true", "dataset_id": "EO:ESA:DAT:SENTINEL-3"}}'
S3_OLCI_L2LFR:
productType: EO:ESA:DAT:SENTINEL-3
providerProductType: OL_2_LFR___
processingLevel: 2
metadata_mapping:
processingLevel:
- '{{"processingLevel": "{processingLevel}"}}'
- '2'
orderLink: 'https://gateway.prod.wekeo2.eu/hda-broker/api/v1/dataaccess/download?{{"location": "{downloadLink}","product_id":"{id}", "cacheable": "true", "dataset_id": "EO:ESA:DAT:SENTINEL-3"}}'
S3_OLCI_L2LRR:
productType: EO:ESA:DAT:SENTINEL-3
providerProductType: OL_2_LRR___
processingLevel: 2
metadata_mapping:
processingLevel:
- '{{"processingLevel": "{processingLevel}"}}'
- '2'
orderLink: 'https://gateway.prod.wekeo2.eu/hda-broker/api/v1/dataaccess/download?{{"location": "{downloadLink}","product_id":"{id}", "cacheable": "true", "dataset_id": "EO:ESA:DAT:SENTINEL-3"}}'
S3_SLSTR_L2:
productType: EO:ESA:DAT:SENTINEL-3
providerProductType: SL_2_LST___
processingLevel: 2
metadata_mapping:
processingLevel:
- '{{"processingLevel": "{processingLevel}"}}'
- '2'
orderLink: 'https://gateway.prod.wekeo2.eu/hda-broker/api/v1/dataaccess/download?{{"location": "{downloadLink}","product_id":"{id}", "cacheable": "true", "dataset_id": "EO:ESA:DAT:SENTINEL-3"}}'
S3_EFR:
productType: EO:EUM:DAT:SENTINEL-3:OL_1_EFR___
metadata_mapping:
id:
- '{{"timeliness": {id#split_id_into_s3_params}["timeliness"], "sat": {id#split_id_into_s3_params}["sat"], "dtstart": {id#split_id_into_s3_params}["startDate"], "dtend": {id#split_id_into_s3_params}["endDate"]}}'
- '{$.id#remove_extension}'
<<: *eo_eum_dates
platform:
- '{{"sat": "{platform}"}}'
- '$.id.`sub(/^[^_]([^_]+)_.*/, Sentinel-\\1)`'
platformSerialIdentifier: '$.id.`sub(/^[^_]([^_]+)_.*/, S\\1)`'
<<: *orbit_cycle
orderLink: 'https://gateway.prod.wekeo2.eu/hda-broker/api/v1/dataaccess/download?{{"location": "{downloadLink}","product_id":"{id}", "cacheable": "true", "dataset_id": "EO:EUM:DAT:SENTINEL-3:OL_1_EFR___"}}'
S3_ERR:
productType: EO:EUM:DAT:SENTINEL-3:OL_1_ERR___
metadata_mapping:
id:
- '{{"timeliness": {id#split_id_into_s3_params}["timeliness"], "sat": {id#split_id_into_s3_params}["sat"], "dtstart": {id#split_id_into_s3_params}["startDate"], "dtend": {id#split_id_into_s3_params}["endDate"]}}'
- '{$.id#remove_extension}'
<<: *eo_eum_dates
platform:
- '{{"sat": "{platform}"}}'
- '$.id.`sub(/^[^_]([^_]+)_.*/, Sentinel-\\1)`'
platformSerialIdentifier: '$.id.`sub(/^[^_]([^_]+)_.*/, S\\1)`'
<<: *orbit_cycle
orderLink: 'https://gateway.prod.wekeo2.eu/hda-broker/api/v1/dataaccess/download?{{"location": "{downloadLink}","product_id":"{id}", "cacheable": "true", "dataset_id": "EO:EUM:DAT:SENTINEL-3:OL_1_ERR___"}}'
S3_OLCI_L2WFR:
productType: EO:EUM:DAT:SENTINEL-3:OL_2_WFR___
metadata_mapping:
id:
- '{{"timeliness": {id#split_id_into_s3_params}["timeliness"], "sat": {id#split_id_into_s3_params}["sat"], "dtstart": {id#split_id_into_s3_params}["startDate"], "dtend": {id#split_id_into_s3_params}["endDate"]}}'
- '{$.id#remove_extension}'
<<: *eo_eum_dates
platform:
- '{{"sat": "{platform}"}}'
- '$.id.`sub(/^[^_]([^_]+)_.*/, Sentinel-\\1)`'
platformSerialIdentifier: '$.id.`sub(/^[^_]([^_]+)_.*/, S\\1)`'
<<: *orbit_cycle
orderLink: 'https://gateway.prod.wekeo2.eu/hda-broker/api/v1/dataaccess/download?{{"location": "{downloadLink}","product_id":"{id}", "cacheable": "true", "dataset_id": "EO:EUM:DAT:SENTINEL-3:OL_2_WFR___"}}'
S3_OLCI_L2WRR:
productType: EO:EUM:DAT:SENTINEL-3:OL_2_WRR___
metadata_mapping:
id:
- '{{"timeliness": {id#split_id_into_s3_params}["timeliness"], "sat": {id#split_id_into_s3_params}["sat"], "dtstart": {id#split_id_into_s3_params}["startDate"], "dtend": {id#split_id_into_s3_params}["endDate"]}}'
- '{$.id#remove_extension}'
<<: *eo_eum_dates
platform:
- '{{"sat": "{platform}"}}'
- '$.id.`sub(/^[^_]([^_]+)_.*/, Sentinel-\\1)`'
platformSerialIdentifier: '$.id.`sub(/^[^_]([^_]+)_.*/, S\\1)`'
<<: *orbit_cycle
orderLink: 'https://gateway.prod.wekeo2.eu/hda-broker/api/v1/dataaccess/download?{{"location": "{downloadLink}","product_id":"{id}", "cacheable": "true", "dataset_id": "EO:EUM:DAT:SENTINEL-3:OL_2_WRR___"}}'
S3_SRA:
productType: EO:EUM:DAT:SENTINEL-3:SR_1_SRA___
metadata_mapping:
id:
- '{{"timeliness": {id#split_id_into_s3_params}["timeliness"], "sat": {id#split_id_into_s3_params}["sat"], "dtstart": {id#split_id_into_s3_params}["startDate"], "dtend": {id#split_id_into_s3_params}["endDate"]}}'
- '{$.id#remove_extension}'
<<: *eo_eum_dates
platform:
- '{{"sat": "{platform}"}}'
- '$.id.`sub(/^[^_]([^_]+)_.*/, Sentinel-\\1)`'
platformSerialIdentifier: '$.id.`sub(/^[^_]([^_]+)_.*/, S\\1)`'
<<: *orbit_cycle
orderLink: 'https://gateway.prod.wekeo2.eu/hda-broker/api/v1/dataaccess/download?{{"location": "{downloadLink}","product_id":"{id}", "cacheable": "true", "dataset_id": "EO:EUM:DAT:SENTINEL-3:SR_1_SRA___"}}'
S3_SRA_A:
productType: EO:EUM:DAT:SENTINEL-3:SR_1_SRA_A_
metadata_mapping:
id:
- '{{"timeliness": {id#split_id_into_s3_params}["timeliness"], "sat": {id#split_id_into_s3_params}["sat"], "dtstart": {id#split_id_into_s3_params}["startDate"], "dtend": {id#split_id_into_s3_params}["endDate"]}}'
- '{$.id#remove_extension}'
<<: *eo_eum_dates
platform:
- '{{"sat": "{platform}"}}'
- '$.id.`sub(/^[^_]([^_]+)_.*/, Sentinel-\\1)`'
platformSerialIdentifier: '$.id.`sub(/^[^_]([^_]+)_.*/, S\\1)`'
<<: *orbit_cycle
orderLink: 'https://gateway.prod.wekeo2.eu/hda-broker/api/v1/dataaccess/download?{{"location": "{downloadLink}","product_id":"{id}", "cacheable": "true", "dataset_id": "EO:EUM:DAT:SENTINEL-3:SR_1_SRA_A_"}}'
S3_SRA_BS:
productType: EO:EUM:DAT:SENTINEL-3:SR_1_SRA_BS
metadata_mapping:
id:
- '{{"timeliness": {id#split_id_into_s3_params}["timeliness"], "sat": {id#split_id_into_s3_params}["sat"], "dtstart": {id#split_id_into_s3_params}["startDate"], "dtend": {id#split_id_into_s3_params}["endDate"]}}'
- '{$.id#remove_extension}'
<<: *eo_eum_dates
platform:
- '{{"sat": "{platform}"}}'
- '$.id.`sub(/^[^_]([^_]+)_.*/, Sentinel-\\1)`'
platformSerialIdentifier: '$.id.`sub(/^[^_]([^_]+)_.*/, S\\1)`'
<<: *orbit_cycle
orderLink: 'https://gateway.prod.wekeo2.eu/hda-broker/api/v1/dataaccess/download?{{"location": "{downloadLink}","product_id":"{id}", "cacheable": "true", "dataset_id": "EO:EUM:DAT:SENTINEL-3:SR_1_SRA_BS"}}'
S3_SLSTR_L1RBT:
productType: EO:EUM:DAT:SENTINEL-3:SL_1_RBT___
providerProductType: SL_1_RBT___
metadata_mapping:
id:
- '{{"productType", "value": "SL_1_RBT___", "timeliness": {id#split_id_into_s3_params}["timeliness"], "sat": {id#split_id_into_s3_params}["sat"], "dtstart": {id#split_id_into_s3_params}["startDate"], "dtend": {id#split_id_into_s3_params}["endDate"]}}'
- '{$.id#remove_extension}'
<<: *eo_eum_dates
platform:
- '{{"sat": "{platform}"}}'
- '$.id.`sub(/^[^_]([^_]+)_.*/, Sentinel-\\1)`'
platformSerialIdentifier: '$.id.`sub(/^[^_]([^_]+)_.*/, S\\1)`'
<<: *orbit_cycle
orderLink: 'https://gateway.prod.wekeo2.eu/hda-broker/api/v1/dataaccess/download?{{"location": "{downloadLink}","product_id":"{id}", "cacheable": "true", "dataset_id": "EO:EUM:DAT:SENTINEL-3:SL_1_RBT___"}}'
S3_WAT:
productType: EO:EUM:DAT:SENTINEL-3:SR_2_WAT___
metadata_mapping:
id:
- '{{"type": "SR_2_WAT___", "timeliness": {id#split_id_into_s3_params}["timeliness"], "sat": {id#split_id_into_s3_params}["sat"], "dtstart": {id#split_id_into_s3_params}["startDate"], "dtend": {id#split_id_into_s3_params}["endDate"]}}'
- '{$.id#remove_extension}'
<<: *eo_eum_dates
orderLink: 'https://gateway.prod.wekeo2.eu/hda-broker/api/v1/dataaccess/download?{{"location": "{downloadLink}","product_id":"{id}", "cacheable": "true", "dataset_id": "EO:EUM:DAT:SENTINEL-3:SR_2_WAT___"}}'
S5P_L1B_IR_ALL:
productType: EO:ESA:DAT:SENTINEL-5P
processingLevel: L1B
metadata_mapping:
processingLevel:
- '{{"processingLevel": "{processingLevel}"}}'
- '$.id.`sub(/^[^_]([^_]+)_([^_]+)_([^_]+)_.*/, \\3)`'
processingMode:
- '{{"processingMode": "{processingMode}"}}'
- '$.null'
orderLink: 'https://gateway.prod.wekeo2.eu/hda-broker/api/v1/dataaccess/download?{{"location": "{downloadLink}","product_id":"{id}", "cacheable": "true", "dataset_id": "EO:ESA:DAT:SENTINEL-5P"}}'
S5P_L2_IR_ALL:
productType: EO:ESA:DAT:SENTINEL-5P
processingLevel: L2
metadata_mapping:
processingLevel:
- '{{"processingLevel": "{processingLevel}"}}'
- '$.id.`sub(/^[^_]([^_]+)_([^_]+)_([^_]+)_.*/, \\3)`'
processingMode:
- '{{"processingMode": "{processingMode}"}}'
- '$.null'
orderLink: 'https://gateway.prod.wekeo2.eu/hda-broker/api/v1/dataaccess/download?{{"location": "{downloadLink}","product_id":"{id}", "cacheable": "true", "dataset_id": "EO:ESA:DAT:SENTINEL-5P"}}'
SEASONAL_ORIGINAL_SL:
productType: EO:ECMWF:DAT:SEASONAL_ORIGINAL_SINGLE_LEVELS
variable:
- 10m_u_component_of_wind
leadtime_hour:
- "0"
origin: ecmwf
system: "51"
format: grib
metadata_mapping:
id: '$.id'
<<: *day_month_year
origin:
- '{{"originating_centre": "{origin}"}}'
- '$.null'
defaultGeometry: 'POLYGON((180 -90, 180 90, -180 90, -180 -90, 180 -90))'
orderLink: 'https://gateway.prod.wekeo2.eu/hda-broker/api/v1/dataaccess/download?{{"location": "{downloadLink}","product_id":"{id}", "cacheable": "true", "dataset_id": "EO:ECMWF:DAT:SEASONAL_ORIGINAL_SINGLE_LEVELS"}}'
SEASONAL_ORIGINAL_PL:
productType: EO:ECMWF:DAT:SEASONAL_ORIGINAL_PRESSURE_LEVELS
variable:
- geopotential
pressure_level:
- "10"
leadtime_hour:
- "12"
origin: ecmwf
system: "51"
format: grib
metadata_mapping:
id: '$.id'
<<: *day_month_year
origin:
- '{{"originating_centre": "{origin}"}}'
- '$.null'
defaultGeometry: 'POLYGON((180 -90, 180 90, -180 90, -180 -90, 180 -90))'
orderLink: 'https://gateway.prod.wekeo2.eu/hda-broker/api/v1/dataaccess/download?{{"location": "{downloadLink}","product_id":"{id}", "cacheable": "true", "dataset_id": "EO:ECMWF:DAT:SEASONAL_ORIGINAL_PRESSURE_LEVELS"}}'
SEASONAL_POSTPROCESSED_SL:
productType: EO:ECMWF:DAT:SEASONAL_POSTPROCESSED_SINGLE_LEVELS
providerProductType:
- ensemble_mean
variable:
- 10m_u_component_of_wind_anomaly
leadtime_month:
- "1"
origin: ecmwf
system: "51"
format: grib
metadata_mapping:
id: '$.id'
providerProductType:
- '{{"product_type": {providerProductType}}}'
- '$.null'
<<: *month_year
origin:
- '{{"originating_centre": "{origin}"}}'
- '$.null'
defaultGeometry: 'POLYGON((180 -90, 180 90, -180 90, -180 -90, 180 -90))'
orderLink: 'https://gateway.prod.wekeo2.eu/hda-broker/api/v1/dataaccess/download?{{"location": "{downloadLink}","product_id":"{id}", "cacheable": "true", "dataset_id": "EO:ECMWF:DAT:SEASONAL_POSTPROCESSED_SINGLE_LEVELS"}}'
SEASONAL_POSTPROCESSED_PL:
productType: EO:ECMWF:DAT:SEASONAL_POSTPROCESSED_PRESSURE_LEVELS
providerProductType:
- ensemble_mean
variable:
- geopotential_anomaly
pressure_level:
- "10"
leadtime_month:
- "1"
origin: ecmwf
system: "5"
format: grib
metadata_mapping:
id: '$.id'
providerProductType:
- '{{"product_type": {providerProductType}}}'
- '$.null'
<<: *month_year
origin:
- '{{"originating_centre": "{origin}"}}'
- '$.null'
defaultGeometry: 'POLYGON((180 -90, 180 90, -180 90, -180 -90, 180 -90))'
orderLink: 'https://gateway.prod.wekeo2.eu/hda-broker/api/v1/dataaccess/download?{{"location": "{downloadLink}","product_id":"{id}", "cacheable": "true", "dataset_id": "EO:ECMWF:DAT:SEASONAL_POSTPROCESSED_PRESSURE_LEVELS"}}'
SEASONAL_MONTHLY_SL:
productType: EO:ECMWF:DAT:SEASONAL_MONTHLY_SINGLE_LEVELS
variable:
- 10m_u_component_of_wind
providerProductType:
- ensemble_mean
leadtime_month:
- "1"
origin: ecmwf
system: "51"
format: grib
metadata_mapping:
id: '$.id'
providerProductType:
- '{{"product_type": {providerProductType}}}'
- '$.null'
<<: *month_year
origin:
- '{{"originating_centre": "{origin}"}}'
- '$.null'
defaultGeometry: 'POLYGON((180 -90, 180 90, -180 90, -180 -90, 180 -90))'
orderLink: 'https://gateway.prod.wekeo2.eu/hda-broker/api/v1/dataaccess/download?{{"location": "{downloadLink}","product_id":"{id}", "cacheable": "true", "dataset_id": "EO:ECMWF:DAT:SEASONAL_MONTHLY_SINGLE_LEVELS"}}'
SEASONAL_MONTHLY_PL:
productType: EO:ECMWF:DAT:SEASONAL_MONTHLY_PRESSURE_LEVELS
variable:
- geopotential
- temperature
providerProductType:
- ensemble_mean
leadtime_month:
- "1"
pressure_level:
- "10"
origin: ecmwf
system: "51"
format: grib
metadata_mapping:
id: '$.id'
providerProductType:
- '{{"product_type": {providerProductType}}}'
- '$.null'
<<: *month_year
origin:
- '{{"originating_centre": "{origin}"}}'
- '$.null'
defaultGeometry: 'POLYGON((180 -90, 180 90, -180 90, -180 -90, 180 -90))'
orderLink: 'https://gateway.prod.wekeo2.eu/hda-broker/api/v1/dataaccess/download?{{"location": "{downloadLink}","product_id":"{id}", "cacheable": "true", "dataset_id": "EO:ECMWF:DAT:SEASONAL_MONTHLY_PRESSURE_LEVELS"}}'
SATELLITE_CARBON_DIOXIDE:
productType: EO:ECMWF:DAT:SATELLITE_CARBON_DIOXIDE
processingLevel:
- level_2
version:
- "4.0"
variable: xco2
sensor_and_algorithm: sciamachy_wfmd
format: zip
metadata_mapping:
id: '$.id'
<<: *day_month_year
variable:
- '{{"variable": "{variable}"}}'
- '$.null'
processingLevel:
- '{{"processing_level": {processingLevel}}}'
- '$.null'
defaultGeometry: 'POLYGON((180 -90, 180 90, -180 90, -180 -90, 180 -90))'
orderLink: 'https://gateway.prod.wekeo2.eu/hda-broker/api/v1/dataaccess/download?{{"location": "{downloadLink}","product_id":"{id}", "cacheable": "true", "dataset_id": "EO:ECMWF:DAT:SATELLITE_CARBON_DIOXIDE"}}'
GLACIERS_DIST_RANDOLPH:
productType: EO:ECMWF:DAT:INSITU_GLACIERS_EXTENT
variable:
- glacier_area
providerProductType:
- gridded
format: zip
version: "6_0"
metadata_mapping:
id: '$.id'
<<: *day_month_year
providerProductType:
- '{{"product_type": {providerProductType}}}'
- '$.null'
version:
- '{{"version": "{version}"}}'
- '$.null'
defaultGeometry: 'POLYGON((180 -90, 180 90, -180 90, -180 -90, 180 -90))'
orderLink: 'https://gateway.prod.wekeo2.eu/hda-broker/api/v1/dataaccess/download?{{"location": "{downloadLink}","product_id":"{id}", "cacheable": "true", "dataset_id": "EO:ECMWF:DAT:INSITU_GLACIERS_EXTENT"}}'
GRIDDED_GLACIERS_MASS_CHANGE:
productType: EO:ECMWF:DAT:DERIVED_GRIDDED_GLACIER_MASS_CHANGE
variable: glacier_mass_change
format: zip
version: "wgms_fog_2022_09"
metadata_mapping:
id: '$.id'
startTimeFromAscendingNode:
- |
{{
"hydrological_year": {startTimeFromAscendingNode#get_hydrological_year}
}}
- $.properties.startDate
completionTimeFromAscendingNode: $.properties.endDate
version:
- '{{"product_version": "{version}"}}'
- '$.null'
variable:
- '{{"variable": "{variable}"}}'
- '$.null'
defaultGeometry: 'POLYGON((180 -90, 180 90, -180 90, -180 -90, 180 -90))'
orderLink: 'https://gateway.prod.wekeo2.eu/hda-broker/api/v1/dataaccess/download?{{"location": "{downloadLink}","product_id":"{id}", "cacheable": "true", "dataset_id": "EO:ECMWF:DERIVED_GRIDDED_GLACIER_MASS_CHANGE"}}'
SATELLITE_METHANE:
productType: EO:ECMWF:DAT:SATELLITE_METHANE
processingLevel:
- level_2
version:
- "4.0"
variable: xch4
sensor_and_algorithm: sciamachy_wfmd
format: zip
metadata_mapping:
id: '$.id'
<<: *day_month_year
processingLevel:
- '{{"processing_level": {processingLevel}}}'
- '$.null'
variable:
- '{{"variable": "{variable}"}}'
- '$.null'
defaultGeometry: 'POLYGON((180 -90, 180 90, -180 90, -180 -90, 180 -90))'
orderLink: 'https://gateway.prod.wekeo2.eu/hda-broker/api/v1/dataaccess/download?{{"location": "{downloadLink}","product_id":"{id}", "cacheable": "true", "dataset_id": "EO:ECMWF:DAT:SATELLITE_METHANE"}}'
UERRA_EUROPE_SL:
productType: EO:ECMWF:DAT:REANALYSIS_UERRA_EUROPE_SINGLE_LEVELS
variable: total_cloud_cover
origin: uerra_harmonie
format: grib # netcdf format may fail
metadata_mapping:
id: '$.id'
<<: *day_month_year
variable:
- '{{"variable": "{variable}"}}'
- '$.null'
defaultGeometry: 'POLYGON((180 -90, 180 90, -180 90, -180 -90, 180 -90))'
orderLink: 'https://gateway.prod.wekeo2.eu/hda-broker/api/v1/dataaccess/download?{{"location": "{downloadLink}","product_id":"{id}", "cacheable": "true", "dataset_id": "EO:ECMWF:DAT:REANALYSIS_UERRA_EUROPE_SINGLE_LEVELS"}}'
SATELLITE_SEA_ICE_EDGE_TYPE:
productType: EO:ECMWF:DAT:SATELLITE_SEA_ICE_EDGE_TYPE
variable:
- sea_ice_type
region: northern_hemisphere
type: cdr
version: "3_0"
format: zip
metadata_mapping:
id: '$.id'
<<: *day_month_year
version:
- '{{"version": "{version}"}}'
- '$.null'
type:
- '{{"cdr_type": "{type}"}}'
- '$.null'
defaultGeometry: 'POLYGON((180 -90, 180 90, -180 90, -180 -90, 180 -90))'
orderLink: 'https://gateway.prod.wekeo2.eu/hda-broker/api/v1/dataaccess/download?{{"location": "{downloadLink}","product_id":"{id}", "cacheable": "true", "dataset_id": "EO:ECMWF:DAT:SATELLITE_SEA_ICE_EDGE_TYPE"}}'
SATELLITE_SEA_LEVEL_BLACK_SEA:
productType: EO:ECMWF:DAT:SATELLITE_SEA_LEVEL_BLACK_SEA
variable: all
format: zip
metadata_mapping:
id: '$.id'
<<: *day_month_year
variable:
- '{{"variable": "{variable}"}}'
- '$.null'
defaultGeometry: 'POLYGON((180 -90, 180 90, -180 90, -180 -90, 180 -90))'
orderLink: 'https://gateway.prod.wekeo2.eu/hda-broker/api/v1/dataaccess/download?{{"location": "{downloadLink}","product_id":"{id}", "cacheable": "true", "dataset_id": "EO:ECMWF:DAT:SATELLITE_SEA_LEVEL_BLACK_SEA"}}'
SATELLITE_SEA_LEVEL_GLOBAL:
productType: EO:ECMWF:DAT:SATELLITE_SEA_LEVEL_GLOBAL
variable:
- daily
format: zip
version: vDT2021
metadata_mapping:
id: '$.id'
<<: *day_month_year
version:
- '{{"version": "{version}"}}'
- '$.null'
defaultGeometry: 'POLYGON((180 -90, 180 90, -180 90, -180 -90, 180 -90))'
orderLink: 'https://gateway.prod.wekeo2.eu/hda-broker/api/v1/dataaccess/download?{{"location": "{downloadLink}","product_id":"{id}", "cacheable": "true", "dataset_id": "EO:ECMWF:DAT:SATELLITE_SEA_LEVEL_GLOBAL"}}'
SATELLITE_SEA_LEVEL_MEDITERRANEAN:
productType: EO:ECMWF:DAT:SATELLITE_SEA_LEVEL_MEDITERRANEAN
variable: all
format: zip
metadata_mapping:
id: '$.id'
<<: *day_month_year
variable:
- '{{"variable": "{variable}"}}'
- '$.null'
defaultGeometry: 'POLYGON((180 -90, 180 90, -180 90, -180 -90, 180 -90))'
orderLink: 'https://gateway.prod.wekeo2.eu/hda-broker/api/v1/dataaccess/download?{{"location": "{downloadLink}","product_id":"{id}", "cacheable": "true", "dataset_id": "EO:ECMWF:DAT:SATELLITE_SEA_LEVEL_MEDITERRANEAN"}}'
ERA5_SL:
productType: EO:ECMWF:DAT:REANALYSIS_ERA5_SINGLE_LEVELS
providerProductType:
- ensemble_mean
variable:
- 10m_u_component_of_wind
format: grib # netcdf format may fail
metadata_mapping:
id: '$.id'
startTimeFromAscendingNode:
- |
{{
"year": {startTimeFromAscendingNode#to_datetime_dict(string)}["year"],
"month": {startTimeFromAscendingNode#to_datetime_dict(list)}["month"],
"day": {startTimeFromAscendingNode#to_datetime_dict(list)}["day"],
"time": {startTimeFromAscendingNode#get_ecmwf_time}
}}
- $.properties.startDate
completionTimeFromAscendingNode: $.properties.endDate
providerProductType:
- '{{"product_type": {providerProductType}}}'
- '$.null'
defaultGeometry: 'POLYGON((180 -90, 180 90, -180 90, -180 -90, 180 -90))'
orderLink: 'https://gateway.prod.wekeo2.eu/hda-broker/api/v1/dataaccess/download?{{"location": "{downloadLink}","product_id":"{id}", "cacheable": "true", "dataset_id": "EO:ECMWF:DAT:REANALYSIS_ERA5_SINGLE_LEVELS"}}'
ERA5_PL:
productType: EO:ECMWF:DAT:REANALYSIS_ERA5_PRESSURE_LEVELS
providerProductType:
- ensemble_mean
variable:
- temperature
pressure_level:
- "1"
format: grib
metadata_mapping:
id: '$.id'
startTimeFromAscendingNode:
- |
{{
"year": {startTimeFromAscendingNode#to_datetime_dict(string)}["year"],
"month": {startTimeFromAscendingNode#to_datetime_dict(list)}["month"],
"day": {startTimeFromAscendingNode#to_datetime_dict(list)}["day"],
"time": {startTimeFromAscendingNode#get_ecmwf_time}
}}
- $.properties.startDate
completionTimeFromAscendingNode: $.properties.endDate
providerProductType:
- '{{"product_type": {providerProductType}}}'
- '$.null'
defaultGeometry: 'POLYGON((180 -90, 180 90, -180 90, -180 -90, 180 -90))'
orderLink: 'https://gateway.prod.wekeo2.eu/hda-broker/api/v1/dataaccess/download?{{"location": "{downloadLink}","product_id":"{id}", "cacheable": "true", "dataset_id": "EO:ECMWF:DAT:REANALYSIS_ERA5_PRESSURE_LEVELS"}}'
ERA5_SL_MONTHLY:
productType: EO:ECMWF:DAT:REANALYSIS_ERA5_SINGLE_LEVELS_MONTHLY_MEANS
providerProductType:
- monthly_averaged_ensemble_members
variable:
- 10m_u_component_of_wind
format: grib
time:
- "00:00"
metadata_mapping:
id: '$.id'
startTimeFromAscendingNode:
- |
{{
"year": {startTimeFromAscendingNode#to_datetime_dict(list)}["year"],
"month": {startTimeFromAscendingNode#to_datetime_dict(list)}["month"]
}}
- $.properties.startDate
completionTimeFromAscendingNode: $.properties.endDate
providerProductType:
- '{{"product_type": {providerProductType}}}'
- '$.null'
defaultGeometry: 'POLYGON((180 -90, 180 90, -180 90, -180 -90, 180 -90))'
orderLink: 'https://gateway.prod.wekeo2.eu/hda-broker/api/v1/dataaccess/download?{{"location": "{downloadLink}","product_id":"{id}", "cacheable": "true", "dataset_id": "EO:ECMWF:DAT:REANALYSIS_ERA5_SINGLE_LEVELS_MONTHLY_MEANS"}}'
ERA5_PL_MONTHLY:
productType: EO:ECMWF:DAT:REANALYSIS_ERA5_PRESSURE_LEVELS_MONTHLY_MEANS
providerProductType:
- monthly_averaged_ensemble_members
variable:
- divergence
pressure_level:
- "1"
format: grib
time:
- "00:00"
metadata_mapping:
id: '$.id'
startTimeFromAscendingNode:
- |
{{
"year": {startTimeFromAscendingNode#to_datetime_dict(list)}["year"],
"month": {startTimeFromAscendingNode#to_datetime_dict(list)}["month"]
}}
- $.properties.startDate
completionTimeFromAscendingNode: $.properties.endDate
providerProductType:
- '{{"product_type": {providerProductType}}}'
- '$.null'
defaultGeometry: 'POLYGON((180 -90, 180 90, -180 90, -180 -90, 180 -90))'
orderLink: 'https://gateway.prod.wekeo2.eu/hda-broker/api/v1/dataaccess/download?{{"location": "{downloadLink}","product_id":"{id}", "cacheable": "true", "dataset_id": "EO:ECMWF:DAT:REANALYSIS_ERA5_PRESSURE_LEVELS_MONTHLY_MEANS"}}'
ERA5_LAND:
productType: EO:ECMWF:DAT:REANALYSIS_ERA5_LAND
variable:
- 2m_dewpoint_temperature
format: grib
metadata_mapping:
id: '$.id'
startTimeFromAscendingNode:
- |
{{
"year": {startTimeFromAscendingNode#to_datetime_dict(string)}["year"],
"month": {startTimeFromAscendingNode#to_datetime_dict(string)}["month"],
"day": {startTimeFromAscendingNode#to_datetime_dict(list)}["day"],
"time": {startTimeFromAscendingNode#get_ecmwf_time}
}}
- $.properties.startDate
completionTimeFromAscendingNode: $.properties.endDate
defaultGeometry: 'POLYGON((180 -90, 180 90, -180 90, -180 -90, 180 -90))'
orderLink: 'https://gateway.prod.wekeo2.eu/hda-broker/api/v1/dataaccess/download?{{"location": "{downloadLink}","product_id":"{id}", "cacheable": "true", "dataset_id": "EO:ECMWF:DAT:REANALYSIS_ERA5_LAND"}}'
ERA5_LAND_MONTHLY:
productType: EO:ECMWF:DAT:REANALYSIS_ERA5_LAND_MONTHLY_MEANS
providerProductType:
- monthly_averaged_reanalysis
variable:
- 2m_dewpoint_temperature
format: grib
time:
- "00:00"
metadata_mapping:
id: '$.id'
startTimeFromAscendingNode:
- |
{{
"year": {startTimeFromAscendingNode#to_datetime_dict(list)}["year"],
"month": {startTimeFromAscendingNode#to_datetime_dict(list)}["month"]
}}
- $.properties.startDate
completionTimeFromAscendingNode: $.properties.endDate
providerProductType:
- '{{"product_type": {providerProductType}}}'
- '$.null'
defaultGeometry: 'POLYGON((180 -90, 180 90, -180 90, -180 -90, 180 -90))'
orderLink: 'https://gateway.prod.wekeo2.eu/hda-broker/api/v1/dataaccess/download?{{"location": "{downloadLink}","product_id":"{id}", "cacheable": "true", "dataset_id": "EO:ECMWF:DAT:REANALYSIS_ERA5_LAND_MONTHLY_MEANS"}}'
CAMS_EAC4:
productType: EO:ECMWF:DAT:CAMS_GLOBAL_REANALYSIS_EAC4
format: grib
variable:
- '2m_dewpoint_temperature'
time:
- '00:00'
metadata_mapping:
id: '$.id'
startTimeFromAscendingNode:
- '{{"dtstart": "{startTimeFromAscendingNode#to_iso_utc_datetime}"}}'
- '$.properties.startdate'
completionTimeFromAscendingNode:
- '{{"dtend": "{completionTimeFromAscendingNode#to_iso_utc_datetime}"}}'
- '$.properties.enddate'
defaultGeometry: 'POLYGON((180 -90, 180 90, -180 90, -180 -90, 180 -90))'
orderLink: 'https://gateway.prod.wekeo2.eu/hda-broker/api/v1/dataaccess/download?{{"location": "{downloadLink}","product_id":"{id}", "cacheable": "true", "dataset_id": "EO:ECMWF:DAT:CAMS_GLOBAL_REANALYSIS_EAC4"}}'
CAMS_GLOBAL_EMISSIONS:
productType: EO:ECMWF:DAT:CAMS_GLOBAL_EMISSION_INVENTORIES
version:
- latest
format: zip
variable:
- acids
source:
- anthropogenic
metadata_mapping:
id: '$.id'
startTimeFromAscendingNode:
- |
{{
"year": {startTimeFromAscendingNode#to_datetime_dict(list)}["year"]
}}
- $.properties.startDate
completionTimeFromAscendingNode: $.properties.endDate
defaultGeometry: 'POLYGON((180 -90, 180 90, -180 90, -180 -90, 180 -90))'
orderLink: 'https://gateway.prod.wekeo2.eu/hda-broker/api/v1/dataaccess/download?{{"location": "{downloadLink}","product_id":"{id}", "cacheable": "true", "dataset_id": "EO:ECMWF:DAT:CAMS_GLOBAL_EMISSION_INVENTORIES"}}'
CAMS_EAC4_MONTHLY:
productType: EO:ECMWF:DAT:CAMS_GLOBAL_REANALYSIS_EAC4_MONTHLY
format: grib
variable:
- 2m_dewpoint_temperature
api_product_type:
- monthly_mean
metadata_mapping:
id: '$.id'
<<: *month_year
providerProductType:
- '{{"product_type": {providerProductType}}}'
- '$.null'
defaultGeometry: 'POLYGON((180 -90, 180 90, -180 90, -180 -90, 180 -90))'
orderLink: 'https://gateway.prod.wekeo2.eu/hda-broker/api/v1/dataaccess/download?{{"location": "{downloadLink}","product_id":"{id}", "cacheable": "true", "dataset_id": "EO:ECMWF:DAT:CAMS_GLOBAL_REANALYSIS_EAC4_MONTHLY"}}'
CAMS_GREENHOUSE_INVERSION:
productType: EO:ECMWF:DAT:CAMS_GLOBAL_GREENHOUSE_GAS_INVERSION
version: latest
variable: carbon_dioxide
quantity: mean_column
input_observations: surface
aggregation: instantaneous
metadata_mapping:
id: '$.id'
<<: *month_year
aggregation:
- '{{"time_aggregation": "{aggregation}"}}'
- '$.null'
variable:
- '{{"variable": "{variable}"}}'
- '$.null'
version:
- '{{"version": "{version}"}}'
- '$.null'
defaultGeometry: 'POLYGON((180 -90, 180 90, -180 90, -180 -90, 180 -90))'
orderLink: 'https://gateway.prod.wekeo2.eu/hda-broker/api/v1/dataaccess/download?{{"location": "{downloadLink}","product_id":"{id}", "cacheable": "true", "dataset_id": "EO:ECMWF:DAT:CAMS_GLOBAL_GREENHOUSE_GAS_INVERSION"}}'
CAMS_EU_AIR_QUALITY_RE:
productType: EO:ECMWF:DAT:CAMS_EUROPE_AIR_QUALITY_REANALYSES
type:
- validated_reanalysis
format: zip
variable:
- nitrogen_dioxide
model:
- ensemble
level:
- '0'
metadata_mapping:
id: '$.id'
<<: *month_year
type:
- '{{"type": {type}}}'
- '$.null'
defaultGeometry: 'POLYGON((180 -90, 180 90, -180 90, -180 -90, 180 -90))'
orderLink: 'https://gateway.prod.wekeo2.eu/hda-broker/api/v1/dataaccess/download?{{"location": "{downloadLink}","product_id":"{id}", "cacheable": "true", "dataset_id": "EO:ECMWF:DAT:CAMS_EUROPE_AIR_QUALITY_REANALYSES"}}'
CAMS_GRF:
productType: EO:ECMWF:DAT:CAMS_GLOBAL_RADIATIVE_FORCINGS
format: zip
variable:
- radiative_forcing_of_carbon_dioxide
forcing_type: instantaneous
band:
- long_wave
sky_type:
- all_sky
level:
- surface
version:
- '2'
metadata_mapping:
id: '$.id'
<<: *month_year
defaultGeometry: 'POLYGON((180 -90, 180 90, -180 90, -180 -90, 180 -90))'
orderLink: 'https://gateway.prod.wekeo2.eu/hda-broker/api/v1/dataaccess/download?{{"location": "{downloadLink}","product_id":"{id}", "cacheable": "true", "dataset_id": "EO:ECMWF:DAT:CAMS_GLOBAL_RADIATIVE_FORCINGS"}}'
CAMS_GRF_AUX:
productType: EO:ECMWF:DAT:CAMS_GLOBAL_RADIATIVE_FORCING_AUXILLIARY_VARIABLES
band:
- short_wave
sky_type:
- clear_sky
version:
- '1.5'
format: zip
variable:
- aerosol_radiation_effect
aerosol_type:
- marine
level:
- surface
metadata_mapping:
id: '$.id'
<<: *month_year
defaultGeometry: 'POLYGON((180 -90, 180 90, -180 90, -180 -90, 180 -90))'
orderLink: 'https://gateway.prod.wekeo2.eu/hda-broker/api/v1/dataaccess/download?{{"location": "{downloadLink}","product_id":"{id}", "cacheable": "true", "dataset_id": "EO:ECMWF:DAT:CAMS_GLOBAL_RADIATIVE_FORCING_AUXILLIARY_VARIABLES"}}'
CAMS_GREENHOUSE_EGG4:
productType: EO:ECMWF:DAT:CAMS_GLOBAL_GHG_REANALYSIS_EGG4
format: grib
variable:
- snow_albedo
step:
- '0'
metadata_mapping:
id: '$.id'
startTimeFromAscendingNode:
- '{{"dtstart": "{startTimeFromAscendingNode#to_iso_utc_datetime}"}}'
- '$.properties.startdate'
completionTimeFromAscendingNode:
- '{{"dtend": "{completionTimeFromAscendingNode#to_iso_utc_datetime}"}}'
- '$.properties.enddate'
defaultGeometry: 'POLYGON((180 -90, 180 90, -180 90, -180 -90, 180 -90))'
orderLink: 'https://gateway.prod.wekeo2.eu/hda-broker/api/v1/dataaccess/download?{{"location": "{downloadLink}","product_id":"{id}", "cacheable": "true", "dataset_id": "EO:ECMWF:DAT:CAMS_GLOBAL_GHG_REANALYSIS_EGG4"}}'
CAMS_GREENHOUSE_EGG4_MONTHLY:
productType: EO:ECMWF:DAT:CAMS_GLOBAL_GHG_REANALYSIS_EGG4_MONTHLY
format: grib
variable:
- snow_albedo
providerProductType:
- monthly_mean_by_hour_of_day
step:
- '3'
time:
- '00:00'
metadata_mapping:
id: '$.id'
<<: *month_year
providerProductType:
- '{{"product_type": {providerProductType}}}'
- '$.null'
defaultGeometry: 'POLYGON((180 -90, 180 90, -180 90, -180 -90, 180 -90))'
orderLink: 'https://gateway.prod.wekeo2.eu/hda-broker/api/v1/dataaccess/download?{{"location": "{downloadLink}","product_id":"{id}", "cacheable": "true", "dataset_id": "EO:ECMWF:DAT:CAMS_GLOBAL_GHG_REANALYSIS_EGG4_MONTHLY"}}'
CAMS_EU_AIR_QUALITY_FORECAST:
productType: EO:ECMWF:DAT:CAMS_EUROPE_AIR_QUALITY_FORECASTS
model:
- ensemble
format: grib
variable:
- alder_pollen
type:
- forecast
time:
- '00:00'
level:
- '0'
leadtime_hour:
- '0'
metadata_mapping:
id: '$.id'
startTimeFromAscendingNode:
- '{{"dtstart": "{startTimeFromAscendingNode#to_iso_utc_datetime}"}}'
- '$.properties.startdate'
completionTimeFromAscendingNode:
- '{{"dtend": "{completionTimeFromAscendingNode#to_iso_utc_datetime}"}}'
- '$.properties.enddate'
type:
- '{{"type": {type}}}'
- '$.null'
defaultGeometry: 'POLYGON((180 -90, 180 90, -180 90, -180 -90, 180 -90))'
orderLink: 'https://gateway.prod.wekeo2.eu/hda-broker/api/v1/dataaccess/download?{{"location": "{downloadLink}","product_id":"{id}", "cacheable": "true", "dataset_id": "EO:ECMWF:DAT:CAMS_EUROPE_AIR_QUALITY_FORECASTS"}}'
CAMS_GAC_FORECAST:
productType: EO:ECMWF:DAT:CAMS_GLOBAL_ATMOSPHERIC_COMPOSITION_FORECASTS
type:
- forecast
format: grib
variable:
- ammonium_aerosol_optical_depth_550nm
time:
- '00:00'
leadtime_hour:
- '0'
metadata_mapping:
id: '$.id'
startTimeFromAscendingNode:
- '{{"dtstart": "{startTimeFromAscendingNode#to_iso_utc_datetime}"}}'
- '$.properties.startdate'
completionTimeFromAscendingNode:
- '{{"dtend": "{completionTimeFromAscendingNode#to_iso_utc_datetime}"}}'
- '$.properties.enddate'
type:
- '{{"type": {type}}}'
- '$.null'
defaultGeometry: 'POLYGON((180 -90, 180 90, -180 90, -180 -90, 180 -90))'
orderLink: 'https://gateway.prod.wekeo2.eu/hda-broker/api/v1/dataaccess/download?{{"location": "{downloadLink}","product_id":"{id}", "cacheable": "true", "dataset_id": "EO:ECMWF:DAT:CAMS_GLOBAL_ATMOSPHERIC_COMPOSITION_FORECASTS"}}'
CAMS_SOLAR_RADIATION:
productType: EO:ECMWF:DAT:CAMS_SOLAR_RADIATION_TIMESERIES
sky_type: clear
step: 1minute
time_reference: true_solar_time
altitude: -999
format: csv
metadata_mapping:
id: '$.id'
startTimeFromAscendingNode:
- '{{"dtstart": "{startTimeFromAscendingNode#to_iso_utc_datetime}"}}'
- '$.properties.startdate'
completionTimeFromAscendingNode:
- '{{"dtend": "{completionTimeFromAscendingNode#to_iso_utc_datetime}"}}'
- '$.properties.enddate'
step:
- '{{"time_step": "{step}"}}'
- '$.null'
sky_type:
- '{{"sky_type": "{sky_type}"}}'
- '$.null'
geometry:
- '{{"longitude": {geometry#to_longitude_latitude}["lon"], "latitude": {geometry#to_longitude_latitude}["lat"]}}'
- '$.null'
defaultGeometry: 'POLYGON((180 -90, 180 90, -180 90, -180 -90, 180 -90))'
orderLink: 'https://gateway.prod.wekeo2.eu/hda-broker/api/v1/dataaccess/download?{{"location": "{downloadLink}","product_id":"{id}", "cacheable": "true", "dataset_id": "EO:ECMWF:DAT:CAMS_SOLAR_RADIATION_TIMESERIES"}}'
EEA_DAILY_VI:
productType: EO:EEA:DAT:CLMS_HRVPP_VI
metadata_mapping:
id:
- '{{"uid": {id}}}'
- '$.id'
<<: *clms_dates
relativeOrbitNumber:
- '{{"relativeOrbitNumber": "{relativeOrbitNumber}"}}'
- '$.null'
version:
- '{{"productVersion": "{version}"}}'
- '$.null'
platformSerialIdentifier:
- '{{"platformSerialIdentifier": "{platformSerialIdentifier}"}}'
- '$.id.`sub(/^[^_]+_[^_]+_([^_]+)_.*/, \\1)`'
tileIdentifier:
- '{{"tileId": "{tileIdentifier}"}}'
- '$.null'
orderLink: 'https://gateway.prod.wekeo2.eu/hda-broker/api/v1/dataaccess/download?{{"location": "{downloadLink}","product_id":"{id}", "cacheable": "true", "dataset_id": "EO:EEA:DAT:CLMS_HRVPP_VI"}}'
COP_DEM_GLO30_DGED:
productType: EO:DEM:DAT:COP-DEM_GLO-30-DGED__2023_1
metadata_mapping:
id: '$.id'
orderLink: 'https://gateway.prod.wekeo2.eu/hda-broker/api/v1/dataaccess/download?{{"location": "{downloadLink}","product_id":"{id}", "cacheable": "true", "dataset_id": "EO:DEM:DAT:COP-DEM_GLO-30-DGED__2023_1"}}'
COP_DEM_GLO30_DTED:
productType: EO:DEM:DAT:COP-DEM_GLO-30-DTED__2023_1
metadata_mapping:
id: '$.id'
orderLink: 'https://gateway.prod.wekeo2.eu/hda-broker/api/v1/dataaccess/download?{{"location": "{downloadLink}","product_id":"{id}", "cacheable": "true", "dataset_id": "EO:DEM:DAT:COP-DEM_GLO-30-DTED__2023_1"}}'
COP_DEM_GLO90_DGED:
productType: EO:DEM:DAT:COP-DEM_GLO-90-DGED__2023_1
metadata_mapping:
id: '$.id'
orderLink: 'https://gateway.prod.wekeo2.eu/hda-broker/api/v1/dataaccess/download?{{"location": "{downloadLink}","product_id":"{id}", "cacheable": "true", "dataset_id": "EO:DEM:DAT:COP-DEM_GLO-90-DGED__2023_1"}}'
COP_DEM_GLO90_DTED:
productType: EO:DEM:DAT:COP-DEM_GLO-90-DTED__2023_1
metadata_mapping:
id: '$.id'
orderLink: 'https://gateway.prod.wekeo2.eu/hda-broker/api/v1/dataaccess/download?{{"location": "{downloadLink}","product_id":"{id}", "cacheable": "true", "dataset_id": "EO:DEM:DAT:COP-DEM_GLO-90-DTED__2023_1"}}'
CLMS_GLO_NDVI_333M:
productType: EO:CLMS:DAT:CLMS_GLOBAL_NDVI_300M_V1_10DAILY_NETCDF
metadata_mapping:
<<: *id_from_date
<<: *clms_dates
orderLink: 'https://gateway.prod.wekeo2.eu/hda-broker/api/v1/dataaccess/download?{{"location": "{downloadLink}","product_id":"{id}", "cacheable": "true", "dataset_id": "EO:CLMS:DAT:CLMS_GLOBAL_NDVI_300M_V1_10DAILY_NETCDF"}}'
CLMS_GLO_NDVI_1KM_LTS:
productType: EO:CLMS:DAT:CLMS_GLOBAL_NDVI_1KM_V2_10DAILY_NETCDF
metadata_mapping:
<<: *id_from_date
<<: *clms_dates
orderLink: 'https://gateway.prod.wekeo2.eu/hda-broker/api/v1/dataaccess/download?{{"location": "{downloadLink}","product_id":"{id}", "cacheable": "true", "dataset_id": "EO:CLMS:DAT:CLMS_GLOBAL_NDVI_1KM_V2_10DAILY_NETCDF"}}'
CLMS_CORINE:
productType: EO:CLMS:DAT:CORINE
providerProductType: Corine Land Cover 2018
format: GeoTiff100mt
metadata_mapping:
id: '$.id'
providerProductType:
- '{{"product_type": "{providerProductType}"}}'
- '$.null'
defaultGeometry: 'POLYGON((180 -90, 180 90, -180 90, -180 -90, 180 -90))'
orderLink: 'https://gateway.prod.wekeo2.eu/hda-broker/api/v1/dataaccess/download?{{"location": "{downloadLink}","product_id":"{id}", "cacheable": "true", "dataset_id": "EO:CLMS:DAT:CORINE"}}'
CLMS_GLO_FCOVER_333M:
productType: EO:CLMS:DAT:CLMS_GLOBAL_FCOVER_300M_V1_10DAILY_NETCDF
metadata_mapping:
<<: *id_from_date
<<: *clms_dates
productGroupId:
- '{{"productGroupId": "{productGroupId}"}}'
- '$.null'
orderLink: 'https://gateway.prod.wekeo2.eu/hda-broker/api/v1/dataaccess/download?{{"location": "{downloadLink}","product_id":"{id}", "cacheable": "true", "dataset_id": "EO:CLMS:DAT:CLMS_GLOBAL_FCOVER_300M_V1_10DAILY_NETCDF"}}'
CLMS_GLO_DMP_333M:
productType: EO:CLMS:DAT:CLMS_GLOBAL_DMP_300M_V1_10DAILY_NETCDF
metadata_mapping:
<<: *id_from_date
<<: *clms_dates
productGroupId:
- '{{"productGroupId": "{productGroupId}"}}'
- '$.null'
orderLink: 'https://gateway.prod.wekeo2.eu/hda-broker/api/v1/dataaccess/download?{{"location": "{downloadLink}","product_id":"{id}", "cacheable": "true", "dataset_id": "EO:CLMS:DAT:CLMS_GLOBAL_DMP_300M_V1_10DAILY_NETCDF"}}'
CLMS_GLO_GDMP_333M:
productType: EO:CLMS:DAT:CLMS_GLOBAL_GDMP_300M_V1_10DAILY_NETCDF
metadata_mapping:
<<: *id_from_date
<<: *clms_dates
productGroupId:
- '{{"productGroupId": "{productGroupId}"}}'
- '$.null'
orderLink: 'https://gateway.prod.wekeo2.eu/hda-broker/api/v1/dataaccess/download?{{"location": "{downloadLink}","product_id":"{id}", "cacheable": "true", "dataset_id": "EO:CLMS:DAT:CLMS_GLOBAL_GDMP_300M_V1_10DAILY_NETCDF"}}'
CLMS_GLO_FAPAR_333M:
productType: EO:CLMS:DAT:CLMS_GLOBAL_FAPAR_300M_V1_10DAILY_NETCDF
metadata_mapping:
<<: *id_from_date
<<: *clms_dates
productGroupId:
- '{{"productGroupId": "{productGroupId}"}}'
- '$.null'
orderLink: 'https://gateway.prod.wekeo2.eu/hda-broker/api/v1/dataaccess/download?{{"location": "{downloadLink}","product_id":"{id}", "cacheable": "true", "dataset_id": "EO:CLMS:DAT:CLMS_GLOBAL_FAPAR_300M_V1_10DAILY_NETCDF"}}'
CLMS_GLO_LAI_333M:
productType: EO:CLMS:DAT:CLMS_GLOBAL_LAI_300M_V1_10DAILY_NETCDF
metadata_mapping:
<<: *id_from_date
<<: *clms_dates
productGroupId:
- '{{"productGroupId": "{productGroupId}"}}'
- '$.null'
orderLink: 'https://gateway.prod.wekeo2.eu/hda-broker/api/v1/dataaccess/download?{{"location": "{downloadLink}","product_id":"{id}", "cacheable": "true", "dataset_id": "EO:CLMS:DAT:CLMS_GLOBAL_LAI_300M_V1_10DAILY_NETCDF"}}'
FIRE_HISTORICAL:
productType: EO:ECMWF:DAT:CEMS_FIRE_HISTORICAL_V1
providerProductType: reanalysis
variable:
- fire_danger_index
version:
- "4_1"
format:
- "grib"
grid: original_grid
type: consolidated_dataset
metadata_mapping:
id: '$.id'
<<: *day_month_year
providerProductType:
- '{{"product_type": "{providerProductType}"}}'
- '$.null'
version:
- '{{"system_version": {version}}}'
- '$.null'
format:
- '{{"format": {format}}}'
- '$.null'
type:
- '{{"dataset_type": "{type}"}}'
- '$.null'
defaultGeometry: 'POLYGON((180 -90, 180 90, -180 90, -180 -90, 180 -90))'
orderLink: 'https://gateway.prod.wekeo2.eu/hda-broker/api/v1/dataaccess/download?{{"location": "{downloadLink}","product_id":"{id}", "cacheable": "true", "dataset_id": "EO:ECMWF:DAT:CEMS_FIRE_HISTORICAL_V1"}}'
GLOFAS_FORECAST:
productType: EO:ECMWF:DAT:CEMS_GLOFAS_FORECAST
variable: river_discharge_in_the_last_24_hours
version:
- operational
leadtime_hour:
- "24"
model:
- lisflood
providerProductType:
- control_forecast
format: grib
metadata_mapping:
id: '$.id'
<<: *day_month_year
providerProductType:
- '{{"product_type": {providerProductType}}}'
- '$.null'
version:
- '{{"system_version": {version}}}'
- '$.null'
model:
- '{{"hydrological_model": {model}}}'
- '$.null'
variable:
- '{{"variable": "{variable}"}}'
- '$.null'
defaultGeometry: 'POLYGON((180 -90, 180 90, -180 90, -180 -90, 180 -90))'
orderLink: 'https://gateway.prod.wekeo2.eu/hda-broker/api/v1/dataaccess/download?{{"location": "{downloadLink}","product_id":"{id}", "cacheable": "true", "dataset_id": "EO:ECMWF:DAT:CEMS_GLOFAS_FORECAST"}}'
GLOFAS_HISTORICAL:
productType: EO:ECMWF:DAT:CEMS_GLOFAS_HISTORICAL
variable:
- river_discharge_in_the_last_24_hours
version:
- version_4_0
model:
- lisflood
providerProductType:
- consolidated
format: grib
metadata_mapping:
id: '$.id'
<<: *hist_day_month_year
providerProductType:
- '{{"product_type": {providerProductType}}}'
- '$.null'
version:
- '{{"system_version": {version}}}'
- '$.null'
model:
- '{{"hydrological_model": {model}}}'
- '$.null'
defaultGeometry: 'POLYGON((180 -90, 180 90, -180 90, -180 -90, 180 -90))'
orderLink: 'https://gateway.prod.wekeo2.eu/hda-broker/api/v1/dataaccess/download?{{"location": "{downloadLink}","product_id":"{id}", "cacheable": "true", "dataset_id": "EO:ECMWF:DAT:CEMS_GLOFAS_HISTORICAL"}}'
GLOFAS_REFORECAST:
productType: EO:ECMWF:DAT:CEMS_GLOFAS_REFORECAST
variable:
- river_discharge_in_the_last_24_hours
providerProductType:
- control_reforecast
leadtime_hour:
- "24"
version:
- version_4_0
model:
- lisflood
format: grib
metadata_mapping:
id: '$.id'
<<: *hist_day_month_year
providerProductType:
- '{{"product_type": {providerProductType}}}'
- '$.null'
version:
- '{{"system_version": {version}}}'
- '$.null'
model:
- '{{"hydrological_model": {model}}}'
- '$.null'
defaultGeometry: 'POLYGON((180 -90, 180 90, -180 90, -180 -90, 180 -90))'
orderLink: 'https://gateway.prod.wekeo2.eu/hda-broker/api/v1/dataaccess/download?{{"location": "{downloadLink}","product_id":"{id}", "cacheable": "true", "dataset_id": "EO:ECMWF:DAT:CEMS_GLOFAS_REFORECAST"}}'
GLOFAS_SEASONAL:
productType: EO:ECMWF:DAT:CEMS_GLOFAS_SEASONAL
variable:
- river_discharge_in_the_last_24_hours
version:
- operational
model:
- htessel_lisflood
leadtime_hour:
- "24"
format: grib
metadata_mapping:
id: '$.id'
<<: *month_year
version:
- '{{"system_version": {version}}}'
- '$.null'
model:
- '{{"hydrological_model": {model}}}'
- '$.null'
defaultGeometry: 'POLYGON((180 -90, 180 90, -180 90, -180 -90, 180 -90))'
orderLink: 'https://gateway.prod.wekeo2.eu/hda-broker/api/v1/dataaccess/download?{{"location": "{downloadLink}","product_id":"{id}", "cacheable": "true", "dataset_id": "EO:ECMWF:DAT:CEMS_GLOFAS_SEASONAL"}}'
GLOFAS_SEASONAL_REFORECAST:
productType: EO:ECMWF:DAT:CEMS_GLOFAS_SEASONAL_REFORECAST
variable:
- river_discharge_in_the_last_24_hours
version:
- version_4_0
model:
- lisflood
leadtime_hour:
- "24"
format: grib
metadata_mapping:
id: '$.id'
<<: *hist_month_year
version:
- '{{"system_version": {version}}}'
- '$.null'
model:
- '{{"hydrological_model": {model}}}'
- '$.null'
defaultGeometry: 'POLYGON((180 -90, 180 90, -180 90, -180 -90, 180 -90))'
orderLink: 'https://gateway.prod.wekeo2.eu/hda-broker/api/v1/dataaccess/download?{{"location": "{downloadLink}","product_id":"{id}", "cacheable": "true", "dataset_id": "EO:ECMWF:DAT:CEMS_GLOFAS_SEASONAL_REFORECAST"}}'
EFAS_FORECAST:
productType: EO:ECMWF:DAT:EFAS_FORECAST
providerProductType:
- control_forecast
soil_level:
- "1"
leadtime_hour:
- "0"
version:
- operational
origin: ecmwf
variable: volumetric_soil_moisture
model_level: soil_levels
time:
- 00:00
format: grib.zip
metadata_mapping:
id: '$.id'
<<: *day_month_year
providerProductType:
- '{{"product_type": {providerProductType}}}'
- '$.null'
version:
- '{{"system_version": {version}}}'
- '$.null'
model_level:
- '{{"model_levels": "{model_level}"}}'
- '$.null'
origin:
- '{{"originating_centre": "{origin}"}}'
- '$.null'
variable:
- '{{"variable": "{variable}"}}'
- '$.null'
defaultGeometry: 'POLYGON((180 -90, 180 90, -180 90, -180 -90, 180 -90))'
orderLink: 'https://gateway.prod.wekeo2.eu/hda-broker/api/v1/dataaccess/download?{{"location": "{downloadLink}","product_id":"{id}", "cacheable": "true", "dataset_id": "EO:ECMWF:DAT:EFAS_FORECAST"}}'
EFAS_HISTORICAL:
productType: EO:ECMWF:DAT:EFAS_HISTORICAL
version:
- version_5_0
variable: volumetric_soil_moisture
model_level: soil_levels
soil_level:
- "1"
time:
- 00:00
metadata_mapping:
id: '$.id'
<<: *hist_day_month_year
version:
- '{{"system_version": {version}}}'
- '$.null'
model_level:
- '{{"model_levels": "{model_level}"}}'
- '$.null'
variable:
- '{{"variable": "{variable}"}}'
- '$.null'
defaultGeometry: 'POLYGON((180 -90, 180 90, -180 90, -180 -90, 180 -90))'
orderLink: 'https://gateway.prod.wekeo2.eu/hda-broker/api/v1/dataaccess/download?{{"location": "{downloadLink}","product_id":"{id}", "cacheable": "true", "dataset_id": "EO:ECMWF:DAT:EFAS_HISTORICAL"}}'
EFAS_REFORECAST:
productType: EO:ECMWF:DAT:EFAS_REFORECAST
providerProductType:
- control_forecast
version:
- version_5_0
soil_level:
- "1"
leadtime_hour:
- "6"
variable: volumetric_soil_moisture
model_level: soil_levels
format: grib.zip
metadata_mapping:
id: '$.id'
<<: *hist_day_month_year
providerProductType:
- '{{"product_type": {providerProductType}}}'
- '$.null'
version:
- '{{"system_version": {version}}}'
- '$.null'
model_level:
- '{{"model_levels": "{model_level}"}}'
- '$.null'
variable:
- '{{"variable": "{variable}"}}'
- '$.null'
defaultGeometry: 'POLYGON((180 -90, 180 90, -180 90, -180 -90, 180 -90))'
orderLink: 'https://gateway.prod.wekeo2.eu/hda-broker/api/v1/dataaccess/download?{{"location": "{downloadLink}","product_id":"{id}", "cacheable": "true", "dataset_id": "EO:ECMWF:DAT:EFAS_REFORECAST"}}'
EFAS_SEASONAL:
productType: EO:ECMWF:DAT:EFAS_SEASONAL
version:
- operational
leadtime_hour:
- "24"
variable: volumetric_soil_moisture
model_level: soil_levels
soil_level:
- "1"
format: grib.zip
metadata_mapping:
id: '$.id'
<<: *month_year
version:
- '{{"system_version": {version}}}'
- '$.null'
model_level:
- '{{"model_levels": "{model_level}"}}'
- '$.null'
variable:
- '{{"variable": "{variable}"}}'
- '$.null'
defaultGeometry: 'POLYGON((180 -90, 180 90, -180 90, -180 -90, 180 -90))'
orderLink: 'https://gateway.prod.wekeo2.eu/hda-broker/api/v1/dataaccess/download?{{"location": "{downloadLink}","product_id":"{id}", "cacheable": "true", "dataset_id": "EO:ECMWF:DAT:EFAS_SEASONAL"}}'
EFAS_SEASONAL_REFORECAST:
productType: EO:ECMWF:DAT:EFAS_SEASONAL_REFORECAST
version:
- "version_5_0"
leadtime_hour:
- "24"
variable: volumetric_soil_moisture
model_level: soil_levels
soil_level:
- "1"
format: grib.zip
metadata_mapping:
id: '$.id'
<<: *hist_month_year
version:
- '{{"system_version": {version}}}'
- '$.null'
model_level:
- '{{"model_levels": "{model_level}"}}'
- '$.null'
variable:
- '{{"variable": "{variable}"}}'
- '$.null'
defaultGeometry: 'POLYGON((180 -90, 180 90, -180 90, -180 -90, 180 -90))'
orderLink: 'https://gateway.prod.wekeo2.eu/hda-broker/api/v1/dataaccess/download?{{"location": "{downloadLink}","product_id":"{id}", "cacheable": "true", "dataset_id": "EO:ECMWF:DAT:EFAS_SEASONAL_REFORECAST"}}'
auth: !plugin
type: TokenAuth
auth_uri: 'https://gateway.prod.wekeo2.eu/hda-broker/gettoken'
refresh_uri: 'https://gateway.prod.wekeo2.eu/hda-broker/refreshtoken'
token_type: json
token_key: access_token
refresh_token_key: refresh_token
headers:
Authorization: "Bearer {token}"
download: !plugin
type: HTTPDownload
base_uri: https://gateway.prod.wekeo2.eu/hda-broker/api/v1/dataaccess
flatten_top_dirs: true
auth_error_code: 401
order_enabled: true
order_method: 'POST'
order_on_response:
metadata_mapping:
order_id: '$.download_id'
orderStatusLink: 'https://gateway.prod.wekeo2.eu/hda-broker/api/v1/dataaccess/downloads?download_id={order_id}'
downloadLink: 'https://gateway.prod.wekeo2.eu/hda-broker/api/v1/dataaccess/download/{order_id}'
order_status_method: 'GET'
order_status_results_entry: 'features'
order_status_success:
status: Done
order_status_error:
status: Error
order_status_in_progress:
response_code: 202
products:
GLOFAS_FORECAST:
outputs_extension: .grib
GLOFAS_REFORECAST:
outputs_extension: .grib
GLOFAS_HISTORICAL:
outputs_extension: .grib
GLOFAS_SEASONAL:
outputs_extension: .grib
GLOFAS_SEASONAL_REFORECAST:
outputs_extension: .grib
FIRE_HISTORICAL:
outputs_extension: .grib
---
!provider # wekeo_cmems
name: wekeo_cmems
priority: 0
roles:
- host
description: WEkEO - Copernicus Marine Service
url: https://www.wekeo.eu/
search: !plugin
type: PostJsonSearch
api_endpoint: 'https://gateway.prod.wekeo2.eu/hda-broker/api/v1/dataaccess/search'
need_auth: true
auth_error_code: 401
results_entry: 'features'
two_passes_id_search: true
pagination:
total_items_nb_key_path: '$.properties.totalResults'
next_page_query_obj: '{{"itemsPerPage":{items_per_page},"startIndex":{skip}}}'
query_params_key: 'search'
discover_product_types:
fetch_url: https://gateway.prod.wekeo2.eu/hda-broker/api/v1/datasets?itemsPerPage=1400&q=EO:MO
single_collection_fetch_url: https://gateway.prod.wekeo2.eu/hda-broker/api/v1/datasets/{productType}
result_type: json
results_entry: 'features'
generic_product_type_id: '$.dataset_id'
generic_product_type_parsable_properties:
collection: '$.dataset_id'
generic_product_type_parsable_metadata:
abstract: '$.metadata.description'
license: '$.terms'
instrument: '$.null'
platform: '$.null'
platformSerialIdentifier: '$.null'
single_product_type_parsable_metadata:
title: '$.metadata._source.datasetTitle'
missionStartDate: '$.metadata._source.tempextent_begin'
missionEndDate: '$.metadata._source.tempextent_end'
processingLevel: '$.null'
keywords: '$.metadata._source.keywords'
metadata_mapping:
productType:
- '{{"dataset_id": "{productType}"}}'
- '$.null'
id:
- '{{"min_date": {id#dates_from_cmems_id}["min_date"], "max_date": {id#dates_from_cmems_id}["max_date"]}}'
- '$.id'
geometry:
- '{{"bbox": {geometry#to_bounds}}}'
- '$.geometry'
startTimeFromAscendingNode:
- '{{"min_date": "{startTimeFromAscendingNode#to_iso_utc_datetime}"}}'
- '$.properties.startdate'
completionTimeFromAscendingNode:
- '{{"max_date": "{completionTimeFromAscendingNode#to_iso_utc_datetime}"}}'
- '$.properties.enddate'
variable:
- '{{"variables": {variable}}}'
- '{$.properties.location#get_variables_from_path}'
downloadLink: '$.properties.location'
title: '$.id'
storageStatus: 'OFFLINE'
orderLink: 'https://gateway.prod.wekeo2.eu/hda-broker/api/v1/dataaccess/download?{{"location": "{downloadLink}","product_id":"{id}", "cacheable": "true", "dataset_id": "productType"}}'
products:
GENERIC_PRODUCT_TYPE:
productType: '{productType}'
auth: !plugin
type: TokenAuth
auth_uri: 'https://gateway.prod.wekeo2.eu/hda-broker/gettoken'
refresh_uri: 'https://gateway.prod.wekeo2.eu/hda-broker/refreshtoken'
token_type: json
token_key: access_token
refresh_token_key: refresh_token
headers:
Authorization: "Bearer {token}"
download: !plugin
type: HTTPDownload
base_uri: https://gateway.prod.wekeo2.eu/hda-broker/api/v1/dataaccess
flatten_top_dirs: true
auth_error_code: 401
order_enabled: true
order_method: 'POST'
order_on_response:
metadata_mapping:
order_id: '$.download_id'
orderStatusLink: 'https://gateway.prod.wekeo2.eu/hda-broker/api/v1/dataaccess/downloads?download_id={order_id}'
downloadLink: 'https://gateway.prod.wekeo2.eu/hda-broker/api/v1/dataaccess/download/{order_id}'
order_status_method: 'GET'
order_status_results_entry: 'features'
order_status_success:
status: Done
order_status_error:
status: Error
order_status_in_progress:
response_code: 202
---
!provider # MARK: creodias_s3
name: creodias_s3
priority: 0
description: CloudFerro DIAS data through S3 protocol
roles:
- host
url: https://creodias.eu/
search: !plugin
type: CreodiasS3Search
api_endpoint: 'http://datahub.creodias.eu/resto/api/collections/{collection}/search.json'
s3_endpoint: 'https://eodata.cloudferro.com'
need_auth: true
timeout: 60
ssl_verify: true
pagination:
next_page_url_tpl: '{url}?{search}&maxRecords={items_per_page}&page={page}&exactCount=1'
total_items_nb_key_path: '$.properties.totalResults'
max_items_per_page: 1_000
sort:
sort_by_default:
- !!python/tuple [startTimeFromAscendingNode, ASC]
sort_by_tpl: '&sortParam={sort_param}&sortOrder={sort_order}'
sort_param_mapping:
startTimeFromAscendingNode: startDate
completionTimeFromAscendingNode: completionDate
publicationDate: published
sort_order_mapping:
ascending: asc
descending: desc
max_sort_params: 1
discover_metadata:
auto_discovery: true
metadata_pattern: '^(?!collection)[a-zA-Z0-9]+$'
search_param: '{metadata}={{{metadata}}}'
metadata_path: '$.properties.*'
discover_product_types:
fetch_url: null
metadata_mapping:
uid: '$.id'
productType:
- productType
- '$.properties.productType'
platform: '$.properties.collection'
platformSerialIdentifier:
- platform
- '$.properties.platform'
instrument:
- instrument
- '$.properties.instrument'
processingLevel:
- processingLevel
- '$.properties.processingLevel'
# INSPIRE obligated OpenSearch Parameters for Collection Search (Table 4)
title: '{$.properties.title#remove_extension}'
topicCategory: '$.properties.topicCategory'
keyword: '$.properties.keywords'
abstract: '$.properties.description'
resolution:
- 'resolution'
- '$.properties.resolution'
organisationName:
- 'organisationName'
- '$.properties.organisationName'
publicationDate: '$.properties.published'
accessConstraint: '$.properties.license'
# OpenSearch Parameters for Product Search (Table 5)
parentIdentifier:
- 'parentIdentifier'
- '$.properties.parentIdentifier'
acquisitionType: '$.properties.acquisitionType'
orbitNumber:
- 'orbitNumber'
- '$.properties.orbitNumber'
orbitDirection:
- 'orbitDirection'
- '$.properties.orbitDirection'
swathIdentifier:
- 'swath'
- '$.properties.swath'
cloudCover:
- 'cloudCover=[0,{cloudCover}]'
- '$.properties.cloudCover'
snowCover:
- 'snowCover=[0,{snowCover}]'
- '$.properties.snowCover'
productVersion: '$.properties.version'
productQualityStatus: '$.properties.onlineQualityCheck'
processorName: '$.properties.processingName'
processingCenter: '$.properties.processingCenter'
creationDate: '$.properties.dhusIngestDate'
modificationDate: '$.properties.updated'
sensorMode:
- 'sensorMode'
- '$.properties.sensorMode'
# OpenSearch Parameters for Acquistion Parameters Search (Table 6)
startTimeFromAscendingNode:
- startDate
- '$.properties.startDate'
completionTimeFromAscendingNode:
- completionDate
- '$.properties.completionDate'
polarizationMode:
- 'polarisation'
- '$.properties.polarisation'
# Custom parameters (not defined in the base document referenced above)
id:
- 'productIdentifier=%{id#remove_extension}%'
- '{$.properties.title#remove_extension}'
tileIdentifier:
- tileId
- '$.null'
# The geographic extent of the product
geometry:
- 'geometry={geometry#to_rounded_wkt}'
# - '$.geometry'
- '($.geometry.`str()`.`sub(/^\\[\\]$/, POLYGON((180 -90, 180 90, -180 90, -180 -90, 180 -90)))`)|($.geometry[*])'
# The url of the quicklook
quicklook: '$.properties.thumbnail'
# The url to download the product "as is" (literal or as a template to be completed either after the search result
# is obtained from the provider or during the eodag download phase)
downloadLink: 'https://zipper.creodias.eu/download/{uid}'
# storageStatus must be one of ONLINE, STAGING, OFFLINE
storageStatus: '$.properties.status'
# Additional metadata provided by the providers but that don't appear in the reference spec
thumbnail: '$.properties.thumbnail'
download: !plugin
type: CreodiasS3Download
flatten_top_dirs: True
base_uri: 'https://eodata.cloudferro.com'
s3_bucket: 'eodata'
ssl_verify: true
auth: !plugin
type: AwsAuth
auth_error_code: 403
ssl_verify: true
products:
# S1
S1_SAR_RAW:
productType: RAW
collection: Sentinel1
metadata_mapping:
cloudCover: '$.null'
S1_SAR_GRD:
productType: GRD
collection: Sentinel1
metadata_mapping:
cloudCover: '$.null'
S1_SAR_SLC:
productType: SLC
collection: Sentinel1
metadata_mapping:
cloudCover: '$.null'
S1_SAR_OCN:
productType: OCN
collection: Sentinel1
metadata_mapping:
cloudCover: '$.null'
# S2
S2_MSI_L1C:
collection: SENTINEL-2
productType: S2MSI1C
prefixes_delimiter: ".jp2"
S2_MSI_L2A:
collection: SENTINEL-2
productType: S2MSI2A
# S3 SRAL
S3_SRA:
productType: SRA
collection: Sentinel3
metadata_mapping:
cloudCover: '$.null'
S3_SRA_A:
productType: SRA_A
collection: Sentinel3
metadata_mapping:
cloudCover: '$.null'
S3_SRA_BS:
productType: SRA_BS
collection: Sentinel3
metadata_mapping:
cloudCover: '$.null'
S3_LAN:
productType: LAN
collection: Sentinel3
metadata_mapping:
cloudCover: '$.null'
S3_WAT:
productType: WAT
collection: Sentinel3
metadata_mapping:
cloudCover: '$.null'
# S3 OLCI
S3_EFR:
productType: EFR
collection: Sentinel3
S3_ERR:
productType: ERR
collection: Sentinel3
S3_OLCI_L2LRR:
productType: LRR
collection: Sentinel3
S3_OLCI_L2LFR:
productType: LFR
collection: Sentinel3
S3_OLCI_L2WRR:
productType: WRR
collection: Sentinel3
S3_OLCI_L2WFR:
productType: WFR
collection: Sentinel3
# S3 SLSTR
S3_SLSTR_L1RBT:
productType: RBT
collection: Sentinel3
S3_SLSTR_L2LST:
productType: LST
collection: Sentinel3
S3_SLSTR_L2WST:
productType: WST
collection: Sentinel3
S3_SLSTR_L2AOD:
productType: AOD___
collection: Sentinel3
S3_SLSTR_L2FRP:
productType: FRP___
collection: Sentinel3
# S3 SY
S3_SY_AOD:
productType: SY_2_AOD___
collection: Sentinel3
S3_SY_SYN:
productType: SY_2_SYN___
collection: Sentinel3
S3_SY_V10:
productType: SY_2_V10___
collection: Sentinel3
S3_SY_VG1:
productType: SY_2_VG1___
collection: Sentinel3
S3_SY_VGP:
productType: SY_2_VGP___
collection: Sentinel3
# S5P L1
S5P_L1B_IR_SIR:
productType: L1B_IR_SIR
collection: Sentinel5P
S5P_L1B_IR_UVN:
productType: L1B_IR_UVN
collection: Sentinel5P
S5P_L1B_RA_BD1:
productType: L1B_RA_BD1
collection: Sentinel5P
S5P_L1B_RA_BD2:
productType: L1B_RA_BD2
collection: Sentinel5P
S5P_L1B_RA_BD3:
productType: L1B_RA_BD3
collection: Sentinel5P
S5P_L1B_RA_BD4:
productType: L1B_RA_BD4
collection: Sentinel5P
S5P_L1B_RA_BD5:
productType: L1B_RA_BD5
collection: Sentinel5P
S5P_L1B_RA_BD6:
productType: L1B_RA_BD6
collection: Sentinel5P
S5P_L1B_RA_BD7:
productType: L1B_RA_BD7
collection: Sentinel5P
S5P_L1B_RA_BD8:
productType: L1B_RA_BD8
collection: Sentinel5P
# S5P L2
S5P_L2_NO2:
productType: L2__NO2___
collection: Sentinel5P
S5P_L2_CLOUD:
productType: L2__CLOUD_
collection: Sentinel5P
S5P_L2_O3:
productType: L2__O3____
collection: Sentinel5P
S5P_L2_CO:
productType: L2__CO____
collection: Sentinel5P
S5P_L2_AER_AI:
productType: L2__AER_AI
collection: Sentinel5P
S5P_L2_O3_PR:
productType: L2__O3__PR
collection: Sentinel5P
S5P_L2_O3_TCL:
productType: L2__O3_TCL
collection: Sentinel5P
S5P_L2_AER_LH:
productType: L2__AER_LH
collection: Sentinel5P
S5P_L2_HCHO:
productType: L2__HCHO__
collection: Sentinel5P
S5P_L2_CH4:
productType: L2__CH4___
collection: Sentinel5P
S5P_L2_NP_BD3:
productType: L2__NP_BD3
collection: Sentinel5P
S5P_L2_NP_BD6:
productType: L2__NP_BD6
collection: Sentinel5P
S5P_L2_NP_BD7:
productType: L2__NP_BD7
collection: Sentinel5P
S5P_L2_SO2:
productType: L2__SO2___
collection: Sentinel5P
# COP DEM
COP_DEM_GLO30_DGED:
productType: DGE_30
collection: CopDem
metadata_mapping:
cloudCover: '$.null'
COP_DEM_GLO30_DTED:
productType: DTE_30
collection: CopDem
metadata_mapping:
cloudCover: '$.null'
COP_DEM_GLO90_DGED:
productType: DGE_90
collection: CopDem
metadata_mapping:
cloudCover: '$.null'
COP_DEM_GLO90_DTED:
productType: DTE_90
collection: CopDem
metadata_mapping:
cloudCover: '$.null'
GENERIC_PRODUCT_TYPE:
productType: '{productType}'
collection: '{collection}'
Parameters mapping#
EODAG maps each provider’s specific metadata parameters to a common model using OGC OpenSearch Extension for Earth Observation. Extra parameters having no equivalent in this model are mapped as is.
Depending on the provider, some parameters are queryable or not. This is configured in providers.yml:
If a parameter metadata-mapping is a list, the first element will help constructing the query (using format()), and the 2nd will help extracting its values from the query result (using jsonpath)
If a parameter metadata-mapping is a string, it will not be queryable and this string will help extracting its values from the query result (using jsonpath).
some_provider:
search:
metadata_mapping:
queryableParameter:
- 'this_is_query_string={queryableParameter}'
- '$.jsonpath.in.result.to.parameter'
nonQueryableParameter: '$.jsonpath.in.result.to.another_parameter'
The following tables list the parameters supported by providers, and if they are queryable or not.
OpenSearch parameters (CSV)#
parameter |
astraea_eod |
cop_ads |
cop_cds |
cop_dataspace |
creodias |
earth_search |
earth_search_cog |
earth_search_gcs |
ecmwf |
onda |
peps |
planetary_computer |
sara |
theia |
usgs_satapi_aws |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
abstract |
metadata only |
metadata only |
metadata only |
metadata only |
metadata only |
metadata only |
metadata only |
metadata only |
metadata only |
||||||
accessConstraint |
metadata only |
metadata only |
metadata only |
metadata only |
|||||||||||
acquisitionStation |
metadata only |
queryable metadata |
queryable metadata |
queryable metadata |
queryable metadata |
metadata only |
queryable metadata |
||||||||
acquisitionSubType |
metadata only |
queryable metadata |
queryable metadata |
queryable metadata |
queryable metadata |
metadata only |
queryable metadata |
||||||||
acquisitionType |
metadata only |
metadata only |
metadata only |
||||||||||||
antennaLookDirection |
metadata only |
||||||||||||||
archivingCenter |
metadata only |
||||||||||||||
availabilityTime |
metadata only |
queryable metadata |
queryable metadata |
queryable metadata |
queryable metadata |
metadata only |
queryable metadata |
||||||||
cloudCover |
metadata only |
queryable metadata |
queryable metadata |
queryable metadata |
queryable metadata |
queryable metadata |
queryable metadata |
queryable metadata |
queryable metadata |
queryable metadata |
queryable metadata |
queryable metadata |
|||
completionTimeFromAscendingNode |
queryable metadata |
queryable metadata |
queryable metadata |
queryable metadata |
queryable metadata |
queryable metadata |
queryable metadata |
queryable metadata |
queryable metadata |
queryable metadata |
queryable metadata |
queryable metadata |
queryable metadata |
queryable metadata |
|
creationDate |
metadata only |
metadata only |
queryable metadata |
queryable metadata |
queryable metadata |
metadata only |
metadata only |
queryable metadata |
metadata only |
metadata only |
queryable metadata |
||||
doi |
metadata only |
queryable metadata |
queryable metadata |
queryable metadata |
queryable metadata |
queryable metadata |
|||||||||
dopplerFrequency |
metadata only |
queryable metadata |
queryable metadata |
queryable metadata |
queryable metadata |
metadata only |
queryable metadata |
||||||||
illuminationAzimuthAngle |
metadata only |
queryable metadata |
queryable metadata |
queryable metadata |
queryable metadata |
metadata only |
queryable metadata |
||||||||
illuminationElevationAngle |
metadata only |
queryable metadata |
queryable metadata |
queryable metadata |
queryable metadata |
metadata only |
queryable metadata |
||||||||
illuminationZenithAngle |
metadata only |
||||||||||||||
incidenceAngleVariation |
metadata only |
||||||||||||||
instrument |
metadata only |
queryable metadata |
queryable metadata |
queryable metadata |
queryable metadata |
queryable metadata |
queryable metadata |
queryable metadata |
queryable metadata |
queryable metadata |
queryable metadata |
queryable metadata |
|||
keyword |
metadata only |
metadata only |
metadata only |
metadata only |
|||||||||||
lineage |
queryable metadata |
||||||||||||||
maximumIncidenceAngle |
metadata only |
||||||||||||||
minimumIncidenceAngle |
metadata only |
||||||||||||||
modificationDate |
metadata only |
metadata only |
metadata only |
queryable metadata |
queryable metadata |
queryable metadata |
metadata only |
queryable metadata |
metadata only |
metadata only |
queryable metadata |
||||
orbitDirection |
metadata only |
queryable metadata |
queryable metadata |
queryable metadata |
queryable metadata |
queryable metadata |
queryable metadata |
queryable metadata |
queryable metadata |
queryable metadata |
queryable metadata |
||||
orbitNumber |
metadata only |
queryable metadata |
queryable metadata |
queryable metadata |
queryable metadata |
queryable metadata |
queryable metadata |
queryable metadata |
queryable metadata |
queryable metadata |
metadata only |
queryable metadata |
|||
organisationName |
queryable metadata |
queryable metadata |
queryable metadata |
queryable metadata |
metadata only |
||||||||||
parentIdentifier |
queryable metadata |
queryable metadata |
queryable metadata |
metadata only |
|||||||||||
platform |
metadata only |
metadata only |
metadata only |
queryable metadata |
queryable metadata |
queryable metadata |
queryable metadata |
metadata only |
queryable metadata |
metadata only |
queryable metadata |
queryable metadata |
|||
platformSerialIdentifier |
metadata only |
queryable metadata |
queryable metadata |
queryable metadata |
metadata only |
queryable metadata |
queryable metadata |
queryable metadata |
queryable metadata |
queryable metadata |
queryable metadata |
queryable metadata |
|||
processingCenter |
metadata only |
metadata only |
metadata only |
||||||||||||
processingDate |
metadata only |
metadata only |
|||||||||||||
processingLevel |
metadata only |
queryable metadata |
queryable metadata |
queryable metadata |
queryable metadata |
queryable metadata |
queryable metadata |
queryable metadata |
queryable metadata |
queryable metadata |
queryable metadata |
queryable metadata |
|||
processingMode |
metadata only |
||||||||||||||
processorName |
metadata only |
metadata only |
metadata only |
||||||||||||
productQualityStatus |
metadata only |
queryable metadata |
metadata only |
metadata only |
|||||||||||
productType |
queryable metadata |
metadata only |
metadata only |
queryable metadata |
queryable metadata |
queryable metadata |
queryable metadata |
queryable metadata |
queryable metadata |
queryable metadata |
queryable metadata |
queryable metadata |
queryable metadata |
queryable metadata |
|
productVersion |
metadata only |
metadata only |
queryable metadata |
queryable metadata |
queryable metadata |
queryable metadata |
metadata only |
queryable metadata |
metadata only |
metadata only |
queryable metadata |
||||
publicationDate |
metadata only |
metadata only |
metadata only |
queryable metadata |
queryable metadata |
queryable metadata |
metadata only |
queryable metadata |
metadata only |
metadata only |
queryable metadata |
||||
resolution |
metadata only |
queryable metadata |
queryable metadata |
queryable metadata |
queryable metadata |
queryable metadata |
queryable metadata |
queryable metadata |
queryable metadata |
metadata only |
queryable metadata |
||||
sensorMode |
metadata only |
queryable metadata |
queryable metadata |
queryable metadata |
queryable metadata |
queryable metadata |
queryable metadata |
queryable metadata |
queryable metadata |
queryable metadata |
metadata only |
queryable metadata |
|||
sensorType |
queryable metadata |
||||||||||||||
snowCover |
queryable metadata |
queryable metadata |
queryable metadata |
metadata only |
|||||||||||
startTimeFromAscendingNode |
metadata only |
metadata only |
metadata only |
queryable metadata |
queryable metadata |
metadata only |
metadata only |
metadata only |
queryable metadata |
queryable metadata |
metadata only |
queryable metadata |
queryable metadata |
metadata only |
|
swathIdentifier |
queryable metadata |
queryable metadata |
queryable metadata |
||||||||||||
title |
metadata only |
metadata only |
metadata only |
metadata only |
metadata only |
metadata only |
metadata only |
metadata only |
metadata only |
metadata only |
metadata only |
metadata only |
metadata only |
metadata only |
|
topicCategory |
metadata only |
queryable metadata |
metadata only |
metadata only |
Provider/eodag specific parameters (CSV)#
parameter |
astraea_eod |
cop_ads |
cop_cds |
cop_dataspace |
creodias |
earth_search |
earth_search_cog |
earth_search_gcs |
ecmwf |
onda |
peps |
planetary_computer |
sara |
theia |
usgs_satapi_aws |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
_date |
metadata only |
metadata only |
|||||||||||||
accuracy |
queryable metadata |
queryable metadata |
|||||||||||||
aerosol_type |
queryable metadata |
||||||||||||||
altitude |
queryable metadata |
||||||||||||||
anoffset |
queryable metadata |
queryable metadata |
|||||||||||||
api_product_type |
queryable metadata |
queryable metadata |
|||||||||||||
assets |
metadata only |
metadata only |
metadata only |
metadata only |
metadata only |
metadata only |
|||||||||
awsProductId |
metadata only |
||||||||||||||
band |
queryable metadata |
||||||||||||||
bitmap |
queryable metadata |
queryable metadata |
|||||||||||||
block |
queryable metadata |
queryable metadata |
|||||||||||||
channel |
queryable metadata |
queryable metadata |
|||||||||||||
class |
queryable metadata |
queryable metadata |
|||||||||||||
database |
queryable metadata |
queryable metadata |
|||||||||||||
dataset |
queryable metadata |
queryable metadata |
|||||||||||||
dataset_type |
queryable metadata |
||||||||||||||
date_range |
queryable metadata |
queryable metadata |
|||||||||||||
day |
queryable metadata |
queryable metadata |
|||||||||||||
defaultGeometry |
metadata only |
metadata only |
|||||||||||||
diagnostic |
queryable metadata |
queryable metadata |
|||||||||||||
direction |
queryable metadata |
queryable metadata |
|||||||||||||
domain |
queryable metadata |
queryable metadata |
|||||||||||||
downloadLink |
metadata only |
metadata only |
metadata only |
metadata only |
metadata only |
metadata only |
metadata only |
metadata only |
metadata only |
metadata only |
metadata only |
metadata only |
metadata only |
metadata only |
|
duplicates |
queryable metadata |
queryable metadata |
|||||||||||||
ensemble_member |
queryable metadata |
||||||||||||||
expect |
queryable metadata |
queryable metadata |
|||||||||||||
experiment |
queryable metadata |
||||||||||||||
expver |
queryable metadata |
queryable metadata |
|||||||||||||
fcperiod |
queryable metadata |
queryable metadata |
|||||||||||||
fieldset |
queryable metadata |
queryable metadata |
|||||||||||||
filter |
queryable metadata |
queryable metadata |
|||||||||||||
forcing_type |
queryable metadata |
||||||||||||||
format |
queryable metadata |
queryable metadata |
|||||||||||||
frequency |
queryable metadata |
queryable metadata |
|||||||||||||
gcm |
queryable metadata |
||||||||||||||
geometry |
queryable metadata |
queryable metadata |
queryable metadata |
queryable metadata |
queryable metadata |
queryable metadata |
queryable metadata |
queryable metadata |
queryable metadata |
queryable metadata |
queryable metadata |
queryable metadata |
queryable metadata |
queryable metadata |
|
grid |
queryable metadata |
queryable metadata |
|||||||||||||
gridSquare |
queryable metadata |
||||||||||||||
hdate |
queryable metadata |
queryable metadata |
|||||||||||||
horizontal_resolution |
queryable metadata |
||||||||||||||
hydrological_model |
queryable metadata |
||||||||||||||
id |
queryable metadata |
metadata only |
metadata only |
queryable metadata |
queryable metadata |
queryable metadata |
queryable metadata |
queryable metadata |
queryable metadata |
queryable metadata |
queryable metadata |
queryable metadata |
metadata only |
queryable metadata |
|
ident |
queryable metadata |
queryable metadata |
|||||||||||||
input_observations |
queryable metadata |
||||||||||||||
interpolation |
queryable metadata |
queryable metadata |
|||||||||||||
intgrid |
queryable metadata |
queryable metadata |
|||||||||||||
iteration |
queryable metadata |
queryable metadata |
|||||||||||||
latitude |
queryable metadata |
queryable metadata |
|||||||||||||
latitudeBand |
queryable metadata |
||||||||||||||
leadtime_hour |
queryable metadata |
queryable metadata |
|||||||||||||
leadtime_month |
queryable metadata |
||||||||||||||
level |
queryable metadata |
||||||||||||||
levelist |
queryable metadata |
queryable metadata |
|||||||||||||
levtype |
queryable metadata |
queryable metadata |
|||||||||||||
location |
queryable metadata |
||||||||||||||
longitude |
queryable metadata |
queryable metadata |
|||||||||||||
lsm |
queryable metadata |
queryable metadata |
|||||||||||||
method |
queryable metadata |
queryable metadata |
|||||||||||||
model |
queryable metadata |
||||||||||||||
model_level |
queryable metadata |
||||||||||||||
model_levels |
queryable metadata |
||||||||||||||
month |
queryable metadata |
queryable metadata |
|||||||||||||
number |
queryable metadata |
queryable metadata |
|||||||||||||
obsgroup |
queryable metadata |
queryable metadata |
|||||||||||||
obstype |
queryable metadata |
queryable metadata |
|||||||||||||
orderLink |
metadata only |
metadata only |
metadata only |
||||||||||||
origin |
queryable metadata |
queryable metadata |
|||||||||||||
originating_centre |
queryable metadata |
||||||||||||||
packing |
queryable metadata |
queryable metadata |
|||||||||||||
padding |
queryable metadata |
queryable metadata |
|||||||||||||
param |
queryable metadata |
queryable metadata |
|||||||||||||
period |
queryable metadata |
||||||||||||||
polarizationChannels |
metadata only |
queryable metadata |
queryable metadata |
queryable metadata |
queryable metadata |
metadata only |
queryable metadata |
metadata only |
queryable metadata |
||||||
polarizationMode |
queryable metadata |
metadata only |
queryable metadata |
queryable metadata |
metadata only |
||||||||||
pressure_level |
queryable metadata |
queryable metadata |
|||||||||||||
priority |
queryable metadata |
queryable metadata |
|||||||||||||
processing_level |
queryable metadata |
||||||||||||||
processing_type |
queryable metadata |
||||||||||||||
product |
queryable metadata |
queryable metadata |
|||||||||||||
quantity |
queryable metadata |
||||||||||||||
quicklook |
metadata only |
metadata only |
metadata only |
metadata only |
metadata only |
metadata only |
metadata only |
metadata only |
metadata only |
metadata only |
metadata only |
metadata only |
|||
range |
queryable metadata |
queryable metadata |
|||||||||||||
rcm |
queryable metadata |
||||||||||||||
refdate |
queryable metadata |
queryable metadata |
|||||||||||||
reference |
queryable metadata |
queryable metadata |
|||||||||||||
reportype |
queryable metadata |
queryable metadata |
|||||||||||||
repres |
queryable metadata |
queryable metadata |
|||||||||||||
resol |
queryable metadata |
queryable metadata |
|||||||||||||
rotation |
queryable metadata |
queryable metadata |
|||||||||||||
section |
queryable metadata |
queryable metadata |
|||||||||||||
sensor_and_algorithm |
queryable metadata |
||||||||||||||
sky_type |
queryable metadata |
||||||||||||||
source |
queryable metadata |
queryable metadata |
|||||||||||||
step |
queryable metadata |
queryable metadata |
|||||||||||||
storageStatus |
metadata only |
metadata only |
metadata only |
metadata only |
metadata only |
metadata only |
metadata only |
metadata only |
metadata only |
metadata only |
metadata only |
metadata only |
metadata only |
metadata only |
|
stream |
queryable metadata |
queryable metadata |
|||||||||||||
system |
queryable metadata |
queryable metadata |
|||||||||||||
system_version |
queryable metadata |
||||||||||||||
target |
queryable metadata |
queryable metadata |
|||||||||||||
thumbnail |
metadata only |
metadata only |
metadata only |
metadata only |
metadata only |
metadata only |
metadata only |
metadata only |
metadata only |
metadata only |
metadata only |
||||
tileIdentifier |
queryable metadata |
queryable metadata |
queryable metadata |
queryable metadata |
queryable metadata |
queryable metadata |
queryable metadata |
||||||||
time |
queryable metadata |
queryable metadata |
|||||||||||||
time_aggregation |
queryable metadata |
queryable metadata |
|||||||||||||
time_reference |
queryable metadata |
||||||||||||||
time_step |
queryable metadata |
||||||||||||||
truncation |
queryable metadata |
queryable metadata |
|||||||||||||
type |
queryable metadata |
queryable metadata |
|||||||||||||
uid |
metadata only |
metadata only |
metadata only |
metadata only |
metadata only |
metadata only |
|||||||||
use |
queryable metadata |
queryable metadata |
|||||||||||||
utmZone |
queryable metadata |
||||||||||||||
variable |
queryable metadata |
queryable metadata |
|||||||||||||
variable_type |
queryable metadata |
||||||||||||||
version |
queryable metadata |
queryable metadata |
|||||||||||||
year |
queryable metadata |
queryable metadata |