Add a Provider
Contents
Add a Provider#
eodag provides a set of plugins which don’t know anything about the
providers themselves, they just implement generic methods required to talk to different kinds of data catalog. For instance:
QueryStringSearch: Search plugin that implements a search protocol that relies on query stringsHTTPHeaderAuth: Authentication plugin that implements HTTP authentication using headersHTTPDownload: Download plugin that implements download over HTTP protocol
Configure a new provider#
The simplest way to add a new provider is to configure existing plugins. This approach requires to
provide the new provider’s configuration in a YAML format. The following example, extracted from
the STAC client page, shows how to add a new STAC provider:
tamn:
search:
type: StacSearch
api_endpoint: https://tamn.snapplanet.io/search
need_auth: false
products:
S2_MSI_L1C:
productType: S2
GENERIC_PRODUCT_TYPE:
productType: '{productType}'
download:
type: AwsDownload
base_uri: https://tamn.snapplanet.io
flatten_top_dirs: True
auth:
type: AwsAuth
credentials:
aws_access_key_id: PLEASE_CHANGE_ME
aws_secret_access_key: PLEASE_CHANGE_ME
It configures the following existing plugins: StacSearch (search),
AwsAuth (authentication) and AwsDownload (download).
Of course, it is also necessary to know how to configure these plugins (which parameters they take, what values they can have, etc.).
You can get some inspiration from the Providers pre-configuration section by analysing how eodag configures the providers it comes installed with.
Add more plugins#
eodag is a plugin-oriented framework which means it can be easily extended. If the plugins it offers are not sufficient for your
own needs (i.e. getting data from a provider not supported by eodag), you should then write your own plugins (possibly by extending one of provided by eodag)
and configure them. What you are the most likely to be willing to do is either to develop a new Search plugin or an
Api plugin (e.g. to create an interface with another program).
eodag-sentinelsat is a good example of an Api plugin. It creates an interface with the sentinalsat library to search and download products from SciHub.
See more details about how to create a new plugin in this dedicated section.
Providers pre-configuration#
All the providers are pre-configured in eodag in a YAML file.
Click on the link below to display its full content.
providers.yml
---
!provider
name: usgs
priority: 0
description: U.S geological survey catalog for Landsat products
roles:
- host
url: https://earthexplorer.usgs.gov/
api: !plugin
type: UsgsApi
need_auth: true
google_base_url: 'http://storage.googleapis.com/earthengine-public/landsat/'
pagination:
max_items_per_page: 5000
total_items_nb_key_path: '$.totalHits'
metadata_mapping:
id: '$.displayId'
geometry: '$.spatialBounds'
productType: '$.productType'
title: '$.displayId'
abstract: '$.summary'
cloudCover: '$.cloudCover'
startTimeFromAscendingNode: '$.temporalCoverage.startDate'
completionTimeFromAscendingNode: '$.temporalCoverage.endDate'
publicationDate: '$.publishDate'
thumbnail: '$.browse[0].thumbnailPath'
quicklook: '$.browse[0].browsePath'
storageStatus: '{$.available#get_group_name((?P<ONLINE>True)|(?P<OFFLINE>False))}'
downloadLink: 'https://earthexplorer.usgs.gov/download/external/options/{productType}/{entityId}/M2M/'
# metadata needed for download
entityId: '$.entityId'
productId: '$.id'
extract: True
order_enabled: true
products:
# datasets list http://kapadia.github.io/usgs/_sources/reference/catalog/ee.txt may be outdated
# see also https://dds.cr.usgs.gov/ee-data/coveragemaps/shp/ee/
LANDSAT_C2L1:
dataset: landsat_ot_c2_l1
outputs_extension: .tar.gz
LANDSAT_C2L2:
dataset: landsat_ot_c2_l2
outputs_extension: .tar.gz
LANDSAT_TM_C1:
dataset: landsat_tm_c1
outputs_extension: .tar.gz
LANDSAT_TM_C2L1:
dataset: landsat_tm_c2_l1
outputs_extension: .tar.gz
LANDSAT_TM_C2L2:
dataset: landsat_tm_c2_l2
outputs_extension: .tar.gz
LANDSAT_ETM_C1:
dataset: landsat_etm_c1
outputs_extension: .tar.gz
LANDSAT_ETM_C2L1:
dataset: landsat_etm_c2_l1
outputs_extension: .tar.gz
LANDSAT_ETM_C2L2:
dataset: landsat_etm_c2_l2
outputs_extension: .tar.gz
S2_MSI_L1C:
dataset: SENTINEL_2A
outputs_extension: .zip
GENERIC_PRODUCT_TYPE:
dataset: '{productType}'
---
!provider
name: aws_eos
priority: 0
description: EOS search for Amazon public datasets
roles:
- host
url: https://developers.eos.com/datasets_description.html
search: !plugin
type: PostJsonSearch
api_endpoint: 'https://gate.eos.com/api/lms/search/v2/{collection}?api_key={apikey}'
need_auth: true
auth_error_code: 403
results_entry: 'results'
pagination:
next_page_query_obj: '{{"limit":{items_per_page},"page":{page}}}'
total_items_nb_key_path: '$.meta.found'
# 2021/04/28: aws_eos doesn't specify a limit in its docs. It says that the default
# value is 500 (https://doc.eos.com/search.api/#single-dataset-search).
# Let's set it to this value for now
max_items_per_page: 500
query_params_key: 'search'
discover_metadata:
auto_discovery: true
metadata_pattern: '^[a-zA-Z0-9_]+$'
search_param: '{{{{"search":{{{{"{metadata}":"{{{metadata}}}" }}}} }}}}'
metadata_path: '$.*'
metadata_mapping:
# landsat8_downloadLink : 's3://landsat-pds/c{storedInCollection}/L8/{path}/{row}/{productID}'
geometry:
- '{{"search":{{"shape": {geometry#to_geojson} }} }}'
- '$.dataGeometry'
# storageStatus set to ONLINE for consistency between providers
storageStatus: '{$.null#replace_str("Not Available","ONLINE")}'
products:
CBERS4_PAN10M_L2:
instrument: PAN10M
collection: cbers4
processingLevel: 2
metadata_mapping:
# OpenSearch Parameters for Collection Search (Table 3)
productType: '$.null'
platformSerialIdentifier:
- '{{"search":{{"satelliteName":"{platformSerialIdentifier}" }} }}'
- '$.satelliteName'
instrument:
- '{{"search":{{"sensor":"{instrument}" }} }}'
- '$.sensor'
processingLevel:
- '{{"search":{{"processingLevel":"{processingLevel}" }} }}'
- '$.processingLevel'
# INSPIRE obligated OpenSearch Parameters for Collection Search (Table 4)
title:
- '{{"search":{{"sceneID":"{title}" }} }}'
- '$.sceneID'
# OpenSearch Parameters for Acquistion Parameters Search (Table 6)
startTimeFromAscendingNode:
- '{{"search":{{"date":{{"from":"{startTimeFromAscendingNode}"}} }} }}'
- '$.date'
completionTimeFromAscendingNode:
- '{{"search":{{"date":{{"to":"{completionTimeFromAscendingNode}"}} }} }}'
- '$.date'
illuminationAzimuthAngle:
- '{{"search":{{"sunAzimuth":"{illuminationAzimuthAngle}" }} }}'
- '$.sunAzimuth'
illuminationElevationAngle:
- '{{"search":{{"sunElevation":"{illuminationElevationAngle}" }} }}'
- '$.sunElevation'
# Custom parameters (not defined in the base document referenced above)
awsPath:
- '{{"search":{{"downloadUrl":"{awsPath}" }} }}'
- '$.downloadUrl'
downloadLink: 's3://cbers-pds/{awsPath}'
metaDownloadLink: 's3://cbers-meta-pds/{awsPath}'
previewBaseName: '{$.sceneID#replace_str("_L2","")}'
thumbnail: 'https://s3.amazonaws.com/cbers-meta-pds/{awsPath}/{previewBaseName}_small.jpeg'
quicklook: 'https://s3.amazonaws.com/cbers-meta-pds/{awsPath}/{previewBaseName}.jpg'
id:
- '{{"search":{{"sceneID":"{id}" }} }}'
- '{title}'
CBERS4_PAN10M_L4:
instrument: PAN10M
collection: cbers4
processingLevel: 4
metadata_mapping_from_product: CBERS4_PAN10M_L2
metadata_mapping:
# Custom parameters (not defined in the base document referenced above)
previewBaseName: '{$.sceneID#replace_str("_L4","")}'
thumbnail: 'https://s3.amazonaws.com/cbers-meta-pds/{awsPath}/{previewBaseName}_small.jpeg'
quicklook: 'https://s3.amazonaws.com/cbers-meta-pds/{awsPath}/{previewBaseName}.jpg'
CBERS4_PAN5M_L2:
instrument: PAN5M
collection: cbers4
processingLevel: 2
metadata_mapping_from_product: CBERS4_PAN10M_L2
CBERS4_PAN5M_L4:
instrument: PAN5M
collection: cbers4
processingLevel: 4
metadata_mapping_from_product: CBERS4_PAN10M_L2
metadata_mapping:
# Custom parameters (not defined in the base document referenced above)
previewBaseName: '{$.sceneID#replace_str("_L4","")}'
thumbnail: 'https://s3.amazonaws.com/cbers-meta-pds/{awsPath}/{previewBaseName}_small.jpeg'
quicklook: 'https://s3.amazonaws.com/cbers-meta-pds/{awsPath}/{previewBaseName}.jpg'
CBERS4_MUX_L2:
instrument: MUX
collection: cbers4
processingLevel: 2
metadata_mapping_from_product: CBERS4_PAN10M_L2
CBERS4_MUX_L4:
instrument: MUX
collection: cbers4
processingLevel: 4
metadata_mapping_from_product: CBERS4_PAN10M_L2
metadata_mapping:
# Custom parameters (not defined in the base document referenced above)
previewBaseName: '{$.sceneID#replace_str("_L4","")}'
thumbnail: 'https://s3.amazonaws.com/cbers-meta-pds/{awsPath}/{previewBaseName}_small.jpeg'
quicklook: 'https://s3.amazonaws.com/cbers-meta-pds/{awsPath}/{previewBaseName}.jpg'
CBERS4_AWFI_L2:
instrument: AWFI
collection: cbers4
processingLevel: 2
metadata_mapping_from_product: CBERS4_PAN10M_L2
CBERS4_AWFI_L4:
instrument: AWFI
collection: cbers4
processingLevel: 4
metadata_mapping_from_product: CBERS4_PAN10M_L2
metadata_mapping:
# Custom parameters (not defined in the base document referenced above)
previewBaseName: '{$.sceneID#replace_str("_L4","")}'
thumbnail: 'https://s3.amazonaws.com/cbers-meta-pds/{awsPath}/{previewBaseName}_small.jpeg'
quicklook: 'https://s3.amazonaws.com/cbers-meta-pds/{awsPath}/{previewBaseName}.jpg'
L8_OLI_TIRS_C1L1:
collection: landsat8
onAmazon: true
metadata_mapping:
# OpenSearch Parameters for Collection Search (Table 3)
productType: '$.null'
platform:
- '{{"search":{{"satelliteName":"{platform}" }} }}'
- '$.satelliteName'
instrument:
- '{{"search":{{"sensor":"{instrument}" }} }}'
- '$.sensor'
# INSPIRE obligated OpenSearch Parameters for Collection Search (Table 4)
title:
- '{{"search":{{"productID":"{title}" }} }}'
- '$.productID'
# OpenSearch Parameters for Product Search (Table 5)
cloudCover:
- '{{"search":{{"cloudCoverage":"{cloudCover}" }} }}'
- '$.cloudCoverage'
# OpenSearch Parameters for Acquistion Parameters Search (Table 6)
availabilityTime:
- '{{"search":{{"IndexingTime":{{"from":"{availabilityTime}"}} }} }}'
- '$.IndexingTime'
acquisitionStation:
- '{{"search":{{"receivingStation":{{"from":"{acquisitionStation}"}} }} }}'
- '$.receivingStation'
startTimeFromAscendingNode:
- '{{"search":{{"date":{{"from":"{startTimeFromAscendingNode}"}} }} }}'
- '$.sceneStartTime'
completionTimeFromAscendingNode:
- '{{"search":{{"date":{{"to":"{completionTimeFromAscendingNode}"}} }} }}'
- '$.sceneStopTime'
illuminationAzimuthAngle:
- '{{"search":{{"sunAzimuth":"{illuminationAzimuthAngle}" }} }}'
- '$.sunAzimuth'
illuminationElevationAngle:
- '{{"search":{{"sunElevation":"{illuminationElevationAngle}" }} }}'
- '$.sunElevation'
# Custom parameters (not defined in the base document referenced above)
onAmazon:
- '{{"search":{{"onAmazon":"{onAmazon}" }} }}'
- '$.onAmazon'
path:
- '{{"search":{{"path":"{path}" }} }}'
- '$.path'
row:
- '{{"search":{{"row":"{row}" }} }}'
- '$.row'
downloadLink: 's3://landsat-pds/c1/L8/{path:03.0f}/{row:03.0f}/{title}/'
thumbnail: '$.thumbnail'
quicklook: 'https://landsat-pds.s3.amazonaws.com/c1/L8/{path:03.0f}/{row:03.0f}/{title}/{title}_thumb_large.jpg'
id:
- '{{"search":{{"productID":"{id}" }} }}'
- '{title}'
MODIS_MCD43A4:
collection: modis
metadata_mapping:
# OpenSearch Parameters for Collection Search (Table 3)
productType: '$.null'
instrument:
- '{{"search":{{"satelliteName":"{instrument}" }} }}'
- '$.satelliteName'
# INSPIRE obligated OpenSearch Parameters for Collection Search (Table 4)
title:
- '{{"search":{{"sceneID":"{title}" }} }}'
- '$.sceneID'
# OpenSearch Parameters for Product Search (Table 5)
cloudCover:
- '{{"search":{{"cloudCoverage":"{cloudCover}" }} }}'
- '$.cloudCoverage'
# OpenSearch Parameters for Acquistion Parameters Search (Table 6)
startTimeFromAscendingNode:
- '{{"search":{{"date":{{"from":"{startTimeFromAscendingNode}"}} }} }}'
- '$.BeginningDateTime'
completionTimeFromAscendingNode:
- '{{"search":{{"date":{{"to":"{completionTimeFromAscendingNode}"}} }} }}'
- '$.EndingDateTime'
# Custom parameters (not defined in the base document referenced above)
verticalTileNumber:
- '{{"search":{{"verticalTileNumber":"{verticalTileNumber}" }} }}'
- '$.verticalTileNumber'
horizontalTileNumber:
- '{{"search":{{"horizontalTileNumber":"{horizontalTileNumber}" }} }}'
- '$.horizontalTileNumber'
doyDate: '{$.sceneID#slice_str(9,16,1)}'
downloadLink: 's3://modis-pds/MCD43A4.006/{horizontalTileNumber:02.0f}/{verticalTileNumber:02.0f}/{doyDate}/'
thumbnail: '$.thumbnail'
quicklook: '$.thumbnail'
id:
- '{{"search":{{"sceneID":"{id}" }} }}'
- '{title}'
NAIP:
collection: naip
metadata_mapping:
# OpenSearch Parameters for Collection Search (Table 3)
productType: '$.null'
platformSerialIdentifier:
- '{{"search":{{"satelliteName":"{platformSerialIdentifier}" }} }}'
- '$.satelliteName'
# INSPIRE obligated OpenSearch Parameters for Collection Search (Table 4)
title:
- '{{"search":{{"sceneID":"{title}" }} }}'
- '$.sceneID'
# OpenSearch Parameters for Acquistion Parameters Search (Table 6)
startTimeFromAscendingNode:
- '{{"search":{{"date":{{"from":"{startTimeFromAscendingNode}"}} }} }}'
- '$.date'
completionTimeFromAscendingNode:
- '{{"search":{{"date":{{"to":"{completionTimeFromAscendingNode}"}} }} }}'
- '$.date'
# Custom parameters (not defined in the base document referenced above)
awsPath:
- '{{"search":{{"awsPath":"{awsPath}" }} }}'
- '$.awsPath'
downloadLink: 's3://naip-analytic/{awsPath}'
id:
- '{{"search":{{"sceneID":"{id}" }} }}'
- '{title}'
S1_SAR_GRD:
productType: GRD
collection: sentinel1
metadata_mapping:
cloudCover: '$.null'
# OpenSearch Parameters for Collection Search (Table 3)
platformSerialIdentifier:
- '{{"search":{{"missionId":"{platformSerialIdentifier}" }} }}'
- '$.missionId'
# INSPIRE obligated OpenSearch Parameters for Collection Search (Table 4)
title:
- '{{"search":{{"sceneID":"{title}" }} }}'
- '$.sceneID'
# OpenSearch Parameters for Product Search (Table 5)
orbitNumber:
- '{{"search":{{"absoluteOrbitNumber":"{orbitNumber}" }} }}'
- '$.absoluteOrbitNumber'
orbitDirection:
- '{{"search":{{"passDirection":"{orbitDirection}" }} }}'
- '$.passDirection'
sensorMode:
- '{{"search":{{"mode":"{sensorMode}" }} }}'
- '$.mode'
# OpenSearch Parameters for Acquistion Parameters Search (Table 6)
startTimeFromAscendingNode:
- '{{"search":{{"date":{{"from":"{startTimeFromAscendingNode}"}} }} }}'
- '$.date'
completionTimeFromAscendingNode:
- '{{"search":{{"date":{{"to":"{completionTimeFromAscendingNode}"}} }} }}'
- '$.date'
polarizationMode:
- '{{"search":{{"polarization":"{polarizationMode}" }} }}'
- '$.polarization'
# Custom parameters (not defined in the base document referenced above)
awsPath:
- '{{"search":{{"awsPath":"{awsPath}" }} }}'
- '$.awsPath'
downloadLink: 's3://sentinel-s1-l1c/{awsPath}'
thumbnail: 'https://render.eosda.com/S1/thumb/{title}.png'
quicklook: 'https://render.eosda.com/S1/thumb/{title}.png'
id:
- '{{"search":{{"sceneID":"{id}" }} }}'
- '{title}'
S2_MSI_L1C:
collection: sentinel2
metadata_mapping:
# OpenSearch Parameters for Collection Search (Table 3)
productType: '$.null'
platformSerialIdentifier:
- '{{"search":{{"satelliteName":"{platformSerialIdentifier}" }} }}'
- '$.satelliteName'
# INSPIRE obligated OpenSearch Parameters for Collection Search (Table 4)
title:
- '{{"search":{{"productName":"{title}" }} }}'
- '$.productName'
# OpenSearch Parameters for Product Search (Table 5)
cloudCover:
- '{{"search":{{"cloudCoverage":"{cloudCover}" }} }}'
- '$.cloudCoverage'
# OpenSearch Parameters for Acquistion Parameters Search (Table 6)
startTimeFromAscendingNode:
- '{{"search":{{"date":{{"from":"{startTimeFromAscendingNode}"}} }} }}'
- '$.timestamp'
completionTimeFromAscendingNode:
- '{{"search":{{"date":{{"to":"{completionTimeFromAscendingNode}"}} }} }}'
- '$.timestamp'
illuminationAzimuthAngle:
- '{{"search":{{"azimuthAngle":"{illuminationAzimuthAngle}" }} }}'
- '$.azimuthAngle'
illuminationZenithAngle:
- '{{"search":{{"zenithAngle":"{illuminationZenithAngle}" }} }}'
- '$.zenithAngle'
# Custom parameters (not defined in the base document referenced above)
originalTileMeta: '{$.originalTileMeta#replace_str("sentinel-s2-l1c.s3.amazonaws.com","roda.sentinel-hub.com/sentinel-s2-l1c")}'
thumbnail: '{$.thumbnail#replace_str("sentinel-s2-l1c.s3.amazonaws.com","roda.sentinel-hub.com/sentinel-s2-l1c")}'
awsPath:
- '{{"search":{{"awsPath":"{awsPath}" }} }}'
- '$.awsPath'
downloadLink: 's3://sentinel-s2-l1c/{awsPath}'
productMetaLink: '{$.productMetaLink#replace_str("sentinel-s2-l1c.s3.amazonaws.com","roda.sentinel-hub.com/sentinel-s2-l1c")}'
productPath:
- '{{"search":{{"productPath":"{productPath}" }} }}'
- '$.productPath'
id:
- '{{"search":{{"productName":"{id}" }} }}'
- '{title}'
processedL2A: '$.null'
awsPathL2A: '$.null'
S2_MSI_L2A:
collection: sentinel2
processedL2A: true
# specific QueryStringSearch usage for these parameters (replaces current query)
specific_qssearch:
parameters:
- title
- id
results_entry: ''
collection:
- tileInfo
- productInfo
merge_responses: true
metadata_mapping:
title:
- 'title'
- '$.name'
id:
- 'title'
- '{title}'
awsPathL2A: '$.tiles[0].path'
downloadLink: 's3://sentinel-s2-l2a/{awsPathL2A}'
productPath: '$.path'
startTimeFromAscendingNode: '$.timestamp'
completionTimeFromAscendingNode: '$.timestamp'
geometry: '$.tileDataGeometry'
productInfo: 'https://roda.sentinel-hub.com/sentinel-s2-l2a/{awsPathL2A}/productInfo.json'
originalSceneID: '$.tiles[0].datastrip.id'
metadata_mapping:
# OpenSearch Parameters for Collection Search (Table 3)
productType: '$.null'
platformSerialIdentifier:
- '{{"search":{{"satelliteName":"{platformSerialIdentifier}" }} }}'
- '$.satelliteName'
# INSPIRE obligated OpenSearch Parameters for Collection Search (Table 4)
title: '{$.productName#fake_l2a_title_from_l1c}'
# OpenSearch Parameters for Product Search (Table 5)
cloudCover:
- '{{"search":{{"cloudCoverage":"{cloudCover}" }} }}'
- '$.cloudCoverage'
# OpenSearch Parameters for Acquistion Parameters Search (Table 6)
startTimeFromAscendingNode:
- '{{"search":{{"date":{{"from":"{startTimeFromAscendingNode}"}} }} }}'
- '$.timestamp'
completionTimeFromAscendingNode:
- '{{"search":{{"date":{{"to":"{completionTimeFromAscendingNode}"}} }} }}'
- '$.timestamp'
illuminationAzimuthAngle:
- '{{"search":{{"azimuthAngle":"{illuminationAzimuthAngle}" }} }}'
- '$.azimuthAngle'
illuminationZenithAngle:
- '{{"search":{{"zenithAngle":"{illuminationZenithAngle}" }} }}'
- '$.zenithAngle'
# Custom parameters (not defined in the base document referenced above)
originalTileMeta: '{$.originalTileMeta#replace_str("sentinel-s2-l1c.s3.amazonaws.com","roda.sentinel-hub.com/sentinel-s2-l2a")}'
thumbnail: '{$.thumbnail#replace_str("sentinel-s2-l1c.s3.amazonaws.com","roda.sentinel-hub.com/sentinel-s2-l1c")}'
quicklook: '{thumbnail}'
downloadLink: 's3://sentinel-s2-l2a/{awsPathL2A}'
awsPath: '$.null'
productMetaLink: '$.null'
productPath: '$.null'
productInfo: 'https://roda.sentinel-hub.com/sentinel-s2-l2a/{awsPathL2A}/productInfo.json'
id:
- '{id#s2msil2a_title_to_aws_productinfo}'
- '{title}'
processedL2A:
- '{{"search":{{"processedL2A":"{processedL2A}" }} }}'
- '$.processedL2A'
awsPathL2A:
- '{{"search":{{"awsPathL2A":"{awsPathL2A}" }} }}'
- '$.awsPathL2A'
download: !plugin
type: AwsDownload
requester_pays: True
products:
CBERS4_MUX_L2:
default_bucket: 'cbers-pds'
complementary_url_key:
- metaDownloadLink
CBERS4_AWFI_L2:
complementary_url_key:
- metaDownloadLink
CBERS4_PAN5M_L2:
complementary_url_key:
- metaDownloadLink
CBERS4_PAN10M_L2:
complementary_url_key:
- metaDownloadLink
S1_SAR_GRD:
default_bucket: 'sentinel-s1-l1c'
build_safe: true
S2_MSI_L1C:
default_bucket: 'sentinel-s2-l1c'
build_safe: true
complementary_url_key:
- productPath
S2_MSI_L2A:
default_bucket: 'sentinel-s2-l2a'
build_safe: true
fetch_metadata:
fetch_url: '{productInfo}'
fetch_format: json
update_metadata:
title: '$.name'
id: '{title}'
productPath: '$.path'
complementary_url_key:
- productPath
auth: !plugin
type: AwsAuth
---
!provider
name: theia
priority: 0
description: French National Space Agency (CNES) catalog for Sentinel 2 products, Pleiades and Landsat products
roles:
- host
url: https://theia.cnes.fr/atdistrib/rocket/
search: !plugin
type: QueryStringSearch
api_endpoint: 'https://theia.cnes.fr/atdistrib/resto2/api/collections/{collection}/search.json'
need_auth: false
pagination:
next_page_url_tpl: '{url}?{search}&maxRecords={items_per_page}&page={page}'
total_items_nb_key_path: '$.properties.totalResults'
# 2021/03/19: Returns a 400 error code if greater than 500.
max_items_per_page: 500
discover_metadata:
auto_discovery: true
metadata_pattern: '^(?!collection)[a-zA-Z0-9_]+$'
search_param: '{metadata}={{{metadata}}}'
metadata_path: '$.properties.*'
metadata_mapping:
# Opensearch resource identifier within the search engine context (in our case
# within the context of the data provider)
uid: '$.id'
# OpenSearch Parameters for Collection Search (Table 3)
productType:
- productType
- '$.properties.productType'
platform:
- platform
- '$.properties.platform'
platformSerialIdentifier:
- 'platform={platformSerialIdentifier}'
- '$.properties.platform'
instrument:
- instrument
- '$.properties.instrument'
processingLevel:
- processingLevel
- '$.properties.processingLevel'
# INSPIRE obligated OpenSearch Parameters for Collection Search (Table 4)
title: '$.properties.title'
keyword: '$.properties.keywords'
resolution: '$.properties.resolution'
organisationName: '$.properties.organisationName'
publicationDate: '$.properties.published'
accessConstraint: '$.properties.license'
# OpenSearch Parameters for Product Search (Table 5)
parentIdentifier: '$.properties.parentIdentifier'
orbitNumber: '$.properties.orbitNumber'
cloudCover:
- 'cloudCover=[0,{cloudCover}]'
- '$.properties.cloudCover'
snowCover: '$.properties.snowCover'
productVersion: '$.properties.version'
creationDate: '$.properties.productionDate'
modificationDate: '$.properties.modificationDate'
processingDate: '$.properties.processingDate'
sensorMode: '$.properties.sensorMode'
archivingCenter: '$.properties.archivingCenter'
processingMode: '$.properties.processingMode'
# OpenSearch Parameters for Acquistion Parameters Search (Table 6)
availabilityTime: '$.properties.availabilityTime'
acquisitionStation: '$.properties.acquisitionStation'
acquisitionSubType: '$.properties.acquisitionSubType'
startTimeFromAscendingNode:
- 'startDate={startTimeFromAscendingNode#to_iso_date}'
- '$.properties.startDate'
completionTimeFromAscendingNode:
- 'completionDate={completionTimeFromAscendingNode#to_iso_date}'
- '$.properties.completionDate'
illuminationAzimuthAngle: '$.properties.illuminationAzimuthAngle'
illuminationZenithAngle: '$.properties.illuminationZenithAngle'
illuminationElevationAngle: '$.properties.illuminationElevationAngle'
polarizationMode: '$.properties.polarizationMode'
polarizationChannels: '$.properties.polarisationChannels'
antennaLookDirection: '$.properties.antennaLookDirection'
minimumIncidenceAngle: '$.properties.minimumIncidenceAngle'
maximumIncidenceAngle: '$.properties.maximumIncidenceAngle'
dopplerFrequency: '$.properties.dopplerFrequency'
incidenceAngleVariation: '$.properties.incidenceAngleVariation'
# Custom parameters (not defined in the base document referenced above)
id: '$.properties.productIdentifier'
tileIdentifier:
- 'location=T{tileIdentifier#replace_str(r"^T(.*)$",r"\1")}'
- '{$.properties.location#replace_str(r"^T(.*)$",r"\1")}'
# The geographic extent of the product
geometry:
- 'geometry={geometry#to_rounded_wkt}'
- '$.geometry'
# The url of the quicklook
quicklook: '$.properties.quicklook'
# The url to download the product "as is" (literal or as a template to be completed either after the search result
# is obtained from the provider or during the eodag download phase)
downloadLink: '$.properties.services.download.url'
# Additional metadata provided by the providers but that don't appear in the reference spec
# Or has a different signification for the provider
thumbnail: '$.properties.thumbnail'
# storageStatus set to ONLINE for consistency between providers
storageStatus: '{$.null#replace_str("Not Available","ONLINE")}'
products:
S2_MSI_L2A_MAJA:
processingLevel: LEVEL2A
collection: SENTINEL2
S2_MSI_L2B_MAJA_SNOW:
processingLevel: L2B-SNOW
collection: Snow
S2_MSI_L2B_MAJA_WATER:
processingLevel: L2B-WATER
collection: WaterQual
S2_MSI_L3A_WASP:
processingLevel: LEVEL3A
collection: SENTINEL2
SPOT_SWH:
collection: SPOTWORLDHERITAGE
processingLevel: LEVEL1C
SPOT_SWH_OLD:
collection: SWH1
processingLevel: LEVEL1C
SPOT5_SPIRIT:
collection: Spirit
processingLevel: 1A
VENUS_L1C:
collection: VENUS
processingLevel: LEVEL1C
VENUS_L2A_MAJA:
collection: VENUS
processingLevel: LEVEL2A
VENUS_L3A_MAJA:
collection: VENUS
processingLevel: LEVEL3A
L8_REFLECTANCE:
collection: LANDSAT
productType: REFLECTANCE
L57_REFLECTANCE:
collection: Landsat57
productType: REFLECTANCE
PLD_PAN:
collection: Pleiades
productType: Panchromatique+(Pan)
PLD_XS:
collection: Pleiades
productType: Multispectral+(XS)
PLD_BUNDLE:
collection: Pleiades
productType: Bundle+(Pan,+XS)
PLD_PANSHARPENED:
collection: Pleiades
productType: Pansharpened+(Pan%2BXS)
OSO:
collection: OSO
GENERIC_PRODUCT_TYPE:
productType: '{productType}'
collection: '{collection}'
processingLevel: '{processingLevel}'
download: !plugin
type: HTTPDownload
base_uri: 'https://theia.cnes.fr/atdistrib/resto2'
extract: true
order_enabled: true
auth_error_code: 403
dl_url_params:
issuerId: theia
auth: !plugin
type: TokenAuth
auth_uri: 'https://theia.cnes.fr/atdistrib/services/authenticate/'
---
!provider
name: peps
priority: 1
description: |
The PEPS platform, the French "mirror site", redistributes the products of Sentinel satellites,
S1A, S1B, S2A and S2B, S3A and S3B from COPERNICUS, the European system for the Earth monitoring.
roles:
- host
url: https://peps.cnes.fr
search: !plugin
type: QueryStringSearch
api_endpoint: 'https://peps.cnes.fr/resto/api/collections/{collection}/search.json'
need_auth: false
pagination:
next_page_url_tpl: '{url}?{search}&maxRecords={items_per_page}&page={page}'
total_items_nb_key_path: '$.properties.totalResults'
# 2021/03/19: 500 is the max, no error if greater
max_items_per_page: 500
discover_metadata:
auto_discovery: true
metadata_pattern: '^(?!collection)[a-zA-Z0-9_]+$'
search_param: '{metadata}={{{metadata}}}'
metadata_path: '$.properties.*'
metadata_mapping:
# Opensearch resource identifier within the search engine context (in our case
# within the context of the data provider)
uid: '$.id'
# OpenSearch Parameters for Collection Search (Table 3)
productType:
- productType
- '$.properties.productType'
platform: '$.properties.collection'
platformSerialIdentifier:
- platform
- '$.properties.platform'
instrument:
- instrument
- '$.properties.instrument'
processingLevel:
- processingLevel
- '$.properties.processingLevel'
# INSPIRE obligated OpenSearch Parameters for Collection Search (Table 4)
title: '$.properties.title'
topicCategory: '$.properties.topicCategory'
keyword: '$.properties.keywords'
abstract: '$.properties.description'
resolution:
- 'resolution'
- '$.properties.resolution'
organisationName:
- 'organisationName'
- '$.properties.organisationName'
publicationDate: '$.properties.published'
accessConstraint: '$.properties.license'
# OpenSearch Parameters for Product Search (Table 5)
parentIdentifier:
- 'parentIdentifier'
- '$.properties.parentIdentifier'
acquisitionType: '$.properties.acquisitionType'
orbitNumber:
- 'orbitNumber'
- '$.properties.orbitNumber'
orbitDirection:
- 'orbitDirection'
- '$.properties.orbitDirection'
swathIdentifier:
- 'swath'
- '$.properties.swath'
cloudCover:
- 'cloudCover=[0,{cloudCover}]'
- '$.properties.cloudCover'
snowCover:
- 'snowCover=[0,{snowCover}]'
- '$.properties.snowCover'
productVersion: '$.properties.version'
productQualityStatus: '$.properties.onlineQualityCheck'
processorName: '$.properties.processingName'
processingCenter: '$.properties.processingCenter'
creationDate: '$.properties.dhusIngestDate'
modificationDate: '$.properties.updated'
sensorMode:
- 'sensorMode'
- '$.properties.sensorMode'
# OpenSearch Parameters for Acquistion Parameters Search (Table 6)
startTimeFromAscendingNode:
- startDate
- '$.properties.startDate'
completionTimeFromAscendingNode:
- completionDate
- '$.properties.completionDate'
polarizationMode:
- 'polarisation'
- '$.properties.polarisation'
# Custom parameters (not defined in the base document referenced above)
id:
- 'productIdentifier={id#remove_extension}'
- '$.properties.productIdentifier'
tileIdentifier:
- 'tileid'
- '$.properties.mgrs'
# The geographic extent of the product
geometry:
- 'geometry={geometry#to_rounded_wkt}'
- '$.geometry'
# The url of the quicklook
quicklook: '$.properties.quicklook'
# The url to download the product "as is" (literal or as a template to be completed either after the search result
# is obtained from the provider or during the eodag download phase)
downloadLink: '$.properties.services.download.url'
# storageStatus: must be one of ONLINE, STAGING, OFFLINE
storageStatus: '{$.properties.storage.mode#get_group_name((?P<ONLINE>disk|tier2)|(?P<STAGING>staging)|(?P<OFFLINE>unknown|tape|tier3))}'
# Additional metadata provided by the providers but that don't appear in the reference spec
thumbnail: '$.properties.thumbnail'
products:
S1_SAR_OCN:
productType: OCN
collection: S1
metadata_mapping:
cloudCover: '$.null'
S1_SAR_GRD:
productType: GRD
collection: S1
metadata_mapping:
cloudCover: '$.null'
S1_SAR_SLC:
productType: SLC
collection: S1
metadata_mapping:
cloudCover: '$.null'
S2_MSI_L1C:
collection: S2ST
productType: S2MSI1C
S2_MSI_L2A:
collection: S2ST
productType: S2MSI2A
GENERIC_PRODUCT_TYPE:
productType: '{productType}'
collection: '{collection}'
download: !plugin
type: HTTPDownload
base_uri: 'https://peps.cnes.fr/resto'
extract: true
archive_depth: 2
order_enabled: true
auth_error_code: 401
dl_url_params:
issuerId: peps
auth: !plugin
type: GenericAuth
auth_uri: 'https://peps.cnes.fr/resto/api/users/connect'
---
!provider
name: creodias
priority: 0
description: CloudFerro DIAS
roles:
- host
url: https://creodias.eu/
search: !plugin
type: ODataV4Search
api_endpoint: 'https://datahub.creodias.eu/odata/v1/Products'
need_auth: false
dont_quote:
- '['
- ']'
- '$'
- '='
- '&'
- ':'
pagination:
next_page_url_tpl: '{url}?{search}&$top={items_per_page}&$skip={skip}&$expand=Attributes&$count=True'
total_items_nb_key_path: '$."@odata.count"'
max_items_per_page: 1_000
results_entry: 'value'
free_text_search_operations:
$filter:
union: ' or '
wrapper: '{}'
operations:
and:
- "Collection/Name eq '{collection}'"
- "OData.CSC.Intersects(area=geography'{geometry#to_ewkt}')"
- "Attributes/OData.CSC.StringAttribute/any(att:att/Name eq 'productType' and att/OData.CSC.StringAttribute/Value eq '{productType}')"
- "Attributes/OData.CSC.StringAttribute/any(att:att/Name eq 'platformShortName' and att/OData.CSC.StringAttribute/Value eq '{platform}')"
- "Attributes/OData.CSC.StringAttribute/any(att:att/Name eq 'platformSerialIdentifier' and att/OData.CSC.StringAttribute/Value eq '{platformSerialIdentifier}')"
- "Attributes/OData.CSC.IntegerAttribute/any(att:att/Name eq 'spatialResolution' and att/OData.CSC.StringAttribute/Value eq '{resolution}')"
- "Attributes/OData.CSC.StringAttribute/any(att:att/Name eq 'authority' and att/OData.CSC.StringAttribute/Value eq '{organisationName}')"
- "Attributes/OData.CSC.IntegerAttribute/any(att:att/Name eq 'orbitNumber' and att/OData.CSC.StringAttribute/Value eq '{orbitNumber}')"
- "Attributes/OData.CSC.StringAttribute/any(att:att/Name eq 'orbitDirection' and att/OData.CSC.StringAttribute/Value eq '{orbitDirection}')"
- "Attributes/OData.CSC.DoubleAttribute/any(att:att/Name eq 'cloudCover' and att/OData.CSC.DoubleAttribute/Value le {cloudCover})"
- "Attributes/OData.CSC.StringAttribute/any(att:att/Name eq 'operationalMode' and att/OData.CSC.StringAttribute/Value eq '{sensorMode}')"
- "Attributes/OData.CSC.StringAttribute/any(att:att/Name eq 'polarisationChannels' and att/OData.CSC.StringAttribute/Value eq '{polarizationChannels}')"
- "Attributes/OData.CSC.StringAttribute/any(att:att/Name eq 'tileId' and att/OData.CSC.StringAttribute/Value eq '{tileIdentifier}')"
- 'ContentDate/Start gt {startTimeFromAscendingNode#to_iso_utc_datetime}'
- 'ContentDate/End lt {completionTimeFromAscendingNode#to_iso_utc_datetime}'
- contains(Name,'{id}')
discover_metadata:
auto_discovery: true
metadata_pattern: '^(?!collection)[a-zA-Z0-9]+$'
search_param:
free_text_search_operations:
$filter:
operations:
and:
- "Attributes/OData.CSC.StringAttribute/any(att:att/Name eq '{metadata}' and att/OData.CSC.StringAttribute/Value eq '{{{metadata}}}')"
metadata_path: '$.Attributes.*'
discover_product_types:
fetch_url: 'https://datahub.creodias.eu/stac/collections'
result_type: json
results_entry: '$.collections[*]'
generic_product_type_id: '$.id'
generic_product_type_parsable_properties:
collection: '$.id'
generic_product_type_parsable_metadata:
abstract: '$.description'
instrument: '{$.summaries.instruments#csv_list}'
platform: '{$.summaries.constellation#csv_list}'
platformSerialIdentifier: '{$.summaries.platform#csv_list}'
processingLevel: '$.summaries."processing:level"'
keywords: '{$.keywords#csv_list}'
license: '$.license'
title: '$.title'
missionStartDate: '$.extent.temporal.interval[0][0]'
per_product_metadata_query: false
metadata_pre_mapping:
metadata_path: '$.Attributes'
metadata_path_id: 'Name'
metadata_path_value: 'Value'
metadata_mapping:
# Opensearch resource identifier within the search engine context (in our case
# within the context of the data provider)
# Queryable parameters are set with null as 1st configuration list value to mark them as queryable,
# but `free_text_search_operations.$filter.operations.and` entries are then used instead.
uid: '$.Id'
# OpenSearch Parameters for Collection Search (Table 3)
productType:
- null
- '$.Attributes.productType'
platform:
- null
- '$.Attributes.platformShortName'
platformSerialIdentifier:
- null
- '$.Attributes.platformSerialIdentifier'
instrument:
- null
- '$.Attributes.instrumentShortName'
processingLevel:
- null
- '$.Attributes.processingLevel'
# INSPIRE obligated OpenSearch Parameters for Collection Search (Table 4)
title: '$.Name'
resolution:
- null
- '$.Attributes.spatialResolution'
organisationName:
- null
- '$.Attributes.authority'
publicationDate: '$.PublicationDate'
# OpenSearch Parameters for Product Search (Table 5)
orbitNumber:
- null
- '$.Attributes.orbitNumber'
orbitDirection:
- null
- '$.Attributes.orbitDirection'
cloudCover:
- null
- '$.Attributes.cloudCover'
modificationDate: '$.ModificationDate'
sensorMode:
- null
- '$.Attributes.operationalMode'
# OpenSearch Parameters for Acquistion Parameters Search (Table 6)
startTimeFromAscendingNode:
- null
- '$.ContentDate.Start'
completionTimeFromAscendingNode:
- null
- '$.ContentDate.End'
polarizationChannels:
- null
- '$.Attributes.polarisationChannels'
# Custom parameters (not defined in the base document referenced above)
id:
- null
- '{$.Name#remove_extension}'
tileIdentifier:
- null
- '$.Attributes.tileId'
# The geographic extent of the product
geometry:
- null
- '{$.Footprint.`sub(/.*(SRID=[0-9]+;.*)/, \\1)`#from_ewkt}'
# The url to download the product "as is" (literal or as a template to be completed either after the search result
# is obtained from the provider or during the eodag download phase)
downloadLink: 'https://datahub.creodias.eu/odata/v1/Products({uid})/$value'
# storageStatus: must be one of ONLINE, STAGING, OFFLINE
storageStatus: '{$.Online#get_group_name((?P<ONLINE>True)|(?P<OFFLINE>False))}'
collection:
- null
- $.null
# Additional metadata provided by the providers but that don't appear in the reference spec
productIdentifier: '$.S3Path'
download: !plugin
type: HTTPDownload
base_uri: 'https://zipper.creodias.eu/download/'
extract: true
order_enabled: false
archive_depth: 2
auth: !plugin
type: KeycloakOIDCPasswordAuth
auth_base_uri: 'https://identity.cloudferro.com/auth'
realm: 'dias'
client_id: 'CLOUDFERRO_PUBLIC'
client_secret: 'dc0aca03-2dc6-4798-a5de-fc5aeb6c8ee1'
token_provision: qs
token_qs_key: 'token'
auth_error_code: 401
products:
# S2
S2_MSI_L1C:
collection: SENTINEL-2
productType: S2MSI1C
S2_MSI_L2A:
collection: SENTINEL-2
productType: S2MSI2A
# S1
S1_SAR_RAW:
productType: RAW
collection: SENTINEL-1
metadata_mapping:
cloudCover: '$.null'
S1_SAR_GRD:
productType: GRD
collection: SENTINEL-1
metadata_mapping:
cloudCover: '$.null'
S1_SAR_SLC:
productType: SLC
collection: SENTINEL-1
metadata_mapping:
cloudCover: '$.null'
S1_SAR_OCN:
productType: OCN
collection: SENTINEL-1
metadata_mapping:
cloudCover: '$.null'
# S3 SRAL
S3_SRA:
productType: SR_1_SRA___
collection: SENTINEL-3
metadata_mapping:
cloudCover: '$.null'
S3_SRA_A:
productType: SR_1_SRA_A_
collection: SENTINEL-3
metadata_mapping:
cloudCover: '$.null'
S3_SRA_BS:
productType: SR_1_SRA_BS
collection: SENTINEL-3
metadata_mapping:
cloudCover: '$.null'
S3_LAN:
productType: SR_2_LAN___
collection: SENTINEL-3
metadata_mapping:
cloudCover: '$.null'
S3_WAT:
productType: SR_2_WAT___
collection: SENTINEL-3
metadata_mapping:
cloudCover: '$.null'
# S3 OLCI
S3_EFR:
productType: OL_1_EFR___
collection: SENTINEL-3
S3_ERR:
productType: OL_1_ERR___
collection: SENTINEL-3
S3_OLCI_L2LRR:
productType: OL_2_LRR___
collection: SENTINEL-3
S3_OLCI_L2LFR:
productType: OL_2_LFR___
collection: SENTINEL-3
S3_OLCI_L2WRR:
productType: OL_2_WRR___
collection: SENTINEL-3
S3_OLCI_L2WFR:
productType: OL_2_WFR___
collection: SENTINEL-3
# S3 SLSTR
S3_SLSTR_L1RBT:
productType: SL_1_RBT___
collection: SENTINEL-3
S3_SLSTR_L2LST:
productType: SL_2_LST___
collection: SENTINEL-3
S3_SLSTR_L2WST:
productType: SL_2_WST___
collection: SENTINEL-3
S3_SLSTR_L2AOD:
productType: SL_2_AOD___
collection: SENTINEL-3
S3_SLSTR_L2FRP:
productType: SL_2_FRP___
collection: SENTINEL-3
# S3 SY
S3_SY_AOD:
productType: SY_2_AOD___
collection: SENTINEL-3
S3_SY_SYN:
productType: SY_2_SYN___
collection: SENTINEL-3
S3_SY_V10:
productType: SY_2_V10___
collection: SENTINEL-3
S3_SY_VG1:
productType: SY_2_VG1___
collection: SENTINEL-3
S3_SY_VGP:
productType: SY_2_VGP___
collection: SENTINEL-3
# S5P L1
S5P_L1B_IR_SIR:
productType: L1B_IR_SIR
collection: SENTINEL-5P
S5P_L1B_IR_UVN:
productType: L1B_IR_UVN
collection: SENTINEL-5P
S5P_L1B_RA_BD1:
productType: L1B_RA_BD1
collection: SENTINEL-5P
S5P_L1B_RA_BD2:
productType: L1B_RA_BD2
collection: SENTINEL-5P
S5P_L1B_RA_BD3:
productType: L1B_RA_BD3
collection: SENTINEL-5P
S5P_L1B_RA_BD4:
productType: L1B_RA_BD4
collection: SENTINEL-5P
S5P_L1B_RA_BD5:
productType: L1B_RA_BD5
collection: SENTINEL-5P
S5P_L1B_RA_BD6:
productType: L1B_RA_BD6
collection: SENTINEL-5P
S5P_L1B_RA_BD7:
productType: L1B_RA_BD7
collection: SENTINEL-5P
S5P_L1B_RA_BD8:
productType: L1B_RA_BD8
collection: SENTINEL-5P
# S5P L2
S5P_L2_NO2:
productType: L2__NO2___
collection: SENTINEL-5P
S5P_L2_CLOUD:
productType: L2__CLOUD_
collection: SENTINEL-5P
S5P_L2_O3:
productType: L2__O3____
collection: SENTINEL-5P
S5P_L2_CO:
productType: L2__CO____
collection: SENTINEL-5P
S5P_L2_AER_AI:
productType: L2__AER_AI
collection: SENTINEL-5P
S5P_L2_O3_PR:
productType: L2__O3__PR
collection: SENTINEL-5P
S5P_L2_O3_TCL:
productType: L2__O3_TCL
collection: SENTINEL-5P
S5P_L2_AER_LH:
productType: L2__AER_LH
collection: SENTINEL-5P
S5P_L2_HCHO:
productType: L2__HCHO__
collection: SENTINEL-5P
S5P_L2_CH4:
productType: L2__CH4___
collection: SENTINEL-5P
S5P_L2_NP_BD3:
productType: L2__NP_BD3
collection: SENTINEL-5P
S5P_L2_NP_BD6:
productType: L2__NP_BD6
collection: SENTINEL-5P
S5P_L2_NP_BD7:
productType: L2__NP_BD7
collection: SENTINEL-5P
S5P_L2_SO2:
productType: L2__SO2___
collection: SENTINEL-5P
GENERIC_PRODUCT_TYPE:
productType: '{productType}'
collection: '{collection}'
---
!provider
name: mundi
priority: 0
description: Atos DIAS
roles:
- host
url: https://mundiwebservices.com/
search: !plugin
type: QueryStringSearch
api_endpoint: 'https://{collection}.browse.catalog.mundiwebservices.com/opensearch'
need_auth: false
result_type: 'xml'
results_entry: '//ns:entry'
literal_search_params:
format: atom
relation: intersects
pagination:
next_page_url_tpl: '{url}?{search}&maxRecords={items_per_page}&startIndex={skip_base_1}'
total_items_nb_key_path: '//os:totalResults/text()'
# 2021/03/19: 50 is the max, no error if greater
max_items_per_page: 50
discover_metadata:
auto_discovery: true
metadata_pattern: '^(?!collection)[a-zA-Z0-9]+$'
search_param: '{metadata}={{{metadata}}}'
metadata_path: '*'
metadata_mapping:
# Opensearch resource identifier within the search engine context (in our case
# within the context of the data provider)
uid: 'ns:id/text()'
# OpenSearch Parameters for Collection Search (Table 3)
productType:
- 'productType'
- 'eo:productType/text()'
doi: 'eo:doi/text()'
platform: 'eo:platform/text()'
platformSerialIdentifier: 'eo:platformSerialIdentifier/text()'
instrument:
- 'instrument'
- 'eo:instrument/text()'
sensorType: 'eo:sensorType/text()'
compositeType: 'eo:compositeType/text()'
processingLevel:
- 'processingLevel'
- 'eo:processingLevel/text()'
orbitType: 'eo:orbitType/text()'
spectralRange: 'eo:spectralRange/text()'
wavelengths: 'eo:wavelengths/text()'
hasSecurityConstraints: 'eo:hasSecurityConstraints/text()'
dissemination: 'eo:dissemination/text()'
# INSPIRE obligated OpenSearch Parameters for Collection Search (Table 4)
title: 'ns:title/text()'
topicCategory: 'ns:category/text()'
keyword: 'eo:keyword/text()'
abstract: 'ns:summary/text()'
resolution: 'eo:resolution/text()'
organisationName: 'dc:creator/text()'
organisationRole: 'eo:organisationRole/text()'
publicationDate: 'ns:published/text()'
lineage: 'eo:lineage/text()'
useLimitation: 'eo:useLimitation/text()'
accessConstraint: 'eo:accessConstraint/text()'
otherConstraint: 'dc:rights/text()'
classification: 'DIAS:productConfidentialityLevel/text()'
language: 'eo:language/text()'
specification: 'eo:specification/text()'
# OpenSearch Parameters for Product Search (Table 5)
parentIdentifier: 'eo:parentIdentifier/text()'
acquisitionType: 'eo:acquisitionType/text()'
orbitNumber: 'eo:orbitNumber/text()'
orbitDirection: 'eo:orbitDirection/text()'
track: 'eo:track/text()'
frame: 'eo:frame/text()'
swathIdentifier: 'eo:swathIdentifier/text()'
cloudCover:
- 'cloudCover=[0,{cloudCover}]'
- 'eo:cloudCover/text()'
snowCover: 'eo:snowCover/text()'
lowestLocation: 'eo:lowestLocation/text()'
highestLocation: 'eo:highestLocation/text()'
productVersion: 'DIAS:processingBaseline/text()'
productQualityStatus: 'eo:productQualityStatus/text()'
productQualityDegradationTag: 'eo:productQualityDegradationTag/text()'
processorName: 'eo:processorName/text()'
processingCenter: 'eo:processingCenter/text()'
creationDate: 'DIAS:onlineDate/text()'
modificationDate: 'ns:updated/text()'
processingDate: 'eo:processingDate/text()'
sensorMode: 'eo:sensorMode/text()'
archivingCenter: 'eo:archivingCenter/text()'
processingMode: 'eo:processingMode/text()'
# OpenSearch Parameters for Acquistion Parameters Search (Table 6)
availabilityTime: 'DIAS:onlineDate/text()'
acquisitionStation: 'eo:acquisitionStation/text()'
acquisitionSubType: 'eo:acquisitionSubType/text()'
startTimeFromAscendingNode:
- 'timeStart={startTimeFromAscendingNode#to_iso_utc_datetime}'
- 'DIAS:sensingStartDate/text()'
completionTimeFromAscendingNode:
- 'timeEnd={completionTimeFromAscendingNode#to_iso_utc_datetime}'
- 'DIAS:sensingStopDate/text()'
illuminationAzimuthAngle: 'eo:illuminationAzimuthAngle/text()'
illuminationZenithAngle: 'eo:illuminationZenithAngle/text()'
illuminationElevationAngle: 'eo:illuminationElevationAngle/text()'
polarizationMode: 'eo:polarizationMode/text()'
polarizationChannels: '{eo:polarisationChannels/text()#replace_str("/"," ")}'
antennaLookDirection: 'eo:antennaLookDirection/text()'
minimumIncidenceAngle: 'eo:minimumIncidenceAngle/text()'
maximumIncidenceAngle: 'eo:maximumIncidenceAngle/text()'
dopplerFrequency: 'eo:dopplerFrequency/text()'
incidenceAngleVariation: 'eo:incidenceAngleVariation/text()'
# Custom parameters (not defined in the base document referenced above)
id:
- 'uid={id#remove_extension}'
- 'dc:identifier/text()'
tileIdentifier:
- 'tileIdentifier'
- 'DIAS:tileIdentifier/text()'
# The geographic extent of the product
geometry:
- 'geometry={geometry#to_rounded_wkt}'
- '{georss:polygon|georss:where#from_georss}'
defaultGeometry: '-180 -90 180 90'
# The url of the quicklook
quicklook: 'media:group/media:content[media:category="QUICKLOOK"]/@url'
# The url to download the product "as is" (literal or as a template to be completed either after the search result
# is obtained from the provider or during the eodag download phase)
downloadLink: 'ns:link[@rel="enclosure"]/@href'
# storageStatus: must be one of ONLINE, STAGING, OFFLINE
storageStatus: 'DIAS:onlineStatus/text()'
# order link formated for POST request usage
orderLink: 'https://apis.mundiwebservices.com/odrapi/0.1/request?{{"productId":"{id}","collectionId":"{platform}"}}'
searchLink: 'https://{platform}.browse.catalog.mundiwebservices.com/opensearch?uid={id}'
# Additional metadata provided by the providers but that don't appear in the reference spec
thumbnail: 'media:group/media:content[media:category="THUMBNAIL"]/@url'
download: !plugin
type: S3RestDownload
base_uri: 'https://mundiwebservices.com/dp'
extract: true
auth_error_code: 401
bucket_path_level: 0
# order mechanism
order_enabled: true
order_method: 'POST'
order_headers:
accept: application/json
Content-Type: application/json
order_on_response:
metadata_mapping:
order_id: '{$.requestId#replace_str("Not Available","")}'
reorder_id: '{$.message.`sub(/.*requestId: ([a-z0-9]+)/, \\1)`#replace_str("Not Available","")}'
orderStatusLink: 'https://apis.mundiwebservices.com/odrapi/0.1/request/{order_id}{reorder_id}'
order_status_method: 'GET'
order_status_percent: status
order_status_success:
status: Success
order_status_on_success:
need_search: true
result_type: 'xml'
results_entry: '//ns:entry'
metadata_mapping:
downloadLink: 'ns:link[@rel="enclosure"]/@href'
storageStatus: 'DIAS:onlineStatus/text()'
auth: !plugin
# Mixed HTTPHeaderAuth and TokenAuth
type: TokenAuth
auth_uri: 'https://apis.mundiwebservices.com/token'
token_type: json
token_key: access_token
credentials:
grant_type: password
headers:
Authorization: Basic cXJic1pTVjBUWTdkZzlNRGQ0Q3ZQVXkyQV9NYToxMU5oNGtFQjFva2NvZlBlRkRYemhudWxGR3dh
Cookie: "seeedtoken={apikey}"
products:
# S1
S1_SAR_GRD:
productType: GRD
collection: Sentinel1
metadata_mapping:
cloudCover: 'null/text()'
S1_SAR_SLC:
productType: SLC
collection: Sentinel1
metadata_mapping:
cloudCover: 'null/text()'
S1_SAR_OCN:
productType: OCN
collection: Sentinel1
metadata_mapping:
cloudCover: 'null/text()'
S1_SAR_RAW:
productType: RAW
collection: Sentinel1
metadata_mapping:
cloudCover: 'null/text()'
# S2
S2_MSI_L1C:
productType: IMAGE
processingLevel: L1C
collection: Sentinel2
S2_MSI_L2A:
productType: IMAGE
processingLevel: L2A
collection: Sentinel2
# S3 SRAL
S3_SRA:
productType: SR_1_SRA___
collection: Sentinel3
metadata_mapping:
cloudCover: 'null/text()'
S3_SRA_A:
productType: SR_1_SRA_A_
collection: Sentinel3
metadata_mapping:
cloudCover: 'null/text()'
S3_SRA_BS:
productType: SR_1_SRA_BS
collection: Sentinel3
metadata_mapping:
cloudCover: 'null/text()'
S3_LAN:
productType: SR_2_LAN___
collection: Sentinel3
metadata_mapping:
cloudCover: 'null/text()'
# S3 OLCI
S3_EFR:
productType: OL_1_EFR___
collection: Sentinel3
S3_ERR:
productType: OL_1_ERR___
collection: Sentinel3
S3_OLCI_L2LRR:
productType: OL_2_LRR___
collection: Sentinel3
S3_OLCI_L2LFR:
productType: OL_2_LFR___
collection: Sentinel3
# S3 SLSTR
S3_SLSTR_L1RBT:
productType: SL_1_RBT___
collection: Sentinel3
S3_SLSTR_L2LST:
productType: SL_2_LST___
collection: Sentinel3
S3_SLSTR_L2FRP:
productType: SL_2_FRP___
collection: Sentinel3
# S3 SY
S3_SY_SYN:
productType: SY_2_SYN___
collection: Sentinel3
S3_SY_V10:
productType: SY_2_V10___
collection: Sentinel3
S3_SY_VG1:
productType: SY_2_VG1___
collection: Sentinel3
S3_SY_VGP:
productType: SY_2_VGP___
collection: Sentinel3
# S5P L1
S5P_L1B_IR_SIR:
productType: L1B_IR_SIR
collection: Sentinel5p
S5P_L1B_IR_UVN:
productType: L1B_IR_UVN
collection: Sentinel5p
S5P_L1B_RA_BD1:
productType: L1B_RA_BD1
collection: Sentinel5p
S5P_L1B_RA_BD2:
productType: L1B_RA_BD2
collection: Sentinel5p
S5P_L1B_RA_BD3:
productType: L1B_RA_BD3
collection: Sentinel5p
S5P_L1B_RA_BD4:
productType: L1B_RA_BD4
collection: Sentinel5p
S5P_L1B_RA_BD5:
productType: L1B_RA_BD5
collection: Sentinel5p
S5P_L1B_RA_BD6:
productType: L1B_RA_BD6
collection: Sentinel5p
S5P_L1B_RA_BD7:
productType: L1B_RA_BD7
collection: Sentinel5p
S5P_L1B_RA_BD8:
productType: L1B_RA_BD8
collection: Sentinel5p
# S5P L2
S5P_L2_NO2:
productType: L2__NO2___
collection: Sentinel5p
S5P_L2_CLOUD:
productType: L2__CLOUD_
collection: Sentinel5p
S5P_L2_O3:
productType: L2__O3____
collection: Sentinel5p
S5P_L2_CO:
productType: L2__CO____
collection: Sentinel5p
S5P_L2_AER_AI:
productType: L2__AER_AI
collection: Sentinel5p
S5P_L2_O3_PR:
productType: L2__O3__PR
collection: Sentinel5p
S5P_L2_O3_TCL:
productType: L2__O3_TCL
collection: Sentinel5p
S5P_L2_AER_LH:
productType: L2__AER_LH
collection: Sentinel5p
S5P_L2_HCHO:
productType: L2__HCHO__
collection: Sentinel5p
S5P_L2_CH4:
productType: L2__CH4___
collection: Sentinel5p
S5P_L2_NP_BD3:
productType: L2__NP_BD3
collection: Sentinel5p
S5P_L2_NP_BD6:
productType: L2__NP_BD6
collection: Sentinel5p
S5P_L2_NP_BD7:
productType: L2__NP_BD7
collection: Sentinel5p
S5P_L2_SO2:
productType: L2__SO2___
collection: Sentinel5p
GENERIC_PRODUCT_TYPE:
productType: '{productType}'
collection: '{collection}'
instrument: '{instrument}'
processingLevel: '{processingLevel}'
---
!provider
name: onda
priority: 0
description: Serco DIAS
roles:
- host
url: https://www.onda-dias.eu/cms/
search: !plugin
type: ODataV4Search
api_endpoint: 'https://catalogue.onda-dias.eu/dias-catalogue/Products'
need_auth: false
dont_quote:
- '['
- ']'
- '$'
- '='
- '&'
- ':'
pagination:
count_endpoint: 'https://catalogue.onda-dias.eu/dias-catalogue/Products/$count'
next_page_url_tpl: '{url}?{search}&$top={items_per_page}&$skip={skip}&$expand=Metadata'
# 2021/03/19: 2000 is the max, if greater 200 response but contains an error message
max_items_per_page: 2_000
results_entry: 'value'
literal_search_params:
$format: json
free_text_search_operations:
$search:
union: ' OR '
wrapper: '"{}"'
operations:
AND:
- 'footprint:"Intersects({geometry#to_rounded_wkt})"'
- 'productType:{productType}'
- 'platformName:{platform}'
- 'cloudCoverPercentage:[0 TO {cloudCover}]'
- 'beginPosition:[{startTimeFromAscendingNode#to_iso_utc_datetime} TO *]'
- 'endPosition:[* TO {completionTimeFromAscendingNode#to_iso_utc_datetime}]'
- '{id#remove_extension}'
- 'platformSerialIdentifier:{platformSerialIdentifier}'
- 'instrumentShortName:{instrument}'
- 'processingLevel:{processingLevel}'
- 'sensorType:{sensorType}'
- 'topicCategory:{topicCategory}'
- 'lineage:{lineage}'
- 'orbitNumber:{orbitNumber}'
- 'orbitDirection:{orbitDirection}'
- 'processingBaseline:{productVersion}'
- 'generalQualityFlag:{productQualityStatus}'
- 'sensorOperationalMode:{sensorMode}'
- 'tileIdentifier:{tileIdentifier}'
discover_metadata:
auto_discovery: true
metadata_pattern: '^[a-zA-Z0-9]+$'
search_param:
free_text_search_operations:
$search:
operations:
AND:
- '{metadata}:{{{metadata}}}'
metadata_path: '$.Metadata.*'
per_product_metadata_query: false
metadata_pre_mapping:
metadata_path: '$.Metadata'
metadata_path_id: 'id'
metadata_path_value: 'value'
metadata_mapping:
# Opensearch resource identifier within the search engine context (in our case
# within the context of the data provider)
# Queryable parameters are set with null as 1st configuration list value to mark them as queryable,
# but `free_text_search_operations.$search.operations.AND` entries are then used instead.
uid: '$.id'
# OpenSearch Parameters for Collection Search (Table 3)
productType:
- null
- '$.Metadata.productType'
platform:
- null
- '$.Metadata.platformName'
platformSerialIdentifier:
- null
- '$.Metadata.platformSerialIdentifier'
instrument:
- null
- '$.Metadata.instrumentShortName'
processingLevel:
- null
- '$.Metadata.processingLevel'
sensorType:
- null
- '$.Metadata.sensorType'
# INSPIRE obligated OpenSearch Parameters for Collection Search (Table 4)
title: '$.Metadata.filename'
topicCategory:
- null
- '$.Metadata.topicCategory'
lineage:
- null
- '$.Metadata.lineage'
# OpenSearch Parameters for Product Search (Table 5)
orbitNumber:
- null
- '$.Metadata.orbitNumber'
orbitDirection:
- null
- '$.Metadata.orbitDirection'
cloudCover:
- null
- '$.Metadata.cloudCoverPercentage'
productVersion:
- null
- '$.Metadata.processingBaseline'
productQualityStatus:
- null
- '$.Metadata.generalQualityFlag'
creationDate: '$.Metadata.creationDate'
processingDate: '$.Metadata.processingDate'
sensorMode:
- null
- '$.Metadata.sensorOperationalMode'
# OpenSearch Parameters for Acquistion Parameters Search (Table 6)
startTimeFromAscendingNode:
- null
- '$.beginPosition'
completionTimeFromAscendingNode:
- null
- '$.endPosition'
polarizationChannels: '{$.Metadata.polarisationChannels#replace_str(","," ")}'
# Custom parameters (not defined in the base document referenced above)
id:
- null
- '{$.Metadata.filename#remove_extension}'
tileIdentifier:
- null
- '$.Metadata.tileIdentifier'
# The geographic extent of the product
geometry:
- null
- '$.footprint'
# The url of the quicklook
quicklook: '$.quicklook'
# The url to download the product "as is" (literal or as a template to be completed either after the search result
# is obtained from the provider or during the eodag download phase)
downloadLink: '%(base_uri)s({uid})/$value'
# storageStatus: must be one of ONLINE, STAGING, OFFLINE
storageStatus: '{$.offline#get_group_name((?P<ONLINE>False)|(?P<OFFLINE>True))}'
# Url used for ordering product if it is offline/archived
orderLink: 'https://catalogue.onda-dias.eu/dias-catalogue/Products({uid})/Ens.Order'
products:
# S1
S1_SAR_OCN:
productType: '*OCN*'
metadata_mapping:
cloudCover: '$.null'
S1_SAR_GRD:
productType: '*GRD*'
metadata_mapping:
cloudCover: '$.null'
S1_SAR_SLC:
productType: '*SLC*'
metadata_mapping:
cloudCover: '$.null'
S1_SAR_RAW:
productType: '*RAW*'
metadata_mapping:
cloudCover: '$.null'
# S2
S2_MSI_L1C:
productType: S2MSI1C
S2_MSI_L2A:
productType: S2MSI2A
# S3 SRAL
S3_SRA:
productType: SR_1_SRA___
metadata_mapping:
cloudCover: '$.null'
S3_SRA_A:
productType: SR_1_SRA_A_
metadata_mapping:
cloudCover: '$.null'
S3_SRA_BS:
productType: SR_1_SRA_BS
metadata_mapping:
cloudCover: '$.null'
S3_LAN:
productType: SR_2_LAN___
metadata_mapping:
cloudCover: '$.null'
S3_WAT:
productType: SR_2_WAT___
metadata_mapping:
cloudCover: '$.null'
# S3 OLCI
S3_EFR:
productType: OL_1_EFR___
S3_ERR:
productType: OL_1_ERR___
S3_OLCI_L2LFR:
productType: OL_2_LFR___
S3_OLCI_L2LRR:
productType: OL_2_LRR___
S3_OLCI_L2WRR:
productType: OL_2_WRR___
S3_OLCI_L2WFR:
productType: OL_2_WFR___
# S3 SLSTR
S3_SLSTR_L1RBT:
productType: SL_1_RBT___
S3_SLSTR_L2LST:
productType: SL_2_LST___
S3_SLSTR_L2WST:
productType: SL_2_WST___
S3_SLSTR_L2FRP:
productType: SL_2_FRP___
# S3 SY
S3_SY_AOD:
productType: SY_2_AOD___
S3_SY_SYN:
productType: SY_2_SYN___
S3_SY_V10:
productType: SY_2_V10___
S3_SY_VG1:
productType: SY_2_VG1___
S3_SY_VGP:
productType: SY_2_VGP___
# S5P L1
S5P_L1B_RA_BD1:
productType: L1B_RA_BD1
S5P_L1B_RA_BD2:
productType: L1B_RA_BD2
S5P_L1B_RA_BD3:
productType: L1B_RA_BD3
S5P_L1B_RA_BD4:
productType: L1B_RA_BD4
S5P_L1B_RA_BD5:
productType: L1B_RA_BD5
S5P_L1B_RA_BD6:
productType: L1B_RA_BD6
S5P_L1B_RA_BD7:
productType: L1B_RA_BD7
S5P_L1B_RA_BD8:
productType: L1B_RA_BD8
# S5P L2
S5P_L2_NO2:
productType: L2__NO2___
S5P_L2_CLOUD:
productType: L2__CLOUD_
S5P_L2_O3:
productType: L2__O3____
S5P_L2_CO:
productType: L2__CO____
S5P_L2_AER_AI:
productType: L2__AER_AI
S5P_L2_O3_PR:
productType: L2__O3__PR
S5P_L2_AER_LH:
productType: L2__AER_LH
S5P_L2_HCHO:
productType: L2__HCHO__
S5P_L2_CH4:
productType: L2__CH4___
S5P_L2_NP_BD3:
productType: L2__NP_BD3
S5P_L2_NP_BD6:
productType: L2__NP_BD6
S5P_L2_NP_BD7:
productType: L2__NP_BD7
S5P_L2_SO2:
productType: L2__SO2___
# Landsat
L8_OLI_TIRS_C1L1:
platform: 'Landsat-*'
GENERIC_PRODUCT_TYPE:
productType: '{productType}'
platform: '{platform}'
download: !plugin
type: HTTPDownload
base_uri: 'https://catalogue.onda-dias.eu/dias-catalogue/Products'
extract: true
auth_error_code: 401
order_enabled: true
order_method: 'POST'
order_headers:
Content-Type: application/json
auth: !plugin
type: GenericAuth
---
!provider
name: astraea_eod
priority: 0
roles:
- host
description: Astraea Earth OnDemand
url: https://earthondemand.astraea.earth
search: !plugin
type: StacSearch
api_endpoint: https://eod-catalog-svc-prod.astraea.earth/search
need_auth: false
pagination:
# 2021/03/19: The docs (https://eod-catalog-svc-prod.astraea.earth/api.html#operation/getSearchSTAC)
# say the max is 10_000. In practice 1_000 products are returned if more are asked (even greater
# than 10_000), without any error.
# This provider doesn't implement any pagination, let's just try to get the maximum number of
# products available at once then, so we stick to 10_000.
max_items_per_page: 10_000
metadata_mapping:
# redefine the following mapppings as the provider does not support advanced queries/filtering,
# these parameters will not be queryable
doi: '$.properties."sci:doi"'
processingLevel: '$.properties."processing:level"'
platform: '$.properties.constellation'
platformSerialIdentifier: '$.properties.platform'
instrument: '$.properties.instruments'
# INSPIRE obligated OpenSearch Parameters for Collection Search (Table 4)
title: '$.id'
abstract: '$.properties.description'
resolution: '$.properties.gsd'
publicationDate: '$.properties.published'
# OpenSearch Parameters for Product Search (Table 5)
orbitNumber: '$.properties."sat:relative_orbit"'
orbitDirection: '$.properties."sat:orbit_state"'
cloudCover: '$.properties."eo:cloud_cover"'
sensorMode: '$.properties."sar:instrument_mode"'
creationDate: '$.properties.created'
modificationDate: '$.properties.updated'
productVersion: '$.properties.version'
# OpenSearch Parameters for Acquistion Parameters Search (Table 6)
availabilityTime: '$.properties.availabilityTime'
acquisitionStation: '$.properties.acquisitionStation'
acquisitionSubType: '$.properties.acquisitionSubType'
illuminationAzimuthAngle: '$.properties."view:sun_azimuth"'
illuminationElevationAngle: '$.properties."view:sun_elevation"'
polarizationChannels: '$.properties."sar:polarizations"'
dopplerFrequency: '$.properties."sar:frequency_band"'
products:
S1_SAR_GRD:
productType: sentinel1_l1c_grd
metadata_mapping:
cloudCover: '$.null'
platformSerialIdentifier: '$.id.`split(_, 0, -1)`'
polarizationMode: '$.id.`sub(/.{14}([A-Z]{2}).*/, \\1)`'
awsPath: '$.assets.productInfo.href.`sub(/(.*)\/productInfo\.json/, \\1)`'
assets: |
{$.assets#dict_update([["manifest.safe",[["title","manifest.safe"],["href","{awsPath}/manifest.safe"],["roles",["metadata"]],["type","application/xml"]]]])}
S2_MSI_L1C:
productType: sentinel2_l1c
metadata_mapping:
platformSerialIdentifier: '$.id.`split(_, 0, -1)`'
tileInfo: '{$.assets.tileInfo.href#replace_str(r"s3(.*)sentinel-s2-l1c/",r"https\1roda.sentinel-hub.com/sentinel-s2-l1c/")}'
tilePath: |
$.assets.tileInfo.href.`sub(/.*\/sentinel-s2-l1c\/(tiles\/.*)\/tileInfo\.json/, \\1)`
S2_MSI_L2A:
productType: sentinel2_l2a
metadata_mapping:
platformSerialIdentifier: '$.id.`split(_, 0, -1)`'
tileInfo: '{$.assets.tileInfo.href#replace_str(r"s3(.*)sentinel-s2-l2a/",r"https\1roda.sentinel-hub.com/sentinel-s2-l2a/")}'
tilePath: |
$.assets.tileInfo.href.`sub(/.*\/sentinel-s2-l2a\/(tiles\/.*)\/tileInfo\.json/, \\1)`
LANDSAT_C2L1:
productType: landsat8_c2l1t1
MODIS_MCD43A4:
productType: mcd43a4
NAIP:
productType: naip
GENERIC_PRODUCT_TYPE:
productType: '{productType}'
download: !plugin
type: AwsDownload
requester_pays: True
flatten_top_dirs: True
products:
S1_SAR_GRD:
default_bucket: 'sentinel-s1-l1c'
build_safe: true
complementary_url_key:
- awsPath
S2_MSI_L1C:
default_bucket: 'sentinel-s2-l1c'
build_safe: true
fetch_metadata:
fetch_url: '{tileInfo}'
fetch_format: json
update_metadata:
title: '$.productName'
productPath: '$.productPath'
complementary_url_key:
- productPath
- tilePath
S2_MSI_L2A:
default_bucket: 'sentinel-s2-l2a'
build_safe: true
fetch_metadata:
fetch_url: '{tileInfo}'
fetch_format: json
update_metadata:
title: '$.productName'
productPath: '$.productPath'
complementary_url_key:
- productPath
- tilePath
auth: !plugin
type: AwsAuth
---
!provider
name: usgs_satapi_aws
priority: 0
roles:
- host
description: USGS Landsatlook SAT API
url: https://landsatlook.usgs.gov/stac-server
search: !plugin
type: StacSearch
api_endpoint: https://landsatlook.usgs.gov/stac-server/search
need_auth: false
pagination:
# 2021/03/19: no more than 10_000 (if greater, returns a 500 error code)
# but in practive if an Internal Server Error is returned for more than
# about 500 products.
max_items_per_page: 500
metadata_mapping:
assets: '{$.assets#recursive_sub_str(r"https?(.*)landsatlook.usgs.gov/data/",r"s3\1usgs-landsat/")}'
awsProductId: '{$.assets.thumbnail.href#replace_str(r".+/([A-Z0-9_]+)/[\w.]+$",r"\1")}'
products:
LANDSAT_C2L1:
productType: landsat-c2l1
LANDSAT_C2L2_SR:
productType: landsat-c2l2-sr
LANDSAT_C2L2_ST:
productType: landsat-c2l2-st
LANDSAT_C2L2ALB_BT:
productType: landsat-c2l2alb-bt
LANDSAT_C2L2ALB_SR:
productType: landsat-c2l2alb-sr
LANDSAT_C2L2ALB_ST:
productType: landsat-c2l2alb-st
LANDSAT_C2L2ALB_TA:
productType: landsat-c2l2alb-ta
GENERIC_PRODUCT_TYPE:
productType: '{productType}'
download: !plugin
type: AwsDownload
requester_pays: True
flatten_top_dirs: True
auth: !plugin
type: AwsAuth
---
!provider
name: earth_search
priority: 0
roles:
- host
description: Earth Search
url: https://www.element84.com/earth-search/
search: !plugin
type: StacSearch
api_endpoint: https://earth-search.aws.element84.com/v0/search
need_auth: false
discover_product_types:
results_entry: '$.collections[?id!="sentinel-s2-l2a-cogs"]'
pagination:
# Override the default next page url key path of StacSearch because the next link returned
# by Earth Search is invalid (as of 2021/04/29). Entry set to null (None) to avoid using the
# next page retrieval mechanism, `next_page_url_tpl` will be used instead (inherited from StacSearch)
# Remove that entry if Earth Search updates that and returns a valid link.
next_page_url_key_path: null
# 2021/04/28: Earth-Search relies on Sat-API whose docs (http://sat-utils.github.io/sat-api/#search-stac-items-by-simple-filtering-)
# say the max is 10_000. In practice a too high number (e.g. 5_000) returns a 502 error ({"message": "Internal server error"}).
# Let's set it to a more robust number: 500
max_items_per_page: 500
metadata_mapping:
utmZone:
- '{{"query":{{"sentinel:utm_zone":{{"eq":"{utmZone}"}}}}}}'
- '$.properties."sentinel:utm_zone"'
latitudeBand:
- '{{"query":{{"sentinel:latitude_band":{{"eq":"{latitudeBand}"}}}}}}'
- '$.properties."sentinel:latitude_band"'
gridSquare:
- '{{"query":{{"sentinel:grid_square":{{"eq":"{gridSquare}"}}}}}}'
- '$.properties."sentinel:grid_square"'
tileIdentifier:
- '{{"query":{{"sentinel:utm_zone":{{"eq":"{tileIdentifier#slice_str(0,2,1)}"}},"sentinel:latitude_band":{{"eq":"{tileIdentifier#slice_str(2,3,1)}"}},"sentinel:grid_square":{{"eq":"{tileIdentifier#slice_str(3,5,1)}"}}}}}}'
- '{utmZone}{latitudeBand}{gridSquare}'
products:
S2_MSI_L1C:
productType: sentinel-s2-l1c
metadata_mapping:
title: '$.properties."sentinel:product_id"'
platformSerialIdentifier: '$.id.`split(_, 0, -1)`'
polarizationMode: '$.id.`sub(/.{14}([A-Z]{2}).*/, \\1)`'
productPath: |
$.properties."sentinel:product_id".`sub(/([S2AB]{3})_MSIL1C_([0-9]{4})([0-9]{2})([0-9]{2})(T.*)/, products!\\2!\\3!\\4!\\1_MSIL1C_\\2\\3\\4\\5)`.`sub(/!0*/, /)`
tilePath: |
$.assets.info.href.`sub(/.*/sentinel-s2-l1c\/(tiles\/.*)\/tileInfo\.json/, \\1)`
utmZone:
- '{{"query":{{"mgrs:utm_zone":{{"eq":"{utmZone}"}}}}}}'
- '$.properties."mgrs:utm_zone"'
latitudeBand:
- '{{"query":{{"mgrs:latitude_band":{{"eq":"{latitudeBand}"}}}}}}'
- '$.properties."mgrs:latitude_band"'
gridSquare:
- '{{"query":{{"mgrs:grid_square":{{"eq":"{gridSquare}"}}}}}}'
- '$.properties."mgrs:grid_square"'
tileIdentifier:
- '{{"query":{{"mgrs:utm_zone":{{"eq":"{tileIdentifier#slice_str(0,2,1)}"}},"mgrs:latitude_band":{{"eq":"{tileIdentifier#slice_str(2,3,1)}"}},"mgrs:grid_square":{{"eq":"{tileIdentifier#slice_str(3,5,1)}"}}}}}}'
- '{utmZone}{latitudeBand}{gridSquare}'
S2_MSI_L2A:
productType: sentinel-s2-l2a
metadata_mapping:
title: '$.properties."sentinel:product_id"'
platformSerialIdentifier: '$.id.`split(_, 0, -1)`'
polarizationMode: '$.id.`sub(/.{14}([A-Z]{2}).*/, \\1)`'
productPath: |
$.properties."sentinel:product_id".`sub(/([S2AB]{3})_MSIL2A_([0-9]{4})([0-9]{2})([0-9]{2})(T.*)/, products!\\2!\\3!\\4!\\1_MSIL2A_\\2\\3\\4\\5)`.`sub(/!0*/, /)`
tilePath: |
$.assets.info.href.`sub(/.*/sentinel-s2-l2a\/(tiles\/.*)\/tileInfo\.json/, \\1)`
L8_OLI_TIRS_C1L1:
productType: landsat-8-l1-c1
GENERIC_PRODUCT_TYPE:
productType: '{productType}'
download: !plugin
type: AwsDownload
requester_pays: True
flatten_top_dirs: True
products:
S2_MSI_L1C:
default_bucket: 'sentinel-s2-l1c'
build_safe: true
complementary_url_key:
- productPath
- tilePath
S2_MSI_L2A:
default_bucket: 'sentinel-s2-l2a'
build_safe: true
complementary_url_key:
- productPath
- tilePath
auth: !plugin
type: AwsAuth
---
!provider
name: earth_search_cog
priority: 0
roles:
- host
description: Earth Search with Cloud Optimized GeoTIFF (COG) formatted assets
url: https://www.element84.com/earth-search/
search: !plugin
type: StacSearch
api_endpoint: https://earth-search.aws.element84.com/v0/search
need_auth: false
discover_product_types:
fetch_url: null
pagination:
# Override the default next page url key path of StacSearch because the next link returned
# by Earth Search is invalid (as of 2021/04/29). Entry set to null (None) to avoid using the
# next page retrieval mechanism, `next_page_url_tpl` will be used instead (inherited from StacSearch)
# Remove that entry if Earth Search updates that and returns a valid link.
next_page_url_key_path: null
# 2021/04/28: Earth-Search relies on Sat-API whose docs (http://sat-utils.github.io/sat-api/#search-stac-items-by-simple-filtering-)
# say the max is 10_000. In practice a too high number (e.g. 5_000) returns a 502 error ({"message": "Internal server error"}).
# Let's set it to a more robust number: 500
max_items_per_page: 500
metadata_mapping:
platformSerialIdentifier: '$.id.`split(_, 0, -1)`'
polarizationMode: '$.id.`sub(/.{14}([A-Z]{2}).*/, \\1)`'
products:
S2_MSI_L2A_COG:
productType: sentinel-s2-l2a-cogs
GENERIC_PRODUCT_TYPE:
productType: '{productType}'
download: !plugin
type: HTTPDownload
base_uri: 'https://sentinel-cogs.s3.us-west-2.amazonaws.com'
---
!provider
name: earth_search_gcs
priority: 0
roles:
- host
description: Google Cloud Storage through Earth Search
url: https://www.element84.com/earth-search/
search: !plugin
type: StacSearch
api_endpoint: https://earth-search.aws.element84.com/v0/search
need_auth: false
discover_product_types:
fetch_url: null
pagination:
# Override the default next page url key path of StacSearch because the next link returned
# by Earth Search is invalid (as of 2021/04/29). Entry set to null (None) to avoid using the
# next page retrieval mechanism, `next_page_url_tpl` will be used instead (inherited from StacSearch)
# Remove that entry if Earth Search updates that and returns a valid link.
next_page_url_key_path: null
# 2021/04/28: Earth-Search relies on Sat-API whose docs (http://sat-utils.github.io/sat-api/#search-stac-items-by-simple-filtering-)
# say the max is 10_000. In practice a too high number (e.g. 5_000) returns a 502 error ({"message": "Internal server error"}).
# Let's set it to a more robust number: 500
max_items_per_page: 500
products:
S2_MSI_L1C:
productType: sentinel-s2-l1c
metadata_mapping:
title: '$.properties."sentinel:product_id"'
platformSerialIdentifier: '$.id.`split(_, 0, -1)`'
polarizationMode: '$.id.`sub(/.{14}([A-Z]{2}).*/, \\1)`'
utmZone: '$.properties."sentinel:utm_zone"'
latitudeBand: '$.properties."sentinel:latitude_band"'
gridSquare: '$.properties."sentinel:grid_square"'
downloadLink: 's3://gcp-public-data-sentinel-2/tiles/{utmZone}/{latitudeBand}/{gridSquare}/{title}.SAFE'
L8_OLI_TIRS_C1L1:
productType: landsat-8-l1-c1
metadata_mapping:
wrsPath: '$.properties."landsat:wrs_path"'
wrsRow: '$.properties."landsat:wrs_row"'
downloadLink: 's3://gcp-public-data-landsat/LC08/01/{wrsPath:03d}/{wrsRow:03d}/{title}'
GENERIC_PRODUCT_TYPE:
productType: '{productType}'
download: !plugin
type: AwsDownload
base_uri: https://storage.googleapis.com
ignore_assets: True
products:
S2_MSI_L1C:
default_bucket: 'gcp-public-data-sentinel-2'
auth: !plugin
type: AwsAuth
---
!provider
name: ecmwf
priority: 0
description: ECMWF archive products
roles:
- host
url: https://www.ecmwf.int
api: !plugin
type: EcmwfApi
api_endpoint: https://api.ecmwf.int/v1
extract: false
metadata_mapping:
productType: '$.productType'
title: '$.id'
startTimeFromAscendingNode: '{$.startTimeFromAscendingNode#to_iso_date}'
completionTimeFromAscendingNode:
- 'date={startTimeFromAscendingNode#to_iso_date}/to/{completionTimeFromAscendingNode#to_iso_date(-1,)}'
- '{$.completionTimeFromAscendingNode#to_iso_date}'
id: '$.id'
# The geographic extent of the product
geometry:
- 'area={geometry#to_nwse_bounds_str(/)}'
- '$.geometry'
# storageStatus set to ONLINE for consistency between providers
storageStatus: '{$.null#replace_str("Not Available","ONLINE")}'
downloadLink: 'https://apps.ecmwf.int/datasets/data/{dataset}'
# ECMWF/MARS specific parameters
# Full list here https://confluence.ecmwf.int/display/UDOC/Keywords+in+MARS+and+Dissemination+requests
accuracy:
- accuracy
- '$.accuracy'
anoffset:
- anoffset
- '$.anoffset'
bitmap:
- bitmap
- '$.bitmap'
block:
- block
- '$.block'
channel:
- channel
- '$.channel'
class:
- class
- '$.class'
database:
- database
- '$.database'
dataset:
- dataset
- '$.dataset'
diagnostic:
- diagnostic
- '$.diagnostic'
direction:
- direction
- '$.direction'
domain:
- domain
- '$.domain'
duplicates:
- duplicates
- '$.duplicates'
expect:
- expect
- '$.expect'
expver:
- expver
- '$.expver'
fcmonth:
- fcmonth
- '$.fcmonth'
fcperiod:
- fcperiod
- '$.fcperiod'
fieldset:
- fieldset
- '$.fieldset'
filter:
- filter
- '$.filter'
format:
- format
- '$.format'
frame:
- frame
- '$.frame'
frequency:
- frequency
- '$.frequency'
grid:
- grid
- '$.grid'
hdate:
- hdate
- '$.hdate'
ident:
- ident
- '$.ident'
interpolation:
- interpolation
- '$.interpolation'
intgrid:
- intgrid
- '$.intgrid'
iteration:
- iteration
- '$.iteration'
latitude:
- latitude
- '$.latitude'
levelist:
- levelist
- '$.levelist'
levtype:
- levtype
- '$.levtype'
longitude:
- longitude
- '$.longitude'
lsm:
- lsm
- '$.lsm'
method:
- method
- '$.method'
number:
- number
- '$.number'
obsgroup:
- obsgroup
- '$.obsgroup'
obstype:
- obstype
- '$.obstype'
origin:
- origin
- '$.origin'
packing:
- packing
- '$.packing'
padding:
- padding
- '$.padding'
param:
- param
- '$.param'
priority:
- priority
- '$.priority'
product:
- product
- '$.product'
range:
- range
- '$.range'
refdate:
- refdate
- '$.refdate'
reference:
- reference
- '$.reference'
reportype:
- reportype
- '$.reportype'
repres:
- repres
- '$.repres'
resol:
- resol
- '$.resol'
rotation:
- rotation
- '$.rotation'
section:
- section
- '$.section'
source:
- source
- '$.source'
step:
- step
- '$.step'
stream:
- stream
- '$.stream'
system:
- system
- '$.system'
target:
- target
- '$.target'
time:
- time
- '$.time'
truncation:
- truncation
- '$.truncation'
type:
- type
- '$.type'
use:
- use
- '$.use'
products:
# See Archive Catalog in https://apps.ecmwf.int/archive-catalogue/
# See available Public Datasets in https://apps.ecmwf.int/datasets/
TIGGE_CF_SFC:
class: ti
dataset: tigge
expver: prod
type: cf
levtype: sfc
origin: ecmf
grid: 0.5/0.5
param: 59/134/136/146/147/151/165/166/167/168/172/176/177/179/189/235/228001/228002/228039/228139/228141/228144/228164/228228
step: 0
time: 00:00
GENERIC_PRODUCT_TYPE:
dataset: '{productType}'
---
!provider
name: cop_ads
priority: 0
description: Copernicus Atmosphere Data Store
roles:
- host
url: https://ads.atmosphere.copernicus.eu/
api: !plugin
type: CdsApi
api_endpoint: https://ads.atmosphere.copernicus.eu/api/v2
extract: false
metadata_mapping:
productType: '$.productType'
title: '$.id'
startTimeFromAscendingNode: '{$.startTimeFromAscendingNode#to_iso_date}'
completionTimeFromAscendingNode:
- 'date={startTimeFromAscendingNode#to_iso_date}/{completionTimeFromAscendingNode#to_iso_date(-1,)}'
- '{$.completionTimeFromAscendingNode#to_iso_date}'
id: '$.id'
# The geographic extent of the product
geometry:
- 'area={geometry#to_nwse_bounds_str(/)}'
- '$.geometry'
# storageStatus set to ONLINE for consistency between providers
storageStatus: '{$.null#replace_str("Not Available","ONLINE")}'
downloadLink: 'https://ads.atmosphere.copernicus.eu/cdsapp#!/dataset/{dataset}'
# Copernicus ADS specific parameters
accuracy:
- accuracy
- '$.accuracy'
anoffset:
- anoffset
- '$.anoffset'
api_product_type:
- product_type
- '$.api_product_type'
bitmap:
- bitmap
- '$.bitmap'
block:
- block
- '$.block'
channel:
- channel
- '$.channel'
class:
- class
- '$.class'
database:
- database
- '$.database'
dataset:
- dataset
- '$.dataset'
date_range:
- date_range
- '$.date_range'
day:
- day
- '$.day'
diagnostic:
- diagnostic
- '$.diagnostic'
direction:
- direction
- '$.direction'
domain:
- domain
- '$.domain'
duplicates:
- duplicates
- '$.duplicates'
expect:
- expect
- '$.expect'
expver:
- expver
- '$.expver'
fcmonth:
- fcmonth
- '$.fcmonth'
fcperiod:
- fcperiod
- '$.fcperiod'
fieldset:
- fieldset
- '$.fieldset'
filter:
- filter
- '$.filter'
format:
- format
- '$.format'
frame:
- frame
- '$.frame'
frequency:
- frequency
- '$.frequency'
grid:
- grid
- '$.grid'
hdate:
- hdate
- '$.hdate'
ident:
- ident
- '$.ident'
interpolation:
- interpolation
- '$.interpolation'
intgrid:
- intgrid
- '$.intgrid'
iteration:
- iteration
- '$.iteration'
latitude:
- latitude
- '$.latitude'
levelist:
- levelist
- '$.levelist'
leadtime_hour:
- leadtime_hour
- '$.leadtime_hour'
levtype:
- levtype
- '$.levtype'
longitude:
- longitude
- '$.longitude'
lsm:
- lsm
- '$.lsm'
method:
- method
- '$.method'
model_level:
- model_level
- '$.model_level'
month:
- month
- '$.month'
number:
- number
- '$.number'
obsgroup:
- obsgroup
- '$.obsgroup'
obstype:
- obstype
- '$.obstype'
origin:
- origin
- '$.origin'
packing:
- packing
- '$.packing'
padding:
- padding
- '$.padding'
param:
- param
- '$.param'
pressure_level:
- pressure_level
- '$.pressure_level'
priority:
- priority
- '$.priority'
product:
- product
- '$.product'
range:
- range
- '$.range'
refdate:
- refdate
- '$.refdate'
reference:
- reference
- '$.reference'
reportype:
- reportype
- '$.reportype'
repres:
- repres
- '$.repres'
resol:
- resol
- '$.resol'
rotation:
- rotation
- '$.rotation'
section:
- section
- '$.section'
source:
- source
- '$.source'
step:
- step
- '$.step'
stream:
- stream
- '$.stream'
system:
- system
- '$.system'
target:
- target
- '$.target'
time:
- time
- '$.time'
truncation:
- truncation
- '$.truncation'
type:
- type
- '$.type'
use:
- use
- '$.use'
variable:
- variable
- '$.variable'
year:
- year
- '$.year'
products:
# See available Public Datasets in https://ads.atmosphere.copernicus.eu/cdsapp#!/search?type=dataset
CAMS_GACF_AOT:
dataset: cams-global-atmospheric-composition-forecasts
stream: oper
class: mc
expver: '0001'
step: 0
levtype: SFC
variable:
- sea_salt_aerosol_optical_depth_550nm
- dust_aerosol_optical_depth_550nm
- organic_matter_aerosol_optical_depth_550nm
- black_carbon_aerosol_optical_depth_550nm
- sulphate_aerosol_optical_depth_550nm
time: '00:00'
leadtime_hour: 0
type: forecast
format: netcdf
CAMS_GACF_RH:
dataset: cams-global-atmospheric-composition-forecasts
stream: oper
class: mc
expver: '0001'
step: 0
variable: 'relative_humidity'
pressure_level: ['1', '2', '3', '5', '7', '10', '20', '30', '50', '70', '100', '150', '200', '250', '300', '400', '500', '600', '700', '850', '925', '1000']
time: '00:00'
leadtime_hour: 0
type: forecast
format: netcdf
CAMS_GACF_MR:
dataset: cams-global-atmospheric-composition-forecasts
stream: oper
class: mc
expver: '0001'
step: 0
variable:
- sea_salt_aerosol_0.03-0.5um_mixing_ratio
- sea_salt_aerosol_0.5-5um_mixing_ratio
- sea_salt_aerosol_5-20um_mixing_ratio
- dust_aerosol_0.03-0.55um_mixing_ratio
- dust_aerosol_0.55-0.9um_mixing_ratio
- dust_aerosol_0.9-20um_mixing_ratio
- hydrophilic_organic_matter_aerosol_mixing_ratio
- hydrophobic_organic_matter_aerosol_mixing_ratio
- hydrophilic_black_carbon_aerosol_mixing_ratio
- hydrophobic_black_carbon_aerosol_mixing_ratio
- sulphate_aerosol_mixing_ratio
model_level: ['1', '2', '3', '4', '5', '6', '7', '8', '9', '10', '11', '12', '13', '14', '15', '16', '17', '18', '19', '20', '21', '22', '23', '24', '25', '26', '27', '28', '29', '30', '31', '32', '33', '34', '35', '36', '37', '38', '39', '40', '41', '42', '43', '44', '45', '46', '47', '48', '49', '50', '51', '52', '53', '54', '55', '56', '57', '58', '59', '60', '61']
time: '00:00'
leadtime_hour: 0
type: forecast
format: netcdf
CAMS_EAC4:
dataset: cams-global-reanalysis-eac4
stream: oper
class: mc
expver: '0001'
step: 0
variable:
- dust_aerosol_0.03-0.55um_mixing_ratio
- dust_aerosol_0.55-0.9um_mixing_ratio
- dust_aerosol_0.9-20um_mixing_ratio
- dust_aerosol_optical_depth_550nm
- hydrophilic_black_carbon_aerosol_mixing_ratio
- hydrophilic_organic_matter_aerosol_mixing_ratio
- hydrophobic_black_carbon_aerosol_mixing_ratio
- hydrophobic_organic_matter_aerosol_mixing_ratio
- sea_salt_aerosol_0.03-0.5um_mixing_ratio
- sea_salt_aerosol_0.5-5um_mixing_ratio
- sea_salt_aerosol_5-20um_mixing_ratio
- sea_salt_aerosol_optical_depth_550nm
- sulphate_aerosol_optical_depth_550nm
model_level: ['1', '2', '3', '4', '5', '6', '7', '8', '9', '10', '11', '12', '13', '14', '15', '16', '17', '18', '19', '20', '21', '22', '23', '24', '25', '26', '27', '28', '29', '30', '31', '32', '33', '34', '35', '36', '37', '38', '39', '40', '41', '42', '43', '44', '45', '46', '47', '48', '49', '50', '51', '52', '53', '54', '55', '56', '57', '58', '59', '60']
time: '00:00'
format: netcdf
GENERIC_PRODUCT_TYPE:
dataset: '{productType}'
stream: oper
class: mc
expver: '0001'
step: 0
time: '00:00'
format: netcdf
---
!provider
name: cop_cds
priority: 0
description: Copernicus Climate Data Store
roles:
- host
url: https://cds.climate.copernicus.eu/
api: !plugin
type: CdsApi
api_endpoint: https://cds.climate.copernicus.eu/api/v2
extract: false
metadata_mapping:
productType: '$.productType'
title: '$.id'
startTimeFromAscendingNode: '{$.startTimeFromAscendingNode#to_iso_date}'
completionTimeFromAscendingNode:
- 'date={startTimeFromAscendingNode#to_iso_date}/{completionTimeFromAscendingNode#to_iso_date(-1,)}'
- '{$.completionTimeFromAscendingNode#to_iso_date}'
id: '$.id'
# The geographic extent of the product
geometry:
- 'area={geometry#to_nwse_bounds_str(/)}'
- '$.geometry'
# storageStatus set to ONLINE for consistency between providers
storageStatus: '{$.null#replace_str("Not Available","ONLINE")}'
downloadLink: 'https://cds.climate.copernicus.eu/cdsapp#!/dataset/{dataset}'
# Copernicus CDS specific parameters
accuracy:
- accuracy
- '$.accuracy'
anoffset:
- anoffset
- '$.anoffset'
api_product_type:
- product_type
- '$.api_product_type'
bitmap:
- bitmap
- '$.bitmap'
block:
- block
- '$.block'
channel:
- channel
- '$.channel'
class:
- class
- '$.class'
database:
- database
- '$.database'
dataset:
- dataset
- '$.dataset'
date_range:
- date_range
- '$.date_range'
day:
- day
- '$.day'
diagnostic:
- diagnostic
- '$.diagnostic'
direction:
- direction
- '$.direction'
domain:
- domain
- '$.domain'
duplicates:
- duplicates
- '$.duplicates'
expect:
- expect
- '$.expect'
expver:
- expver
- '$.expver'
fcmonth:
- fcmonth
- '$.fcmonth'
fcperiod:
- fcperiod
- '$.fcperiod'
fieldset:
- fieldset
- '$.fieldset'
filter:
- filter
- '$.filter'
format:
- format
- '$.format'
frame:
- frame
- '$.frame'
frequency:
- frequency
- '$.frequency'
grid:
- grid
- '$.grid'
hdate:
- hdate
- '$.hdate'
ident:
- ident
- '$.ident'
interpolation:
- interpolation
- '$.interpolation'
intgrid:
- intgrid
- '$.intgrid'
iteration:
- iteration
- '$.iteration'
latitude:
- latitude
- '$.latitude'
levelist:
- levelist
- '$.levelist'
leadtime_hour:
- leadtime_hour
- '$.leadtime_hour'
levtype:
- levtype
- '$.levtype'
longitude:
- longitude
- '$.longitude'
lsm:
- lsm
- '$.lsm'
method:
- method
- '$.method'
model_level:
- model_level
- '$.model_level'
month:
- month
- '$.month'
number:
- number
- '$.number'
obsgroup:
- obsgroup
- '$.obsgroup'
obstype:
- obstype
- '$.obstype'
origin:
- origin
- '$.origin'
packing:
- packing
- '$.packing'
padding:
- padding
- '$.padding'
param:
- param
- '$.param'
pressure_level:
- pressure_level
- '$.pressure_level'
priority:
- priority
- '$.priority'
product:
- product
- '$.product'
range:
- range
- '$.range'
refdate:
- refdate
- '$.refdate'
reference:
- reference
- '$.reference'
reportype:
- reportype
- '$.reportype'
repres:
- repres
- '$.repres'
resol:
- resol
- '$.resol'
rotation:
- rotation
- '$.rotation'
section:
- section
- '$.section'
source:
- source
- '$.source'
step:
- step
- '$.step'
stream:
- stream
- '$.stream'
system:
- system
- '$.system'
target:
- target
- '$.target'
time:
- time
- '$.time'
truncation:
- truncation
- '$.truncation'
type:
- type
- '$.type'
use:
- use
- '$.use'
variable:
- variable
- '$.variable'
year:
- year
- '$.year'
products:
# See available Public Datasets in https://cds.climate.copernicus.eu/cdsapp#!/search?type=dataset
ERA5_SL:
dataset: reanalysis-era5-single-levels
api_product_type: reanalysis
time: '00:00'
format: netcdf
GENERIC_PRODUCT_TYPE:
dataset: '{productType}'
time: '00:00'
format: netcdf
---
!provider
name: sara
priority: 0
description: Sentinel Australasia Regional Access
search: !plugin
type: QueryStringSearch
# The endpoint is based off of the collection. There is a generic endpoint,
# but can be very slow if not enough metdata is provided.
api_endpoint: 'https://copernicus.nci.org.au/sara.server/1.0/api/collections/{collection}/search.json'
need_auth: false
pagination:
next_page_url_tpl: '{url}?{search}&maxRecords={items_per_page}&page={page}'
total_items_nb_key_path: '$.properties.totalResults'
# 2021/03/19: 500 is the max, no error if greater
max_items_per_page: 500
discover_metadata:
auto_discovery: true
metadata_pattern: '^(?!collection)[a-zA-Z0-9_]+$'
search_param: '{metadata}={{{metadata}}}'
metadata_path: '$.properties.*'
metadata_mapping:
# Opensearch resource identifier within the search engine context (in our case
# within the context of the data provider)
uid: '$.id'
# OpenSearch Parameters for Collection Search (Table 3)
productType:
- productType
- '$.properties.productType'
platform: '$.properties.collection'
platformSerialIdentifier:
- platform
- '$.properties.platform'
instrument:
- instrument
- '$.properties.instrument'
processingLevel:
- processingLevel
- '$.properties.processingLevel'
# INSPIRE obligated OpenSearch Parameters for Collection Search (Table 4)
title: '$.properties.title'
topicCategory: '$.properties.topicCategory'
keyword: '$.properties.keywords'
abstract: '$.properties.description'
resolution:
- 'resolution'
- '$.properties.resolution'
organisationName:
- 'organisationName'
- '$.properties.organisationName'
publicationDate: '$.properties.published'
accessConstraint: '$.properties.license'
# OpenSearch Parameters for Product Search (Table 5)
parentIdentifier:
- 'parentIdentifier'
- '$.properties.parentIdentifier'
acquisitionType: '$.properties.acquisitionType'
orbitNumber:
- 'orbitNumber'
- '$.properties.orbitNumber'
orbitDirection:
- 'orbitDirection'
- '$.properties.orbitDirection'
swathIdentifier:
- 'swath'
- '$.properties.swath'
cloudCover:
- 'cloudCover=[0,{cloudCover}]'
- '$.properties.cloudCover'
snowCover:
- 'snowCover=[0,{snowCover}]'
- '$.properties.snowCover'
productVersion: '$.properties.version'
productQualityStatus: '$.properties.onlineQualityCheck'
processorName: '$.properties.processingName'
processingCenter: '$.properties.processingCenter'
creationDate: '$.properties.dhusIngestDate'
modificationDate: '$.properties.updated'
sensorMode:
- 'sensorMode'
- '$.properties.sensorMode'
# OpenSearch Parameters for Acquistion Parameters Search (Table 6)
startTimeFromAscendingNode:
- startDate
- '$.properties.startDate'
completionTimeFromAscendingNode:
- completionDate
- '$.properties.completionDate'
polarizationMode:
- 'polarisation'
- '$.properties.polarisation'
# Custom parameters (not defined in the base document referenced above)
id:
- 'productIdentifier={id#remove_extension}'
- '$.properties.productIdentifier'
# The geographic extent of the product
geometry:
- 'geometry={geometry#to_rounded_wkt}'
- '$.geometry'
# The url of the quicklook
quicklook: '$.properties.quicklook'
# The url to download the product "as is" (literal or as a template to be completed either after the search result
# is obtained from the provider or during the eodag download phase)
downloadLink: '$.properties.services.download.url'
# storageStatus set to ONLINE for consistency between providers
storageStatus: '{$.null#replace_str("Not Available","ONLINE")}'
# Additional metadata provided by the providers but that don't appear in the reference spec
thumbnail: '$.properties.thumbnail'
products:
# Sentinel 1
S1_SAR_OCN:
productType: OCN
collection: S1
instrument: C-SAR
S1_SAR_GRD:
productType: GRD
collection: S1
instrument: C-SAR
S1_SAR_SLC:
productType: SLC
collection: S1
instrument: C-SAR
# Sentinel 2
S2_MSI_L1C:
collection: S2
productType: S2MSIL1C
instrument: MSI
processingLevel: L1C
S2_MSI_L2A:
collection: S2
productType: S2MSIL2A
instrument: MSI
processingLevel: L2A
# OLCI products
# L1
S3_EFR:
productType: OL_1_EFR___
collection: S3
instrument: OLCI
processingLevel: LEVEL-1
S3_ERR:
productType: OL_1_ERR___
collection: S3
instrument: OLCI
processingLevel: LEVEL-1
# output during radiometric calibration mode
S3_RAC:
productType: OL_1_RAC___
collection: S3
instrument: OLCI
processingLevel: LEVEL-1
# L2
S3_OLCI_L2LRR:
productType: OL_2_LRR___
collection: S3
instrument: OLCI
processingLevel: LEVEL-2
S3_OLCI_L2LFR:
productType: OL_2_LFR___
collection: S3
instrument: OLCI
processingLevel: LEVEL-2
S3_OLCI_L2WRR:
productType: OL_2_WRR___
collection: S3
instrument: OLCI
processingLevel: LEVEL-2
S3_OLCI_L2WFR:
productType: OL_2_WFR___
collection: S3
instrument: OLCI
processingLevel: LEVEL-2
# SLSTR products
S3_SLSTR_L1RBT:
productType: SL_1_RBT___
collection: S3
instrument: SLSTR
processingLevel: LEVEL-1
S3_SLSTR_L2LST:
productType: SL_2_LST___
collection: S3
instrument: SLSTR
processingLevel: LEVEL-2
S3_SLSTR_L2WST:
productType: SL_2_WST___
collection: S3
instrument: SLSTR
processingLevel: LEVEL-2
S3_SLSTR_L2AOD:
productType: SL_2_AOD___
collection: S3
instrument: SLSTR
processingLevel: LEVEL-2
S3_SLSTR_L2FRP:
productType: SL_2_FRP___
collection: S3
instrument: SLSTR
processingLevel: LEVEL-2
# SRAL
# L1
S3_SRA:
productType: SR_1_SRA___
collection: S3
instrument: SRAL
processingLevel: LEVEL-1
S3_SRA_A:
productType: SR_1_SRA_A_
collection: S3
instrument: SRAL
processingLevel: LEVEL-1
S3_SRA_BS:
productType: SR_1_SRA_BS
collection: S3
instrument: SRAL
processingLevel: LEVEL-1
# L2
S3_LAN:
productType: SR_2_LAN___
collection: S3
instrument: SRAL
processingLevel: LEVEL-2
S3_WAT:
productType: SR_2_WAT___
collection: S3
instrument: SRAL
processingLevel: LEVEL-2
# Synergy products
S3_SY_AOD:
productType: SY_2_AOD___
collection: S3
instrument: SYNERGY
processingLevel: LEVEL-2
S3_SY_SYN:
productType: SY_2_SYN___
collection: S3
instrument: SYNERGY
processingLevel: LEVEL-2
S3_SY_V10:
productType: SY_2_V10___
collection: S3
instrument: SYNERGY
processingLevel: LEVEL-2W
S3_SY_VG1:
productType: SY_2_VG1___
collection: S3
instrument: SYNERGY
processingLevel: LEVEL-2
S3_SY_VGP:
productType: SY_2_VGP___
collection: S3
instrument: SYNERGY
processingLevel: LEVEL-2
GENERIC_PRODUCT_TYPE:
productType: '{productType}'
collection: '{collection}'
download: !plugin
type: HTTPDownload
base_uri: 'https://copernicus.nci.org.au/'
extract: true
archive_depth: 2
order_enabled: true
auth_error_code: 403
auth: !plugin
type: GenericAuth
method: basic
---
!provider
name: meteoblue
priority: 0
roles:
- host
description: Meteoblue
url: https://www.meteoblue.com
search: !plugin
type: BuildPostSearchResult
api_endpoint: 'https://my.meteoblue.com/dataset/query'
need_auth: true
pagination:
next_page_query_obj: '{{"checkOnly":true}}'
discover_metadata:
auto_discovery: true
metadata_pattern: '^[a-zA-Z0-9_]+$'
search_param: '{{{{"{metadata}":{{{metadata}#to_geojson}} }}}}'
metadata_path: '$.*'
metadata_mapping:
startTimeFromAscendingNode: '{$.startTimeFromAscendingNode#to_iso_utc_datetime}'
completionTimeFromAscendingNode:
- '{{"timeIntervals": [ "{startTimeFromAscendingNode#to_iso_date}/{completionTimeFromAscendingNode#to_iso_date(-1,)}" ] }}'
- '{$.completionTimeFromAscendingNode#to_iso_utc_datetime}'
geometry:
- '{{"geometry": {geometry#to_geojson} }}'
- '$.geometry'
productType: '$.queries[0].domain'
storageStatus: '{$.requiresJobQueue#get_group_name((?P<ONLINE>False)|(?P<OFFLINE>True))}'
downloadLink: 'https://my.meteoblue.com/dataset/query'
# Meteoblue specific parameters
datapoints: '$.datapoints'
requiresJobQueue: '$.requiresJobQueue'
requiresComplexJobQueue: '$.requiresComplexJobQueue'
units:
- '{{"units": {units#to_geojson} }}'
- '$.units'
queries:
- '{{"queries": {queries#to_geojson} }}'
- '$.queries'
format:
- '{{"format": {format#to_geojson} }}'
- '$.format'
timeIntervalsAlignment:
- '{{"timeIntervalsAlignment": {timeIntervalsAlignment#to_geojson} }}'
- '$.timeIntervalsAlignment'
orderLink: '{downloadLink#replace_str(r"^(.*)(\")(queries\")(.)",r"\1\2runOnJobQueue\2\4 true, \2\3\4")}'
products:
NEMSGLOBAL_TCDC:
queries: [{'domain':'NEMSGLOBAL','gapFillDomain':null,'timeResolution':'daily','codes':[{'code':71,'level':'sfc','aggregation':'mean'}]}]
format: netCDF
units: {'temperature':'C','velocity':'km/h','length':'metric','energy':'watts'}
timeIntervalsAlignment:
geometry: {"type": "Polygon", "coordinates": [[[180, -90], [180, 90], [-180, 90], [-180, -90], [180, -90]]]}
NEMSAUTO_TCDC:
queries: [{'domain':'NEMSAUTO','gapFillDomain':null,'timeResolution':'daily','codes':[{'code':71,'level':'sfc','aggregation':'mean'}]}]
format: netCDF
units: {'temperature':'C','velocity':'km/h','length':'metric','energy':'watts'}
timeIntervalsAlignment:
geometry: {"type": "Polygon", "coordinates": [[[180, -90], [180, 90], [-180, 90], [-180, -90], [180, -90]]]}
GENERIC_PRODUCT_TYPE:
productType: '{productType}'
geometry: {"type": "Polygon", "coordinates": [[[180, -90], [180, 90], [-180, 90], [-180, -90], [180, -90]]]}
download: !plugin
type: HTTPDownload
base_uri: 'https://my.meteoblue.com/dataset/query'
method: POST
extract: False
order_enabled: true
order_method: 'POST'
order_on_response:
metadata_mapping:
order_id: '$.id'
downloadLink: 'http://queueresults.meteoblue.com/{order_id}'
downloadMethod: '{$.null#replace_str("Not Available","GET")}'
orderStatusLink: 'http://my.meteoblue.com/queue/status/{order_id}'
order_status_method: 'GET'
order_status_percent: percentCompleted
order_status_error:
status: error
outputs_extension: .nc
auth: !plugin
type: HttpQueryStringAuth
auth_uri: 'http://my.meteoblue.com/dataset/meta?dataset=NEMSAUTO'
---
!provider
name: cop_dataspace
priority: 0
description: Copernicus Data Space Ecosystem
roles:
- host
url: https://dataspace.copernicus.eu/
search: !plugin
type: ODataV4Search
api_endpoint: 'https://catalogue.dataspace.copernicus.eu/odata/v1/Products'
need_auth: false
dont_quote:
- '['
- ']'
- '$'
- '='
- '&'
- ':'
pagination:
next_page_url_tpl: '{url}?{search}&$top={items_per_page}&$skip={skip}&$expand=Attributes&$count=True'
total_items_nb_key_path: '$."@odata.count"'
max_items_per_page: 1_000
results_entry: 'value'
free_text_search_operations:
$filter:
union: ' or '
wrapper: '{}'
operations:
and:
- "Collection/Name eq '{collection}'"
- "OData.CSC.Intersects(area=geography'{geometry#to_ewkt}')"
- "Attributes/OData.CSC.StringAttribute/any(att:att/Name eq 'productType' and att/OData.CSC.StringAttribute/Value eq '{productType}')"
- "Attributes/OData.CSC.StringAttribute/any(att:att/Name eq 'platformShortName' and att/OData.CSC.StringAttribute/Value eq '{platform}')"
- "Attributes/OData.CSC.StringAttribute/any(att:att/Name eq 'platformSerialIdentifier' and att/OData.CSC.StringAttribute/Value eq '{platformSerialIdentifier}')"
- "Attributes/OData.CSC.IntegerAttribute/any(att:att/Name eq 'spatialResolution' and att/OData.CSC.StringAttribute/Value eq '{resolution}')"
- "Attributes/OData.CSC.StringAttribute/any(att:att/Name eq 'authority' and att/OData.CSC.StringAttribute/Value eq '{organisationName}')"
- "Attributes/OData.CSC.IntegerAttribute/any(att:att/Name eq 'orbitNumber' and att/OData.CSC.StringAttribute/Value eq '{orbitNumber}')"
- "Attributes/OData.CSC.StringAttribute/any(att:att/Name eq 'orbitDirection' and att/OData.CSC.StringAttribute/Value eq '{orbitDirection}')"
- "Attributes/OData.CSC.DoubleAttribute/any(att:att/Name eq 'cloudCover' and att/OData.CSC.DoubleAttribute/Value le {cloudCover})"
- "Attributes/OData.CSC.StringAttribute/any(att:att/Name eq 'operationalMode' and att/OData.CSC.StringAttribute/Value eq '{sensorMode}')"
- "Attributes/OData.CSC.StringAttribute/any(att:att/Name eq 'polarisationChannels' and att/OData.CSC.StringAttribute/Value eq '{polarizationChannels}')"
- "Attributes/OData.CSC.StringAttribute/any(att:att/Name eq 'tileId' and att/OData.CSC.StringAttribute/Value eq '{tileIdentifier}')"
- 'ContentDate/Start gt {startTimeFromAscendingNode#to_iso_utc_datetime}'
- 'ContentDate/End lt {completionTimeFromAscendingNode#to_iso_utc_datetime}'
- contains(Name,'{id}')
discover_metadata:
auto_discovery: true
metadata_pattern: '^(?!collection)[a-zA-Z0-9]+$'
search_param:
free_text_search_operations:
$filter:
operations:
and:
- "Attributes/OData.CSC.StringAttribute/any(att:att/Name eq '{metadata}' and att/OData.CSC.StringAttribute/Value eq '{{{metadata}}}')"
metadata_path: '$.Attributes.*'
per_product_metadata_query: false
metadata_pre_mapping:
metadata_path: '$.Attributes'
metadata_path_id: 'Name'
metadata_path_value: 'Value'
metadata_mapping:
# Opensearch resource identifier within the search engine context (in our case
# within the context of the data provider)
# Queryable parameters are set with null as 1st configuration list value to mark them as queryable,
# but `free_text_search_operations.$filter.operations.and` entries are then used instead.
uid: '$.Id'
# OpenSearch Parameters for Collection Search (Table 3)
productType:
- null
- '$.Attributes.productType'
platform:
- null
- '$.Attributes.platformShortName'
platformSerialIdentifier:
- null
- '$.Attributes.platformSerialIdentifier'
instrument:
- null
- '$.Attributes.instrumentShortName'
processingLevel:
- null
- '$.Attributes.processingLevel'
# INSPIRE obligated OpenSearch Parameters for Collection Search (Table 4)
title: '$.Name'
resolution:
- null
- '$.Attributes.spatialResolution'
organisationName:
- null
- '$.Attributes.authority'
publicationDate: '$.PublicationDate'
# OpenSearch Parameters for Product Search (Table 5)
orbitNumber:
- null
- '$.Attributes.orbitNumber'
orbitDirection:
- null
- '$.Attributes.orbitDirection'
cloudCover:
- null
- '$.Attributes.cloudCover'
modificationDate: '$.ModificationDate'
sensorMode:
- null
- '$.Attributes.operationalMode'
# OpenSearch Parameters for Acquistion Parameters Search (Table 6)
startTimeFromAscendingNode:
- null
- '$.ContentDate.Start'
completionTimeFromAscendingNode:
- null
- '$.ContentDate.End'
polarizationChannels:
- null
- '$.Attributes.polarisationChannels'
# Custom parameters (not defined in the base document referenced above)
id:
- null
- '{$.Name#remove_extension}'
tileIdentifier:
- null
- '$.Attributes.tileId'
# The geographic extent of the product
geometry:
- null
- '{$.Footprint.`sub(/.*(SRID=[0-9]+;.*)/, \\1)`#from_ewkt}'
# The url to download the product "as is" (literal or as a template to be completed either after the search result
# is obtained from the provider or during the eodag download phase)
downloadLink: 'https://catalogue.dataspace.copernicus.eu/odata/v1/Products({uid})/$value'
# storageStatus: must be one of ONLINE, STAGING, OFFLINE
storageStatus: '{$.Online#get_group_name((?P<ONLINE>True)|(?P<OFFLINE>False))}'
collection:
- null
- $.null
# Additional metadata provided by the providers but that don't appear in the reference spec
productIdentifier: '$.S3Path'
download: !plugin
type: HTTPDownload
base_uri: 'https://catalogue.dataspace.copernicus.eu/odata/v1/Products'
extract: true
order_enabled: false
archive_depth: 2
auth: !plugin
type: KeycloakOIDCPasswordAuth
auth_base_uri: 'https://identity.dataspace.copernicus.eu/auth'
realm: 'CDSE'
client_id: 'cdse-public'
client_secret: null
token_provision: qs
token_qs_key: 'token'
auth_error_code: 401
products:
# S2
S2_MSI_L1C:
collection: SENTINEL-2
productType: S2MSI1C
S2_MSI_L2A:
collection: SENTINEL-2
productType: S2MSI2A
# S1
S1_SAR_RAW:
productType: RAW
collection: SENTINEL-1
metadata_mapping:
cloudCover: '$.null'
S1_SAR_GRD:
productType: GRD
collection: SENTINEL-1
metadata_mapping:
cloudCover: '$.null'
S1_SAR_SLC:
productType: SLC
collection: SENTINEL-1
metadata_mapping:
cloudCover: '$.null'
S1_SAR_OCN:
productType: OCN
collection: SENTINEL-1
metadata_mapping:
cloudCover: '$.null'
# S3 SRAL
S3_SRA:
productType: SR_1_SRA___
collection: SENTINEL-3
metadata_mapping:
cloudCover: '$.null'
S3_SRA_A:
productType: SR_1_SRA_A_
collection: SENTINEL-3
metadata_mapping:
cloudCover: '$.null'
S3_SRA_BS:
productType: SR_1_SRA_BS
collection: SENTINEL-3
metadata_mapping:
cloudCover: '$.null'
S3_LAN:
productType: SR_2_LAN___
collection: SENTINEL-3
metadata_mapping:
cloudCover: '$.null'
S3_WAT:
productType: SR_2_WAT___
collection: SENTINEL-3
metadata_mapping:
cloudCover: '$.null'
# S3 OLCI
S3_EFR:
productType: OL_1_EFR___
collection: SENTINEL-3
S3_ERR:
productType: OL_1_ERR___
collection: SENTINEL-3
S3_OLCI_L2LRR:
productType: OL_2_LRR___
collection: SENTINEL-3
S3_OLCI_L2LFR:
productType: OL_2_LFR___
collection: SENTINEL-3
S3_OLCI_L2WRR:
productType: OL_2_WRR___
collection: SENTINEL-3
S3_OLCI_L2WFR:
productType: OL_2_WFR___
collection: SENTINEL-3
# S3 SLSTR
S3_SLSTR_L1RBT:
productType: SL_1_RBT___
collection: SENTINEL-3
S3_SLSTR_L2LST:
productType: SL_2_LST___
collection: SENTINEL-3
S3_SLSTR_L2WST:
productType: SL_2_WST___
collection: SENTINEL-3
S3_SLSTR_L2AOD:
productType: SL_2_AOD___
collection: SENTINEL-3
S3_SLSTR_L2FRP:
productType: SL_2_FRP___
collection: SENTINEL-3
# S3 SY
S3_SY_AOD:
productType: SY_2_AOD___
collection: SENTINEL-3
S3_SY_SYN:
productType: SY_2_SYN___
collection: SENTINEL-3
S3_SY_V10:
productType: SY_2_V10___
collection: SENTINEL-3
S3_SY_VG1:
productType: SY_2_VG1___
collection: SENTINEL-3
S3_SY_VGP:
productType: SY_2_VGP___
collection: SENTINEL-3
# S5P L1
S5P_L1B_RA_BD1:
productType: L1B_RA_BD1
collection: SENTINEL-5P
S5P_L1B_RA_BD2:
productType: L1B_RA_BD2
collection: SENTINEL-5P
S5P_L1B_RA_BD3:
productType: L1B_RA_BD3
collection: SENTINEL-5P
S5P_L1B_RA_BD4:
productType: L1B_RA_BD4
collection: SENTINEL-5P
S5P_L1B_RA_BD5:
productType: L1B_RA_BD5
collection: SENTINEL-5P
S5P_L1B_RA_BD6:
productType: L1B_RA_BD6
collection: SENTINEL-5P
S5P_L1B_RA_BD7:
productType: L1B_RA_BD7
collection: SENTINEL-5P
S5P_L1B_RA_BD8:
productType: L1B_RA_BD8
collection: SENTINEL-5P
# S5P L2
S5P_L2_NO2:
productType: L2__NO2___
collection: SENTINEL-5P
S5P_L2_CLOUD:
productType: L2__CLOUD_
collection: SENTINEL-5P
S5P_L2_O3:
productType: L2__O3____
collection: SENTINEL-5P
S5P_L2_CO:
productType: L2__CO____
collection: SENTINEL-5P
S5P_L2_AER_AI:
productType: L2__AER_AI
collection: SENTINEL-5P
S5P_L2_O3_PR:
productType: L2__O3__PR
collection: SENTINEL-5P
S5P_L2_AER_LH:
productType: L2__AER_LH
collection: SENTINEL-5P
S5P_L2_HCHO:
productType: L2__HCHO__
collection: SENTINEL-5P
S5P_L2_CH4:
productType: L2__CH4___
collection: SENTINEL-5P
S5P_L2_NP_BD3:
productType: L2__NP_BD3
collection: SENTINEL-5P
S5P_L2_NP_BD6:
productType: L2__NP_BD6
collection: SENTINEL-5P
S5P_L2_NP_BD7:
productType: L2__NP_BD7
collection: SENTINEL-5P
S5P_L2_SO2:
productType: L2__SO2___
collection: SENTINEL-5P
GENERIC_PRODUCT_TYPE:
productType: '{productType}'
collection: '{collection}'
---
!provider
name: planetary_computer
priority: 0
roles:
- host
description: Microsoft Planetary Computer
url: https://planetarycomputer.microsoft.com
search: !plugin
type: StacSearch
api_endpoint: https://planetarycomputer.microsoft.com/api/stac/v1/search
need_auth: false
pagination:
max_items_per_page: 1000
metadata_mapping:
tileIdentifier:
- '{{"query":{{"s2:mgrs_tile":{{"eq":"{tileIdentifier}"}}}}}}'
- '$.properties."s2:mgrs_tile"'
products:
S1_SAR_GRD:
productType: sentinel-1-grd
S2_MSI_L2A:
productType: sentinel-2-l2a
LANDSAT_C2L1:
productType: landsat-c2-l1
LANDSAT_C2L2:
productType: landsat-c2-l2
MODIS_MCD43A4:
productType: modis-43A4-061
NAIP:
productType: naip
GENERIC_PRODUCT_TYPE:
productType: '{productType}'
download: !plugin
type: HTTPDownload
base_uri: 'https://planetarycomputer.microsoft.com/api/stac/download'
flatten_top_dirs: True
auth_error_code: 403
auth: !plugin
type: SASAuth
auth_uri: 'https://planetarycomputer.microsoft.com/api/sas/v1/sign?href={url}'
signed_url_key: href
headers:
Ocp-Apim-Subscription-Key: "{apikey}"
---
!provider
name: hydroweb_next
priority: 0
roles:
- host
description: hydroweb.next thematic hub for hydrology data access
url: https://hydroweb.next.theia-land.fr
search: !plugin
type: StacSearch
api_endpoint: https://hydroweb.next.theia-land.fr/api/v1/rs-catalog/stac/search
need_auth: true
auth_error_code: 401
pagination:
max_items_per_page: 10_000
metadata_mapping:
startTimeFromAscendingNode:
- '{{"query":{{"end_datetime":{{"gte":"{startTimeFromAscendingNode#to_iso_utc_datetime}"}}}}}}'
- '$.properties.start_datetime'
completionTimeFromAscendingNode:
- '{{"query":{{"start_datetime":{{"lte":"{completionTimeFromAscendingNode#to_iso_utc_datetime}"}}}}}}'
- '$.properties.end_datetime'
products:
GENERIC_PRODUCT_TYPE:
productType: '{productType}'
download: !plugin
type: HTTPDownload
base_uri: https://hydroweb.next.theia-land.fr
flatten_top_dirs: true
auth_error_code: 401
auth: !plugin
type: HTTPHeaderAuth
headers:
X-API-Key: "{apikey}"
Parameters mapping#
EODAG maps each provider’s specific metadata parameters to a common model using OGC OpenSearch Extension for Earth Observation. Extra parameters having no equivalent in this model are mapped as is.
Depending on the provider, some parameters are queryable or not. This is configured in providers.yml:
If a parameter metadata-mapping is a list, the first element will help constructing the query (using format()), and the 2nd will help extracting its values from the query result (using jsonpath)
If a parameter metadata-mapping is a string, it will not be queryable and this string will help extracting its values from the query result (using jsonpath).
some_provider:
search:
metadata_mapping:
queryableParameter:
- 'this_is_query_string={queryableParameter}'
- '$.jsonpath.in.result.to.parameter'
nonQueryableParameter: '$.jsonpath.in.result.to.another_parameter'
The following tables list the parameters supported by providers, and if they are queryable or not.
OpenSearch parameters (CSV)#
parameter |
astraea_eod |
cop_ads |
cop_cds |
cop_dataspace |
creodias |
earth_search |
earth_search_cog |
earth_search_gcs |
ecmwf |
mundi |
onda |
peps |
planetary_computer |
sara |
theia |
usgs_satapi_aws |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
abstract |
metadata only |
metadata only |
metadata only |
metadata only |
metadata only |
metadata only |
metadata only |
metadata only |
metadata only |
|||||||
accessConstraint |
metadata only |
metadata only |
metadata only |
metadata only |
||||||||||||
acquisitionStation |
metadata only |
queryable metadata |
queryable metadata |
queryable metadata |
metadata only |
queryable metadata |
metadata only |
queryable metadata |
||||||||
acquisitionSubType |
metadata only |
queryable metadata |
queryable metadata |
queryable metadata |
metadata only |
queryable metadata |
metadata only |
queryable metadata |
||||||||
acquisitionType |
metadata only |
metadata only |
metadata only |
|||||||||||||
antennaLookDirection |
metadata only |
metadata only |
||||||||||||||
archivingCenter |
metadata only |
metadata only |
||||||||||||||
availabilityTime |
metadata only |
queryable metadata |
queryable metadata |
queryable metadata |
metadata only |
queryable metadata |
metadata only |
queryable metadata |
||||||||
classification |
metadata only |
|||||||||||||||
cloudCover |
metadata only |
queryable metadata |
queryable metadata |
queryable metadata |
queryable metadata |
queryable metadata |
queryable metadata |
queryable metadata |
queryable metadata |
queryable metadata |
queryable metadata |
queryable metadata |
queryable metadata |
|||
completionTimeFromAscendingNode |
queryable metadata |
queryable metadata |
queryable metadata |
queryable metadata |
queryable metadata |
queryable metadata |
queryable metadata |
queryable metadata |
queryable metadata |
queryable metadata |
queryable metadata |
queryable metadata |
queryable metadata |
|||
compositeType |
metadata only |
|||||||||||||||
creationDate |
metadata only |
queryable metadata |
queryable metadata |
queryable metadata |
metadata only |
metadata only |
metadata only |
queryable metadata |
metadata only |
metadata only |
queryable metadata |
|||||
dissemination |
metadata only |
|||||||||||||||
doi |
metadata only |
queryable metadata |
queryable metadata |
queryable metadata |
metadata only |
queryable metadata |
queryable metadata |
|||||||||
dopplerFrequency |
metadata only |
queryable metadata |
queryable metadata |
queryable metadata |
metadata only |
queryable metadata |
metadata only |
queryable metadata |
||||||||
frame |
metadata only |
|||||||||||||||
hasSecurityConstraints |
metadata only |
|||||||||||||||
highestLocation |
metadata only |
|||||||||||||||
illuminationAzimuthAngle |
metadata only |
queryable metadata |
queryable metadata |
queryable metadata |
metadata only |
queryable metadata |
metadata only |
queryable metadata |
||||||||
illuminationElevationAngle |
metadata only |
queryable metadata |
queryable metadata |
queryable metadata |
metadata only |
queryable metadata |
metadata only |
queryable metadata |
||||||||
illuminationZenithAngle |
metadata only |
metadata only |
||||||||||||||
incidenceAngleVariation |
metadata only |
metadata only |
||||||||||||||
instrument |
metadata only |
queryable metadata |
queryable metadata |
queryable metadata |
queryable metadata |
queryable metadata |
queryable metadata |
queryable metadata |
queryable metadata |
queryable metadata |
queryable metadata |
queryable metadata |
queryable metadata |
|||
keyword |
metadata only |
metadata only |
metadata only |
metadata only |
||||||||||||
language |
metadata only |
|||||||||||||||
lineage |
metadata only |
queryable metadata |
||||||||||||||
lowestLocation |
metadata only |
|||||||||||||||
maximumIncidenceAngle |
metadata only |
metadata only |
||||||||||||||
minimumIncidenceAngle |
metadata only |
metadata only |
||||||||||||||
modificationDate |
metadata only |
metadata only |
metadata only |
queryable metadata |
queryable metadata |
queryable metadata |
metadata only |
metadata only |
queryable metadata |
metadata only |
metadata only |
queryable metadata |
||||
orbitDirection |
metadata only |
queryable metadata |
queryable metadata |
queryable metadata |
queryable metadata |
queryable metadata |
metadata only |
queryable metadata |
queryable metadata |
queryable metadata |
queryable metadata |
queryable metadata |
||||
orbitNumber |
metadata only |
queryable metadata |
queryable metadata |
queryable metadata |
queryable metadata |
queryable metadata |
metadata only |
queryable metadata |
queryable metadata |
queryable metadata |
queryable metadata |
metadata only |
queryable metadata |
|||
orbitType |
metadata only |
|||||||||||||||
organisationName |
queryable metadata |
queryable metadata |
metadata only |
queryable metadata |
queryable metadata |
metadata only |
||||||||||
organisationRole |
metadata only |
|||||||||||||||
otherConstraint |
metadata only |
|||||||||||||||
parentIdentifier |
metadata only |
queryable metadata |
queryable metadata |
metadata only |
||||||||||||
platform |
metadata only |
queryable metadata |
queryable metadata |
queryable metadata |
queryable metadata |
queryable metadata |
metadata only |
queryable metadata |
metadata only |
queryable metadata |
metadata only |
queryable metadata |
queryable metadata |
|||
platformSerialIdentifier |
metadata only |
queryable metadata |
queryable metadata |
queryable metadata |
metadata only |
queryable metadata |
metadata only |
queryable metadata |
queryable metadata |
queryable metadata |
queryable metadata |
queryable metadata |
queryable metadata |
|||
processingCenter |
metadata only |
metadata only |
metadata only |
|||||||||||||
processingDate |
metadata only |
metadata only |
metadata only |
|||||||||||||
processingLevel |
metadata only |
queryable metadata |
queryable metadata |
queryable metadata |
queryable metadata |
queryable metadata |
queryable metadata |
queryable metadata |
queryable metadata |
queryable metadata |
queryable metadata |
queryable metadata |
queryable metadata |
|||
processingMode |
metadata only |
metadata only |
||||||||||||||
processorName |
metadata only |
metadata only |
metadata only |
|||||||||||||
productQualityDegradationTag |
metadata only |
|||||||||||||||
productQualityStatus |
metadata only |
queryable metadata |
metadata only |
metadata only |
||||||||||||
productType |
queryable metadata |
queryable metadata |
queryable metadata |
queryable metadata |
queryable metadata |
queryable metadata |
queryable metadata |
queryable metadata |
queryable metadata |
queryable metadata |
queryable metadata |
queryable metadata |
queryable metadata |
|||
productVersion |
metadata only |
queryable metadata |
queryable metadata |
queryable metadata |
metadata only |
queryable metadata |
metadata only |
queryable metadata |
metadata only |
metadata only |
queryable metadata |
|||||
publicationDate |
metadata only |
metadata only |
metadata only |
queryable metadata |
queryable metadata |
queryable metadata |
metadata only |
metadata only |
queryable metadata |
metadata only |
metadata only |
queryable metadata |
||||
resolution |
metadata only |
queryable metadata |
queryable metadata |
queryable metadata |
queryable metadata |
queryable metadata |
metadata only |
queryable metadata |
queryable metadata |
queryable metadata |
metadata only |
queryable metadata |
||||
sensorMode |
metadata only |
queryable metadata |
queryable metadata |
queryable metadata |
queryable metadata |
queryable metadata |
metadata only |
queryable metadata |
queryable metadata |
queryable metadata |
queryable metadata |
metadata only |
queryable metadata |
|||
sensorType |
metadata only |
queryable metadata |
||||||||||||||
snowCover |
metadata only |
queryable metadata |
queryable metadata |
metadata only |
||||||||||||
spectralRange |
metadata only |
|||||||||||||||
startTimeFromAscendingNode |
metadata only |
queryable metadata |
queryable metadata |
metadata only |
metadata only |
metadata only |
queryable metadata |
queryable metadata |
queryable metadata |
metadata only |
queryable metadata |
queryable metadata |
metadata only |
|||
swathIdentifier |
metadata only |
queryable metadata |
queryable metadata |
|||||||||||||
title |
metadata only |
metadata only |
metadata only |
metadata only |
metadata only |
metadata only |
metadata only |
metadata only |
metadata only |
metadata only |
metadata only |
metadata only |
metadata only |
|||
topicCategory |
metadata only |
queryable metadata |
metadata only |
metadata only |
||||||||||||
track |
metadata only |
|||||||||||||||
useLimitation |
metadata only |
|||||||||||||||
wavelengths |
metadata only |
Provider/eodag specific parameters (CSV)#
parameter |
astraea_eod |
cop_ads |
cop_cds |
cop_dataspace |
creodias |
earth_search |
earth_search_cog |
earth_search_gcs |
ecmwf |
mundi |
onda |
peps |
planetary_computer |
sara |
theia |
usgs_satapi_aws |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
assets |
metadata only |
metadata only |
metadata only |
metadata only |
metadata only |
metadata only |
||||||||||
awsProductId |
metadata only |
|||||||||||||||
collection |
queryable metadata |
queryable metadata |
||||||||||||||
defaultGeometry |
metadata only |
|||||||||||||||
downloadLink |
metadata only |
metadata only |
metadata only |
metadata only |
metadata only |
metadata only |
metadata only |
metadata only |
metadata only |
metadata only |
metadata only |
metadata only |
metadata only |
|||
geometry |
queryable metadata |
queryable metadata |
queryable metadata |
queryable metadata |
queryable metadata |
queryable metadata |
queryable metadata |
queryable metadata |
queryable metadata |
queryable metadata |
queryable metadata |
queryable metadata |
queryable metadata |
|||
gridSquare |
queryable metadata |
|||||||||||||||
id |
queryable metadata |
queryable metadata |
queryable metadata |
queryable metadata |
queryable metadata |
queryable metadata |
queryable metadata |
queryable metadata |
queryable metadata |
queryable metadata |
queryable metadata |
metadata only |
queryable metadata |
|||
latitudeBand |
queryable metadata |
|||||||||||||||
orderLink |
metadata only |
metadata only |
||||||||||||||
polarizationChannels |
metadata only |
queryable metadata |
queryable metadata |
queryable metadata |
queryable metadata |
queryable metadata |
metadata only |
metadata only |
queryable metadata |
metadata only |
queryable metadata |
|||||
polarizationMode |
metadata only |
metadata only |
queryable metadata |
queryable metadata |
metadata only |
|||||||||||
productIdentifier |
metadata only |
metadata only |
||||||||||||||
quicklook |
metadata only |
metadata only |
metadata only |
metadata only |
metadata only |
metadata only |
metadata only |
metadata only |
metadata only |
metadata only |
metadata only |
|||||
searchLink |
metadata only |
|||||||||||||||
specification |
metadata only |
|||||||||||||||
storageStatus |
metadata only |
metadata only |
metadata only |
metadata only |
metadata only |
metadata only |
metadata only |
metadata only |
metadata only |
metadata only |
metadata only |
metadata only |
metadata only |
|||
thumbnail |
metadata only |
metadata only |
metadata only |
metadata only |
metadata only |
metadata only |
metadata only |
metadata only |
metadata only |
metadata only |
||||||
tileIdentifier |
queryable metadata |
queryable metadata |
queryable metadata |
queryable metadata |
queryable metadata |
queryable metadata |
queryable metadata |
queryable metadata |
||||||||
uid |
metadata only |
metadata only |
metadata only |
metadata only |
metadata only |
metadata only |
metadata only |
|||||||||
utmZone |
queryable metadata |