Add a Provider#

eodag provides a set of plugins which don’t know anything about the providers themselves, they just implement generic methods required to talk to different kinds of data catalog. For instance:

Configure a new provider#

The simplest way to add a new provider is to configure existing plugins. This approach requires to provide the new provider’s configuration in a YAML format. The following example, extracted from the STAC client page, shows how to add a new STAC provider:

tamn:
   search:
      type: StacSearch
      api_endpoint: https://tamn.snapplanet.io/search
      need_auth: false
   products:
      S2_MSI_L1C:
            productType: S2
      GENERIC_PRODUCT_TYPE:
            productType: '{productType}'
   download:
      type: AwsDownload
      base_uri: https://tamn.snapplanet.io
      flatten_top_dirs: True
   auth:
      type: AwsAuth
      credentials:
            aws_access_key_id: PLEASE_CHANGE_ME
            aws_secret_access_key: PLEASE_CHANGE_ME

It configures the following existing plugins: StacSearch (search), AwsAuth (authentication) and AwsDownload (download).

Of course, it is also necessary to know how to configure these plugins (which parameters they take, what values they can have, etc.). You can get some inspiration from the Providers pre-configuration section by analysing how eodag configures the providers it comes installed with.

Add more plugins#

eodag is a plugin-oriented framework which means it can be easily extended. If the plugins it offers are not sufficient for your own needs (i.e. getting data from a provider not supported by eodag), you should then write your own plugins (possibly by extending one of provided by eodag) and configure them. What you are the most likely to be willing to do is either to develop a new Search plugin or an Api plugin (e.g. to create an interface with another program).

eodag-sentinelsat is a good example of an Api plugin. It creates an interface with the sentinalsat library to search and download products from SciHub.

See more details about how to create a new plugin in this dedicated section.

Providers pre-configuration#

All the providers are pre-configured in eodag in a YAML file. Click on the link below to display its full content.

providers.yml
---
!provider
  name: usgs
  priority: 0
  description: U.S geological survey catalog for Landsat products
  roles:
    - host
  url: https://earthexplorer.usgs.gov/
  api: !plugin
    type: UsgsApi
    need_auth: true
    google_base_url: 'http://storage.googleapis.com/earthengine-public/landsat/'
    pagination:
      max_items_per_page: 5000
      total_items_nb_key_path: '$.totalHits'
    metadata_mapping:
      id: '$.entityId'
      geometry: '$.spatialBounds'
      productType: '$.productType'
      title: '$.displayId'
      abstract: '$.summary'
      cloudCover: '$.cloudCover'
      startTimeFromAscendingNode: '$.temporalCoverage.startDate'
      completionTimeFromAscendingNode: '$.temporalCoverage.endDate'
      quicklook: '$.browse[0].thumbnailPath'
      # storageStatus set to ONLINE for consistency between providers
      storageStatus: '{$.null#replace_str("Not Available","ONLINE")}'
      downloadLink: 'https://dds.cr.usgs.gov/{id}'
    extract: True
  products:
    # datasets list http://kapadia.github.io/usgs/_sources/reference/catalog/ee.txt
    L8_OLI_TIRS_C1L1:
      dataset: LANDSAT_8_C1
    GENERIC_PRODUCT_TYPE:
      dataset: '{productType}'

---
!provider
  name: aws_eos
  priority: 0
  description: EOS search for Amazon public datasets
  roles:
    - host
  url: https://developers.eos.com/datasets_description.html
  search: !plugin
    type: PostJsonSearch
    api_endpoint: 'https://gate.eos.com/api/lms/search/v2/{collection}?api_key={apikey}'
    need_auth: true
    auth_error_code: 403
    pagination:
      next_page_query_obj: '{{"limit":{items_per_page},"page":{page}}}'
      total_items_nb_key_path: '$.meta.found'
      # 2021/04/28: aws_eos doesn't specify a limit in its docs. It says that the default
      # value is 500 (https://doc.eos.com/search.api/#single-dataset-search).
      # Let's set it to this value for now
      max_items_per_page: 500
    query_params_key: 'search'
    discover_metadata:
      auto_discovery: true
      metadata_pattern: '^[a-zA-Z0-9_]+$'
      search_param: '{{{{"search":{{{{"{metadata}":"{{{metadata}}}" }}}} }}}}'
      metadata_path: '$.*'
    metadata_mapping:
      # landsat8_downloadLink : 's3://landsat-pds/c{storedInCollection}/L8/{path}/{row}/{productID}'
      geometry:
        - '{{"search":{{"shape": {geometry#to_geo_interface} }} }}'
        - '$.dataGeometry'
      # storageStatus set to ONLINE for consistency between providers
      storageStatus: '{$.null#replace_str("Not Available","ONLINE")}'
  products:
    CBERS4_PAN10M_L2:
      instrument: PAN10M
      collection: cbers4
      processingLevel: 2
      metadata_mapping:
        # OpenSearch Parameters for Collection Search (Table 3)
        productType: '$.null'
        platformSerialIdentifier:
          - '{{"search":{{"satelliteName":"{platformSerialIdentifier}" }} }}'
          - '$.satelliteName'
        instrument:
          - '{{"search":{{"sensor":"{instrument}" }} }}'
          - '$.sensor'
        processingLevel:
          - '{{"search":{{"processingLevel":"{processingLevel}" }} }}'
          - '$.processingLevel'
        # INSPIRE obligated OpenSearch Parameters for Collection Search (Table 4)
        title:
          - '{{"search":{{"sceneID":"{title}" }} }}'
          - '$.sceneID'
        # OpenSearch Parameters for Acquistion Parameters Search (Table 6)
        startTimeFromAscendingNode:
          - '{{"search":{{"date":{{"from":"{startTimeFromAscendingNode}"}} }} }}'
          - '$.date'
        completionTimeFromAscendingNode:
          - '{{"search":{{"date":{{"to":"{completionTimeFromAscendingNode}"}} }} }}'
          - '$.date'
        illuminationAzimuthAngle:
          - '{{"search":{{"sunAzimuth":"{illuminationAzimuthAngle}" }} }}'
          - '$.sunAzimuth'
        illuminationElevationAngle:
          - '{{"search":{{"sunElevation":"{illuminationElevationAngle}" }} }}'
          - '$.sunElevation'
        # Custom parameters (not defined in the base document referenced above)
        awsPath:
          - '{{"search":{{"downloadUrl":"{awsPath}" }} }}'
          - '$.downloadUrl'
        downloadLink: 's3://cbers-pds/{awsPath}'
        metaDownloadLink: 's3://cbers-meta-pds/{awsPath}'
        previewBaseName: '{$.sceneID#replace_str("_L2","")}'
        thumbnail: 'https://s3.amazonaws.com/cbers-meta-pds/{awsPath}/{previewBaseName}_small.jpeg'
        quicklook: 'https://s3.amazonaws.com/cbers-meta-pds/{awsPath}/{previewBaseName}.jpg'
        id:
          - '{{"search":{{"sceneID":"{id}" }} }}'
          - '{title}'
    CBERS4_PAN10M_L4:
      instrument: PAN10M
      collection: cbers4
      processingLevel: 4
      metadata_mapping_from_product: CBERS4_PAN10M_L2
      metadata_mapping:
        # Custom parameters (not defined in the base document referenced above)
        previewBaseName: '{$.sceneID#replace_str("_L4","")}'
        thumbnail: 'https://s3.amazonaws.com/cbers-meta-pds/{awsPath}/{previewBaseName}_small.jpeg'
        quicklook: 'https://s3.amazonaws.com/cbers-meta-pds/{awsPath}/{previewBaseName}.jpg'
    CBERS4_PAN5M_L2:
      instrument: PAN5M
      collection: cbers4
      processingLevel: 2
      metadata_mapping_from_product: CBERS4_PAN10M_L2
    CBERS4_PAN5M_L4:
      instrument: PAN5M
      collection: cbers4
      processingLevel: 4
      metadata_mapping_from_product: CBERS4_PAN10M_L2
      metadata_mapping:
        # Custom parameters (not defined in the base document referenced above)
        previewBaseName: '{$.sceneID#replace_str("_L4","")}'
        thumbnail: 'https://s3.amazonaws.com/cbers-meta-pds/{awsPath}/{previewBaseName}_small.jpeg'
        quicklook: 'https://s3.amazonaws.com/cbers-meta-pds/{awsPath}/{previewBaseName}.jpg'
    CBERS4_MUX_L2:
      instrument: MUX
      collection: cbers4
      processingLevel: 2
      metadata_mapping_from_product: CBERS4_PAN10M_L2
    CBERS4_MUX_L4:
      instrument: MUX
      collection: cbers4
      processingLevel: 4
      metadata_mapping_from_product: CBERS4_PAN10M_L2
      metadata_mapping:
        # Custom parameters (not defined in the base document referenced above)
        previewBaseName: '{$.sceneID#replace_str("_L4","")}'
        thumbnail: 'https://s3.amazonaws.com/cbers-meta-pds/{awsPath}/{previewBaseName}_small.jpeg'
        quicklook: 'https://s3.amazonaws.com/cbers-meta-pds/{awsPath}/{previewBaseName}.jpg'
    CBERS4_AWFI_L2:
      instrument: AWFI
      collection: cbers4
      processingLevel: 2
      metadata_mapping_from_product: CBERS4_PAN10M_L2
    CBERS4_AWFI_L4:
      instrument: AWFI
      collection: cbers4
      processingLevel: 4
      metadata_mapping_from_product: CBERS4_PAN10M_L2
      metadata_mapping:
        # Custom parameters (not defined in the base document referenced above)
        previewBaseName: '{$.sceneID#replace_str("_L4","")}'
        thumbnail: 'https://s3.amazonaws.com/cbers-meta-pds/{awsPath}/{previewBaseName}_small.jpeg'
        quicklook: 'https://s3.amazonaws.com/cbers-meta-pds/{awsPath}/{previewBaseName}.jpg'
    L8_OLI_TIRS_C1L1:
      collection: landsat8
      onAmazon: true
      metadata_mapping:
        # OpenSearch Parameters for Collection Search (Table 3)
        productType: '$.null'
        platform:
          - '{{"search":{{"satelliteName":"{platform}" }} }}'
          - '$.satelliteName'
        instrument:
          - '{{"search":{{"sensor":"{instrument}" }} }}'
          - '$.sensor'
        # INSPIRE obligated OpenSearch Parameters for Collection Search (Table 4)
        title:
          - '{{"search":{{"productID":"{title}" }} }}'
          - '$.productID'
        # OpenSearch Parameters for Product Search (Table 5)
        cloudCover:
          - '{{"search":{{"cloudCoverage":"{cloudCover}" }} }}'
          - '$.cloudCoverage'
        # OpenSearch Parameters for Acquistion Parameters Search (Table 6)
        availabilityTime:
          - '{{"search":{{"IndexingTime":{{"from":"{availabilityTime}"}} }} }}'
          - '$.IndexingTime'
        acquisitionStation:
          - '{{"search":{{"receivingStation":{{"from":"{acquisitionStation}"}} }} }}'
          - '$.receivingStation'
        startTimeFromAscendingNode:
          - '{{"search":{{"date":{{"from":"{startTimeFromAscendingNode}"}} }} }}'
          - '$.sceneStartTime'
        completionTimeFromAscendingNode:
          - '{{"search":{{"date":{{"to":"{completionTimeFromAscendingNode}"}} }} }}'
          - '$.sceneStopTime'
        illuminationAzimuthAngle:
          - '{{"search":{{"sunAzimuth":"{illuminationAzimuthAngle}" }} }}'
          - '$.sunAzimuth'
        illuminationElevationAngle:
          - '{{"search":{{"sunElevation":"{illuminationElevationAngle}" }} }}'
          - '$.sunElevation'
        # Custom parameters (not defined in the base document referenced above)
        onAmazon:
          - '{{"search":{{"onAmazon":"{onAmazon}" }} }}'
          - '$.onAmazon'
        path:
          - '{{"search":{{"path":"{path}" }} }}'
          - '$.path'
        row:
          - '{{"search":{{"row":"{row}" }} }}'
          - '$.row'
        downloadLink: 's3://landsat-pds/c1/L8/{path:03.0f}/{row:03.0f}/{title}/'
        thumbnail: '$.thumbnail'
        quicklook: 'https://landsat-pds.s3.amazonaws.com/c1/L8/{path:03.0f}/{row:03.0f}/{title}/{title}_thumb_large.jpg'
        id:
          - '{{"search":{{"productID":"{id}" }} }}'
          - '{title}'
    MODIS_MCD43A4:
      collection: modis
      metadata_mapping:
        # OpenSearch Parameters for Collection Search (Table 3)
        productType: '$.null'
        instrument:
          - '{{"search":{{"satelliteName":"{instrument}" }} }}'
          - '$.satelliteName'
        # INSPIRE obligated OpenSearch Parameters for Collection Search (Table 4)
        title:
          - '{{"search":{{"sceneID":"{title}" }} }}'
          - '$.sceneID'
        # OpenSearch Parameters for Product Search (Table 5)
        cloudCover:
          - '{{"search":{{"cloudCoverage":"{cloudCover}" }} }}'
          - '$.cloudCoverage'
        # OpenSearch Parameters for Acquistion Parameters Search (Table 6)
        startTimeFromAscendingNode:
          - '{{"search":{{"date":{{"from":"{startTimeFromAscendingNode}"}} }} }}'
          - '$.BeginningDateTime'
        completionTimeFromAscendingNode:
          - '{{"search":{{"date":{{"to":"{completionTimeFromAscendingNode}"}} }} }}'
          - '$.EndingDateTime'
        # Custom parameters (not defined in the base document referenced above)
        verticalTileNumber:
          - '{{"search":{{"verticalTileNumber":"{verticalTileNumber}" }} }}'
          - '$.verticalTileNumber'
        horizontalTileNumber:
          - '{{"search":{{"horizontalTileNumber":"{horizontalTileNumber}" }} }}'
          - '$.horizontalTileNumber'
        doyDate: '{$.sceneID#slice_str(9,16,1)}'
        downloadLink: 's3://modis-pds/MCD43A4.006/{horizontalTileNumber:02.0f}/{verticalTileNumber:02.0f}/{doyDate}/'
        thumbnail: '$.thumbnail'
        quicklook: '$.thumbnail'
        id:
          - '{{"search":{{"sceneID":"{id}" }} }}'
          - '{title}'
    NAIP:
      collection: naip
      metadata_mapping:
        # OpenSearch Parameters for Collection Search (Table 3)
        productType: '$.null'
        platformSerialIdentifier:
          - '{{"search":{{"satelliteName":"{platformSerialIdentifier}" }} }}'
          - '$.satelliteName'
        # INSPIRE obligated OpenSearch Parameters for Collection Search (Table 4)
        title:
          - '{{"search":{{"sceneID":"{title}" }} }}'
          - '$.sceneID'
        # OpenSearch Parameters for Acquistion Parameters Search (Table 6)
        startTimeFromAscendingNode:
          - '{{"search":{{"date":{{"from":"{startTimeFromAscendingNode}"}} }} }}'
          - '$.date'
        completionTimeFromAscendingNode:
          - '{{"search":{{"date":{{"to":"{completionTimeFromAscendingNode}"}} }} }}'
          - '$.date'
        # Custom parameters (not defined in the base document referenced above)
        awsPath:
          - '{{"search":{{"awsPath":"{awsPath}" }} }}'
          - '$.awsPath'
        downloadLink: 's3://naip-analytic/{awsPath}'
        id:
          - '{{"search":{{"sceneID":"{id}" }} }}'
          - '{title}'
    S1_SAR_GRD:
      productType: GRD
      collection: sentinel1
      metadata_mapping:
        cloudCover: '$.null'
        # OpenSearch Parameters for Collection Search (Table 3)
        platformSerialIdentifier:
          - '{{"search":{{"missionId":"{platformSerialIdentifier}" }} }}'
          - '$.missionId'
        # INSPIRE obligated OpenSearch Parameters for Collection Search (Table 4)
        title:
          - '{{"search":{{"sceneID":"{title}" }} }}'
          - '$.sceneID'
        # OpenSearch Parameters for Product Search (Table 5)
        orbitNumber:
          - '{{"search":{{"absoluteOrbitNumber":"{orbitNumber}" }} }}'
          - '$.absoluteOrbitNumber'
        orbitDirection:
          - '{{"search":{{"passDirection":"{orbitDirection}" }} }}'
          - '$.passDirection'
        sensorMode:
          - '{{"search":{{"mode":"{sensorMode}" }} }}'
          - '$.mode'
        # OpenSearch Parameters for Acquistion Parameters Search (Table 6)
        startTimeFromAscendingNode:
          - '{{"search":{{"date":{{"from":"{startTimeFromAscendingNode}"}} }} }}'
          - '$.date'
        completionTimeFromAscendingNode:
          - '{{"search":{{"date":{{"to":"{completionTimeFromAscendingNode}"}} }} }}'
          - '$.date'
        polarizationMode:
          - '{{"search":{{"polarization":"{polarizationMode}" }} }}'
          - '$.polarization'
        # Custom parameters (not defined in the base document referenced above)
        awsPath:
          - '{{"search":{{"awsPath":"{awsPath}" }} }}'
          - '$.awsPath'
        downloadLink: 's3://sentinel-s1-l1c/{awsPath}'
        thumbnail: 'https://render.eosda.com/S1/thumb/{title}.png'
        quicklook: 'https://render.eosda.com/S1/thumb/{title}.png'
        id:
          - '{{"search":{{"sceneID":"{id}" }} }}'
          - '{title}'
    S2_MSI_L1C:
      collection: sentinel2
      metadata_mapping:
        # OpenSearch Parameters for Collection Search (Table 3)
        productType: '$.null'
        platformSerialIdentifier:
          - '{{"search":{{"satelliteName":"{platformSerialIdentifier}" }} }}'
          - '$.satelliteName'
        # INSPIRE obligated OpenSearch Parameters for Collection Search (Table 4)
        title:
          - '{{"search":{{"productName":"{title}" }} }}'
          - '$.productName'
        # OpenSearch Parameters for Product Search (Table 5)
        cloudCover:
          - '{{"search":{{"cloudCoverage":"{cloudCover}" }} }}'
          - '$.cloudCoverage'
        # OpenSearch Parameters for Acquistion Parameters Search (Table 6)
        startTimeFromAscendingNode:
          - '{{"search":{{"date":{{"from":"{startTimeFromAscendingNode}"}} }} }}'
          - '$.timestamp'
        completionTimeFromAscendingNode:
          - '{{"search":{{"date":{{"to":"{completionTimeFromAscendingNode}"}} }} }}'
          - '$.timestamp'
        illuminationAzimuthAngle:
          - '{{"search":{{"azimuthAngle":"{illuminationAzimuthAngle}" }} }}'
          - '$.azimuthAngle'
        illuminationZenithAngle:
          - '{{"search":{{"zenithAngle":"{illuminationZenithAngle}" }} }}'
          - '$.zenithAngle'
        # Custom parameters (not defined in the base document referenced above)
        originalTileMeta: '{$.originalTileMeta#replace_str("sentinel-s2-l1c.s3.amazonaws.com","roda.sentinel-hub.com/sentinel-s2-l1c")}'
        thumbnail: '{$.thumbnail#replace_str("sentinel-s2-l1c.s3.amazonaws.com","roda.sentinel-hub.com/sentinel-s2-l1c")}'
        awsPath:
          - '{{"search":{{"awsPath":"{awsPath}" }} }}'
          - '$.awsPath'
        downloadLink: 's3://sentinel-s2-l1c/{awsPath}'
        productMetaLink: '{$.productMetaLink#replace_str("sentinel-s2-l1c.s3.amazonaws.com","roda.sentinel-hub.com/sentinel-s2-l1c")}'
        productPath:
          - '{{"search":{{"productPath":"{productPath}" }} }}'
          - '$.productPath'
        id:
          - '{{"search":{{"productName":"{id}" }} }}'
          - '{title}'
        processedL2A: '$.null'
        awsPathL2A: '$.null'
    S2_MSI_L2A:
      collection: sentinel2
      processedL2A: true
      # specific QueryStringSearch usage for these parameters (replaces current query)
      specific_qssearch:
        parameters:
          - title
          - id
        results_entry: ''
        collection:
          - tileInfo
          - productInfo
        merge_responses: true
        metadata_mapping:
          title:
            - 'title'
            - '$.name'
          id:
            - 'title'
            - '{title}'
          awsPathL2A: '$.tiles[0].path'
          downloadLink: 's3://sentinel-s2-l2a/{awsPathL2A}'
          productPath: '$.path'
          startTimeFromAscendingNode: '$.timestamp'
          completionTimeFromAscendingNode: '$.timestamp'
          geometry: '$.tileDataGeometry'
          productInfo: 'https://roda.sentinel-hub.com/sentinel-s2-l2a/{awsPathL2A}/productInfo.json'
          originalSceneID: '$.tiles[0].datastrip.id'
      metadata_mapping:
        # OpenSearch Parameters for Collection Search (Table 3)
        productType: '$.null'
        platformSerialIdentifier:
          - '{{"search":{{"satelliteName":"{platformSerialIdentifier}" }} }}'
          - '$.satelliteName'
        # INSPIRE obligated OpenSearch Parameters for Collection Search (Table 4)
        title: '{$.productName#fake_l2a_title_from_l1c}'
        # OpenSearch Parameters for Product Search (Table 5)
        cloudCover:
          - '{{"search":{{"cloudCoverage":"{cloudCover}" }} }}'
          - '$.cloudCoverage'
        # OpenSearch Parameters for Acquistion Parameters Search (Table 6)
        startTimeFromAscendingNode:
          - '{{"search":{{"date":{{"from":"{startTimeFromAscendingNode}"}} }} }}'
          - '$.timestamp'
        completionTimeFromAscendingNode:
          - '{{"search":{{"date":{{"to":"{completionTimeFromAscendingNode}"}} }} }}'
          - '$.timestamp'
        illuminationAzimuthAngle:
          - '{{"search":{{"azimuthAngle":"{illuminationAzimuthAngle}" }} }}'
          - '$.azimuthAngle'
        illuminationZenithAngle:
          - '{{"search":{{"zenithAngle":"{illuminationZenithAngle}" }} }}'
          - '$.zenithAngle'
        # Custom parameters (not defined in the base document referenced above)
        originalTileMeta: '{$.originalTileMeta#replace_str("sentinel-s2-l1c.s3.amazonaws.com","roda.sentinel-hub.com/sentinel-s2-l2a")}'
        thumbnail: '{$.thumbnail#replace_str("sentinel-s2-l1c.s3.amazonaws.com","roda.sentinel-hub.com/sentinel-s2-l1c")}'
        quicklook: '{thumbnail}'
        downloadLink: 's3://sentinel-s2-l2a/{awsPathL2A}'
        awsPath: '$.null'
        productMetaLink: '$.null'
        productPath: '$.null'
        productInfo: 'https://roda.sentinel-hub.com/sentinel-s2-l2a/{awsPathL2A}/productInfo.json'
        id:
          - '{id#s2msil2a_title_to_aws_productinfo}'
          - '{title}'
        processedL2A:
          - '{{"search":{{"processedL2A":"{processedL2A}" }} }}'
          - '$.processedL2A'
        awsPathL2A:
          - '{{"search":{{"awsPathL2A":"{awsPathL2A}" }} }}'
          - '$.awsPathL2A'

  download: !plugin
    type: AwsDownload
    products:
      CBERS4_MUX_L2:
        default_bucket: 'cbers-pds'
        complementary_url_key:
          - metaDownloadLink
      CBERS4_AWFI_L2:
        complementary_url_key:
          - metaDownloadLink
      CBERS4_PAN5M_L2:
        complementary_url_key:
          - metaDownloadLink
      CBERS4_PAN10M_L2:
        complementary_url_key:
          - metaDownloadLink
      S1_SAR_GRD:
        default_bucket: 'sentinel-s1-l1c'
        build_safe: true
      S2_MSI_L1C:
        default_bucket: 'sentinel-s2-l1c'
        build_safe: true
        complementary_url_key:
          - productPath
      S2_MSI_L2A:
        default_bucket: 'sentinel-s2-l2a'
        build_safe: true
        fetch_metadata:
          fetch_url: '{productInfo}'
          fetch_format: json
          update_metadata:
            title: '$.name'
            id: '{title}'
            productPath: '$.path'
        complementary_url_key:
          - productPath
  auth: !plugin
    type: AwsAuth

---
!provider
  name: theia
  priority: 0
  description: French National Space Agency (CNES) catalog for Sentinel 2 products, Pleiades and Landsat products
  roles:
    - host
  url: https://theia.cnes.fr/atdistrib/rocket/
  search: !plugin
    type: QueryStringSearch
    api_endpoint: 'https://theia.cnes.fr/atdistrib/resto2/api/collections/{collection}/search.json'
    need_auth: false
    pagination:
      next_page_url_tpl: '{url}?{search}&maxRecords={items_per_page}&page={page}'
      total_items_nb_key_path: '$.properties.totalResults'
      # 2021/03/19: Returns a 400 error code if greater than 500.
      max_items_per_page: 500
    discover_metadata:
      auto_discovery: true
      metadata_pattern: '^(?!collection)[a-zA-Z0-9_]+$'
      search_param: '{metadata}={{{metadata}}}'
      metadata_path: '$.properties.*'
    metadata_mapping:
      # Opensearch resource identifier within the search engine context (in our case
      # within the context of the data provider)
      uid: '$.id'
      # OpenSearch Parameters for Collection Search (Table 3)
      productType:
        - productType
        - '$.properties.productType'
      platform:
        - platform
        - '$.properties.platform'
      platformSerialIdentifier:
        - 'platform={platformSerialIdentifier}'
        - '$.properties.platform'
      instrument:
        - instrument
        - '$.properties.instrument'
      processingLevel:
        - processingLevel
        - '$.properties.processingLevel'

      # INSPIRE obligated OpenSearch Parameters for Collection Search (Table 4)
      title: '$.properties.title'
      keyword: '$.properties.keywords'
      resolution: '$.properties.resolution'
      organisationName: '$.properties.organisationName'
      publicationDate: '$.properties.published'
      accessConstraint: '$.properties.license'

      # OpenSearch Parameters for Product Search (Table 5)
      parentIdentifier: '$.properties.parentIdentifier'
      orbitNumber: '$.properties.orbitNumber'
      cloudCover:
        - 'cloudCover=[0,{cloudCover}]'
        - '$.properties.cloudCover'
      snowCover: '$.properties.snowCover'
      productVersion: '$.properties.version'
      creationDate: '$.properties.productionDate'
      modificationDate: '$.properties.modificationDate'
      processingDate: '$.properties.processingDate'
      sensorMode: '$.properties.sensorMode'
      archivingCenter: '$.properties.archivingCenter'
      processingMode: '$.properties.processingMode'

      # OpenSearch Parameters for Acquistion Parameters Search (Table 6)
      availabilityTime: '$.properties.availabilityTime'
      acquisitionStation: '$.properties.acquisitionStation'
      acquisitionSubType: '$.properties.acquisitionSubType'
      startTimeFromAscendingNode:
        - 'startDate={startTimeFromAscendingNode#to_iso_date}'
        - '$.properties.startDate'
      completionTimeFromAscendingNode:
        - 'completionDate={completionTimeFromAscendingNode#to_iso_date}'
        - '$.properties.completionDate'
      illuminationAzimuthAngle: '$.properties.illuminationAzimuthAngle'
      illuminationZenithAngle: '$.properties.illuminationZenithAngle'
      illuminationElevationAngle: '$.properties.illuminationElevationAngle'
      polarizationMode: '$.properties.polarizationMode'
      polarizationChannels: '$.properties.polarisationChannels'
      antennaLookDirection: '$.properties.antennaLookDirection'
      minimumIncidenceAngle: '$.properties.minimumIncidenceAngle'
      maximumIncidenceAngle: '$.properties.maximumIncidenceAngle'
      dopplerFrequency: '$.properties.dopplerFrequency'
      incidenceAngleVariation: '$.properties.incidenceAngleVariation'

      # Custom parameters (not defined in the base document referenced above)
      id: '$.properties.productIdentifier'
      # The geographic extent of the product
      geometry:
        - 'geometry={geometry#to_rounded_wkt}'
        - '$.geometry'
      # The url of the quicklook
      quicklook: '$.properties.quicklook'
      # The url to download the product "as is" (literal or as a template to be completed either after the search result
      # is obtained from the provider or during the eodag download phase)
      downloadLink: '$.properties.services.download.url'

      # Additional metadata provided by the providers but that don't appear in the reference spec
      # Or has a different signification for the provider
      thumbnail: '$.properties.thumbnail'

      # storageStatus set to ONLINE for consistency between providers
      storageStatus: '{$.null#replace_str("Not Available","ONLINE")}'

  products:
    S2_MSI_L2A_MAJA:
      processingLevel: LEVEL2A
      collection: SENTINEL2
    S2_MSI_L2B_MAJA_SNOW:
      processingLevel: L2B-SNOW
      collection: Snow
    S2_MSI_L2B_MAJA_WATER:
      processingLevel: L2B-WATER
      collection: WaterQual
    S2_MSI_L3A_WASP:
      processingLevel: LEVEL3A
      collection: SENTINEL2
    SPOT_SWH:
      collection: SPOTWORLDHERITAGE
      processingLevel: LEVEL1C
    SPOT_SWH_OLD:
      collection: SWH1
      processingLevel: LEVEL1C
    SPOT5_SPIRIT:
      collection: Spirit
      processingLevel: 1A
    VENUS_L1C:
      collection: VENUS
      processingLevel: LEVEL1C
    VENUS_L2A_MAJA:
      collection: VENUS
      processingLevel: LEVEL2A
    VENUS_L3A_MAJA:
      collection: VENUS
      processingLevel: LEVEL3A
    L8_REFLECTANCE:
      collection: LANDSAT
      productType: REFLECTANCE
    L57_REFLECTANCE:
      collection: Landsat57
      productType: REFLECTANCE
    PLD_PAN:
      collection: Pleiades
      productType: Panchromatique+(Pan)
    PLD_XS:
      collection: Pleiades
      productType: Multispectral+(XS)
    PLD_BUNDLE:
      collection: Pleiades
      productType: Bundle+(Pan,+XS)
    PLD_PANSHARPENED:
      collection: Pleiades
      productType: Pansharpened+(Pan%2BXS)
    OSO:
      collection: OSO
    GENERIC_PRODUCT_TYPE:
      productType: '{productType}'
      collection: '{collection}'
      processingLevel: '{processingLevel}'
  download: !plugin
    type: HTTPDownload
    base_uri: 'https://theia.cnes.fr/atdistrib/resto2'
    extract: true
    order_enabled: true
    auth_error_code: 403
    dl_url_params:
      issuerId: theia
  auth: !plugin
    type: TokenAuth
    auth_uri: 'https://theia.cnes.fr/atdistrib/services/authenticate/'

---
!provider
  name: peps
  priority: 1
  description: |
    The PEPS platform, the French "mirror site", redistributes the products of Sentinel satellites,
    S1A, S1B, S2A and S2B, S3A and S3B from COPERNICUS, the European system for the Earth monitoring.
  roles:
    - host
  url: https://peps.cnes.fr
  search: !plugin
    type: QueryStringSearch
    api_endpoint: 'https://peps.cnes.fr/resto/api/collections/{collection}/search.json'
    need_auth: false
    pagination:
      next_page_url_tpl: '{url}?{search}&maxRecords={items_per_page}&page={page}'
      total_items_nb_key_path: '$.properties.totalResults'
      # 2021/03/19: 500 is the max, no error if greater
      max_items_per_page: 500
    discover_metadata:
      auto_discovery: true
      metadata_pattern: '^(?!collection)[a-zA-Z0-9_]+$'
      search_param: '{metadata}={{{metadata}}}'
      metadata_path: '$.properties.*'
    metadata_mapping:
      # Opensearch resource identifier within the search engine context (in our case
      # within the context of the data provider)
      uid: '$.id'
      # OpenSearch Parameters for Collection Search (Table 3)
      productType:
        - productType
        - '$.properties.productType'
      platform: '$.properties.collection'
      platformSerialIdentifier:
        - platform
        - '$.properties.platform'
      instrument:
        - instrument
        - '$.properties.instrument'
      processingLevel:
        - processingLevel
        - '$.properties.processingLevel'

      # INSPIRE obligated OpenSearch Parameters for Collection Search (Table 4)
      title: '$.properties.title'
      topicCategory: '$.properties.topicCategory'
      keyword: '$.properties.keywords'
      abstract: '$.properties.description'
      resolution:
        - 'resolution'
        - '$.properties.resolution'
      organisationName:
        - 'organisationName'
        - '$.properties.organisationName'
      publicationDate: '$.properties.published'
      accessConstraint: '$.properties.license'

      # OpenSearch Parameters for Product Search (Table 5)
      parentIdentifier:
        - 'parentIdentifier'
        - '$.properties.parentIdentifier'
      acquisitionType: '$.properties.acquisitionType'
      orbitNumber:
        - 'orbitNumber'
        - '$.properties.orbitNumber'
      orbitDirection:
        - 'orbitDirection'
        - '$.properties.orbitDirection'
      swathIdentifier:
        - 'swath'
        - '$.properties.swath'
      cloudCover:
        - 'cloudCover=[0,{cloudCover}]'
        - '$.properties.cloudCover'
      snowCover:
        - 'snowCover=[0,{snowCover}]'
        - '$.properties.snowCover'
      productVersion: '$.properties.version'
      productQualityStatus: '$.properties.onlineQualityCheck'
      processorName: '$.properties.processingName'
      processingCenter: '$.properties.processingCenter'
      creationDate: '$.properties.dhusIngestDate'
      modificationDate: '$.properties.updated'
      sensorMode:
        - 'sensorMode'
        - '$.properties.sensorMode'

      # OpenSearch Parameters for Acquistion Parameters Search (Table 6)
      startTimeFromAscendingNode:
        - startDate
        - '$.properties.startDate'
      completionTimeFromAscendingNode:
        - completionDate
        - '$.properties.completionDate'
      polarizationMode:
        - 'polarisation'
        - '$.properties.polarisation'

      # Custom parameters (not defined in the base document referenced above)
      id:
        - 'productIdentifier={id#remove_extension}'
        - '$.properties.productIdentifier'
      # The geographic extent of the product
      geometry:
        - 'geometry={geometry#to_rounded_wkt}'
        - '$.geometry'
      # The url of the quicklook
      quicklook: '$.properties.quicklook'
      # The url to download the product "as is" (literal or as a template to be completed either after the search result
      # is obtained from the provider or during the eodag download phase)
      downloadLink: '$.properties.services.download.url'
      # storageStatus: must be one of ONLINE, STAGING, OFFLINE
      storageStatus: '{$.properties.storage.mode#get_group_name((?P<ONLINE>disk)|(?P<STAGING>staging)|(?P<OFFLINE>tape))}'

      # Additional metadata provided by the providers but that don't appear in the reference spec
      thumbnail: '$.properties.thumbnail'

  products:
    S1_SAR_OCN:
      productType: OCN
      collection: S1
      metadata_mapping:
        cloudCover: '$.null'
    S1_SAR_GRD:
      productType: GRD
      collection: S1
      metadata_mapping:
        cloudCover: '$.null'
    S1_SAR_SLC:
      productType: SLC
      collection: S1
      metadata_mapping:
        cloudCover: '$.null'
    S2_MSI_L1C:
      collection: S2ST
      productType: S2MSI1C
    S2_MSI_L2A:
      collection: S2ST
      productType: S2MSI2A
    GENERIC_PRODUCT_TYPE:
      productType: '{productType}'
      collection: '{collection}'
  download: !plugin
    type: HTTPDownload
    base_uri: 'https://peps.cnes.fr/resto'
    extract: true
    archive_depth: 2
    order_enabled: true
    auth_error_code: 401
    dl_url_params:
      issuerId: peps
  auth: !plugin
    type: GenericAuth
    auth_uri: 'https://peps.cnes.fr/resto/api/users/connect'

---
!provider
  name: sobloo
  priority: 0
  description: Airbus DIAS
  roles:
    - host
  url: https://sobloo.eu/
  products:
    S2_MSI_L1C:
      productType: S2MSI1C
    S2_MSI_L2A:
      productType: S2MSI2A
    S1_SAR_RAW:
      productType: RAW
      metadata_mapping:
        cloudCover: '$.null'
    S1_SAR_GRD:
      productType: GRD
      metadata_mapping:
        cloudCover: '$.null'
    S1_SAR_SLC:
      productType: SLC
      metadata_mapping:
        cloudCover: '$.null'
    S1_SAR_OCN:
      productType: OCN
      metadata_mapping:
        cloudCover: '$.null'
    GENERIC_PRODUCT_TYPE:
      productType: '{productType}'
    # S3_LAN:
    #   productType: SR_2_LAN___
    # S3_SRA:
    #   productType: SR_1_SRA___
    # S3_SRA_BS:
    #   productType: SR_1_SRA_BS
    # S3_SRA_A_BS:
    #   productType: SR_1_SRA_A_
  search: !plugin
    type: QueryStringSearch
    api_endpoint: 'https://sobloo.eu/api/v1/services/search'
    need_auth: false
    collection: catalog
    result_type: 'json'
    pagination:
      next_page_url_tpl: '{url}?{search}&size={items_per_page}&from={skip}'
      total_items_nb_key_path: '$.totalnb'
      # 2021/03/19: It's not possible to get more than 10_000 products
      # Returns a 400 Bad Request if more products are requested.
      max_items_per_page: 10_000
    results_entry: 'hits'
    discover_metadata:
      auto_discovery: true
      metadata_pattern: '^[a-zA-Z0-9\.%]+$'
      search_param: 'f={metadata}:eq:{{{metadata}}}'
      metadata_path: '$.data.*'
    metadata_mapping:
      # Opensearch resource identifier within the search engine context (in our case
      # within the context of the data provider)
      uid: '$.data.uid'
      # OpenSearch Parameters for Collection Search (Table 3)
      productType:
        - 'f=identification.type:eq:{productType}'
        - '$.data.identification.type'
      platform: '$.data.acquisition.mission'
      platformSerialIdentifier: '$.data.acquisition.missionCode'
      instrument: '$.data.acquisition.sensorId'
      processingLevel: '$.data.production.levelCode'

      # INSPIRE obligated OpenSearch Parameters for Collection Search (Table 4)
      title: '$.data.identification.externalId'
      publicationDate: '{$.data.timestamp#to_iso_utc_datetime_from_milliseconds}'

      # OpenSearch Parameters for Product Search (Table 5)
      orbitNumber: '$.data.orbit.number'
      orbitDirection: '$.data.orbit.direction'
      cloudCover:
        - 'f=contentDescription.cloudCoverPercentage:lte:{cloudCover}'
        - '$.data.contentDescription.cloudCoverPercentage'
      creationDate: '{$.data.state.insertionDate#to_iso_utc_datetime_from_milliseconds}'

      # OpenSearch Parameters for Acquistion Parameters Search (Table 6)
      availabilityTime: '$.data.availabilityTime'
      startTimeFromAscendingNode:
        - 'f=acquisition.beginViewingDate:gte:{startTimeFromAscendingNode#datetime_to_timestamp_milliseconds}'
        - '{$.data.acquisition.beginViewingDate#to_iso_utc_datetime_from_milliseconds}'
      completionTimeFromAscendingNode:
        - 'f=acquisition.endViewingDate:lte:{completionTimeFromAscendingNode#datetime_to_timestamp_milliseconds}'
        - '{$.data.acquisition.endViewingDate#to_iso_utc_datetime_from_milliseconds}'
      polarizationChannels: '$.data.acquisition.polarization'
      sensorMode: '$.data.acquisition.sensorMode'

      # Custom parameters (not defined in the base document referenced above)
      id:
        - 'f=identification.externalId:eq:{id#remove_extension}'
        - '$.data.identification.externalId'
      # The geographic extent of the product
      geometry:
        - 'gintersect={geometry#to_rounded_wkt}'
        - '$.md.geometry'
      # The url of the quicklook
      quicklook: 'https://sobloo.eu/api/v1/services/quicklook/{uid}'
      # The url to download the product "as is" (literal or as a template to be completed either after the search result
      # is obtained from the provider or during the eodag download phase)
      downloadLink: '%(base_uri)s/{uid}'
      # storageStatus: must be one of ONLINE, STAGING, OFFLINE
      storageStatus: '{$.data.state.services.download#get_group_name((?P<ONLINE>internal)|(?P<OFFLINE>external))}'

  download: !plugin
    type: HTTPDownload
    base_uri: 'https://sobloo.eu/api/v1/services/download'
    archive_depth: 2
    order_enabled: true
    auth_error_code:
      - 401
      - 403
      - 500
    extract: true
  auth: !plugin
    type: HTTPHeaderAuth
    headers:
      Authorization: "Apikey {apikey}"

---
!provider
  name: creodias
  priority: 0
  description: CloudFerro DIAS
  roles:
    - host
  url: https://creodias.eu/
  search: !plugin
    type: QueryStringSearch
    api_endpoint: 'https://finder.creodias.eu/resto/api/collections/{collection}/search.json'
    need_auth: false
    pagination:
      next_page_url_tpl: '{url}?{search}&maxRecords={items_per_page}&page={page}'
      total_items_nb_key_path: '$.properties.totalResults'
      # 2021/03/19: 2000 is the max, 400 error if greater
      max_items_per_page: 2_000
    discover_metadata:
      auto_discovery: true
      metadata_pattern: '^(?!collection)[a-zA-Z0-9]+$'
      search_param: '{metadata}={{{metadata}}}'
      metadata_path: '$.properties.*'
    metadata_mapping:
      # Opensearch resource identifier within the search engine context (in our case
      # within the context of the data provider)
      uid: '$.id'
      # OpenSearch Parameters for Collection Search (Table 3)
      productType:
        - productType
        - '$.properties.productType'
      platform: '$.properties.collection'
      platformSerialIdentifier:
        - platform
        - '$.properties.platform'
      instrument:
        - instrument
        - '$.properties.instrument'
      processingLevel:
        - processingLevel
        - '$.properties.processingLevel'

      # INSPIRE obligated OpenSearch Parameters for Collection Search (Table 4)
      title: '$.properties.title'
      keyword: '$.properties.keywords'
      abstract: '$.properties.description'
      resolution:
        - resolution
        - '$.properties.resolution'
      organisationName:
        - organisationName
        - '$.properties.organisationName'
      publicationDate: '$.properties.published'
      accessConstraint: '$.properties.license'

      # OpenSearch Parameters for Product Search (Table 5)
      parentIdentifier:
        - parentIdentifier
        - '$.properties.parentIdentifier'
      orbitNumber:
        - orbitNumber
        - '$.properties.orbitNumber'
      orbitDirection:
        - orbitDirection
        - '$.properties.orbitDirection'
      cloudCover:
        - 'cloudCover=[0,{cloudCover}]'
        - '$.properties.cloudCover'
      snowCover:
        - snowCover
        - '$.properties.snowCover'
      modificationDate: '$.properties.updated'
      sensorMode:
        - sensorMode
        - '$.properties.sensorMode'

      # OpenSearch Parameters for Acquistion Parameters Search (Table 6)
      startTimeFromAscendingNode:
        - startDate
        - '$.properties.startDate'
      completionTimeFromAscendingNode:
        - completionDate
        - '$.properties.completionDate'
      polarizationChannels: '$.properties.polarisation'

      # Custom parameters (not defined in the base document referenced above)
      id:
        - 'productIdentifier=%{id#remove_extension}%'
        - '{$.properties.title#remove_extension}'
      # The geographic extent of the product
      geometry:
        - 'geometry={geometry#to_rounded_wkt}'
        - '$.geometry'
      # The url of the quicklook
      quicklook: '{thumbnail}'
      # The url to download the product "as is" (literal or as a template to be completed either after the search result
      # is obtained from the provider or during the eodag download phase)
      downloadLink: 'https://zipper.creodias.eu/download/{uid}'
      # storageStatus: must be one of ONLINE, STAGING, OFFLINE
      storageStatus: '{$.properties.status#get_group_name((?P<ONLINE>(0|34|37))|(?P<STAGING>32)|(?P<OFFLINE>31))}'

      # Additional metadata provided by the providers but that don't appear in the reference spec
      thumbnail: '$.properties.thumbnail'
      cultivatedCover:
        - cultivatedCover
        - '$.properties.cultivatedCover'
      desertCover:
        - desertCover
        - '$.properties.desertCover'
      floodedCover:
        - floodedCover
        - '$.properties.floodedCover'
      forestCover:
        - forestCover
        - '$.properties.forestCover'
      herbaceousCover:
        - herbaceousCover
        - '$.properties.herbaceousCover'
      iceCover:
        - iceCover
        - '$.properties.iceCover'
      urbanCover:
        - urbanCover
        - '$.properties.urbanCover'
      waterCover:
        - waterCover
        - '$.properties.waterCover'
      processingBaseline:
        - processingBaseline
        - '$.properties.processingBaseline'
      publishedAfter:
        - publishedAfter
        - '$.properties.publishedAfter'
      publishedBefore:
        - publishedBefore
        - '$.properties.publishedBefore'
      relativeOrbitNumber:
        - relativeOrbitNumber
        - '$.properties.relativeOrbitNumber'
      relativeOrbitNumber:
        - relativeOrbitNumber
        - '$.properties.relativeOrbitNumber'
      sortOrder:
        - sortOrder
        - '$.properties.sortOrder'
      sortParam:
        - sortParam
        - '$.properties.sortParam'
      status:
        - status
        - '$.properties.status'
      timeliness:
        - timeliness
        - '$.properties.timeliness'
  download: !plugin
    type: HTTPDownload
    base_uri: 'https://zipper.creodias.eu/download/'
    extract: true
    order_enabled: false
    archive_depth: 2
  auth: !plugin
    type: KeycloakOIDCPasswordAuth
    auth_base_uri: 'https://auth.creodias.eu/auth'
    realm: 'dias'
    client_id: 'CLOUDFERRO_PARTNERS'
    client_secret: 'dc0aca03-2dc6-4798-a5de-fc5aeb6c8ee1'
    token_provision: qs
    token_qs_key: 'token'
    auth_error_code: 401
  products:
    S2_MSI_L1C:
      productType: L1C
      collection: Sentinel2
    S2_MSI_L2A:
      productType: L2A
      collection: Sentinel2
    S1_SAR_RAW:
      productType: RAW
      collection: Sentinel1
      metadata_mapping:
        cloudCover: '$.null'
    S1_SAR_GRD:
      productType: GRD
      collection: Sentinel1
      metadata_mapping:
        cloudCover: '$.null'
    S1_SAR_SLC:
      productType: SLC
      collection: Sentinel1
      metadata_mapping:
        cloudCover: '$.null'
    S1_SAR_OCN:
      productType: OCN
      collection: Sentinel1
      metadata_mapping:
        cloudCover: '$.null'
    S3_LAN:
      productType: LAN
      collection: Sentinel3
      metadata_mapping:
        cloudCover: '$.null'
    S3_SRA:
      productType: SRA
      collection: Sentinel3
      metadata_mapping:
        cloudCover: '$.null'
    S3_SRA_BS:
      productType: SRA_BS
      collection: Sentinel3
      metadata_mapping:
        cloudCover: '$.null'
    S3_SRA_A_BS:
      productType: SRA_A
      collection: Sentinel3
      metadata_mapping:
        cloudCover: '$.null'
    S3_EFR:
      productType: EFR
      collection: Sentinel3
    S3_ERR:
      productType: ERR
      collection: Sentinel3
    S3_WAT:
      productType: WAT
      collection: Sentinel3
    S3_SLSTR_L2LST:
      productType: LST
      collection: Sentinel3
    S3_OLCI_L2LRR:
      productType: LRR
      collection: Sentinel3
    S3_OLCI_L2LFR:
      productType: LFR
      collection: Sentinel3
    S3_SLSTR_L1RBT:
      productType: RBT
      collection: Sentinel3
    GENERIC_PRODUCT_TYPE:
      productType: '{productType}'
      collection: '{collection}'
---
!provider
  name: mundi
  priority: 0
  description: Atos DIAS
  roles:
    - host
  url: https://mundiwebservices.com/
  search: !plugin
    type: QueryStringSearch
    api_endpoint: 'https://mundiwebservices.com/acdc/catalog/proxy/search/{collection}/opensearch'
    need_auth: false
    result_type: 'xml'
    results_entry: '//ns:entry'
    literal_search_params:
      format: atom
      relation: intersects
    pagination:
      next_page_url_tpl: '{url}?{search}&maxRecords={items_per_page}&startIndex={skip_base_1}'
      total_items_nb_key_path: '//os:totalResults/text()'
      # 2021/03/19: 50 is the max, no error if greater
      max_items_per_page: 50
    discover_metadata:
      auto_discovery: true
      metadata_pattern: '^(?!collection)[a-zA-Z0-9]+$'
      search_param: '{metadata}={{{metadata}}}'
      metadata_path: '*'
    metadata_mapping:
      # Opensearch resource identifier within the search engine context (in our case
      # within the context of the data provider)
      uid: 'ns:id/text()'
      # OpenSearch Parameters for Collection Search (Table 3)
      productType:
        - 'productType'
        - 'eo:productType/text()'
      doi: 'eo:doi/text()'
      platform: 'eo:platform/text()'
      platformSerialIdentifier: 'eo:platformSerialIdentifier/text()'
      instrument:
        - 'instrument'
        - 'eo:instrument/text()'
      sensorType: 'eo:sensorType/text()'
      compositeType: 'eo:compositeType/text()'
      processingLevel:
        - 'processingLevel'
        - 'eo:processingLevel/text()'
      orbitType: 'eo:orbitType/text()'
      spectralRange: 'eo:spectralRange/text()'
      wavelengths: 'eo:wavelengths/text()'
      hasSecurityConstraints: 'eo:hasSecurityConstraints/text()'
      dissemination: 'eo:dissemination/text()'

      # INSPIRE obligated OpenSearch Parameters for Collection Search (Table 4)
      title: 'ns:title/text()'
      topicCategory: 'ns:category/text()'
      keyword: 'eo:keyword/text()'
      abstract: 'ns:summary/text()'
      resolution: 'eo:resolution/text()'
      organisationName: 'dc:creator/text()'
      organisationRole: 'eo:organisationRole/text()'
      publicationDate: 'ns:published/text()'
      lineage: 'eo:lineage/text()'
      useLimitation: 'eo:useLimitation/text()'
      accessConstraint: 'eo:accessConstraint/text()'
      otherConstraint: 'dc:rights/text()'
      classification: 'DIAS:productConfidentialityLevel/text()'
      language: 'eo:language/text()'
      specification: 'eo:specification/text()'

      # OpenSearch Parameters for Product Search (Table 5)
      parentIdentifier: 'eo:parentIdentifier/text()'
      acquisitionType: 'eo:acquisitionType/text()'
      orbitNumber: 'eo:orbitNumber/text()'
      orbitDirection: 'eo:orbitDirection/text()'
      track: 'eo:track/text()'
      frame: 'eo:frame/text()'
      swathIdentifier: 'eo:swathIdentifier/text()'
      cloudCover:
        - 'cloudCover=[0,{cloudCover}]'
        - 'eo:cloudCover/text()'
      snowCover: 'eo:snowCover/text()'
      lowestLocation: 'eo:lowestLocation/text()'
      highestLocation: 'eo:highestLocation/text()'
      productVersion: 'DIAS:processingBaseline/text()'
      productQualityStatus: 'eo:productQualityStatus/text()'
      productQualityDegradationTag: 'eo:productQualityDegradationTag/text()'
      processorName: 'eo:processorName/text()'
      processingCenter: 'eo:processingCenter/text()'
      creationDate: 'DIAS:onlineDate/text()'
      modificationDate: 'ns:updated/text()'
      processingDate: 'eo:processingDate/text()'
      sensorMode: 'eo:sensorMode/text()'
      archivingCenter: 'eo:archivingCenter/text()'
      processingMode: 'eo:processingMode/text()'

      # OpenSearch Parameters for Acquistion Parameters Search (Table 6)
      availabilityTime: 'DIAS:onlineDate/text()'
      acquisitionStation: 'eo:acquisitionStation/text()'
      acquisitionSubType: 'eo:acquisitionSubType/text()'
      startTimeFromAscendingNode:
        - 'timeStart={startTimeFromAscendingNode#to_iso_utc_datetime}'
        - 'DIAS:sensingStartDate/text()'
      completionTimeFromAscendingNode:
        - 'timeEnd={completionTimeFromAscendingNode#to_iso_utc_datetime}'
        - 'DIAS:sensingStopDate/text()'
      illuminationAzimuthAngle: 'eo:illuminationAzimuthAngle/text()'
      illuminationZenithAngle: 'eo:illuminationZenithAngle/text()'
      illuminationElevationAngle: 'eo:illuminationElevationAngle/text()'
      polarizationMode: 'eo:polarizationMode/text()'
      polarizationChannels: '{eo:polarisationChannels/text()#replace_str("/"," ")}'
      antennaLookDirection: 'eo:antennaLookDirection/text()'
      minimumIncidenceAngle: 'eo:minimumIncidenceAngle/text()'
      maximumIncidenceAngle: 'eo:maximumIncidenceAngle/text()'
      dopplerFrequency: 'eo:dopplerFrequency/text()'
      incidenceAngleVariation: 'eo:incidenceAngleVariation/text()'

      # Custom parameters (not defined in the base document referenced above)
      id:
        - 'uid={id#remove_extension}'
        - 'dc:identifier/text()'
      # The geographic extent of the product
      geometry:
        - 'geometry={geometry#to_rounded_wkt}'
        - 'georss:box/text()'
      # The url of the quicklook
      quicklook: 'media:group/media:content[media:category="QUICKLOOK"]/@url'
      # The url to download the product "as is" (literal or as a template to be completed either after the search result
      # is obtained from the provider or during the eodag download phase)
      downloadLink: 'ns:link[@rel="enclosure"]/@href'
      # storageStatus: must be one of ONLINE, STAGING, OFFLINE
      storageStatus: 'DIAS:onlineStatus/text()'

      # Additional metadata provided by the providers but that don't appear in the reference spec
      thumbnail: 'media:group/media:content[media:category="THUMBNAIL"]/@url'
  download: !plugin
    type: S3RestDownload
    base_uri: 'https://mundiwebservices.com/dp'
    extract: true
    auth_error_code: 401
  auth: !plugin
    type: HTTPHeaderAuth
    headers:
      Cookie: "seeedtoken={apikey}"
  products:
    S1_SAR_GRD:
      productType: GRD
      processingLevel: L1_
      collection: Sentinel1
      metadata_mapping:
        cloudCover: 'null/text()'
    S1_SAR_SLC:
      productType: SLC
      processingLevel: L1_
      collection: Sentinel1
      metadata_mapping:
        cloudCover: 'null/text()'
    S2_MSI_L1C:
      productType: IMAGE
      processingLevel: L1C
      collection: Sentinel2
    S2_MSI_L2A:
      productType: IMAGE
      processingLevel: L2A
      collection: Sentinel2
    S3_OLCI_L2LFR:
      productType: OL_2_LFR___
      instrument: OLCI
      collection: Sentinel3
    GENERIC_PRODUCT_TYPE:
      productType: '{productType}'
      collection: '{collection}'
      instrument: '{instrument}'
      processingLevel: '{processingLevel}'
---
!provider
  name: onda
  priority: 0
  description: Serco DIAS
  roles:
    - host
  url: https://www.onda-dias.eu/cms/
  search: !plugin
    type: ODataV4Search
    api_endpoint: 'https://catalogue.onda-dias.eu/dias-catalogue/Products'
    need_auth: false
    dont_quote:
      - '['
      - ']'
      - '$'
      - '='
      - '&'
      - ':'
    pagination:
      count_endpoint: 'https://catalogue.onda-dias.eu/dias-catalogue/Products/$count'
      next_page_url_tpl: '{url}?{search}&$top={items_per_page}&$skip={skip}'
      # 2021/03/19: 2000 is the max, if greater 200 response but contains an error message
      max_items_per_page: 2_000
    results_entry: 'value'
    literal_search_params:
      $format: json
    free_text_search_operations:
      $search:
        union: ' OR '
        wrapper: '"{}"'
        operations:
          AND:
            - 'footprint:"Intersects({geometry#to_rounded_wkt})"'
            - 'productType:{productType}'
            - 'platformName:{platform}'
            - 'cloudCoverPercentage:[0 TO {cloudCover}]'
            - 'beginPosition:[{startTimeFromAscendingNode#to_iso_utc_datetime} TO *]'
            - 'endPosition:[* TO {completionTimeFromAscendingNode#to_iso_utc_datetime}]'
            - '{id#remove_extension}'
    discover_metadata:
      auto_discovery: true
      metadata_pattern: '^[a-zA-Z0-9]+$'
      search_param:
        free_text_search_operations:
          $search:
            operations:
              AND:
                - '{metadata}:{{{metadata}}}'
      metadata_path: '$.*'
    metadata_mapping:
      # Opensearch resource identifier within the search engine context (in our case
      # within the context of the data provider)
      uid: '$.id'
      # OpenSearch Parameters for Collection Search (Table 3)
      productType:
        - 'productType'
        - '$.productType'
      platform:
        - platform
        - '$.platformName'
      platformSerialIdentifier: '$.platformSerialIdentifier'
      instrument: '$.instrumentName'
      processingLevel: '$.processingLevel'

      # INSPIRE obligated OpenSearch Parameters for Collection Search (Table 4)
      title: '$.filename'

      # OpenSearch Parameters for Product Search (Table 5)
      orbitNumber: '$.orbitNumber'
      orbitDirection: '$.orbitDirection'
      cloudCover: '$.cloudCoverPercentage'
      productVersion: '$.processingBaseline'
      productQualityStatus: '$.generalQualityFlag'
      creationDate: '$.creationDate'
      processingDate: '$.processingDate'
      sensorMode: '$.sensorOperationalMode'

      # OpenSearch Parameters for Acquistion Parameters Search (Table 6)
      startTimeFromAscendingNode: '$.beginPosition'
      completionTimeFromAscendingNode: '$.endPosition'
      polarizationChannels: '{$.polarisationChannels#replace_str(","," ")}'

      # Custom parameters (not defined in the base document referenced above)
      id: '{$.filename#remove_extension}'
      # The geographic extent of the product
      geometry: '$.footprint'
      # The url of the quicklook
      quicklook: '$.quicklook'
      # The url to download the product "as is" (literal or as a template to be completed either after the search result
      # is obtained from the provider or during the eodag download phase)
      downloadLink: '%(base_uri)s({uid})/$value'
      # storageStatus: must be one of ONLINE, STAGING, OFFLINE
      storageStatus: '{$.offline#get_group_name((?P<ONLINE>False)|(?P<OFFLINE>True))}'
      # Url used for ordering product if it is offline/archived
      orderLink: 'https://catalogue.onda-dias.eu/dias-catalogue/Products({uid})/Ens.Order'
  products:
    S1_SAR_OCN:
      productType: '*OCN*'
      metadata_mapping:
        cloudCover: '$.null'
    S1_SAR_GRD:
      productType: '*GRD*'
      metadata_mapping:
        cloudCover: '$.null'
    S1_SAR_SLC:
      productType: '*SLC*'
      metadata_mapping:
        cloudCover: '$.null'
    S1_SAR_RAW:
      productType: '*RAW*'
      metadata_mapping:
        cloudCover: '$.null'
    S2_MSI_L1C:
      productType: S2MSI1C
    S2_MSI_L2A:
      productType: S2MSI2A
    S3_EFR:
      productType: OL_1_EFR___
    S3_ERR:
      productType: OL_1_ERR___
    S3_OLCI_L2LFR:
      productType: OL_2_LFR___
    S3_OLCI_L2LRR:
      productType: OL_2_LRR___
    S3_SLSTR_L1RBT:
      productType: SL_1_RBT___
    S3_SLSTR_L2LST:
      productType: SL_2_LST___
    S3_LAN:
      productType: SR_2_LAN___
      metadata_mapping:
        cloudCover: '$.null'
    S3_SRA:
      productType: SR_1_SRA_A_
      metadata_mapping:
        cloudCover: '$.null'
    S3_SRA_BS:
      productType: SR_1_SRA_BS
      metadata_mapping:
        cloudCover: '$.null'
    S3_SRA_A_BS:
      productType: SR_1_SRA___
      metadata_mapping:
        cloudCover: '$.null'
    S3_WAT:
      productType: SR_2_WAT___
      metadata_mapping:
        cloudCover: '$.null'
    L8_OLI_TIRS_C1L1:
      platform: 'Landsat-*'
    GENERIC_PRODUCT_TYPE:
      productType: '{productType}'
      platform: '{platform}'
  download: !plugin
    type: HTTPDownload
    base_uri: 'https://catalogue.onda-dias.eu/dias-catalogue/Products'
    extract: true
    auth_error_code: 401
    order_enabled: true
    order_method: 'POST'
    order_headers:
      Content-Type: application/json
  auth: !plugin
    type: GenericAuth

---
!provider
  name: astraea_eod
  priority: 0
  roles:
    - host
  description: Astraea Earth OnDemand
  url: https://earthondemand.astraea.earth
  search: !plugin
    type: StacSearch
    api_endpoint: https://eod-catalog-svc-prod.astraea.earth/search
    need_auth: false
    pagination:
      # 2021/03/19: The docs (https://eod-catalog-svc-prod.astraea.earth/api.html#operation/getSearchSTAC)
      # say the max is 10_000. In practice 1_000 products are returned if more are asked (even greater
      # than 10_000), without any error.
      # This provider doesn't implement any pagination, let's just try to get the maximum number of
      # products available at once then, so we stick to 10_000.
      max_items_per_page: 10_000
    metadata_mapping:
      # redefine the following mapppings as the provider does not support advanced queries/filtering,
      # these parameters will not be queryable
      doi: '$.properties."sci:doi"'
      processingLevel: '$.properties."processing:level"'
      platform: '$.properties.constellation'
      platformSerialIdentifier: '$.properties.platform'
      instrument: '$.properties.instruments'
      # INSPIRE obligated OpenSearch Parameters for Collection Search (Table 4)
      title: '$.id'
      abstract: '$.properties.description'
      resolution: '$.properties.gsd'
      publicationDate: '$.properties.published'
      # OpenSearch Parameters for Product Search (Table 5)
      orbitNumber: '$.properties."sat:relative_orbit"'
      orbitDirection: '$.properties."sat:orbit_state"'
      cloudCover: '$.properties."eo:cloud_cover"'
      sensorMode: '$.properties."sar:instrument_mode"'
      creationDate: '$.properties.created'
      modificationDate: '$.properties.updated'
      productVersion: '$.properties.version'
      # OpenSearch Parameters for Acquistion Parameters Search (Table 6)
      availabilityTime: '$.properties.availabilityTime'
      acquisitionStation: '$.properties.acquisitionStation'
      acquisitionSubType: '$.properties.acquisitionSubType'
      illuminationAzimuthAngle: '$.properties."view:sun_azimuth"'
      illuminationElevationAngle: '$.properties."view:sun_elevation"'
      polarizationChannels: '$.properties."sar:polarizations"'
      dopplerFrequency: '$.properties."sar:frequency_band"'
  products:
    S1_SAR_GRD:
      productType: sentinel1_l1c_grd
      metadata_mapping:
        cloudCover: '$.null'
        platformSerialIdentifier: '$.id.`split(_, 0, -1)`'
        polarizationMode: '$.id.`sub(/.{14}([A-Z]{2}).*/, \\1)`'
        awsPath: '$.assets.productInfo.href.`sub(/(.*)\/productInfo\.json/, \\1)`'
        assets: |
          {$.assets#dict_update([["manifest.safe",[["title","manifest.safe"],["href","{awsPath}/manifest.safe"],["roles",["metadata"]],["type","application/xml"]]]])}
    S2_MSI_L1C:
      productType: sentinel2_l1c
      metadata_mapping:
        platformSerialIdentifier: '$.id.`split(_, 0, -1)`'
        tileInfo: '{$.assets.tileInfo.href#replace_str(r"s3(.*)sentinel-s2-l1c/",r"https\1roda.sentinel-hub.com/sentinel-s2-l1c/")}'
        tilePath: |
          $.assets.tileInfo.href.`sub(/.*\/sentinel-s2-l1c\/(tiles\/.*)\/tileInfo\.json/, \\1)`
    S2_MSI_L2A:
      productType: sentinel2_l2a
      metadata_mapping:
        platformSerialIdentifier: '$.id.`split(_, 0, -1)`'
        tileInfo: '{$.assets.tileInfo.href#replace_str(r"s3(.*)sentinel-s2-l2a/",r"https\1roda.sentinel-hub.com/sentinel-s2-l2a/")}'
        tilePath: |
          $.assets.tileInfo.href.`sub(/.*\/sentinel-s2-l2a\/(tiles\/.*)\/tileInfo\.json/, \\1)`
    L8_OLI_TIRS_C1L1:
      productType: landsat8_l1tp
    MODIS_MCD43A4:
      productType: mcd43a4
    NAIP:
      productType: naip
    GENERIC_PRODUCT_TYPE:
      productType: '{productType}'
  download: !plugin
    type: AwsDownload
    base_uri: https://eod-catalog-svc-prod.astraea.earth
    flatten_top_dirs: True
    products:
      S1_SAR_GRD:
        default_bucket: 'sentinel-s1-l1c'
        build_safe: true
        complementary_url_key:
          - awsPath
      S2_MSI_L1C:
        default_bucket: 'sentinel-s2-l1c'
        build_safe: true
        fetch_metadata:
          fetch_url: '{tileInfo}'
          fetch_format: json
          update_metadata:
            title: '$.productName'
            productPath: '$.productPath'
        complementary_url_key:
          - productPath
          - tilePath
      S2_MSI_L2A:
        default_bucket: 'sentinel-s2-l2a'
        build_safe: true
        fetch_metadata:
          fetch_url: '{tileInfo}'
          fetch_format: json
          update_metadata:
            title: '$.productName'
            productPath: '$.productPath'
        complementary_url_key:
          - productPath
          - tilePath
  auth: !plugin
    type: AwsAuth

---
!provider
  name: usgs_satapi_aws
  priority: 0
  roles:
    - host
  description: USGS Landsatlook SAT API
  url: https://landsatlook.usgs.gov/stac-server
  search: !plugin
    type: StacSearch
    api_endpoint: https://landsatlook.usgs.gov/stac-server/search
    need_auth: false
    pagination:
      # 2021/03/19: no more than 10_000 (if greater, returns a 500 error code)
      # but in practive if an Internal Server Error is returned for more than
      # about 500 products.
      max_items_per_page: 500
    metadata_mapping:
      assets: '{$.assets#recursive_sub_str(r"https?(.*)landsatlook.usgs.gov/data/",r"s3\1usgs-landsat/")}'
      awsProductId: '{$.assets.thumbnail.href#replace_str(r".+/([A-Z0-9_]+)/[\w.]+$",r"\1")}'
  products:
    LANDSAT_C2L1:
      productType: landsat-c2l1
    LANDSAT_C2L2_SR:
      productType: landsat-c2l2-sr
    LANDSAT_C2L2_ST:
      productType: landsat-c2l2-st
    LANDSAT_C2L2ALB_BT:
      productType: landsat-c2l2alb-bt
    LANDSAT_C2L2ALB_SR:
      productType: landsat-c2l2alb-sr
    LANDSAT_C2L2ALB_ST:
      productType: landsat-c2l2alb-st
    LANDSAT_C2L2ALB_TA:
      productType: landsat-c2l2alb-ta
    GENERIC_PRODUCT_TYPE:
      productType: '{productType}'
  download: !plugin
    type: AwsDownload
    base_uri: https://landsatlook.usgs.gov/stac-server
    flatten_top_dirs: True
  auth: !plugin
    type: AwsAuth

---
!provider
  name: earth_search
  priority: 0
  roles:
    - host
  description: Earth Search
  url: https://www.element84.com/earth-search/
  search: !plugin
    type: StacSearch
    api_endpoint: https://earth-search.aws.element84.com/v0/search
    need_auth: false
    pagination:
      # Override the default next page url key path of StacSearch because the next link returned
      # by Earth Search is invalid (as of 2021/04/29). Entry set to null (None) to avoid using the
      # next page retrieval mechanism, `next_page_url_tpl` will be used instead (inherited from StacSearch)
      # Remove that entry if Earth Search updates that and returns a valid link.
      next_page_url_key_path: null
      # 2021/04/28: Earth-Search relies on Sat-API whose docs (http://sat-utils.github.io/sat-api/#search-stac-items-by-simple-filtering-)
      # say the max is 10_000. In practice a too high number (e.g. 5_000) returns a 502 error ({"message": "Internal server error"}).
      # Let's set it to a more robust number: 500
      max_items_per_page: 500
  products:
    S2_MSI_L1C:
      productType: sentinel-s2-l1c
      metadata_mapping:
        title: '$.properties."sentinel:product_id"'
        platformSerialIdentifier: '$.id.`split(_, 0, -1)`'
        polarizationMode: '$.id.`sub(/.{14}([A-Z]{2}).*/, \\1)`'
        productPath: |
          $.properties."sentinel:product_id".`sub(/([S2AB]{3})_MSIL1C_([0-9]{4})([0-9]{2})([0-9]{2})(T.*)/, products/\\2/\\3/\\4/\\1_MSIL1C_\\2\\3\\4\\5)`.`sub(/\/0+/, \/)`
        tilePath: |
          $.assets.info.href.`sub(/.*/sentinel-s2-l1c\/(tiles\/.*)\/tileInfo\.json/, \\1)`
    S2_MSI_L2A:
      productType: sentinel-s2-l2a
      metadata_mapping:
        title: '$.properties."sentinel:product_id"'
        platformSerialIdentifier: '$.id.`split(_, 0, -1)`'
        polarizationMode: '$.id.`sub(/.{14}([A-Z]{2}).*/, \\1)`'
        productPath: |
          $.properties."sentinel:product_id".`sub(/([S2AB]{3})_MSIL2A_([0-9]{4})([0-9]{2})([0-9]{2})(T.*)/, products/\\2/\\3/\\4/\\1_MSIL2A_\\2\\3\\4\\5)`.`sub(/\/0+/, \/)`
        tilePath: |
          $.assets.info.href.`sub(/.*/sentinel-s2-l2a\/(tiles\/.*)\/tileInfo\.json/, \\1)`
    L8_OLI_TIRS_C1L1:
      productType: landsat-8-l1-c1
    GENERIC_PRODUCT_TYPE:
      productType: '{productType}'
  download: !plugin
    type: AwsDownload
    base_uri: https://earth-search.aws.element84.com/v0
    flatten_top_dirs: True
    products:
      S2_MSI_L1C:
        default_bucket: 'sentinel-s2-l1c'
        build_safe: true
        complementary_url_key:
          - productPath
          - tilePath
      S2_MSI_L2A:
        default_bucket: 'sentinel-s2-l2a'
        build_safe: true
        complementary_url_key:
          - productPath
          - tilePath
  auth: !plugin
    type: AwsAuth

---
!provider
  name: earth_search_cog
  priority: 0
  roles:
    - host
  description: Earth Search
  url: https://www.element84.com/earth-search/
  search: !plugin
    type: StacSearch
    api_endpoint: https://earth-search.aws.element84.com/v0/search
    need_auth: false
    pagination:
      # Override the default next page url key path of StacSearch because the next link returned
      # by Earth Search is invalid (as of 2021/04/29). Entry set to null (None) to avoid using the
      # next page retrieval mechanism, `next_page_url_tpl` will be used instead (inherited from StacSearch)
      # Remove that entry if Earth Search updates that and returns a valid link.
      next_page_url_key_path: null
      # 2021/04/28: Earth-Search relies on Sat-API whose docs (http://sat-utils.github.io/sat-api/#search-stac-items-by-simple-filtering-)
      # say the max is 10_000. In practice a too high number (e.g. 5_000) returns a 502 error ({"message": "Internal server error"}).
      # Let's set it to a more robust number: 500
      max_items_per_page: 500
    metadata_mapping:
      platformSerialIdentifier: '$.id.`split(_, 0, -1)`'
      polarizationMode: '$.id.`sub(/.{14}([A-Z]{2}).*/, \\1)`'
  products:
    S2_MSI_L2A_COG:
      productType: sentinel-s2-l2a-cogs
    GENERIC_PRODUCT_TYPE:
      productType: '{productType}'
  download: !plugin
    type: HTTPDownload
    base_uri: 'https://sentinel-cogs.s3.us-west-2.amazonaws.com'
---
!provider
  name: ecmwf
  priority: 0
  description: ECMWF archive products
  roles:
    - host
  url: https://www.ecmwf.int
  api: !plugin
    type: EcmwfApi
    api_endpoint: https://api.ecmwf.int/v1
    extract: false
    metadata_mapping:
      productType: '$.productType'
      title: '$.id'
      startTimeFromAscendingNode: '{$.startTimeFromAscendingNode#to_iso_date}'
      completionTimeFromAscendingNode:
        - 'date={startTimeFromAscendingNode#to_iso_date}/to/{completionTimeFromAscendingNode#to_iso_date(-1,)}'
        - '{$.completionTimeFromAscendingNode#to_iso_date}'
      id: '$.id'
      # The geographic extent of the product
      geometry:
        - 'area={geometry#to_nwse_bounds_str(/)}'
        - '$.geometry'
      # storageStatus set to ONLINE for consistency between providers
      storageStatus: '{$.null#replace_str("Not Available","ONLINE")}'
      downloadLink: 'https://apps.ecmwf.int/datasets/data/{dataset}'
      # ECMWF/MARS specific parameters
      # Full list here https://confluence.ecmwf.int/display/UDOC/Keywords+in+MARS+and+Dissemination+requests
      accuracy:
        - accuracy
        - '$.accuracy'
      anoffset:
        - anoffset
        - '$.anoffset'
      bitmap:
        - bitmap
        - '$.bitmap'
      block:
        - block
        - '$.block'
      channel:
        - channel
        - '$.channel'
      class:
        - class
        - '$.class'
      database:
        - database
        - '$.database'
      dataset:
        - dataset
        - '$.dataset'
      diagnostic:
        - diagnostic
        - '$.diagnostic'
      direction:
        - direction
        - '$.direction'
      domain:
        - domain
        - '$.domain'
      duplicates:
        - duplicates
        - '$.duplicates'
      expect:
        - expect
        - '$.expect'
      expver:
        - expver
        - '$.expver'
      fcmonth:
        - fcmonth
        - '$.fcmonth'
      fcperiod:
        - fcperiod
        - '$.fcperiod'
      fieldset:
        - fieldset
        - '$.fieldset'
      filter:
        - filter
        - '$.filter'
      format:
        - format
        - '$.format'
      frame:
        - frame
        - '$.frame'
      frequency:
        - frequency
        - '$.frequency'
      grid:
        - grid
        - '$.grid'
      hdate:
        - hdate
        - '$.hdate'
      ident:
        - ident
        - '$.ident'
      interpolation:
        - interpolation
        - '$.interpolation'
      intgrid:
        - intgrid
        - '$.intgrid'
      iteration:
        - iteration
        - '$.iteration'
      latitude:
        - latitude
        - '$.latitude'
      levelist:
        - levelist
        - '$.levelist'
      levtype:
        - levtype
        - '$.levtype'
      longitude:
        - longitude
        - '$.longitude'
      lsm:
        - lsm
        - '$.lsm'
      method:
        - method
        - '$.method'
      number:
        - number
        - '$.number'
      obsgroup:
        - obsgroup
        - '$.obsgroup'
      obstype:
        - obstype
        - '$.obstype'
      origin:
        - origin
        - '$.origin'
      packing:
        - packing
        - '$.packing'
      padding:
        - padding
        - '$.padding'
      param:
        - param
        - '$.param'
      priority:
        - priority
        - '$.priority'
      product:
        - product
        - '$.product'
      range:
        - range
        - '$.range'
      refdate:
        - refdate
        - '$.refdate'
      reference:
        - reference
        - '$.reference'
      reportype:
        - reportype
        - '$.reportype'
      repres:
        - repres
        - '$.repres'
      resol:
        - resol
        - '$.resol'
      rotation:
        - rotation
        - '$.rotation'
      section:
        - section
        - '$.section'
      source:
        - source
        - '$.source'
      step:
        - step
        - '$.step'
      stream:
        - stream
        - '$.stream'
      system:
        - system
        - '$.system'
      target:
        - target
        - '$.target'
      time:
        - time
        - '$.time'
      truncation:
        - truncation
        - '$.truncation'
      type:
        - type
        - '$.type'
      use:
        - use
        - '$.use'
  products:
    # See Archive Catalog in https://apps.ecmwf.int/archive-catalogue/
    # See available Public Datasets in https://apps.ecmwf.int/datasets/
    TIGGE_CF_SFC:
      class: ti
      dataset: tigge
      expver: prod
      type: cf
      levtype: sfc
      origin: ecmf
      grid: 0.5/0.5
      param: 59/134/136/146/147/151/165/166/167/168/172/176/177/179/189/235/228001/228002/228039/228139/228141/228144/228164/228228
      step: 0
      time: 00:00
    GENERIC_PRODUCT_TYPE:
      dataset: '{productType}'

Parameters mapping#

EODAG maps each provider’s specific metadata parameters to a common model using OGC OpenSearch Extension for Earth Observation. Extra parameters having no equivalent in this model are mapped as is.

Depending on the provider, some parameters are queryable or not. This is configured in providers.yml:

  • If a parameter metadata-mapping is a list, the first element will help constructing the query (using format()), and the 2nd will help extracting its values from the query result (using jsonpath)

  • If a parameter metadata-mapping is a string, it will not be queryable and this string will help extracting its values from the query result (using jsonpath).

some_provider:
   search:
      metadata_mapping:
         queryableParameter:
            - 'this_is_query_string={queryableParameter}'
            - '$.jsonpath.in.result.to.parameter'
         nonQueryableParameter: '$.jsonpath.in.result.to.another_parameter'

The following tables list the parameters supported by providers, and if they are queryable or not.

OpenSearch parameters (CSV)#

parameter

astraea_eod

creodias

earth_search

earth_search_cog

ecmwf

mundi

onda

peps

sobloo

theia

usgs_satapi_aws

abstract

metadata only

metadata only

metadata only

metadata only

metadata only

metadata only

metadata only

accessConstraint

metadata only

metadata only

metadata only

metadata only

acquisitionStation

metadata only

queryable metadata

queryable metadata

metadata only

metadata only

queryable metadata

acquisitionSubType

metadata only

queryable metadata

queryable metadata

metadata only

metadata only

queryable metadata

acquisitionType

metadata only

metadata only

antennaLookDirection

metadata only

metadata only

archivingCenter

metadata only

metadata only

availabilityTime

metadata only

queryable metadata

queryable metadata

metadata only

metadata only

metadata only

queryable metadata

classification

metadata only

cloudCover

metadata only

queryable metadata

queryable metadata

queryable metadata

queryable metadata

metadata only

queryable metadata

queryable metadata

queryable metadata

queryable metadata

completionTimeFromAscendingNode

queryable metadata

queryable metadata

queryable metadata

queryable metadata

queryable metadata

metadata only

queryable metadata

queryable metadata

queryable metadata

queryable metadata

compositeType

metadata only

creationDate

metadata only

queryable metadata

queryable metadata

metadata only

metadata only

metadata only

metadata only

metadata only

queryable metadata

dissemination

metadata only

doi

metadata only

queryable metadata

queryable metadata

metadata only

queryable metadata

dopplerFrequency

metadata only

queryable metadata

queryable metadata

metadata only

metadata only

queryable metadata

frame

metadata only

hasSecurityConstraints

metadata only

highestLocation

metadata only

illuminationAzimuthAngle

metadata only

queryable metadata

queryable metadata

metadata only

metadata only

queryable metadata

illuminationElevationAngle

metadata only

queryable metadata

queryable metadata

metadata only

metadata only

queryable metadata

illuminationZenithAngle

metadata only

metadata only

incidenceAngleVariation

metadata only

metadata only

instrument

metadata only

queryable metadata

queryable metadata

queryable metadata

queryable metadata

metadata only

queryable metadata

metadata only

queryable metadata

queryable metadata

keyword

metadata only

metadata only

metadata only

metadata only

language

metadata only

lineage

metadata only

lowestLocation

metadata only

maximumIncidenceAngle

metadata only

metadata only

minimumIncidenceAngle

metadata only

metadata only

modificationDate

metadata only

metadata only

queryable metadata

queryable metadata

metadata only

metadata only

metadata only

queryable metadata

orbitDirection

metadata only

queryable metadata

queryable metadata

queryable metadata

metadata only

metadata only

queryable metadata

metadata only

queryable metadata

orbitNumber

metadata only

queryable metadata

queryable metadata

queryable metadata

metadata only

metadata only

queryable metadata

metadata only

metadata only

queryable metadata

orbitType

metadata only

organisationName

queryable metadata

metadata only

queryable metadata

metadata only

organisationRole

metadata only

otherConstraint

metadata only

parentIdentifier

queryable metadata

metadata only

queryable metadata

metadata only

platform

metadata only

metadata only

queryable metadata

queryable metadata

metadata only

queryable metadata

metadata only

metadata only

queryable metadata

queryable metadata

platformSerialIdentifier

metadata only

queryable metadata

queryable metadata

metadata only

metadata only

metadata only

queryable metadata

metadata only

queryable metadata

queryable metadata

processingCenter

metadata only

metadata only

processingDate

metadata only

metadata only

metadata only

processingLevel

metadata only

queryable metadata

queryable metadata

queryable metadata

queryable metadata

metadata only

queryable metadata

metadata only

queryable metadata

queryable metadata

processingMode

metadata only

metadata only

processorName

metadata only

metadata only

productQualityDegradationTag

metadata only

productQualityStatus

metadata only

metadata only

metadata only

productType

queryable metadata

queryable metadata

queryable metadata

queryable metadata

queryable metadata

queryable metadata

queryable metadata

queryable metadata

queryable metadata

queryable metadata

productVersion

metadata only

queryable metadata

queryable metadata

metadata only

metadata only

metadata only

metadata only

queryable metadata

publicationDate

metadata only

metadata only

queryable metadata

queryable metadata

metadata only

metadata only

metadata only

metadata only

queryable metadata

resolution

metadata only

queryable metadata

queryable metadata

queryable metadata

metadata only

queryable metadata

metadata only

queryable metadata

sensorMode

metadata only

queryable metadata

queryable metadata

queryable metadata

metadata only

metadata only

queryable metadata

metadata only

metadata only

queryable metadata

sensorType

metadata only

snowCover

queryable metadata

metadata only

queryable metadata

metadata only

spectralRange

metadata only

startTimeFromAscendingNode

metadata only

queryable metadata

metadata only

metadata only

queryable metadata

metadata only

queryable metadata

queryable metadata

queryable metadata

metadata only

swathIdentifier

metadata only

queryable metadata

title

metadata only

metadata only

metadata only

metadata only

metadata only

metadata only

metadata only

metadata only

metadata only

metadata only

topicCategory

metadata only

metadata only

track

metadata only

useLimitation

metadata only

wavelengths

metadata only

Provider/eodag specific parameters (CSV)#

parameter

astraea_eod

creodias

earth_search

earth_search_cog

ecmwf

mundi

onda

peps

sobloo

theia

usgs_satapi_aws

assets

metadata only

metadata only

metadata only

metadata only

awsProductId

metadata only

cultivatedCover

queryable metadata

desertCover

queryable metadata

downloadLink

metadata only

metadata only

metadata only

metadata only

metadata only

metadata only

metadata only

metadata only

metadata only

metadata only

floodedCover

queryable metadata

forestCover

queryable metadata

geometry

queryable metadata

queryable metadata

queryable metadata

queryable metadata

queryable metadata

metadata only

queryable metadata

queryable metadata

queryable metadata

queryable metadata

herbaceousCover

queryable metadata

iceCover

queryable metadata

id

queryable metadata

queryable metadata

queryable metadata

queryable metadata

queryable metadata

metadata only

queryable metadata

queryable metadata

metadata only

queryable metadata

orderLink

metadata only

polarizationChannels

metadata only

metadata only

queryable metadata

queryable metadata

metadata only

metadata only

metadata only

metadata only

queryable metadata

polarizationMode

metadata only

metadata only

queryable metadata

metadata only

processingBaseline

queryable metadata

publishedAfter

queryable metadata

publishedBefore

queryable metadata

quicklook

metadata only

metadata only

metadata only

metadata only

metadata only

metadata only

metadata only

metadata only

metadata only

metadata only

relativeOrbitNumber

queryable metadata

sortOrder

queryable metadata

sortParam

queryable metadata

specification

metadata only

status

queryable metadata

storageStatus

metadata only

metadata only

metadata only

metadata only

metadata only

metadata only

metadata only

metadata only

metadata only

metadata only

thumbnail

metadata only

metadata only

metadata only

metadata only

metadata only

metadata only

metadata only

metadata only

timeliness

queryable metadata

uid

metadata only

metadata only

metadata only

metadata only

metadata only

metadata only

urbanCover

queryable metadata

waterCover

queryable metadata