Hint

You can run this notebook in a live session with .

Providers and products

from eodag import EODataAccessGateway
dag = EODataAccessGateway()

Providers available

The method available_providers() returns a list of the pre-configured providers.

available_providers = dag.available_providers()
available_providers

['astraea_eod',
 'cop_ads',
 'cop_cds',
 'cop_dataspace',
 'creodias',
 'earth_search',
 'earth_search_cog',
 'earth_search_gcs',
 'ecmwf',
 'mundi',
 'onda',
 'peps',
 'sara',
 'theia',
 'usgs_satapi_aws']

print(f"eodag has {len(available_providers)} providers already configured.")

eodag has 15 providers already configured.

It can take a product type as an argument and will return the providers known to eodag that offer this product.

dag.available_providers("S2_MSI_L1C")

['astraea_eod',
 'cop_dataspace',
 'creodias',
 'earth_search',
 'earth_search_gcs',
 'mundi',
 'onda',
 'peps',
 'sara']

Note

If a provider is configured to need authentication for search, and has no crendentials set, it will be pruned on EODAG initialization, and will not appear in available providers list.

Product types available

The method list_product_types() returns a dictionary that represents eodag’s internal product type catalog if used with fetch_providers=False. It will fetch providers for new product types and return an extended list if used with fetch_providers=True (default behavior).

internal_catalog = dag.list_product_types(fetch_providers=False)
print(f"EODAG has {len(internal_catalog)} product types stored in its internal catalog.")

EODAG has 86 product types stored in its internal catalog.

extended_catalog = dag.list_product_types()
print(f"EODAG has {len(extended_catalog)} product types stored in its extended catalog, after having fetched providers.")

EODAG has 109 product types stored in its extended catalog, after having fetched providers.

When providers are fetched for new product types, eodag’s product types configuration is updated in EODataAccessGateway instance. Extended product types list is then returned independantly of fetch_providers option in list_product_types():

called_again_catalog = dag.list_product_types(fetch_providers=False)
print(f"list_product_types() keeps returning {len(called_again_catalog)} product types.")

list_product_types() keeps returning 109 product types.

internal_catalog[0]

{'ID': 'CAMS_EAC4',
 'abstract': 'CAMS (Copernicus Atmosphere Monitoring Service) ECMWF Atmospheric Composition Reanalysis 4\nfrom Copernicus ADS\n',
 'instrument': None,
 'platform': 'CAMS',
 'platformSerialIdentifier': 'CAMS',
 'processingLevel': None,
 'keywords': 'Copernicus,Atmosphere,Atmospheric,Reanalysis,CAMS,EAC4,ADS,ECMWF',
 'sensorType': 'ATMOSPHERIC',
 'license': 'proprietary',
 'title': 'CAMS ECMWF Atmospheric Composition Reanalysis 4',
 'missionStartDate': '2003-01-01T00:00:00Z'}

products_id = [p["ID"] for p in internal_catalog]
products_id

['CAMS_EAC4',
 'CAMS_GACF_AOT',
 'CAMS_GACF_MR',
 'CAMS_GACF_RH',
 'ERA5_SL',
 'L57_REFLECTANCE',
 'L8_OLI_TIRS_C1L1',
 'L8_REFLECTANCE',
 'LANDSAT_C2L1',
 'LANDSAT_C2L2ALB_BT',
 'LANDSAT_C2L2ALB_SR',
 'LANDSAT_C2L2ALB_ST',
 'LANDSAT_C2L2ALB_TA',
 'LANDSAT_C2L2_SR',
 'LANDSAT_C2L2_ST',
 'MODIS_MCD43A4',
 'NAIP',
 'OSO',
 'PLD_BUNDLE',
 'PLD_PAN',
 'PLD_PANSHARPENED',
 'PLD_XS',
 'S1_SAR_GRD',
 'S1_SAR_OCN',
 'S1_SAR_RAW',
 'S1_SAR_SLC',
 'S2_MSI_L1C',
 'S2_MSI_L2A',
 'S2_MSI_L2A_COG',
 'S2_MSI_L2A_MAJA',
 'S2_MSI_L2B_MAJA_SNOW',
 'S2_MSI_L2B_MAJA_WATER',
 'S2_MSI_L3A_WASP',
 'S3_EFR',
 'S3_ERR',
 'S3_LAN',
 'S3_OLCI_L2LFR',
 'S3_OLCI_L2LRR',
 'S3_OLCI_L2WFR',
 'S3_OLCI_L2WRR',
 'S3_RAC',
 'S3_SLSTR_L1RBT',
 'S3_SLSTR_L2AOD',
 'S3_SLSTR_L2FRP',
 'S3_SLSTR_L2LST',
 'S3_SLSTR_L2WST',
 'S3_SRA',
 'S3_SRA_A',
 'S3_SRA_BS',
 'S3_SY_AOD',
 'S3_SY_SYN',
 'S3_SY_V10',
 'S3_SY_VG1',
 'S3_SY_VGP',
 'S3_WAT',
 'S5P_L1B_IR_SIR',
 'S5P_L1B_IR_UVN',
 'S5P_L1B_RA_BD1',
 'S5P_L1B_RA_BD2',
 'S5P_L1B_RA_BD3',
 'S5P_L1B_RA_BD4',
 'S5P_L1B_RA_BD5',
 'S5P_L1B_RA_BD6',
 'S5P_L1B_RA_BD7',
 'S5P_L1B_RA_BD8',
 'S5P_L2_AER_AI',
 'S5P_L2_AER_LH',
 'S5P_L2_CH4',
 'S5P_L2_CLOUD',
 'S5P_L2_CO',
 'S5P_L2_HCHO',
 'S5P_L2_NO2',
 'S5P_L2_NP_BD3',
 'S5P_L2_NP_BD6',
 'S5P_L2_NP_BD7',
 'S5P_L2_O3',
 'S5P_L2_O3_PR',
 'S5P_L2_O3_TCL',
 'S5P_L2_SO2',
 'SPOT5_SPIRIT',
 'SPOT_SWH',
 'SPOT_SWH_OLD',
 'TIGGE_CF_SFC',
 'VENUS_L1C',
 'VENUS_L2A_MAJA',
 'VENUS_L3A_MAJA']

The method can take a provider name as an argument and will return the product types known to eodag that are offered by this provider.

peps_products = dag.list_product_types("peps")
[p["ID"] for p in peps_products]

['S1_SAR_GRD', 'S1_SAR_OCN', 'S1_SAR_SLC', 'S2_MSI_L1C', 'S2_MSI_L2A']

Combine these two methods

These two methods can be combined to find which product type is the most common in eodag’s catalog among all the providers.

availability_per_product = []
for product in products_id:
    providers = dag.available_providers(product)
    availability_per_product.append((product, len(providers)))
availability_per_product = sorted(availability_per_product, key=lambda x: x[1], reverse=True)
most_common_p_type, nb_providers = availability_per_product[0]
print(f"The most common product type is '{most_common_p_type}' with {nb_providers} providers offering it.")

The most common product type is 'S2_MSI_L1C' with 9 providers offering it.

These can be also used to find out which provider (as configured by eodag) offers the hights number of different product types.

availability_per_provider = []
for provider in dag.available_providers():
    provider_products_id = [
        p["ID"]
        for p in dag.list_product_types(provider, fetch_providers=False)
    ]
    availability_per_provider.append(
        (provider, len(provider_products_id))
    )
availability_per_provider = sorted(availability_per_provider, key=lambda x: x[1], reverse=True)
provider, nb_p_types = availability_per_provider[0]
print(f"The provider with the largest number of product types is '{provider}' with {nb_p_types}.")

The provider with the largest number of product types is 'creodias' with 61.

Product types discovery

EODAG comes with a large list of pre-configured product types. Some others are available from providers catalogs but will not be configured, or are not yet configured in EODAG.

Some providers, like STAC providers, come in EODAG with a configuration describing how to discover these not-already-configured product types.

With the method discover_product_types() or CLI command eodag discover we can obtain a JSON configuration file that will be used as EODAG external product types configuration file.

In EODAG, the discovered EODAG external product types configuration file can be set to:

• a file automatically built from github actions and stored in eodag/resources/ext_product_types.json (default settings)

• a custom remote or local file by setting its path in EODAG_EXT_PRODUCT_TYPES_CFG_FILE environment variable (if the file is not readable, only user-modified providers will be fetched).

Then, when listing product types using list_product_types(fetch_providers=True), EODAG will first read the content of the EODAG external product types configuration file using fetch_product_types_list() then update EODataAccessGateway instance product types configuration, if needed.

The obtained product types list will contain both pre-configured and discovered product types.