Hint

You can run this notebook in a live session with Binder.

Copernicus Atmosphere using ECMWFSearch plugin#

In this tutorial we will show you how to use eodag to download data from providers using ECMWFSearch eodag plugin. You can currently find three providers that uses it, cop_ads, cop_cds and cop_ewds. For this tutorial we will use cop_ads, but cop_cds and cop_ewds are used the same way.

[1]:
# Switch to minimal mode for the exception handlers
%xmode Minimal

from eodag import EODataAccessGateway

dag = EODataAccessGateway()
Exception reporting mode: Minimal

Search (build download request)#

Validation#

Requests on these collections require some search-parameters to be set. If none is set, validation will prompt an error:

[2]:
dag.search(provider="cop_ads", collection="CAMS_EAC4", raise_errors=True)
ValidationError: 6 validation errors for Queryables
ecmwf:data_format
  Field required [type=missing, input_value={'collection': 'CAMS_EAC4'}, input_type=dict]
    For further information visit https://errors.pydantic.dev/2.11/v/missing
ecmwf:date
  Field required [type=missing, input_value={'collection': 'CAMS_EAC4'}, input_type=dict]
    For further information visit https://errors.pydantic.dev/2.11/v/missing
ecmwf:model_level
  Field required [type=missing, input_value={'collection': 'CAMS_EAC4'}, input_type=dict]
    For further information visit https://errors.pydantic.dev/2.11/v/missing
ecmwf:pressure_level
  Field required [type=missing, input_value={'collection': 'CAMS_EAC4'}, input_type=dict]
    For further information visit https://errors.pydantic.dev/2.11/v/missing
ecmwf:time
  Field required [type=missing, input_value={'collection': 'CAMS_EAC4'}, input_type=dict]
    For further information visit https://errors.pydantic.dev/2.11/v/missing
ecmwf:variable
  Field required [type=missing, input_value={'collection': 'CAMS_EAC4'}, input_type=dict]
    For further information visit https://errors.pydantic.dev/2.11/v/missing


The above exception was the direct cause of the following exception:

ValidationError: 6 error(s). ecmwf:data_format: Field required; ecmwf:pressure_level: Field required; ecmwf:time: Field required; ecmwf:model_level: Field required; ecmwf:variable: Field required; ecmwf:date: Field required

Check available queryables and default values:#

Available queryables parameters and associated values can be checked using list_queryables() method, or through cop_ads or cop_cds websites:

[3]:
queryables = dag.list_queryables(provider="cop_ads", collection="CAMS_EAC4")
queryables
[3]:
QueryablesDict (9) - additional_properties=False
'ecmwf_area':  typing.Annotated[tuple[Annotated[float, FieldInfo(annotation=NoneType, required=False, default=None, description='West border of the bounding box', metadata=[Ge(ge=-180), ...])], Annotated[float, FieldInfo(annotation=NoneType, required=False, default=None, description='South border of the bounding box', metadata=[Ge(ge=-90), ...])], Annotated[float, FieldInfo(annotation=NoneType, required=False, default=None, description='East border of the bounding box', metadata=[Ge(ge=-180), ...])], Annotated[float, FieldInfo(annotation=NoneType, required=False, default=None, description='North border of the bounding box', metadata=[Ge(ge=-90), ...])]],  FieldInfo( 'required': False, ... )] typing.Annotated[
tuple[typing.Annotated[float, FieldInfo(annotation=NoneType, required=False, default=None, description='West border of the bounding box', metadata=[Ge(ge=-180), Le(le=180)])], typing.Annotated[float, FieldInfo(annotation=NoneType, required=False, default=None, description='South border of the bounding box', metadata=[Ge(ge=-90), Le(le=90)])], typing.Annotated[float, FieldInfo(annotation=NoneType, required=False, default=None, description='East border of the bounding box', metadata=[Ge(ge=-180), Le(le=180)])], typing.Annotated[float, FieldInfo(annotation=NoneType, required=False, default=None, description='North border of the bounding box', metadata=[Ge(ge=-90), Le(le=90)])]],
FieldInfo(annotation=NoneType, required=False, default=None, alias_priority=2, validation_alias=AliasChoices(choices=['ecmwf:area', 'area']), serialization_alias='ecmwf:area', title='Sub-region extraction', description='Select a sub-region of the available area by providing its limits on latitude and longitude')
]
'ecmwf_data_format':  typing.Annotated[Literal['grib', ...],  FieldInfo( 'required': True, ... )] typing.Annotated[
typing.Literal['grib', 'netcdf_zip'],
FieldInfo(annotation=NoneType, required=True, alias_priority=2, validation_alias=AliasChoices(choices=['ecmwf:data_format', 'data_format']), serialization_alias='ecmwf:data_format', title='Data format', description='Please select a format for the data files, the native format of MARS dataset is GRIB.')
]
'ecmwf_date':  typing.Annotated[Literal['2003-01-01/2024-12-31'],  FieldInfo( 'required': True, ... )] typing.Annotated[
typing.Literal['2003-01-01/2024-12-31'],
FieldInfo(annotation=NoneType, required=True, alias_priority=2, validation_alias=AliasChoices(choices=['ecmwf:date', 'date']), serialization_alias='ecmwf:date', title='Date', description='date formatted like yyyy-mm-dd/yyyy-mm-dd')
]
'ecmwf_model_level':  typing.Annotated[list[Literal['1', ...]],  FieldInfo( 'required': True, ... )] typing.Annotated[
list[typing.Literal['1', '10', '11', '12', '13', '14', '15', '16', '17', '18', '19', '2', '20', '21', '22', '23', '24', '25', '26', '27', '28', '29', '3', '30', '31', '32', '33', '34', '35', '36', '37', '38', '39', '4', '40', '41', '42', '43', '44', '45', '46', '47', '48', '49', '5', '50', '51', '52', '53', '54', '55', '56', '57', '58', '59', '6', '60', '7', '8', '9']],
FieldInfo(annotation=NoneType, required=True, alias_priority=2, validation_alias=AliasChoices(choices=['ecmwf:model_level', 'model_level']), serialization_alias='ecmwf:model_level', title='Model level', description="Model level 1 is the top of the atmosphere. Model level 60 is the Earth's surface.")
]
'ecmwf_pressure_level':  typing.Annotated[list[Literal['1', ...]],  FieldInfo( 'required': True, ... )] typing.Annotated[
list[typing.Literal['1', '10', '100', '1000', '150', '2', '20', '200', '250', '3', '30', '300', '400', '5', '50', '500', '600', '7', '70', '700', '800', '850', '900', '925', '950']],
FieldInfo(annotation=NoneType, required=True, alias_priority=2, validation_alias=AliasChoices(choices=['ecmwf:pressure_level', 'pressure_level']), serialization_alias='ecmwf:pressure_level', title='Pressure level')
]
'ecmwf_time':  typing.Annotated[list[Literal['00:00', ...]],  FieldInfo( 'required': True, ... )] typing.Annotated[
list[typing.Literal['00:00', '03:00', '06:00', '09:00', '12:00', '15:00', '18:00', '21:00']],
FieldInfo(annotation=NoneType, required=True, alias_priority=2, validation_alias=AliasChoices(choices=['ecmwf:time', 'time']), serialization_alias='ecmwf:time', title='Time', description='Model base time as HH:MM (UTC)')
]
'ecmwf_variable':  typing.Annotated[list[Literal['10m_u_component_of_wind', ...]],  FieldInfo( 'required': True, ... )] typing.Annotated[
list[typing.Literal['10m_u_component_of_wind', '10m_v_component_of_wind', '2m_dewpoint_temperature', '2m_temperature', 'acetone', 'acetone_product', 'aldehydes', 'amine', 'ammonia', 'ammonium', 'black_carbon_aerosol_optical_depth_550nm', 'carbon_monoxide', 'dimethyl_sulfide', 'dinitrogen_pentoxide', 'dust_aerosol_0.03-0.55um_mixing_ratio', 'dust_aerosol_0.55-0.9um_mixing_ratio', 'dust_aerosol_0.9-20um_mixing_ratio', 'dust_aerosol_optical_depth_550nm', 'ethane', 'ethanol', 'ethene', 'formaldehyde', 'formic_acid', 'fraction_of_cloud_cover', 'geopotential', 'high_cloud_cover', 'high_vegetation_cover', 'hydrogen_peroxide', 'hydroperoxy_radical', 'hydrophilic_black_carbon_aerosol_mixing_ratio', 'hydrophilic_organic_matter_aerosol_mixing_ratio', 'hydrophobic_black_carbon_aerosol_mixing_ratio', 'hydrophobic_organic_matter_aerosol_mixing_ratio', 'hydroxyl_radical', 'isoprene', 'lake_cover', 'land_sea_mask', 'lead', 'leaf_area_index_high_vegetation', 'leaf_area_index_low_vegetation', 'low_cloud_cover', 'low_vegetation_cover', 'mean_altitude_of_maximum_injection', 'mean_sea_level_pressure', 'medium_cloud_cover', 'methacrolein_mvk', 'methacrylic_acid', 'methane_chemistry', 'methane_sulfonic_acid', 'methanol', 'methyl_glyoxal', 'methyl_peroxide', 'methylperoxy_radical', 'near_ir_albedo_for_diffuse_radiation', 'near_ir_albedo_for_direct_radiation', 'nitrate', 'nitrate_radical', 'nitric_acid', 'nitrogen_dioxide', 'nitrogen_monoxide', 'olefins', 'organic_ethers', 'organic_matter_aerosol_optical_depth_550nm', 'organic_nitrates', 'ozone', 'paraffins', 'particulate_matter_10um', 'particulate_matter_1um', 'particulate_matter_2.5um', 'pernitric_acid', 'peroxides', 'peroxy_acetyl_radical', 'peroxyacetyl_nitrate', 'potential_vorticity', 'propane', 'propene', 'radon', 'relative_humidity', 'sea_ice_cover', 'sea_salt_aerosol_0.03-0.5um_mixing_ratio', 'sea_salt_aerosol_0.5-5um_mixing_ratio', 'sea_salt_aerosol_5-20um_mixing_ratio', 'sea_salt_aerosol_optical_depth_550nm', 'sea_surface_temperature', 'skin_reservoir_content', 'skin_temperature', 'snow_albedo', 'snow_depth', 'soil_clay_content', 'soil_type', 'specific_cloud_ice_water_content', 'specific_cloud_liquid_water_content', 'specific_humidity', 'specific_rain_water_content', 'specific_snow_water_content', 'stratospheric_ozone_tracer', 'sulphate_aerosol_mixing_ratio', 'sulphate_aerosol_optical_depth_550nm', 'sulphur_dioxide', 'surface_geopotential', 'surface_pressure', 'surface_roughness', 'temperature', 'terpenes', 'total_aerosol_optical_depth_1240nm', 'total_aerosol_optical_depth_469nm', 'total_aerosol_optical_depth_550nm', 'total_aerosol_optical_depth_670nm', 'total_aerosol_optical_depth_865nm', 'total_cloud_cover', 'total_column_acetone', 'total_column_aldehydes', 'total_column_carbon_monoxide', 'total_column_ethane', 'total_column_ethanol', 'total_column_ethene', 'total_column_formaldehyde', 'total_column_formic_acid', 'total_column_hydrogen_peroxide', 'total_column_hydroxyl_radical', 'total_column_isoprene', 'total_column_methane', 'total_column_methanol', 'total_column_methyl_peroxide', 'total_column_nitric_acid', 'total_column_nitrogen_dioxide', 'total_column_nitrogen_monoxide', 'total_column_olefins', 'total_column_organic_nitrates', 'total_column_ozone', 'total_column_paraffins', 'total_column_peroxyacetyl_nitrate', 'total_column_propane', 'total_column_sulphur_dioxide', 'total_column_water', 'total_column_water_vapour', 'type_of_high_vegetation', 'type_of_low_vegetation', 'u_component_of_wind', 'uv_visible_albedo_for_diffuse_radiation', 'uv_visible_albedo_for_direct_radiation', 'v_component_of_wind', 'vertical_velocity', 'vertically_integrated_mass_of_dust_aerosol_0.03-0.55um', 'vertically_integrated_mass_of_dust_aerosol_0.55-9um', 'vertically_integrated_mass_of_dust_aerosol_9-20um', 'vertically_integrated_mass_of_hydrophilic_black_carbon_aerosol', 'vertically_integrated_mass_of_hydrophilic_organic_matter_aerosol', 'vertically_integrated_mass_of_hydrophobic_black_carbon_aerosol', 'vertically_integrated_mass_of_hydrophobic_organic_matter_aerosol', 'vertically_integrated_mass_of_sea_salt_aerosol_0.03-0.5um', 'vertically_integrated_mass_of_sea_salt_aerosol_0.5-5um', 'vertically_integrated_mass_of_sea_salt_aerosol_5-20um', 'vertically_integrated_mass_of_sulphate_aerosol']],
FieldInfo(annotation=NoneType, required=True, alias_priority=2, validation_alias=AliasChoices(choices=['ecmwf:variable', 'variable']), serialization_alias='ecmwf:variable', title='Variable', description='Please, consult the product user guide in the documentation section for more information on these variables.')
]
'end':  typing.Annotated[str,  FieldInfo( 'required': False, ... )] typing.Annotated[
str,
FieldInfo(annotation=NoneType, required=False, default=None, alias='end_datetime', alias_priority=2, description="Date/time as string in ISO 8601 format (e.g. '2024-06-10T12:00:00Z')")
]
'start':  typing.Annotated[str,  FieldInfo( 'required': False, ... )] typing.Annotated[
str,
FieldInfo(annotation=NoneType, required=False, default=None, alias='start_datetime', alias_priority=2, description="Date/time as string in ISO 8601 format (e.g. '2024-06-10T12:00:00Z')")
]

Queryables values are returned using typing.Annoteted format. It contains parameter expected type, allowed alias for query, constraints and other metadata:

[4]:
queryables["ecmwf_data_format"]
[4]:
typing.Annotated[typing.Literal['grib', 'netcdf_zip'], FieldInfo(annotation=NoneType, required=True, alias_priority=2, validation_alias=AliasChoices(choices=['ecmwf:data_format', 'data_format']), serialization_alias='ecmwf:data_format', title='Data format', description='Please select a format for the data files, the native format of MARS dataset is GRIB.'), 'json_schema_required']
[5]:
from typing import get_args

queryable_type, *queryable_metadata= get_args(queryables["ecmwf_data_format"])
[6]:
queryable_type
[6]:
typing.Literal['grib', 'netcdf_zip']
[7]:
queryable_metadata[0].validation_alias
[7]:
AliasChoices(choices=['ecmwf:data_format', 'data_format'])
[8]:
queryable_metadata[0].is_required()
[8]:
True

Search from an existing collection:#

[9]:
products_from_collection = dag.search(
    provider="cop_ads",
    collection="CAMS_EAC4",
    variable=["2m_dewpoint_temperature"],
    data_format="grib",
    start="2021-01-01",
    end="2021-01-02",
    time=["00:00"],
)
products_from_collection
[9]:
SearchResult (1)
0  EOProduct(id=CAMS_EAC4_ORDERABLE_5bf2d5f16176c40994a963e611f42c653cbcccfa, provider=cop_ads)
EOProduct
provider: 'cop_ads',
collection: 'CAMS_EAC4',
properties["id"]: 'CAMS_EAC4_ORDERABLE_5bf2d5f16176c40994a963e611f42c653cbcccfa',
properties["start_datetime"]: '2021-01-01T00:00:00.000Z',
properties["end_datetime"]: '2021-01-02T00:00:00.000Z',
properties: (14){
datetime: '2021-01-01T00:00:00.000Z',
end_datetime: '2021-01-02T00:00:00.000Z',
id: 'CAMS_EAC4_ORDERABLE_5bf2d5f16176c40994a963e611f42c653cbcccfa',
qs: { 'data_format': 'grib' , 'date': '2021-01-01/2021-01-02' , 'time': ['00:00' ] , 'variable': ['2m_dewpoint_temperature' ] },
start_datetime: '2021-01-01T00:00:00.000Z',
title: 'CAMS_EAC4_ORDERABLE_5bf2d5f16176c40994a963e611f42c653cbcccfa',
ecmwf:data_format: 'grib',
ecmwf:dataset: 'cams-global-reanalysis-eac4',
ecmwf:date: '2021-01-01/2021-01-02',
ecmwf:time: ['00:00' ],
ecmwf:variable: ['2m_dewpoint_temperature' ],
eodag:default_geometry: 'POLYGON((180 -90, 180 90, -180 90, -180 -90, 180 -90))',
eodag:order_link: 'https://ads.atmosphere.copernicus.eu/api/retrieve/v1/processes/cams-global-reanalysis-eac4/execution?{"inputs": {"data_format": "grib", "date": "2021-01-01/2021-01-02", "time": ["00:00"], "variable": ["2m_dewpoint_temperature"]}}',
order:status: 'orderable',
}
assets: (0)
geometry

Search using a custom request:#

If you know the collection / dataset id from cop_ads provider, you can also directly use it as collection, you will get the same result:

[10]:
products_from_ads_req = dag.search(
    provider="cop_ads",
    collection="cams-global-reanalysis-eac4",
    variable=["2m_dewpoint_temperature"],
    data_format="grib",
    start="2021-01-01",
    end="2021-01-02",
    time=["00:00"],
)

# "eodag:order_link" property must be the same with the two request methods,
# as they are built from the same ADS request arguments
if (
    products_from_ads_req[0].properties["eodag:order_link"]
    == products_from_collection[0].properties["eodag:order_link"]
):
    print(
        "Request using collection or directly ADS dataset result to the\n",
        "same 'eodag:order_link' %s"
        % (
            products_from_ads_req[0].properties["eodag:order_link"],
        )
    )

Request using collection or directly ADS dataset result to the
 same 'eodag:order_link' https://ads.atmosphere.copernicus.eu/api/retrieve/v1/processes/cams-global-reanalysis-eac4/execution?{"inputs": {"data_format": "grib", "date": "2021-01-01/2021-01-02", "time": ["00:00"], "variable": ["2m_dewpoint_temperature"]}}

Send product retrieval request, download when available and return an xarray.DataArray#

  • download performed using ADS credentials set in ~/.config/eodag/eodag.yml as for other EO providers:

cop_ads:
    priority:
    download:
        output_dir: /my/path/to/data/eodag_data
    auth:
        credentials:
            apikey: myapikey
[11]:
# Get XarrayDict
xd = products_from_ads_req[0].to_xarray()
xd
[Retry #1] Waiting 11.999867s until next order status check (retry every 0.2' for 10')
[11]:
XarrayDict (1)
'CAMS-GLOBAL-REANALYSIS-EAC4_.grib':  xarray.Dataset (time: 2, latitude: 241, longitude: 480)  Size: 931kB
<xarray.Dataset> Size: 931kB
Dimensions:     (time: 2, latitude: 241, longitude: 480)
Coordinates:
    number      int64 8B ...
  * time        (time) datetime64[ns] 16B 2021-01-01 2021-01-02
    step        timedelta64[ns] 8B ...
    surface     float64 8B ...
  * latitude    (latitude) float64 2kB 90.0 89.25 88.5 ... -88.5 -89.25 -90.0
  * longitude   (longitude) float64 4kB 0.0 0.75 1.5 2.25 ... 357.8 358.5 359.2
    valid_time  (time) datetime64[ns] 16B ...
Data variables:
    d2m         (time, latitude, longitude) float32 925kB ...
Attributes: (12/31)
    GRIB_edition:            1
    GRIB_centre:             ecmf
    GRIB_centreDescription:  European Centre for Medium-Range Weather Forecasts
    GRIB_subCentre:          0
    Conventions:             CF-1.7
    institution:             European Centre for Medium-Range Weather Forecasts
    ...                      ...
    order:date:              2026-01-19T16:11:55.584412
    created:                 2026-01-19T16:11:55.584412
    updated:                 2026-01-19T16:12:06.736935
    eodag:request_params:    {'date': '2021-01-01/2021-01-02', 'grid': '0.75/...
    published:               2026-01-19T16:12:06.736935
    eodag:download_link:     https://object-store.os-api.cci2.ecmwf.int:443/c...
[12]:
# DataArray from XarrayDict first value
da = next(iter(xd.values())).d2m
da
[12]:
<xarray.DataArray 'd2m' (time: 2, latitude: 241, longitude: 480)> Size: 925kB
[231360 values with dtype=float32]
Coordinates:
    number      int64 8B ...
  * time        (time) datetime64[ns] 16B 2021-01-01 2021-01-02
    step        timedelta64[ns] 8B ...
    surface     float64 8B ...
  * latitude    (latitude) float64 2kB 90.0 89.25 88.5 ... -88.5 -89.25 -90.0
  * longitude   (longitude) float64 4kB 0.0 0.75 1.5 2.25 ... 357.8 358.5 359.2
    valid_time  (time) datetime64[ns] 16B ...
Attributes: (12/31)
    GRIB_paramId:                             168
    GRIB_dataType:                            an
    GRIB_numberOfPoints:                      115680
    GRIB_typeOfLevel:                         surface
    GRIB_stepUnits:                           1
    GRIB_stepType:                            instant
    ...                                       ...
    GRIB_shortName:                           2d
    GRIB_totalNumber:                         0
    GRIB_units:                               K
    long_name:                                2 metre dewpoint temperature
    units:                                    K
    standard_name:                            unknown

Plot using cartopy#

[13]:
import matplotlib.pyplot as plt
import cartopy.crs as ccrs

ax = plt.axes(projection=ccrs.Orthographic(0, 20))
ax.coastlines()
ax.gridlines(draw_labels=True)
da[0].plot.contourf(ax=ax, transform=ccrs.PlateCarree())
[13]:
<cartopy.mpl.contour.GeoContourSet at 0x70f3fa6c18e0>
../../_images/notebooks_tutos_tuto_cds_20_1.png