Hint

You can run this notebook in a live session with .

Copernicus Atmosphere using BuildSearchResult plugin

In this tutorial we will show you how to use eodag to download data from providers using BuildSearchResult eodag plugin. You can currently find two providers that uses it, cop_ads and cop_cds. For this tutorial we will use cop_ads, but cop_cds is used the same way.

from eodag import EODataAccessGateway, setup_logging

setup_logging(1)  # 0: nothing, 1: only progress bars, 2: INFO, 3: DEBUG
dag = EODataAccessGateway()
dag.set_preferred_provider("cop_ads")

Search (build download request)

There are two use case, a search for a product already configured in EODAG, or a search for a dataset not already configured, where you will have a little more to do.

We can add a variable (Temperature, temperature) and one model_level to the request because CAMS_EAC4 is configured to request for some pre-configured values. Check the dataset available values to make your selection.

Note: specifying your own variables will completely overwrite default values configured for CAMS_EAC4 product type.

Check available queryables and default values:

Available queryables parameters and associated values can be checked using list_queryables() method, or through cop_ads or cop_cds websites:

queryables = dag.list_queryables(provider="cop_ads", productType="CAMS_EAC4")
queryables["variable"]

typing.Annotated[typing.Literal['10m_u_component_of_wind', '10m_v_component_of_wind', '2m_dewpoint_temperature', '2m_temperature', 'acetone', 'acetone_product', 'aldehydes', 'amine', 'ammonia', 'ammonium', 'black_carbon_aerosol_optical_depth_550nm', 'carbon_monoxide', 'dimethyl_sulfide', 'dinitrogen_pentoxide', 'dust_aerosol_0.03-0.55um_mixing_ratio', 'dust_aerosol_0.55-0.9um_mixing_ratio', 'dust_aerosol_0.9-20um_mixing_ratio', 'dust_aerosol_optical_depth_550nm', 'ethane', 'ethanol', 'ethene', 'formaldehyde', 'formic_acid', 'fraction_of_cloud_cover', 'geopotential', 'high_cloud_cover', 'high_vegetation_cover', 'hydrogen_peroxide', 'hydroperoxy_radical', 'hydrophilic_black_carbon_aerosol_mixing_ratio', 'hydrophilic_organic_matter_aerosol_mixing_ratio', 'hydrophobic_black_carbon_aerosol_mixing_ratio', 'hydrophobic_organic_matter_aerosol_mixing_ratio', 'hydroxyl_radical', 'isoprene', 'lake_cover', 'land_sea_mask', 'lead', 'leaf_area_index_high_vegetation', 'leaf_area_index_low_vegetation', 'lifting_threshold_speed', 'low_cloud_cover', 'low_vegetation_cover', 'mean_altitude_of_maximum_injection', 'mean_sea_level_pressure', 'medium_cloud_cover', 'methacrolein_mvk', 'methacrylic_acid', 'methane_chemistry', 'methane_sulfonic_acid', 'methanol', 'methyl_glyoxal', 'methyl_peroxide', 'methylperoxy_radical', 'near_ir_albedo_for_diffuse_radiation', 'near_ir_albedo_for_direct_radiation', 'nitrate', 'nitrate_radical', 'nitric_acid', 'nitrogen_dioxide', 'nitrogen_monoxide', 'olefins', 'organic_ethers', 'organic_matter_aerosol_optical_depth_550nm', 'organic_nitrates', 'ozone', 'paraffins', 'particulate_matter_10um', 'particulate_matter_1um', 'particulate_matter_2.5um', 'pernitric_acid', 'peroxides', 'peroxy_acetyl_radical', 'peroxyacetyl_nitrate', 'potential_vorticity', 'propane', 'propene', 'radon', 'relative_humidity', 'sea_ice_cover', 'sea_salt_aerosol_0.03-0.5um_mixing_ratio', 'sea_salt_aerosol_0.5-5um_mixing_ratio', 'sea_salt_aerosol_5-20um_mixing_ratio', 'sea_salt_aerosol_optical_depth_550nm', 'sea_surface_temperature', 'skin_reservoir_content', 'skin_temperature', 'snow_albedo', 'snow_depth', 'soil_clay_content', 'soil_type', 'specific_cloud_ice_water_content', 'specific_cloud_liquid_water_content', 'specific_humidity', 'specific_rain_water_content', 'specific_snow_water_content', 'stratospheric_ozone_tracer', 'sulphate_aerosol_mixing_ratio', 'sulphate_aerosol_optical_depth_550nm', 'sulphur_dioxide', 'surface_geopotential', 'surface_pressure', 'surface_roughness', 'temperature', 'terpenes', 'total_aerosol_optical_depth_1240nm', 'total_aerosol_optical_depth_469nm', 'total_aerosol_optical_depth_550nm', 'total_aerosol_optical_depth_670nm', 'total_aerosol_optical_depth_865nm', 'total_cloud_cover', 'total_column_acetone', 'total_column_aldehydes', 'total_column_carbon_monoxide', 'total_column_ethane', 'total_column_ethanol', 'total_column_ethene', 'total_column_formaldehyde', 'total_column_formic_acid', 'total_column_hydrogen_peroxide', 'total_column_hydroxyl_radical', 'total_column_isoprene', 'total_column_methane', 'total_column_methanol', 'total_column_methyl_peroxide', 'total_column_nitric_acid', 'total_column_nitrogen_dioxide', 'total_column_nitrogen_monoxide', 'total_column_olefins', 'total_column_organic_nitrates', 'total_column_ozone', 'total_column_paraffins', 'total_column_peroxyacetyl_nitrate', 'total_column_propane', 'total_column_sulphur_dioxide', 'total_column_water', 'total_column_water_vapour', 'type_of_high_vegetation', 'type_of_low_vegetation', 'u_component_of_wind', 'uv_visible_albedo_for_diffuse_radiation', 'uv_visible_albedo_for_direct_radiation', 'v_component_of_wind', 'vertical_velocity', 'vertically_integrated_mass_of_dust_aerosol_0.03-0.55um', 'vertically_integrated_mass_of_dust_aerosol_0.55-9um', 'vertically_integrated_mass_of_dust_aerosol_9-20um', 'vertically_integrated_mass_of_hydrophilic_black_carbon_aerosol', 'vertically_integrated_mass_of_hydrophilic_organic_matter_aerosol', 'vertically_integrated_mass_of_hydrophobic_black_carbon_aerosol', 'vertically_integrated_mass_of_hydrophobic_organic_matter_aerosol', 'vertically_integrated_mass_of_sea_salt_aerosol_0.03-0.5um', 'vertically_integrated_mass_of_sea_salt_aerosol_0.5-5um', 'vertically_integrated_mass_of_sea_salt_aerosol_5-20um', 'vertically_integrated_mass_of_sulphate_aerosol'], FieldInfo(annotation=NoneType, required=False, default='2m_dewpoint_temperature')]

Here we can see the list of available values for variable, and that the default value configured for CAMS_EAC4 is 2m_dewpoint_temperature

Search from an existing product type:

# Request for all parameters
products_from_product_type, total_count = dag.search(
    start="2021-01-01",
    end="2021-01-02",
    productType="CAMS_EAC4",
)
print(
    "%s product built %s,\n having variable = %s\nand model_level=%s\n"
    % (
        total_count,
        products_from_product_type[0],
        products_from_product_type[0].properties.get("variable"),
        products_from_product_type[0].properties.get("model_level"),
    )
)
# Request for temperature on one model level
products_from_product_type, total_count = dag.search(
    start="2021-01-01",
    end="2021-01-02",
    productType="CAMS_EAC4",
    variable="temperature",
    model_level="1",
)
print(
    "%s product built %s,\n having variable = %s\nand model_level=%s\n"
    % (
        total_count,
        products_from_product_type[0],
        products_from_product_type[0].properties.get("variable"),
        products_from_product_type[0].properties.get("model_level"),
    )
)

1 product built EOProduct(id=CAMS_EAC4_20210101_e86a2b26cd2bcf90eaa14d1b9f58592169b10a36, provider=cop_ads),
 having variable = 2m_dewpoint_temperature
and model_level=None

1 product built EOProduct(id=CAMS_EAC4_20210101_5cc16f9d5dbf711e31cda8c77c6efc5414a1c4f1, provider=cop_ads),
 having variable = temperature
and model_level=1

Search using a custom request:

Here we will use a set of custom parameters corresponding to CAMS_EAC4, which should result to the same request sent to ads.

ads_req_params = {
    "dataset": "cams-global-reanalysis-eac4",
    "variable": "temperature",
    "model_level": "1",
    "time": "00:00",
    "format": "grib",
}

products_from_ads_req, total_count = dag.search(
    provider="cop_ads",
    start="2021-01-01",
    end="2021-01-02",
    **ads_req_params,
)
# orderLink property must be the same with the two request methods,
# as they are built from the same ADS request arguments
if (
    products_from_ads_req[0].properties["orderLink"]
    == products_from_product_type[0].properties["orderLink"]
):
    print(
        "Request using productType or directly ADS parameters result to the\n",
        "same orderLink %s"
        % (
            products_from_ads_req[0].properties["orderLink"],
        )
    )

Request using productType or directly ADS parameters result to the
 same orderLink https://ads.atmosphere.copernicus.eu/api/v2/resources/cams-global-reanalysis-eac4?{"date": "2021-01-01/2021-01-01", "format": "grib", "model_level": 1, "time": "00:00", "variable": "temperature"}

Send product retrieval request and download when available

• download performed using ADS credentials set in ~/.config/eodag/eodag.yml as for other EO providers:

cop_ads:
    priority:
    download:
        outputs_prefix: /my/path/to/data/eodag_data
    auth:
        credentials:
            username: my-ads-uid
            password: my-ads-api-key

• you can check your request status from https://ads.atmosphere.copernicus.eu/cdsapp#!/yourrequests

• See support for any problem related to the provider

product_path = dag.download(products_from_product_type[0], wait=0.2)
product_path

'/data/eodag_data/CAMS_EAC4_20210101_5cc16f9d5dbf711e31cda8c77c6efc5414a1c4f1'

Open dataset with xarray and cfgrib, then plot using matplotlib

import os
import xarray as xr

# the product ouput file to load is the only one located in "product_path" directory
ds = xr.load_dataset(os.path.join(product_path, os.listdir(product_path)[0]), engine="cfgrib")
ds

<xarray.Dataset>
Dimensions:     (latitude: 241, longitude: 480)
Coordinates:
    time        datetime64[ns] 2021-01-01
    step        timedelta64[ns] 00:00:00
    hybrid      float64 1.0
  * latitude    (latitude) float64 90.0 89.25 88.5 87.75 ... -88.5 -89.25 -90.0
  * longitude   (longitude) float64 0.0 0.75 1.5 2.25 ... 357.8 358.5 359.2
    valid_time  datetime64[ns] 2021-01-01
Data variables:
    t           (latitude, longitude) float32 257.2 257.2 257.2 ... 243.7 243.7
Attributes:
    GRIB_edition:            2
    GRIB_centre:             ecmf
    GRIB_centreDescription:  European Centre for Medium-Range Weather Forecasts
    GRIB_subCentre:          0
    Conventions:             CF-1.7
    institution:             European Centre for Medium-Range Weather Forecasts
    history:                 2024-04-08T21:46 GRIB to CDM+CF via cfgrib-0.9.1...

xarray.Dataset

• Dimensions:
  • latitude: 241
  • longitude: 480
• Coordinates: (6)
  • time
    ()
    datetime64[ns]
    2021-01-01

    long_name :

    initial time of forecast

    standard_name :

    forecast_reference_time

    array('2021-01-01T00:00:00.000000000', dtype='datetime64[ns]')

  • step
    ()
    timedelta64[ns]
    00:00:00

    long_name :

    time since forecast_reference_time

    standard_name :

    forecast_period

    array(0, dtype='timedelta64[ns]')

  • hybrid
    ()
    float64
    1.0

    long_name :

    hybrid level

    units :

    1

    positive :

    down

    standard_name :

    atmosphere_hybrid_sigma_pressure_coordinate

    array(1.)

  • latitude
    (latitude)
    float64
    90.0 89.25 88.5 ... -89.25 -90.0

    units :

    degrees_north

    standard_name :

    latitude

    long_name :

    latitude

    stored_direction :

    decreasing

    array([ 90.  ,  89.25,  88.5 , ..., -88.5 , -89.25, -90.  ])

  • longitude
    (longitude)
    float64
    0.0 0.75 1.5 ... 357.8 358.5 359.2

    units :

    degrees_east

    standard_name :

    longitude

    long_name :

    longitude

    array([  0.  ,   0.75,   1.5 , ..., 357.75, 358.5 , 359.25])

  • valid_time
    ()
    datetime64[ns]
    2021-01-01

    standard_name :

    time

    long_name :

    time

    array('2021-01-01T00:00:00.000000000', dtype='datetime64[ns]')

• Data variables: (1)
  • t
    (latitude, longitude)
    float32
    257.2 257.2 257.2 ... 243.7 243.7

    GRIB_paramId :

    130

    GRIB_dataType :

    an

    GRIB_numberOfPoints :

    115680

    GRIB_typeOfLevel :

    hybrid

    GRIB_stepUnits :

    1

    GRIB_stepType :

    instant

    GRIB_gridType :

    regular_ll

    GRIB_NV :

    122

    GRIB_Nx :

    480

    GRIB_Ny :

    241

    GRIB_cfName :

    air_temperature

    GRIB_cfVarName :

    t

    GRIB_gridDefinitionDescription :

    Latitude/longitude

    GRIB_iDirectionIncrementInDegrees :

    0.75

    GRIB_iScansNegatively :

    0

    GRIB_jDirectionIncrementInDegrees :

    0.75

    GRIB_jPointsAreConsecutive :

    0

    GRIB_jScansPositively :

    0

    GRIB_latitudeOfFirstGridPointInDegrees :

    90.0

    GRIB_latitudeOfLastGridPointInDegrees :

    -90.0

    GRIB_longitudeOfFirstGridPointInDegrees :

    0.0

    GRIB_longitudeOfLastGridPointInDegrees :

    359.25

    GRIB_missingValue :

    3.4028234663852886e+38

    GRIB_name :

    Temperature

    GRIB_shortName :

    t

    GRIB_units :

    K

    long_name :

    Temperature

    units :

    K

    standard_name :

    air_temperature

    array([[257.1941 , 257.1941 , 257.1941 , ..., 257.1941 , 257.1941 ,
            257.1941 ],
           [258.28687, 258.28687, 258.28687, ..., 258.2849 , 258.2859 ,
            258.2859 ],
           [259.25073, 259.2527 , 259.25464, ..., 259.24194, 259.24585,
            259.2478 ],
           ...,
           [243.4861 , 243.48512, 243.48317, ..., 243.49196, 243.49   ,
            243.48805],
           [243.63454, 243.63454, 243.63356, ..., 243.63747, 243.63649,
            243.63551],
           [243.7488 , 243.7488 , 243.7488 , ..., 243.7488 , 243.7488 ,
            243.7488 ]], dtype=float32)

• Indexes: (2)
  • latitude
    PandasIndex

    PandasIndex(Index([  90.0,  89.25,   88.5,  87.75,   87.0,  86.25,   85.5,  84.75,   84.0,
            83.25,
           ...
           -83.25,  -84.0, -84.75,  -85.5, -86.25,  -87.0, -87.75,  -88.5, -89.25,
            -90.0],
          dtype='float64', name='latitude', length=241))

  • longitude
    PandasIndex

    PandasIndex(Index([   0.0,   0.75,    1.5,   2.25,    3.0,   3.75,    4.5,   5.25,    6.0,
             6.75,
           ...
            352.5, 353.25,  354.0, 354.75,  355.5, 356.25,  357.0, 357.75,  358.5,
           359.25],
          dtype='float64', name='longitude', length=480))

• Attributes: (7)

  GRIB_edition :

  2

  GRIB_centre :

  ecmf

  GRIB_centreDescription :

  European Centre for Medium-Range Weather Forecasts

  GRIB_subCentre :

  0

  Conventions :

  CF-1.7

  institution :

  European Centre for Medium-Range Weather Forecasts

  history :

  2024-04-08T21:46 GRIB to CDM+CF via cfgrib-0.9.11.0/ecCodes-2.24.2 with {"source": "../../../../../../../data/eodag_data/CAMS_EAC4_20210101_5cc16f9d5dbf711e31cda8c77c6efc5414a1c4f1/CAMS_EAC4_20210101_5cc16f9d5dbf711e31cda8c77c6efc5414a1c4f1.grib", "filter_by_keys": {}, "encode_cf": ["parameter", "time", "geography", "vertical"]}

ds.t.plot()

<matplotlib.collections.QuadMesh at 0x764b4d4151e0>