Hint

You can run this notebook in a live session with Binder.

Overview#

This overview demonstrates the basic capabilities of eodag in a simple worflow, whose steps are introduced in more details in the following pages.

The workflow consists of the following steps:

  1. Configure: eodag is configured to use the provider PEPS (registering to PEPS is required to download the products, see how to register to a provider)

  2. Search: Sentinel 2 Level-1C products (i.e. images) are searched for over an area in France in March 2021.

  3. Crunch: A decicated filter - that we call cruncher - provided by eodag is used to select products with a cloud cover less than 10%.

  4. Serialize: Save the filtered products as a GeoJSON file.

  5. Download: The products are downloaded.

  6. Post-process: eodag-cube is an external package that is used to access a product’s data, it is going to be used to calculate the NDVI of a product.

Configure#

Let us first create a workspace directory to save products downloaded during this workflow.

[1]:

import os

workspace = 'eodag_workspace_overview'
if not os.path.isdir(workspace):
    os.mkdir(workspace)

Now we will configure eodag to be able to download using PEPS. For that we need to fill our credentials:

  • in the user configuration file ~/.config/eodag.eodag.yml:

peps:
    priority: # Lower value means lower priority (Default: 1)
    search:  # Search parameters configuration
    download:
        extract:  # whether to extract the downloaded products (true or false).
        outputs_prefix: # where to store downloaded products.
        dl_url_params:  # additional parameters to pass over to the download url as an url parameter
        delete_archive: # whether to delete the downloaded archives (true or false, Default: true).
    auth:
        credentials:
            username: PLEASE_CHANGE_ME
            password: PLEASE_CHANGE_ME

  • or using environment variables: (we also set outputs_prefix, the directory where to store downloaded products)

[2]:

os.environ["EODAG__PEPS__AUTH__CREDENTIALS__USERNAME"] = "PLEASE_CHANGE_ME"
os.environ["EODAG__PEPS__AUTH__CREDENTIALS__PASSWORD"] = "PLEASE_CHANGE_ME"
os.environ["EODAG__PEPS__DOWNLOAD__OUTPUTS_PREFIX"] = os.path.abspath(workspace)

Logging is then activated with the setup_logging() method. It’s a useful way to see what eodag does under the hood, e.g. requesting the provider, adapting the response. It’s also useful to detect when things go wrong, and, if relevant, create an issue on GitHub with the log messages.

[3]:

from eodag import setup_logging
setup_logging(2)  # 3 for even more information

The next object to import, and this is certainly one of the most important objects provided by eodag, is the EODataAccessGateway class. The creation of a single instance of this class is enough in a workflow, it is going to take care of configuring the providers, exposing the products configured off-the-shelf by eodag, and many more things.

[4]:

from eodag import EODataAccessGateway
dag = EODataAccessGateway()

2023-01-13 10:54:32,231 eodag.config                     [INFO    ] Loading user configuration from: /home/sylvain/.config/eodag/eodag.yml
2023-01-13 10:54:32,461 eodag.core                       [INFO    ] Locations configuration loaded from /home/sylvain/.config/eodag/locations.yml

eodag stores an internal catalog of products it makes easily accessible. Sentinel 2 Level-1C products are designated with the identifier S2_MSI_L1C. It’s possible to check that it’s made available by PEPS with the method list_product_types() which returns a list of dictionnaries, each one of them containing general metadata (eodag’s product type ID, platform, sensor type, etc.).

[5]:

[product_type["ID"] for product_type in dag.list_product_types("peps")]

[5]:

['S1_SAR_GRD', 'S1_SAR_OCN', 'S1_SAR_SLC', 'S2_MSI_L1C', 'S2_MSI_L2A']

The method available_providers() allows to get the list of providers that make available a given product.

[6]:

dag.available_providers("S2_MSI_L1C")

[6]:

['astraea_eod',
 'aws_eos',
 'creodias',
 'earth_search',
 'earth_search_gcs',
 'mundi',
 'onda',
 'peps',
 'sara',
 'usgs']

Finally PEPS is declared as the preferred provider, which means that eodag will look for products with this provider (all the providers are pre-configured).

[7]:

dag.set_preferred_provider("peps")

Crunch#

Crunching as defined in eodag is a way to filter the products contained in a SearchResult object. Several crunchers/filters are available (e.g. FilterProperty to filter products products according to one of their properties). They can be called by passing the cruncher object to SearchResult.crunch() or by running a dedicated method (e.g. SearchResult.filter_property()). The following example shows how to filter products to keep only those whose cloud cover is less than 10%.

[18]:

filtered_products = all_products.filter_property(cloudCover=10, operator="lt")

2021-04-20 18:02:44,440-15s eodag.plugins.crunch.filter_property [INFO    ] Finished filtering products. 10 resulting products

[19]:

print(f"Got now {len(filtered_products)} products after filtering.")

Got now 10 products after filtering.

Serialize#

The method serialize() allows to save a SearchResult as a GeoJSON file.

[20]:

filtered_prods_filepath = dag.serialize(filtered_products, filename=os.path.join(workspace, "filtered_products.geojson"))

This method can come in handy to save the state of a search and restore it later with deserialize_and_register().

[21]:

# restored_filtered_prods = dag.deserialize_and_register(filtered_prods_filepath)

Download#

Before downloading any product, it can be useful to have a quick look at them. EO products usually offer a quicklook image, a low resolution by-product of the original data. An EOProduct has a get_quicklook method that takes care of downloading the quicklook image and returns its path. matplotlib can be used here to display the images collected.

[23]:

import matplotlib.pyplot as plt
import matplotlib.image as mpimg

quicklooks_dir = os.path.join(workspace, "quicklooks")
if not os.path.isdir(quicklooks_dir):
    os.mkdir(quicklooks_dir)

fig = plt.figure(figsize=(10, 8))
for i, product in enumerate(filtered_products, start=1):

    # This line takes care of downloading the quicklook
    quicklook_path = product.get_quicklook(base_dir=quicklooks_dir)

    img = mpimg.imread(quicklook_path)
    ax = fig.add_subplot(3, 4, i)
    ax.set_title(i)
    plt.imshow(img)
plt.tight_layout()

2021-04-20 18:02:45,068-15s eodag.api.product                [INFO    ] Download recorded in /home/maxime/TRAVAIL/06_EODAG/01_eodag/eodag/docs/notebooks/api_user_guide/eodag_workspace_overview/quicklooks/S2B_MSIL1C_20210328T103629_N0209_R008_T31TCJ_20210328T124650

2021-04-20 18:02:45,335-15s eodag.api.product                [INFO    ] Download recorded in /home/maxime/TRAVAIL/06_EODAG/01_eodag/eodag/docs/notebooks/api_user_guide/eodag_workspace_overview/quicklooks/S2B_MSIL1C_20210328T103629_N0209_R008_T31TCH_20210328T124650

2021-04-20 18:02:45,636-15s eodag.api.product                [INFO    ] Download recorded in /home/maxime/TRAVAIL/06_EODAG/01_eodag/eodag/docs/notebooks/api_user_guide/eodag_workspace_overview/quicklooks/S2B_MSIL1C_20210328T103629_N0209_R008_T31TDH_20210328T124650





2021-04-20 18:02:45,947-15s eodag.api.product                [INFO    ] Download recorded in /home/maxime/TRAVAIL/06_EODAG/01_eodag/eodag/docs/notebooks/api_user_guide/eodag_workspace_overview/quicklooks/S2B_MSIL1C_20210328T103629_N0209_R008_T31TDJ_20210328T124650



2021-04-20 18:02:46,255-15s eodag.api.product                [INFO    ] Download recorded in /home/maxime/TRAVAIL/06_EODAG/01_eodag/eodag/docs/notebooks/api_user_guide/eodag_workspace_overview/quicklooks/S2A_MSIL1C_20210323T104021_N0209_R008_T31TCJ_20210323T141236



2021-04-20 18:02:46,530-15s eodag.api.product                [INFO    ] Download recorded in /home/maxime/TRAVAIL/06_EODAG/01_eodag/eodag/docs/notebooks/api_user_guide/eodag_workspace_overview/quicklooks/S2A_MSIL1C_20210323T104021_N0209_R008_T31TDJ_20210323T141236



2021-04-20 18:02:46,802-15s eodag.api.product                [INFO    ] Download recorded in /home/maxime/TRAVAIL/06_EODAG/01_eodag/eodag/docs/notebooks/api_user_guide/eodag_workspace_overview/quicklooks/S2A_MSIL1C_20210323T104021_N0209_R008_T31TDH_20210323T141236



2021-04-20 18:02:47,165-15s eodag.api.product                [INFO    ] Download recorded in /home/maxime/TRAVAIL/06_EODAG/01_eodag/eodag/docs/notebooks/api_user_guide/eodag_workspace_overview/quicklooks/S2B_MSIL1C_20210301T104859_N0209_R051_T31TCH_20210301T115849

2021-04-20 18:02:47,455-15s eodag.api.product                [INFO    ] Download recorded in /home/maxime/TRAVAIL/06_EODAG/01_eodag/eodag/docs/notebooks/api_user_guide/eodag_workspace_overview/quicklooks/S2B_MSIL1C_20210301T104859_N0209_R051_T31TDJ_20210301T115849



2021-04-20 18:02:47,739-15s eodag.api.product                [INFO    ] Download recorded in /home/maxime/TRAVAIL/06_EODAG/01_eodag/eodag/docs/notebooks/api_user_guide/eodag_workspace_overview/quicklooks/S2B_MSIL1C_20210301T104859_N0209_R051_T31TCJ_20210301T115849
../../_images/notebooks_api_user_guide_1_overview_44_27.png

EOProducts can either be downloaded individually with download() or together with download_all() from a SearchResult. The last image is going to be downloaded, it is cloud-free and has no no-data pixel.

[24]:

product_to_download = filtered_products[-1]
product_path = dag.download(product_to_download)
product_path

2021-04-20 18:02:48,884-15s eodag.plugins.download.base      [INFO    ] Download url: https://peps.cnes.fr/resto/collections/S2ST/f913081f-eead-5709-ab65-cbaa1884150c/download

2021-04-20 18:04:11,370-15s eodag.plugins.download.base      [INFO    ] Extraction activated

2021-04-20 18:04:12,745-15s eodag.api.product                [INFO    ] Remote location of the product is still available through its 'remote_location' property: https://peps.cnes.fr/resto/collections/S2ST/f913081f-eead-5709-ab65-cbaa1884150c/download




[24]:

'file:///home/maxime/TRAVAIL/06_EODAG/01_eodag/eodag/docs/notebooks/api_user_guide/eodag_workspace_overview/S2B_MSIL1C_20210301T104859_N0209_R051_T31TCJ_20210301T115849/S2B_MSIL1C_20210301T104859_N0209_R051_T31TCJ_20210301T115849.SAFE'

The location property of this product now points to a local path.

[25]:

product_to_download.location

[25]:

'file:///home/maxime/TRAVAIL/06_EODAG/01_eodag/eodag/docs/notebooks/api_user_guide/eodag_workspace_overview/S2B_MSIL1C_20210301T104859_N0209_R051_T31TCJ_20210301T115849/S2B_MSIL1C_20210301T104859_N0209_R051_T31TCJ_20210301T115849.SAFE'

Post-process#

Now the product is downloaded, it can be post-processed with softwares such as Sen2Cor or SNAP.

At some point eodag had some capabilities to directly post-process a product, i.e. to access its data. These capabilities, which relied on rasterio, have been ported to the Python package eodag-cube to avoid the heavy dependencies associated with GDAL in particular. Installing this package is enough to benefit from its capabilities, it is going to extend EOProduct with a get_data() method which returns a product’s image band as a xarray.DataArray.

The capabilities of eodag-cube are used hereafter to compute the NDVI of the downloaded product over a sub-extent of the original product (this is actually Toulouse, France).

Warning

eodag-cube needs to be installed to run correcly the following code.

[26]:

crs = "epsg:4326"
resolution = 0.0001
sub_extent = (1.435905, 43.586857, 1.458907, 43.603827)
VIR = product_to_download.get_data(band="B04", extent=sub_extent, crs=crs, resolution=resolution)
NIR = product_to_download.get_data(band="B08", extent=sub_extent, crs=crs, resolution=resolution)
NDVI = (NIR - VIR * 1.) / (NIR + VIR)

The NDVI can quickly be plotted directly with xarray which uses matplotlib under the hood.

[27]:

NDVI.plot(cmap="RdYlGn", center=False)

[27]:

<matplotlib.collections.QuadMesh at 0x7fb6e5f7da90>
../../_images/notebooks_api_user_guide_1_overview_54_1.png