Ctrl+K

Overview

Hint

You can run this notebook in a live session with Binder.

Overview#

This overview demonstrates the basic capabilities of eodag in a simple worflow, whose steps are introduced in more details in the following pages.

The workflow consists of the following steps:

  1. Configure: eodag is configured to use the provider PEPS (registering to PEPS is required to download the products, see how to register to a provider)

  2. Search: Sentinel 2 Level-1C products (i.e. images) are searched for over an area in France in March 2021.

  3. Crunch: Two dedicated filters - that we call crunchers - provided by eodag are used to select products with a cloud cover less than 30% and covering the city of Toulouse (France).

  4. Serialize: Save the filtered products as a GeoJSON file.

  5. Download: The products are downloaded.

  6. Post-process: eodag-cube is an external package that is used to access a product’s data, it is going to be used to calculate the NDVI of a product.

Configure#

Let us first create a workspace directory to save products downloaded during this workflow.

[1]:

import os

workspace = 'eodag_workspace_overview'
if not os.path.isdir(workspace):
    os.mkdir(workspace)

Now we will configure eodag to be able to download using PEPS. For that we need to fill our credentials:

  • in the user configuration file ~/.config/eodag.eodag.yml:

peps:
    priority: # Lower value means lower priority (Default: 1)
    search:  # Search parameters configuration
    download:
        extract:  # whether to extract the downloaded products (true or false, Default: true).
        outputs_prefix: # where to store downloaded products.
        dl_url_params:  # additional parameters to pass over to the download url as an url parameter
        delete_archive: # whether to delete the downloaded archives (true or false, Default: true).
    auth:
        credentials:
            username: PLEASE_CHANGE_ME
            password: PLEASE_CHANGE_ME

  • or using environment variables: (we also set outputs_prefix, the directory where to store downloaded products)

[2]:

# os.environ["EODAG__PEPS__AUTH__CREDENTIALS__USERNAME"] = "PLEASE_CHANGE_ME"
# os.environ["EODAG__PEPS__AUTH__CREDENTIALS__PASSWORD"] = "PLEASE_CHANGE_ME"
os.environ["EODAG__PEPS__DOWNLOAD__OUTPUTS_PREFIX"] = os.path.abspath(workspace)

Logging is then activated with the setup_logging() method. It’s a useful way to see what eodag does under the hood, e.g. requesting the provider, adapting the response. It’s also useful to detect when things go wrong, and, if relevant, create an issue on GitHub with the log messages.

[3]:

from eodag import setup_logging
setup_logging(2)  # 3 for even more information

The next object to import, and this is certainly one of the most important objects provided by eodag, is the EODataAccessGateway class. The creation of a single instance of this class is enough in a workflow, it is going to take care of configuring the providers, exposing the products configured off-the-shelf by eodag, and many more things.

[4]:

from eodag import EODataAccessGateway
dag = EODataAccessGateway()

2023-10-17 16:51:54,405 eodag.config                     [INFO    ] Loading user configuration from: /home/anesson/.config/eodag/eodag.yml
2023-10-17 16:51:54,592 eodag.core                       [INFO    ] Locations configuration loaded from /home/anesson/.config/eodag/locations.yml

eodag stores an internal catalog of products it makes easily accessible. Sentinel 2 Level-1C products are designated with the identifier S2_MSI_L1C. It’s possible to check that it’s made available by PEPS with the method list_product_types() which returns a list of dictionnaries, each one of them containing general metadata (eodag’s product type ID, platform, sensor type, etc.).

[5]:

[product_type["ID"] for product_type in dag.list_product_types("peps")]

[5]:

['S1_SAR_GRD', 'S1_SAR_OCN', 'S1_SAR_SLC', 'S2_MSI_L1C', 'S2_MSI_L2A']

The method available_providers() allows to get the list of providers that make available a given product.

[6]:

dag.available_providers("S2_MSI_L1C")

[6]:

['astraea_eod',
 'aws_eos',
 'cop_dataspace',
 'creodias',
 'earth_search',
 'earth_search_gcs',
 'onda',
 'peps',
 'sara',
 'usgs',
 'wekeo']

Finally PEPS is declared as the preferred provider, which means that eodag will look for products with this provider (all the providers are pre-configured).

[7]:

dag.set_preferred_provider("peps")

Crunch#

Crunching as defined in eodag is a way to filter the products contained in a SearchResult object. Several crunchers/filters are available (e.g. FilterProperty to filter products products according to one of their properties). They can be called by passing the cruncher object to SearchResult.crunch() or by running a dedicated method (e.g. SearchResult.filter_property()). The following example shows how to filter products to keep only those whose cloud cover is less than 30%.

[16]:

filtered_products = all_products.filter_property(cloudCover=30, operator="lt")

2023-10-17 16:51:58,539 eodag.crunch.property            [INFO    ] Finished filtering products. 27 resulting products

[17]:

print(f"Got now {len(filtered_products)} products after filtering by cloud coverage.")

Got now 27 products after filtering by cloud coverage.

Now, we use another cruncher to filter products containing a given geometry (Toulouse, France). We will use these products in Post-process.

[18]:

filtered_products = filtered_products.filter_overlap(geometry=(1.435905, 43.586857, 1.458907, 43.603827), contains=True)

2023-10-17 16:51:58,940 eodag.crunch.overlap             [INFO    ] Finished filtering products. 3 resulting products

[19]:

print(f"Got now {len(filtered_products)} products after filtering by geometry.")

Got now 3 products after filtering by geometry.

Serialize#

The method serialize() allows to save a SearchResult as a GeoJSON file.

[20]:

filtered_prods_filepath = dag.serialize(filtered_products, filename=os.path.join(workspace, "filtered_products.geojson"))

This method can come in handy to save the state of a search and restore it later with deserialize_and_register().

[21]:

restored_filtered_prods = dag.deserialize_and_register(filtered_prods_filepath)

Download#

Before downloading any product, it can be useful to have a quick look at them. EO products usually offer a quicklook image, a low resolution by-product of the original data. An EOProduct has a get_quicklook method that takes care of downloading the quicklook image and returns its path. matplotlib can be used here to display the images collected.

[22]:

import matplotlib.pyplot as plt
import matplotlib.image as mpimg

quicklooks_dir = os.path.join(workspace, "quicklooks")
if not os.path.isdir(quicklooks_dir):
    os.mkdir(quicklooks_dir)

fig = plt.figure(figsize=(10, 8))
for i, product in enumerate(filtered_products, start=1):

    # This line takes care of downloading the quicklook
    quicklook_path = product.get_quicklook(base_dir=quicklooks_dir)

    img = mpimg.imread(quicklook_path)
    ax = fig.add_subplot(1, 3, i)
    ax.set_title(i)
    plt.imshow(img)
plt.tight_layout()

2023-10-17 16:52:00,327 eodag.product                    [INFO    ] Download recorded in /home/anesson/workspace/EODAG/dev/eodag/docs/notebooks/api_user_guide/eodag_workspace_overview/quicklooks/S2B_MSIL1C_20210321T104639_N0209_R051_T31TCJ_20210321T130616

2023-10-17 16:52:00,791 eodag.product                    [INFO    ] Download recorded in /home/anesson/workspace/EODAG/dev/eodag/docs/notebooks/api_user_guide/eodag_workspace_overview/quicklooks/S2B_MSIL1C_20210321T104639_N0500_R051_T31TCJ_20230526T002554

2023-10-17 16:52:01,129 eodag.product                    [INFO    ] Download recorded in /home/anesson/workspace/EODAG/dev/eodag/docs/notebooks/api_user_guide/eodag_workspace_overview/quicklooks/S2B_MSIL1C_20210301T104859_N0500_R051_T31TCJ_20230604T191601
../../_images/notebooks_api_user_guide_1_overview_47_6.png

EOProducts can either be downloaded individually with download() or together with download_all() from a SearchResult. The last image is going to be downloaded, it is cloud-free and has no no-data pixel.

[23]:

product_to_download = filtered_products[-1]
product_path = dag.download(product_to_download)
product_path

2023-10-17 16:52:02,381 eodag.download.base              [INFO    ] Download url: https://peps.cnes.fr/resto/collections/S2ST/569f40e6-b634-5a63-ad5b-7ddc242a39dc/download

2023-10-17 16:59:38,005 eodag.download.base              [INFO    ] Extraction activated
2023-10-17 16:59:40,653 eodag.download.base              [INFO    ] Deleting archive S2B_MSIL1C_20210301T104859_N0500_R051_T31TCJ_20230604T191601.zip
2023-10-17 16:59:40,812 eodag.product                    [INFO    ] Remote location of the product is still available through its 'remote_location' property: https://peps.cnes.fr/resto/collections/S2ST/569f40e6-b634-5a63-ad5b-7ddc242a39dc/download

[23]:

'/home/anesson/workspace/EODAG/dev/eodag/docs/notebooks/api_user_guide/eodag_workspace_overview/S2B_MSIL1C_20210301T104859_N0500_R051_T31TCJ_20230604T191601'

The location property of this product now points to a local path.

[24]:

product_to_download.location

[24]:

'file:///home/anesson/workspace/EODAG/dev/eodag/docs/notebooks/api_user_guide/eodag_workspace_overview/S2B_MSIL1C_20210301T104859_N0500_R051_T31TCJ_20230604T191601'

Post-process#

Now the product is downloaded, it can be post-processed with softwares such as Sen2Cor or SNAP.

At some point eodag had some capabilities to directly post-process a product, i.e. to access its data. These capabilities, which relied on rasterio, have been ported to the Python package eodag-cube to avoid the heavy dependencies associated with GDAL in particular. Installing this package is enough to benefit from its capabilities, it is going to extend EOProduct with a get_data() method which returns a product’s image band as a xarray.DataArray.

The capabilities of eodag-cube are used hereafter to compute the NDVI of the downloaded product over a sub-extent of the original product (this is actually Toulouse, France).

Warning

eodag-cube needs to be installed to run correcly the following code.

[25]:

import warnings
import rasterio

# mute rasterio warnings
warnings.filterwarnings("ignore", category=rasterio.errors.NotGeoreferencedWarning)

crs = "epsg:4326"
resolution = 0.0001
sub_extent = (1.435905, 43.586857, 1.458907, 43.603827)
VIR = product_to_download.get_data(band="B04", extent=sub_extent, crs=crs, resolution=resolution)
NIR = product_to_download.get_data(band="B08", extent=sub_extent, crs=crs, resolution=resolution)
NDVI = (NIR - VIR * 1.) / (NIR + VIR)

The NDVI can quickly be plotted directly with xarray which uses matplotlib under the hood.

[26]:

NDVI.plot(cmap="RdYlGn", center=False)

[26]:

<matplotlib.collections.QuadMesh at 0x7fce8fb66a60>
../../_images/notebooks_api_user_guide_1_overview_57_1.png
Contents