Ctrl+K

EODAG as STAC client

Hint

You can run this notebook in a live session with Binder.

EODAG as STAC client#

STAC API#

EODAG can perform an item search over a STAC compliant API. Found STAC items are returned as EOProduct objects with STAC metadata mapped to OGC OpenSearch Extension for Earth Observation.

EODAG comes with already configured providers, but you can also add new ones dynamically.

[1]:

import os
# To have some basic feedback on what eodag is doing, we configure logging to output minimum information
from eodag import setup_logging
setup_logging(verbose=2)

from eodag.api.core import EODataAccessGateway

dag = EODataAccessGateway()

2021-03-08 18:07:00,193-15s eodag.config                     [INFO    ] Loading user configuration from: /home/sylvain/.config/eodag/eodag.yml
2021-03-08 18:07:00,405-15s eodag.core                       [INFO    ] Locations configuration loaded from /home/sylvain/.config/eodag/locations.yml

List already configured providers providing a STAC API compliant service:

[2]:

[p.name for p in dag.providers_config.values() if hasattr(p, "search") and p.search.type == 'StacSearch']

[2]:

['astraea_eod', 'earth_search', 'usgs_satapi_aws']

Then, let’s update EODAG’s configuration with a new STAC provider

[3]:

dag.update_providers_config("""
    tamn:
        search:
            type: StacSearch
            api_endpoint: https://tamn.snapplanet.io/search
        products:
            S2_MSI_L1C:
                productType: S2
            GENERIC_PRODUCT_TYPE:
                productType: '{productType}'
        download:
            type: AwsDownload
            base_uri: https://tamn.snapplanet.io
            flatten_top_dirs: True
        auth:
            type: AwsAuth
            credentials:
                aws_access_key_id: PLEASE_CHANGE_ME
                aws_secret_access_key: PLEASE_CHANGE_ME
""")
dag.set_preferred_provider("tamn")

2021-03-08 18:07:00,425-15s eodag.config                     [INFO    ] tamn: unknown provider found in user conf, trying to use provided configuration

Search some S2_MSI_L1C products over Luxembourg:

[4]:

prods_S2L1C, _ = dag.search(productType="S2_MSI_L1C", locations=dict(country="LUX"), start="2020-05-01", end="2020-05-15", items_per_page=50)

2021-03-08 18:07:00,602-15s eodag.core                       [INFO    ] Searching product type 'S2_MSI_L1C' on provider: tamn
2021-03-08 18:07:00,603-15s eodag.plugins.search.qssearch    [INFO    ] Sending count request: https://tamn.snapplanet.io/search?datetime=2020-05-01T00:00:00.000Z/2020-05-15T00:00:00.000Z&bbox=5.674051954784829,49.44266714130711,6.242751092156993,50.128051662794235&collections=S2&limit=1&page=1
2021-03-08 18:07:00,821-15s eodag.plugins.search.qssearch    [INFO    ] Sending search request: https://tamn.snapplanet.io/search?datetime=2020-05-01T00:00:00.000Z/2020-05-15T00:00:00.000Z&bbox=5.674051954784829,49.44266714130711,6.242751092156993,50.128051662794235&collections=S2&limit=50&page=1
2021-03-08 18:07:01,525-15s eodag.core                       [INFO    ] Found 35 result(s) on provider 'tamn'

Filter over any item property using crunchers:

[5]:

from eodag.plugins.crunch.filter_property import FilterProperty

prods_S2L1C_filtered = prods_S2L1C.crunch(FilterProperty({"cloudCover": 10, "operator": "lt"}))
len(prods_S2L1C_filtered)

2021-03-08 18:07:01,528-15s eodag.plugins.crunch.filter_property [INFO    ] Finished filtering products. 11 resulting products

[5]:

11

List available assets from the first retrieved product

[6]:

[(key, asset["href"]) for key, asset in prods_S2L1C[0].assets.items()]

[6]:

[('thumbnail',
  'https://roda.sentinel-hub.com/sentinel-s2-l1c/tiles/31/U/GQ/2020/5/14/0/preview.jpg'),
 ('metadata',
  'https://roda.sentinel-hub.com/sentinel-s2-l1c/tiles/31/U/GQ/2020/5/14/0/metadata.xml'),
 ('tileInfo',
  'https://roda.sentinel-hub.com/sentinel-s2-l1c/tiles/31/U/GQ/2020/5/14/0/tileInfo.json'),
 ('productInfo',
  'https://roda.sentinel-hub.com/sentinel-s2-l1c/tiles/31/U/GQ/2020/5/14/0/productInfo.json'),
 ('B1', 's3://sentinel-s2-l1c/tiles/31/U/GQ/2020/5/14/0/B1.jp2'),
 ('B2', 's3://sentinel-s2-l1c/tiles/31/U/GQ/2020/5/14/0/B2.jp2'),
 ('B3', 's3://sentinel-s2-l1c/tiles/31/U/GQ/2020/5/14/0/B3.jp2'),
 ('B4', 's3://sentinel-s2-l1c/tiles/31/U/GQ/2020/5/14/0/B4.jp2'),
 ('B5', 's3://sentinel-s2-l1c/tiles/31/U/GQ/2020/5/14/0/B5.jp2'),
 ('B6', 's3://sentinel-s2-l1c/tiles/31/U/GQ/2020/5/14/0/B6.jp2'),
 ('B7', 's3://sentinel-s2-l1c/tiles/31/U/GQ/2020/5/14/0/B7.jp2'),
 ('B8', 's3://sentinel-s2-l1c/tiles/31/U/GQ/2020/5/14/0/B8.jp2'),
 ('B8A', 's3://sentinel-s2-l1c/tiles/31/U/GQ/2020/5/14/0/B8A.jp2'),
 ('B9', 's3://sentinel-s2-l1c/tiles/31/U/GQ/2020/5/14/0/B9.jp2'),
 ('B10', 's3://sentinel-s2-l1c/tiles/31/U/GQ/2020/5/14/0/B10.jp2'),
 ('B11', 's3://sentinel-s2-l1c/tiles/31/U/GQ/2020/5/14/0/B11.jp2'),
 ('B12', 's3://sentinel-s2-l1c/tiles/31/U/GQ/2020/5/14/0/B12.jp2')]

Same thing with an unconfigured product type (should match available collections).

For Tamn Landsat-8 products are available in L8 Collection. Let’s search them over Spain:

[7]:

prods_L8, _ = dag.search(productType="L8", country="ESP", start="2020-05-01", end="2020-05-15")

2021-03-08 18:07:01,562-15s eodag.plugins.manager            [INFO    ] UnsupportedProductType: L8, using generic settings
2021-03-08 18:07:01,562-15s eodag.core                       [INFO    ] Searching product type 'L8' on provider: tamn
2021-03-08 18:07:01,563-15s eodag.plugins.search.qssearch    [INFO    ] Sending count request: https://tamn.snapplanet.io/search?datetime=2020-05-01T00:00:00.000Z/2020-05-15T00:00:00.000Z&bbox=-9.392883673530648,35.946850083961465,3.0394840836805486,43.74833771420099&collections=L8&limit=1&page=1
2021-03-08 18:07:01,830-15s eodag.plugins.search.qssearch    [INFO    ] Sending search request: https://tamn.snapplanet.io/search?datetime=2020-05-01T00:00:00.000Z/2020-05-15T00:00:00.000Z&bbox=-9.392883673530648,35.946850083961465,3.0394840836805486,43.74833771420099&collections=L8&limit=20&page=1
2021-03-08 18:07:02,393-15s eodag.core                       [INFO    ] Found 98 result(s) on provider 'tamn'

[8]:

[(key, asset["href"]) for key, asset in prods_L8[0].assets.items()]

[8]:

[('thumbnail',
  'https://landsat-pds.s3.amazonaws.com/c1/L8/196/035/LC08_L1TP_196035_20200514_20200527_01_T1/LC08_L1TP_196035_20200514_20200527_01_T1_thumb_large.jpg'),
 ('metadata',
  'https://landsat-pds.s3.amazonaws.com/c1/L8/196/035/LC08_L1TP_196035_20200514_20200527_01_T1/LC08_L1TP_196035_20200514_20200527_01_T1_MTL.txt'),
 ('B1',
  'https://landsat-pds.s3.amazonaws.com/c1/L8/196/035/LC08_L1TP_196035_20200514_20200527_01_T1/LC08_L1TP_196035_20200514_20200527_01_T1_B1.TIF'),
 ('B2',
  'https://landsat-pds.s3.amazonaws.com/c1/L8/196/035/LC08_L1TP_196035_20200514_20200527_01_T1/LC08_L1TP_196035_20200514_20200527_01_T1_B2.TIF'),
 ('B3',
  'https://landsat-pds.s3.amazonaws.com/c1/L8/196/035/LC08_L1TP_196035_20200514_20200527_01_T1/LC08_L1TP_196035_20200514_20200527_01_T1_B3.TIF'),
 ('B4',
  'https://landsat-pds.s3.amazonaws.com/c1/L8/196/035/LC08_L1TP_196035_20200514_20200527_01_T1/LC08_L1TP_196035_20200514_20200527_01_T1_B4.TIF'),
 ('B5',
  'https://landsat-pds.s3.amazonaws.com/c1/L8/196/035/LC08_L1TP_196035_20200514_20200527_01_T1/LC08_L1TP_196035_20200514_20200527_01_T1_B5.TIF'),
 ('B6',
  'https://landsat-pds.s3.amazonaws.com/c1/L8/196/035/LC08_L1TP_196035_20200514_20200527_01_T1/LC08_L1TP_196035_20200514_20200527_01_T1_B6.TIF'),
 ('B7',
  'https://landsat-pds.s3.amazonaws.com/c1/L8/196/035/LC08_L1TP_196035_20200514_20200527_01_T1/LC08_L1TP_196035_20200514_20200527_01_T1_B7.TIF'),
 ('B8',
  'https://landsat-pds.s3.amazonaws.com/c1/L8/196/035/LC08_L1TP_196035_20200514_20200527_01_T1/LC08_L1TP_196035_20200514_20200527_01_T1_B8.TIF'),
 ('B9',
  'https://landsat-pds.s3.amazonaws.com/c1/L8/196/035/LC08_L1TP_196035_20200514_20200527_01_T1/LC08_L1TP_196035_20200514_20200527_01_T1_B9.TIF'),
 ('B10',
  'https://landsat-pds.s3.amazonaws.com/c1/L8/196/035/LC08_L1TP_196035_20200514_20200527_01_T1/LC08_L1TP_196035_20200514_20200527_01_T1_B10.TIF'),
 ('B11',
  'https://landsat-pds.s3.amazonaws.com/c1/L8/196/035/LC08_L1TP_196035_20200514_20200527_01_T1/LC08_L1TP_196035_20200514_20200527_01_T1_B11.TIF')]

STAC Static catalog#

EODAG can search for items over a STAC static catalog. Path to the catalog must be set as provider.search.api_endpoint with provider.search.type=StaticStacSearch. A download plugin must also be set, and depends of the provider.

Here is an example with a catalog from https://stacindex.org/catalogs/spot-orthoimages-canada-2005, which will use HTTPDownload as download plugin, without credentials as no authentication needed for download.

See download plugins documentation for other available plugins.

Warning

Please note that StaticStacSearch plugin development is still at an early stage. If search is too slow using this plugin, please use a catalog with less elements.

[9]:

# Decrease logging level
setup_logging(verbose=1)

# create a workspace
workspace = 'eodag_workspace_stac_client'
if not os.path.isdir(workspace):
    os.mkdir(workspace)

# add the provider
dag.update_providers_config("""
stac_http_provider:
    search:
        type: StaticStacSearch
        api_endpoint: https://canada-spot-ortho.s3.amazonaws.com/canada_spot_orthoimages/canada_spot5_orthoimages/S5_2007/catalog.json
    products:
        GENERIC_PRODUCT_TYPE:
            productType: '{productType}'
    download:
        type: HTTPDownload
        base_uri: https://fake-endpoint
        flatten_top_dirs: True
        outputs_prefix: %s
""" % os.path.abspath(workspace))

dag.set_preferred_provider("stac_http_provider")

Let’s perform search :

[10]:

from shapely.geometry import Polygon

search_polygon = Polygon([(-70, 45), (-75, 47), (-80, 47), (-80, 44)])
query_args = {"start": "2007-05-01", "end": "2007-05-06" , "geom": search_polygon}

products, found = dag.search(**query_args)
print("%s product(s) found" % found)

3 product(s) found

Before downloading, make some cleanup in the products as canada-spot-ortho.s3.amazonaws.com thumbnails are not available for download : - remove thumbnail assets - remove products with no assets

[11]:

for idx, product in enumerate(products):
    # remove thumbnail
    if "thumbnail" in product.assets:
        del products[idx].assets["thumbnail"]
    # remove items with empty assets
    if not product.assets:
        del products[idx]
print("%s product(s) with valid assets" % len(products))

2 product(s) with valid assets

[12]:

# plot products and search polygon on map
import ipyleaflet as ipyl

m = ipyl.Map(center=(45, -75), zoom=5)

polygon_layer = ipyl.GeoJSON(data=search_polygon.__geo_interface__, style=dict(color='blue'))
m.add_layer(polygon_layer)

items_layer = ipyl.GeoJSON(data=products.as_geojson_object(), style=dict(color='green'))
m.add_layer(items_layer)
m

Download items from the filtered search results:

[13]:

paths = dag.download_all(products)
paths

[13]:

['/home/sylvain/workspace/eodag/examples/eodag_workspace_stac_client/S5_07702_4605_20070505',
 '/home/sylvain/workspace/eodag/examples/eodag_workspace_stac_client/S5_07724_4507_20070505']

[14]:

for path in paths:
    !basename $path
    !ls {path.replace("file://","")}

S5_07702_4605_20070505
S5_07702_4605_20070505_m20_1_lcc00_cog.tif
S5_07702_4605_20070505_m20_2_lcc00_cog.tif
S5_07702_4605_20070505_m20_3_lcc00_cog.tif
S5_07702_4605_20070505_m20_4_lcc00_cog.tif
S5_07702_4605_20070505_p10_1_lcc00_cog.tif
S5_07724_4507_20070505
s5_07724_4507_20070505_m20_1_lcc00_cog.tif
s5_07724_4507_20070505_m20_2_lcc00_cog.tif
s5_07724_4507_20070505_m20_3_lcc00_cog.tif
s5_07724_4507_20070505_m20_4_lcc00_cog.tif
s5_07724_4507_20070505_p10_1_lcc00_cog.tif
Contents