EODAG as STAC client
Contents
EODAG as STAC client#
STAC API#
EODAG can perform an item search over a STAC compliant API. Found STAC items are returned as EOProduct objects with STAC metadata mapped to OGC OpenSearch Extension for Earth Observation.
EODAG comes with already configured providers, but you can also add new ones dynamically.
[1]:
import os
# To have some basic feedback on what eodag is doing, we configure logging to output minimum information
from eodag import setup_logging
setup_logging(verbose=2)
from eodag.api.core import EODataAccessGateway
dag = EODataAccessGateway()
2021-03-08 18:07:00,193-15s eodag.config [INFO ] Loading user configuration from: /home/sylvain/.config/eodag/eodag.yml
2021-03-08 18:07:00,405-15s eodag.core [INFO ] Locations configuration loaded from /home/sylvain/.config/eodag/locations.yml
List already configured providers providing a STAC API compliant service:
[2]:
[p.name for p in dag.providers_config.values() if hasattr(p, "search") and p.search.type == 'StacSearch']
[2]:
['astraea_eod', 'earth_search', 'usgs_satapi_aws']
Then, let’s update EODAG’s configuration with a new STAC provider
[3]:
dag.update_providers_config("""
tamn:
search:
type: StacSearch
api_endpoint: https://tamn.snapplanet.io/search
products:
S2_MSI_L1C:
productType: S2
GENERIC_PRODUCT_TYPE:
productType: '{productType}'
download:
type: AwsDownload
base_uri: https://tamn.snapplanet.io
flatten_top_dirs: True
auth:
type: AwsAuth
credentials:
aws_access_key_id: PLEASE_CHANGE_ME
aws_secret_access_key: PLEASE_CHANGE_ME
""")
dag.set_preferred_provider("tamn")
2021-03-08 18:07:00,425-15s eodag.config [INFO ] tamn: unknown provider found in user conf, trying to use provided configuration
Search some S2_MSI_L1C products over Luxembourg:
[4]:
prods_S2L1C, _ = dag.search(productType="S2_MSI_L1C", locations=dict(country="LUX"), start="2020-05-01", end="2020-05-15", items_per_page=50)
2021-03-08 18:07:00,602-15s eodag.core [INFO ] Searching product type 'S2_MSI_L1C' on provider: tamn
2021-03-08 18:07:00,603-15s eodag.plugins.search.qssearch [INFO ] Sending count request: https://tamn.snapplanet.io/search?datetime=2020-05-01T00:00:00.000Z/2020-05-15T00:00:00.000Z&bbox=5.674051954784829,49.44266714130711,6.242751092156993,50.128051662794235&collections=S2&limit=1&page=1
2021-03-08 18:07:00,821-15s eodag.plugins.search.qssearch [INFO ] Sending search request: https://tamn.snapplanet.io/search?datetime=2020-05-01T00:00:00.000Z/2020-05-15T00:00:00.000Z&bbox=5.674051954784829,49.44266714130711,6.242751092156993,50.128051662794235&collections=S2&limit=50&page=1
2021-03-08 18:07:01,525-15s eodag.core [INFO ] Found 35 result(s) on provider 'tamn'
Filter over any item property using crunchers:
[5]:
from eodag.plugins.crunch.filter_property import FilterProperty
prods_S2L1C_filtered = prods_S2L1C.crunch(FilterProperty({"cloudCover": 10, "operator": "lt"}))
len(prods_S2L1C_filtered)
2021-03-08 18:07:01,528-15s eodag.plugins.crunch.filter_property [INFO ] Finished filtering products. 11 resulting products
[5]:
11
List available assets from the first retrieved product
[6]:
[(key, asset["href"]) for key, asset in prods_S2L1C[0].assets.items()]
[6]:
[('thumbnail',
'https://roda.sentinel-hub.com/sentinel-s2-l1c/tiles/31/U/GQ/2020/5/14/0/preview.jpg'),
('metadata',
'https://roda.sentinel-hub.com/sentinel-s2-l1c/tiles/31/U/GQ/2020/5/14/0/metadata.xml'),
('tileInfo',
'https://roda.sentinel-hub.com/sentinel-s2-l1c/tiles/31/U/GQ/2020/5/14/0/tileInfo.json'),
('productInfo',
'https://roda.sentinel-hub.com/sentinel-s2-l1c/tiles/31/U/GQ/2020/5/14/0/productInfo.json'),
('B1', 's3://sentinel-s2-l1c/tiles/31/U/GQ/2020/5/14/0/B1.jp2'),
('B2', 's3://sentinel-s2-l1c/tiles/31/U/GQ/2020/5/14/0/B2.jp2'),
('B3', 's3://sentinel-s2-l1c/tiles/31/U/GQ/2020/5/14/0/B3.jp2'),
('B4', 's3://sentinel-s2-l1c/tiles/31/U/GQ/2020/5/14/0/B4.jp2'),
('B5', 's3://sentinel-s2-l1c/tiles/31/U/GQ/2020/5/14/0/B5.jp2'),
('B6', 's3://sentinel-s2-l1c/tiles/31/U/GQ/2020/5/14/0/B6.jp2'),
('B7', 's3://sentinel-s2-l1c/tiles/31/U/GQ/2020/5/14/0/B7.jp2'),
('B8', 's3://sentinel-s2-l1c/tiles/31/U/GQ/2020/5/14/0/B8.jp2'),
('B8A', 's3://sentinel-s2-l1c/tiles/31/U/GQ/2020/5/14/0/B8A.jp2'),
('B9', 's3://sentinel-s2-l1c/tiles/31/U/GQ/2020/5/14/0/B9.jp2'),
('B10', 's3://sentinel-s2-l1c/tiles/31/U/GQ/2020/5/14/0/B10.jp2'),
('B11', 's3://sentinel-s2-l1c/tiles/31/U/GQ/2020/5/14/0/B11.jp2'),
('B12', 's3://sentinel-s2-l1c/tiles/31/U/GQ/2020/5/14/0/B12.jp2')]
Same thing with an unconfigured product type (should match available collections).
For Tamn Landsat-8 products are available in L8 Collection. Let’s search them over Spain:
[7]:
prods_L8, _ = dag.search(productType="L8", country="ESP", start="2020-05-01", end="2020-05-15")
2021-03-08 18:07:01,562-15s eodag.plugins.manager [INFO ] UnsupportedProductType: L8, using generic settings
2021-03-08 18:07:01,562-15s eodag.core [INFO ] Searching product type 'L8' on provider: tamn
2021-03-08 18:07:01,563-15s eodag.plugins.search.qssearch [INFO ] Sending count request: https://tamn.snapplanet.io/search?datetime=2020-05-01T00:00:00.000Z/2020-05-15T00:00:00.000Z&bbox=-9.392883673530648,35.946850083961465,3.0394840836805486,43.74833771420099&collections=L8&limit=1&page=1
2021-03-08 18:07:01,830-15s eodag.plugins.search.qssearch [INFO ] Sending search request: https://tamn.snapplanet.io/search?datetime=2020-05-01T00:00:00.000Z/2020-05-15T00:00:00.000Z&bbox=-9.392883673530648,35.946850083961465,3.0394840836805486,43.74833771420099&collections=L8&limit=20&page=1
2021-03-08 18:07:02,393-15s eodag.core [INFO ] Found 98 result(s) on provider 'tamn'
[8]:
[(key, asset["href"]) for key, asset in prods_L8[0].assets.items()]
[8]:
[('thumbnail',
'https://landsat-pds.s3.amazonaws.com/c1/L8/196/035/LC08_L1TP_196035_20200514_20200527_01_T1/LC08_L1TP_196035_20200514_20200527_01_T1_thumb_large.jpg'),
('metadata',
'https://landsat-pds.s3.amazonaws.com/c1/L8/196/035/LC08_L1TP_196035_20200514_20200527_01_T1/LC08_L1TP_196035_20200514_20200527_01_T1_MTL.txt'),
('B1',
'https://landsat-pds.s3.amazonaws.com/c1/L8/196/035/LC08_L1TP_196035_20200514_20200527_01_T1/LC08_L1TP_196035_20200514_20200527_01_T1_B1.TIF'),
('B2',
'https://landsat-pds.s3.amazonaws.com/c1/L8/196/035/LC08_L1TP_196035_20200514_20200527_01_T1/LC08_L1TP_196035_20200514_20200527_01_T1_B2.TIF'),
('B3',
'https://landsat-pds.s3.amazonaws.com/c1/L8/196/035/LC08_L1TP_196035_20200514_20200527_01_T1/LC08_L1TP_196035_20200514_20200527_01_T1_B3.TIF'),
('B4',
'https://landsat-pds.s3.amazonaws.com/c1/L8/196/035/LC08_L1TP_196035_20200514_20200527_01_T1/LC08_L1TP_196035_20200514_20200527_01_T1_B4.TIF'),
('B5',
'https://landsat-pds.s3.amazonaws.com/c1/L8/196/035/LC08_L1TP_196035_20200514_20200527_01_T1/LC08_L1TP_196035_20200514_20200527_01_T1_B5.TIF'),
('B6',
'https://landsat-pds.s3.amazonaws.com/c1/L8/196/035/LC08_L1TP_196035_20200514_20200527_01_T1/LC08_L1TP_196035_20200514_20200527_01_T1_B6.TIF'),
('B7',
'https://landsat-pds.s3.amazonaws.com/c1/L8/196/035/LC08_L1TP_196035_20200514_20200527_01_T1/LC08_L1TP_196035_20200514_20200527_01_T1_B7.TIF'),
('B8',
'https://landsat-pds.s3.amazonaws.com/c1/L8/196/035/LC08_L1TP_196035_20200514_20200527_01_T1/LC08_L1TP_196035_20200514_20200527_01_T1_B8.TIF'),
('B9',
'https://landsat-pds.s3.amazonaws.com/c1/L8/196/035/LC08_L1TP_196035_20200514_20200527_01_T1/LC08_L1TP_196035_20200514_20200527_01_T1_B9.TIF'),
('B10',
'https://landsat-pds.s3.amazonaws.com/c1/L8/196/035/LC08_L1TP_196035_20200514_20200527_01_T1/LC08_L1TP_196035_20200514_20200527_01_T1_B10.TIF'),
('B11',
'https://landsat-pds.s3.amazonaws.com/c1/L8/196/035/LC08_L1TP_196035_20200514_20200527_01_T1/LC08_L1TP_196035_20200514_20200527_01_T1_B11.TIF')]
STAC Static catalog#
EODAG can search for items over a STAC static catalog. Path to the catalog must be set as provider.search.api_endpoint
with provider.search.type=StaticStacSearch
. A download plugin must also be set, and depends of the provider.
Here is an example with a catalog from https://stacindex.org/catalogs/spot-orthoimages-canada-2005, which will use HTTPDownload
as download plugin, without credentials as no authentication needed for download.
See download plugins documentation for other available plugins.
Warning
Please note that StaticStacSearch
plugin development is still at an early stage. If search is too slow using this plugin, please use a catalog with less elements.
[9]:
# Decrease logging level
setup_logging(verbose=1)
# create a workspace
workspace = 'eodag_workspace_stac_client'
if not os.path.isdir(workspace):
os.mkdir(workspace)
# add the provider
dag.update_providers_config("""
stac_http_provider:
search:
type: StaticStacSearch
api_endpoint: https://canada-spot-ortho.s3.amazonaws.com/canada_spot_orthoimages/canada_spot5_orthoimages/S5_2007/catalog.json
products:
GENERIC_PRODUCT_TYPE:
productType: '{productType}'
download:
type: HTTPDownload
base_uri: https://fake-endpoint
flatten_top_dirs: True
outputs_prefix: %s
""" % os.path.abspath(workspace))
dag.set_preferred_provider("stac_http_provider")
Let’s perform search :
[10]:
from shapely.geometry import Polygon
search_polygon = Polygon([(-70, 45), (-75, 47), (-80, 47), (-80, 44)])
query_args = {"start": "2007-05-01", "end": "2007-05-06" , "geom": search_polygon}
products, found = dag.search(**query_args)
print("%s product(s) found" % found)
3 product(s) found
Before downloading, make some cleanup in the products as canada-spot-ortho.s3.amazonaws.com
thumbnails are not available for download : - remove thumbnail assets - remove products with no assets
[11]:
for idx, product in enumerate(products):
# remove thumbnail
if "thumbnail" in product.assets:
del products[idx].assets["thumbnail"]
# remove items with empty assets
if not product.assets:
del products[idx]
print("%s product(s) with valid assets" % len(products))
2 product(s) with valid assets
[12]:
# plot products and search polygon on map
import ipyleaflet as ipyl
m = ipyl.Map(center=(45, -75), zoom=5)
polygon_layer = ipyl.GeoJSON(data=search_polygon.__geo_interface__, style=dict(color='blue'))
m.add_layer(polygon_layer)
items_layer = ipyl.GeoJSON(data=products.as_geojson_object(), style=dict(color='green'))
m.add_layer(items_layer)
m
Download items from the filtered search results:
[13]:
paths = dag.download_all(products)
paths
[13]:
['/home/sylvain/workspace/eodag/examples/eodag_workspace_stac_client/S5_07702_4605_20070505',
'/home/sylvain/workspace/eodag/examples/eodag_workspace_stac_client/S5_07724_4507_20070505']
[14]:
for path in paths:
!basename $path
!ls {path.replace("file://","")}
S5_07702_4605_20070505
S5_07702_4605_20070505_m20_1_lcc00_cog.tif
S5_07702_4605_20070505_m20_2_lcc00_cog.tif
S5_07702_4605_20070505_m20_3_lcc00_cog.tif
S5_07702_4605_20070505_m20_4_lcc00_cog.tif
S5_07702_4605_20070505_p10_1_lcc00_cog.tif
S5_07724_4507_20070505
s5_07724_4507_20070505_m20_1_lcc00_cog.tif
s5_07724_4507_20070505_m20_2_lcc00_cog.tif
s5_07724_4507_20070505_m20_3_lcc00_cog.tif
s5_07724_4507_20070505_m20_4_lcc00_cog.tif
s5_07724_4507_20070505_p10_1_lcc00_cog.tif