{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# Search for products by tile" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Built-in search by tile\n", "\n", "Many providers already support search by tile (*Sentinel 2 MGRS tiling grid*):\n", "`peps`, `theia`, `onda`, `creodias`, `cop_dataspace`, `planetary_computer`, `earth_search`.\n", "\n", "For these providers, you can use `tileIdentifier` as search parameter using EODAG:" ] }, { "cell_type": "code", "execution_count": 1, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "[(EOProduct(id=S2A_MSIL1C_20180614T103021_N0206_R108_T31TFK_20180614T124154, provider=peps),\n", " '31TFK'),\n", " (EOProduct(id=S2B_MSIL1C_20180612T104019_N0206_R008_T31TFK_20180612T124717, provider=peps),\n", " '31TFK'),\n", " (EOProduct(id=S2B_MSIL1C_20180609T103019_N0206_R108_T31TFK_20180609T131037, provider=peps),\n", " '31TFK'),\n", " (EOProduct(id=S2A_MSIL1C_20180607T104021_N0206_R008_T31TFK_20180607T124742, provider=peps),\n", " '31TFK'),\n", " (EOProduct(id=S2A_MSIL1C_20180604T103021_N0206_R108_T31TFK_20180604T141551, provider=peps),\n", " '31TFK'),\n", " (EOProduct(id=S2B_MSIL1C_20180602T104019_N0206_R008_T31TFK_20180602T132118, provider=peps),\n", " '31TFK')]\n" ] } ], "source": [ "from pprint import pprint\n", "from eodag import EODataAccessGateway, setup_logging\n", "\n", "dag = EODataAccessGateway()\n", "dag.set_preferred_provider(\"peps\")\n", "products, _ = dag.search(\n", " productType=\"S2_MSI_L1C\", \n", " start=\"2018-06-01\", \n", " end=\"2018-06-15\", \n", " tileIdentifier=\"31TFK\"\n", ")\n", "pprint([(product, product.properties[\"tileIdentifier\"]) for product in products])" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Search by tile using custom locations\n", "`eodag` allows to search for products by geometric features that match a *location query*, see the [API user guide](../api_user_guide/4_search.ipynb#locations-search) for an introduction to this concept.\n", "\n", "In this tutorial we will use a shapefile that represents the Sentinel 2 tiling grid to search for *Sentinel 2 Level-1C* products with *PEPS* **at a specific tile**. In this shapefile each tile is defined by its centroid and a `tile_id` attribute (e.g. *29PMT*). This shapefile was created by downloading first the Sentinel 2 tiling grid (MGRS) provided [by ESA as a KML file](https://web.archive.org/web/20200907072744/https://sentinel.esa.int/web/sentinel/missions/sentinel-2/data-products). It was then converted as a shapefile and processed to compute the centroids. We use the tile's centroid here as `eodag` returns products that intersects the user defined search area. Since tiles overlap with each other, using the polygons instead of the centroids would return more tiles than just the one we target. " ] }, { "cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [], "source": [ "import os\n", "from zipfile import ZipFile\n", "\n", "# Interactive mapping\n", "import folium\n", "from folium.plugins import TimestampedGeoJson\n", "# pyshp: to read shapefiles\n", "import shapefile" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Setup" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "A workspace directory is created to store the files that will be generated." ] }, { "cell_type": "code", "execution_count": 3, "metadata": {}, "outputs": [], "source": [ "workspace = \"eodag_workspace_locations_tiles\"\n", "if not os.path.isdir(workspace):\n", " os.mkdir(workspace)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "You should have an `auxdata` folder next to this tutorial's file. It contains a shapefile that is needed to run this tutorial correctly." ] }, { "cell_type": "code", "execution_count": 4, "metadata": {}, "outputs": [], "source": [ "sentinel2_grid_zip = os.path.join(\"auxdata\", \"sentinel2_tiling_grid_centroids.zip\")\n", "if not os.path.isfile(sentinel2_grid_zip):\n", " raise FileNotFoundError(\"Auxdata not found, please check your configuration.\")" ] }, { "cell_type": "code", "execution_count": 5, "metadata": {}, "outputs": [], "source": [ "# We unzip the archived shapefile.\n", "with ZipFile(sentinel2_grid_zip, \"r\") as fzip:\n", " fzip.extractall(\"auxdata\")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "In this tutorial products will just be searched for, not downloaded. We don't need to set up PEPS credentials to search for products. If you wish to download them, you should set the credentials beforehand, using these two environment variables for instance." ] }, { "cell_type": "code", "execution_count": 6, "metadata": {}, "outputs": [], "source": [ "# os.environ[\"EODAG__PEPS__AUTH__CREDENTIALS__USERNAME\"] = \"PLEASE_CHANGE_ME\"\n", "# os.environ[\"EODAG__PEPS__AUTH__CREDENTIALS__PASSWORD\"] = \"PLEASE_CHANGE_ME\"" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Logging is activated to better inspect what `eodag` does internally." ] }, { "cell_type": "code", "execution_count": 7, "metadata": {}, "outputs": [], "source": [ "setup_logging(2) # INFO level" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The default search criteria consists of a time period in June 2018 and `eodag`'s product type identifier for *Sentinel 2 Level-1C* products." ] }, { "cell_type": "code", "execution_count": 8, "metadata": {}, "outputs": [], "source": [ "default_search_criteria = dict(\n", " productType=\"S2_MSI_L1C\",\n", " start=\"2018-06-01\",\n", " end=\"2018-06-15\"\n", ")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Add a locations configuration" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We check and store the content of this shapefile." ] }, { "cell_type": "code", "execution_count": 9, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "shapefile Reader\n", " 56686 shapes (type 'POINT')\n", " 56686 records (2 fields) \n", "\n", "fields: [('DeletionFlag', 'C', 1, 0), ['tile_id', 'C', 5, 0]]\n" ] } ], "source": [ "sentinel2_shp = os.path.join('auxdata', 'sentinel2_tiling_grid_centroids.shp')\n", "with shapefile.Reader(sentinel2_shp) as shp:\n", " print(shp, \"\\n\")\n", " print(\"fields:\", shp.fields)\n", " shaperecs = shp.shapeRecords()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "It has about 57 000 tiles/polygons and a field `tile_id`.\n", "\n", "We create a YAML file to configure this new location selector, we will refer to it with `s2_tile_centroid`." ] }, { "cell_type": "code", "execution_count": 10, "metadata": {}, "outputs": [], "source": [ "# Save the locations configuration file.\n", "locations_yaml_content = \"\"\"\n", "shapefiles:\n", " - name: s2_tile_centroid\n", " path: {}\n", " attr: tile_id\n", "\"\"\".format(os.path.abspath(sentinel2_shp))\n", "\n", "locations_filepath = os.path.abspath(os.path.join(workspace, \"custom_locations.yml\"))\n", "\n", "with open(locations_filepath, \"w\") as f_yml:\n", " f_yml.write(locations_yaml_content.strip())" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "An instance of an [EODataAccessGateway](../../api_reference/core.rst#eodag.api.core.EODataAccessGateway) class is created, it makes use of this location configuration file." ] }, { "cell_type": "code", "execution_count": 11, "metadata": {}, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "2023-10-17 15:39:30,945 eodag.config [INFO ] Loading user configuration from: /home/sylvain/.config/eodag/eodag.yml\n", "2023-10-17 15:39:31,063 eodag.core [INFO ] Locations configuration loaded from /home/sylvain/workspace/eodag/docs/notebooks/tutos/eodag_workspace_locations_tiles/custom_locations.yml\n" ] } ], "source": [ "dag = EODataAccessGateway(locations_conf_path=locations_filepath)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We want to look for *Sentinel 2 Level-1C* products. We can check whether this product type is offered by *PEPS* (as configured in `eodag`). If so, *PEPS* is set as the provider used to search for products." ] }, { "cell_type": "code", "execution_count": 12, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "True" ] }, "execution_count": 12, "metadata": {}, "output_type": "execute_result" } ], "source": [ "\"peps\" in dag.available_providers(\"S2_MSI_L1C\")" ] }, { "cell_type": "code", "execution_count": 13, "metadata": {}, "outputs": [], "source": [ "dag.set_preferred_provider(\"peps\")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Search" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### A single tile" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Our target tile is `31TFK` and is located in the South-East of France. Its feature is retrieved from the shapefil to be displayed later on an interactive map." ] }, { "cell_type": "code", "execution_count": 14, "metadata": {}, "outputs": [], "source": [ "targeted_tile_name = \"31TFK\"\n", "\n", "# Get the targeted tile feature\n", "targeted_tile = [\n", " sr\n", " for sr in shaperecs\n", " if sr.record[\"tile_id\"] == \"31TFK\"\n", "][0]" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We search for all the products that intersect with the centroid of this tile." ] }, { "cell_type": "code", "execution_count": 15, "metadata": {}, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "2023-10-17 15:39:41,948 eodag.core [INFO ] Searching product type 'S2_MSI_L1C' on provider: peps\n", "2023-10-17 15:39:41,949 eodag.core [INFO ] Iterate search over multiple pages: page #1\n", "2023-10-17 15:39:41,950 eodag.search.qssearch [INFO ] Sending search request: https://peps.cnes.fr/resto/api/collections/S2ST/search.json?startDate=2018-06-01&completionDate=2018-06-15&geometry=POINT (4.9533 44.6422)&productType=S2MSI1C&maxRecords=500&page=1\n", "2023-10-17 15:39:42,796 eodag.core [INFO ] Found 6 result(s) on provider 'peps'\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "6 were found given the above search criteria\n" ] } ], "source": [ "products = dag.search_all(\n", " locations=dict(s2_tile_centroid=\"31TFK\"),\n", " **default_search_criteria\n", ")\n", "print(f\"{len(products)} were found given the above search criteria\")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The products found are displayed on an interactive map along with the centroid of the targeted tile. A time player allows to see when the products were sensed." ] }, { "cell_type": "code", "execution_count": 16, "metadata": {}, "outputs": [ { "data": { "text/html": [ "