EODataAccessGateway

EODataAccessGateway#

The EODataAccessGateway class is the core component of the EODAG library. It provides a unified interface for interacting with various Earth Observation data providers. This class allows users to configure providers, search for products, download data, and perform other operations related to Earth Observation data management. Below is an overview of its main functionalities, categorized for ease of use.

Constructor#

EODataAccessGateway([user_conf_file_path, ...])

An API for downloading a wide variety of geospatial products originating from different types of providers.

Configuration#

EODataAccessGateway.add_provider(name[, ...])

Adds a new provider.

EODataAccessGateway.set_preferred_provider(...)

Set max priority for the given provider.

EODataAccessGateway.get_preferred_provider()

Get the provider currently set as the preferred one for searching products, along with its priority.

EODataAccessGateway.update_providers_config([...])

Update providers configuration with given input.

EODataAccessGateway.update_collections_list(...)

Update eodag collections list

Catalog#

EODataAccessGateway.providers

Providers of eodag configuration sorted by priority in descending order and by name in ascending order.

EODataAccessGateway.available_providers([...])

Gives the sorted list of the available providers or groups

EODataAccessGateway.list_collections([...])

Lists supported collections.

EODataAccessGateway.guess_collection([...])

Find EODAG collection IDs that best match a set of search parameters.

EODataAccessGateway.fetch_collections_list([...])

Fetch collections list and update if needed.

EODataAccessGateway.discover_collections([...])

Fetch providers for collections

Crunch#

EODataAccessGateway.crunch(results, **kwargs)

Apply the filters given through the keyword arguments to the results

EODataAccessGateway.get_cruncher(name, **options)

Build a crunch plugin from a configuration

Download#

EODataAccessGateway.download(product[, ...])

Download a single product.

EODataAccessGateway.download_all(search_result)

Download all products resulting from a search.

Serialize/Deserialize#

EODataAccessGateway.serialize(search_result)

Registers results of a search into a geojson file.

EODataAccessGateway.deserialize(filename)

Loads results of a search from a geojson file.

EODataAccessGateway.deserialize_and_register(...)

Loads results of a search from a geojson file and register products with the information needed to download itself.

Misc#

EODataAccessGateway.group_by_extent(searches)

Combines multiple SearchResults and return a list of SearchResults grouped by extent (i.e. bounding box).

EODataAccessGateway.guess_collection([...])

Find EODAG collection IDs that best match a set of search parameters.

EODataAccessGateway.list_queryables([...])

Fetch the queryable properties for a given collection and/or provider.

EODataAccessGateway.available_sortables()

For each provider, gives its available sortable parameter(s) and its maximum number of them if it supports the sorting feature, otherwise gives None.

EODataAccessGateway.import_stac_items(items_urls)

Import STAC items from a list of URLs and convert them to SearchResult.
class eodag.api.core.EODataAccessGateway(user_conf_file_path=None, locations_conf_path=None)[source]#

An API for downloading a wide variety of geospatial products originating from different types of providers.

Parameters:

  • user_conf_file_path (str | None, default: None) – (optional) Path to the user configuration file

  • locations_conf_path (str | None, default: None) – (optional) Path to the locations configuration file

add_provider(name, url=None, priority=None, search={'type': 'StacSearch'}, products={'GENERIC_COLLECTION': {'_collection': '{collection}'}}, download={'auth_error_code': 401, 'type': 'HTTPDownload'}, **kwargs)[source]#

Adds a new provider.

search, products & download already have default values that will be updated (not replaced), with user provided ones:

  • search : {"type": "StacSearch"}

  • products : {"GENERIC_COLLECTION": {"_collection": "{collection}"}}

  • download : {"type": "HTTPDownload", "auth_error_code": 401}

Parameters:

  • name (str) – Name of provider

  • url (str | None, default: None) – Provider url, also used as search["api_endpoint"] if not defined

  • priority (int | None, default: None) – Provider priority. If None, provider will be set as preferred (highest priority)

  • search (dict[str, Any], default: {'type': 'StacSearch'}) – Search PluginConfig mapping

  • products (dict[str, Any], default: {'GENERIC_COLLECTION': {'_collection': '{collection}'}}) – Provider collections mapping

  • download (dict[str, Any], default: {'type': 'HTTPDownload', 'auth_error_code': 401}) – Download PluginConfig mapping

  • kwargs (dict[str, Any]) – Additional ProviderConfig mapping

available_providers(collection=None, by_group=False)[source]#

Gives the sorted list of the available providers or groups

Deprecated since version v4.0.0: Please use eodag.api.core.EODataAccessGateway.providers instead.

The providers or groups are sorted first by their priority level in descending order, and then alphabetically in ascending order for providers or groups with the same priority level.

Parameters:

  • collection (str | None, default: None) – (optional) Only list providers configured for this collection

  • by_group (bool, default: False) – (optional) If set to True, list groups when available instead of providers, mixed with other providers

Return type:

list[str]

Returns:

the sorted list of the available providers or groups

available_sortables()[source]#

For each provider, gives its available sortable parameter(s) and its maximum number of them if it supports the sorting feature, otherwise gives None.

Return type:

dict[str, ProviderSortables | None]

Returns:

A dictionary with providers as keys and dictionary of sortable parameter(s) and its (their) maximum number as value(s).

Raises:

UnsupportedProvider

crunch(results, **kwargs)[source]#

Apply the filters given through the keyword arguments to the results

Parameters:

  • results (SearchResult) – The results of a eodag search request

  • kwargs (Any)

Return type:

SearchResult

Returns:

The result of successively applying all the filters to the results

static deserialize(filename)[source]#

Loads results of a search from a geojson file.

Parameters:

filename (str) – A filename containing a search result encoded as a geojson

Return type:

SearchResult

Returns:

The search results encoded in filename

deserialize_and_register(filename)[source]#

Loads results of a search from a geojson file and register products with the information needed to download itself.

This method also sets the internal EODataAccessGateway instance on the products, enabling pagination (e.g. access to next pages) if available.

Parameters:

filename (str) – A filename containing a search result encoded as a geojson

Return type:

SearchResult

Returns:

The search results encoded in filename, ready for download and pagination

discover_collections(provider=None)[source]#

Fetch providers for collections

Parameters:

provider (str | None, default: None) – The name of a provider or provider-group to fetch. Defaults to all providers (None value).

Return type:

dict[str, Any] | None

Returns:

external collections configuration

download(product, progress_callback=None, executor=None, wait=0.2, timeout=10, **kwargs)[source]#

Download a single product.

This is an alias to the method of the same name on EOProduct, but it performs some additional checks like verifying that a downloader and authenticator are registered for the product before trying to download it.

If the metadata mapping for eodag:download_link is set to something that can be interpreted as a link on a local filesystem, the download is skipped (by now, only a link starting with file:/ is supported). Therefore, any user that knows how to extract product location from product metadata on a provider can override the eodag:download_link metadata mapping in the right way. For example, using the environment variable: EODAG__CREODIAS__SEARCH__METADATA_MAPPING__EODAG_DOWNLOAD_LINK="file:///{id}" will lead to all EOProduct’s originating from the provider creodias to have their eodag:download_link metadata point to something like: file:///12345-678, making this method immediately return the later string without trying to download the product.

Parameters:

  • product (EOProduct) – The EO product to download

  • progress_callback (ProgressCallback | None, default: None) – (optional) A method or a callable object which takes a current size and a maximum size as inputs and handle progress bar creation and update to give the user a feedback on the download progress

  • executor (ThreadPoolExecutor | None, default: None) – (optional) An executor to download assets of product in parallel if it has any. If None , a default executor will be created

  • wait (float, default: 0.2) – (optional) If download fails, wait time in minutes between two download tries

  • timeout (float, default: 10) – (optional) If download fails, maximum time in minutes before stop retrying to download

  • kwargs (Unpack[DownloadConf]) –

    Additional keyword arguments from the download plugin configuration class that can be provided to override any other values defined in a configuration file or with environment variables:

    • output_dir - where to store downloaded products, as an absolute file path (Default: local temporary directory)

    • output_extension - downloaded file extension

    • extract - whether to extract the downloaded products, only applies to archived products

    • dl_url_params - additional parameters to pass over to the download url as an url parameter

    • delete_archive - whether to delete the downloaded archives

    • asset - regex filter to identify assets to download

Return type:

str

Returns:

The absolute path to the downloaded product in the local filesystem

Raises:

PluginImplementationError

Raises:

RuntimeError

download_all(search_result, downloaded_callback=None, progress_callback=None, executor=None, wait=0.2, timeout=10, **kwargs)[source]#

Download all products resulting from a search.

Parameters:

  • search_result (SearchResult) – A set of EO products resulting from a search

  • downloaded_callback (DownloadedCallback | None, default: None) – (optional) A method or a callable object which takes as parameter the product. You can use the base class DownloadedCallback and override its __call__ method. Will be called each time a product finishes downloading

  • progress_callback (ProgressCallback | None, default: None) – (optional) A method or a callable object which takes a current size and a maximum size as inputs and handle progress bar creation and update to give the user a feedback on the download progress

  • executor (ThreadPoolExecutor | None, default: None) – (optional) An executor to download EO products of search_result in parallel which will also be reused to download assets of these products in parallel.

  • wait (float, default: 0.2) – (optional) If download fails, wait time in minutes between two download tries of the same product

  • timeout (float, default: 10) – (optional) If download fails, maximum time in minutes before stop retrying to download

  • kwargs (Unpack[DownloadConf]) –

    Additional keyword arguments from the download plugin configuration class that can be provided to override any other values defined in a configuration file or with environment variables:

    • output_dir - where to store downloaded products, as an absolute file path (Default: local temporary directory)

    • output_extension - downloaded file extension

    • extract - whether to extract the downloaded products, only applies to archived products

    • dl_url_params - additional parameters to pass over to the download url as an url parameter

    • delete_archive - whether to delete the downloaded archives

    • asset - regex filter to identify assets to download

Return type:

list[str]

Returns:

A collection of the absolute paths to the downloaded products

fetch_collections_list(provider=None)[source]#

Fetch collections list and update if needed.

If strict mode is enabled (by setting the EODAG_STRICT_COLLECTIONS environment variable to a truthy value), this method will not fetch or update collections and will return immediately.

Parameters:

provider (str | None, default: None) – The name of a provider or provider-group for which collections list should be updated. Defaults to all providers (None value).

Return type:

None

get_cruncher(name, **options)[source]#

Build a crunch plugin from a configuration

Parameters:

  • name (str) – The name of the cruncher to build

  • options (Any) – The configuration options of the cruncher

Return type:

Crunch

Returns:

The cruncher named name

get_preferred_provider()[source]#

Get the provider currently set as the preferred one for searching products, along with its priority.

Return type:

tuple[str, int]

Returns:

The provider with the maximum priority and its priority

static group_by_extent(searches)[source]#

Combines multiple SearchResults and return a list of SearchResults grouped by extent (i.e. bounding box).

Parameters:

searches (list[SearchResult]) – List of eodag SearchResult

Return type:

list[SearchResult]

Returns:

list of SearchResult

guess_collection(free_text=None, intersect=False, instruments=None, platform=None, constellation=None, processing_level=None, sensor_type=None, keywords=None, description=None, title=None, start_date=None, end_date=None, **kwargs)[source]#

Find EODAG collection IDs that best match a set of search parameters.

When using several filters, collections that match most of them will be returned at first.

Parameters:

  • free_text (str | None, default: None) – Free text search filter used to search accross all the following parameters. Handles logical operators with parenthesis (AND/OR/NOT), quoted phrases ("exact phrase"), * and ? wildcards.

  • intersect (bool, default: False) – Join results for each parameter using INTERSECT instead of UNION.

  • instruments (str | None, default: None) – Instruments parameter.

  • platform (str | None, default: None) – Platform parameter.

  • constellation (str | None, default: None) – Constellation parameter.

  • processing_level (str | None, default: None) – Processing level parameter.

  • sensor_type (str | None, default: None) – Sensor type parameter.

  • keywords (str | None, default: None) – Keywords parameter.

  • description (str | None, default: None) – description parameter.

  • title (str | None, default: None) – Title parameter.

  • start_date (str | None, default: None) – start date for datetime filtering. Not used by free_text

  • end_date (str | None, default: None) – end date for datetime filtering. Not used by free_text

  • kwargs (Any)

Return type:

CollectionsList

Returns:

The best match for the given parameters.

Raises:

NoMatchingCollection

import_stac_items(items_urls, provider=None)[source]#

Import STAC items from a list of URLs and convert them to SearchResult.

  • Origin provider and download links will be set if item comes from an EODAG server.

  • If item comes from a known EODAG provider, result will be registered to it, ready to download and its metadata normalized.

  • If item comes from an unknown provider, a generic STAC provider will be used.

Parameters:

  • items_urls (list[str]) – A list of STAC items URLs to import

  • provider (str | None, default: None) – (optional) The EODAG provider to which the STAC items belong, if known.

Return type:

SearchResult

Returns:

A SearchResult containing the imported STAC items

list_collections(provider=None, fetch_providers=True)[source]#

Lists supported collections.

Parameters:

  • provider (str | None, default: None) – (optional) The name of a provider that must support the product types we are about to list

  • fetch_providers (bool, default: True) – (optional) Whether to fetch providers for new product types or not

Return type:

CollectionsList

Returns:

The list of the collections that can be accessed using eodag.

Raises:

UnsupportedProvider

list_queryables(provider=None, fetch_providers=True, **kwargs)[source]#

Fetch the queryable properties for a given collection and/or provider.

Parameters:

  • provider (str | None, default: None) – (optional) The provider.

  • fetch_providers (bool, default: True) – If new collections should be fetched from the providers; default: True

  • kwargs (Any) – additional filters for queryables (collection or other search arguments)

Return type:

QueryablesDict

Returns:

A QueryablesDict containing the EODAG queryable properties, associating parameters to their annotated type, and a additional_properties attribute

property providers: ProvidersDict#

Providers of eodag configuration sorted by priority in descending order and by name in ascending order.

search(page=1, limit=20, items_per_page=20, raise_errors=False, start=None, end=None, geom=None, locations=None, provider=None, count=False, validate=True, **kwargs)[source]#

Look for products matching criteria on known providers.

The default behaviour is to look for products on the provider with the highest priority supporting the requested collection. These priorities are configurable through user configuration file or individual environment variable. If the request to the provider with the highest priority fails or is empty, the data will be request from the provider with the next highest priority. Only if the request fails for all available providers, an error will be thrown.

Parameters:

  • page (int, default: 1) – (optional) The page number to return (deprecated, use eodag.api.search_result.SearchResult.next_page() instead)

  • limit (int | None, default: 20) – (optional) The number of results that must appear in one single page. If None, the maximum number possible will be used.

  • items_per_page (int | None, default: 20) – (optional) The number of results that must appear in one single page. If None, the maximum number possible will be used. (deprecated, use limit instead)

  • raise_errors (bool, default: False) – (optional) When an error occurs when searching, if this is set to True, the error is raised

  • start (str | None, default: None) – (optional) Start sensing time in ISO 8601 format (e.g. “1990-11-26”, “1990-11-26T14:30:10.153Z”, “1990-11-26T14:30:10+02:00”, …). If no time offset is given, the time is assumed to be given in UTC.

  • end (str | None, default: None) – (optional) End sensing time in ISO 8601 format (e.g. “1990-11-26”, “1990-11-26T14:30:10.153Z”, “1990-11-26T14:30:10+02:00”, …). If no time offset is given, the time is assumed to be given in UTC.

  • geom (str | dict[str, float] | BaseGeometry | None, default: None) –

    (optional) Search area that can be defined in different ways:

    • with a Shapely geometry object: shapely.geometry.base.BaseGeometry

    • with a bounding box (dict with keys: “lonmin”, “latmin”, “lonmax”, “latmax”): dict.fromkeys(["lonmin", "latmin", "lonmax", "latmax"])

    • with a bounding box as list of float: [lonmin, latmin, lonmax, latmax]

    • with a WKT str

  • locations (dict[str, str] | None, default: None) – (optional) Location filtering by name using locations configuration {"<location_name>"="<attr_regex>"}. For example, {"country"="PA."} will use the geometry of the features having the property ISO3 starting with ‘PA’ such as Panama and Pakistan in the shapefile configured with name=country and attr=ISO3

  • provider (str | None, default: None) – (optional) the provider to be used. If set, search fallback will be disabled. If not set, the configured preferred provider will be used at first before trying others until finding results.

  • count (bool, default: False) – (optional) Whether to run a query with a count request or not

  • validate (bool | None, default: True) – (optional) Set to True to validate search parameters before sending the query to the provider

  • kwargs (Any) – Some other criteria that will be used to do the search, using paramaters compatibles with the provider

Return type:

SearchResult

Returns:

A set of EO products matching the criteria

Changed in version v3.0.0b1: search() method now returns only a single SearchResult instead of a 2 values tuple.

Note

The search interfaces, which are implemented as plugins, are required to return a list as a result of their processing. This requirement is enforced here.

search_all(limit=None, items_per_page=None, start=None, end=None, geom=None, locations=None, **kwargs)[source]#

Search and return all the products matching the search criteria.

It iterates over the pages of a search query and collects all the returned products into a single SearchResult instance.

Requests are attempted to all providers of the product ordered by descending piority.

Parameters:

  • limit (int | None, default: None) – (optional) The number of results requested internally per page. The maximum number of items than can be requested at once to a provider has been configured in EODAG for some of them. If limit is None and this number is available for the searched provider, it is used to limit the number of requests made. This should also reduce the time required to collect all the products matching the search criteria. If this number is not available, a default value of 50 is used instead. limit can also be set to any arbitrary value.

  • items_per_page (int | None, default: None) – (optional) The number of results requested internally per page (deprecated, use limit instead)

  • start (str | None, default: None) – (optional) Start sensing time in ISO 8601 format (e.g. “1990-11-26”, “1990-11-26T14:30:10.153Z”, “1990-11-26T14:30:10+02:00”, …). If no time offset is given, the time is assumed to be given in UTC.

  • end (str | None, default: None) – (optional) End sensing time in ISO 8601 format (e.g. “1990-11-26”, “1990-11-26T14:30:10.153Z”, “1990-11-26T14:30:10+02:00”, …). If no time offset is given, the time is assumed to be given in UTC.

  • geom (str | dict[str, float] | BaseGeometry | None, default: None) –

    (optional) Search area that can be defined in different ways:

    • with a Shapely geometry object: shapely.geometry.base.BaseGeometry

    • with a bounding box (dict with keys: “lonmin”, “latmin”, “lonmax”, “latmax”): dict.fromkeys(["lonmin", "latmin", "lonmax", "latmax"])

    • with a bounding box as list of float: [lonmin, latmin, lonmax, latmax]

    • with a WKT str

  • locations (dict[str, str] | None, default: None) – (optional) Location filtering by name using locations configuration {"<location_name>"="<attr_regex>"}. For example, {"country"="PA."} will use the geometry of the features having the property ISO3 starting with ‘PA’ such as Panama and Pakistan in the shapefile configured with name=country and attr=ISO3

  • kwargs (Any) – Some other criteria that will be used to do the search, using parameters compatible with the provider

Return type:

SearchResult

Returns:

An iterator that yields page per page a set of EO products matching the criteria

search_iter_page(limit=20, items_per_page=20, start=None, end=None, geom=None, locations=None, **kwargs)[source]#

Iterate over the pages of a products search.

Deprecated since version v4.0.0: Please use eodag.api.search_result.SearchResult.next_page() instead.

Parameters:

  • limit (int, default: 20) – (optional) The number of results requested per page

  • items_per_page (int | None, default: 20) – (optional) The number of results requested per page (deprecated, use limit instead)

  • start (str | None, default: None) – (optional) Start sensing time in ISO 8601 format (e.g. “1990-11-26”, “1990-11-26T14:30:10.153Z”, “1990-11-26T14:30:10+02:00”, …). If no time offset is given, the time is assumed to be given in UTC.

  • end (str | None, default: None) – (optional) End sensing time in ISO 8601 format (e.g. “1990-11-26”, “1990-11-26T14:30:10.153Z”, “1990-11-26T14:30:10+02:00”, …). If no time offset is given, the time is assumed to be given in UTC.

  • geom (str | dict[str, float] | BaseGeometry | None, default: None) –

    (optional) Search area that can be defined in different ways:

    • with a Shapely geometry object: shapely.geometry.base.BaseGeometry

    • with a bounding box (dict with keys: “lonmin”, “latmin”, “lonmax”, “latmax”): dict.fromkeys(["lonmin", "latmin", "lonmax", "latmax"])

    • with a bounding box as list of float: [lonmin, latmin, lonmax, latmax]

    • with a WKT str

  • locations (dict[str, str] | None, default: None) – (optional) Location filtering by name using locations configuration {"<location_name>"="<attr_regex>"}. For example, {"country"="PA."} will use the geometry of the features having the property ISO3 starting with ‘PA’ such as Panama and Pakistan in the shapefile configured with name=country and attr=ISO3

  • kwargs (Any) – Some other criteria that will be used to do the search, using paramaters compatibles with the provider

Return type:

Iterator[SearchResult]

Returns:

An iterator that yields page per page a set of EO products matching the criteria

static serialize(search_result, filename='search_results.geojson', skip_invalid=True)[source]#

Registers results of a search into a geojson file. The output is a FeatureCollection containing the EO products as features, with additional metadata such as number_matched, next_page_token, and search_params stored in the properties.

Parameters:

  • search_result (SearchResult) – A set of EO products resulting from a search

  • filename (str, default: 'search_results.geojson') – (optional) The name of the file to generate

  • skip_invalid (bool, default: True) – Whether to skip properties whose values are not valid according to the STAC specification.

Return type:

str

Returns:

The name of the created file

set_preferred_provider(provider)[source]#

Set max priority for the given provider.

Parameters:

provider (str) – The name of the provider that should be considered as the preferred provider to be used for this instance

Return type:

None

update_collections_list(ext_collections_conf)[source]#

Update eodag collections list

Parameters:

ext_collections_conf (dict[str, dict[str, dict[str, Any]] | None]) – external collections configuration

Return type:

None

update_providers_config(yaml_conf=None, dict_conf=None)[source]#

Update providers configuration with given input. Can be used to add a provider to existing configuration or update an existing one.

Parameters:

  • yaml_conf (str | None, default: None) – YAML formated provider configuration

  • dict_conf (dict[str, Any] | None, default: None) – provider configuration as dictionary in place of yaml_conf

Return type:

None