eodag.plugins.search.qssearch.QueryStringSearch#

class eodag.plugins.search.qssearch.QueryStringSearch(provider, config)[source]#

A plugin that helps implementing any kind of search protocol that relies on query strings (e.g: opensearch).

The available configuration parameters for this kind of plugin are:

  • result_type: (optional) One of “json” or “xml”, depending on the representation of the provider’s search results. The default is “json”

  • results_entry: (mandatory) The name of the key in the provider search result that gives access to the result entries

  • api_endpoint: (mandatory) The endpoint of the provider’s search interface

  • literal_search_params: (optional) A mapping of (search_param => search_value) pairs giving search parameters to be passed as is in the search url query string. This is useful for example in situations where the user wants to pass-in a search query as it is done on the provider interface. In such a case, the user can put in his configuration file the query he needs to pass to the provider.

  • pagination: (mandatory) The configuration of how the pagination is done on the provider. It is a tree with the following nodes:

    • next_page_url_tpl: The template for pagination requests. This is a simple Python format string which will be resolved using the following keywords: url (the base url of the search endpoint), search (the query string corresponding to the search request), items_per_page (the number of items to return per page), skip (the number of items to skip) or skip_base_1 (the number of items to skip, starting from 1) and page (which page to return).

    • total_items_nb_key_path: (optional) An XPath or JsonPath leading to the total number of results satisfying a request. This is used for providers which provides the total results metadata along with the result of the query and don’t have an endpoint for querying the number of items satisfying a request, or for providers for which the count endpoint returns a json or xml document

    • count_endpoint: (optional) The endpoint for counting the number of items satisfying a request

    • next_page_url_key_path: (optional) A JSONPATH expression used to retrieve the URL of the next page in the response of the current page.

  • free_text_search_operations: (optional) A tree structure of the form:

    <search-param>:     # e.g: $search
    union: # how to join the operations below (e.g: ' AND ' -->
           # '(op1 AND op2) AND (op3 OR op4)')
    wrapper: # a pattern for how each operation will be wrapped
             # (e.g: '({})' --> '(op1 AND op2)')
    operations:     # The operations to build
      <opname>:     # e.g: AND
        - <op1>    # e.g:
                   # 'sensingStartDate:[{startTimeFromAscendingNode}Z TO *]'
        - <op2>    # e.g:
            # 'sensingStopDate:[* TO {completionTimeFromAscendingNode}Z]'
        ...
      ...
...

    With the structure above, each operation will become a string of the form: ‘(<op1> <opname> <op2>)’, then the operations will be joined together using the union string and finally if the number of operations is greater than 1, they will be wrapped as specified by the wrapper config key.

The search plugins of this kind can detect when a metadata mapping is “query-able”, and get the semantics of how to format the query string parameter that enables to make a query on the corresponding metadata. To make a metadata query-able, just configure it in the metadata mapping to be a list of 2 items, the first one being the specification of the query string search formatting. The later is a string following the specification of Python string formatting, with a special behaviour added to it. For example, an entry in the metadata mapping of this kind:

completionTimeFromAscendingNode:
    - 'f=acquisition.endViewingDate:lte:{completionTimeFromAscendingNode#timestamp}'
    - '$.properties.acquisition.endViewingDate'

means that the search url will have a query string parameter named “f” with a value of “acquisition.endViewingDate:lte:1543922280.0” if the search was done with the value of completionTimeFromAscendingNode being 2018-12-04T12:18:00. What happened is that {completionTimeFromAscendingNode#timestamp} was replaced with the timestamp of the value of completionTimeFromAscendingNode. This example shows all there is to know about the semantics of the query string formatting introduced by this plugin: any eodag search parameter can be referenced in the query string with an additional optional conversion function that is separated from it by a # (see format_metadata() for further details on the available converters). Note that for the values in the free_text_search_operations configuration parameter follow the same rule.

Parameters

  • provider (dict) – An eodag providers configuration dictionary

  • config (str) – Path to the user configuration file

__init__(provider, config)[source]#

Methods

__init__(provider, config)

build_query_string(product_type, **kwargs)

Build The query string using the search parameters

clear()

Clear search context

collect_search_urls([page, items_per_page, ...])

Build paginated urls

count_hits(count_url[, result_type])

Count the number of results satisfying some criteria

discover_product_types()

Fetch product types list from provider using discover_product_types conf

do_search([items_per_page])

Perform the actual search request.

format_free_text_search(**kwargs)

Build the free text search parameter using the search parameters

get_collections(**kwargs)

Get the collection to which the product belongs

get_product_type_def_params(product_type, ...)

Get the provider product type definition parameters

get_queryables(search_params)

Retrieve the metadata mappings that are query-able

map_product_type(product_type, **kwargs)

Map the eodag product type to the provider product type

normalize_results(results, **kwargs)

Build EOProducts from provider results

query([items_per_page, page, count])

Perform a search on an OpenSearch-like interface

update_metadata_mapping(metadata_mapping)

Update plugin metadata_mapping with input metadata_mapping configuration

Attributes

COMPLEX_QS_REGEX

DEFAULT_ITEMS_PER_PAGE

extract_properties

plugins