ElasticSearchView

This view creates an ElasticSearch index and stores the targeted Json resources into an ElasticSearch Document.

The documents created on each view are isolated from documents created on other views by using different ElasticSearch indices.

A default view gets automatically created when the project is created but other views can be created.

Processing pipeline

An asynchronous process gets triggered for every view. This process can be visualized as a pipeline with different stages.

The first stage is the input of the pipeline: a stream of resource updates scoped for the project where the view was created.

For each incoming update, a pipeline is applied to this resource.

A pipeline is made of pipes which are the processing units of a pipeline for an Elasticsearch view.

They are applied sequentially as defined by the user in the view payload and allow to transform and/or filter a resource before indexing it to Elasticsearch.

Filters

Filters allow to discard a resource if it does not match a given predicate (ex: it has been deprecated or does not match an expected type). If a resource is filtered out by a pipe:

  • it won’t be indexed so the execution and it will be deleted from the index if a previous revision of this resource has been indexed
  • the execution of the next pipes is short-circuited to avoid useless computation

It is therefore encouraged to apply the filtering pipes at the beginning of the pipeline.

Existing pipes and how to create new ones is defined here.

Transformations

Transformations allow to modify:

  • The original payload of the resource which has been submitted by the user
  • The metadata graph of the resource (the fields that have been generated from Nexus)
  • The data graph of the resource (that has been computed from the original payload at ingestion time)

Reconciliation and indexing

When a resource makes it to the end of the pipeline, we produce a json document by merging:

  • the original payload,
  • the data and metadata graphs that have been translated to the compacted JSON-LD format with a default context and the one provided by the view

The resulting json is then stripped of any @context fields and pushed to Elasticsearch.

ElasticSearchView pipeline

Payload

The payload includes a pipeline of transformations and filters to apply to the different resources. The stages of the pipeline are applied sequentially on the resource as defined in the payload.

{
  "@id": "{someid}",
  "@type": "ElasticSearchView",
  "resourceTag": "{tag}",
  "pipeline": [
    {
      "name" : "{pipeName}",
      "config" : _pipe_config_
    },
    ...
  ],
  "context": _context_,
  "mapping": _elasticsearch mapping_,
  "settings": _elasticsearch settings_,
  "permission": "{permission}"
}

where…

  • {tag}: String - Selects only resources with the provided tag. This field is optional.
  • {pipeName}: String - Identifier of the pipe to apply. More information about pipe is available here
  • pipe_config : Json object - Configuration for the pipe {pipeName}. This field can be optional depending on {pipeName}
  • ._context_ : Json - Additional JSON-LD context value applied when compacting the resource before indexing it to Elasticsearch.
  • _elasticsearch mapping_: Json object - Defines the value types for the Json keys, as stated at the ElasticSearch mapping documentation.
  • _elasticssearch settings_: Json object - defines Elasticsearch index settings for the underlying Elasticsearch index. Default settings are applied, if not specified.
  • {someid}: Iri - The @id value for this view.
  • {permission}: String - permission required to query this view. Defaults to views/query.

Please note that for retro-compatibility purposes, omitting the pipeline will apply a default one including filtering deprecated resources, discarding metadata and selecting default label predicates

Legacy payload

Retro-compatibility is ensured with the legacy payload as defined:

{
  "@id": "{someid}",
  "@type": "ElasticSearchView",
  "resourceSchemas": [ "{resourceSchema}", ...],
  "resourceTypes": [ "{resourceType}", ...],
  "resourceTag": "{tag}",
  "sourceAsText": {sourceAsText},
  "includeMetadata": {includeMetadata},
  "includeDeprecated": {includeDeprecated},
  "mapping": _elasticsearch mapping_,
  "settings": _elasticsearch settings_,
  "permission": "{permission}"
}

where…

  • {resourceSchema}: Iri - Selects only resources that are validated against the provided schema Iri. This field is optional.
  • {resourceType}: Iri - Select only resources of the provided type Iri. This field is optional.
  • {tag}: String - Selects only resources with the provided tag. This field is optional.
  • _elasticsearch mapping_: Json object - Defines the value types for the Json keys, as stated at the ElasticSearch mapping documentation.
  • _elasticssearch settings_: Json object - defines Elasticsearch index settings for the underlying Elasticsearch index. Default settings are applied, if not specified.
  • {sourceAsText}: Boolean - If true, the resource’s payload will be stored in the ElasticSearch document as a single escaped string value under the key _original_source. If false, the resource’s payload will be stored normally in the ElasticSearch document. The default value is false.
  • {includeMetadata}: Boolean - If true, the resource’s nexus metadata (_constrainedBy, _deprecated, …) will be stored in the ElasticSearch document. Otherwise it won’t. The default value is false.
  • {includeDeprecated}: Boolean - If true, deprecated resources are also indexed. The default value is false.
  • {someid}: Iri - The @id value for this view.
  • {permission}: String - permission required to query this view. Defaults to views/query.

The legacy payload is now deprecated and will be removed in an upcoming version.

Example

The following example creates an ElasticSearch view that will index resources validated against the schema with id https://bluebrain.github.io/nexus/schemas/myschema. If a resource is deprecated, it won’t be selected for indexing.

The resulting ElasticSearch Documents fields will be indexed according to the provided mapping rules and they won’t include the resource metadata fields.

{
  "@id": "https://bluebrain.github.io/nexus/vocabulary/myview",
  "@type": [
    "ElasticSearchView"
  ],
  "mapping": {
    "dynamic": false,
    "properties": {
      "@id": {
        "type": "keyword"
      },
      "@type": {
        "type": "keyword"
      },
      "name": {
        "type": "keyword"
      },
      "number": {
        "type": "long"
      },
      "bool": {
        "type": "boolean"
      }
    }
  },
  "pipeline": [
    {
      "name" : "filterDeprecated"
    },
    {
      "name" : "filterBySchema",
      "config" : {
        "types" : [
          "https://bluebrain.github.io/nexus/schemas/myschema"
        ]
      }
    },
    {
      "name" : "discardMetadata"
    }
  ]
}

Create using POST

POST /v1/views/{org_label}/{project_label}
  {...}

The json payload:

  • If the @id value is found on the payload, this @id will be used.
  • If the @id value is not found on the payload, an @id will be generated as follows: base:{UUID}. The base is the prefix defined on the resource’s project ({project_label}).

Example

Request
sourcecurl -XPOST \
  -H "Content-Type: application/json" \
  "http://localhost:8080/v1/views/myorg/myproj" -d \
'{
    "@type": "ElasticSearchView",
    "mapping": {
      "dynamic": false,
      "properties": {
        "@id": {
          "type": "keyword"
        },
        "@type": {
          "type": "keyword"
        },
        "name": {
          "type": "keyword"
        },
        "number": {
          "type": "long"
        },
        "bool": {
          "type": "boolean"
        }
      }
    },
    "pipeline": []
}'
Payload
source{
  "@type": [
    "ElasticSearchView"
  ],
  "mapping": {
    "dynamic": false,
    "properties": {
      "@id": {
        "type": "keyword"
      },
      "@type": {
        "type": "keyword"
      },
      "name": {
        "type": "keyword"
      },
      "number": {
        "type": "long"
      },
      "bool": {
        "type": "boolean"
      }
    }
  },
  "pipeline": []
}
Response
source{
  "@context": [
    "https://bluebrain.github.io/nexus/contexts/elasticsearch-metadata.json",
    "https://bluebrain.github.io/nexus/contexts/metadata.json"
  ],
  "@id": "http://localhost:8080/v1/resources/myorg/myproj/_/386a8c21-3acf-4f10-90ab-329466f5e04c",
  "@type": [
    "ElasticSearchView",
    "View"
  ],
  "_constrainedBy": "https://bluebrain.github.io/nexus/schemas/views.json",
  "_createdAt": "2021-05-12T09:44:35.609Z",
  "_createdBy": "http://localhost:8080/v1/anonymous",
  "_deprecated": false,
  "_incoming": "http://localhost:8080/v1/views/myorg/myproj/386a8c21-3acf-4f10-90ab-329466f5e04c/incoming",
  "_outgoing": "http://localhost:8080/v1/views/myorg/myproj/386a8c21-3acf-4f10-90ab-329466f5e04c/outgoing",
  "_project": "http://localhost:8080/v1/projects/myorg/myproj",
  "_rev": 1,
  "_self": "http://localhost:8080/v1/views/myorg/myproj/386a8c21-3acf-4f10-90ab-329466f5e04c",
  "_updatedAt": "2021-05-12T09:44:35.609Z",
  "_updatedBy": "http://localhost:8080/v1/anonymous",
  "_uuid": "ea4b3858-c1ac-4afb-ac94-13fdb1b53681"
}

Create using PUT

This alternative endpoint to create a view is useful in case the json payload does not contain an @id but you want to specify one. The @id will be specified in the last segment of the endpoint URI.

PUT /v1/views/{org_label}/{project_label}/{view_id}
  {...}

Note that if the payload contains an @id different from the {view_id}, the request will fail.

Example

Request
sourcecurl -XPUT \
  -H "Content-Type: application/json" \
  "http://localhost:8080/v1/views/myorg/myproj/myview" -d \
'{
    "@type": "ElasticSearchView",
    "mapping": {
      "dynamic": false,
      "properties": {
        "@id": {
          "type": "keyword"
        },
        "@type": {
          "type": "keyword"
        },
        "name": {
          "type": "keyword"
        },
        "number": {
          "type": "long"
        },
        "bool": {
          "type": "boolean"
        }
      }
    },
    "pipeline": []
}'
Payload
source{
  "@type": [
    "ElasticSearchView"
  ],
  "mapping": {
    "dynamic": false,
    "properties": {
      "@id": {
        "type": "keyword"
      },
      "@type": {
        "type": "keyword"
      },
      "name": {
        "type": "keyword"
      },
      "number": {
        "type": "long"
      },
      "bool": {
        "type": "boolean"
      }
    }
  },
  "pipeline": []
}
Response
source{
  "@context": [
    "https://bluebrain.github.io/nexus/contexts/elasticsearch-metadata.json",
    "https://bluebrain.github.io/nexus/contexts/metadata.json"
  ],
  "@id": "http://localhost:8080/v1/resources/myorg/myproj/_/myview",
  "@type": [
    "ElasticSearchView",
    "View"
  ],
  "_constrainedBy": "https://bluebrain.github.io/nexus/schemas/views.json",
  "_createdAt": "2021-05-12T09:54:28.171Z",
  "_createdBy": "http://localhost:8080/v1/anonymous",
  "_deprecated": false,
  "_incoming": "http://localhost:8080/v1/views/myorg/myproj/myview/incoming",
  "_outgoing": "http://localhost:8080/v1/views/myorg/myproj/myview/outgoing",
  "_project": "http://localhost:8080/v1/projects/myorg/myproj",
  "_rev": 1,
  "_self": "http://localhost:8080/v1/views/myorg/myproj/myview",
  "_updatedAt": "2021-05-12T09:54:28.171Z",
  "_updatedBy": "http://localhost:8080/v1/anonymous",
  "_uuid": "7e737b83-30a0-4ea3-b6c9-cd1ed481d743"
}

Update

This operation overrides the payload.

In order to ensure a client does not perform any changes to a resource without having had seen the previous revision of the view, the last revision needs to be passed as a query parameter.

PUT /v1/views/{org_label}/{project_label}/{view_id}?rev={previous_rev}
  {...}

… where {previous_rev} is the last known revision number for the view.

Note

Updating a view creates a new Elasticsearch index and deletes the existing one. The indexing process will start from the beginning.

Example

Request
sourcecurl -XPUT \
-H "Content-Type: application/json" \
"http://localhost:8080/v1/views/myorg/myproj/myview?rev=1" -d \
'{
    "@type": [
        "ElasticSearchView"
    ],
    "mapping": {
    "dynamic": false,
    "properties": {
      "@id": {
        "type": "keyword"
      },
      "@type": {
        "type": "keyword"
      },
      "name": {
        "type": "keyword"
      },
      "number": {
        "type": "long"
      },
      "bool": {
        "type": "boolean"
      }
    }
  },
  "pipeline": []
}'
Payload
source{
  "@type": [
    "ElasticSearchView"
  ],
  "mapping": {
    "dynamic": false,
    "properties": {
      "@id": {
        "type": "keyword"
      },
      "@type": {
        "type": "keyword"
      },
      "name": {
        "type": "keyword"
      },
      "number": {
        "type": "long"
      },
      "bool": {
        "type": "boolean"
      }
    }
  },
  "pipeline": []
}
Response
source{
  "@context": [
    "https://bluebrain.github.io/nexus/contexts/elasticsearch-metadata.json",
    "https://bluebrain.github.io/nexus/contexts/metadata.json"
  ],
  "@id": "http://localhost:8080/v1/resources/myorg/myproj/_/myview",
  "@type": [
    "ElasticSearchView",
    "View"
  ],
  "_constrainedBy": "https://bluebrain.github.io/nexus/schemas/views.json",
  "_createdAt": "2021-05-12T09:54:28.171Z",
  "_createdBy": "http://localhost:8080/v1/anonymous",
  "_deprecated": false,
  "_incoming": "http://localhost:8080/v1/views/myorg/myproj/myview/incoming",
  "_outgoing": "http://localhost:8080/v1/views/myorg/myproj/myview/outgoing",
  "_project": "http://localhost:8080/v1/projects/myorg/myproj",
  "_rev": 2,
  "_self": "http://localhost:8080/v1/views/myorg/myproj/myview",
  "_updatedAt": "2021-05-12T09:56:26.156Z",
  "_updatedBy": "http://localhost:8080/v1/anonymous",
  "_uuid": "7e737b83-30a0-4ea3-b6c9-cd1ed481d743"
}

Deprecate

Locks the view, so no further operations can be performed. It also stops indexing any more resources into it and deletes the underlying index.

Deprecating a view is considered to be an update as well.

Note

Deprecating a view deletes the view index, making the view not searchable.

DELETE /v1/views/{org_label}/{project_label}/{view_id}?rev={previous_rev}

… where {previous_rev} is the last known revision number for the view.

Example

Request
sourcecurl -XDELETE \
  "http://localhost:8080/v1/views/myorg/myproj/myview?rev=3"
Response
source{
  "@context": [
    "https://bluebrain.github.io/nexus/contexts/elasticsearch-metadata.json",
    "https://bluebrain.github.io/nexus/contexts/metadata.json"
  ],
  "@id": "http://localhost:8080/v1/resources/myorg/myproj/_/myview",
  "@type": [
    "ElasticSearchView",
    "View"
  ],
  "_constrainedBy": "https://bluebrain.github.io/nexus/schemas/views.json",
  "_createdAt": "2021-05-12T12:56:09.676Z",
  "_createdBy": "http://localhost:8080/v1/anonymous",
  "_deprecated": true,
  "_incoming": "http://localhost:8080/v1/views/myorg/myproj/myview/incoming",
  "_outgoing": "http://localhost:8080/v1/views/myorg/myproj/myview/outgoing",
  "_project": "http://localhost:8080/v1/projects/myorg/myproj",
  "_rev": 4,
  "_self": "http://localhost:8080/v1/views/myorg/myproj/myview",
  "_updatedAt": "2021-05-12T13:01:01.232Z",
  "_updatedBy": "http://localhost:8080/v1/anonymous",
  "_uuid": "7e737b83-30a0-4ea3-b6c9-cd1ed481d743"
}

Undeprecate

Unlocks the view, so further operations can be performed. It also restarts indexing resources into it.

Undeprecating a view is considered to be an update as well.

PUT /v1/views/{org_label}/{project_label}/{view_id}/undeprecate?rev={previous_rev}

… where {previous_rev} is the last known revision number for the view.

Example

Request
sourcecurl -XPUT \
  "http://localhost:8080/v1/views/myorg/myproj/myview/undeprecate?rev=4"
Response
source{
  "@context": [
    "https://bluebrain.github.io/nexus/contexts/elasticsearch-metadata.json",
    "https://bluebrain.github.io/nexus/contexts/metadata.json"
  ],
  "@id": "http://localhost:8080/v1/resources/myorg/myproj/_/myview",
  "@type": [
    "ElasticSearchView",
    "View"
  ],
  "_constrainedBy": "https://bluebrain.github.io/nexus/schemas/views.json",
  "_createdAt": "2021-05-12T12:56:09.676Z",
  "_createdBy": "http://localhost:8080/v1/anonymous",
  "_deprecated": false,
  "_incoming": "http://localhost:8080/v1/views/myorg/myproj/myview/incoming",
  "_outgoing": "http://localhost:8080/v1/views/myorg/myproj/myview/outgoing",
  "_project": "http://localhost:8080/v1/projects/myorg/myproj",
  "_rev": 5,
  "_self": "http://localhost:8080/v1/views/myorg/myproj/myview",
  "_updatedAt": "2021-05-12T13:01:01.232Z",
  "_updatedBy": "http://localhost:8080/v1/anonymous",
  "_uuid": "7e737b83-30a0-4ea3-b6c9-cd1ed481d743"
}

Fetch

GET /v1/views/{org_label}/{project_label}/{view_id}?rev={rev}

where …

  • {rev}: Number - the targeted revision to be fetched. This field is optional and defaults to the latest revision.

Example

Request
sourcecurl "http://localhost:8080/v1/views/myorg/myproj/myview"
Response
source{
  "@context": [
    "https://bluebrain.github.io/nexus/contexts/elasticsearch.json",
    "https://bluebrain.github.io/nexus/contexts/metadata.json"
  ],
  "@id": "http://localhost:8080/v1/resources/myorg/myproj/_/myview",
  "@type": [
    "ElasticSearchView",
    "View"
  ],
  "includeDeprecated": true,
  "includeMetadata": true,
  "mapping": {
    "dynamic": false,
    "properties": {
      "@id": {
        "type": "keyword"
      },
      "@type": {
        "type": "keyword"
      },
      "bool": {
        "type": "boolean"
      },
      "name": {
        "type": "keyword"
      },
      "number": {
        "type": "long"
      }
    }
  },
  "permission": "views/query",
  "resourceSchemas": [],
  "resourceTypes": [],
  "pipeline": [],
  "settings": {
    "analysis": {
      "analyzer": {
        "nexus": {
          "filter": [
            "trim",
            "word_delimiter_nexus",
            "lowercase"
          ],
          "tokenizer": "classic",
          "type": "custom"
        }
      },
      "filter": {
        "word_delimiter_nexus": {
          "split_on_numerics": false,
          "type": "word_delimiter_graph"
        }
      }
    },
    "index": {
      "number_of_shards": 1
    }
  },
  "sourceAsText": false,
  "_constrainedBy": "https://bluebrain.github.io/nexus/schemas/views.json",
  "_createdAt": "2021-05-12T09:54:28.171Z",
  "_createdBy": "http://localhost:8080/v1/anonymous",
  "_deprecated": false,
  "_incoming": "http://localhost:8080/v1/views/myorg/myproj/myview/incoming",
  "_outgoing": "http://localhost:8080/v1/views/myorg/myproj/myview/outgoing",
  "_project": "http://localhost:8080/v1/projects/myorg/myproj",
  "_rev": 3,
  "_self": "http://localhost:8080/v1/views/myorg/myproj/myview",
  "_updatedAt": "2021-05-12T09:58:20.507Z",
  "_updatedBy": "http://localhost:8080/v1/anonymous",
  "_uuid": "7e737b83-30a0-4ea3-b6c9-cd1ed481d743"
}

If the redirect to Fusion feature is enabled and if the Accept header is set to text/html, a redirection to the fusion representation of the resource will be returned.

Note that for retro-compatibility purposes, fetching an elasticsearch view returns legacy fields.

Fetch original payload

GET /v1/views/{org_label}/{project_label}/{view_id}/source?rev={rev}

where …

  • {rev}: Number - the targeted revision to be fetched. This field is optional and defaults to the latest revision.

Example

Request
sourcecurl "http://localhost:8080/v1/views/myorg/myproj/myview/source"
Response
source{
  "@type": [
    "ElasticSearchView"
  ],
  "mapping": {
    "dynamic": false,
    "properties": {
      "@id": {
        "type": "keyword"
      },
      "@type": {
        "type": "keyword"
      },
      "name": {
        "type": "keyword"
      },
      "number": {
        "type": "long"
      },
      "bool": {
        "type": "boolean"
      }
    }
  },
  "pipeline": []
}

Search

POST /v1/views/{org_label}/{project_label}/{view_id}/_search
  {...}

The supported payload is defined on the ElasticSearch documentation

The string documents is used as a prefix of the default ElasticSearch view_id

Example

Request
sourcecurl -XPOST \
-H "Content-Type: application/json" \
"http://localhost:8080/v1/views/myorg/myproj/myview/_search" -d \
'{
  "query": {
    "term": {
      "@type": "https://bluebrain.github.io/nexus/vocabulary/ElasticSearchView"
    }
  }
}'
Payload
source{
  "query": {
    "term": {
      "@type": "https://bluebrain.github.io/nexus/vocabulary/ElasticSearchView"
    }
  }
}
Response
source{
  "hits": {
    "hits": [
      {
        "_id": "https://bluebrain.github.io/nexus/vocabulary/defaultElasticSearchIndex",
        "_index": "delta_7e737b83-30a0-4ea3-b6c9-cd1ed481d743_3",
        "_score": 0.8566987,
        "_source": {
          "@id": "https://bluebrain.github.io/nexus/vocabulary/defaultElasticSearchIndex",
          "@type": [
            "https://bluebrain.github.io/nexus/vocabulary/ElasticSearchView",
            "https://bluebrain.github.io/nexus/vocabulary/View"
          ],
          "_constrainedBy": "https://bluebrain.github.io/nexus/schemas/views.json",
          "_createdAt": "2021-05-12T09:44:22.259Z",
          "_createdBy": "http://localhost:8080/v1/realms/internal/users/delta",
          "_deprecated": false,
          "_incoming": "http://localhost:8080/v1/views/myorg/myproj/documents/incoming",
          "_original_source": "{\"resourceSchemas\":[],\"resourceTypes\":[],\"sourceAsText\":true,\"includeMetadata\":true,\"includeDeprecated\":true,\"permission\":\"views/query\",\"@type\":\"ElasticSearchView\",\"@id\":\"https://bluebrain.github.io/nexus/vocabulary/defaultElasticSearchIndex\"}",
          "_outgoing": "http://localhost:8080/v1/views/myorg/myproj/documents/outgoing",
          "_project": "http://localhost:8080/v1/projects/myorg/myproj",
          "_rev": 1,
          "_self": "http://localhost:8080/v1/views/myorg/myproj/documents",
          "_updatedAt": "2021-05-12T09:44:22.259Z",
          "_updatedBy": "http://localhost:8080/v1/realms/internal/users/delta",
          "_uuid": "a1147f0f-1650-49f2-8704-30ad97be8f31"
        }
      },
      {
        "_id": "http://localhost:8080/v1/resources/myorg/myproj/_/386a8c21-3acf-4f10-90ab-329466f5e04c",
        "_index": "delta_7e737b83-30a0-4ea3-b6c9-cd1ed481d743_3",
        "_score": 0.8566987,
        "_source": {
          "@id": "http://localhost:8080/v1/resources/myorg/myproj/_/386a8c21-3acf-4f10-90ab-329466f5e04c",
          "@type": [
            "https://bluebrain.github.io/nexus/vocabulary/ElasticSearchView",
            "https://bluebrain.github.io/nexus/vocabulary/View"
          ],
          "_constrainedBy": "https://bluebrain.github.io/nexus/schemas/views.json",
          "_createdAt": "2021-05-12T09:44:35.609Z",
          "_createdBy": "http://localhost:8080/v1/anonymous",
          "_deprecated": false,
          "_incoming": "http://localhost:8080/v1/views/myorg/myproj/386a8c21-3acf-4f10-90ab-329466f5e04c/incoming",
          "_original_source": "{\"@type\":[\"ElasticSearchView\"],\"mapping\":{\"dynamic\":false,\"properties\":{\"@id\":{\"type\":\"keyword\"},\"@type\":{\"type\":\"keyword\"},\"name\":{\"type\":\"keyword\"},\"number\":{\"type\":\"long\"},\"bool\":{\"type\":\"boolean\"}}},\"includeMetadata\":true,\"includeDeprecated\":true,\"sourceAsText\":true,\"resourceSchemas\":[],\"resourceTypes\":[]}",
          "_outgoing": "http://localhost:8080/v1/views/myorg/myproj/386a8c21-3acf-4f10-90ab-329466f5e04c/outgoing",
          "_project": "http://localhost:8080/v1/projects/myorg/myproj",
          "_rev": 1,
          "_self": "http://localhost:8080/v1/views/myorg/myproj/386a8c21-3acf-4f10-90ab-329466f5e04c",
          "_updatedAt": "2021-05-12T09:44:35.609Z",
          "_updatedBy": "http://localhost:8080/v1/anonymous",
          "_uuid": "ea4b3858-c1ac-4afb-ac94-13fdb1b53681"
        }
      },
      {
        "_id": "http://localhost:8080/v1/resources/myorg/myproj/_/indexing1",
        "_index": "delta_7e737b83-30a0-4ea3-b6c9-cd1ed481d743_3",
        "_score": 0.8566987,
        "_source": {
          "@id": "http://localhost:8080/v1/resources/myorg/myproj/_/indexing1",
          "@type": [
            "https://bluebrain.github.io/nexus/vocabulary/ElasticSearchView",
            "https://bluebrain.github.io/nexus/vocabulary/View"
          ],
          "_constrainedBy": "https://bluebrain.github.io/nexus/schemas/views.json",
          "_createdAt": "2021-05-12T09:53:32.704Z",
          "_createdBy": "http://localhost:8080/v1/anonymous",
          "_deprecated": false,
          "_incoming": "http://localhost:8080/v1/views/myorg/myproj/indexing1/incoming",
          "_original_source": "{\"@type\":[\"ElasticSearchView\"],\"mapping\":{\"dynamic\":false,\"properties\":{\"@id\":{\"type\":\"keyword\"},\"@type\":{\"type\":\"keyword\"},\"name\":{\"type\":\"keyword\"},\"number\":{\"type\":\"long\"},\"bool\":{\"type\":\"boolean\"}}},\"includeMetadata\":true,\"includeDeprecated\":true,\"sourceAsText\":true,\"resourceSchemas\":[],\"resourceTypes\":[]}",
          "_outgoing": "http://localhost:8080/v1/views/myorg/myproj/indexing1/outgoing",
          "_project": "http://localhost:8080/v1/projects/myorg/myproj",
          "_rev": 1,
          "_self": "http://localhost:8080/v1/views/myorg/myproj/indexing1",
          "_updatedAt": "2021-05-12T09:53:32.704Z",
          "_updatedBy": "http://localhost:8080/v1/anonymous",
          "_uuid": "ab97491a-0f30-4616-9906-28b92a8c0b79"
        }
      },
      {
        "_id": "http://localhost:8080/v1/resources/myorg/myproj/_/myview",
        "_index": "delta_7e737b83-30a0-4ea3-b6c9-cd1ed481d743_3",
        "_score": 0.8566987,
        "_source": {
          "@id": "http://localhost:8080/v1/resources/myorg/myproj/_/myview",
          "@type": [
            "https://bluebrain.github.io/nexus/vocabulary/ElasticSearchView",
            "https://bluebrain.github.io/nexus/vocabulary/View"
          ],
          "_constrainedBy": "https://bluebrain.github.io/nexus/schemas/views.json",
          "_createdAt": "2021-05-12T09:54:28.171Z",
          "_createdBy": "http://localhost:8080/v1/anonymous",
          "_deprecated": false,
          "_incoming": "http://localhost:8080/v1/views/myorg/myproj/myview/incoming",
          "_original_source": "{\"@type\":[\"ElasticSearchView\"],\"mapping\":{\"dynamic\":false,\"properties\":{\"@id\":{\"type\":\"keyword\"},\"@type\":{\"type\":\"keyword\"},\"name\":{\"type\":\"keyword\"},\"number\":{\"type\":\"long\"},\"bool\":{\"type\":\"boolean\"}}},\"includeMetadata\":true,\"includeDeprecated\":true,\"sourceAsText\":true,\"resourceSchemas\":[],\"resourceTypes\":[]}",
          "_outgoing": "http://localhost:8080/v1/views/myorg/myproj/myview/outgoing",
          "_project": "http://localhost:8080/v1/projects/myorg/myproj",
          "_rev": 3,
          "_self": "http://localhost:8080/v1/views/myorg/myproj/myview",
          "_updatedAt": "2021-05-12T09:58:20.507Z",
          "_updatedBy": "http://localhost:8080/v1/anonymous",
          "_uuid": "7e737b83-30a0-4ea3-b6c9-cd1ed481d743"
        }
      }
    ],
    "max_score": 0.8566987,
    "total": {
      "relation": "eq",
      "value": 4
    }
  },
  "timed_out": false,
  "took": 10,
  "_shards": {
    "failed": 0,
    "skipped": 0,
    "successful": 1,
    "total": 1
  }
}

Create a point in time

Creates a point in time on the underlying index of the view to be used in further search requests.

See the Elasticsearch documentation for more details

POST /v1/views/{org_label}/{project_label}/{view_id}/_pit?keep_alive={keep_alive}
  {...}

where …

  • {keep_alive}: Number - the time to live in seconds of the corresponding point in time.
Request
sourcecurl -XPOST \
-H "Content-Type: application/json" \
"http://localhost:8080/v1/views/myorg/myproj/myview/_pit?keep_alive=300"
Response
source{
  "id": "xxx"
}

Delete a point in time

Closes a point in time on the underlying index of the view when it is no longer used.

See the Elasticsearch documentation for more details

DELETE /v1/views/{org_label}/{project_label}/{view_id}/_pit
  {...}
Request
sourcecurl -XDELETE \
-H "Content-Type: application/json" \
"http://localhost:8080/v1/views/myorg/myproj/myview/_pit" -d \
'{
  "id": "xxx"
}'

Fetch Elasticsearch mapping

GET /v1/views/{org_label}/{project_label}/{view_id}/_mapping

Retrieves the view’s ElasticSearch mapping.

Example

Request
sourcecurl "http://localhost:8080/v1/views/myorg/myproj/myview/_mapping"
Response
source{
  "delta_f488d131-5f29-4721-979e-277a0850fed2_1": {
    "mappings": {
      "properties": {
        "title": {
          "type": "text",
          "fields": {
            "keyword": {
              "type": "keyword"
            }
          }
        },
        "description": {
          "type": "text"
        }
      }
    }
  }
}

Fetch statistics

GET /v1/views/{org_label}/{project_label}/{view_id}/statistics

Example

Request
sourcecurl "http://localhost:8080/v1/views/myorg/myproj/nxv:myview/statistics"
Response
source{
  "@context": "https://bluebrain.github.io/nexus/contexts/statistics.json",
  "@type": "ViewStatistics",
  "totalEvents": 3754,
  "processedEvents": 3754,
  "evaluatedEvents": 3754,
  "remainingEvents": 0,
  "discardedEvents": 0,
  "failedEvents": 0,
  "delayInSeconds": 0,
  "lastEventDateTime": "2021-04-30T15:04:44.021Z",
  "lastProcessedEventDateTime": "2021-04-30T15:04:44.021Z"
}

where…

  • totalEvents - total number of events in the project
  • processedEvents - number of events that have been considered by the view
  • remainingEvents - number of events that remain to be considered by the view
  • discardedEvents - number of events that have been discarded (were not evaluated due to filters, e.g. did not match schema, tag or type defined in the view)
  • evaluatedEvents - number of events that have been used to update an index
  • lastEventDateTime - timestamp of the last event in the project
  • lastProcessedEventDateTime - timestamp of the last event processed by the view
  • delayInSeconds - number of seconds between the last processed event timestamp and the last known event timestamp

Fetch indexing

GET /v1/views/{org_label}/{project_label}/{view_id}/offset

Example

Request
sourcecurl "http://localhost:8080/v1/views/myorg/myproj/myview/offset"
Response
source{
  "@context": "https://bluebrain.github.io/nexus/contexts/offset.json",
  "@type": "TimeBasedOffset",
  "instant": "2021-05-12T12:57:19.375Z",
  "value": "95d159f6-b321-11eb-a5f9-e5c0b5ea0976"
}

where…

  • instant - timestamp of the last event processed by the view
  • value - the value of the offset

Restart indexing

This endpoint restarts the view indexing process. It does not delete the created indices but it overrides the resource Document when going through the event log.

DELETE /v1/views/{org_label}/{project_label}/{view_id}/offset

Example

Request
sourcecurl -XDELETE "http://localhost:8080/v1/views/myorg/myproj/nxv:myview/offset"
Response
source{
  "@context": "https://bluebrain.github.io/nexus/contexts/offset.json",
  "@type": "ViewStatistics",
  "delayInSeconds": 0,
  "discardedEvents": 2,
  "evaluatedEvents": 8,
  "failedEvents": 0,
  "lastEventDateTime": "2021-05-12T09:58:20.507Z",
  "lastProcessedEventDateTime": "2021-05-12T09:58:20.507Z",
  "processedEvents": 10,
  "remainingEvents": 0,
  "totalEvents": 10
}
v1.11.x