ElasticSearchView
This view creates an ElasticSearch index
and stores the targeted Json resources into an ElasticSearch Document.
The documents created on each view are isolated from documents created on other views by using different ElasticSearch indices.
A default view gets automatically created when the project is created but other views can be created.
Processing pipeline
An asynchronous process gets trigger for every view. This process can be visualized as a pipeline with different stages.
The first stage is the input of the pipeline: a stream of events scoped for the project where the view was created.
The last stage takes the JSON document, generated through the pipeline steps, and stores it as a Document in an ElasticSearch index
Payload
{
"@id": "{someid}",
"@type": "ElasticSearchView",
"resourceSchemas": [ "{resourceSchema}", ...],
"resourceTypes": [ "{resourceType}", ...],
"resourceTag": "{tag}",
"sourceAsText": {sourceAsText},
"includeMetadata": {includeMetadata},
"includeDeprecated": {includeDeprecated},
"mapping": _elasticsearch mapping_
}
where…
{resourceSchema}
: Iri - Selects only resources that are validated against the provided schema Iri. This field is optional.{resourceType}
: Iri - Select only resources of the provided type Iri. This field is optional.{tag}
: String - Selects only resources with the provided tag. This field is optional._elasticsearch mapping_
: Json object - Defines the value types for the Json keys, as stated at the ElasticSearch mapping documentation.{sourceAsText}
: Boolean - If true, the resource’s payload will be stored in the ElasticSearch document as a single escaped string value of the key_original_source
. If false, the resource’s payload will be stored normally in the ElasticSearch document. The default value isfalse
.{includeMetadata}
: Boolean - If true, the resource’s nexus metadata (_constrainedBy
,_deprecated
, …) will be stored in the ElasticSearch document. Otherwise it won’t. The default value isfalse
.{includeDeprecated}
: Boolean - If true, deprecated resources are also indexed. The default value isfalse
.{someid}
: Iri - The @id value for this view.
Example
The following example creates an ElasticSearch view that will index resources validated against the schema with id https://bluebrain.github.io/nexus/schemas/myschema
. If a resource is deprecated, it won’t be selected for indexing.
The resulting ElasticSearch Documents fields will be indexed according to the provided mapping rules and they won’t include the resource metadata fields.
{
"@id": "https://bluebrain.github.io/nexus/vocabulary/myview",
"@type": [
"ElasticSearchView"
],
"mapping": {
"dynamic": false,
"properties": {
"@id": {
"type": "keyword"
},
"@type": {
"type": "keyword"
},
"name": {
"type": "keyword"
},
"number": {
"type": "long"
},
"bool": {
"type": "boolean"
}
}
},
"includeMetadata": false,
"includeDeprecated": false,
"sourceAsText": false,
"resourceSchemas": [
"https://bluebrain.github.io/nexus/schemas/myschema"
],
"resourceTypes": []
}
Endpoints
The following sections describe the endpoints that are specific to an ElasticSearchView.
The general view endpoints are described on the parent page.
Search Documents
POST /v1/views/{org_label}/{project_label}/{view_id}/_search
{...}
The supported payload is defined on the ElasticSearch documentation
The string documents
is used as a prefix of the default ElasticSearch view_id
Example
- Request
-
curl -XPOST -H "Content-Type: application/json" "https://nexus.example.com/v1/views/myorg/myproj/nxv:myview/_search" -d \ '{ "query": { "term": { "_deprecated": true } } }'
- Payload
-
{ "query": { "term": { "_deprecated": true } } }
- Response
-
{ "_shards": { "failed": 0, "skipped": 0, "successful": 0, "total": 0 }, "hits": { "hits": [ { "_score": 1.2039728, "_id": "http://example.com/myview", "_index": "kg_v1_803da242-d510-4e1a-a12d-d0e0b7018a41_684bd815-9273-46f4-ac1c-0383d4a98254_1", "_source": { "@id": "http://example.com/myview", "@type": [ "https://bluebrain.github.io/nexus/vocabulary/View", "https://bluebrain.github.io/nexus/vocabulary/ElasticSearchView", "https://bluebrain.github.io/nexus/vocabulary/Alpha" ], "_original_source": "{\"mapping\":\"{\\\"dynamic\\\":false,\\\"properties\\\":{\\\"@id\\\":{\\\"type\\\":\\\"keyword\\\"},\\\"@type\\\":{\\\"type\\\":\\\"keyword\\\"},\\\"name\\\":{\\\"type\\\":\\\"keyword\\\"},\\\"number\\\":{\\\"type\\\":\\\"long\\\"},\\\"bool\\\":{\\\"type\\\":\\\"boolean\\\"}}}\",\"_uuid\":\"89dd9858-cf87-4c46-94c2-24da0fcae0aa\",\"@context\":\"https://bluebrain.github.io/nexus/contexts/view\",\"@type\":[\"View\",\"ElasticSearchView\"],\"includeMetadata\":true,\"sourceAsText\":false}", "_incoming": "https://nexus.example.com/v1/schemas/myorg/myproj/nxv:myview/incoming", "_outgoing": "https://nexus.example.com/v1/schemas/myorg/myproj/nxv:myview/outgoing", "_self": "https://nexus.example.com/v1/schemas/myorg/myproj/nxv:myview", "_constrainedBy": "https://bluebrain.github.io/nexus/schemas/view", "_project": "https://nexus.example.com/v1/projects/myorg/myproj", "_rev": 6, "_deprecated": true, "_createdAt": "2018-09-18T16:39:22.748Z", "_createdBy": "https://nexus.example.com/v1/realms/myrealm/users/john", "_updatedAt": "2018-09-18T17:10:22.748Z", "_updatedBy": "https://nexus.example.com/v1/realms/myrealm/users/john" }, "_type": "doc" } ], "max_score": 1.2039728, "total": 1 }, "timed_out": false, "took": 11 }
Fetch statistics
GET /v1/views/{org_label}/{project_label}/{view_id}/statistics
Example
- Request
-
curl "https://nexus.example.com/v1/views/myorg/myproj/nxv:myview/statistics"
- Response
-
{ "@context": "https://bluebrain.github.io/nexus/contexts/statistics.json", "@type": "ViewStatistics", "totalEvents": 3754, "processedEvents": 3754, "evaluatedEvents": 3754, "remainingEvents": 0, "discardedEvents": 0, "failedEvents": 0, "delayInSeconds": 0, "lastEventDateTime": "2020-01-31T15:04:44.021Z", "lastProcessedEventDateTime": "2020-01-31T15:04:44.021Z" }
where…
totalEvents
- total number of events in the projectprocessedEvents
- number of events that have been considered by the viewremainingEvents
- number of events that remain to be considered by the viewdiscardedEvents
- number of events that have been discarded (were not evaluated due to filters, e.g. did not match schema, tag or type defined in the view)evaluatedEvents
- number of events that have been used to update an indexlastEventDateTime
- timestamp of the last event in the projectlastProcessedEventDateTime
- timestamp of the last event processed by the viewdelayInSeconds
- number of seconds between the last processed event timestamp and the last known event timestamp
Restart view
This endpoint restarts the view indexing process. It does not delete the created indices but it overrides the resource Document when going through the event log.
DELETE /v1/views/{org_label}/{project_label}/{view_id}/offset
Example