v1.11 To v1.12 Migration

Schema changes

Run the differents scripts starting with V1_12 in order:

https://github.com/BlueBrain/nexus/tree/$git.branch$/delta/sourcing-psql/src/main/resources/scripts/postgres/init

Indexing in Elasticsearch and Blazegraph

Those different changes in 1.12 require a full reindexing in Elasticsearch and Blazegraph:

• default indexing in Elasticsearch now pushes to a single index
• projects, resolvers, schemas, storages and views are not indexed anymore
• some metadata fields are not indexed anymore

The choice of migration will depend on your current setup, needs, and uptime requirements that you want to uphold for your users.

If you can accept partial results in Elasticsearch/Blazegraph during the reindexing process, this migration can be achieved as follows:

• Make a backup of your PostgreSQL instance
• Deploy new instances of Elasticsearch and Blazegraph
• Scale down Nexus Delta, update the image version and change its configuration to point to those new instances
• Configure Delta via the plugins.elasticsearch.main-index.shards depending on your amount of data and following the Elastic recommendations about sharding
• You can optionally allocate more resources to Delta, PostgreSQL, Blazegraph and Elasticsearch to accommodate for the load related to the indexing
• You may also want to reindex in Elasticsearch first and then in Blazegraph if the load is too high

• Run the following SQL script to delete the former default elasticsearch views and their progress:

  DELETE FROM scoped_events WHERE type = 'elasticsearch' AND id = 'https://bluebrain.github.io/nexus/vocabulary/defaultElasticSearchIndex';
  DELETE FROM scoped_states WHERE type = 'elasticsearch' AND id = 'https://bluebrain.github.io/nexus/vocabulary/defaultElasticSearchIndex';
  DELETE FROM projection_offsets WHERE module = 'elasticsearch' AND resource_id = 'https://bluebrain.github.io/nexus/vocabulary/defaultElasticSearchIndex';
  DELETE FROM projection_offsets WHERE name = 'event-metrics';
  

• Run the following SQL script to delete the progress of Blazegraph indexing:

  DELETE FROM projection_offsets WHERE module = 'blazegraph';
  

• Start Nexus Delta

• Monitor the load of the different components and the indexing process by running the following query

  SELECT * FROM projection_offsets ORDER BY updated_at DESC;