v1.2 To v1.3 Migration

As part of the v1.3 release CompositeViews were introduced requiring a change in the data model used to record the indexing progress of a View. The change is not backwards compatible, it requires dropping two Cassandra tables and a rebuilding all indices. Additionally, in order to benefit from the new ElasticSearch index configuration, a migration of the default ElasticSearchView of each project is required.

The migration steps are as follows:

1. Stop the 3 services: iam, admin and kg.
2. Backup the cassandra store.
3. Delete all ElasticSearch indices:

   curl -XDELETE 'http://{elasticsearch_host}/kg_*'
   

4. Delete all BlazeGraph namespaces:

   for i in `curl -s 'http://{blazegraph_host}/blazegraph/namespace?describe-each-named-graph=false' | grep sparqlEndpoint | grep -o --color "rdf:resource=\"[^\"]*" | sed 's/rdf:resource="//' | sed 's#/sparql$##' | grep -v kb | grep -v LBS`
      do curl -X DELETE "$i"
   done
   

5. Drop the kg.projections_progress and the kg.projections_failures tables:

   drop table kg.projections_progress;
   drop table kg.projections_failures;
   

6. Deploy the new service images (tag 1.3) for iam and admin.

7. Deploy the new kg (tag 1.3) image with the MIGRATE_V12_TO_V13=true environment variable. This will instruct the service to automatically migrate the default views. The service can be restarted without the migration environment variable after the following message is presented in the log:

   Finished migrating views mappings. 
   

Once the services are restarted with the new images all ElasticSearch and BlazeGraph indices will be rebuilt automatically. The progress for each view can be tracked either in Nexus Web or using the statistics endpoint for each view:

    GET /v1/views/{org_label}/{project_label}/{view_id}/statistics