Starting with version 3.0.0 of tia contentserver core, the database (both protocol and metadata) is managed by flyway. Flyway is a framework that allows the management of databases using scripts and records the used ones. This allows applications to self-manage the database even when changing structure, migrating data or complex sql based operations.
In contrast to using JPA to manage the database, flyway needs sql scripts for each database supported. By default we currently provide scripts for H2, MSSQL server and Oracle.
Flyway concept and naming patterns
The database migrations are done using sql scripts which are applied and the usage is recorded in a flyway history table in the database. Upon each start, the history table is compared to the existing scripts and if scripts are missing, the migrations will be applied.
We can use two types of files: versioned migrations and repeatable migrations.
Version migrations have the pattern V<Version>__<description>.sql. The capital letter V is important, lowercase will not be recognized. The version can either be in the from 2_9_7 or 2.9.7 both are identical for flyway. The description can be any text, spaces must be replaced by lowercase stash and should not be too long. Those migrations are executed in the versioning order. Any already applied file must not be changed.
Repeatable migrations have the pattern R__<description>.sql. Both the capital letter R and the restrictions for the description apply as for the versioned migrations. The repeatable migrations are executed after all versioned migrations are applied. In contrast to versioned, the repeatable migrations can be changed once applied. Every time the content changes, the files are re-executed. Those migrations can be used for example, when the filling of a field changes and the existing values need to be adapted to the new value.
Customizing
The database services are managed for each repository, therefor each database based service in the repository.cfg has an optional parameter scripts. The default value is classpath:sql/[meta|metav2|prot]/migration/[h2|sqlserver|oracle] depending on the service and the database vendor.
When using custom scripts, either additional or complete replacement, the vendors must not be mixed, i.e. the locations specified must point to a directory containing the scripts for only one vendor.
Adding additional scripts
In the case additional scripts are required, the scripts parameter can be used to provide them. For example for the oracle version of the protocol service additional scripts in the location /data/custom/kgs/protocol.
<repo>.protocolservice.database.scripts=classpath:sql/prot/migration/oracle,filesystem:/data/custom/kgs/protocol
In order to avoid conflicts with the versioned migrations contained by default, only repeatable migrations should be used. The order of the locations does not matter.
Adding support for non-default vendors
When adding custom support for a database vendor, before each update it is required to check if there are changes which need to be added to those scripts.
In case a database vendor not supported by default is needed, the scripts parameter can also be used. In addition to the JDBC-Driver some vendors also require an additional flyway dependency (e.g. mysql). The list of supported databases can be found here: https://documentation.red-gate.com/fd/supported-databases-184127454.html .
For example, using a custom vendor for metasaervice database v2, and the scripts are located in /kgs/data/sql/vendor
<repo>.metaservice.databasev2.scripts=filesystem:/kgs/data/sql/vendor
The jar files for both the driver and the flyway dependency can then be placed in a lib folder, say /kgs/lib and must then be added to the application, depending on the deployment, using the parameter loader.path, e.g. for container deployments LOADER_PATH=/kgs/lib.