1 Introduction
2 System Requirements
3 Licensing
4 Migration Modes
- 4.1 Online Migration
- 4.2 Offline Migration
5 Installation
- 5.1 Sample installation for Apache Tomcat 7 via WebUI
- 5.2 Sample installation via webapps-folder
6 Setting up a migration process with an external SQL database
7 Additional Requirements for Online Migrations
8 Configuring KGS Migration4ArchiveLink
9 Defining and Editing Datasources
10 Configuring an Online Migration
11 Configuring an Offline Migration
12 Starting and running a Migration
- 12.1 Starting / Running the Online Migration process
  - 12.1.1 Generating a DMIS File for the migration process
    - 12.1.1.1 TOA01
    - 12.1.1.2 TOA02
    - 12.1.1.3 TOA03
    - 12.1.1.4 TOAHR
    - 12.1.1.5 TOADL
    - 12.1.1.6 ADMI_FILES
- 12.2 Starting / Running the offline Migration process
13 Migration Monitoring
14 Errors and Duplicates in Migration runs
15 Performance Logging
16 Logging

Introduction

KGS Migration4ArchiveLink is a Java-based software application which allows you to migrate the contents of any SAP ArchiveLink® compliant document storage and archiving system.

Ever since archiving solutions exist, document migrations have been required. KGS Migration4ArchiveLink allows you to define and control migration processes for SAP data and documents.

You may manage and monitor every aspect of the migration while necessary steps are being executed in the background.

System Requirements

For the system requirements please see: Prerequisites for KGS Software Components

Licensing

In order to use KGS Migration4ArchiveLink follow the guide for requesting licences: KGS Software licensing

Migration Modes

KGS Migration4ArchiveLink supports migration runs in two different modes:

Online Migration
Offline Migration

Both migration modes have in common, that you may define a list of identifying data which determines which documents have to be migrated. The difference is, how this identifying data is being obtained.

Online Migration

This feature is no longer available from version 3.2.6

When setting up an online migration, the KGS Migration4ArchiveLink uses an RFC (Remote Function Call) session to obtain all document references visible to your SAP system.

At this point, only basic metadata such as content repositories and object categories (we currently support the following: Images, Print Lists, Data Objects) are being read from the SAP configuration tables.

This information will be stored in the KGS Migration4ArchiveLink repository.

Please note: The table BDS_BAR_EX will not be automatically part of an online migration and has to be exported manully. You may use SAP transaction SE16

SAP tables being used

SAP tables being used
TOA01
TOA02
TOA03
TOAHR
TOADL
ADMI_FILES

Offline Migration

When setting up an offline migration the KGS Migration4ArchiveLink doesn't connect to your SAP system at all.

Instead you'll have to manually add the repositories which should be included in the migration run. You'll also have to provide a list with the Archive Document IDs, representing all documents that are going to be migrated.

You may either create this kind of list by an SAP transaction or through SQL statemets. This approach has the advantage that the user will be able to exclude documents from the migration process (e.g.: if the documents are too old or unused).

Installation

KGS Migration4ArchiveLink is being shipped as a web archive container (WAR file). Depending on your webserver the installation procedure might differ.

Sample installation for Apache Tomcat 7 via WebUI

Open your tomcat's "Manager App", go to section WAR file to deploy, select "KGSAdmin-MIG.war" and click on the button Deploy.

After the deployment procedure has been completed a new web application called KGSAdmin-MIG will appear in the list of the installed web applications.

Please note: If the name of the *.war-file was changed, that the name of the deployed web application has been changed as well.

Sample installation via webapps-folder

After downloading KGS Migration4ArchiveLink you unzip the contents of the downloaded zip-archive to <Webserver-installation-folder>/webapps.

The zip-archive contains KGS Migration4ArchiveLink in form of a war-container, which will be automatically decompressed by Tomcat Webserver.

After tomcat has finished decompressing the war-container you may restart Tomcat webserver one time.

Please note: Depending on the hardware and system Tomcat Webserver is being executed on it might take some time to decompress the content of the war-archive.

Setting up a migration process with an external SQL database

KGS Migration4Archivelink is being shipped with an integrated database (H2 database). If you intend to use an external database (i.e.: Oracle, Microsoft SQL Server, etc.), you'll have to define and configure a database connection to the external database ( Main → Edit Configuration → Database).

If you intend to use the database which has been shipped with KGS Migration4ArchiveLink you may skip this step.

Additional Requirements for Online Migrations

In order to be able to perform online migrations an SAP connection has to be setup in the KGS SAP Connector (Main → KGS SAP Connector).

Please find additional information about the configuration of KGS SAP Connector here: KGS SAP Connector.

In addition to that KGS Migration4ArchiveLink requires a user thich has access to the SAP system.

A CPIC user ID (sometimes also referred to as RFC user ID) has to be created with the following authorization profile in order to give access to the relevant data to KGS Migration4ArchiveLink.

All access from KGS Migration4ArchiveLink software is read only.

Configuring KGS Migration4ArchiveLink

In case you wish to define and change the global configuration of KGS Migration4ArchiveLinke please go to "Main → Migration4ArchiveLink → Edit Configuration".

Within this section you may define and change the global settings of the application.

All changes made to the global configuration of the application might impact every migration process.

Configuring mail notifications

In case you wish to setup mail notifications set the following parameters.

Configuring data conversion during migration runs

KGS Migration4ArchiveLink offers you the option to convert components during the migration process.

If you wish to define conversions set the following parameters in a way which fits your requirements.

Configuring global RFC settings

Within this tab you may change the global record count for data retrievals from your SAP system.

Configuring the usage of an external database

In case you decide to use your own database instead of the database that had been shipped with KGS Migration4ArchiveLink you'll have to provide and define the information listed below in order to use your own database.

Please note that the user provided by you needs to have the access rights to create the tables and schemas which are being used by Migration4Archivelink

Configuring common settings of KGS Migration4ArchiveLink

Within this tab you may change the behavior of KGS Migration4ArchiveLink regarding internal aspects like logging, which directory is being used by Migration4ArchiveLink as working directory and so on.

Entering the license key

In order to enter the license key which has been provided by KGS, please enter it within this tab.

Configuring the behavior of KGS Migration4ArchiveLink

The following settings can be made in this section:

Dealing with different time zones

When dealing with servers distributed around the globe, it's possible that urls might be marked as expired due to a difference in time zone. Please change the settings in this section if necessary.

Defining and Editing Datasources

Datasources define where and how KGS Migration4Archive shall look for documents to migrate as well as the target server to which the retrieved documents shall be archived.

Please find the window to define datasources under Main → Migration4Archivelink → Edit Datasources.

Datasource window without any sources defined	Datasource window with the source "MIG1" defined

Datasource window without any sources defined	Datasource window with the source "MIG1" defined

Here you may either define a new datasource by clicking the "Add Datasource"-button, edit an existing datasource by clicking the "Edit"-button or delete an existing datasource by clicking the "Delete"-button as well as jump to the migration monitor.

Adding a new Datasource

In order to add a new datasource please click the "Add Datasource" button which opens the "Add/Update DataSource"-modal, with which you may define a new datasource.

In case you wish to add an online migration and get the warning "No SAP Configuration:" please install and setup the KGS SAP Connector

Modal to add a datasource for an Online Miration	Modal to add a datasource for an Offline Migration

Modal to add a datasource for an Online Miration	Modal to add a datasource for an Offline Migration

Adding a DataSource for an Online Migration

This feature is no longer available from version 3.2.6

To define a datasource for an online migration the "SAP System ID" and the "RFC Configuration" are mandatory. The "Migration Type" needs to be set to "Online Migration".

It's recommended to test the chosen "RFC Configuration" by clicking the "Test"-button. A message will show whether the chosen "RFC Configuration" led to a connection to the SAP system.

In case of an error message please make sure that the entered information (e.g.: IP address/hostname, user ID, and / or password) is correct and that the server or virtual machine the KGS Migration4ArchiveLink is being executed on is able to establish a connection to the SAP system server.

Messagebox shown on failed test	Messagebox shown on successful test

Messagebox shown on failed test	Messagebox shown on successful test

After clicking the "Add/Update"-button KGS Migration4ArchiveLink system will start to collect and download content repository related information from the specified SAP system.

Adding a DataSource for an Offline Migration

In order to define a datasource for an offline migration the parameters "SAP System ID", "Alias" as well as "Offline Repositories" are all mandatory parameters. The "Migration Type" needs to be set to "Offline Migration".

Parameterdefinitions for "Add Datasource"

Available in Migration Type	Parameter	Value	Description	Default

Available in Migration Type	Parameter	Value	Description	Default
	SAP System ID		Please enter die SAP System ID of your source system.
	Alias		Please choose an alias for the data source definition. Recommendation: Choose the content repository name of the source system as alias.
	Migration Type	Combobox-Entry	Defines the type of migration, that shall be performed in this migration, Online Migration Offline Migration	Online Migration
Offline Migration	Offline Repositories	Comma-separated repository list	Please enter the repository / -ies for which migrations shall be executed. In case you wish to migrate more than one repository please provide the repositories as a comma-separated list. e.g.: FI,B3,H4,TC
Online Migration	Select RFC Configuration	Combox-Entry	Please choose here the SAP-System/-s that have been defined in KGS SAP Connector	First entry in the list of SAP-systems in the SAP Connector

Editing an existing Datasource

By clicking the "Edit"-button, a defined datasource may be edited and parameters may be changed.

Please note that the "Migration Type" parameter may not be changed after the creation of a datasource.

Delete an existing Datasource

An existing datasource may be deleted by switching within the tab pane to the datasource and clicking the "Delete"-button.

Please note that a datasource might not be deletable after the migration process has been started.

Configuring an Online Migration

This feature is no longer available from version 3.2.6

Before starting a migration process, KGS Migration4ArchiveLink requires a set of parameters which describe the source and destination storage systems. In order to define, inspect or edit these parameters please use the -button.

You'll be asked to provide the following information.

Please note: The HTTP Source parameters may be prefilled with the information obtained from the content repository settings found in the SAP system.

After entering all required content repository parameters you may use the Update-button to confirm the new content repository configuration.

Now you may start to download ArchiveLink table entries by clicking the -button. All available link table entries are being downloaded and checked for consistency. After the downloads have been finished the system may display a number of accurate / faulty entries downloaded so far.

The result may look similar to the one in the picture below.

Datatype information retrieved from repository 1D

Datatype information retrieved from repository 1D

Configuring an Offline Migration

Before starting a migration process, KGS Migration4ArchiveLink requires a set of parameters which describe the source and destination storage systems. In order to define, inspect or edit these parameters please use the -button.

You'll be asked to provide the following information.

Please note: The HTTP Source parameters may be prefilled with the information obtained from the content repository settings found in the SAP system.

After entering all required content repository parameters you may use the Update-button to confirm the new content repository configuration.

Depending on the number and names of the content repositories defined in the parameter "Offline Repositories" the result may look similar to the picture below.

Three repositories defined for Offline Migration process

Three repositories defined for Offline Migration process

Starting and running a Migration

After configuring an online or offline migration you may start and define multiple migration runs per repository. This allows you to control which documents are being migrated and how many documents are being migrated for example during working hours.

Starting / Running the Online Migration process

KGS Migration4ArchiveLink allows you to either migrate the whole contents of the content repository or provide a list of documents which shall be migrated to the new archiving system.

If you plan to replace an old archiving system and wish to migrate all documents archived in the current system, press the -button in order to retrieve datatype information from the SAP system.

You may get a list of contents which may be migrated to the new archiving system similar to the one below.

Contents of the content repository 2F after retrieving information from the SAP system

Contents of the content repository 2F after retrieving information from the SAP system

By pressing the 🗘-button next to the found datatype Migration4ArchiveLink will start to retrieve detailed information regarding this type. Once the information has been retrieved you may start the migration process by clicking the ⚙-button.

The view will switch to the "Migration Monitor" which provides you with information regarding the currently started migration process.

If you wish to migrate only a specific list of documents you may open a file-upload-dialogue by clicking the "+"-button, entering the Datatype and uploading a file which contains the content repository, the DocumentID as well as the original archiving date.

Example:

FI 0A1A66A7086942969469594488AAFC28 2020-04-23

FI FBC683A91EFB4D22A913D1D472DA02A7 2019-03-16

Migration4ArchiveLink will process the file provided by you and create a migrationstep that includes only the documentIDs from that list.

Generating a DMIS File for the migration process

DMIS files may either be created by using a standard report offered by SAP or by creating a custom report comprised of standard SQL statements. By defining the document set yourself you may

exclude outdated DocumentID (i.e.: all DocumentIDs that have already exceeded their retention period)
intend to include those documents that are attached to SAP records where the corresponding organizational unit was part of a divestiture, reorganization or compny carve-out.

In order to create a custom report, you may use the following SQL statements.

SQL to get DMIS for Images	SQL to get DMIS for Print Lists	SQL to get DMIS for Data Objects

SQL to get DMIS for Images

SQL to get DMIS for Print Lists

SQL to get DMIS for Data Objects

TOA01

select || ARCHIV_ID || ' ' || ARC_DOC_ID || ' ' || AR_DATE || 
from SAP<SID>.TOA01 
where ARCHIV_ID = <ContRepID>;

TOA02

select || ARCHIV_ID || ' ' || ARC_DOC_ID || ' ' || AR_DATE || 
from SAP<SID>.TOA02
where ARCHIV_ID = <ContRepID>;

TOA03

select || ARCHIV_ID || ' ' || ARC_DOC_ID || ' ' || AR_DATE || 
from SAP<SID>.TOA03 
where ARCHIV_ID = <ContRepID>;

TOAHR

select || ARCHIV_ID || ' ' || ARC_DOC_ID || ' ' || AR_DATE || 
from SAP<SID>.TOAHR 
where ARCHIV_ID = <ContRepID>;

TOADL

select || ARCHIV_ID || ' ' || ARC_DOC_ID || ' ' || AR_DATE || 
from SAP<SID>.TOADL 
where ARCHIV_ID = <ContRepID>;

ADMI_FILES

select || CREP || ' ' || ARCH_DOCID || ' ' || CREAT_DATE || 
from SAP<SID>.ADMI_FILES 
where CREP = <ContRepID>;

Starting / Running the offline Migration process

In case you decided to setup an offline migration, Migration4ArchiveLink needs to get a list of documentIDs which shall be migrated.

Please provide a list that contains the following information:

Example:

FI 0A1A66A7086942969469594488AAFC28 2020-04-23

FI FBC683A91EFB4D22A913D1D472DA02A7 2019-03-16

After processing this list, you'll be shown the number of entries which shall be migrated due to the information provided as well as the option to either start the migration process by clicking on the -button or delete the uploaded list.

Preview after a list of DocumentIDs to be migrated from content repository FI had been processed

Preview after a list of DocumentIDs to be migrated from content repository FI had been processed

Migration Monitoring

While a migration is running the Migration Monitor offers latest status information. Migration Monitor is being found under "Main → Migration4ArchiveLink → Migration Monitor". On the left-hand side you'll find an overview of the repositories as well as the datatypes for which migration runs have been started. On the right-hand side you'll see detailed information about migration run selected.

This information includes the following details:

Detail	Description

Detail	Description
Threads	Number of threads which are being used to simultaneously migrate documents.
Total Documents	Number of documents existing in the repository.
Migrated Documents	Number of documents migrated in this run.
Reprocessed Documents	Number of documents reprocessed in this run.
Duplicates	Number of duplicates.
Migration Errors	Number of documents that couldn't be migrated due to errors.
Migration Volume	Size of the content that has been migrated.
Time elapsed	Time that passed since the start of the migration run.
Time left	Presumable time until the migration has been finished.
Documents / day	Average number of documents that can be migrated per day.
Documents / hour	Average number of documents that can be migrated per hour.
Documents / min	Average number of documents that can be migrated per minute.
Documents / sec	Average number of documents that can be migrated per second.

In addition, it is possible to edit the selected content repository. To do so, use the edit button. A dialog box will pop up showing the current configuration data.

Caution: Changing the source or the destination parameters while a migration is running will lead to an incomplete migration! It is highly recommended not to change the HTTP Source or HTTP Destination section while a migration is running.

While a migration is running you are able to stop/start this run. You can also de-/increase the number of threads that are being used to migrate documents in parallel using the arrow buttons (left → decrease, right → increase).

After a migration process has finished, the start button will change to if any migration error has occurred. In this case, the button provides a reprocessing option for erroneous documents. In case of unexpected errors resulting in configuration changes, this button ensures complete migration pro-cessing without adding a new data source.

The Migration Monitor will show you in the right-hand details view how many documents are included in the selected migration run, how many documents already have been migrated and in case of errors (i.e.: Documents, that are not accessable right now or can't be retrieved from the source repository) the number of errors.

In case of errors you may click on the number of erros to switch to the Errors/Duplicates view.

Migration run with nine migration errors.

Migration run with nine migration errors.

Errors and Duplicates in Migration runs

KGS Migration4ArchiveLink provides you with an overview of the migration runs executed against a source repository, which may help dealing with errors (i.e.: Missing files, etc.). Please find the "Errors/Duplicates"-overview under "Main → Migration4ArchiveLink → Errors/Duplicates"

Errors/Duplicates overview of an offline migration done against the content repository MH

Errors/Duplicates overview of an offline migration done against the content repository MH

By clicking on the button next to the number of duplicates / errors you may open the details overview, which contains information about every document that couldn't get migrated and the reason why the migration failed for this document.

Expanded errors overview for run against content repository MH

Expanded errors overview for run against content repository MH

Performance Logging

KGS Migration4ArchiveLink is being shipped with a performance monitor which allows you to get an overview about the currently running miration processes. Please find the Performance monitor under "Main → Migration4ArchiveLink → Performance Log".

In order to check the performance of the currently running migration, you may choose which information shall be collected by ticking the corresponding checkbox and pressing the "Start"-button.

Migration4ArchiveLink will start generating a graph which shows the overall time it took to migrate a specific document as well as every time information checked.

You may choose from the following information.

Parameter	Description

Parameter	Description
Overall Time (ms)	Shows the overal time it took to migrate a specific document (always active)
Source Access Time (ms)	Shows in milliseconds how long it took to access the sources content server.
Destination Access Time (ms)	Shows in milliseconds how long it took to access the destination content server.
Internal Processing Time (ms)	Shows in milliseconds how long it took for Migration4ArchiveLink to process the information internally.

Depending on the information checked your graph might look similar to the one below.

Performance overview of started migration process on repository MH

Performance overview of started migration process on repository MH

Logging

The log file configuration can be accessed through the OSGi configuration menu:

OSGi → Configuration:

Click on the name “KGS File Logger” or use the edit button to enter the configuration.

The following details may be specified:

You may now specify the file name and log directory to be used. Please keep in mind that the log file may need a lot of free disk space when running a migration with debug level 4 enabled.

You may specify the period (in days) a log file will be kept before automatic deletion takes place. A value of 0 means no automatic deletion at all.

Setting parameter “Rotate LogFile” = 1 will result in a new log file being created every day