Number of threads (MIG)

This view from the task manager should also be used to determine the thread count.

This article relates to 64 Bit platforms. In older x86 systems the architecture and the core usage looks different.

In the migration you have the option to set the number of threads manually but there are some factors to consider.

First of all: More does not always mean better. More about this later in this article. A thread is a logical unit that can process a task independently of other factors.

Every machine has physical CPUs (Central Processing Unit). This is also referred to as a "core".

Almost every modern CPU supports the so-called "Hyperthreading" (Hyper-threading ). Here the CPU is divided into logical processor units. Usually there are two per CPU (See screenshot)

This view from the task manager should also be used to determine the thread count.

On Linux this would be possible with the command:.

lscpu

Sample calculation:

In this example, we see a server that has been optimized for many concurrent processes (usually server systems). The mainboard has 4 sockets with 12 cores each.
Each of these cores has 2 logical threads per core. So here would be the maximum number:

9*12*2 = 96 threads (with 48 cores). The heart rate of the individual cores (here 2.4Ghz) indicates the speed at which an operation is processed.
The following applies in principle: the more Ghz, the faster (and the more power consumption).
In the server environment on which many operations are to be executed (as e.g. with the KGS migration) the speed of the individual cores plays only a subordinate role, since a single routine is not CPU-intensive and thus also gets along well with a lower number of Ghz. Here the 96 threads are much more important.

Conclusion: The computer from the screenshot (Linux) can run 96 threads in parallel. Thus, the migration can also be increased to 96 threads to get an optimal utilization.

While the upper computer is a desktop computer (Windows), which was optimized for fewer parallel steps but faster (e.g. calling Office products, browser etc.). In this case, we already reach the optimal load with 4 threads.

What happens if I specify "too few" or "too many" threads in the migration?

Too few: Basically, the computer is not fully utilized. However, this may be intentional, so that resources are "reserved" for other applications.
Fewer threads leads to a "slower" migration, because less concurrent operations are executed.
Assuming the migration of a document takes exactly one second, with 30 threads we would also manage 30 documents per second. With 96 threads also 96 documents per second.
How long a migration takes per document depends on other factors: Retrieval time of the source document (network, data carrier on which the source is located etc.)
as well as the storage time to the target (network, data carrier on the target archive). S3, for example, is not considered the best performing system).
Too many: If more threads are requested than the machine actually owns/accesses, there is a "switch" between the threads in the machine.
This is effectively always the case in reality. In the Windows screenshot you can see the thread count 3916. This is the sum of threads of all processes (programs, here: 334) that are currently in use. If some of these threads would use 100% CPU this would have an influence on the other threads, because only then the thread with the 100% CPU load takes away the resources from the other threads. If this is not the case (assuming each thread uses only 0.5 % CPU) this process is processed and the next thread is processed. This process is also called context switch (Context switch).

So if we specify 60 threads in the migration (from the Windows screenshot with 4 effective threads), this causes the processor to "schedule" the 56 threads and whenever an operation is waiting for in one of the effective threads (e.g. waiting for the I/O operation like file reading or similar) a thread switch to one of the remaining 56 threads takes place. If here again something is waited for or the process is completed it is switched to the next one or a previously incomplete one is continued.

In mass, however, this leads to the opposite effect: Managing the additional 56 threads and jumping into just these in between ensures that the individual processes (and thus the migration of a document) takes longer.

How does it behave with multiple webapps / webservers on the same machine?

Since the physical hardware does not change, programs on a server/computer must also share the same resources. This results in the effect of context switching (see point 2 "too many").
Thus, it is also not advisable to deploy multiple web apps / web servers on the same machine with the maximum number of threads.
So the optimal rule is: number of logical threads of the machine / number of webapps with corresponding threads each.
In short: Two migrations on the same server with 2 threads each has the same (optimal) effect as one migration with 4 threads (if the system supports only 4 logical threads).
With 96 threads, the optimum for two migrations would be 48 threads each.

Conclusion:

It is recommended to display the hardware specifications either via the task manager or via the lscpu command for Linux. The value given here should be used as a guideline for the number of threads.
A too high "fantasy" number leads to a negative effect. Too few threads will result in a slow migration because the resources are not fully utilized.