Generally an S3 enabled storage system is supported as content service.
Following S3 functions will be used: Required S3 functions
List of officially support S3 Systems: Supported S3 storage products
Using S3 managed buckets: Using S3 managed content service
Integration of Credential provider: Credentials provider
In case of using retention or legal holds, the user configured to access the buckets must also be the owner of the buckets, otherwise the check if object lock is enabled will fail.
From the S3 documentation:
x-amz-expected-bucket-owner
The account ID of the expected bucket owner. If the bucket is owned by a different account, the request fails with the HTTP status code 403 Forbidden (access denied). (https://docs.aws.amazon.com/AmazonS3/latest/API/API_GetObjectLockConfiguration.html )
Description of configuration parameters in repository.cfg
There are two types for S3 content service: s3blobstore and s3managedbucketblobstore. First the common parameters for both of them are listed before the specific parameters for the managed S3 service.
Content-Service
Service | Präfix | Servicetyp | Parameter | Optional | Unterstützte Werte | Default | Typ | Funktion | Verfübar ab |
|---|---|---|---|---|---|---|---|---|---|
Content-Service | <repo>.contentservice |
| type | n | s3blobstore, s3managedbucketblobstore | noop | String | type definition of the content service | 1.0.1 |
s3blobstore, | 1.0.1 | ||||||||
Content-Service | <repo>.contentservice | s3blobstore | connectionuser | n, wenn credentialsprovider auf “basic“ gestellt wird | <user> |
| String | User for S3 connection | 1.0.1 |
Content-Service | <repo>.contentservice | s3blobstore | connectionpass | n, when credentialsprovider is“basic“ | <user> |
| String | Connection-password for S3 (may be an Alias for a password in keystore) | 1.0.1 |
Content-Service | <repo>.contentservice | s3blobstore | maxconnections | y | <user> | 50 (AWS default) | Num | Max Connection-Pool of client | 1.0.1 |
Content-Service | <repo>.contentservice | s3blobstore | endpoint | y | <user> |
| URI witout protocol, e.g.: 127.0.0.1:9000 | Endpoint for S3 access (without Schema). The Endpoint is usally defined using parameter “region”. If Enpoint is defined, the parameter “region” will be ignored. | 1.0.1 |
Content-Service | <repo>.contentservice | s3blobstore | protocol | y | http,https | https | String | Client connection protocol | 1.0.1 |
Content-Service | <repo>.contentservice | s3blobstore | connecttimeout | y | <user> | 10000 | Num | Client connection-timeout (ms) | 1.0.1 |
Content-Service | <repo>.contentservice | s3blobstore | requesttimeout | y | <user> | 0 | Num | Client request timeout (ms) 0=Disabled | 1.0.1 |
Content-Service | <repo>.contentservice | s3blobstore | requestsigner | y | NoOpSignerType, QueryStringSignerType, AWS4SignerType, AWS4UnsignedPayloadSignerType, AWS3SignerType | String | Signer for signing of requests to AWS | 1.0.1 Signer “AWS4UnsignedPayloadSignerType“ may lead to problems when using certain special characters | |
Content-Service | <repo>.contentservice | s3blobstore | region | y | <user> | String | S3 Region | 1.0.1 | |
Content-Service | <repo>.contentservice | s3blobstore | allowcreatebuckets | y | true, false | true | Boolean | Enable creation of Buckets | 1.0.1 |
Content-Service | <repo>.contentservice | s3blobstore | bucketname | n | <user> |
| String | S3 Bucket for content files | 1.0.1 |
Content-Service | <repo>.contentservice | s3blobstore | clientoptions | y | pathstyleaccess:true |
| String | S3 ClientOption | 1.0.1 |
Content-Service | <repo>.contentservice | s3blobstore | contrepinpath | y | true, false | false | Bool | Use Repository Name as Root Folder | 1.0.1 |
Content-Service | <repo>.contentservice | s3blobstore | onbucketcreate | y | <user> | String | Script to execute when Bucket was created | 1.0.1 | |
Content-Service | <repo>.contentservice | s3blobstore | onbucketcreateworkingdir | y | <user> | Pfad | Needs to be defined when onbucketcreate is enabled | 1.0.1 | |
Content-Service | <repo>.contentservice | s3blobstore | calculatestreamhash | y | true, false | true | Boolean | After encryption the hash values in ContentServices are calculated again | 1.0.2 |
Content-Service | <repo>.contentservice | s3blobstore | EnforceUTF8ForContentDisposition | y | true, false | false | Boolean | false: should be set, if a ContentDisposition filename needs to be encoded, otherwise the original format is delivered, true: always encode in UTF-8 | 1.4.1 |
Content-Service | <repo>.contentservice | s3blobstore | objectlockenabled | y | true, false | true | Boolean | Setting objectLockEnabled when creating a new Bucket | 1.9.0 |
Content-Service | <repo>.contentservice | s3blobstore | maxidle | y | <user> | 60000 | Num | Maximum time in ms after removing an unused connection from ConnectionPool | 1.9.0 |
Content-Service | <repo>.contentservice | s3blobstore | validateafterinactivity | y | <user> | 5000 | Num | Polling time in ms for checking if connection in pool is still open. | 1.9.0 |
Content-Service | <repo>.contentservice | s3blobstore | cleanVersions | y | true,false | true | boolean | true: Delete all previous versions for update and delete operations when using buckets with enabled versioning. false: keep all versions | 1.9.0 |
Content-Service | <repo>.contentservice | s3blobstore | credentialsprovider | y | basic, instanceprofile | basic | String | basic: Authentication whith username and password instanceprofile: can be used when both, tia Core and S3 bucket S3 Bucket are deployed in AWS - then no direct authentification is necessary, as this is handled by the internal permission group. | 2.0.3 |
s3managedbucketsblobstore | 1.0.1 | ||||||||
Content-Service | <repo>.contentservice | s3blobstore | region | y | <user> | String | S3 Region | 1.0.1 | |
Content-Service | <repo>.contentservice | s3blobstore | AllowCreateBuckets | y | true, false | true | Boolean | Enable creation of Buckets | 1.0.1 |
Content-Service | <repo>.contentservice | s3blobstore | bucketgroups | y | <user> | 1 | Num | Number of bucket groups to use for new storage files | 1.0.1 |
Content-Service | <repo>.contentservice | s3blobstore | bucketspergroup | y | <user> | 5 | Num | Number of buckets in a group (Attention: don’t change after first use!!) | 1.0.1 |
Content-Service | <repo>.contentservice | s3blobstore | bucketnameformat | y | <user>, z.B. %8.8s |
| String | Number of characters of generated bucket names. When e.g. set to 8 characters, the name is filled with leading zeros to be 8 characters long. | 1.0.1 |
Content-Service | <repo>.contentservice | s3blobstore | bucketprefix | y | <user> |
| String | Name prefix of Buckets | 1.0.1 |
Example configuration for S3-Blobstore:
<repoName>.contentservice.type = s3blobstore <repoName>.contentservice.s3blobstore.connectionuser=kgsarchive <repoName>.contentservice.s3blobstore.connectionpass=kgsarchivePassword #<repoName>.contentservice.s3blobstore.maxconnections= #default: 50 <repoName>.contentservice.s3blobstore.endpoint=localhost:9000 <repoName>.contentservice.s3blobstore.protocol=http #<repoName>.contentservice.s3blobstore.connecttimeout= #default: 10000 #<repoName>.contentservice.s3blobstore.requestsigner= #default: #<repoName>.contentservice.s3blobstore.region= #default: #<repoName>.contentservice.s3blobstore.EnforceUTF8ForContentDisposition= #default: false #<repoName>.contentservice.s3blobstore.AllowCreateBuckets= #default: true #<repoName>.contentservice.s3blobstore.MaxIdle= #default: 60000 #<repoName>.contentservice.s3blobstore.ValidateAfterInactivity= #default: 5000 #<repoName>.contentservice.s3blobstore.CleanVersions= #default: true #<repoName>.contentservice.s3blobstore.ObjectLockEnabled= #default: #<repoName>.contentservice.s3blobstore.clientoptions= #default: <repoName>.contentservice.s3blobstore.bucketname=mass #<repoName>.contentservice.s3blobstore.bucketgroups= #default: 1 #<repoName>.contentservice.s3blobstore.bucketspergroup= #default: 5 <repoName>.contentservice.s3blobstore.bucketnameformat=%.2s #<repoName>.contentservice.s3blobstore.bucketprefix= #default: #<repoName>.contentservice.s3blobstore.contrepinpath = #default: false
Example configuration for S3-Managedbucketsblobstore:
<repoName>.contentservice.type = s3managedbucketsblobstore <repoName>.contentservice.s3blobstore.connectionuser=kgsarchive <repoName>.contentservice.s3blobstore.connectionpass=kgsarchivePassword #<repoName>.contentservice.s3blobstore.maxconnections= #default: 50 <repoName>.contentservice.s3blobstore.endpoint=localhost:9000 <repoName>.contentservice.s3blobstore.protocol=http #<repoName>.contentservice.s3blobstore.connecttimeout= #default: 10000 #<repoName>.contentservice.s3blobstore.requesttimeout= #default: 0 #<repoName>.contentservice.s3blobstore.requestsigner= #default: #<repoName>.contentservice.s3blobstore.region= #default: #<repoName>.contentservice.s3blobstore.AllowCreateBuckets= #default: true <repoName>.contentservice.s3blobstore.bucketname=mass #<repoName>.contentservice.s3blobstore.clientoptions= #default: #<repoName>.contentservice.s3blobstore.contrepinpath = #default: false #<repoName>.contentservice.s3blobstore.onbucketcreate = #<repoName>.contentservice.s3blobstore.onbucketcreateworkingdir = #<repoName>.contentservice.s3blobstore.calculatestreamhash = #default: true #<repoName>.contentservice.s3blobstore.EnforceUTF8ForContentDisposition= #default: false #<repoName>.contentservice.s3blobstore.ObjectLockEnabled= #default: true #<repoName>.contentservice.s3blobstore.MaxIdle= #default: 60000 #<repoName>.contentservice.s3blobstore.validateafterinactivity = #default: 5000 #<repoName>.contentservice.s3blobstore.CleanVersions= #default: true #<repoName>.contentservice.s3blobstore.credentialsprovider = #default: basic #<repoName>.contentservice.s3blobstore.bucketgroups = #default: 1 #<repoName>.contentservice.s3blobstore.bucketspergroup = #default: 5 #<repoName>.contentservice.s3blobstore.bucketnameformat = %.2s #<repoName>.contentservice.s3blobstore.bucketprefix = #default: