S3

Generally an S3 enabled storage system is supported as content service.

Following S3 functions will be used: Required S3 functions

List of officially support S3 Systems: Supported S3 storage products

Using S3 managed buckets: Using S3 managed content service

Integration of Credential provider: Credentials provider

In case of using retention or legal holds, the user configured to access the buckets must also be the owner of the buckets, otherwise the check if object lock is enabled will fail.

From the S3 documentation:
x-amz-expected-bucket-owner

The account ID of the expected bucket owner. If the bucket is owned by a different account, the request fails with the HTTP status code 403 Forbidden (access denied). (GetObjectLockConfiguration - Amazon Simple Storage Service )

Description of configuration parameters in repository.cfg

There are two types for S3 content service: s3blobstore and s3managedbucketblobstore. First the common parameters for both of them are listed before the specific parameters for the managed S3 service.

Content-Service

Service	Präfix	Servicetyp	Parameter	Optional	Unterstützte Werte	Default	Typ	Funktion	Verfübar ab

Service	Präfix	Servicetyp	Parameter	Optional	Unterstützte Werte	Default	Typ	Funktion	Verfübar ab
Content-Service	<repo>.contentservice		type	n	s3blobstore, s3managedbucketblobstore	noop	String	type definition of the content service	1.0.1
		s3blobstore, s3managedbucketsblobstore, s3netappblobstore, s3netappmanagedbucketsblobstore							1.0.1, netapp from 3.2.0 on
Content-Service	<repo>.contentservice	s3blobstore	connectionuser	n, wenn credentialsprovider auf “basic“ gestellt wird	<user>		String	User for S3 connection	1.0.1
Content-Service	<repo>.contentservice	s3blobstore	connectionpass	n, when credentialsprovider is“basic“	<user>		String	Connection-password for S3 (may be an Alias for a password in keystore)	1.0.1
Content-Service	<repo>.contentservice	s3blobstore	maxconnections	y	<user>	50 (AWS default)	Num	Max Connection-Pool of client	1.0.1
Content-Service	<repo>.contentservice	s3blobstore	endpoint	y	<user>		URI witout protocol, e.g.: 127.0.0.1:9000	Endpoint for S3 access (without Schema). The Endpoint is usally defined using parameter “region”. If Enpoint is defined, the parameter “region” will be ignored.	1.0.1
Content-Service	<repo>.contentservice	s3blobstore	protocol	y	http,https	https	String	Client connection protocol	1.0.1
Content-Service	<repo>.contentservice	s3blobstore	connecttimeout	y	<user>	10000	Num	Client connection-timeout (ms)	1.0.1
Content-Service	<repo>.contentservice	s3blobstore	requesttimeout	y	<user>	0	Num	Client request timeout (ms) 0=Disabled	1.0.1
Content-Service	<repo>.contentservice	s3blobstore	requestsigner	y	NoOpSignerType, QueryStringSignerType, AWS4SignerType, AWS4UnsignedPayloadSignerType, AWS3SignerType		String	Signer for signing of requests to AWS	1.0.1 Signer “AWS4UnsignedPayloadSignerType“ may lead to problems when using certain special characters
Content-Service	<repo>.contentservice	s3blobstore	region	y	<user>		String	S3 Region	1.0.1
Content-Service	<repo>.contentservice	s3blobstore	allowcreatebuckets	y	true, false	true	Boolean	Enable creation of Buckets	1.0.1
Content-Service	<repo>.contentservice	s3blobstore	bucketname	n	<user>		String	S3 Bucket for content files	1.0.1
Content-Service	<repo>.contentservice	s3blobstore	clientoptions	y	pathstyleaccess:true		String	S3 ClientOption	1.0.1
Content-Service	<repo>.contentservice	s3blobstore	contrepinpath	y	true, false	false	Bool	Use Repository Name as Root Folder	1.0.1
Content-Service	<repo>.contentservice	s3blobstore	onbucketcreate	y	<user>		String	Script to execute when Bucket was created	1.0.1
Content-Service	<repo>.contentservice	s3blobstore	onbucketcreateworkingdir	y	<user>		Pfad	Needs to be defined when onbucketcreate is enabled	1.0.1
Content-Service	<repo>.contentservice	s3blobstore	calculatestreamhash	y	true, false	true	Boolean	After encryption the hash values in ContentServices are calculated again	1.0.2
Content-Service	<repo>.contentservice	s3blobstore	EnforceUTF8ForContentDisposition	y	true, false	false	Boolean	false: should be set, if a ContentDisposition filename needs to be encoded, otherwise the original format is delivered, true: always encode in UTF-8	1.4.1
Content-Service	<repo>.contentservice	s3blobstore	objectlockenabled	y	true, false	true	Boolean	Setting objectLockEnabled when creating a new Bucket	1.9.0
Content-Service	<repo>.contentservice	s3blobstore	maxidle	y	<user>	60000	Num	Maximum time in ms after removing an unused connection from ConnectionPool	1.9.0
Content-Service	<repo>.contentservice	s3blobstore	validateafterinactivity	y	<user>	5000	Num	Polling time in ms for checking if connection in pool is still open.	1.9.0
Content-Service	<repo>.contentservice	s3blobstore	cleanVersions	y	true,false	true	boolean	true: Delete all previous versions for update and delete operations when using buckets with enabled versioning. false: keep all versions	1.9.0
Content-Service	<repo>.contentservice	s3blobstore	credentialsprovider	y	basic, instanceprofile	basic	String	basic: Authentication whith username and password instanceprofile: can be used when both, tia Core and S3 bucket S3 Bucket are deployed in AWS - then no direct authentification is necessary, as this is handled by the internal permission group.	2.0.3
		s3managedbucketsblobstore, s3netappmanagedbucketsblobstore							1.0.1, netapp from 3.2.0 on
Content-Service	<repo>.contentservice	s3blobstore	region	y	<user>		String	S3 Region	1.0.1
Content-Service	<repo>.contentservice	s3blobstore	AllowCreateBuckets	y	true, false	true	Boolean	Enable creation of Buckets	1.0.1
Content-Service	<repo>.contentservice	s3blobstore	bucketgroups	y	<user>	1	Num	Number of bucket groups to use for new storage files	1.0.1
Content-Service	<repo>.contentservice	s3blobstore	bucketspergroup	y	<user>	5	Num	Number of buckets in a group (Attention: don’t change after first use!!)	1.0.1
Content-Service	<repo>.contentservice	s3blobstore	bucketnameformat	y	<user>, z.B. %8.8s		String	Number of characters of generated bucket names. When e.g. set to 8 characters, the name is filled with leading zeros to be 8 characters long.	1.0.1
Content-Service	<repo>.contentservice	s3blobstore	bucketprefix	y	<user>		String	Name prefix of Buckets	1.0.1

Example configuration for S3-Blobstore:

<repoName>.contentservice.type = s3blobstore
<repoName>.contentservice.s3blobstore.connectionuser=kgsarchive
<repoName>.contentservice.s3blobstore.connectionpass=kgsarchivePassword
#<repoName>.contentservice.s3blobstore.maxconnections= #default: 50 
<repoName>.contentservice.s3blobstore.endpoint=localhost:9000
<repoName>.contentservice.s3blobstore.protocol=http
#<repoName>.contentservice.s3blobstore.connecttimeout= #default: 10000
#<repoName>.contentservice.s3blobstore.requestsigner= #default:
#<repoName>.contentservice.s3blobstore.region= #default:
#<repoName>.contentservice.s3blobstore.EnforceUTF8ForContentDisposition= #default: false
#<repoName>.contentservice.s3blobstore.AllowCreateBuckets= #default: true
#<repoName>.contentservice.s3blobstore.MaxIdle= #default: 60000
#<repoName>.contentservice.s3blobstore.ValidateAfterInactivity= #default: 5000
#<repoName>.contentservice.s3blobstore.CleanVersions= #default: true
#<repoName>.contentservice.s3blobstore.ObjectLockEnabled= #default:
#<repoName>.contentservice.s3blobstore.clientoptions= #default:
<repoName>.contentservice.s3blobstore.bucketname=mass
#<repoName>.contentservice.s3blobstore.bucketgroups= #default: 1
#<repoName>.contentservice.s3blobstore.bucketspergroup= #default: 5
<repoName>.contentservice.s3blobstore.bucketnameformat=%.2s
#<repoName>.contentservice.s3blobstore.bucketprefix= #default: 
#<repoName>.contentservice.s3blobstore.contrepinpath = #default: false

Example configuration for S3-Managedbucketsblobstore:

<repoName>.contentservice.type = s3managedbucketsblobstore
<repoName>.contentservice.s3blobstore.connectionuser=kgsarchive
<repoName>.contentservice.s3blobstore.connectionpass=kgsarchivePassword
#<repoName>.contentservice.s3blobstore.maxconnections= #default: 50 
<repoName>.contentservice.s3blobstore.endpoint=localhost:9000
<repoName>.contentservice.s3blobstore.protocol=http
#<repoName>.contentservice.s3blobstore.connecttimeout= #default: 10000
#<repoName>.contentservice.s3blobstore.requesttimeout= #default: 0
#<repoName>.contentservice.s3blobstore.requestsigner= #default:
#<repoName>.contentservice.s3blobstore.region= #default:
#<repoName>.contentservice.s3blobstore.AllowCreateBuckets= #default: true
<repoName>.contentservice.s3blobstore.bucketname=mass
#<repoName>.contentservice.s3blobstore.clientoptions= #default:
#<repoName>.contentservice.s3blobstore.contrepinpath = #default: false
#<repoName>.contentservice.s3blobstore.onbucketcreate = 
#<repoName>.contentservice.s3blobstore.onbucketcreateworkingdir =
#<repoName>.contentservice.s3blobstore.calculatestreamhash = #default: true
#<repoName>.contentservice.s3blobstore.EnforceUTF8ForContentDisposition= #default: false
#<repoName>.contentservice.s3blobstore.ObjectLockEnabled= #default: true
#<repoName>.contentservice.s3blobstore.MaxIdle= #default: 60000
#<repoName>.contentservice.s3blobstore.validateafterinactivity = #default: 5000
#<repoName>.contentservice.s3blobstore.CleanVersions= #default: true
#<repoName>.contentservice.s3blobstore.credentialsprovider = #default: basic
#<repoName>.contentservice.s3blobstore.bucketgroups = #default: 1
#<repoName>.contentservice.s3blobstore.bucketspergroup = #default: 5
#<repoName>.contentservice.s3blobstore.bucketnameformat = %.2s
#<repoName>.contentservice.s3blobstore.bucketprefix = #default: