Configure a Ceph target

You can migrate data to a Ceph Storage bucket by configuring one as a target filesystem.

Follow these steps to create a Ceph target:

UI

Configure a Ceph target filesystem in the UI

1. From the Dashboard, select an instance under Instances.
2. In the Filesystems & Agents menu, select Filesystems.
3. Select Add target filesystem.
4. Enter the following details:
   • Filesystem Type - The type of filesystem target. Select Ceph Storage.
   • Display Name - Enter a name for your target filesystem.
   • Hostname - The endpoint for your Ceph filesystem. For example, http://cephhost:7480.
   • Bucket Name - The name of the Ceph bucket you're using. For example mybucket-07.
   • Authentication Method - Select the appropriate method from the list. The Authentication Method options available include:
     • Access Key and Secret
       • Access Key - Enter the access key. For example, mykey.
       • Secret Key - Enter the secret key. For example, mypassword.
     • STS assumed role with web identity
       • Role ARN - Enter the Role ARN. For example, arn:aws:iam::123456789012:role/MyExampleRole.
       • Token File Location - Enter the token file location. For example, /path/security/keycloak/token.txt.
5. Within the Filesystem Properties section, enter override properties or additional properties for your Ceph filesystem by adding key/value pairs. The following filesystem options are also available:
   • Chunked Encoding - Enabled by default and should be disabled for Ceph Reef version 18.2.4 and below, otherwise the creation of empty directories or zero-byte files will fail
   • Path Style Access - Private stores generally expect path style access to be enabled so that buckets are visible, therefore this is enabled by default
6. Select Save. You can now use your Ceph Storage target in data migrations.

CLI

Configure a Ceph target filesystem in the CLI

To create a Ceph target in the Data Migrator CLI, run the filesystem add ceph command:

Add a Ceph filesystem

filesystem add ceph [--access-key] string 
                    [--bucket-name] string
                    [--credentials-provider] string
                    [--endpoint] string
                    [--file-system-id] string
                    [--properties] string
                    [--properties-files] list
                    [--s3-path-style-access] ENABLE|DISABLE
                    [--s3a-chunked-encoding] ENABLE|DISABLE
                    [--secret-key] string
                    [--sts-access-token-path] string
                    [--sts-role-arn] string

For guidance about access, permissions, and security when adding an Amazon S3 bucket as a target filesystem, see Security best practices in IAM.

Ceph mandatory parameters

• --access-key The Ceph access key. For example, mykey.
• --bucket-name The name of your Ceph bucket. In the UI, this is called Bucket Name.
• --endpoint The endpoint for your Ceph bucket. For example, http://cephtarget:7480. In the UI, this is called Hostname.
• --file-system-id The ID for the new filesystem resource. In the UI, this is called Display Name.
• --secret-key The secret key to use with your access key. For example, mypassword.

Ceph optional parameters

• --credentials-provider The Java class name of a credentials provider for authenticating with the Amazon S3 endpoint.
  The Provider options available include:
  • org.apache.hadoop.fs.s3a.SimpleAWSCredentialsProvider Use this provider to offer credentials as an access key and secret access key with the --access-key and --secret-key Parameters. This is the default option if no credentials provider is specified.
  • StsWithEndpointAssumeRoleWithWebIdentityCredentialProvider Use this option to offer credentials as a Role ARN and access token path with the --sts-role-arn and --sts-access-token-path Parameters.
• --properties Enter properties to use in a comma-separated key/value list. In the UI, this is called Filesystem Properties.
• --properties-files Reference a list of existing properties files, each containing configuration properties in the format used by core-site.xml. This is not available in the UI as we believe this would not be needed for Ceph.
• --s3-path-style-access Private stores generally expect path style access to be enabled so that buckets are visible. Use either 'ENABLE' or 'DISABLE'. The default if this parameter is not specified is for this to be enabled.
• --s3a-chunked-encoding Set this parameter to be disabled for Ceph Reef version 18.2.4 and below, otherwise the creation of empty directories or zero-byte files will fail. Use either 'ENABLE' or 'DISABLE'. The default if this parameter is not specified is for this to be enabled.
• --sts-access-token-path The path to a file containing the access token if the authentication method specified in --credentials-provider is STS assumed role with web identity.
• --sts-role-arn The Role ARN if the authentication method specified in --credentials-provider is STS assumed role with web identity.

Example

filesystem add ceph --file-system-id ceph-tgt
                    --bucket-name mybucket-07
                    --access-key mykey
                    --secret-key mypassword
                    --endpoint http://cephtarget:7480

S3a properties

Enter additional properties for Ceph filesystems by adding them as key-value pairs in the UI or as a comma-separated key-value pair list with the --properties parameter in the CLI. You can overwrite default property values or add new properties.

Additional properties

Find an additional list of S3a properties in the S3a documentation.

Next steps

If you haven't already, configure a source filesystem from which to migrate data. Then, you can create a migration to migrate data to your new Ceph target.