Add An Amazon S3 Storage

In order to integrate Amazon S3 with MetaDefender for Secure Storage please follow these steps:

  1. From the left side menu, navigate to Storage units, and from the Object Storage tab, choose Add Amazon S3 Storage
  1. Give your storage a name so you can easily identify it later
  2. There are two ways to connect:
    • via the IAM role from the EC2 instance: please skip to step 6
    • using security credentials
  1. Enter your Access Key ID. Scroll to the end of this page for more details.
  2. Enter your Secret Access Key. Scroll to the end of this page for more details.
  3. Choose the R egion endpoint where your S3 endpoint is located.
  4. Enter the name of the bucket you wish to process with MetaDefender for Secure Storage.
  5. If you wish to only process a particular folder enter the name or path of the folder in the Folder location field. Leave this field empty if you wish to process the entire bucket.
  6. Select Add in order to finish the process.

Where can I find the Access Key ID and Secret Access Key?

To generate an Access Key ID / Secret Access Key pair please follow these steps:

  1. Log in to IAM Console and go to Users
  2. In the Details pane, click the appropriate IAM user or create one if none exists
  3. From the Security Credentials, tab click Create Access Key

If you already have a maximum of two access keys—active or inactive—you must delete one first before proceeding. If you need more than two root access keys, IAM users (each of whom can be assigned their own access keys) would probably better suit your requirements.

Using IAM role for Amazon S3 integration

On S3 integration with IAM, the role configured on the EC2 machine will be used to authenticate.

With the optional field of Assume Role ARN, MetaDefender for Secure Storage allows integration with other buckets that are in different accounts from the current AWS account. This is possible by configuring Assume Role ARN field, with the ARN of the role that has access to the other account's S3 bucket.

If you are deploying MetaDefender for Secure Storage on an EC2 instance you will need to run the following AWS CLI command in order to increase the default hop-limit

aws ec2 modify-instance-metadata-options --instance-id <INSTANCE_ID> --http-put-response-hop-limit 3 --http-endpoint enabled

This is required because our services run within Docker and IMDSv2 restricts the number of "hops" each request to the metadata service can make.

Necessary AWS policies

These are the necessary AWS policies required in order to add an Amazon S3 Storage:

  • s3:GetObject
  • s3:PutObject
  • s3:GetObjectVersion
  • s3:ListBucket
  • s3:ListBucketVersions
  • s3:GetObjectTagging
  • s3:GetObjectVersionTagging
  • s3:PutObjectTagging
  • s3:PutObjectVersionTagging
  • s3:DeleteObject
  • s3:DeleteObjectVersion
  • s3:PutLifeCycleConfiguration

S3 Compatible integrations

In order to use an S3 compatible storage unit the following requirements should be met:

  • A valid TLS endpoint. since we are enforcing HTTPS for all S3 integrations. A valid certificate is also required. Please note that there is an option to add a custom certificate
  • The storage should be configured to use or support the virtual-hosted addressing model, not path based.

The following object storage providers have been validated but any provider that meets the requirements and implements the S3 compatible open standard will work with MDSS:

  • MinIO Object Storage
  • Ceph Object Gateway
Type to search, ESC to discard
Type to search, ESC to discard
Type to search, ESC to discard
Next to read:
Add A Cloudian Storage
On This Page
Add An Amazon S3 StorageWhere can I find the Access Key ID and Secret Access Key?Using IAM role for Amazon S3 integrationNecessary AWS policiesS3 Compatible integrations