Add A Google Cloud Storage
In order to integrate Google Cloud with MetaDefender Storage Security please follow these steps:
- From the left side menu, navigate to Storage units, and from the Object Storage tab, choose Add Google Cloud Storage
Give your storage a name so you can easily identify it later
There are two ways to connect:
- via the ADC from the ECS instance (If you choose this option, skip the following step and continue from step 5)
- using security credentials
Upload your Credentials file. Scroll down for more details.
Enter the name of the bucket you wish to process with MetaDefender Storage Security.
If you wish to only process a particular folder enter the name or path of the folder in the Folder location field. Leave this field empty if you wish to process the entire bucket.
Select Add in order to finish the process.
Where can I generate the Credentials file?
To generate a credentials file, please follow these steps, explained in the official documentation of Google Cloud.
https://developers.google.com/workspace/guides/create-credentials#service-account
Using ADC for Google Cloud integration
On Google Cloud integration with ADC, the default credentials configured on the ECS instance will be used to authenticate.
Please use the following links to attach application default credentials to the ESC instance:
https://cloud.google.com/docs/authentication/application-default-credentials
https://cloud.google.com/iam/docs/impersonating-service-accounts#attaching-to-resources
If the google credentials are not set in the GOOGLE_APPLICATION_CREDENTIALS variable, then the service account assigned to the VM instance requires the iam.serviceAccounts.signBlob permission (included in the Service Account Token Creator role)
Google Cloud: Handling Rate Limiting
For transient errors on Google Cloud requests a retry with exponential backoff can be configured to adjust the required needs. The HTTP error codes covered are the TooManyRequests, RequestTimeoutand ServiceUnavailable.
The retry policy can be customized using the following environment variables:
| Parameters | Value |
|---|---|
| GCP_MAX_RETRY_ATTEMPTS | 10 |
| GCP_MAX RETRY_DELAY | 120 (seconds) |
The retry policy implements exponential backoff, starting with a 200-millisecond delay between retries and capping at a maximum delay of 120 seconds after 10 retries.