How to Migrate Amazon S3 Data Across Accounts and Regions Using the AWS CLI

Migrating data between Amazon S3 buckets across AWS accounts and Regions can be a common requirement during cloud restructuring, account isolation, or data archiving strategies.

We’ll copy data from a source bucket in one AWS account to a destination bucket in another account, potentially in a different Region. This process ensures that the destination account becomes the new owner of the objects, and we’ll achieve this through AWS S3 data transfer using AWS CLI.

Tools & Services Used

• Amazon S3 – Object storage service
• AWS CLI – Command line tool for AWS
• AWS IAM – Identity and Access Management

Prerequisites

• Two AWS accounts (source and destination)
• Existing S3 buckets in both accounts
• IAM access to configure users, roles, and policies
• AWS CLI installed on your system

Step-by-Step Guide

Create IAM User and Role in the Destination Account

Create an IAM User

• Go to IAM in the AWS Management Console
• Create a IAM user with programmatic access
• Download the access and secret keys

Create a Policy (S3MigrationPolicy)

Create a policy with the following permissions, replacing bucket names with your actual bucket names:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "s3:ListBucket",
                "s3:GetObject",
                "s3:GetObjectTagging",
                "s3:GetObjectVersion",
                "s3:GetObjectVersionTagging"
            ],
            "Resource": [
                "arn:aws:s3:::amazon-s3-source-bucket",
                "arn:aws:s3:::amazon-s3-source-bucket/*"
            ]
        },
        {
            "Effect": "Allow",
            "Action": [
                "s3:ListBucket",
                "s3:PutObject",
                "s3:PutObjectAcl",
                "s3:PutObjectTagging",
                "s3:GetObjectTagging",
                "s3:GetObjectVersion",
                "s3:GetObjectVersionTagging"
            ],
            "Resource": [
                "arn:aws:s3:::amazon-s3-destination-bucket",
                "arn:aws:s3:::amazon-s3-destination-bucket/*"
            ]
        }
    ]
}

Create IAM Role (S3MigrationRole)

Create a role that can be assumed by the destination account IAM user:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Principal": {

              "AWS": "arn:aws:iam::<destination_account>:user/<user_name>"
            },
            "Action": "sts:AssumeRole",
            "Condition": {}
        }
    ]
}

Change the Amazon Resource Name (ARN) of the destination IAM user name according to your use case.

Attach the S3MigrationPolicy to this role S3MigrationRole.

Configure Bucket Policy in the Source Account

In the source S3 bucket, attach a bucket policy to allow access to the IAM role in the destination account:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "DelegateS3Access",
      "Effect": "Allow",
      "Principal": {
        "AWS": "arn:aws:iam::<destination-account-id>:role/S3MigrationRole"
      },
      "Action": [
        "s3:ListBucket",
        "s3:GetObject",
        "s3:GetObjectTagging",
        "s3:GetObjectVersion",
        "s3:GetObjectVersionTagging"
      ],
      "Resource": [
        "arn:aws:s3:::source-bucket-name/*",
        "arn:aws:s3:::source-bucket-name"
      ]
    }

Replace <destination-account-id> with your actual AWS account ID and S3MigrationRole with the name of the IAM role you created in your destination account.

Create the Destination S3 Bucket

If you haven’t already:

• Go to S3 console in the destination account
• Create a bucket in the desired Region

Configure AWS CLI & Assume Role

Install the AWS CLI and configure it with the IAM user credentials. See the following link to configure AWS CLI & Assume Role:- https://www.skynats.com/blog/how-do-i-assume-an-iam-role-using-the-aws-cli/

Copy or Sync the Data

Use one of the following commands to migrate data:

Copy all objects:

aws s3 cp s3://amazon-s3-source-bucket/ \
    s3://amazon-s3-destination-bucket/ \
    --recursive --source-region source-region-name --region destination-region-name

Synchronize objects:

aws s3 sync s3://amazon-s3-source-bucket/ \
    s3://amazon-s3-destination-bucket/ \
    --source-region source-region-name --region destination-region-name

Replace the bucket and region names according to your setup in the above command.

Conclusion

This guide showed you how to perform a secure, one-time migration of S3 data across AWS accounts and Regions using the AWS CLI.

Handling AWS S3 data transfer using AWS CLI across accounts and regions can be complex. Skynats offers expert AWS Management Services to simplify migrations, ensure data security, and optimize performance. Contact us today for professional support.