Synchronizing Objects to a Bucket

You can synchronize entire directories from your local machine or a bucket to another bucket using the sync command, ensuring that only new or modified objects are transferred.

sync synchronizes objects by first comparing their sizes; if sizes match, it then checks the last modified timestamps. Any discrepancy triggers synchronization.

You can also remove objects that have been deleted locally.

This is useful for optimizing storage while avoiding unnecessary manual action, and for managing your bucket websites with efficient content updates.

sync ignores empty directories. This means empty folders in the source directory will not be copied to the destination. This feature cannot be disabled.

Synchronizing an Object to a Bucket Using AWS CLI (s3)

Before you begin: Install and configure AWS CLI. For more information, see Installing and Configuring AWS CLI.

To synchronize an object to one of your buckets, use the sync command following this syntax:

Request sample

$ aws s3 sync PATH/TO/DIRECTORY_TO_SYNC or s3://BUCKET_TO_SYNC s3://YOUR_BUCKET/ \
    --profile YOUR_PROFILE \
    --dryrun \
    --include \
    --exclude \
    --follow-symlinks \
    --acl \
    --no-guess-mime-type \
    --website-redirect \
    --cache-control \
    --content-disposition \
    --content-encoding \
    --content-language \
    --content-type \
    --expires \
    --only-show-errors \
    --no-progress \
    --page-size \
    --request-payer \
    --metadata \
    --metadata-directive \
    --copy-props \
    --size-only \
    --exact-timestamps \
    --delete \
    --debug \
    --no-sign-request \
    --no-verify-ssl \
    --no-paginate \
    --endpoint https://oos.eu-west-2.outscale.com

This command contains the following attributes that you need to specify:

PATH/TO/DIRECTORY_TO_SYNC or s3://BUCKET_TO_SYNC/: The path to the directory on your local machine or the content of the bucket (if you have the appropriate permissions) you want to synchronize.
s3://YOUR_BUCKET: The name of the bucket where you want the directory to be synchronized.
(optional) profile: The named profile you want to use, created when configuring AWS CLI. For more information, see Installing and Configuring AWS CLI.
(optional) dryrun: Provides an overview of the operations performed by the command without actually running it. This is used as a safety measure to review the impact of a command before executing it.
(optional) include: Specifies patterns to include files in the sync operation, overriding any global exclude patterns, and ensuring specific files are always synced.
(optional) exclude: Specifies patterns to exclude files from the sync operation, helping to filter out unwanted files or directories.
(optional) follow-symkinks: Follows symbolic links in the source directory, uploading the target content to the bucket under the symlink’s name, as S3 does not support symlinks.
(optional) acl: The permissions you grant for your bucket (private | public-read | public-read-write | authenticated-read | bucket-owner-read | bucket-owner-full-control).
(optional) no-guess-mime-type: Prevents from automatically setting a MIME type based on file extension. This is useful if you need to ensure a specific content-type is used.
(optional) website-redirect: If you configured the bucket as a website, redirects requests for this object to another object in the same bucket or to an external URL.
(optional) cache-control: How you want the copy of the object to be handled in terms of caching (max-age | max-stale | min-fresh | no-cache | no-store | no-transform | only-if-cached | stale-if-error).
(optional) content-disposition: How you want the object to be displayed when accessed via a browser ("inline" | "attachment" | "attachment; filename="<NAME_OF_DOWNLOADED_FILE>"").
- inline: When possible, prompts the browser to display the content within the browser window itself. This is useful for images, PDFs, and other media types that browsers can easily render.
- attachment: Regardless of file type, prompts the browser to download the content instead of displaying it inline. The file is thus saved locally.
- "attachment; filename="<NAME_OF_DOWNLOADED_FILE>"": Regardless of file type, prompts the browser to download the content instead of displaying it inline. The file is thus saved locally with the specified filename.
(optional) content-encoding: The encoding format of the object (gzip | compress | deflate | identity | br).
(optional) content-language: The language the content of the object is in, in language code (ISO 639 format).

(optional) content-type: The MIME (Multipurpose Internet Mail Extensions) type of the object.

An inaccurately set or absent content-type attribute can cause objects to be misinterpreted or mishandled by browsers. As a result, you may encounter difficulties accessing or viewing your objects using your preferred browser.

(optional) expires: The date and time, in UTC format, at which you consider the object can no longer be cached, and is considered stale.
(optional) only-show-errors: Limits output to errors only, making it easier to spot problems in scripts or automated environments.
(optional) no-progress: Suppresses the progress display during the sync operation. This is useful for scripts or logs where you do not need real-time progress updates.
(optional) page-size: Determines the number of requests made in a call to retrieve all objects. A smaller page size increases the number of calls but helps prevent timeouts by retrieving fewer items in each request. For example, if you have a bucket of 120 objects and you set the page size to 10, the CLI will need to perform 12 requests to retrieve all objects. If you set the page size to 60, only 2 requests will need to be performed.
The default value is 1000.

The minimum and maximum values are 1 and 1000.
(optional) request-payer: Configures the operation to be performed as a requester-pays, meaning the requester rather than the bucket owner pays the cost of the data transfer.
(optional) metadata: A map of additional metadata entries you can freely specify.
- key: The name of the metadata.
- value: The value of the metadata.
(optional) metadata-directive: Whether you want the metadata to be copied from the source objects or replaced with the metadata provided in the request (COPY or REPLACE).

(optional) copy-props: Determines the properties you want copied from the source object (none | metadata-directive | default).

none refrains from copying any of the properties from the source object.
metadata-directive copies the following values from the source object: content-type, content-language, content-encoding, content-disposition, cache-control, expires, and metadata.
default-value copies tags and metadata specified by the metadata-directive value from the source object.

(optional) size-only: Directs the command to only use file size, not last-modified time, to determine if a file should be copied. This can reduce the number of HTTP HEAD requests during sync.
(optional) exact-timestamps: Uses precise timestamps to compare files, ensuring that only files with newer timestamps are copied, providing more granular control over what is considered a 'newer' file.
(optional) delete: Removes files from the destination bucket that do not have counterparts in the source directory, ensuring that the destination exactly mirrors the source.
(optional) debug: When included, returns the detailed log of the operation. This is useful to analyze and troubleshoot issues you may be encountering.
(optional) no-sign-request: Performs the operation without signing the request. This is useful for accessing publicly accessible resources without using your credentials.
(optional) no-paginate: Disables pagination of the output. This is useful for commands that produce a large volume of data.
endpoint: The endpoint corresponding to the Region you want to send the request to. For more information, see Installing and Configuring AWS CLI.

Related page

Installing and Configuring AWS CLI

AWS™ and Amazon Web Services™ are trademarks of Amazon Technologies, Inc or its affiliates in the United States and/or other countries.