Fixed
Status Update
No update yet.
Comments
ma...@google.com
I have filed a feature request with our product engineering team. However, I can't provide you with an ETA or guarantee the implementation of this feature, but rest assured that Google strives on improving its products and that your feedback helps us do just that.
Any future updates to this feature will be posted here.
Any future updates to this feature will be posted here.
ma...@google.com
nh...@google.com
The GCS service does have support for v4 signatures, and has for ~1 yr. There is one existing known issue in bucket.storage.googleapis.com/object ) isn't working, and we have a solution in progress.
The issue field with engineering was for support for using v4 signatures in gsutil, which is, as yet, unimplemented.
If there's a specific tool you had in mind using that isn't working, especially after
The issue field with engineering was for support for using v4 signatures in gsutil, which is, as yet, unimplemented.
If there's a specific tool you had in mind using that isn't working, especially after
ya...@gmail.com
AWS has stopped supporting Signature v2 for more regions and since gsutil has still not implemented v4, I would remove stating support of s3 in gsutil.
Transfer service is a good alternative for one way but for GCS-to-S3, one must use aws toolkit or an external app e.g. cyberduck, unfortunately both requiring a local copy of files hence a VM.
I wish Google had published a warning that gsutil only supports v2 of AWS Signature, it would have saved time and effort. Seehttps://stackoverflow.com/questions/47929964/gsutil-cannot-copy-to-s3-due-to-authentication
Since this was raised a few months ago, is there a firm plan to continue supporting s3 in gsutil ? If the Transfer service is the only alternative, let's be explicit and transparent about it.
Transfer service is a good alternative for one way but for GCS-to-S3, one must use aws toolkit or an external app e.g. cyberduck, unfortunately both requiring a local copy of files hence a VM.
I wish Google had published a warning that gsutil only supports v2 of AWS Signature, it would have saved time and effort. See
Since this was raised a few months ago, is there a firm plan to continue supporting s3 in gsutil ? If the Transfer service is the only alternative, let's be explicit and transparent about it.
ho...@google.com
Support for accessing S3 with Signature V4 was added to gsutil in version 4.28:
https://github.com/GoogleCloudPlatform/gsutil/commit/d2ce76c8e699285d659f43013a4bcd7a4caedb77
It can be specified in the [s3] section of your boto configuration file, e.g.:
[s3]
# Note that we specify region as part of the host, as mentioned in the AWS docs:
#http://docs.aws.amazon.com/general/latest/gr/rande.html#s3_region
host =s3.eu-central-1.amazonaws.com
use-sigv4 = True
It can be specified in the [s3] section of your boto configuration file, e.g.:
[s3]
# Note that we specify region as part of the host, as mentioned in the AWS docs:
#
host =
use-sigv4 = True
ya...@gmail.com
Glad to read core functionality is covered. Please update documentation accordingly and make "gsutil config" create above [s3] section in .boto as this is likely to become the default signature.
Ni...@rath.org
As far as I can see, having to hardcopy the host in the .boto file makes it impossible to copy between buckets in different regions.
`gsutil rsync s3://foobucket s3://barbucket` always gives a 'PermanentRedirect' error for one of the buckets.
`gsutil rsync s3://foobucket s3://barbucket` always gives a 'PermanentRedirect' error for one of the buckets.
pt...@google.com
ap...@google.com
🤖 This is an automatic update. The status of the related internal issue(s) listed in "Blocked By" has changed, so we've done the following:
* Moved this issue to a locked sub-component, because the issue has been closed. Please open a new issue if you need Google to re-visit this topic.
You can read more about what these statuses mean athttps://developers.google.com/issue-tracker/concepts/issues#fields
* Moved this issue to a locked sub-component, because the issue has been closed. Please open a new issue if you need Google to re-visit this topic.
You can read more about what these statuses mean at
Description
The authentication API that Google supports however is the older AWS Signature Version 2 (
Newer tools for interacting with S3 are likely to support only Version 4, meaning they will no longer work with GCS. Additionally, newer S3 regions support only the newer signature method, meaning interoperability and gsutil don't work on them (for example, AWS's us-east-2 region).
Extending interoperability support to accept the newer signature would make transitions from S3 to GCS more consistent in the future, and allow GCS to continue to be used with newer file transfer programs.