IssueTracker

I think if more people realized what this entails they would be starring this issue in ernest.  The cloudfront custom origin gives GAE developers a HUGE win.  If this is as simple as whitelisting a User-Agent I hope the team will consider getting this into near-future release.

Status: Accepted! Exciting.

I wound up just using CloudFront on my SVG anyway (it turns out most of my traffic isn't from North America, so edge servers could help), so when/if it's done, I can tell you at least one production app's users will benefit.

Some testing with Analytics has shown that cloudfront is still worth it for us as the average increase in Page Download Time is more than offset by the average reduction in Server Response Time.

So we are moving to cloudfront and eager to get this fixed.

This is marked Language: Python but I think it affects all languages as I have the same problem in the go runtime.

Yes, this is not language-specific.

We are fairly conservative about when we allow responses to go out with Content-Encoding: gzip. Most of the time, this is what you want, because this behavior makes sure that buggy browsers correctly interpret responses from your app. In other cases (such as this one), it results in inefficient (but otherwise correct) response transmission.

Is there any way you can tell CloudFront to use a different User-Agent? As described in our docs, if the User-Agent mentions gzip, you can get around User-Agent limitations. Other limitations still apply though e.g. if gzip is not listed in Accept-Encoding, then we won't allow gzipped responses.

> Is there any way you can tell CloudFront to use a different User-Agent?

No.

My take is that caches in general should get the gzipped content they claim to support, and then it's the cache's responsibility to decide if individual clients should get the gzipped response. Otherwise, it's not possible to implement an efficient cache that forwards responses in their original encoding.

In other words, if you allow gzip for my cache and some users get glitches, it's a bug report I should take to my cache's authors, not to you.

Practically speaking, my app is using a few times the bandwidth to end-users that it would use if this particular cache were whitelisted (or if a user-agent blacklist was used instead of a whitelist, or if apps could output X-GAE-Force-Encoding to turn on gzip, or whatever). I pay by the GB, though my app's too small for that to be a big problem for me.

If there's some kind of compatible Google edge hosting that will gzip responses, I'm interested. :)

How common are broken browser gzip implementations?  I did a bit of searching and mostly found references to IE6 (pre-service pack), Netscape 4, and similarly-ancient browser versions.  I'd be genuinely surprised to see any new browser version come out without functioning gzip support (meaning that it claims to support gzip correctly and does, or does not claim to support it at all).  I think it is reasonable to assume that the list of broken browsers is as long as it is going to get.

While a conservative approach is generally commendable, I think this is a case where it is hurting far more users than it is helping.  Blacklisting the handful of known offenders and treating this as a legacy issue may be a better approach for this particular problem than using a heuristic that is proving to be too restrictive.

Asking Amazon to add some indication that it supports gzip in its User-Agent string is just silly.  It already sends Accept-Encoding headers to indicate its support; this is what the standard calls for.  Do we really need to add assertions of gzip support to the list of junk User-Agent strings have to include forever (Mozilla compatible anyone?)

So, it's been over a year.  Any progress on this?

any news regarding this issue? i'm having the same problem and it was great to have a real solution

I have forwarded this request to the engineering team. We will update this issue with any progress updates and a resolution.

Best regards,

What'd you hear back from the engineering team?

Hey all, just a quick update.

At this moment, we are actively discussing and investigating this change. The concern is that this would change established behavior for a lot of possibly un-maintained programs.

Indeed this would change the behavior of unmaintained apps. However, you could find other possible solutions, such as :
 - allow us to do a content-encoding whitelist of user agents in the app.yaml file
 - allow us to disable the user-agent gzip compression dependency in the app.yaml file : the gzip compression should not depend on the user agent. It should depend only on the accept-encoding header, which is the exact purpose of that header, and I'd like at least an option to make it that way.

I'm looking forward to hear your thoughts

I am happy to say that this is fixed across the board.

If you run into any circumstances where Cloudfront is still not being served gzipped responses, please open a new defect report with specifics so that we may investigate.

Thank you!

And it seems that there are still cases where Cloudfront does not get a gzipped response. Reopening and de-duplicating some reports into this one.

Any update in this?