Provide hosted connection pooling [71581796]

Assigned

Feature Request

Status Update

No update yet.

Description

dn...@google.com

created issue #1

Jan 4, 2018 05:56PM

Connection pooling is the recommended solution for controlling the number of concurrent connections to a cloud sql instance. It is mentioned in the FAQ, many bugs and stack overflow issues.

As suggested at

https://issuetracker.google.com/issues/37271935#comment28 it might be possible for Cloud SQL to provide this connection pooling for customers and simplify deploying it.

Comments

[Deleted User] <[Deleted User]> #2Jan 4, 2018 05:59PM

+1000

kn...@google.com <kn...@google.com> #3Jan 4, 2018 06:40PM

Assigned to gc...@google.com.

I already notified our engineering team about this feature. Future update(s) will be posted here.

[Deleted User] <[Deleted User]> #4Jan 8, 2018 01:43PM

ji...@enjine.com <ji...@enjine.com> #5Jan 9, 2018 03:27PM

ps...@gmail.com <ps...@gmail.com> #6Jan 9, 2018 09:26PM

gr...@gmail.com <gr...@gmail.com> #7Jan 12, 2018 02:36AM

ma...@gmail.com <ma...@gmail.com> #8Mar 2, 2018 04:38PM

gl...@gmail.com <gl...@gmail.com> #9Mar 2, 2018 04:40PM

pe...@gmail.com <pe...@gmail.com> #10Apr 3, 2018 01:20AM

pa...@gmail.com <pa...@gmail.com> #11Aug 28, 2018 03:01PM

[Deleted User] <[Deleted User]> #12Sep 19, 2018 11:09AM

[Deleted User] <[Deleted User]> #13Sep 24, 2019 08:11AM

jr...@gmail.com <jr...@gmail.com> #14Apr 19, 2021 10:11PM

Has there been any development for this feature?
It would super helpful to have something like pgbouncer transparently available as a simple config option when provisioning a cloud sql instance. As a newcomer to postgresql I actually thought this would be the default config for cloud sql since it is such a widely recommended setup. It wasn't until I noticed the complains from people much more familiar with postgres than me, that I noticed the oob limits on connections and that many users end up hosting their own pgbouncer instance. Which is of course possible but not ideal, specially for a product whose core value proposition is management/deployment/operational simplicity.

co...@efugulin.com <co...@efugulin.com> #15Dec 11, 2021 07:01PM

ch...@bergersenweb.com <ch...@bergersenweb.com> #16Jan 8, 2022 01:53PM

aj...@google.com <aj...@google.com> #17Jan 10, 2022 05:25PM

Hello Cloud SQL users,

We are interested in learning more about this feature request. If you star this item, please comment in with (1) what database engine/version you are using and (2) what connection pool tools, if any, that you use right now with your Cloud SQL instances or other similar databases.

All the best,

Akhil

Cloud SQL Product Manager

Message last modified on Jan 24, 2022 04:03PM

me...@kieranbenton.com <me...@kieranbenton.com> #18Jan 10, 2022 05:33PM

(1) Currently on 11.13 but upgrading to latest stable soon.
(2) Application instances maintain their own pools (using the Npgsql library), these are aggregated via a seperate deployment of pgbouncer (in GKE).

Managing pgbouncer manually in GKE is a pain, as scaling it down in particular causes transient connection failures in the application pools. This is work aroundable but not without significant engineering effort we've not had chance to put in place yet.

Message last modified on Jan 10, 2022 05:33PM

al...@gmail.com <al...@gmail.com> #19Jan 18, 2022 05:32AM

We're on postgres 11 but also have it in our plans to upgrade to 14 in Q1.

Our applications running in App Engine Node Flexible use their own connection pools, provided under the covers by the slonik NPM library.

The biggest issue we hit is when we have a lot of simultaneous events that trigger Cloud Functions for us that need to hit the DB. Even without a lot of load, we can spin up a bunch of Cloud Functions concurrently which can easily max out or DB connection limits. Would be really nice if we could just flip a switch to turn on pgbouncer on our DB instance. Basically would like this to work exactly how Azure does it on their managed postgres service:

https://docs.microsoft.com/en-us/azure/postgresql/flexible-server/concepts-pgbouncer

sc...@leanpath.com <sc...@leanpath.com> #20Feb 24, 2022 07:11PM

Other service providers provide PGBouncer as a feature that can be natively enabled/managed, like Digital Ocean ( https://docs.digitalocean.com/products/databases/postgresql/how-to/manage-connection-pools/ )

With the latest incident https://status.cloud.google.com/incidents/MqpDy4VdUKuQkw1GXPTo having such a feature available as part of the managed service would allow customers to mitigate this incident rapidly. I expect many people are using Cloud SQL to avoid having a DBA, but how you scale connections on say MySQL and PostgreSQL are a bit different as I'm sure some people recently found out.

Edit: Using PostgreSQL 13. Managing my connection pooling in app with HikariCP combined with the cloud-sql-jdbc-socket-factory. We have manually examined our peak application scaling and per application pool size to stay under out db limits. Recently migrated from CloudSQL MySQL

Message last modified on Feb 24, 2022 07:15PM

go...@thingo.nl <go...@thingo.nl> #21Apr 4, 2022 12:34PM

1) mysql 5.7 and 8.0
2) proxysql

Would be very nice to have a managed service for running proxysql for handling serverless peak scaling.
(its a must actually; and a pain to setup and manage on GCE or alternative)

re...@inteji.com <re...@inteji.com> #22Jul 28, 2022 09:53AM

We are using PostgreSQL and currently maintain a pgbouncer instance pool.

This prevents us to use IAM authentication onto CloudSQL for our services, and would really like to be able to switch to it, but it would create too many connections to the database without pooling.

Feel free to contact me if you want more details about our setup.

Message last modified on Jul 28, 2022 09:55AM