IssueTracker

I have filed your feature request to our product team. However, there is no guarantee that this feature will be implemented, and the ETA cannot be provided. Rest assured that your feedback is always taken into account, as it allows us to improve the platform.

Any future updates to this feature will be posted here.

Hi,

I understand that the issue is potentially low priority and that numerous workarounds exists but I would like to share some background info about this.

We have a Cloud SQL Second Gen instance and that went nuts one day, consuming all disk space in the universe up until it reached the maximum disk capacity of ~9TB. That issue "self resolved" (some Cloud magick is not bad...) and we did not saw any impact on any of our queries or none of our database users noticed.

This is to some extent good: something that would otherwise cause service disruption did not, the issue self-resolved and the service continued to work. However, our data usage for the instance is around 20G. But our database now have 9TB of disk space allocated to it. Huge amounts of *wasted* disk space in the Cloud era.

I have lots os services that use this database and rebuilding it from scratch (*shutdown services*, *cause service disruption for users*, mysqldump, backup, stop old instance, start new one, use mysql to restore data, reconfigure all services to new database, new IP, etc.) is laborious and, well, can happen again in the future.

It would be a nice feature to reduce the size, but I understand as a developer that "it is not that simple". Lots of infrastructure and key design decisions prevent me as a user to "just change the size down", but I also have no way/tools to assist me into moving the relevant data (20GB) out of all that wasted reserved "empty" disk space I'm paying for, without causing service disruption. Instead of spending time testing and simulating that DB work to rebuild our DB, I would like to use it to improve our system to our customers. But I'll have to go to the office on a Sunday to make this slightly laborious change because I have no easy way around this one.

+1 to this.

I've wasted hours on this problem and still haven't found why my new database won't connect to my infrastructure.

Perhaps a useful approach to this would be a duplicate function that let's me set the new DB size and keep all the other config the same.

Or even the ability to export the full config of the instance as a JSON file (or whatever) - then I could do that for both instances and figure out what the heck is different between them!!

deleted

This issue can have a major side effect from a cost perspective as well. If a customer uses a "temp" table for a data-load or test, the database will grow to make room for that table. However, once that data-load is complete, there is no way to shrink the DB back to its original size.

The problem can compound if a customer then clones this db for other environments, as the excess (now-unused) disk space will also be reserved in the clones.

The end result can be a cloudSQL bill that is dramatically higher than expected or forecasted.

> The problem can compound if a customer then clones this db for other environments, as the excess (now-unused) disk space will also be reserved in the clones.

I just confirm that this is a big problem for us as well. We rely on the backup restores to create development environments, thus the resized disk now would have to propagate to all development instances as well.

Also for us this is a big problem.
If we have to restore a copy of database to take back some data and then delete the restored db we have to do it in another istance, else where first istance grow
and you have to pay for that extra space forever.
It's also annoying that if you have to make a copy of db for some test you have always to do it in another istance, and at the end delete the test istance
elsewhere you pay forever for unused free space.
And if you have some script that refer to the istance name, it is a problem becouse you have to change script everytime.

To be honest I was a bit surprised when I found out that you can't decrease the SQL instance size and I'm surprised that after 2 years not much was done, it happens often that you first store the data for example in the db and later compress them or you just need those data for a certain amount of time. It's a waste of space and money for the customers that are paying for something they are not using
Hope this issue will be solved soon

Need a way out myself too :/ We run ops 24/7 and I want an easy way to reduce size w/o downtime.

+1 ... had a runaway storage issue that was blowing up a table size, that was solved but now we can't resize without significant downtime. This is a very real issue that I think a lot of other people are facing. Would love to see some movement on it!

+1

+1; Also the thing is that every time we need to resize, the whole database is needed to be dumped to sql/csv file, and then loaded up, which takes ages.

+1 - an issue caused the db size to balloon to 9TB, and im using a fraction of this storage. If the ability to reduce the DB size is not feasible, perhaps a multi-step way to live migrate to newly resized DB could be a way.

Here's how it could work:
1. Create a new read replica (allow specifying a new size - if the new size is too small, this operation can fail with an error)
2. Once read replica is created, allow triggering a failover to make the newly resized read replica the new master and drop the existing master (with excessive storage allocation).

Is that feasible? I understand it's easier said than done.

+1

+1

+1

+1

+1

+1

+1

+1

+1

+1 pleaseeee

+1

+1 pleaseeeee!!!!!!!

+1

+1

+1

+1

+1

+1

+1

+1

+1

+1

+1

+1

+1

+1

We recently went through a project to clean up and remove a bunch of old data. We have multiple instances now where we're using >3TB less than the allotted storage, costing us > $1,000 a month for unused space.

+1

+1

Same comment here, using auto scale HDD size put you at risk to have a surge in the storage then you are caught with size increase for nothing. And the workaround consisting of creating a new instance to downsize it maybe conplicated for actual production databases

Thanks

+1

+1

Paying double as what we should be paying now due to performing a VCAUUM on a very large table.

+1

+1

+1

"assigned" 3 years ago...

+1

+1 and wow ...

+1 Thanks

+1

+1

+1

+1

+1

+1, please, resolve this ...

+1

+1

+1... I had a case where my storage grew to 19TB, while we use just 70GB of storage. Now we are paying thousands of dollars for a storage we don't use and won't use for a long time into the future. And as you already know the possible workarounds available will lead to a downtime that we can't afford. This feature is very critical.

+1

After more than 3 years this issue is still open..... you guys are really doing a great job, well done! take it easy! no hurry please!

+1

+1

+1

+1

+1

I guess they just want us to keeping paying more. Heroku makes downgrade very easy. Google, shame on you!

+1

+1

+1

+1

+1

As a workaround, you can create a new replica of the database with smaller allocated storage. Normally when you do this, GCP requires that you allocate the same amount of storage to the new replica as the current primary. However, if you set it up as an "EXTERNAL" primary then you can trick GCP into bypassing this requirement. It makes sense, since GCP cannot know how big an external database is and therefore can't enforce the storage requirements.

I just did this migrating from a mostly empty 2TB MySQL 5.7 database to a <300GB MySQl 8.0 database. The instructions here will work:

  https://cloud.google.com/sql/docs/mysql/replication/replication-from-external

The main difference is that the IP address of the external source database will be the public IP of your current Google Cloud SQL database. You may have to create a public IP temporarily to facilitate the transfer.

After you have a live replica with a smaller footprint, you can promote it and migrate your dependent services to it.

+1

+1

@78 That's a good suggestion. However I spent a couple of days trying to get that to work, but synchronization would simply fail with "ERROR" every time, which makes it quite hard to figure out where I was going wrong unfortunately.

+1

+1

+1

+1

+1

+1  - application error caused increased WAL that has been repaired, but the storage size damage is done.

+1

+1

+1

+1

+1

+1

+1

+1

+1

+1

+1

+1

+1

+1

+1

+1

+1

+1

+1

+1

+1

+1

+1

+1

+1

+1

+1

+1 -- same issues as others, temporary dev testing ballooned the size and now that the process is mature we can't right-size.

+1

+1

+1

+1

+1, c'mon Google

+1

+1

Hey google - how about an update?

It's only been a *few years* since the original issue was raised...
Or is this such a cash-cow for you it'll hurt your profits?

+1

+1

+1

+1

+1

+1

+1

+1

+1

+1

+1

+1

+1

+1

+1

+1

I have automatic increments enabled and my storage increased to 5.24 TB, and I only use 1.8 TB.

There is no way for me to decrease, I'm stuck. Because;

It's not possible to export/import sql file since 1.8 TB takes more than a week to export/import, even though I use the highest spec VM.
Google Cloud SQL also doesn't allow me to access mysql data folder, so it's also not possible to export by copying data folder.

I planned to create a read-replica outside of Google Cloud SQL, then switch master to it, then get rid of cloud sql, but its impossible because it takes more than a week to export/import sql file, and Google Cloud only keeps 7 days of mysql bin logs, so new instance can't keep up with the master because of the missing bin logs.

I'M STUCK ON GOOGLE CLOUD SQL.

F

+1

+1

+1

+1

+1 plzzzz

+1

+1 and your defaults don't help I feel I was tricked into start the project with 100 GB for nothing

+1

+1

+1 pleaseeee

+1

+1

+1

+1

+1

+1

+1

+1

+1

+1

I have the same issue - after initial import of the databases the storage capacity is roughly at 200% of what we actually use for data... (after the WAL files have timed out and got removed)

+1

+1

+1.  it's crazy that this is not basic functionality.

+1

+1

+1

+1

+1