Serverless VPC Connector in asia-south1 is out of capacity [164378672]

Fixed

Bug

Status Update

No update yet.

Description

dd...@google.com

created issue #1

Aug 14, 2020 12:42PM

It is not possible to create any Serverless VPC access connector in `asia-south1` as the capacity seems to have been reached.

This can be reproduced by navigating the console to `VPC Network` > `Serverless VPC Access`, and creating a new connector will not list the `asia-south1` region as it is not available. When trying to create one by running `gcloud compute networks vpc-access connectors create [name] --network [network] --region asia-south1 --range [IP range]`, you could be getting this error [1] while choosing another region will run fine. Some customers have reported seeing this error [2].

The expectation is that the `asia-south1` region be available as per this doc [3].

This is issue is known by the Engineering Team, but there will be no ETA on the fix. You may periodically check this entry for any updates on the matter.
At the moment, it is recommended to select another region than `south-asia1`

[1] - '@type':

type.googleapis.com/google.rpc.PreconditionFailure
violations:
- description: Location 'asia-south1' violates constraint 'constraints/gcp.resourceLocations'
on the resource 'projects/ddworks-dev/locations/asia-south1/connectors/asdf'.
subject: orgpolicy:projects/ddworks-dev/locations/asia-south1
type: constraints/gcp.resourceLocations
[2] "Connector is in a bad state, manual deletion recommended."
[3]

https://cloud.google.com/vpc/docs/configure-serverless-vpc-access#supported_regions

Comments

my...@gmail.com <my...@gmail.com> #2Oct 19, 2020 05:51PM

Project: platform/frameworks/support
Branch: androidx-master-dev

commit 1e5efa0794e826bc1b3b153796ed5659fb97fca2
Author: Ian Lake <ilake@google.com>
Date: Fri Feb 08 10:23:54 2019

Prefer exact matching deep links

Instead of only taking into account the
number of matching arguments for deep link
matching, always prefer exact matches.

Test: updated tests
BUG: 123969518
Change-Id: Ibd8ab5b6ba251991fa994ebd297bcb9ffb5851bd

M navigation/common/src/androidTest/java/androidx/navigation/NavDestinationAndroidTest.kt
M navigation/common/src/androidTest/java/androidx/navigation/NavGraphAndroidTest.kt
M navigation/common/src/main/java/androidx/navigation/NavDeepLink.java
M navigation/common/src/main/java/androidx/navigation/NavDestination.java

https://android-review.googlesource.com/898367

https://goto.google.com/android-sha1/1e5efa0794e826bc1b3b153796ed5659fb97fca2

Message last modified on Oct 22, 2020 10:59AM

pl...@gmail.com <pl...@gmail.com> #3Oct 22, 2020 04:20AM

This is fixed internally and will be available in Navigation 1.0.0-beta02 - we'll always prefer an exact deep match link (where exact means your deep link does not contain any .* or {param} tags)

ma...@bambuser.com <ma...@bambuser.com> #4Oct 22, 2020 10:05AM

Had the same issue in the following regions,
us-central1
us-east1
us-east4
us-west2
us-west3
us-west4
europe-west1
europe-west2
europe-west3
europe-west6
northamerica-northeast1

le...@gmail.com <le...@gmail.com> #5Nov 4, 2020 04:33PM

Have same issue with asia-south1 here. Any progress?

bo...@gmail.com <bo...@gmail.com> #6Nov 25, 2020 06:26AM

is there any ETA to fix this , or what is the alternate to connect to cloud SQL & cloud functions ?

pl...@gmail.com <pl...@gmail.com> #7Dec 21, 2020 02:55AM

Any update???

hi...@gmail.com <hi...@gmail.com> #8Dec 26, 2020 08:29PM

It's unbelievable that such functionality does not work. I have tried europe-west1 and europe-west4, error is reproduced every time. The error message is completely useless, the workaround was to set max throughput to 300mbps.

ha...@gmail.com <ha...@gmail.com> #9Dec 27, 2020 08:24PM

same issue :( poor cloud infra by google

mg...@google.com <mg...@google.com> Feb 12, 2021 08:17PM

Assigned to se...@google.com.

ma...@gmail.com <ma...@gmail.com> #10Mar 15, 2021 11:37AM

Our App engine instance and Redis instance are created in asia-south1. But, we are not able to create VPC connector to access the Redis from App engine.

pa...@srellik.com <pa...@srellik.com> #11Mar 29, 2021 07:48AM

Same issue: Our App engine instance and Redis instance are created in asia-south1. But, we are not able to create VPC connector to access the Redis from App engine.

Please update

pa...@srellik.com <pa...@srellik.com> #12Mar 29, 2021 06:01PM

An internal error occurred: VPC Access connector failed to get healthy. Please check GCE quotas, logs and org policies and recreate.

pa...@srellik.com <pa...@srellik.com> #13Mar 29, 2021 06:03PM

We are facing the same issue reported here:

https://stackoverflow.com/questions/64373810/error-while-creating-gcp-serverless-vpc-connection-code-13

so...@gmail.com <so...@gmail.com> #14Apr 10, 2021 02:00PM

Same here for `europe-west4`, and as advised above works with max throughput 300.

yo...@gmail.com <yo...@gmail.com> #15May 2, 2021 04:20AM

I tried to create VPC connector in following regions:
us-central1
us-west1
northamerica-northeast1
asia-northeast1

All prompted：An internal error occurred: Failed to create a VPC Access connector. Please delete the connector manually.

But it is weird that I had been successfully created VPC connector in northamerica-northeast1 regions on few month ago, but now it isn't work

[Deleted User] <[Deleted User]> #16May 16, 2021 11:04PM

Comment has been deleted.

Message last modified on May 18, 2021 12:28AM

sn...@gmail.com <sn...@gmail.com> #17May 25, 2021 11:56PM

also facing the issue for
us-central1

su...@rocketguestposting.com <su...@rocketguestposting.com> #18Jun 7, 2021 04:23PM

"An internal error occurred: Failed to create a VPC Access connector. Please delete the connector manually."

In all regions

re...@google.com <re...@google.com> #19Jun 11, 2021 08:13PM

Reassigned to gc...@google.com.

The Engineering team still working on the issue but there will be no ETA on the fix. You may periodically check this entry for any updates on the matter. Thanks

da...@google.com <da...@google.com> #20Jun 24, 2021 10:27AM

Hi,
I am checking for status update on this Bug .I am contacting regarding case i am working on ,where customer is unable to create VPC Connector.

Message last modified on Jun 24, 2021 10:30AM

al...@google.com <al...@google.com> #21Jul 1, 2021 09:10PM

Hi,
The capacity check has been deployed to all regions in prod. Customers will now see an Insufficient CPU quota in region error message when connector creation fails due to a lack of quota.

al...@google.com <al...@google.com> Jul 8, 2021 10:17PM

Marked as fixed.

li...@superbtech.asia <li...@superbtech.asia> #22Aug 18, 2021 06:46AM

hi, i've face this issue too.
is there a specific way of what to do to solve this problem?
im not pretty clear of what this "The capacity check has been deployed to all regions in prod. Customers will now see an Insufficient CPU quota in region error message when connector creation fails due to a lack of quota" means.

thank you

ma...@stone.com.br <ma...@stone.com.br> #23Aug 18, 2021 02:04PM

The same here!

Message last modified on Aug 18, 2021 02:05PM

dm...@deskree.com <dm...@deskree.com> #24Sep 2, 2021 02:26PM

Same here. Trying to create a serverless VPC connector for us-east4

ro...@gmail.com <ro...@gmail.com> #25Sep 13, 2021 02:07PM

We have the same issue for us-central1

ob...@gmail.com <ob...@gmail.com> #26Sep 13, 2021 02:08PM

The same issue for us-central1 as well

ca...@cleafy.com <ca...@cleafy.com> #27Sep 23, 2021 09:45AM

Still, same issue for us-central1, and several weeks have passed.

Error is the same: "ERROR: (gcloud.compute.networks.vpc-access.connectors.create) {
"code": 13,
"message": "An internal error occurred: Failed to create a VPC Access connector. Please delete the connector manually."
}"

This is critical for us because without VPC Connectors we can't connect to other services from serverless products (App Engine mainly).
Maybe it's time to move to a different cloud provider...

sh...@gmail.com <sh...@gmail.com> #28Oct 9, 2021 07:44PM

Hi,

Hitting this also -
{
"code": 13,
"message": "An internal error occurred: Failed to create a VPC Access connector. Please delete the connector manually."
}"

Tried every region and every combination of subnets.
Any workaround? How do I connect a Cloud Run to Compute Engine without VPC connector?
Please help.

sh...@gmail.com <sh...@gmail.com> #29Oct 13, 2021 07:39AM

Hi.,

Facing similar issue

"status": { "code": 13, "message": "An internal error occurred: Failed to create a VPC Access connector. Please delete the connector manually." },

Any workaround., I am trying to connect Cloud function to GKE resources., Its a critical business requirement.. Please help ASAP.

ha...@gmail.com <ha...@gmail.com> #30Oct 21, 2021 08:24AM

Hi,

Exact issue in europe-west3 region:

Error code 13, message: An internal error occurred: Failed to create a VPC Access connector. Please delete the connector manually.

an...@gmail.com <an...@gmail.com> #31Dec 19, 2021 09:37AM

Hi All,

Same story here. Tried all regions in EU and US - none of them works even with limiting the max throughput to 300.

It does not look like fixed although someone marked is as such on July 9, 2021.

Very disappointing.
Anton

Sun Dec 19 2021 10:33:47 GMT+0100 (CET).png

267 KB

View

Download

an...@gmail.com <an...@gmail.com> #32Dec 19, 2021 09:49AM

Reported this again here:

https://issuetracker.google.com/issues/211197245

ma...@gmail.com <ma...@gmail.com> #33Dec 24, 2021 01:36AM

I just encountered this problem, in my case it was because I tried creating one in a project without the default compute engine service account. When I recreated the default compute engine service account it worked.

I've also seen some other people mention a similar problem occurs if you don't have sufficient CPU quota in the associated region

[Deleted User] <[Deleted User]> #34Jun 4, 2022 05:32PM

this is still not working. Do we have any solution?

ka...@searce.com <ka...@searce.com> #35Jun 29, 2022 07:12AM

Tried for us-central1 and its not working.

[Deleted User] <[Deleted User]> #36Sep 8, 2022 03:28PM

Running into the same issue.. in europe-west2

[Deleted User] <[Deleted User]> #37Sep 12, 2022 08:32AM

Same issue today with europe-west1

br...@gmail.com <br...@gmail.com> #38Sep 14, 2022 11:29PM

+1 us-central1 today. I was testing out gcp for a new application. Probably going to abort and switch to AWS...

de...@gmail.com <de...@gmail.com> #39Sep 22, 2022 10:40PM

Facing the exact same issue with asia-south1
us-central1 seems to work though, but need it to work in asia-south1

jo...@gsoi.fr <jo...@gsoi.fr> #40Oct 12, 2022 02:42PM

Hello, same issue with europe-west1 region.

ma...@metro-external.digital <ma...@metro-external.digital> #41Oct 27, 2022 09:55AM

Hi I am facing the same issue for region europe-west1. Any leads on how to resolve this issue?

se...@classroomscreen.com <se...@classroomscreen.com> #42Nov 28, 2022 06:31PM

Same here, us-central1. This is a critical service for us and I have not found any workaround. I guess the status 'fixed' means that it will not be looked at anymore?

se...@classroomscreen.com <se...@classroomscreen.com> #43Nov 29, 2022 10:43AM

Update: After trying for hours yesterday evening, this morning it just decides to work. Maybe the issue is also time/traffic dependent? I don't know. But to all people affected, perhaps retrying in a different moment will work... Our setup is done completely scripted, so I am 100% positive I used the same settings today as yesterday...

[Deleted User] <[Deleted User]> #44Feb 8, 2023 05:08AM

Error -An internal error occurred: VPC Access connector failed to get healthy. Please check GCE quotas, logs and org policies and recreate. while creating serverless vpc connector ?
Is there any thing to fix this issue ?

Message last modified on Feb 9, 2023 03:04AM

li...@ebay.com <li...@ebay.com> #45Feb 24, 2023 04:36AM

message: An internal error occurred: VPC Access connector failed to get healthy. Please check GCE quotas, logs and org policies and recreate.

I have "unlimited" quota of compute engine on my project.

ma...@carma.com.au <ma...@carma.com.au> #46Mar 28, 2023 03:41AM

I am seeing the same issue, intermittent. Worked yesterday, same script today consistently fails.

Error waiting to create Connector: Error waiting for Creating Connector: Error code 13, message: An internal error occurred: GCP region quota data malformed.

ge...@gmail.com <ge...@gmail.com> #47May 31, 2023 09:09PM

I am facing the same issue! 2023 in USWEST1

no...@gmail.com <no...@gmail.com> #48Jul 1, 2023 07:02AM

facing the same issue with asia-south1

so...@hdruk.cloud <so...@hdruk.cloud> #49Aug 9, 2023 02:50PM

This is happening in europe-west1 (and others) any news about a proper fix?

ma...@quickplay.com <ma...@quickplay.com> #50Sep 11, 2024 10:27PM

is this still occuring?
I'm unable to create in any region today

Message last modified on Sep 11, 2024 10:28PM

ab...@worldline.com <ab...@worldline.com> #51Oct 10, 2024 08:12AM

Seems like there is no proper answer to this ? I am also facing the same issue.
The logs also do not show anything informative.