IssueTracker

Also adding my +1 that we just had to put in significant effort to move one of our core flows off of Cloud Functions due to this and have plans to migrate the rest of our infrastructure as well because 12 seconds is pretty ridiculous latency.

I'm really scared to shift to AWS or other alternatives at this moment but seems there are no other options. If it's related to my own dependencies leading to cold start, I accept it. But here, I have no control when my code execution gets started.

At this point we are tightly coupled with GCP and down the line if this issue is not yet addressed, we may need to start migrating to other alternatives. I understand Google doesn't commit to any ETA' s as per their policies but this issue is a hidden massive loop hole in the whole cloud functions system.

I hope someone from official team updates on the status.

Can anyone from team update incase if this issue is under resolution?

Looks like they corrected the trace now. Now its showing that our code starts at 0th ms (actually showing in negative values which is impossible). But atleast now its showing the function is getting called as soon as its triggered, which is great.

Anyone else experiencing the same? I'm using the latest versions, btw.

Will debug more.

Looks like it got reverted again. I see huge delays again before start of the function!!!

Hi folks,

Apologies for the delay in responding here. The quick summary is "we're still working on it" (The last update on the internal bug was yesterday). I can't give an ETA because honestly we don't know, but the "good" news is that we're able to reproduce the issue and we're investigating the cause. This is not expected behavior, which is why it's taking some time to track down (it unfortunately was not an obvious cause)

We will update again once we know more.

Thanks Vinod for sharing some info. May I know what fixes are getting soon? By soon is it in days or weeks or months? I understand Google can't commit to timeline but at-least info about a tentative date to developers is pretty much required(considering you are already on the fixes)

It at-least helps us to evaluate based on that. Also, the fixes we saw couple of days got reverted and again its showing around 5 secs just to start the function. Initially I thought its because of firestore libs and took the pain to convert all to REST api's. But now again we see it happening.

It would be really great if you can share what fixes we may seen soon so that we can see if our issues are covered.

Thanks, Ayyappa

We still see the delays however we would like to try with cloud run as suggested by the team. We will update here if we have further updates.

Also incase if you have updated from Node8 to Node10, make sure you change your code to the latest environment variables. Mainly, replace FUNCTION_NAME to K_SERVICE if you are using it in your project.

Thanks, Ayyappa

We moved to Cloud Run with the help of google team and its good w.r.t cold starts but with some drawbacks. However coming from functions we need to figure out few things

1. Cost comparison (looks like its 10 times more pricey but concurrency might help to reduce)
2. Debugging with trace becomes difficult as it won't show any logs
3. Concurrency number per cloud run instance
4. Additional Min instances cost to keep instances warm
5. Thread safe code as cloud run allows concurrency compared to functions where it's more isolated
6. Unable to deploy multiple functions as different cloud run services(as functions framework allowing only one target). Functions offered more control where we can set the required memory per function.

Unfortunately, I can't exactly measure the cold start as its not displaying more details on the trace. Will keep you informed.

On another note, we totally migrated from firestore client libs to REST api which saved couple of seconds.

Is this affecting all Node runtimes- 8, 10, and 12- or only some of them?

Any updates on this Vinod? We still have firebase triggers on functions which have lot of delays.

Hey Vinod, Coming to the firestore triggers, It's been more than 4 days I ran some tests. I gave a quick test now and see the cold starts are around 2secs which is ok for a background trigger. But I can tell it's far much better than earlier which used to be in the order of 5-10secs.

Thanks, Ayyappa

Firebase customers: the FUNCTION_NAME env var has been changed to FUNCTION_TARGET, and this may be impacting your cold start performance.

If you are using code such as in this Github issue, e.g. like this example and many similar examples in that issue, that code is now falling back to the "load all functions" behavior because the FUNCTION_NAME env var has gone away.

Adjusting the code to look at FUNCTION_TARGET should improve cold start performance in that case.

K_SERVICE also works (it will have the same value as FUNCTION_TARGET).

The wording on http://cloud/functions/docs/env-var is a bit unclear, but I think that FUNCTION_TARGET is the thing that explicitly maps to the in-code function.

For example, K_SERVICE does not appear in the Node.js Functions Framework [1], but FUNCTION_TARGET does [2].

[1] https://github.com/GoogleCloudPlatform/functions-framework-nodejs/search?q=K_SERVICE

[2] https://github.com/GoogleCloudPlatform/functions-framework-nodejs/search?q=FUNCTION_TARGET

The Functions Framework contract also explicitly mentions FUNCTION_TARGET: https://github.com/GoogleCloudPlatform/functions-framework#specification-summary

While I appreciate that (ostensibly) you guys have been actively working on this, I think we're all starting to feel that your team's definition of "active" may be different than that of other developers.

I'm the engineering director at Google responsible for Cloud Functions. For a multitude of reasons, multiple changes/updates haven't yet made their way to production which we believe would have improved the performance issues cited in this bug report. Although I cannot promise that it will be fully resolved once those changes and updates have rolled out.

What I can promise you is that starting in January you'll see us making a lot more progress on this, and with regular updates as well as transparency on what we believe is causing the issue and what we're doing to address it.

An update

👋 stefernet@ brandon@, other folks in this thread, I'd like to give some perspective and explain why this ticket has taken quite some time to address.

When this ticket was opened we were having intermittent infrastructure issues with Cloud Functions resulting in poor performance. This thread helped surface a P0 issue, which was addressed.

However...

As this ongoing discussion demonstrates, this isn't the whole story.

A variety of people continue to have a poor experience with Cloud Functions. Internally, we've put together a group of folks across several teams (Cloud Functions, Developer Relations, support, Firebase), and have been discussing how to better meet people's expectations around performance.

What's clear, is there's no one problem:

• there were infrastructure issues; some of which have been addressed, some of which where there's ongoing work around (e.g., improving gVisor read performance).
• large dependency graphs hurt cold start performance.
• Cloud Functions can be a tricky paradigm to code for, which can contribute to cold start issues.
• there are product improvements we can make to make cold starts less frequent.

Our plan

• I've worked with support to pull together this comprehensive article "Tips for writing and deploying Node.js apps on Cloud Functions". It's my hope that this helps any people in this thread who were bumping into issues due to coding hiccups (such as unhandled promises).
• We have features in the works that will allow people to avoid frequent cold starts; concretely, we intend to support minimum instances for Cloud Functions.
• We intend to continue having cross- team meetings to get to the bottom of specific problems customers are having (for instance, why slow cold starts seem to frequently be correlating with certain libraries).

Unfortunately, this thread has turned into a bit of a catch- all issue (for a problem that is nuanced).

In the new year, I would like to close this issue, in favor of more specific issues where we can dig into specific categories of problems (this will make it easier for us to help individual customers).

We want Cloud Functions to be an awesome product for everyone's use cases. And I apologize for the frustration this long-lived thread has caused.

Edit: in response to brandon@, we will update the "best practices" post, as we release features in the new year. There will also be posts to cloud.google.com, and @googlecloud/(@BenjaminCoe) on Twitter.

An update

👋 I’ve updated my initial post and the title to be more specific, based on the problems still being discussed in this thread:

Cold Start performance issues seem to correlate closely with gRPC libraries (like Firestore). Folks switching to HTTP dependencies, from gRPC, have seen performance improvements (this seems to point to gRPC as well).

If the problems you’re running into do not seem to correlate to gRPC SDKs, such as Firestore, please don’t hesitate to open an issue and we will investigate (Also, I’ve pulled together a Tips & Tricks post, based on some of what I’ve learned investigating this thread which might help).

• We continue to work on cross cutting features that will help cold start performance in general, e.g., Min Instances for Cloud Functions, and will keep this thread updated as features roll out.
• There's an ongoing internal conversation about how we can improve cold start issues brought about by dependencies.

Sounds like I'm having the same issue (https://twitter.com/sambecker/status/1352676553671532544).

Basically, created a blank firebase project with an https function that modifies one field of one firestore document.

Cold start times (which seem to reset within 30m) are in the 6000ms range.

10:28:41.934 ColdStartTest.tsx?6c99:21 Request starting ...
10:28:48.176 ColdStartTest.tsx?6c99:24 Success
10:28:48.176 ColdStartTest.tsx?6c99:29 Cold start time: 6242
10:28:53.802 ColdStartTest.tsx?6c99:21 Request starting ...
10:28:53.979 ColdStartTest.tsx?6c99:24 Success
10:28:53.979 ColdStartTest.tsx?6c99:29 Cold start time: 177
11:07:39.914 ColdStartTest.tsx?6c99:21 Request starting ...
11:07:46.073 ColdStartTest.tsx?6c99:24 Success
11:07:46.073 ColdStartTest.tsx?6c99:29 Cold start time: 6159
12:13:45.491 ColdStartTest.tsx?6c99:21 Request starting ...
12:13:52.109 ColdStartTest.tsx?6c99:24 Success
12:13:52.110 ColdStartTest.tsx?6c99:29 Cold start time: 6619

I can post the repo if it's helpful, but honestly it just seems like it's a problem using firestore within a firebase project which doesn't sound like it makes any sense? Please correct me if I'm misunderstanding this thread! I was encouraged by @mbleigh to contribute.

EDIT 1:

I should mention that I'm using the dependency firestore-admin vs @google-cloud/firestore as that's what the default Firebase project starts you with. Would love to know whether that distinction's related to this issue or not.

EDIT 2:

Switching from default function memory of 256MB to 512MB seems to bring cold starts down to 2000-3000ms range. Amazing to think that would be necessary on an empty firebase project that makes use of firestore but very good to know.

remove the firestore libraries from Google and just go straight to REST API calls straight to the firestore API

This is a remarkable observation, and I'm frankly amazed that the official shipped SDK would be significantly less performant than effectively writing one's own SDK to wrap the underlying firestore API.

In common with others, I've also deduced that cold start times on our FB projects are largely impacted by the first call to admin.firestore() - not module imports, or global initialisers. Just this. We wrapped a timer around that call and we could see it represented the majority % of the first run time.

console.time("admin.firestore() " + process.env.K_SERVICE);
const db = admin.firestore();
console.timeEnd("admin.firestore() " + process.env.K_SERVICE);

I can concur we've also upped the memory allocation on some of our busier functions simply to reduce this firestore latency a bit. We also went so far as to use the RTDB for some datastore functionality we needed to have faster response times on, since it spins up in ~80-100ms rather than ~300-800ms for firestore.

In common with others, I've also deduced that cold start times on our FB projects are largely impacted by the first call to admin.firestore() - not module imports, or global initialisers. Just this. We wrapped a timer around that call and we could see it represented the majority % of the first run time.

Totally agree with this. I spent a lot of time refactoring our cloud functions to follow the best dependency practices outlined in the various articles published. After completing that we saw only a minor improvements in startup times. We then discovered this issue and moved all of our services to Cloud Run. We still use firestore trigger cloud functions but they simply forward the request to our cloud run instance. By avoiding any calls to the admin SDK in the firestore triggers themselves we're now down to sub-second cold start times.

This problem should be mentioned in the performance articles. Don't have people waste a bunch of time optimizing dependencies and import patterns if you have such a blatant performance problem in the admin SDK.

Given how long this has been a P1 issue I've given up on it. I would really like to see Eventarc include firestore trigger events so that Cloud Run can receive those directly.

Meanwhile google is pretending to work at this with P1. How can it take over a year to fix this for the firebase team when even a quick bodge by some users has a such a big impact.

There's a reason why Google cloud has lost $5.6B in 2020.

I'm waiting around +20 seconds to save a document + firestore trigger + Save other document - which is a lot since is just an audit behaviour.

Not through pubsub, just a normal HTTP call to our Cloud Run instance. ie.

const host = functions.config().api.functions

const httpClient = axios.create({
  baseURL: host,
});

export const onMessageCreated = functions.firestore
  .document('threads/{threadId}/messages/{messageId}')
  .onCreate(async (snapshot, context) => {
    const threadId = context.params.threadId;
    const messageId = context.params.messageId;
    const message = snapshot.data() as any;
    message.messageId = messageId;
    await httpClient.post('/functions/onMessageCreated', {
      threadId: threadId,
      snapshot: message
    });
  });

Not ideal but brought our cold start times from ~20 sec to < 1 sec. Getting trigger events to Cloud Run (or fixing this issue) would be ideal because all cloud functions are doing for us now is adding latency and cost.

The advantage to this setup is that you can configure the host to point to your dev machine using something like ngrok. This makes debugging triggers code much easier.

There are three different paths that we're taking to resolve this issue. For many, I believe #3 will be the most practical and near term outcome.

1. Generally speaking, Cloud Functions cold start time is not as good as industry leaders in this benchmark, and we expect to improve this over time, but I don't have a time table to communicate right now. There are ideas that we're investigating, but they're too early for me to give you a definitive timeline, and I would categorize this as incremental product performance improvements that we're going to deliver over time.

2. Firebase client libraries seem to be particularly slow to start up. Which we think is specifically related to the presence of gRPC and Node.js. For the purposes of resolving this issue, I want to delegate this to the Firebase team, as this does not appear to be a problem specific to Cloud Functions. Unfortunately, Firebase does not seem to have a public issue tracker, and I need to figure out where to file this issue.

3. Finally, there are cold starts improvements coming to Cloud Functions in the form of the Min Instances feature. We've created a feature request here https://buganizer.corp.google.com/issues/181884353 which you can subscribe to for more updates. We expect to have this feature in Private Preview in the next 4-6 weeks. For many users here, this may be a satisfactory resolution.

#1 is true but I find without the firebase SDK it's acceptable. Any improvements would be appreciated.

Min instances are a great addition for specific use cases but overall they are not providing a huge benefit to us. In our application we have many scaling events during the day. Our app is spanned across multiple time zones and there are time periods within each timezone where functions need to scale up to meet demand. When our application needs to scale a user was often waiting 20s+ as a new instance came online to handle their request. Then they would perform another action in the application which would cause yet another scaling event for a different function. Again waiting 20s+.

Solution #3 just means we have to set min instances on a large number of our functions to avoid this experience for our users. We still run the risk of the app scaling past the point of my configured min instances. While this will band-aid the issue, the value proposition for cloud functions is lost once you start using the Firebase SDK.

#2 is the solution I am waiting for... OR the ability to deliver firestore triggers and callable functions via EventArc so we can handle them directly in Cloud Run.

Hi all, for Firestore improvements, I talked to the Firestore team and they asked me to pass this along:

The Firestore SDK team has examined the impact of loading the Firebase Admin SDK pretty extensively, and while there are some small gains to be had code loading/weight doesn't explain anything like the multi-second cold starts folks on this thread have reported. We believe the issue seems to lie specifically with the gRPC connection the Firestore SDK uses to read data, but our internal testing has not been able to reproduce the same effect that has been described in this thread. We're going to keep investigating (including options such as providing a non-gRPC SDK), and if you can reliably reproduce 5s+ cold starts with a minimal code sample, we'd love to know more about it so we can take a look (including how you're measuring the cold start duration).

I'm going to move this issue to the Firestore component so that the Firestore team can continue to action this issue.

(For GCF specific improvements to cold start, as mentioned earlier, I would direct you to https://buganizer.corp.google.com/issues/181884353).

We're still experiencing 5+ second cold starts for functions that perform a simple Firestore transaction. We love the ease-of-use of GCF, but we're now forced to reevaluate other options as this behavior is not acceptable in production. Frankly, I'm surprised Google hasn't pulled out all of the stops on this one, as it completely cripples one of their flagship cloud products.

The new min-instances feature does help some, but it's only a bandaid that does not resolve the problem. Even with min-instances, customers are still vulnerable to cold-starts when concurrent requests exceed the allocated capacity.

Hi everyone. Thank you for being patient with us while we have looked into this issue. We understand the impact it has and how frustrating it can be to deal with. We take this issue very seriously however there are a lot of complex moving parts involved which must be adjusted very carefully. To date we’ve done the following:

1. Worked directly with numerous customers that reached out to us and helped them address their issue
2. Published best practices on writing performant functions
3. Rolled out a series of performance enhancements to cut function startup time for common workloads.
4. Released the min instances feature
5. Sped up function invocations from Firestore Triggers when only reading the triggered document (pull request).

We have more improvements coming. Please stay tuned.

As we explore and prioritize other improvements it would help if we knew the following:

1. What are your startup times at the 95th vs 99th percentile when testing a minimal reproduction of the issue? How does it account for network overhead between the origin and GCP? Please share your code and data with us.
2. What is your specific pattern of traffic at the time you see the issue? For example, is it during significant spikes, idle times, steady state, etc? What is the traffic volume at the time?

Thanks for posting an update. I think everyone who's still bothering to follow this is trying to be patient, but this is an egregious length of time for a significant blocker to usage. A lot of the frustration seems to come from the obviousness of the solution. The requisite dependencies for interacting with many Firebase components cause this issue, and they are absolutely the prime offender regardless of how many other dependencies are included: they need to be trimmed/deferred/modularized/all of the above.

It's great that there's at least one deferral change in place is starting to get attention - #5 is an example of this - but at this one-use-case-per-year pace, many of us may never see results which make expected use cases for Functions w/NodeJS work properly.

There are numerous examples provided by users of the exact behaviors which are issues here (e.g., comment #94). The fact that we can get rid of much of this problem just by making our own wrappers and skipping the Firestore SDK is an indictment of the state of that package (though Firebase Admin is also an offender). Most of us would like to use the official resources, but maybe they should have a huge asterisk in documentation about the performance implications if they won't be addressed anytime soon.

As a note, the #4 min instances "feature" doesn't actually address the problem at all; cold starts are still exactly the same, you just hit them less often since you front-loaded the lag time. You also have to anticipate your load well enough to set a min instances count that will prevent cold starts and that means functionally paying for peak capacity 100% of the time. Either that, or you hit cold starts when your traffic has any sort of a spike.

One thing that could solve this for the majority of use cases is not de-allocating shards after such a short span. If you allowed each to live for 24+ hours, I suspect you'd find that most concerns would evaporate, though traffic spikes could still cause cold starts. Another approach could partner with this to analyze load increases and anticipate spikes, making available shards in advance of calls being made. Obviously only helpful for Functions with a steady flow of traffic and intermittent ramped spikes, but that is another major use case (for instance, site hosting via Functions would likely be fixed with this).

To answer your questions in our case:

1. We consistently observe 5-8 seconds for cold starts in most cases, with it rarely dipping below that range. Statistical analysis is unavailable. We look at the logs while loading an asset, observe the lag, observe that the function doesn't start for that length of time, note the response is received after the delay. The next request completes without delay, as do all following that fall within a window where there has been prior activity. Not sure if it's enough to "account" for network overhead. Honestly, if you're still at this phase of investigation, this is likely a lost cause - just try out any minimum viable examples that weren't explicitly fixed by the special patch in #5 and you can easily observe this behavior yourself. We will not be able to share our specific code or data.
2. This is at any time when a Function is called on a new shard (or whatever the term is for the container running the Function). It's especially obvious if a Function has been dormant for more than ~20 minutes, guessing that shards are deallocated at that point and so any call needs to cold start one. I'd expect any deviation from a completely steady stream of requests would eventually trigger this.

Hi everyone,

Yes, we are still looking into this issue and it is still a priority. As I mentioned back in June it is a very complex problem due to all the moving parts involved and improvements will be incremental. I'm not at liberty to discuss specifics but we have improvements in progress and this ticket will be updated when we have something public we can share.

Thank you for your continued patience.

If I understand correctly https://serverless-battleground.vercel.app/ is using AWS Lambda, right? So what it shows is that the Firestore library also has high latency on AWS Lambda, similarly to what it observed on Cloud Functions.

Hi,

As an update on this Issue Tracker, our Engineering Team is still working on this issue and they will provide an update as soon as they have something relevant to share.

Note that customers who wish to keep their Cloud Functions warm to avoid cold start can leverage the "min instances" feature https://cloud.google.com/functions/docs/configuring/min-instances

comment

Hi,

Thanks for contacting Google Cloud Platform Support,

The library that mentioned is not supported by us. As such, we cannot answer any questions about its functionality.

Regards

The fact that now two Google customers have provided meaningful workarounds to a product-breaking, 16-month old problem that a nearly two trillion dollar company hasn't provided any solutions for does not inspire confidence.

It looks like there are about 30 Googlers CC'd on this issue, to which I say this: Please fix your product. We really want to use it, because when it works it works really well. However, this problem makes it completely unusable in production. We cannot reasonably ask our users to wait 6+ seconds for a function to cold start. The workarounds provided by Google can reduce some of the impact but are not sufficient. It is clear that Google is trying to grow their cloud market share, and we're cheering for you, but the inaction on this issue is producing the opposite effect.

If Google cannot or will not solve this issue in a reasonable amount of time (which, frankly, was a long time ago), my org is going to be forced to abandon this product. We cannot gamble our own success on hints of an eventual resolution. After 510 days of waiting, we need more than the unfulfilled promises we've been given thus far.

Sadly, we've had to migrate away from Firebase Functions for hosting our Node.js programs. We had assurances from their team in December 2020 that this was being fixed ASAP, and here we are in October 2021 with no progress to speak of. We've moved on to Cloud Run, but frankly, if we weren't already built out for other portions of the Firebase environment such as auth, we'd have migrated back to AWS and likely never revisited GCP again. That may still happen once we have time to port functionality - my faith in critical issues being resolved by the GCP team is entirely eroded.

This is a severe black eye for any aspiring cloud platform, and many times more so given this is Google! Either tell us you're not going to fix this, or commit to fixing this on a certain time table and do it. Not that it'll benefit my team any longer, but for the sake of everyone else here, get your head in the game.

TLDR: Try writing critical client-facing functions in Go

Yes, Go is going to be better than Node.js, but I was curious to see how it performs in the Functions environment, especially considering this issue. This is my first post here, there are plenty of complaints here, I thought I might try to share my own experience. This is absolutely not a silver bullet and not applicable to everyone, but it might help certain use cases. I've been increasingly frustrated by the 5-15 second cold-starts, and this is just to execute a simple Firestore query returning a single document ID, seeing as the client libraries do not play well with Node.js SSR. Cold-starts are crucial for client-facing functions. Minimum instances are expensive for some, and they still affect that random unlucky client.

Serverless is supposed to be fast, not a background processing platform (although it excels at that too!). It's caused me to reconsider Firebase several times and try Cloud Run with Go, but then I realise that Functions aren't much different, and I keep coming back. Some things, like storage rules and auth, are baked into the Firebase platform. It's exhausting and frustrating, but at least I don't have the pressures of a company, my heart goes out to you that do.

I haven't had time to try the above community-improved libraries, what I did try is re-writing a function in Go after seriously considering starting from scratch on Cloud Run, and I'm shocked at how much better it performs.

• Before: 1G instance, 8000-15000ms cold-starts in Node.js (depending on which VM is allocated, and probably also somewhat dependent on the direction of the wind), ~100ms warm execution times (as little as 3ms for CORS pre-flight requests)
• After: ~80ms cold-starts using 128M instances written in Go, using around 20MBs of memory. In fact, the 128M seems to perform better than 1G, strangely. I haven't measured the overhead due to the actual allocation of a machine and download of the container, but it is negligible with network latencies.

This is not a viable solution for everyone. I myself have built significant tooling in Typescript to get rid of boilerplate, stuff that would have been amazing to have in a firebase-contrib type standard library. It's a struggle to set into a new language with no generics and increased verbosity, but the performance gains are worth it. I just wish that Firebase natively supported Go.

Some of the pros 👍 are:

• No need to lazy-import files to help that cold start
• No more worries about exceeding the maximum function upload size (node_modules), Go is naturally smaller and has dead-code elimination I believe
• Go doesn't have to synchronously require(...) 10/100/1000s of files from the disk (future ES Modules and bundling could help, but it's extra setup boilerplate)
• The SDKs seem to be high quality, perhaps better than Node.js (native Query iterators that I avoided in Node.js due to lack of documentation), the bonus of being Google's star language, it pretty much feels like import-and-go (also an excellent language for bad puns), like with Deno
• The std library is amazing, it even has image manipulation for thumbnail processing
• It integrates nicely with the Firebase dashboard and error tracker, as if it was meant to be
• Did I mention the cold-start times that are faster than Node.js's warm-start times (OK, maybe exaggerating a bit...)?

Bear in mind that you will have to manually enable APIs and set up Scheduler timers, and there is no native request auth verification (big bummer, there is no onCall equivalent yet, but no problem if the function is public serving), you have to verify that yourself. If you're dependant on Node.js for SSR or have a very large app/function, you're out of luck, but I would recommend taking a look at Cloud Run, it processes several requests concurrently and will further reduce the chance of hitting a cold start, and supports background events (via push). From my experiments, it also seems to allocate extra instances pre-emptively during spikes, but this is pure speculation. Testing & emulators are also going to be more of a pain, but to be frank, it's already a pain with Node.js.

To be clear, I consider this a workaround. The issue lies clearly with the Node.js SDK, not the language or runtime itself. Go is more efficient, but unless you're doing compute-heavy work, it's not magically 10x faster, and it also has its quirks. There is little need for CSP when behaviour is mostly synchronous on a single-core instance. It just feels like the Node.js SDK (or the underlying gRPC dependency) isn't natively JS, but rather ported from Java/Go.

Until GCP addresses this issue, I would recommend to try a gradual adoption with Go for critical parts.

• Create a new directory in your Firebase project
• Add a go.mod file (Go 1.16)
• Create a functions.go file with a non-main package name. This file can house any number of functions, you specify the entry-point when deploying each function
• Each function is a normal Go-type HTTP request handler, very similar to Node.js/Express' request handler.

The good news is that deployment is still very automatic, a single command with no Cloudbuild configuration. The firebase functions config you're used to pretty much maps 1:1 to gcloud functions deploy CLI arguments, it's reproducible and there's no need to mess with the GCP GUI.

Marcus

wbattel@sentryware.com, brandon@askgms.com -- Thanks for taking the time to express your frustrations while also showing support for the product. We absolutely care about this issue and on a personal note it pains me to read comments like yours.

As I mentioned in September Google Cloud Functions is a large product built on a LOT of very complex interconnected systems. Since then work has been done on this issue but I can't share specifics. I know this is extremely frustrating to hear but want to reassure you that this is important to Google and is regularly checked on. It's unlikely there will be a dramatic update anytime soon but it is moving forward.

Has anyone here had luck sorting this out with min instances?

min instances in firebase cloud functions did not solve it for us

we moved all functions that require immediate response to a more traditional web server running in cloud run, and are using traditional rest api calls rather than invoking a cloud function directly