+1 Please add this feature to tall kinds of load balancers.

Thanks for the valuable input! Here's a quick update. I'd encourage everyone to focus on what feature they need rather than on an implementation. Though "firewalls at the forwarding rule" sounds simple, it's very complex to build. Hopefully the following provides some background as to why and shows you how to accomplish what you need today.

If we focus on the pass-through GCP load balancers – internal TCP/UDP and network load balancing, this feature is already available. The firewall rules you create – whether VPC firewalls or hierarchical firewall policies – are applied right in-front of the NIC for the VM in the packet coprocessor on the physical system running your VM.

For the pass-through load balancers, the forwarding rule is merely configuration telling us how to route traffic within Andromeda for internal TCP/UDP load balancing or in Maglev for external network load balancing. Maglevs direct traffic to the external IP addresses of GCP VMs, network load balancers, external protocol forwarding, Cloud VPN gateways, etc. We apply firewall rules as close to the VM as possible so that your VM can make use of the same shared infrastructure that others use – your firewall rules are almost certainly not the same as another customer's firewall rules.

If we focus on external proxy load balancers – external HTTP(S), SSL Proxy, and TCP Proxy – those are all powered by Google Front End (GFE) proxies. Enforcing a desired set of deny-list IP addresses at the GFE is desirable. The configuration described in the forwarding rule directs traffic to Maglev then onto GFEs, but the forwarding rule isn't a single device, router, or proxy dedicated to the load balancer. Once again, each customer's deny- and allow-lists are unique, and both Maglev and GFE systems are shared systems. For example, an external HTTP(S) load balancer does not get a dedicated GFE – instead, your external HTTP(S) load balancer is implemented by all serving GFEs worldwide (always when you use Premium Tier, and less likely, though still possible in certain situations, when you use Standard Tier).

For those reasons and those proxy load balancers, Cloud Armor is our path forward because it lets you create a per-load-balancer configuration on our shared GFE fleet. You can configure both L4 and "next generation" (L5+) "firewall" settings. We're constantly updating the Cloud Armor features. For example, I just created a single Cloud Armor security policy blocking over 100 CIDRs. My policy references ten rules and each rule has ten CIDRs in a deny-list. I can reference this policy on multiple backend services on one or more external HTTP(S) load balancers. So my single policy denies from 100 CIDRs, and it can apply to as many external HTTP(S) load balancers as I configure. One 100 CIDR deny list, multiple load balancers.

Another helpful point when discussing IP address deny-lists is to consider the purpose of the deny list. Talking about GFE-based proxy load balancers first, you get a certain amount of automatic DDoS protection from just using Google infrastructure. For pass-through external network load balancing, note that each VM in the load balancer can only receive 1,800,000 packets per second or 20Gb/s, whichever is reached first. My point here is that the need to play whack-a-mole with deny lists might be diminished by leveraging some of the other capabilities we have, including some that you get automatically.

Does that help? If anyone has specific use-cases, please let us know here.

Following up on comment #129:

Thank you for the specific use case – that's very clearly worded. You're correct that we currently don't offer IP address allow/deny controls for GCP internal HTTP(S) load balancers. (We do offer that kind of control using Cloud Armor for GCP external HTTP(S) load balancers.)

Here are my recommendations for regulated workloads:

• The load balancer doesn't have to be completely unprotected even if any system in your VPC network (or on-premises network connected to your VPC network) can send packets to its IP address. To be clear, "its IP address" is the IP address of the load balancer's forwarding rule, as implemented by Google-managed Envoy proxies [*]. The load balancer's backends – your VMs – could verify that load-balanced requests they receive are legitimate. The standard way to do this is to use HTTP Authorization headers. In addition to using HTTPS transport, this model focuses on which systems are authorized to send requests to your backends, rather than looking at what IP addresses the requesting systems might use.

  This option is achievable with our current internal HTTP(S) load balancer offering, but it requires that the load balancer's backends – your VMs – parse the Authorization headers that a client sends the load balancer. Our internal HTTP(S) load balancers preserve these headers when they make requests to the backends.

• Another approach in the same spirit of access control is to use mutual TLS (mTLS, or client certificate based authentication). Instead of passing HTTP Authorization headers to the load balancer's backends, a client establishes a TLS connection with either a proxy or a backend. In terms of our current offerings, mTLS isn't currently available for our internal HTTP(S) load balancers; however, you can configure a GCP internal (pass-through) TCP load balancer to distribute traffic to backends which perform mTLS. In other words, instead of a proxy load balancer being the TLS server, you can use a pass-through load balancer whose backends are TLS servers. This is how service mesh systems like Istio are implemented in GKE Kubernetes. Services outside a cluster authenticate to services in a cluster using mTLS, and the Istio ingress "gateway" can be an internal TCP load balancer. TLS termination happens on the Envoy proxy Pods running on VMs, and those VMs can be backends of a pass-through internal TCP load balancer.

• Whether you can use mTLS or not, it's worth considering an internal TCP load balancer for your current needs because you can control the client IP addresses which are permitted to establish TCP connections "to the load balancer" using firewall rules applicable to the load balancer's backend VMs. An internal TCP load balancer is implemented using routing in our software-defined VPC network – there's no proxy or device; it's just all our SDN. With pass-through GCP load balancers, you can control client source IP addresses using GCP firewall rules (or even hierarchical firewall policies).

An overview for internal TCP load balancing is here: https://cloud.google.com/load-balancing/docs/internal/

__

[*] One thing to keep in mind about the proxy systems for internal HTTP(S) load balancers: These proxies are managed by Google, and various components in our control plane can connect to them, in order to stop the proxy, start the proxy, and to do rolling updates. (Keeping the Envoy proxies current is important from a security perspective!) So there are control systems "outside of your VPC network" which can "send packets to the load balancer" according to certain interpretations.

+1

Have you considered using IP based access levels with IAM conditions using IAP on the load balancer?

Here is how this works.

You enable IAP on the BackendServices that you need protected. https://cloud.google.com/iap/docs/tutorial-gce The org admin defines access levels based on IP addresses: https://cloud.google.com/access-context-manager/docs/overview#ip-address IAP permissions are assigned using IAM. These permissions can have access level conditions. https://cloud.google.com/iam/docs/conditions-overview#request_attributes