IssueTracker

To solve this problem, create the two routes with different priorities, like this:

Route1: Destination:0.0.0.0/0<-------> Next hop: ILB(IN us-east1) <------> Priority:950

Route2: Destination:0.0.0.0/0<-------> Next Hop: ILB(IN us-west1) <------> Priority:951

As long as the forwarding rules aren't configured to be globally accessible, the next hops are regional. That is:

From the perspective of us-west1, this route is available:

Route2: Destination:0.0.0.0/0<-------> Next Hop: ILB(IN us-west1) <------> Priority:951

From the perspective of us-east1, this one is available:

Route1: Destination:0.0.0.0/0<-------> Next hop: ILB(IN us-east1) <------> Priority:950

It doesn't matter that these have different priorities in the route table because only one is "active" in each region (as long as global access isn't configured).

For comment #1, we really should be troubleshooting this via a Support ticket because "an account that lacks organization level privileges" could mean several things, and the specifics do matter.

In general, if there does not exist an IAM permission binding for an IAM principal where both the following would be true:

• The IAM permission binding would be in the IAM policy of the folder or organization where the hierarchical firewall policy is to be created.
• The IAM permission binding would grant the role of compute.orgFirewallPolicyAdmin (or a custom role that at least contains the compute.firewallPolicies.create permission) to the IAM principal.

...Then the IAM principal should receive an Unauthorized error when attempting to create a hierarchical firewall policy on the folder or organization. The IAM permissions check does happen before resource creation.

There is almost certainly more going on here, and we should troubleshoot this in a Support ticket. Please open one for help.

I was able to reproduce the issue. Looking also at the original report the error is:

 Error: Error creating FirewallPolicy: googleapi: Error 403: Required 'compute.globalOperations.get' permission for 'locations/global/operations/org-xxxxxxxxxx-xxxxxxxxx-xxxxx-xxxxxxx-xxxxxx', forbidden
│ 
│   with google_compute_firewall_policy.default,
│   on resource.tf line 1, in resource "google_compute_firewall_policy" "default":
│    1: resource "google_compute_firewall_policy" "default" {
│ 
╵

So the problem is, that in specific situation (i.e. having roles/compute.orgFirewallPolicyAdmin given on a folder level) - user has necessary permissions to create a policy, hence creations succeeds, but lacks permissions to check the status of the operation (compute.globalOperations.get) - so the command fails, as the end result is unknown.

compute.globalOperations.get is part of roles/compute.orgFirewallPolicyAdmin, but only after compute.globalOperations.get is granted on the org level, the problem disappears.

Since 297898006 is an issue visible only to Googlers, and since this issue is publicly visible, it's helpful to re-open this so our shared customers and colleagues outside of Google know what's going on. Thank you, Wiktor and John, for bringing clarity to the issue. I wasn't quite sure what comment #1 was referring to, but I now have a clear idea. With that in mind, I can formalize the problem statement of comment #1 and explain what's happening.

Problem statement

1. Suppose an IAM principal (user, group, service account) has been granted the compute.orgFirewallPolicyAdmin role via binding in the IAM policy of an example folder. Further, suppose that the same IAM principal does not have any bindings which grant it the compute.globalOperations.get permission in the IAM policy of the parent organization.
   • To dispel confusion, if the IAM principal does have a binding which grants it the compute.globalOperations.get permission in the IAM policy of the parent organization, there is no problem.
2. Suppose that the IAM principal attempts to create a hierarchical firewall policy at the same level of the example folder, using gcloud or the Terraform module. An example gcloud command is: gcloud compute firewall-policies create --folder=FOLDER_NUMBER --short-name=POLICY_NAME
3. Because the IAM principal has been granted the compute.orgFirewallPolicyAdmin to the example folder, and because the compute.orgFirewallPolicyAdmin role includes the compute.firewallPolicies.create permission, the gcloud command from (2) – or its equivalent by using Terraform – does successfully create the firewall policy.
   • To dispel confusion, the newly-created hierarchical firewall policy is not "orphaned" – it's fully functional, and can be deleted if desired.
   • However, if the IAM principal does not have any bindings which grant it the compute.globalOperations.get permission in the IAM policy of the parent organization, there's no way for the IAM principal to interrogate the operation for its status. This leads to an unexpected HTTP 403 Unauthorized error, so unless the IAM principal issues a subsequent command to list hierarchical firewall policies, that principal is likely unaware that the hierarchical firewall policy has been created!

An astute observer would note that the compute.globalOperations.get permission is included in the compute.orgFirewallPolicyAdmin role. The issue in play here is whether the IAM principal has been granted the compute.globalOperations.get permission in the IAM policy of the organization itself. Because creation operations for hierarchical firewall policies are currently scoped to the whole organization, the IAM principal having the compute.globalOperations.get permission on the folder where the hierarchical firewall policy is created is irrelevant.

Next steps

We're discussing this issue internally, thinking about ways we can remove or better explain this unexpected HTTP 403 error. Even though the HTTP error is unexpected, the hierarchical firewall policy is only created if the IAM principal has the permission to do so.

We'll use the internal 297898006 issue as the parent of this issue.