IssueTracker

Symptoms:

GKE clusters using version 1.16 or later may fail to autoscale workloads that depend on Horizontal Pod Autoscaling using external metrics.
This is due to cluster role "external-metrics-reader" missing from the cluster configuration, preventing the Horizontal Pod Autoscaling (HPA) agent from accessing the cluster.
The workaround is to manually recreate role external-metrics-reader in the affected cluster.

Workaround:

Manually recreate role external-metrics-reader in the affected cluster. Users with workloads that depend on HPA using external metrics can also avoid manually upgrading to 1.16.

Workaround example:

To enable external metrics to hpa, the missing cluster role should be added.
The cluster role yaml is as following:
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: external-metrics-reader
rules:
- apiGroups:
  - "external.metrics.k8s.io"
  resources:
  - "*"
  verbs:
  - list
  - get
  - watch

This cluster role can be applied using kubectl:
kubectl apply -f ./external_rbac.yaml
Where ./external_rbac.yaml contains the cluster role provided above.

Our engineering team has determined the root cause of the issue and is developing a fix. There is no ETA for the rollout at this stage. Further updates will be provided here.