IssueTracker

When deploying a custom predictor on ai-platform I get the following error

ERROR: (gcloud.beta.ai-platform.versions.create) Create Version failed. Bad model detected with error:  "Error loading the model"

The logs show

Failure: Could not reach metadata service: Internal Server Error.
ERROR:root:Command '['/tools/google-cloud-sdk/bin/gsutil', '-o', 'GoogleCompute:service_account=default', 'cp', '-R', 'gs://cml-784264224121-1607624275983022/models/testmodel/testversion/1061379786517087871/user_code/*', '/tmp/custom_code']' returned non-zero exit status 1.
ERROR:root:Error loading model: 'generator' object has no attribute 'next'
ERROR:root:Error loading the model

I'm using the following command to deploy

gcloud beta ai-platform versions create testversion --model testmodel --runtime-version 2.2 --python-version 3.7 --origin gs://$BUCKET_NAME  --package-uris=$URI --prediction-class $PCLASS

All the environment variables have the correct values.

The deploy works without a custom prediction on a n1-standard-4 instance in us-central1. The custom predictor is deployed globally on a mls1-c4-m2 but my bucket is still on us-central1. I tried to deploy the custom predictor on us-central1 but n1-standard-4 doesn't support custom predictors and us-central1 doesn't have mls1-c4-m2 instances.

My barebones predictor is as follows

class Predictor(object):
    def __init__(self, model):
        self.model = model

    def predict(self, instances, **kwargs):
        return [0.12345 for i in range(len(instances))]

    @classmethod
    def from_path(cls, model_dir):
        return cls(None)

Any help would be appreciated. I'm struggling to debug the internal error messages.