Fixed
Status Update
Comments
ab...@google.com <ab...@google.com>
ab...@google.com <ab...@google.com> #2
Hi,
Can you provide the link of the steps by steps that you used in this case? that will allow me to verify the documentation that you have been following
Can you provide the link of the steps by steps that you used in this case? that will allow me to verify the documentation that you have been following
mk...@google.com <mk...@google.com> #3
Thank you for your repro steps - we will look into this issue.
di...@google.com <di...@google.com> #4
Can you elaborate on the benefits of normalizing the URI for your use case?
We don't currently normalize because Dataproc isn't always sure of the proper canonicalization:
"https://www.googleapis.com/compute/.. ." vs "https://compute.googleapis.com/.. ."
"/compute/v1/..." vs "/compute/beta/..."
We don't currently normalize because Dataproc isn't always sure of the proper canonicalization:
"
"/compute/v1/..." vs "/compute/beta/..."
de...@cleardata.com <de...@cleardata.com> #5
We want to validate which image a cluster is running to check for regulatory compliance issues. We can handle both formats, but 1) it feels hacky, and 2) when multiple formats appear for a field, its hard to know if we cover all of the possible cases.
I assumed there was a reason for including the full uri - but really, the relative resource name is sufficient for us (we need to know the project & image name. The host and API version are not material). Since there are two formats, anyone depending on this cannot assume they get full uris, so it should be safe to return the relative resource name in all cases.
Ross
I assumed there was a reason for including the full uri - but really, the relative resource name is sufficient for us (we need to know the project & image name. The host and API version are not material). Since there are two formats, anyone depending on this cannot assume they get full uris, so it should be safe to return the relative resource name in all cases.
Ross
Description
Stock images always seem to return the full resource name. The API should always normalize imageUri so apps can depend on the format being a full resource name.
Steps to reproduce:
1) Create two clusters via the API, one with a full name, the other with a relative name for the custom image. (I'm using python with the discovery API here):
>>> from googleapiclienthelpers.discovery import build_subresource
>>> dataproc = build_subresource('dataproc.projects.regions.clusters', 'v1beta2')
>>> dataproc.create(projectId='cleardata-lab-krv', region='us-central1', body={'clusterName': 'api-relative-name', 'config': {'masterConfig': {'imageUri': 'projects/cleardata-images/global/images/cleardataproc-debian9-1-4-10-20190804'}}}).execute()
{'name': 'projects/cleardata-lab-krv/regions/us-central1/operations/31d2b055-c099-4c58-9126-93dd05562df0',
'metadata': {'@type': '
'clusterName': 'api-relative-name',
'clusterUuid': 'b0e9ebf2-e385-41a3-acef-2b5b5ff16b12',
'status': {'state': 'PENDING',
'innerState': 'PENDING',
'stateStartTime': '2019-08-20T21:55:57.801Z'},
'operationType': 'CREATE',
'description': 'Create cluster with 2 workers',
'warnings': ['For PD-Standard without local SSDs, we strongly recommend provisioning 1TB or larger to ensure consistently high I/O performance. See
>>> dataproc.create(projectId='cleardata-lab-krv', region='us-central1', body={'clusterName': 'api-full-name', 'config': {'masterConfig': {'imageUri': '
{'name': 'projects/cleardata-lab-krv/regions/us-central1/operations/95890614-cf09-4d03-ab3c-55a6fea36a1c',
'metadata': {'@type': '
'clusterName': 'api-full-name',
'clusterUuid': '7f0a6cb3-87fd-4a1c-a39b-b8af4b060dcf',
'status': {'state': 'PENDING',
'innerState': 'PENDING',
'stateStartTime': '2019-08-20T21:56:23.205Z'},
'operationType': 'CREATE',
'description': 'Create cluster with 2 workers',
'warnings': ['For PD-Standard without local SSDs, we strongly recommend provisioning 1TB or larger to ensure consistently high I/O performance. See
2) wait for all clusters to complete creating.
3) Describe them and note the different formats for imageUri:
$ gcloud dataproc clusters describe api-relative-name --region=us-central1 | grep imageUri
imageUri: projects/cleardata-images/global/images/cleardataproc-debian9-1-4-10-20190804
imageUri: projects/cleardata-images/global/images/cleardataproc-debian9-1-4-10-20190804
$ gcloud dataproc clusters describe api-full-name --region=us-central1 | grep imageUri
imageUri:
imageUri:
Thanks,
Ross