Run ID in Kubeflow Pipelines on Vertex AI [193880774]

Assigned

Bug

Status Update

No update yet.

Description

se...@google.com

created issue #1

Jul 16, 2021 01:35PM

This will create a public issue which anybody can view and comment on.

Please provide as much information as possible. At least, this should include a description of your issue and steps to reproduce the problem. If possible please provide a summary of what steps or workarounds you have already tried, and any docs or articles you found (un)helpful.

Problem you have encountered:

Hi,

Trying to implement a Kubeflow Pipeline with Vertex AI, I found that the workflow.uid doesn't seem to resolve properly.

Previously in kubeflow, we could use the dsl.RUN_ID_PLACEHOLDER, which resolves to {{workflow.uid}} at compile time, and then to the run ID at run time.

However in my tests it seems that while dsl.RUN_ID_PLACEHOLDER resolves at compile time to {{workflow.uid}}, this does not subsequently resolve to the run ID at run time.

Is this not supported by Vertex Pipelines? I couldn't see anything in the documentation about this kubeflow feature not being compatible with Vertex AI.

What you expected to happen:

How to Access the Run ID using the KFP SDK with Vertex AI Pipelines

Comments

se...@google.com <se...@google.com> #2Jul 16, 2021 01:36PM

Assigned to gc...@google.com.

Hello,

Thank you for reaching out.

This issue was forwarded to our product team. Please keep in mind that this issue has to be analyzed and considered by the product team and I can't provide you ETA for it to be delivered. However, you can keep track of the status by following this thread.

Kind Regards

li...@ml6.eu <li...@ml6.eu> #3Aug 31, 2021 08:11AM

Is there any update on this issue?

se...@google.com <se...@google.com> #4Sep 1, 2021 08:58AM

Reassigned to se...@google.com.

Hello,

Thank you for your patience.

Our product team is currently working on it. Once there will be an update it will be shared in this thread.

Kind Regards

se...@google.com <se...@google.com> Sep 8, 2021 11:26AM

Reassigned to gc...@google.com.

er...@spacewell.com <er...@spacewell.com> #5Dec 22, 2021 09:58AM

Are there any alternative solutions or workarounds that would allow us to access the pipeline run id? This functionality is useful for us to be able to correlate our predictions with the pipeline runs that created them. And that in turn can help analysis/troubleshooting.

Message last modified on Dec 22, 2021 09:59AM

sj...@google.com <sj...@google.com> #6Dec 27, 2021 12:45PM

Assigned to sj...@google.com.

Hello,

Thank you for reaching out. Our internal team is handling the case. Unfortunately there are no updates about it. For now, please stay tune to this thread for updates.

Regards,

sj...@google.com <sj...@google.com> #7Jan 10, 2022 06:19PM

Accepted by sj...@google.com.

Hello,

Quick update about this. Our team mentions that for Vertex Pipelines, we have a different set of placeholders, the currently implemented ones are names and uuids. Here's an example of using them: https://github.com/kubeflow/pipelines/blob/master/sdk/python/kfp/v2/compiler_cli_tests/test_data/pipeline_with_placeholders.py

Let me know if this helps you with your case.

Regards,

sj...@google.com <sj...@google.com> #8Jan 19, 2022 11:36AM

Hello

Quick update, this has been delayed. Please stay tune for further for updates.

Regards

[Deleted User] <[Deleted User]> #9Jan 23, 2022 11:55PM

As a workaround, you can specify a run ID when you invoke the pipeline and then additionally pass that run ID as an input parameter to your pipeline so that you can access it.

sj...@google.com <sj...@google.com> #10Jan 26, 2022 01:17PM

Assigned to gc...@google.com.

Hello

Thank you for providing the workaround, I will forward that to the google team. Please stay tune to this thread for updates.

Regards,

Issue 193880774

Description

Issue summary

Comments

se...@google.com <se...@google.com> #2Jul 16, 2021 01:36PM

li...@ml6.eu <li...@ml6.eu> #3Aug 31, 2021 08:11AM

se...@google.com <se...@google.com> #4Sep 1, 2021 08:58AM

se...@google.com <se...@google.com> Sep 8, 2021 11:26AM

er...@spacewell.com <er...@spacewell.com> #5Dec 22, 2021 09:58AM

sj...@google.com <sj...@google.com> #6Dec 27, 2021 12:45PM

sj...@google.com <sj...@google.com> #7Jan 10, 2022 06:19PM

sj...@google.com <sj...@google.com> #8Jan 19, 2022 11:36AM

[Deleted User] <[Deleted User]> #9Jan 23, 2022 11:55PM

sj...@google.com <sj...@google.com> #10Jan 26, 2022 01:17PM

Add comment

Issue metadata