Feature request - Live migrate GPU instances during host maintenance events [69114248]

Assigned

Feature Request

Status Update

No update yet.

Description

sj...@google.com

created issue #1

Nov 9, 2017 06:51PM

Can you help to ensure uninterrupted run-time of our GPU instances during maintenance by adding or / supporting live migrate feature or option.

Comments

cb...@google.com <cb...@google.com> #2Nov 10, 2017 11:54PM

Assigned to gc...@google.com.

I have forwarded your request to our engineering team. I cannot provide an E.T.A. or guarantee that this feature will be deployed. Nevertheless rest assured that your feedback is always seriously taken.

Any future updates will be posted on this thread. Once again let me thank you for helping us to improve the functionality of our products.

am...@google.com <am...@google.com> #3Nov 7, 2018 08:57PM

Hi,

Any news on this one?

t4...@gmail.com <t4...@gmail.com> #4Dec 27, 2018 03:05PM

Please work on this feature request. I was running another simulation for a
couple days for machine learning and it migrated on me again... I lost all
my info.

On Fri, Dec 21, 2018 at 2:21 PM <buganizer-system@google.com> wrote:

- Show quoted text -

--
All my best,

Journey McDowell
Master's Mechanical Engineering Student
Technical Director FSAE 2016
Formula Student Driverless 2017
+1-805-215-4742
[image:

https://www.norelem.com/xs_db/BILD_DB/website/norelem/www/750/municHMotorsport_Logo.jpg]

kv...@mit.edu <kv...@mit.edu> #5Jan 21, 2020 05:52PM

+1 for this feature request; I'm working with large neural networks and resuming training from a network snapshot can be a significant challenge for some architectures.

Message last modified on Jan 21, 2020 05:53PM

vi...@walmart.com <vi...@walmart.com> #6Jan 27, 2020 06:00AM

+1 for this feature request.

ra...@walmart.com <ra...@walmart.com> #7Feb 5, 2020 03:45AM

+1 for this feature request. We have a lot of Ml and DL workloads which run on GPU VMs.. and they run for a long duration.
VM maintenance can force such workload restarts which is a loss of effort/time and cost.

hi...@google.com <hi...@google.com> #8Sep 24, 2020 10:28PM

Hi team,

Another customer is having issues with the GPU VM maintenance events. Do we have any news on this one?

si...@google.com <si...@google.com> #9Sep 25, 2020 02:50PM

Hey all - I'm a Product Manager on the Compute Engine team. We've been working full-steam on a feature which aims to provide significantly more control and predictability over when maintenance events happen on our instances (including those with GPUs). I don't have any details we can share yet, but when I can I will update this bug.

si...@google.com <si...@google.com> #10Nov 17, 2020 04:59PM

Hi everyone - we have an initial alpha for the feature I mentioned above, which provides significantly more predictability and reduced frequency for maintenance events for GCE GPU VMs. Availability is limited, but if you're interested, please fill out this form and we'll reach out for additional details:

https://forms.gle/BKL3yX4yAV65Pmqn7