Dataproc GKE - Spark Version Upgrade [303822123]

Assigned

Feature Request

Status Update

No update yet.

Description

ma...@flumehealth.com

created issue #1

Oct 6, 2023 06:13PM

It seems that the Dataproc versions for clusters deployed to GCE are updated regularly and their Spark, Hadoop, Hive, etc versions are updated along with those. Is it possible for someone to create new GKE image versions with newer versions of Spark 3.2-3.4 and the corresponding Hadoop and Hive versions? I noticed in the docs it says explicitly not to do this on our own because the images run the risk of failing or being rejected.

Specifically, it would be great if there were matching images for Spark 3.2-3.4 that match the GCE versions listed here:

https://cloud.google.com/dataproc/docs/concepts/versioning/dataproc-release-2.1

Comments

ma...@flumehealth.com <ma...@flumehealth.com> #2Oct 6, 2023 06:15PM

Hi , Could you please clarify the issue description or share any screen shot of the problem where you are facing issue ?

ma...@flumehealth.com <ma...@flumehealth.com> #3Oct 6, 2023 07:27PM

Hi, its a feature request. Can you change it to feature request?
I am requesting a dataproc image version that supports spark 3.4

https://cloud.google.com/dataproc/docs/concepts/versioning/dataproc-release-2.2

spark3.1.png

75 KB

View

Download

pu...@google.com <pu...@google.com> Oct 9, 2023 05:44AM

Assigned to pu...@google.com.

pu...@google.com <pu...@google.com> #4Oct 9, 2023 02:48PM

Reassigned to gc...@google.com.

Hello,

Thank you for reaching out to us with your request.

We have duly noted your feedback and will thoroughly validate it. While we cannot provide an estimated time of implementation or guarantee the fulfillment of the feature request , please be assured that your input is highly valued. Your feedback enables us to enhance our products and services.

Issue 303822123

Description

Issue summary

Comments

ma...@flumehealth.com <ma...@flumehealth.com> #2Oct 6, 2023 06:15PM

ma...@flumehealth.com <ma...@flumehealth.com> #3Oct 6, 2023 07:27PM

pu...@google.com <pu...@google.com> Oct 9, 2023 05:44AM

pu...@google.com <pu...@google.com> #4Oct 9, 2023 02:48PM

Add comment

Issue metadata