Duplicate

of 251880583

Bug

Status Update

No update yet.

Description

si...@openlabsusa.org

created issue #1

Jul 29, 2022 05:50PM

Problem you have encountered:

When I run a DataFlow job with the option --enable_google_cloud_profiler (and also with these experiments: ['use_runner_v2', 'use_unified_worker', 'beam_fn_api', 'use_multiple_sdk_containers']), the python SDK raises this warning:

Unable to start google cloud profiler due to error: Unable to find the job id or job name from envvar.

Looking into this futher, I found that the Go code in the Container SDK logs the following:

Provision info:

... metadata:{key:"job_id" value:"2022-07-27_14_10_37-13912882228797168880"} metadata:{key:"job_name" value:""}

So it looks like the provisioning API is returning an empty job_name, causing boot.go to set the JOB_NAME env var to "", which causes the Python SDK code to fail when trying to activate googlecloudprofiler.

What you expected to happen:

sdk_worker_main.py should successfully find a non-falsy JOB_NAME and JOB_ID from the env vars and should log "'Turning on Google Cloud Profiler.'" and the results should show up in the profiler output.

Steps to reproduce:

We're using a custom image that includes the following lines, to ensure the image works with Dataflow:

COPY --from=apache/beam_python3.8_sdk:2.38.0 /opt/apache/beam /opt/apache/beam
# Set the entrypoint to Apache Beam SDK launcher.
ENTRYPOINT ["/opt/apache/beam/boot"]

Kicking off the DataFlow job with code that looks like

beam_options = PipelineOptions(
   ['--runner', 'DataflowRunner',
    '--job_name', 'my_job',
    '--experiments', 'use_runner_v2',
    '--sdk_container_image', my_image]
)
with beam.Pipeline(options=beam_options) as pipeline:
    # run the pipeline

Using the python library apache_beam[gcp]==2.38.0

Other information (workarounds you have tried, documentation consulted, etc):

Comments

bu...@gmail.com <bu...@gmail.com> #2Oct 3, 2022 09:31AM

datastore-core and datastore-preferences have dependencies on android for SharedPreferencesMigrations and convenience constructors. We should extract these dependencies so users can use this in kotlin-only parts of their code.

kl...@google.com <kl...@google.com> #3Oct 27, 2022 02:22AM

Status: Duplicate of

251880583

Would this enable Kotlin Multiplatform usage? Or just Kotlin/JVM in non-Android settings?

[Deleted User] <[Deleted User]> #4Nov 15, 2022 04:21PM

This work is intended to support kotlin-only modules in android projects using Kotlin/JVM. We haven't tested this in non-android settings and haven't built it using Kotlin Multiplatform.

bu...@gmail.com <bu...@gmail.com> #5Nov 16, 2022 05:50AM

Project: platform/frameworks/support
Branch: androidx-master-dev

commit c9bbc6f26740be8b2ca486032238442e1138c914
Author: rohitsat13 <rohitsat@google.com>
Date: Tue Sep 29 09:36:28 2020

Split out datastore and datastore-core modules

Test: Ran all datastore tests
Bug: 168512698
Relnote: Created a pure kotlin dependency for datastore to allow for faster compilation
Change-Id: I42d75d0f66dc0cac19bd0f452a84b58806a65a46

M buildSrc/src/main/kotlin/androidx/build/PublishDocsRules.kt
A datastore/datastore-core/api/1.0.0-alpha01.txt
M datastore/datastore-core/api/current.txt
A datastore/datastore-core/api/public_plus_experimental_1.0.0-alpha01.txt
M datastore/datastore-core/api/public_plus_experimental_current.txt
A datastore/datastore-core/api/restricted_1.0.0-alpha01.txt
M datastore/datastore-core/api/restricted_current.txt
M datastore/datastore-core/build.gradle
D datastore/datastore-core/lint-baseline.xml
M datastore/datastore-core/src/main/java/androidx/datastore/core/CorruptionHandler.kt
M datastore/datastore-core/src/main/java/androidx/datastore/core/DataMigration.kt
M datastore/datastore-core/src/main/java/androidx/datastore/core/DataMigrationInitializer.kt
M datastore/datastore-core/src/main/java/androidx/datastore/core/DataStore.kt
M datastore/datastore-core/src/main/java/androidx/datastore/core/DataStoreFactory.kt
M datastore/datastore-core/src/main/java/androidx/datastore/core/InitializerApi.kt
M datastore/datastore-core/src/main/java/androidx/datastore/core/Serializer.kt
M datastore/datastore-core/src/main/java/androidx/datastore/core/SingleProcessDataStore.kt
M datastore/datastore-core/src/main/java/androidx/datastore/core/handlers/NoOpCorruptionHandler.kt
M datastore/datastore-core/src/main/java/androidx/datastore/core/handlers/ReplaceFileCorruptionHandler.kt
M datastore/datastore-core/src/test/java/androidx/datastore/core/DataMigrationInitializerTest.kt
M datastore/datastore-core/src/test/java/androidx/datastore/core/DataStoreFactoryTest.kt
M datastore/datastore-core/src/test/java/androidx/datastore/core/SingleProcessDataStoreTest.kt
M datastore/datastore-core/src/test/java/androidx/datastore/core/TestingSerializer.kt
M datastore/datastore-core/src/test/java/androidx/datastore/core/handlers/ReplaceFileCorruptionHandlerTest.kt
M datastore/datastore-preferences/api/api_lint.ignore
M datastore/datastore-preferences/api/current.txt
M datastore/datastore-preferences/api/public_plus_experimental_current.txt
M datastore/datastore-preferences/api/restricted_current.txt
M datastore/datastore-preferences/build.gradle
M datastore/datastore-preferences/datastore-preferences-proto/src/main/java/androidx/datastore/preferences/PreferencesMapCompat.kt
M datastore/datastore-preferences/src/androidTest/java/androidx/datastore/preferences/PreferenceDataStoreFactoryTest.kt
M datastore/datastore-preferences/src/androidTest/java/androidx/datastore/preferences/SharedPreferencesToPreferencesTest.kt
M datastore/datastore-preferences/src/main/java/androidx/datastore/preferences/PreferenceDataStoreFactory.kt
M datastore/datastore-preferences/src/main/java/androidx/datastore/preferences/Preferences.kt
M datastore/datastore-preferences/src/main/java/androidx/datastore/preferences/PreferencesSerializer.kt
M datastore/datastore-preferences/src/test/java/androidx/datastore/preferences/PreferencesSerializerTest.kt
M datastore/datastore-proto/src/main/java/androidx/datastore/protos/ProtoSerializer.kt
M datastore/datastore-proto/src/test/java/androidx/datastore/protos/ProtoSerializerTest.kt
M datastore/datastore-sampleapp/build.gradle
M datastore/datastore-sampleapp/src/main/java/com/example/datastoresampleapp/PreferencesDataStoreActivity.kt
M datastore/datastore-sampleapp/src/main/java/com/example/datastoresampleapp/ProtoDataStoreActivity.kt
M datastore/datastore-sampleapp/src/main/java/com/example/datastoresampleapp/SettingsFragment.kt
M datastore/datastore/api/api_lint.ignore
A datastore/datastore/api/current.txt
A datastore/datastore/api/public_plus_experimental_current.txt
A datastore/datastore/api/res-current.txt
A datastore/datastore/api/restricted_current.txt
A datastore/datastore/build.gradle
M datastore/datastore/src/androidTest/AndroidManifest.xml
A datastore/datastore/src/androidTest/java/androidx/datastore/DataStoreFactoryTest.kt
M datastore/datastore/src/androidTest/java/androidx/datastore/TestingSerializer.kt
M datastore/datastore/src/androidTest/java/androidx/datastore/migrations/SharedPreferencesMigrationTest.kt
M datastore/datastore/src/main/AndroidManifest.xml
A datastore/datastore/src/main/java/androidx/datastore/DataStoreFactory.kt
M datastore/datastore/src/main/java/androidx/datastore/migrations/SharedPreferencesMigration.kt
M settings.gradle

https://android-review.googlesource.com/1441928

IssueTracker

enable_google_cloud_profiler doesn't work with Beam in Python

Status Update

Description

Problem you have encountered:

What you expected to happen:

Steps to reproduce:

Other information (workarounds you have tried, documentation consulted, etc):

Comments

bu...@gmail.com <bu...@gmail.com> #2Oct 3, 2022 09:31AM

kl...@google.com <kl...@google.com> #3Oct 27, 2022 02:22AM

[Deleted User] <[Deleted User]> #4Nov 15, 2022 04:21PM

bu...@gmail.com <bu...@gmail.com> #5Nov 16, 2022 05:50AM

Issue 240585931

Description

Problem you have encountered:

What you expected to happen:

Steps to reproduce:

Other information (workarounds you have tried, documentation consulted, etc):

Issue summary

Comments

bu...@gmail.com <bu...@gmail.com> #2Oct 3, 2022 09:31AM

kl...@google.com <kl...@google.com> #3Oct 27, 2022 02:22AM

[Deleted User] <[Deleted User]> #4Nov 15, 2022 04:21PM

bu...@gmail.com <bu...@gmail.com> #5Nov 16, 2022 05:50AM

Add comment

Issue metadata