Assigned
Status Update
No update yet.
Comments
va...@google.com
ja...@google.com
This feature request has been forwarded to the Data Fusion engineering team so that they may evaluate it. Note that there are no ETAs or guarantees of implementation for feature requests. All communication regarding this feature request is to be done here.
Description
The User is facing issues like System lag and also with draining their Dataflow job. It is getting stuck in a draining state and they have to forcefully cancel the job. They expect the Dataflow job to process messages from the Pub/Sub subscription in a timely and reliable manner without experiencing significant system lag. The draining process should complete successfully.
Encountered an error: com.google.api.gax.rpc.OutOfRangeException: io.grpc.StatusRuntimeException: OUT_OF_RANGE: Offset 0 is beyond the end of the stream Entity
What you expected to happen:
The below mentioned points need to be mitigated.
Due to this issue user is worried about-
1. Manual Intervention to restart the jobs.
2. Data Loss, due to forced cancellation of jobs.
3. Unpredictability, The intermittent nature of these issues makes it difficult user to plan and schedule their data processing activities.