Option to properly split batch prediction results for Vertex AI [202988557]

Assigned

Feature Request

Status Update

No update yet.

Description

gu...@google.com

created issue #1

Oct 13, 2021 09:14PM

Problem you have encountered:

When performing batch prediction with Vertex AI, results are saved within a destination bucket
resulting files produce inconsistent behavior where the pipeline sometimes concatenates results

I.E GCS bucket contains files file1.txt, file2.txt & file3.txt Based on backend optimization algorithm, results from above files are saved as follows:

File1.txt & File2.txt --> output1.txt File3.txt --> output2.txt

File1.txt --> output1.txt File2.txt & File3.txt --> output2.txt

File1.txt ,File2.txt & File3.txt --> output1.txt

the number of output file is non-deterministic

What you expected to happen:

Input files should have their own output results. Results should not be concatenated:

File1.txt --> output1.txt

File2.txt --> output2.txt

File3.txt --> output3.txt

The above can be achieved by implementing an optional flag to separate results. Default API behavior can remain the same.

Steps to reproduce:

Execute multi file batch prediction:

batch_pred_job = create_batch_prediction_job_sample(
project="PROJECT_ID",
display_name="DISPLAY_NAME",
model_id="MODEL_ID",
instances_format='file-list',
gcs_source_uri="gs://BUCKET/INPUT_DIR/*.txt",
predictions_format='jsonl',
gcs_destination_output_uri_prefix='gs://BUCKET/OUTPUT_DESTINATION_DIR/'
)

Other information (workarounds you have tried, documentation consulted, etc):

Comments

gu...@google.com <gu...@google.com> #2Oct 13, 2021 09:15PM

Application get crashed when Recylerview reach at end(last page). Inside GithubRemoteMediator.kt I have set val endOfPaginationReached = page > 2 to allow maximum 2 pages for testing. So, the problem is when I reach at the last page of list, application get crashed as remoteKeys.nextKey is null(as we have set nextKey=null for last page of list).

Do you have a stack trace for the crash you could share?

Offline cache broken How to support caching when application is open without internet connection. currently it is showing retry button. How to show previously loaded data. I have tried to fix this by removing

This might be due to loadStateFlow / listener logic on the PagingDataAdapter which hides / shows UI elements based on load state. You'll want to modify that logic to only listen to remote errors and not the local ones as well.

Issue 202988557