Infeasible
Status Update
No update yet.
Comments
rb...@google.com
Public facing bug created so the customer can track progress.
zi...@garnercorp.com
We encountered the same issue with our production data. Schema auto-detection currently only scans the first 500 rows of a CSV file.
If the rest of the rows have different kinds of data, the load job fails.
It would be nice to introduce some fallback mechanism, such that when BigQuery detected a different type of data when importing a column, it would switch to a different data type for that column.
If the rest of the rows have different kinds of data, the load job fails.
It would be nice to introduce some fallback mechanism, such that when BigQuery detected a different type of data when importing a column, it would switch to a different data type for that column.
va...@google.com
va...@google.com
Hello,
Could you please confirm whether the issue is still persisting on your end or not?....
va...@google.com
Hello,
Since there is no responese from your side,I will proceed to mark this issue as won't fix(infeasible), In case the issue persists on your side or you want to report a new issue, please do not hesitate to create a new Issue Tracker thread describing your situation.
Description
BigQuery schema auto-detection doesn't include enough data during it's detection process so the generated schemes are incorrect
What you expected to happen:
Enhance BQ schema auto detection to,
(a) Include a larger number of rows (currently 500) in its analysis
(b) If the table is an external table, process as many files as possible (rather than data from one file which is randomly picked).
Other information (workarounds you have tried, documentation consulted, etc):