IssueTracker

This will create a public issue which anybody can view and comment on.

Please provide as much information as possible. At least, this should include a description of your issue and steps to reproduce the problem. If possible please provide a summary of what steps or workarounds you have already tried, and any docs or articles you found (un)helpful.

Problem you have encountered:

I'm attempting to use a Pub/Sub BigQuery Subscription with the use_table_schema feature to write a BIGNUMERIC field. Writing 7 and "7" both failed. Instead I had to write the Base64 encoding of the BigDecimalByteStringEncoder encoding of a BigDecimal: AAAAAMDvxkJWX695mhifDgI= to get 7 to show up in the table.

What you expected to happen: If it's expected that a certain datatype should be written in a certain way, I would have expected the documentation to call that out (it is for the pubsub schema doc). Some data types are explicitly called out (DATE, TIME, TIMESTAMP, and DATETIME are all called out here: https://cloud.google.com/pubsub/docs/create-bigquery-subscription#use-table-schema as needing to be integers with a specific meaning. However BIGNUMERIC and likely others are not. Other mechanisms of loading data would have accepted 7 or "7" and thus would be the first things a user would try.

Ultimately, I'd expect that data formats would be explicitly documented so that users can appropriately translate their messages.

Steps to reproduce:

Create a table with a BIGNUMERIC column:

CREATE TABLE test.bignumeric (
  id STRING,
  big_numeric BIGNUMERIC,
  PRIMARY KEY (id) NOT ENFORCED
)
CLUSTER BY id

Create a PubSub Topic with a BigQuery (use_table_schema) Subscription to write to this table.

Publish:

{
    "id": "id",
    "big_numeric": 7,
    "_CHANGE_TYPE": "UPSERT",
    "_CHANGE_SEQUENCE_NUMBER": 0
}
OR
{
    "id": "id",
    "big_numeric": "7",
    "_CHANGE_TYPE": "UPSERT",
    "_CHANGE_SEQUENCE_NUMBER": 0
}

And observe that neither work (this might be easier to observe if you attach a deadletter topic with a default subscription to the BigQuery subscription).

Next publish:

{
    "id": "id",
    "big_numeric": "AAAAAMDvxkJWX695mhifDgI=",
    "_CHANGE_TYPE": "UPSERT",
    "_CHANGE_SEQUENCE_NUMBER": 0
}

And observe that a row is entered.

Other information (workarounds you have tried, documentation consulted, etc): To determine how to format a BIGNUMERIC I tried the following doc: https://cloud.google.com/pubsub/docs/create-bigquery-subscription#use-table-schema

This doc explicitly calls out date/time/datetime/timestamp formats, but nothing else.