follow up `prefect-gcp` issue using the `bigquery_insert_stream` task. I’m not able to pass `Timesta...

Bob Colner

05/10/2022, 4:14 PM

follow up

prefect-gcp

issue using the

bigquery_insert_stream

task. I’m not able to pass

Timestamp

data-types -getting:

TypeError: Object of type Timestamp is not JSON serializable

Bob Colner

05/10/2022, 4:16 PM

not a prefect issue -but wondering if there is a good work-around

Bob Colner

05/10/2022, 4:17 PM

I guess I just need to pass the

Timestamp

as an iso formatted string -hope this ends up in bigquery as a Timestamp

Bob Colner

05/10/2022, 4:20 PM

that worked 😉

Andrew Huang

05/10/2022, 4:45 PM

is it similar to the one below with the dataset column as Timestamp type?

Copy code

```python
        from prefect import flow
        from prefect_gcp import GcpCredentials
        from prefect_gcp.bigquery import bigquery_insert_stream
        from google.cloud.bigquery import SchemaField
        @flow
        def example_bigquery_insert_stream_flow():
            gcp_credentials = GcpCredentials(project="project")
            records = [
                {"timestamp": "2022-02-03 10:00", "text": "abc", "bool": True},
            ]
            result = bigquery_insert_stream(
                dataset="integrations",
                table="test_table",
                records=records,
                gcp_credentials=gcp_credentials
            )
            return result

Bob Colner

05/10/2022, 5:00 PM

yes, my BQ schema had a Timestamp col, and I passed a isoformated timestamp string to

biquery_insert_stream

4 Views

Open in Slack

Previous Next

Prefect Community

Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.