b

    Brett Naul

    2 years ago
    having a weird issue that I haven’t seen anything similar to before: I’m trying to run a
    BigQueryLoadGoogleCloudStorage
    task (which I wrote but that isn’t really helping 😬) and passing in
    schema
    as a list; by the time
    schema
    is used in
    run
    , somehow the order of the list is being altered, causing the columns to be mis-identified. any idea what might be messing with the
    schema
    input to my task? some kind of argument validation or copy maybe…? context: https://github.com/PrefectHQ/prefect/blob/master/src/prefect/tasks/google/bigquery.py#L346-L409
    Jeremiah

    Jeremiah

    2 years ago
    Try wrapping the schema in a
    Constant(schema)
    to force the order to be preserved
    We need to improve introspection of lists-of-constants I think
    b

    Brett Naul

    2 years ago
    yep that fixed it! I can see now why it works the way it does but it can definitely be counterintuitive (and you might not notice right away)
    Jeremiah

    Jeremiah

    2 years ago
    I’ve attempted to improve the situation here https://github.com/PrefectHQ/prefect/pull/1829