Naga Sravika Bodapati

    Naga Sravika Bodapati

    4 months ago
    Seeing this error: can't pickle CompiledFFI objects , can someone help what might be the issue?
    Anna Geller

    Anna Geller

    4 months ago
    you would need to provide a bit more information 🙂
    is it for Prefect 1.0 or 2.0?
    usually, pickle errors occur due to flow serialization issues, often happens when you serialized your flow with one Python and Prefect version during registration, but then ran it in an environment that had different versions
    Naga Sravika Bodapati

    Naga Sravika Bodapati

    4 months ago
    prefect 1.0
    I am trying to register the flow
    and seeing this error
    in prefect 1.0, is there a way to call one task from another task function?
    Kevin Kho

    Kevin Kho

    4 months ago
    You need to call
    mytask.run()
    but that’s calling the Python function under the hood. It’s not a task with results and retries etc
    Anna Geller

    Anna Geller

    4 months ago
    theoretically yes, you can invoke the .run() method but it's not recommended
    Naga Sravika Bodapati

    Naga Sravika Bodapati

    4 months ago
    ok
    found the issue - i am using storage client for bigquery in one of the tasks and that is throwing this error. Why is it happening?
    can we remove this pickling?
    Kevin Kho

    Kevin Kho

    4 months ago
    What storage are you using? You can store it as a script, but a client instantiated inside a task should be fine for registration because that code is not actually run yet
    Naga Sravika Bodapati

    Naga Sravika Bodapati

    4 months ago
    yes, i am not sure why i am seeing this because i have used similar logic elsewhere and it worked
    GCS bucket
    Kevin Kho

    Kevin Kho

    4 months ago
    You can use the
    stored_as_script
    =
    True
    to not have serialization during the registration time
    You can find more info the in docstrings here
    Naga Sravika Bodapati

    Naga Sravika Bodapati

    4 months ago
    i tried to run the script on a different prefect vm
    and registration went through!
    what must have changed?
    this first vm is on GCP and the second is on premise - will it make a difference?
    Kevin Kho

    Kevin Kho

    4 months ago
    No that shouldn’t 😅.
    Naga Sravika Bodapati

    Naga Sravika Bodapati

    4 months ago
    yes! I am not understanding this! Prefect version on both is the same.