Kieran
02/25/2021, 6:13 PMBeginning health checks...
System Version check: OK
Traceback (most recent call last):
File "/opt/prefect/healthcheck.py", line 151, in <module>
flows = cloudpickle_deserialization_check(flow_file_paths)
File "/opt/prefect/healthcheck.py", line 44, in cloudpickle_deserialization_check
flows.append(cloudpickle.loads(flow_bytes))
ModuleNotFoundError: No module named 'flows.tasks'
Registering the same flow locally is passing my check.
from prefect.utilities.debug import is_serializable
if is_serializable(flow): ...
My PYTHONPATH is the root of the directory and when SSH'ing into the container I am able to import the "missing" module...
Any pointers?Zanie
if __name__ == "__main__":
block around your flow register/storage building? Sometimes cloud pickle can be finicky about that.Zanie
Kieran
02/25/2021, 7:34 PM@task
def some_function():
return ...
And importing a helper function (which wasn't prefaced with @task
). Moving my helpers inside this @task ovecame the pickle problem. But ideally, I would be able to share helper functions across tasks without having to make them their own respective task?Zanie
pip install
rather than extending your PYTHONPATH
it's generally more reliable.Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.
Powered by