https://prefect.io logo
Title
j

Jake

07/07/2022, 10:14 PM
We want to be able to turn any function into single-task flows. I’ve created a wrapper to do this (as pictured). Calling
.run()
on the flow that gets returned works fine, and registering the flow seems to be fine too (using k8s run). However, when we try to actually run it, we get the following error:
Failed to load and execute flow run: FlowStorageError('An error occurred while unpickling the flow:\n  ModuleNotFoundError("No module named \'build_index\'")\nThis may be due to a missing Python module in your current environment. Please ensure you have all required flow dependencies installed.')
Am I missing something obvious?
k

Kevin Kho

07/07/2022, 10:20 PM
Do you recognize
build_index
? I don’t know where that is from
j

Jake

07/07/2022, 10:21 PM
That’s used by the function I’m passing into the wrapper
flow = make_into_flow(flow_name, build_index_from_cache
and we have
from directory.build_index import build_index_from_cache
k

Kevin Kho

07/07/2022, 10:24 PM
Ah then that just needs to be available in the execution environment for that it’s understood when unpickled
j

Jake

07/07/2022, 10:26 PM
How do I make that available? It should already exist in the docker image
k

Kevin Kho

07/07/2022, 10:45 PM
It seems like it can’t resolve the import though. Either add it to PYTHONPATH or install it as a library with pip? Have you done that before?