Troy Sankey

04/24/2020, 6:44 PM
We're just starting to build out a re-usable library of prefect tasks, but ran into the fact that we can't actually serialize any of our flows because they refer to tasks defined in separate (common) python files. For my testing, this is led me to copy+paste common code into each flow file (very not-DRY), so I'm wondering if there's clearly something I'm missing? We're using the Docker storage environment.


04/24/2020, 6:52 PM
Hey @Troy Sankey to accomplish this I believe you have to add the files to your Docker storage via the kwarg
Copy code
- files (dict, optional): a dictionary of files to copy into the image when building
and have them in a place where the tasks they contain are importable (i.e. on the PYTHONPATH).

Chris White

04/24/2020, 6:55 PM
Depending on the size and complexity of the files, a completely custom
might also be an option here (this is how people typically install custom non published packages I believe)