https://prefect.io logo
Title
s

Sven Aoki

08/15/2022, 8:22 AM
Hello, has anyone run a separate docker image in a flow? My prefect deployment goes as:
prefect deployment build fx_run.py:fx_rates_flow -n fx_deploy --infra docker-container -sb remote-file-system/minio
and then I also created a custom docker image with s3fs installed which I pull during deployment. However I keep getting the error:
No such file or directory: 'docker': 'docker'
. Can anyone advise?
a

Anna Geller

08/15/2022, 8:36 AM
Can you use S3 instead of minio? Otherwise you may solve it by switching to process infra e.g. --infra process
The problem is docker in docker - you can google this term to read more about problems with that approach
Alternatively, you can bake your flow code into your Docker image itself - we are currently working on that feature, this way you could skip minio and have everything (storage and infra) based on the same Docker image
ETA for this is 1-2 weeks as it's a high priority request from the community
🙌 1
s

Sven Aoki

08/15/2022, 8:54 AM
Thanks for the help. Running it as a local Process requires s3fs to be installed. Is that not required for s3 anymore and already baked into the offical image? Declaring additional EXTRA_PIP_PACKAGES in the env field, won't work as our company cluster is shielded by proxies. If I add those too, the flow isn't executed which I noticed a couple of time, i.e. when I created a separate docker agent in a Dockerfile I had to declare the http_proxy, etc first, install packages and remove the env vars again. I am not sure what is happening under the hood...
a

Anna Geller

08/15/2022, 9:41 AM
yup exactly, no s3fs required when flow code is baked into the image
follow the #announcements - should be released rather soon
🙌 1