https://prefect.io logo
b

Brian Mesick

12/03/2020, 5:05 PM
Hey folks. We’re running into a weird issue where running a Docker stored flow version 0.13.18 (also tried 0.13.13) against a 0.13.13 Kubernetes agent and getting the error:
Copy code
Usage: prefect execute [OPTIONS] COMMAND [ARGS]...
Try 'prefect execute -h' for help.

Error: No such command 'flow-run'.
Looking through the history of the Slack it looks like this was a known issue with 0.13.0 agents, but I’m not seeing any other possible explanations. Any hints?
Apparently we were able to work around it by renaming the flow and ECR repo. ¯\_(ツ)_/¯
d

Dylan

12/03/2020, 5:36 PM
Hi @Brian Mesick, Glad you were able to work around this, even though I am a little perplexed. In general, best practice is to keep your Prefect Agent running on the latest version. Sometimes, if we introduce new functionality, having a Flow’s Prefect version that’s ahead of the Agent’s Prefect version can cause issues
b

Brian Mesick

12/03/2020, 5:39 PM
Interesting, ok. We seem to have a greater-than-usual amount of troubles productionalizing flows. I’m not sure what it is about our process, but pretty much every time we move a flow from local development to run in the agent we run into issues that require us to pull logs off the container before it gets cleaned up. Probably our biggest issue with Prefect at this point.
d

Dylan

12/03/2020, 6:01 PM
Can you describe some of your issues a bit more?
Also, we just rolled out Run Configs, which is our new recommended way to configure Flows to run in Kubernetes. https://docs.prefect.io/orchestration/flow_config/run_configs.html#run-configuration
s

Skip Breidbach

12/03/2020, 6:59 PM
@Dylan this looks great. I've got a couple of questions. 1. For the
KubernetesRun
configuration, the documentation says "Specify an image to use, if not using 
Docker
 storage" - I assume this means that if the flow was created with
storage=Docker(..)
then the image will be picked up there and doesn't need to be specified in the run configuration. 2. Related to #1, any issues with multi-flow images? 3. How does Dask fit in? I imagine for the worker-spec we'd specify that
dask-worker
would run as it does today, but what about the job config? Should that start
dask-scheduler
? I ask because the existing
job.yaml
in the repo runs
'python -c "import prefect; prefect.environments.execution.load_and_run_flow()"'
- would that stay the same?
d

Dylan

12/03/2020, 7:00 PM
1. Correct 👍 2. Nope, if you have an image will all of your Flows, the Storage + Run Config knows how to run the correct Flow 3. You configure Dask through an executor on the Flow object
👍 1
s

Skip Breidbach

12/03/2020, 7:03 PM
Ok! We've been using
DaskKubernetesEnvironment
(marked legacy but it works :-)) so we'd switch over to
DaskExecutor
.
d

Dylan

12/03/2020, 8:22 PM
💯
👍 1
3 Views