is there a way to test run a flow locally in Docker somethin Prefect Community #ask-community

is there a way to test-run a flow locally in Docke...

Luke Orland

11/24/2020, 6:02 PM

is there a way to test-run a flow locally in Docker, something like

flow.run(run_on_schedule=False)

or using FlowRunner, without using an agent?

Luke Orland

11/24/2020, 6:05 PM

Copy code

runner = FlowRunner(flow=built_flow)
with raise_on_exception():
    flow_state = runner.run(return_tasks=built_flow.tasks)

ERROR - prefect.TaskRunner | Unexpected error: KeyError('flow_run_id')

Luke Orland

11/24/2020, 6:08 PM

I suppose using the docker agent would have its benefits: • nearly identical to executing in Fargate • quicker startup than Fargate I guess I'm trying to avoid cluttering our Prefect Cloud with flow runs during development.

Spencer

11/24/2020, 6:40 PM

I simply mount my flows into my container and run the flow from inside the container interactively for dev

Luke Orland

11/24/2020, 6:48 PM

Copy code

storage = Docker(...)
storage.add_flow(flow)
storage = storage.build(push=False)
flow.storage = storage

Is that essentially how you mount the flow in the container?

Luke Orland

11/24/2020, 6:49 PM

And then how do you run it interactively, ssh into the container, then execute the

.prefect

file? Or some

docker run ...

command?

Spencer

11/24/2020, 7:38 PM

No,

docker run -v $(pwd):/opt -it <image> /bin/bash

then calling

python <flow_file>.py

. Each flow has a main block to do

flow.run()

(

__name__ == "__main__":

etc)

Luke Orland

11/24/2020, 7:44 PM

Cool, thanks for the advice!

Open in Slack

Previous Next