https://prefect.io logo
Title
b

Ben Fogelson

09/15/2020, 3:17 PM
Is it possible to use
LocalResult
to save results from a flow running in a docker container to the host filesystem? Here’s a toy example of what I’m trying to do:
from prefect.environments.storage import Docker, Local
from prefect.engine.results import LocalResult
from prefect import Flow, task

@task
def foo():
    return "foo"

with Flow('run_local', storage=Local(), result=LocalResult()) as run_local_flow:
    foo()
run_local_flow.register("proj")

with Flow('run_docker', storage=Docker(), result=LocalResult()) as run_docker_flow:
    foo()
run_docker_flow.register("proj")
When I kick off a run of
run_local_flow
, the
foo
task produces a result on my host filesystem (e.g.
/Users/ben.fogelson/.prefect/results/prefect-result-2020-09-15t15-11-21-771486-00-00
) as expected. When I kick off a run of
run_docker_flow
, the UI says that there is a result saved to an analogous location, but there isn’t anything on my host filesystem (I’m guessing it is getting saved to the container filesystem). Being able to run a flow with docker storage but inspect the results locally would be super useful for development.
j

Jim Crist-Harif

09/15/2020, 3:22 PM
Since your flow is running inside a docker container, it can only write to directories inside that docker container. You could mount a local directory as a volume in that container, and then point your
LocalResult
add that directory to save results there. This should be doable with the
--volume
flag for
prefect agent start
.
b

Ben Fogelson

09/15/2020, 3:23 PM
Thanks @Jim Crist-Harif!
c

Chris White

09/15/2020, 3:26 PM
@Marvin archive “How to use LocalResult with Docker Storage?”