https://prefect.io logo
Title
m

Matt Denno

10/19/2022, 7:05 PM
One more question: we are trying to run flows in a docker container locally for testing. I see in the docs that: • You must configure remote Storage. Local storage is not supported for Docker. Does this mean that we need to setup remote storage to test running the flows in the container? I currently have something like the following:
deploy_import_1 = Deployment.build_from_flow(
    flow=import_1,
    name="import1",
    work_queue_name="imports",
    schedule=IntervalSchedule(
        interval=timedelta(hours=1),
        anchor_date=datetime(2021, 1, 1, 2, 30, tzinfo=pytz.UTC)
    ),
    infrastructure=DockerContainer(
        image="path_to_image/import-prefect:python3.10",
        image_pull_policy="NEVER",
        auto_remove=False
    )
)
Do I need to include remote storage to test? I am a bit unclear on all the pieces of the puzzle still.
k

Kalise Richmond

10/19/2022, 7:35 PM
Hi Matt, that is correct, you will need to add the storage configuration to your build_from_flow()
m

Matt Denno

10/19/2022, 7:40 PM
Hi @Kalise Richmond - and can you confirm that it can't be a LocalFileSystem instance. Is there an easy way to get this to work for testing that doesn't involve spinning up remote resources?
k

Kalise Richmond

10/19/2022, 7:51 PM
Yes, it can't be local file system because when the process goes to run the flow inside the docker container, it's going to look for that local file path inside the docker container and not on your computer.
m

Matt Denno

10/19/2022, 8:36 PM
Thanks for the clarification. I wonder if it would be possible to use volume mounts to work around that. I am still wrapping my head around all of the Prefect concepts so maybe that it not even a possibility?