Hello Everyone I use `local agent` and try to bind local fol Prefect Community #ask-community

Hello Everyone! I use `local agent` and try to bin...

Ruslan Aliev

09/23/2021, 12:36 PM

Hello Everyone! I use

local agent

and try to bind local folder with folder inside docker container using

volumes

, but looks like it doesn’t work or I do it wrong 👀:

Copy code

container_name = StringFormatter(name="Container name", template="scenario_{scenario_id}_task_{task_id}")
container_id = create_container(image_name=image_name,
                                container_name=container_name(scenario_id=scenario_id, task_id=task_id),
                                # command=command,
                                command=['ls'],
                                volumes=['data:/home/data',
                                         'temp:/home/temp'],
                                ).set_upstream(input_file_path)
started = start_container(container_id=container_id)
status_code = wait_on_container(container_id=container_id, upstream_tasks=[started])
logs = get_container_logs(container_id=container_id, upstream_tasks=[status_code])
log([logs])

The

log([logs])

shows, that the

data

and

temp

folders are missed. What am I missing here?

Kevin Kho

09/23/2021, 2:07 PM

Hey @Ruslan Aliev, I think this will help you. That is for Docker agent but I think the code snippet there might work

👍 1

Kevin Kho

09/23/2021, 2:08 PM

I think you need to pass a host_config as well to the Prefect task

Ruslan Aliev

09/24/2021, 1:02 PM

@Kevin Kho I tried to pass

host_config

prefect.tasks.docker.containers.CreateContainer

, but on run got that error:

TypeError: __init__() got an unexpected keyword argument 'NetworkMode'

But your code snippet works fine - thank you! I don’t know why

host_config

didn’t work correctly with Prefect

CreateContainer

Kevin Kho

09/24/2021, 1:50 PM

Ah that might be Prefect version? What is your version?

Kevin Kho

09/25/2021, 1:57 AM

Could you also share with me your task call?

Ruslan Aliev

09/28/2021, 11:05 AM

@Kevin Kho Hi! Sorry for delay. System apps:

docker -v

Docker version 20.10.8, build 3967b7d

Python packages:

prefect.__version__ = '0.15.5'

docker.__version__ = '5.0.0'

Flow code: https://gist.github.com/RuslanTau/b184386aad252d1a46a7bc3f98c3f36f

Kevin Kho

09/28/2021, 2:12 PM

I think the problem here is that the Prefect task for

CreateContiner

takes in a

dict

for host_config. You actually created a host_config and passed it. That Prefect task will take care of handling it. Can you try:

Copy code

host_config = {"binds": absolute_volumes}

Ruslan Aliev

09/29/2021, 8:49 AM

@Kevin Kho

host_config = {"binds": absolute_volumes}

works fine - thank you! It took me a while to figure out why you hand over the key “binds” instead “Binds”, which I get by create_host_config
method 🙃

👍 1

Kevin Kho

09/29/2021, 1:51 PM

Nice!

2 Views

Open in Slack

Previous Next