https://prefect.io
Join Slack
I’m trying to create a Docker container and then run the container - using the task library docker t...
# ask-community
t
I’m trying to create a Docker container and then run the container - using the task library docker tasks - but i think i’m running into an issue
Copy code
[16 Dec 2021 6:03am]: Error during execution of task: DockerException("Error while fetching server API version: ('Connection aborted.', FileNotFoundError(2, 'No such file or directory'))")
is it related to the fact i’m using a Docker agent? or a Docker storage?
this is the step i tried to create
Copy code
create_lichy = CreateContainer(image_name='lichy:1.0', container_name='my_lichy', command='npm run run')
k
I think the issue is trying to run Docker in Docker because the docker agent spins up a container for your Flow and then that Flow in the container is trying to spin another one. You need to mount the Docker sock for this Docker in Docker to work. Not super sure though
upvote 1
t
ya that’s what i thought though im not sure how to do that 😕
looks like it should be something like
Copy code
-v /var/run/docker.sock:/var/run/docker.sock -v /usr/bin/docker:/usr/bin/docker
k
I think you have it but here is an archived issue
This user said they got it with 
-v /var/run/docker.sock:/var/run/docker.sock
on the agent run command I think
t
hmm, ya i was just looking for which docker to put it on, quite a few flying around
i’ll try that 🙂
they say they put it on the 
docker run
though… not sure that’s the agent
k
I saw that but I don’t think you use 
docker run
anywhere when you do this, and the agent takes in the 
volume
argument also
t
hah. it worked
btw, i only saw examples for logging from tasks, is there no way to use the prefect.context in the flow?
k
No because the Flow block is not deferred. Only tasks are. So a logging statement in the Flow block gets executed at registration time
t
ah i see, so i guess i should just create like a 
print_log
task or something if i need to ? (since otherwise i have no way of logging task-library tasks) in any case: 
ListContainers
- for whatever reason, this one behaves really weird, while all its siblings behave fine it forces me to do 
.run()
on the class, otherwise it behaves as if just running the task returns the class or something…
Copy code
create_lichy = CreateContainer(image_name='lichy:1.0', container_name='my_lichy', command='npm run run',log_stdout=True)
run_lichy = StartContainer(log_stdout=True)
wait_on_lichy = WaitOnContainer(log_stdout=True)
list_lichy = ListContainers(all_containers=True, filters={"name":"my_lichy"}, log_stdout=True)
lichy_logs = GetContainerLogs(log_stdout=True)
k
yes or log inside the task
t
Copy code
lichy_list = list_lichy()
    if len(lichy_list) == 0:
        lichy_id = create_lichy()
        print_stuff(f"Lichy container does not exist, created one with id: {lichy_id}")
    else:
        lichy_id = lichy_list[0].get('Id')
        print_stuff(f"Lichy containr already exists, id: {lichy_id}")
    run_lichy(container_id=lichy_id)
    lichy_res = wait_on_lichy(container_id=lichy_id)
    print_stuff(lichy_res)
logging inside the task is impossible when it’s not my task, right?
in any case, first line there ^ if i don’t add 
.run()
it complains:
Copy code
~/Prefect/hb-prefect$ python3 build.py 

Traceback (most recent call last):
  File "build.py", line 2, in <module>
    from flows import xkcd, hello, seg_pred_backfill
  File "/home/ubuntu/Prefect/hb-prefect/flows/seg_pred_backfill.py", line 80, in <module>
    if len(lichy_list) == 0:
TypeError: object of type 'ListContainers' has no len()
k
Ah I see what you mean. Yeah. You can’t use 
if
inside the Flow block because it’s not a task so it’s executed immediately. You need to use the 
case
task instead
t
this error goes away if i do 
.run()
, but --- then i don’t see the task in the cloud UI….?
oh interesting
but why would it not appear in the cloud UI if it actually ran?
k
Because .
run()
forces the run during your registration I think and it becomes registered with whatever the list was as a constant
t
ok that makes sense, interesting that just the fact i was using 
if
seemed to make the IDE complain about the type mismatch 🤔 (prompting me to add the 
.run()
)
3 Views
Open in Slack
PreviousNext
Powered by

Prefect Community

Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.

Powered by