https://prefect.io logo
Title
d

David Steiner Sand

12/21/2022, 6:12 PM
Firstly, thank you all for building and maintaining Prefect, I’ve been using it in production for a while now and it works wonderfully. Similar to the previous question, currently I’m trying to orchestrate a non Python application using Prefect. The ideia is to use the Prefect to periodically run PHP scripts inside Docker containers, however we might also use Prefect for other scripts in the future, written in other languages. I’ve managed to successfully run non-python scripts inside Docker containers through Prefect, but the agents that run these containers does not seem to update the server with the flow run status. Please see this thread to understand what I did.
1
To explain and exemplify my approach, I’ve created a Docker Container Block with the
alpine
image and I’ve edited the
command
to
ls
(This will make the agent run
ls
instead of running the flow). To build and apply the deploy, I created an empty dummy flow (just because it is required by the command), called
flow_1
, and ran:
prefect deployment build flow_1.py:flow_1 -n deploy -q test -ib docker-container/not-python -a
flow_1
:
@flow
def flow_1():
    pass
After running the deploy, the agent pulls the
alpine
image, runs the containers, runs the command
ls
and kills the container, as expected. The only problem is that the agent does not send updates regarding the flow run status to the server, since the flow was not run. So the status of the flow run remains
Pending
forever. Is there a way to update the status in this circumstances? Maybe I could use an image which contains python and edit the
command
of the Docker Container Block to something like
ls && python flow_1.py
(just an example, probably the python command required will be more complex than that) ? This way both
ls
and the flow will run, and the server will be updated with the
Complete
status of the flow run.
Docker Container Block:
k

Kalise Richmond

12/21/2022, 6:34 PM
Hi @David Steiner Sand that's definitely one solution for making sure the dummy flow gets ran and put into a Completed Status. More interesting would be also trying to identify if the process fails and then also setting the state for the flow into a Failed so that you could take action when needed. A couple things you could think about for the different scripts is we do have a prefect-shell collection that you could use to wrap your scripts with. Another idea would be from inside your script, using the prefect client API to then set the state of the flow run that is triggered.
d

David Steiner Sand

12/21/2022, 6:38 PM
Thanks for the quick response and tips @Kalise Richmond, I’ll for sure try them out 🙂