Tim Helfensdörfer
09/27/2022, 4:40 PM[]
) in my yaml file the command points to my prefect entry script but it needs the UUID of the flow run. So it boils down to two questions:
1. How does the queue worker provide the UUID into the ecs task
2. What does Command
do?
(in the second screenshot I changed the task definition command from [] to python3 /app/tasks/run.py
- but it still needs the flow run uuid)ECSTask(command=["echo", "hello world"], task_definition_arn="arn:aws:ecs:...")
Yes, haha hello world, but doesn't it more sense to show an actual example on how to run an actual worker? Isn't this the purpose of prefect?Ryan Peden
09/27/2022, 5:51 PMECSTask
sets that before it starts the container. The normal process looks like:
• The container starts and runs the command provided by the deployment (the default is python -m prefect.engine
)
• prefect.engine
runs, reads the flow ID from the env variable, pulls details about the deployment, downloads the code from whichever external storage you specifed in the deployment, then runs itTim Helfensdörfer
09/27/2022, 6:41 PM-m prefect.engine
is because I need to override ignore_storage:
flow = await load_flow_from_flow_run(
flow_run, client=client, ignore_storage=True
)
But the other parts are just copy pasted for now. At the beginning of the script i just dumped all env variables:
if __name__ == "__main__":
# Standard Library
import os
import sys
print(os.environ.items())
Besides my own env variables which i set through a s3 arn resource in the task definition and a few default docker env variables, no prefect variables are set.Khuyen Tran
09/28/2022, 4:13 PM[]
, which overwrites the default command to start an agent.
We raised the issue here