https://prefect.io logo
s

shijas km

01/21/2022, 11:59 AM
hi I have a question, I am new to prefect , I have configured prefect cloud to run the flow, the flow is created in an aws ec2 and I run the job and scheduled it , now its running on cloud ui my question is the actual execution happens in my prefect cloud or in my aws ec2 ? is there any way i can run my prefect flow with docker in prefect cloud ?
a

Anna Geller

01/21/2022, 12:18 PM
Your flow is running entirely on your infrastructure, in your case, your EC2 instance. You can read more about Prefect’s hybrid execution model here. To run your flows in Docker containers, you can leverage a Docker agent which can be started the same way as a local agent on your EC2 instance.
s

shijas km

01/21/2022, 1:01 PM
Thanks, but when I start with docker agent my flow not running its just shows scheduled only one more question, in my code I am writing some debug messages to my local log file, but when the flow runs in cloud this log file not updating with the python logs, how can I get the python logs jn my local when flow run in cloud. when I run the flow in my local the log messages properly updating
a

Anna Geller

01/21/2022, 1:08 PM
there are several reasons why your flow may be stuck in a scheduled state, but usually this happens when labels not set correctly between the agent and Flow. Did you assign a label to your agent and the same label to your flow in a run configuration? Here is an example:
Copy code
prefect agent docker start --label dockerEC2
and in your flow:
Copy code
from prefect.run_configs import DockerRun

...
flow.run_config = DockerRun(
    labels=["dockerEC2"]   
)
Writing to a local log file is a bit more involved to set up with Docker agent. You would need to: 1. Mount a volume 2. Attach a FileHandler to your Prefect logger - here is an example
4 Views