Yaron Levi
09/03/2022, 9:24 PMRyan Peden
09/04/2022, 12:54 PMfrom prefect import flow
@flow(name="My excellent flow")
def my_etl_flow()
read_data_from_mongo()
write_data_to_snowflake()
if __name__ == '__main__':
my_etl_flow()
Then, when your Render container starts up, tell it to run something like python my_script.py
instead of prefect agent start
. If you set the PREFECT_API_URL
and (if you're using Prefect Cloud) PREFECT_API_KEY
environment variables, the flow will still show up in the Prefect UI the way it does now.
Since this approach wouldn't use a deployment, you'd need to include your flow code into your container image (if I remember our conversation a couple of weeks ago correctly, I think you were going to use S3 storage.)
It's difficult for me to say which approach is best for your situation, but running the flow directly instead of using the Prefect scheduler might be simpler in this case if you're only expecting the agent to start up, run this one flow, and then shut down again.Yaron Levi
09/04/2022, 2:25 PMRyan Peden
09/04/2022, 2:47 PMYaron Levi
09/04/2022, 3:05 PM