Hi community! I am trying to use a monorepo where each folder represents a flow, currently I am using ECS Agent to run the flow on AWS Fargate. I am a little bit lost on the errors I am currently having. I am trying to build a docker image for each folder since it has a flow.py file and multiple others like extract.py, load.py, etc ... that are imported from flow.py and then registering the flow, is this process correct and if not which would be a better solution ?
✅ 1
a
Anna Geller
07/19/2022, 9:15 PM
there is no right or wrong here - your setup is totally valid and the only thing you need to take care of is copying all modules over to the Dockerfile - check out this repo for an example https://github.com/anna-geller/packaging-prefect-flows/
🙌 1
k
Kyle McChesney
07/19/2022, 9:33 PM
Just wanted to add some anecdotal evidence on this pattern. We have a setup very similar to this at my current company and it is working great!
Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.