<@ULVA73B9P> exploring work pools in ECS and weig...
# ask-marvin
d
@Marvin exploring work pools in ECS and weighing the tradeoffs between push and poll. a few questions: • is push considered experimental? i've noticed some outage concerns in channels so want to understand if it's a "use at your own risk" situation • is there considerable performance differences people have noticed with not having a continuously running worker to poll scheduled events? mostly curious if there's a reasonable run frequency threshold before considering deploying a flow as to poll. • are there any downsides people have seen to using push over poll? for example are there downsides/limitations to retries?
m
Here's what I found regarding your questions about ECS work pools and the push vs. poll tradeoffs: 1. Push Considered Experimental? - There is documentation and ongoing development regarding push work pools, which suggests they are actively being integrated and documented (see GitHub PR). However, there might be some concerns about stability or outages, as you mentioned. It's advisable to monitor community channels and official release notes for any updates on this front. 2. Performance Differences with Polling: - Polling involves having a continuously running worker that checks for tasks to execute. This can lead to increased resource usage compared to push, where tasks are sent to workers as needed. The tradeoff here is between resource usage and responsiveness. If tasks need to be executed frequently, polling might make more sense despite the overhead. The documentation does not provide specific performance benchmarks, so this might require empirical testing in your specific setup. 3. Downsides of Push Over Poll: - The primary concern with push might be related to reliability and the potential for outages, as you noted. Additionally, there could be limitations on retries or error handling, but specific details weren't highlighted in the sources I reviewed. The decision between push and poll should consider both the reliability of the infrastructure and the specific needs of your workflows. For more detailed information, you can explore the ECS Guide in Prefect's documentation and the work pools concept page. If you have specific concerns about your use case, it might be beneficial to engage with the Prefect community or support channels for firsthand experiences and advice.