haris khan
10/26/2024, 11:13 PMMarvin
10/26/2024, 11:14 PMDaskTaskRunner
in the Prefect 3.x documentation or the open GitHub issues. This kind of behavior might not be typical, but there are a few areas you might want to verify:
1. Check for Retries: Ensure that your task or flow isn't configured with retries. If a task is configured to retry on completion or failure, it might appear as though it is restarting.
2. Loop Configuration: Double-check how tasks are being submitted in your loop. Ensure that the logic to submit tasks doesn't accidentally resubmit completed tasks.
3. Using Dask Properly: Make sure you’re using DaskTaskRunner
and dask
functionalities correctly. Here's a basic reference from the Prefect 3.x documentation for integrating Dask: Prefect Dask.
If you still face issues, you might consider reaching out to the Prefect community on their Slack or GitHub for more targeted help. If you need more specifics on the Dask setup or a code review, I would be happy to help with that too!