Task 'sync_canvas': Exception encountered during task execution!
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/prefect/engine/task_runner.py", line 859, in get_task_run_state
    value = prefect.utilities.executors.run_task_with_timeout(
  File "/usr/local/lib/python3.8/site-packages/prefect/utilities/executors.py", line 445, in run_task_with_timeout
    return task.run(*args, **kwargs)  # type: ignore
  File "<string>", line 28, in sync_canvas
TypeError: 'NoneType' object is not iterable

We are using prefect on EC2 spot instances and we think this is likely caused by AWS dropping our spot instances. Is there a way around this while still using spot instances or is this unavoidable with spot instances?

Due to the nature of spot instances being able to be killed by AWS at any time, I would say it is unavoidable 100%. But what version of PRefect are you running?

prefect==0.15.10

The error is telling me that you have a NoneType that you are trying to iterate over. I found that to be possible if I return a None from a Task/Function whose result I map over. The error comes like that from the executor, not either Task. My workaround was to return [] when I had no data, then it is iterable, and can be evaluated in the mapped Task

Also interestingly enough, rescheduling the flow works, so if there was a way to automate rescheduling then that should solve this.

Is the Task on a Spot instance, or the Flow?

also yes our code returns [] if events is empty and never returns None. Thus why we think it is spot instances problem. I believe both the task and the flow are hosted on a spot instance as that is where we build our prefect agents.

Hmmm if the Task is on its own instance, I think that Prefect might return

None

from the crash when a spot is closed down? If it is within the Flow instance it all should come crashing down and not give you that error. I think. Might need someone to weigh in from Prefect