Hello - I'm getting the following error:
`ValueEr...
# prefect-communityr
Riley Hun
08/12/2020, 6:41 PMHello - I'm getting the following error:
ValueError: Task <Task: ThinkNumGetHistoryTask> has retry settings but some upstream dependencies do not have result typesRiley Hun
08/12/2020, 6:43 PMMy flow looks like this:
Copy code
token_task = ThinkNumGetTokenTask(
max_retries=3,
retry_delay=timedelta(seconds=10)
)
history_task = ThinkNumGetHistoryTask(
max_retries=3,
retry_delay=timedelta(seconds=10)
)
with Flow("thinknum-etl") as flow:
token = token_task(
version=version
)
history = history_task(
dataset_id=dataset_id,
version=version,
token=token
)j
josh
08/12/2020, 6:45 PMHi @Riley Hun this is due to the assumption that in order to retry a task the upstream result needs to be persisted somewhere so when it attempts to retry it can grab that data to start the task run. You can read more about results here: https://docs.prefect.io/core/concepts/results.html#persisting-results
I do however also think that that shouldn’t be a ValueError and instead should only raise a warning because retries have the potential to occur in the same process where the data already lives. I’m going to make a PR for that real quick
josh
08/12/2020, 6:51 PMr
Riley Hun
08/12/2020, 6:54 PMThanks @josh! Appreciate it. Looks like I'll have to start persisting my results for my tasks.
But in this example [1], how come we don't have to persist the upstream result?
[1] https://docs.prefect.io/core/examples/retries_with_mapping.html
Riley Hun
08/12/2020, 7:19 PMSo I did something like this:
Copy code
token_task = ThinkNumGetTokenTask(
max_retries=3,
retry_delay=timedelta(seconds=10),
result=GCSResult(bucket='prefect')
)
history_task = ThinkNumGetHistoryTask(
max_retries=3,
retry_delay=timedelta(seconds=10)
)