Saranya Elumalai
09/01/2020, 7:54 PM@task(max_retries=3, retry_delay=timedelta(seconds=5))
def write_delta_lookup_table():
......
My image build failed for the reason
ValueError: Task <Task: write_delta_lookup_table> has retry settings but some upstream dependencies do not have result types. See <https://docs.prefect.io/core/concepts/results.html> for more details.
Why the task retry require result types. I am not passing values from one task to another. Also https://docs.prefect.io/core/concepts/tasks.html#overview retry example doesnt show the use of result type. What am I missing here. ??
(Any documentation with simple retry usecase will be very helpful)Chris White
09/01/2020, 9:37 PMfrom prefect.engine.results import LocalResult
# when you initialize your Flow
Flow(..., result=LocalResult(), ...)
to your Flow initialization, this check will not failSaranya Elumalai
09/01/2020, 9:44 PMChris White
09/01/2020, 9:48 PMSaranya Elumalai
09/01/2020, 9:49 PMUnexpected error: TypeError("can't pickle SSLContext objects")
Traceback (most recent call last):
File "/usr/local/lib/python3.7/site-packages/prefect/engine/runner.py", line 48, in inner
new_state = method(self, state, *args, **kwargs)
File "/usr/local/lib/python3.7/site-packages/prefect/engine/task_runner.py", line 977, in get_task_run_state
result = self.result.write(value, filename="output", **prefect.context)
Here is my code
with Flow(f"store-etl",result=LocalResult()) as flow:
company = write_delta_lookup_table()
.......
@task(max_retries=3, retry_delay=timedelta(seconds=10))
def get_S3_connection():
What am I missing hereChris White
09/02/2020, 2:25 AMcloudpickle
, else there is no way to persist them to disk. I see that you are creating an S3 client; for situations like this we have a WIP for shareable non-pickelable objects here: https://github.com/PrefectHQ/prefect/pull/3139Saranya Elumalai
09/02/2020, 2:36 AMChris White
09/02/2020, 2:51 AMAnish Chhaparwal
09/30/2020, 5:28 PMChris White
09/30/2020, 5:29 PMSaranya Elumalai
09/30/2020, 5:46 PMAnish Chhaparwal
09/30/2020, 5:48 PM