https://prefect.io
Join Slack
Hey everyone, I'm encountering an issue with retrying tasks in the Prefect UI. When I try to retry ...
# ask-community
j
Hey everyone, I'm encountering an issue with retrying tasks in the Prefect UI. When I try to retry a task, I'm getting the following error: 
raise MissingResult("The result was not persisted and is no longer available.")
Here are some details: - I'm using the Dask runner. - My task results are stored in S3. - The error occurs when I try to retry a failed task from the UI. Has anyone else experienced a similar issue with the Dask runner and S3 result storage? I'd appreciate any insights or suggestions on how to resolve this problem. Thanks in advance for your help!
c
How are you persisting results, and using them in code?
j
for persisting results I am using S3
i am not sure how to use the result back in code..
Here's an overview of my workflow: 1. I retrieve a list of files to process. 2. I submit this list to Dask for distributed processing. Each file is treated as a separate task. 3. Dask processes each file and stores the output in S3. 4. After processing all the files, I move the output from S3 to my database. Everything seems to be working well, but I'm considering a scenario where one of the files fails during processing. In such cases, I want to be able to retry the entire flow by using the retry button in the Prefect UI. My concern is that when I attempt to retry a failed task in the UI, I encounter the following error: 
raise MissingResult("The result was not persisted and is no longer available.")
c
Hrmm, are the inputs or outputs different between these? I’d have to review results to ensure the configuration is right here, but each result should be unique - if the inputs change (either in total, or in between tasks) then the output results would be different. That said, I’m not sure what might be the case here
d
Ive had this issue a lot myself.
j
@Deceivious do you have any suggestion on this...
d
None. I just rerun when this happens and hope it passes.
j
rerun as in a new flow rather than retry the failed flow
d
Retry....
We usually run 3x the frequency of data we need. ie if we need hourly we run every 20 mins. SO if one fails and the next passes we just ignore 😄
j
ohh ok in my case i have a batch of 27 Million rows
d
There have been multiple listing of such cases in community before. Its hard to replicate so not much to be done.
j
also I am not sure if I am doing right can you quickly eyeball the code and tell me if I am missing something... is that ok..
d
I dont use Dask so not sure if m the right person
j
ohh ok
i am stuck with this issue like 5hr now
d
I personally think this should be treated as a cache miss and should be recomputed? Or atleast allow the dev to select
Try run sequentially and see if that passes
j
i want the dask runner reason being I will have to run this all 27M as soon as possible
so I am using multi-processing along with multi-threading
d
yes just as test is what i mean
j
ohh ok sure let me do that
also noticed this I am passing result_storage_key, but in the bucket the output is a file with random name @Christopher Boyd @Deceivious
d
which versionof preffect?
j
v2.10.17
c
I’m not 100% sure here unfortunately, and might need to test this out myself. The problem as @Deceivious mentioned, is I can’t quite confirm if this is configured correctly and it’s not working as intended, or if it’s not configured correctly - but in that case, I don’t have enough knowledge here to make the recommendation of what to fix just yet
j
@Deceivious @Christopher Boyd thanks for your time
23 Views
Open in Slack
PreviousNext
Powered by

Prefect Community

Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.

Powered by