https://prefect.io
Join Slack
How do I deal with the following error? `[2021-07-29 12:41:09-0500] INFO - prefect.LocalDaskExecutor...
# ask-community
t
How do I deal with the following error? 
[2021-07-29 12:41:09-0500] INFO - prefect.LocalDaskExecutor | Attempting to interrupt and cancel all running tasks...
This is a long running Flow with a lot of mapped tasks (about 83k) and it mostly fails with this error.
k
Hey @Tim Enders, could you run the Flow with debug level logs and see if that yields anything? Does the Flow work for a sample of the 83k?
t
How do I run the flow with Debug level logs?
k
You can pass it in the RunConfig with 
RunConfig(env={PREFECT__LOGGING__LEVEL: "DEBUG"})
or you can set the env var on the agent.
t
I am running this locall with 
flow.run()
can I pass it in as a parameter?
k
You need to do
Copy code
import os
os.environ['PREFECT__LOGGING__LEVEL'] = "DEBUG"
before importing prefect. Doesn’t seem like it can be done through parameters
t
ok
Thanks, I will get back to you after I have more information.
đź‘Ť 1
There is nothing aroun that message in the logs. but it does run straight into an exception. I am running two maps, and then attempting a flattne, but the 
flatten_jeson
task here doesn't seem to get data, which I think is because the 
flatten
call fails.
Copy code
details = fetch_environment_details.map(unmapped(client), proj_envs)
    real_details = really_get_the_details.map(unmapped(client), flatten(details))
    flat_data = flatten_json(
        flatten(real_details), "platform", "project_environment_details"
    )
Logs (only a few entries)
Copy code
2021-07-29 14:22:26-0500] INFO - prefect.TaskRunner | Task 'really_get_the_details[72976]': Finished task run for task with final state: 'Success'
[2021-07-29 14:22:26-0500] DEBUG - prefect.TaskRunner | Task 'really_get_the_details[70221]': Handling state change from Running to Success
[2021-07-29 14:22:26-0500] DEBUG - prefect.TaskRunner | Task 'really_get_the_details[78073]': Handling state change from Running to Success
[2021-07-29 14:22:26-0500] INFO - prefect.LocalDaskExecutor | Attempting to interrupt and cancel all running tasks...
[2021-07-29 14:22:33-0500] ERROR - prefect.FlowRunner | Unexpected error: TypeError("object of type 'NoneType' has no len()")
k
Got it will take a look
Looks like your Executor is running into an error and being interrupted and closing. Is there something with the 
really_get_the_details
that can potentially fail?
t
If the task hits an exception it should be raised as I am using 
raise_on_exception
k
That’s right, one sec let me ask the team if there are any more ideas.
t
Thanks,
Alright, I get a 
killed
message when I run it with 
prefect run
instead of raw python
k
I chatted with the team. 
raise_on_exception
is more for stopping the Flow run abruptly instead of continuing downstream tasks when an error is raised. It does seem that the map/flatten might be 
null
. What was the 
killed
message?
t
That message is linux killing my process because of memory issues. I need to re-work the whole thing
Is there a way to access the context from within a task?
looking to get the task run id out
k
Yep you can do 
prefect.context.get('task_run_id')
inside the task
t
thanks
2 Views
Open in Slack
PreviousNext
Powered by

Prefect Community

Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.

Powered by