Hello, I have 4 tasks that run successfully when r...
# ask-communitya
Adam Roderick
12/12/2022, 3:53 PMHello, I have 4 tasks that run successfully when running locally using FlowRunner. When running in Prefect Cloud, I get the following results How can I troubleshoot this?
• TypeError("'generator' object is not callable")
• SystemError('unknown opcode')
• SystemError('Objects/dictobject.c1527 bad argument to internal function')
• The 4th task hangs indefinitely
Adam Roderick
12/12/2022, 4:29 PMAny idea on this @Anna Geller or @Dylan?
Adam Roderick
12/12/2022, 4:30 PMI can pull down the docker image that is being used, manually spin up a container, and successfully run the flow using FlowRunner
d
Dylan
12/12/2022, 4:45 PM
Hey Adam 👋
Can you share a minimum reproducible example here?
a
Adam Roderick
12/12/2022, 4:48 PM👋 I will work on that. The problem may be in our infrastructure/devops, so I'll do my best. Any ideas on troubleshooting errors like these so I can try and narrow it down?
z
Zanie
12/12/2022, 4:50 PM
I’ve never seen these
SystemErrorZanie
12/12/2022, 4:50 PM
It seems like something very weird must be going on. Are you mixing Python versions?
Zanie
12/12/2022, 4:50 PM
(Pickling on deploy with one version and running with another?)
a
Adam Roderick
12/12/2022, 4:52 PMWe've confirmed that the local dev environments, the build container, and the execution environment are all on python 3.8
Adam Roderick
12/12/2022, 4:52 PMOur searches do seem to indicate a python version mismatch, but I don't know where else to look for a version mismatch
z
Zanie
12/12/2022, 4:57 PM
Perhaps it’d be helpful to run it without an agent using
prefect run -n "<flow-name>" --executeZanie
12/12/2022, 4:57 PM
It may also be helpful to use script-based storage instead of pickle-based (if you are).
Zanie
12/12/2022, 4:58 PM
Are you using Cython?
Zanie
12/12/2022, 4:59 PM
a
Adam Roderick
12/12/2022, 4:59 PMThanks for the thoughts Michael. We aren't using Cython but perhaps one of the dependencies is
Adam Roderick
12/12/2022, 4:59 PMScript-based storage is new to me, will check it out
Adam Roderick
12/12/2022, 4:59 PMWould
prefect run -n "<flow-name>" --executez
Zanie
12/12/2022, 5:00 PM
It’ll run it as though it is the agent — it’ll connect to Cloud.
👍 1
Zanie
12/12/2022, 5:01 PM
I think it’ll use a subprocess instead of a container though, which may be helpful.
a
Adam Roderick
12/12/2022, 8:43 PMThis can down to an issue of the snowflake connector and something about prefect's execution model. When we implement following the pattern at https://github.com/PrefectHQ/prefect/blob/1.x/src/prefect/tasks/snowflake/snowflake.py#L187 (using
fetchall()2 Views