https://prefect.io logo
Join Slack
Powered by
hello, i updated my application to prefect 2.0.2 n...
# ask-community
t
hello, i updated my application to prefect 2.0.2 now the flow are crashing, even in very sample code..
15:59:11.628 | INFO    | prefect.engine - Created flow run 'huge-lorikeet' for flow 'my_test_endpoint'
15:59:11.628 | INFO    | Flow run 'huge-lorikeet' - Starting 'ConcurrentTaskRunner'; submitted tasks will be run concurrently...
15:59:11.695 | ERROR   | Flow run 'huge-lorikeet' - Crash detected! Execution was interrupted by an unexpected exception.
15:59:11.728 | ERROR   | root - TypeError
Traceback (most recent call last):
File "/usr/local/lib/python3.8/site-packages/starlette/middleware/errors.py", line 162, in __call__
await <http://self.app|self.app>(scope, receive, _send)
File "/usr/local/lib/python3.8/site-packages/starlette/middleware/base.py", line 68, in __call__
response = await self.dispatch_func(request, call_next)
File "/usr/local/lib/python3.8/site-packages/starlette_context/middleware/context_middleware.py", line 70, in dispatch
response = await call_next(request)
File "/usr/local/lib/python3.8/site-packages/starlette/middleware/base.py", line 46, in call_next
raise app_exc
File "/usr/local/lib/python3.8/site-packages/starlette/middleware/base.py", line 36, in coro
await <http://self.app|self.app>(scope, request.receive, send_stream.send)
File "/usr/local/lib/python3.8/site-packages/starlette/exceptions.py", line 93, in __call__
raise exc
File "/usr/local/lib/python3.8/site-packages/starlette/exceptions.py", line 82, in __call__
await <http://self.app|self.app>(scope, receive, sender)
File "/usr/local/lib/python3.8/site-packages/fastapi/middleware/asyncexitstack.py", line 21, in __call__
raise e
File "/usr/local/lib/python3.8/site-packages/fastapi/middleware/asyncexitstack.py", line 18, in __call__
await <http://self.app|self.app>(scope, receive, send)
File "/usr/local/lib/python3.8/site-packages/starlette/routing.py", line 670, in __call__
await route.handle(scope, receive, send)
File "/usr/local/lib/python3.8/site-packages/starlette/routing.py", line 266, in handle
await <http://self.app|self.app>(scope, receive, send)
File "/usr/local/lib/python3.8/site-packages/starlette/routing.py", line 65, in app
response = await func(request)
File "/usr/local/lib/python3.8/site-packages/fastapi/routing.py", line 227, in app
raw_response = await run_endpoint_function(
File "/usr/local/lib/python3.8/site-packages/fastapi/routing.py", line 162, in run_endpoint_function
return await run_in_threadpool(dependant.call, **values)
File "/usr/local/lib/python3.8/site-packages/starlette/concurrency.py", line 41, in run_in_threadpool
return await <http://anyio.to|anyio.to>_thread.run_sync(func, *args)
File "/usr/local/lib/python3.8/site-packages/anyio/to_thread.py", line 31, in run_sync
return await get_asynclib().run_sync_in_worker_thread(
File "/usr/local/lib/python3.8/site-packages/anyio/_backends/_asyncio.py", line 937, in run_sync_in_worker_thread
return await future
File "/usr/local/lib/python3.8/site-packages/anyio/_backends/_asyncio.py", line 867, in run
result = context.run(func, *args)
File "/usr/local/lib/python3.8/site-packages/prefect/flows.py", line 390, in __call__
return enter_flow_run_engine_from_flow_call(
File "/usr/local/lib/python3.8/site-packages/prefect/engine.py", line 152, in enter_flow_run_engine_from_flow_call
return anyio.run(begin_run)
File "/usr/local/lib/python3.8/site-packages/anyio/_core/_eventloop.py", line 70, in run
return asynclib.run(func, *args, **backend_options)
File "/usr/local/lib/python3.8/site-packages/anyio/_backends/_asyncio.py", line 292, in run
return native_run(wrapper(), debug=debug)
File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
return loop.run_until_complete(main)
File "uvloop/loop.pyx", line 1501, in uvloop.loop.Loop.run_until_complete
File "/usr/local/lib/python3.8/site-packages/anyio/_backends/_asyncio.py", line 287, in wrapper
return await func(*args)
File "/usr/local/lib/python3.8/site-packages/prefect/client.py", line 104, in with_injected_client
return await fn(*args, **kwargs)
File "/usr/local/lib/python3.8/site-packages/prefect/engine.py", line 221, in create_then_begin_flow_run
state = await begin_flow_run(
File "/usr/local/lib/python3.8/site-packages/prefect/engine.py", line 333, in begin_flow_run
await result_filesystem._save(is_anonymous=True)
File "/usr/local/lib/python3.8/site-packages/prefect/blocks/core.py", line 618, in _save
await self.register_type_and_schema(client=client)
TypeError: object NoneType can't be used in 'await' expression
15:59:11.733 | ERROR   | uvicorn.error - Exception in ASGI application
Traceback (most recent call last):
File "/usr/local/lib/python3.8/site-packages/uvicorn/protocols/http/httptools_impl.py", line 372, in run_asgi
result = await app(self.scope, self.receive, self.send)
File "/usr/local/lib/python3.8/site-packages/uvicorn/middleware/proxy_headers.py", line 75, in __call__
return await <http://self.app|self.app>(scope, receive, send)
File "/usr/local/lib/python3.8/site-packages/fastapi/applications.py", line 269, in __call__
await super().__call__(scope, receive, send)
File "/usr/local/lib/python3.8/site-packages/starlette/applications.py", line 124, in __call__
await self.middleware_stack(scope, receive, send)
File "/usr/local/lib/python3.8/site-packages/starlette/middleware/errors.py", line 184, in __call__
raise exc
File "/usr/local/lib/python3.8/site-packages/starlette/middleware/errors.py", line 162, in __call__
await <http://self.app|self.app>(scope, receive, _send)
File "/usr/local/lib/python3.8/site-packages/starlette/middleware/base.py", line 68, in __call__
response = await self.dispatch_func(request, call_next)
File "/usr/local/lib/python3.8/site-packages/starlette_context/middleware/context_middleware.py", line 70, in dispatch
response = await call_next(request)
File "/usr/local/lib/python3.8/site-packages/starlette/middleware/base.py", line 46, in call_next
raise app_exc
File "/usr/local/lib/python3.8/site-packages/starlette/middleware/base.py", line 36, in coro
await <http://self.app|self.app>(scope, request.receive, send_stream.send)
File "/usr/local/lib/python3.8/site-packages/starlette/exceptions.py", line 93, in __call__
raise exc
File "/usr/local/lib/python3.8/site-packages/starlette/exceptions.py", line 82, in __call__
await <http://self.app|self.app>(scope, receive, sender)
File "/usr/local/lib/python3.8/site-packages/fastapi/middleware/asyncexitstack.py", line 21, in __call__
raise e
File "/usr/local/lib/python3.8/site-packages/fastapi/middleware/asyncexitstack.py", line 18, in __call__
await <http://self.app|self.app>(scope, receive, send)
File "/usr/local/lib/python3.8/site-packages/starlette/routing.py", line 670, in __call__
await route.handle(scope, receive, send)
File "/usr/local/lib/python3.8/site-packages/starlette/routing.py", line 266, in handle
await <http://self.app|self.app>(scope, receive, send)
File "/usr/local/lib/python3.8/site-packages/starlette/routing.py", line 65, in app
response = await func(request)
File "/usr/local/lib/python3.8/site-packages/fastapi/routing.py", line 227, in app
raw_response = await run_endpoint_function(
File "/usr/local/lib/python3.8/site-packages/fastapi/routing.py", line 162, in run_endpoint_function
return await run_in_threadpool(dependant.call, **values)
File "/usr/local/lib/python3.8/site-packages/starlette/concurrency.py", line 41, in run_in_threadpool
return await <http://anyio.to|anyio.to>_thread.run_sync(func, *args)
File "/usr/local/lib/python3.8/site-packages/anyio/to_thread.py", line 31, in run_sync
return await get_asynclib().run_sync_in_worker_thread(
File "/usr/local/lib/python3.8/site-packages/anyio/_backends/_asyncio.py", line 937, in run_sync_in_worker_thread
return await future
File "/usr/local/lib/python3.8/site-packages/anyio/_backends/_asyncio.py", line 867, in run
result = context.run(func, *args)
File "/usr/local/lib/python3.8/site-packages/prefect/flows.py", line 390, in __call__
return enter_flow_run_engine_from_flow_call(
File "/usr/local/lib/python3.8/site-packages/prefect/engine.py", line 152, in enter_flow_run_engine_from_flow_call
return anyio.run(begin_run)
File "/usr/local/lib/python3.8/site-packages/anyio/_core/_eventloop.py", line 70, in run
return asynclib.run(func, *args, **backend_options)
File "/usr/local/lib/python3.8/site-packages/anyio/_backends/_asyncio.py", line 292, in run
return native_run(wrapper(), debug=debug)
File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
return loop.run_until_complete(main)
File "uvloop/loop.pyx", line 1501, in uvloop.loop.Loop.run_until_complete
File "/usr/local/lib/python3.8/site-packages/anyio/_backends/_asyncio.py", line 287, in wrapper
return await func(*args)
File "/usr/local/lib/python3.8/site-packages/prefect/client.py", line 104, in with_injected_client
return await fn(*args, **kwargs)
File "/usr/local/lib/python3.8/site-packages/prefect/engine.py", line 221, in create_then_begin_flow_run
state = await begin_flow_run(
File "/usr/local/lib/python3.8/site-packages/prefect/engine.py", line 333, in begin_flow_run
await result_filesystem._save(is_anonymous=True)
File "/usr/local/lib/python3.8/site-packages/prefect/blocks/core.py", line 618, in _save
await self.register_type_and_schema(client=client)
TypeError: object NoneType can't be used in 'await' expression
k
Can you paste the code that has this error?
t
the sample python (fastapi endpoint)
Copy code
@task(name="get_name")
def get_name():
    return "world"

@task(name="hello_name")
def hello_name(name):
    print("hello " + name)
    return "hello " + name


@flow(name="call_name")
def call_name():
    name = get_name()
    print("the name is: " + name)  # 2
    return name


@router.get('/test')
def get_test_endpoint():
    res = call_name()
    return "test"
am calling get_test_endpoint()
k
get_test_endpoint
is not a flow. Is that correct?
t
not a flow, i removed the flow notation above it..the same error
k
Can you try to run the flow 
call_name
by itself and see if you get the error?
t
hm,, this works, if i have it outside the endpoint
i used before 2.0b6 and all endpoints were working fine calling the flows, today i updated to 2.0.2, all having crash
k
Can you check to make sure the output of your flow is not None?
t
i have this :
Copy code
res = call_name()
print(res)
the output is 
world
..correct
k
Can you add 
.submit
to the task call within your flow and see if you see the same error?
Copy code
@flow(name="call_name")
def call_name():
    name = get_name.submit()
    return name
m
Hey @Tarek I was able to take your example code as is and it ran correctly, This might seem a bit tedious but can you try setting up a new Environment to test this there
k
Do you use 2.0.2 @Mason Menges?
t
@Khuyen Tran with submit and outside the endpoint its working..but the print output is "completed" but within the endpoint the same error
i see this in the log: 
File "/usr/local/lib/python3.8/site-packages/prefect/blocks/core.py", line 618, in _save
TypeError: object NoneType can't be used in 'await' expression
do i need a block object in the UI? i dont think so
m
The code snippet works for me on version 2.0.2 and 2.0.3 so I don't think this is version specific, you could also try resetting the orion database if you feel comfortable doing that though that seems like the nuclear option in this case assuming a new environment with a fresh installation of the dependencies doesn't address the issue.
Copy code
prefect orion database reset -y
t
i have rested the db, with the command, the ui is clean now, but still this error, its strange that the flow call outside the endpoint is working..
m
I know I probably sound like a broken record haha but are you testing this in a Virtual Environment and if so can you reproduce this same error in a New environment setup?
👍 1
t
ok i will try
hello @Mason Menges, hello @Khuyen Tran i can now reproduce the error with sample python file 🙂
Copy code
from prefect import flow, task

@task(name="get_name")
def get_name():
    return "world"


@flow(name="call_name")
def call_name():
    name = get_name()
    print("the name is: " + name) 
    return name


from fastapi import APIRouter
router = APIRouter(redirect_slashes=False)

@router.get('/test')
def get_test_endpoint():
    return call_name()


@router.get('/ok')
def get_ok():
    return "ok"

import uvicorn
if __name__ == "__main__":
    uvicorn.run(router, host="0.0.0.0", port=8000)
then call http://localhost:8000/test in the webbrowser
i use fastapi 0.78.0 and uvicorn 0.17.6
hi @Anna Geller, would you please look into this, the last sample python file produces an error in prefect package, but only within fastapi request..also in 2.0.3 reproducable
a
please don't tag anyone
can you explain what you're trying to do here?
I can open an issue for the error here and we can investigate; until then you can trigger any flow from deployment using https://discourse.prefect.io/t/how-to-create-a-flow-run-from-deployment-orchestrator-pattern/803
t
Hi Anna, i am just calling a flow within an api request...this causing a crash in prefect library, if i execute the same code outside the fastapi request call, i dont have this problem, but this last sample python standalone file reproduces the problem
a
yup sure, opening an issue @Marvin open "Running a flow from FastAPI app crashes with block's `register_type_and_schema`"
m
t
Thank you :-)
🙌 1
Hi Anna, kindly am asking, just to plan our stuff, when do you nearly plan to fix this? thanks
a
to be fully transparent, this is not a high-priority item -- if this is important for you, you can contribute a PR and we'll review
t
hi Anna, i removed the @sync_compatible decorator from the register_type_and_schema method in: prefect\blocks\core.py now both flows are working with and without fastapi request.. could this be the fix? currently i cant create a branch in prefect github repository, i think my company blocks contribution. 
git push --set-upstream origin fix_running_a_flow_from_fastapi
remote: Permission to PrefectHQ/prefect.git denied to
...
a
I don't know but I will add this information to the GitHub issue
thanks for diving deeper into this, I appreciate it! 🙌
39 Views
Open in Slack
PreviousNext
Powered by