Milenko Beslic
10/03/2022, 9:59 PMtrigger_dbt_cloud_job_run_and_wait_for_completion
fails with AttributeError: 'NoneType' object has no attribute 'call'
.
Here is the log:
21:33:15.658 | INFO | Flow run 'charcoal-harrier' - Created subflow run 'hot-civet' for flow 'emails-gold'
21:33:15.958 | INFO | Flow run 'hot-civet' - Created subflow run 'cocky-pony' for flow 'Trigger dbt Cloud job run and wait for completion'
21:33:16.069 | ERROR | Flow run 'cocky-pony' - Encountered exception during execution:
Traceback (most recent call last):
File "/usr/local/lib/python3.10/site-packages/prefect/engine.py", line 589, in orchestrate_flow_run
result = await flow_call()
File "/usr/local/lib/python3.10/site-packages/prefect_dbt/cloud/jobs.py", line 287, in trigger_dbt_cloud_job_run_and_wait_for_completion
run_id_future = get_run_id.submit(triggered_run_data_future)
File "/usr/local/lib/python3.10/site-packages/prefect/tasks.py", line 492, in submit
return enter_task_run_engine(
File "/usr/local/lib/python3.10/site-packages/prefect/engine.py", line 741, in enter_task_run_engine
return flow_run_context.sync_portal.call(begin_run)
AttributeError: 'NoneType' object has no attribute 'call'
21:33:16.133 | INFO | Flow run 'cocky-pony' - Created task run 'Trigger dbt Cloud job run-e4229069-0' for task 'Trigger dbt Cloud job run'
21:33:16.134 | INFO | Flow run 'cocky-pony' - Submitted task run 'Trigger dbt Cloud job run-e4229069-0' for execution.
21:33:16.264 | INFO | Task run 'Trigger dbt Cloud job run-e4229069-0' - Triggering run for job with ID 135917
21:33:16.558 | INFO | Task run 'Trigger dbt Cloud job run-e4229069-0' - Run successfully triggered for job with ID 135917. You can view the status of this run at <https://cloud.getdbt.com/#/accounts/xxxxx/projects/156718/runs/86440872/>
21:33:16.683 | INFO | Task run 'Trigger dbt Cloud job run-e4229069-0' - Finished in state Completed()
21:33:16.812 | ERROR | Flow run 'cocky-pony' - Finished in state Failed('Flow run encountered an exception.')
21:33:16.812 | ERROR | Flow run 'hot-civet' - Encountered exception during execution:
Traceback (most recent call last):
File "/usr/local/lib/python3.10/site-packages/prefect/engine.py", line 596, in orchestrate_flow_run
result = await run_sync(flow_call)
File "/usr/local/lib/python3.10/site-packages/prefect/utilities/asyncutils.py", line 57, in run_sync_in_worker_thread
return await anyio.to_thread.run_sync(call, cancellable=True)
File "/usr/local/lib/python3.10/site-packages/anyio/to_thread.py", line 31, in run_sync
return await get_asynclib().run_sync_in_worker_thread(
File "/usr/local/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 937, in run_sync_in_worker_thread
return await future
File "/usr/local/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 867, in run
result = context.run(func, *args)
File "/opt/prefect/flows/emails_deploy.py", line 35, in emails_gold
run = trigger_dbt_cloud_job_run_and_wait_for_completion(
File "/usr/local/lib/python3.10/site-packages/prefect/flows.py", line 390, in __call__
return enter_flow_run_engine_from_flow_call(
File "/usr/local/lib/python3.10/site-packages/prefect/engine.py", line 163, in enter_flow_run_engine_from_flow_call
return run_async_from_worker_thread(begin_run)
File "/usr/local/lib/python3.10/site-packages/prefect/utilities/asyncutils.py", line 137, in run_async_from_worker_thread
return anyio.from_thread.run(call)
File "/usr/local/lib/python3.10/site-packages/anyio/from_thread.py", line 49, in run
return asynclib.run_async_from_thread(func, *args)
File "/usr/local/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 970, in run_async_from_thread
return f.result()
File "/usr/local/lib/python3.10/concurrent/futures/_base.py", line 458, in result
return self.__get_result()
File "/usr/local/lib/python3.10/concurrent/futures/_base.py", line 403, in __get_result
raise self._exception
File "/usr/local/lib/python3.10/site-packages/prefect/client/orion.py", line 82, in with_injected_client
return await fn(*args, **kwargs)
File "/usr/local/lib/python3.10/site-packages/prefect/engine.py", line 522, in create_and_begin_subflow_run
return terminal_state.result()
File "/usr/local/lib/python3.10/site-packages/prefect/orion/schemas/states.py", line 145, in result
raise data
File "/usr/local/lib/python3.10/site-packages/prefect/engine.py", line 589, in orchestrate_flow_run
result = await flow_call()
File "/usr/local/lib/python3.10/site-packages/prefect_dbt/cloud/jobs.py", line 287, in trigger_dbt_cloud_job_run_and_wait_for_completion
run_id_future = get_run_id.submit(triggered_run_data_future)
File "/usr/local/lib/python3.10/site-packages/prefect/tasks.py", line 492, in submit
return enter_task_run_engine(
File "/usr/local/lib/python3.10/site-packages/prefect/engine.py", line 741, in enter_task_run_engine
return flow_run_context.sync_portal.call(begin_run)
AttributeError: 'NoneType' object has no attribute 'call'
21:33:16.892 | ERROR | Flow run 'hot-civet' - Finished in state Failed('Flow run encountered an exception.')
...
merlin
10/03/2022, 11:58 PMp102-13.py
If I leave .result()
in the code, it fails with error:
letter = get_letter().result()
AttributeError: 'str' object has no attribute 'result'
But remove the .result()
bit and I get success:
poetry run python src/quickstart/p102-13.py
16:48:31.385 | INFO | prefect.engine - Created flow run 'jasper-uakari' for flow 'Result Flow'
16:48:31.454 | INFO | Flow run 'jasper-uakari' - Created task run 'Get a letter-c414aff6-0' for task 'Get a letter'
16:48:31.454 | INFO | Flow run 'jasper-uakari' - Executing 'Get a letter-c414aff6-0' immediately...
16:48:31.468 | INFO | Task run 'Get a letter-c414aff6-0' - A
16:48:31.478 | INFO | Task run 'Get a letter-c414aff6-0' - Finished in state Completed()
16:48:31.478 | INFO | Flow run 'jasper-uakari' - You got: A
16:48:31.488 | INFO | Flow run 'jasper-uakari' - Finished in state Completed('All states completed.')
The poetry environment is:
[tool.poetry.dependencies]
python = "^3.10"
prefect = "^2.4.5"
greenlet = "^1.1.3"
wonsun
10/04/2022, 4:33 AMIgor Kotua
10/04/2022, 7:32 AMThomas Fredriksen
10/04/2022, 9:02 AMDeepanshu Aggarwal
10/04/2022, 9:54 AMBogdan Serban
10/04/2022, 10:05 AM/
, for instance /prefect/
. I am having an issue with loading the UI when I access <http://localhost/prefect/>
as the UI cannot load the corresponding assets, because it ignores the location prefix when doing the requests, for instance: <http://localhost/assets/index.1a4d60fb.js>
is available at <http://localhost/prefect/assets/index.1a4d60fb.js>
. Is there a way to specify a prefix or some sort of redirect URL for the UI to use when doing requests?Connor Vaid
10/04/2022, 11:10 AM{
"errors": [
{
"path": [
"flow_run",
0,
"id"
],
"message": "Cannot return null for non-nullable field flow_run.id.",
"extensions": {
"code": "INTERNAL_SERVER_ERROR"
}
}
],
"data": null
}
When I add the following “where” filter it seems to work (this would suggest I have some flows with null ids?), however when running a query to find flows with null ids nothing is returned.
{
"where": {
"flow": {
"id": {
"_is_null": false
}
}
}
}
Please let me know if you have any ideas about why this may happen, thanks!Hamza Naanani
10/04/2022, 2:21 PMJustin Trautmann
10/04/2022, 3:15 PMCrash detected! Request to <https://api.prefect.cloud/api/accounts/[...]/workspaces/[...]/task_runs/> failed.
...
RuntimeError: The connection pool was closed while 65 HTTP requests/responses were still in-flight.
this is most likely not related to local network issues as it is reproducible across different networks. when using a local orion server instead, the flow succeeds without any issues and a couple of days ago, the flow ran successfully on prefect cloud.
I am submitting ~100 parallel tasks using the RayTaskRunner with a local cluster.
Any help is much appreciated.
python 3.8.10
prefect 2.4.5
prefect-ray 0.2.0.post2Sean Turner
10/04/2022, 3:30 PMkubernetesJob
on EKS
orion, agent, and prefect cli are all 2.4.5
When I submit a simple task I get the following error in the agent logs:
HTTP response body: {"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"jobs.batch is forbidden: User \"system:anonymous\" cannot create resource \"jobs\" in API group \"batch\" in the namespace \"prefect\"","reason":"Forbidden","details":{"group":"batch","kind":"jobs"},"code":403}
Seems that for some reason the service account called agent
isn't being observed? system:anonymous
is being used instead. The role
and roleBinding
both match what is in the helm chart so permissions should be fine.
$ kgp agent-5fbdcf4bbb-zrrg7 -o yaml | grep serviceAccount
serviceAccount: agent
serviceAccountName: agent
I was not having these problems before upgrading from 2.4.0 -> 2.4.5 😞. I created a new kubernetesjob block for 2.4.5 to silence an errorIgor Morgunov
10/04/2022, 3:54 PMAndrei Tulbure
10/04/2022, 4:14 PMGarren Moore
10/04/2022, 4:59 PMdatabricks
currently supported as a connection/profile type when using prefect-dbt cli commands (like trigger_dbt_cli_command
?
RuntimeError: Command failed with exit code 1:
Could not find adapter type databricks!
Philip MacMenamin
10/04/2022, 6:12 PMSchuyler Duveen
10/04/2022, 6:38 PMSean Turner
10/04/2022, 7:50 PMKubernetesJob
infra blocks in a Deployment
?
e.g.
flow a
task b (uses KubernetesJob with docker image foo)
task c (uses KubernetesJob with docker image bar)
It kind of seems like I'm limited to to having one KubernetesJob
per Deployment
?Bradley McLaughlin
10/04/2022, 8:23 PMYZ
10/04/2022, 9:43 PMmy_task(run_date="20221004")
retrying run: my_task(run_date=None)
Any idea what I might have done wrong, or any workaround?
@task(retry_delay=timedelta(minutes=30), max_retries=10) # wait for up to 5 hours
def my_task(run_date: str):
Sander
10/04/2022, 9:47 PMWill Hayes
10/04/2022, 10:25 PMTan Li
10/04/2022, 11:50 PMSecret.load("github-access-token").get()
in normal python scripts?Jelle Vegter
10/05/2022, 7:32 AMHa Pham
10/05/2022, 8:02 AMHieu Tran
10/05/2022, 8:35 AMtimeout_seconds
of the flow as parameter to customize it when trigger a deployment of that flow manually from UI?
@flow(timeout_seconds = 15)
def log_flow(name: str):
log_task(name)
As timeout_seconds is in the flow decorator, it won't be shown as a parameter of the flow runErik
10/05/2022, 10:36 AMprefect agent
are:
╭─ Options ──────────────────────────────────────────────────────────────────────
│ --work-queue -q TEXT One or more work queue names for the agent to pull from. [default: None] │
│ --hide-welcome │
│ --api TEXT [default: (from PREFECT_API_URL)] │
│ --tag -t TEXT DEPRECATED: One or more optional tags that will be used to create a work queue [default: None] │
│ --help Show this message and exit. │
╰─────────────────────────────────────────────────────────────────────────────────
What I would like to do is launch an agent once, grab the flow_runs that are sitting in the queue (a single call), once those are completed, the agent would close itself. Is there any kind of option for that?Steph Clacksman
10/05/2022, 10:57 AMTypeError: Dict key must be str
but the other doesn't. The one causing the error does have ~1 million rows though.Q
10/05/2022, 12:42 PMprojname/flows/myflow.py
imports projname.utils.utilfunc
).
This flow is scheduled from a deployment and uses Process
as its infrastructure block.
I would like to ensure that the latest version of projname
is installed (to guarantee that utilfunc
is up-to-date), so I can't install the package when setting up agent environment.
I think I can install the package before running the flow if I set Process.command
to something like ["pip", "install", ".", "&&", "python3", "-m", "prefect.engine"]
, but maybe there's a better way to go about it?Aaron Lanzen
10/05/2022, 1:07 PMGarren Moore
10/05/2022, 2:49 PMERROR prefect.infrastructure.process - Process 'turquoise-woodlouse' exited with status code: -9
No more messaging beyond that.
In Prefect Cloud UI, the flow runs indefinitely and doesn’t show this log message (level set to DEBUG).
Anyone familiar with exit code -9? Can’t find any docs.Garren Moore
10/05/2022, 2:49 PMERROR prefect.infrastructure.process - Process 'turquoise-woodlouse' exited with status code: -9
No more messaging beyond that.
In Prefect Cloud UI, the flow runs indefinitely and doesn’t show this log message (level set to DEBUG).
Anyone familiar with exit code -9? Can’t find any docs.Taylor Curran
10/05/2022, 3:02 PMGarren Moore
10/05/2022, 3:10 PMRyan Peden
10/05/2022, 3:10 PMGarren Moore
10/05/2022, 3:44 PMRyan Peden
10/05/2022, 3:55 PMMichael Adkins
10/05/2022, 4:43 PMGarren Moore
10/05/2022, 4:45 PM