Steven Trimboli
09/30/2022, 2:56 PMTony Piazza
09/30/2022, 3:11 PMFile "/opt/conda/envs/coiled/lib/python3.9/site-packages/prefect/client/base.py", line 125, in raise_for_status
Exception: PrefectHTTPStatusError("Client error '404 Not Found' for url '<http://ephemeral-orion/api/task_runs/add94276-cb3e-417d-a3f9-033f935ae5ec/set_state>'\nResponse: {'exception_message': 'Task run with id add94276-cb3e-417d-a3f9-033f935ae5ec not found'}\nFor more information check: <https://httpstatuses.com/404>")
Any ideas as to what might cause this?Taylor Babin
09/30/2022, 3:17 PMAlex Shea
09/30/2022, 4:15 PMYingbo
09/30/2022, 4:28 PMLocalRun
. I have a single Prefect agent running in a container. I use IntervalSchedule
and set interval=timedelta(seconds=60)
. I expected to see my flow run once every minute indefinitely, but instead, I see 10 flows scheduled on the Dashboard. I tried other schedulers -- Cron, etc -- but it always schedules 10 and only 10 runs. Am I missing something or misunderstanding how schedulers work? ThanksAndrew Pruchinski
09/30/2022, 4:46 PMINTERNAL_SERVER_ERROR
and prefect's secret client throwing the KeyError: 'SLACK_WEBHOOK_URL'
. I know there is a current intermittent issue in regards to retrieving secrets. We pass our secrets to the dask worker via the docker container so we aren't using the prefect secrets stored in the UI. However we are still getting this error and it has brought down multiple flows this morning. The error, other information and questions will be in the threads. Thank you in advance!Evan Dong
09/30/2022, 4:54 PMThomas Fredriksen
09/30/2022, 5:59 PMKubernetesJob
. However, just before posting a question in this channel, I figured I should "give it one last try" and to my surprise it actually worked 🤡
As the post is already written, I figured I might as well share was I experienced and the solutions so that anyone else having simiular issues may have somewhere it start.
I was trying to create a deployment using KubernetesJob
as infrastructure. The deployment looked like this (see code further down in the thread)
The container image is built from a dockerfile based on prefecthq/prefect:2-latest
and copies the script containing the flow to /opt/prefect/flows/flow.py
.
This deploys just fine, and I can see that the kubernetes job is created and the Prefect engine starts, however it does not seem like the flow runst, and I instead get the following error (see traceback further down in the thread)
It turns out that the base docker-image I had chosen was the issue, as the prefect version did not match the version the server was running (2.4.5
vs 2.4.2
). From the release-notes for 2.4.3
-2.4.5
i see that block protection is an issue, and I speculate that this might be the root cause.
Hope it helps 🙂
Edit: Moved code and traceback to the threadJoe
09/30/2022, 6:15 PMNick Coy
09/30/2022, 6:34 PMDavid Cupp
09/30/2022, 7:29 PMJosh Paulin
09/30/2022, 7:46 PMMatthew Seligson
09/30/2022, 8:20 PMNavin Chaganti
09/30/2022, 8:22 PMents/Developer/Library/Frameworks/Python3.framework/Versions/3.9/Headers -arch arm64 -arch x86_64 -Werror=implicit-function-declaration -I/Users/navinchaganti/codebase/prefect-dev/prefect_env/include -I/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.9/Headers -c asyncpg/pgproto/pgproto.c -o build/temp.macosx-10.9-universal2-cpython-39/asyncpg/pgproto/pgproto.o -O2 -fsigned-char -Wall -Wsign-compare -Wconversion
asyncpg/pgproto/pgproto.c:31:10: fatal error: 'Python.h' file not found
#include "Python.h"
^~~~~~~~~~
1 error generated.
error: command '/usr/bin/clang' failed with exit code 1
----------------------------------------
ERROR: Failed building wheel for asyncpg
Failed to build asyncpg
ERROR: Could not build wheels for asyncpg which use PEP 517 and cannot be installed directly
Anyone faced the asyncpg error while installation ?Meghan Franklin
09/30/2022, 8:45 PMfail_signal = signals.FAIL(message_here); raise fail_signal
-> the state handler isn’t triggering, although it does trigger when a task otherwise changes status AND it triggers if a task fails for a reason other than a raise fail_signal
. What even more baffling is that I can run the sample code with mission_critical_task
from here and I get the expected behavior and the expected triggers. It’s a complicated flow, but I’ve ripped out as many pieces and simplified as much as possible and now I’m out of ideas on why I’m getting this lack of behavior. I’m tied to prefect 0.15.13Trevor Sweeney
09/30/2022, 10:34 PM~\.dbt\profiles.yml
, I receive the following runtime error and I'm having difficulty interpreting the issue, exit code 4294967295
. Has anyone else countered this?
Error:
"..~\venv\lib\site-packages\prefect_shell\commands.py", line 90, in shell_run_command raise RuntimeError(msg)
RuntimeError: Command failed with exit code 4294967295:
Script:
from prefect import flow
from prefect_dbt.cli.commands import trigger_dbt_cli_command
@flow
def trigger_dbt_cli_command_flow():
result = trigger_dbt_cli_command("dbt debug")
return result
trigger_dbt_cli_command_flow()
Versions:
Windows 10
Prefect: 2.4.5
Prefect-dbt: 0.2.2
Thanks very much!Miremad Aghili
09/30/2022, 10:36 PMapp.prefect.cloud
today. Is this maintenance time?Hedgar
10/01/2022, 8:17 AMawswrangler
package to wrangle data and move same to an S3 bucket. Before signing on prefect cloud I could use wr
by simply indicating an s3 bucket path and my data would go to this path. Recently I created an S3 block via prefect cli.
Now in an attempt to build deployment with the -sb
flag I got an error that I indicated wrong s3 path.
Clarifying questions:
1. Must I use the prefect cloud s3 storage block as the path
under wr
package?
2. Can I have a separate s3 bucket for storing output of my data wrangling different from the s3 bucket where my flow code should reside?
I would appreciate if your answer include some basic scenarios. I have gone through all your repo and none appear to address this issue probably there is, maybe implicitly .Bal Raj
10/02/2022, 8:53 AMFailed to load and execute flow run: ClientError([{'path': ['secret_value'], 'message': 'Unable to complete operation. An internal API error occurred.', 'extensions': {'code': 'API_ERROR'}}])
facing this issue multiple times? Any solutions?Pekka
10/02/2022, 3:56 PMpypy 3.9
since it has JIT. I'm trying to install prefect as the minimal dependency but there are some problems that will probably be outside of the 'supported things' -- prefect has orjson
as a dependency which is not pypy-supported. I wonder how big of a role that dependency plays in prefect and whether there's some way to hack a bit to make it use some other lib?Tony Popov
10/02/2022, 5:39 PMPREFECT_LOCAL_STORAGE_PATH
does in this context, I guess points to pod’s filesystem and persistent storage/bucket has to be mounted to a pathWilliam Wolfe-McGuire
10/02/2022, 7:26 PMis_serializable
to check if entire flows can be serialized but that doesnt allow you to narrow down which tasks are responsible for your serialization issues. what is the best way to debug in this situation?Tony Popov
10/03/2022, 1:16 AMclass Result:
pass
@task
async def my_task_1(arg: str) -> Result:
...
@task
async def my_task_2(arg: Result) -> None:
...
Question
Doc says I can do await my_task_1('arg')
in a flow, although mypy says my_task_1('arg')
is not awaitable
Is following a best practice?
task_result = await my_task.submit('arg')
result = await task_result.result()
await my_task_2.submit(result)
Jessica Smith
10/03/2022, 2:00 AMZac Hooper
10/03/2022, 2:41 AMKrzysztof Ciesielski
10/03/2022, 8:39 AMThomas Opsomer
10/03/2022, 8:40 AMMichal Luščon
10/03/2022, 8:41 AMJoshua Greenhalgh
10/03/2022, 8:42 AMException raised while calling state handlers: ClientError([{'path': ['secret_value'], 'message': 'Unable to complete operation. An internal API error occurred.', 'extensions': {'code': 'API_ERROR'}}])
Traceback (most recent call last):
File "/usr/local/lib/python3.9/site-packages/prefect/client/secrets.py", line 140, in get
value = secrets[self.name]
KeyError: 'SLACK_WEBHOOK_URL'
Stefan
10/03/2022, 8:42 AMfrom prefect import task, flow
@task
def task1(arg1):
pass
@flow
def main_flow():
task1(arg1_wrong=1)
main_flow()
Logs on Prefect Cloud: