Nimesh Kumar
01/28/2023, 1:19 PMLee Mendelowitz
01/29/2023, 4:18 PMDaniel Davee
01/30/2023, 2:43 AMHamza Kazmi
01/30/2023, 8:50 AMAnkit
01/30/2023, 8:53 AMFerdinand von den Eichen
01/30/2023, 9:24 AMeddy davies
01/30/2023, 1:10 PMMac
01/30/2023, 1:38 PMMia
01/30/2023, 2:44 PMDownloading flow code from storage at ''
03:31:49 PM
Flow could not be retrieved from deployment.
Traceback (most recent call last):
File "<frozen importlib._bootstrap_external>", line 846, in exec_module
File "<frozen importlib._bootstrap_external>", line 982, in get_code
File "<frozen importlib._bootstrap_external>", line 1039, in get_data
FileNotFoundError: [Errno 2] No such file or directory: 'example_flow.py'
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/usr/local/lib/python3.9/site-packages/prefect/engine.py", line 266, in retrieve_flow_then_begin_flow_run
flow = await load_flow_from_flow_run(flow_run, client=client)
File "/usr/local/lib/python3.9/site-packages/prefect/client/utilities.py", line 47, in with_injected_client
return await fn(*args, **kwargs)
File "/usr/local/lib/python3.9/site-packages/prefect/deployments.py", line 186, in load_flow_from_flow_run
flow = await run_sync_in_worker_thread(load_flow_from_entrypoint, str(import_path))
File "/usr/local/lib/python3.9/site-packages/prefect/utilities/asyncutils.py", line 91, in run_sync_in_worker_thread
return await anyio.to_thread.run_sync(
File "/usr/local/lib/python3.9/site-packages/anyio/to_thread.py", line 31, in run_sync
return await get_asynclib().run_sync_in_worker_thread(
File "/usr/local/lib/python3.9/site-packages/anyio/_backends/_asyncio.py", line 937, in run_sync_in_worker_thread
return await future
File "/usr/local/lib/python3.9/site-packages/anyio/_backends/_asyncio.py", line 867, in run
result = context.run(func, *args)
File "/usr/local/lib/python3.9/site-packages/prefect/flows.py", line 772, in load_flow_from_entrypoint
flow = import_object(entrypoint)
File "/usr/local/lib/python3.9/site-packages/prefect/utilities/importtools.py", line 195, in import_object
module = load_script_as_module(script_path)
File "/usr/local/lib/python3.9/site-packages/prefect/utilities/importtools.py", line 158, in load_script_as_module
raise ScriptError(user_exc=exc, path=path) from exc
prefect.exceptions.ScriptError: Script at 'example_flow.py' encountered an exception: FileNotFoundError(2, 'No such file or directory')
Guy Altman
01/30/2023, 3:35 PMeddy davies
01/30/2023, 4:42 PM[]
as input but the function would fail with that input and output to s3 show it has received an input?Hans Lellelid
01/30/2023, 4:50 PMAnkit
01/30/2023, 5:15 PMLeon Kozlowski
01/30/2023, 6:34 PMexpected_start_time
resolved TaskRun
within context
? I'm looking at docs and I see
expected_start_time: Optional[DateTimeTZ] = Field(
default=None,
description="The task run's expected start time.",
)
How would the expected start time of a task run be determined?Leo Kacenjar
01/30/2023, 8:01 PMJean-Michel Provencher
01/30/2023, 9:06 PMIsmail Cenik
01/30/2023, 9:35 PMLuke
01/30/2023, 11:01 PMtrigger_project_run_and_wait_for_completion
, which is highlighted in the tutorial, actually available in the package. I'm using prefect 2.7.10.Joshua Grant
01/30/2023, 11:48 PMprefect==2.7.10
using ECSTask
from prefect-aws==0.2.4
, in which running the flow is baked into the Docker image but prefect
copies over the code, which results in a shutil.Error
. Before upgrading, we used the GitHub
storage block in the command to Deployment.build_from_flow()
as we encountered this error when using the path
parameter with prefect==2.6.9
. Details in 🧵.FuETL
01/31/2023, 12:57 AMFailed to set task state with error: ClientError([{'message': 'State payload is too large.', 'locations': [{'line': 2, 'column': 5}], 'path': ['set_task_run_states'], 'extensions': {'code': 'INTERNAL_SERVER_ERROR', 'exception': {'message': 'State payload is too large.'}}}])
The stacktrace
Traceback (most recent call last):
File "/app/src/prefect/engine/cloud/task_runner.py", line 91, in call_runner_target_handlers
state = self.client.set_task_run_state(
File "/app/src/prefect/client/client.py", line 1622, in set_task_run_state
result = self.graphql(
File "/app/src/prefect/client/client.py", line 465, in graphql
raise ClientError(result["errors"])
Daniel Davee
01/31/2023, 2:01 AMTomás Emilio Silva Ebensperger
01/31/2023, 2:36 AMfastapi.BackgroundTasks
but i can't find a way to return the flow_run_id as the route response and also have the flow run in the background. Thank you.Dylan Lim
01/31/2023, 3:49 AMIshan Anilbhai Koradiya
01/31/2023, 5:17 AMAnkit
01/31/2023, 6:39 AMpd.read_parquet(bucket, key)
to read the parquet files as pandas df earlier which doesn't seem to work with my new setup probably compatibility issue of pandas or some other module's version. I tried using s3 blocks to read but it keeps giving the same error of file not found
. Anyone aware about what might be wrong?
from prefect.filesystems import S3
s3_block = S3.load("block-name")
s3_block.read_path('key')
Satsuki Nagae
01/31/2023, 7:29 AMMuhammad Husnain
01/31/2023, 7:42 AMcontext = prefect.context.get_run_context().dict()
context['custom_value'] = "some_value"
print("prefect_context_dict", context)
The above code does not update the prefect dictionary, even though it should since it is a dictionary?
Is it even possible to do that in prefect 2?Michal Luščon
01/31/2023, 9:43 AMprefect deployment build
with a remote storage set. It seems to me that the prefect doesn't delete old/already removed .py files from remote storage and it also downloads them to worker during the execution of a flow.Abhishek Mitra
01/31/2023, 11:01 AMWatching flow run execution...
└── 16:18:49 | INFO | Entered state <Scheduled>: Flow run scheduled.
└── 16:18:57 | INFO | Entered state <Submitted>: Submitted for execution
└── 16:18:57 | ERROR | An error occurred (ClientException) when calling the RegisterTaskDefinition operation: Role is not valid
└── 16:18:58 | INFO | Entered state <Failed>: An error occurred (ClientException) when calling the RegisterTaskDefinition operation: Role is not valid
Flow run failed!
Torstein Molland
01/31/2023, 1:18 PMprefect deployment build -n $PROJECT -q $PROJECT -a -sb s3/$S3_BLOCK_NAME -v $GITHUB_SHA -o $FLOW_NAME.yaml flows/test/test.py:$FLOW_NAME --timezone 'UTC' --cron '0 22 * * 1,2,3,5'
but the schedule was not updated. Am I perhaps misunderstanding how to use the -a
parameter?