Patrick Tan12/20/2022, 7:09 PM
Patrick Tan12/20/2022, 7:13 PM
Amruth VVKP12/20/2022, 8:08 PM
Santhosh Solomon (Fluffy)12/20/2022, 9:35 PM
Adam Roderick12/20/2022, 10:41 PM
which isn't usable for large datasets. Any other attempts we've done to use similar logic but using a cursor fails. What is it about the prefect execution environment that prevents us from using snowflake cursors?
Samuel Hinton12/21/2022, 2:04 AM
Wingmore12/21/2022, 2:39 AM
Gows Shaik12/21/2022, 5:38 AM
Gows Shaik12/21/2022, 5:40 AM
Malavika S Menon12/21/2022, 6:06 AM
Deepanshu Aggarwal12/21/2022, 6:35 AM
no extra logs nothing, it also leaves the tasks in running state. any idea why this happens ?
Flow run infrastructure exited with non-zero status code -1.
YD12/21/2022, 7:41 AM
which sent me an SMS alert on task failure. how can i achieve this kind of behavior in Prefect 2.0? thanks
YD12/21/2022, 7:51 AM
but I get an error:
What is the proper way to set a timeout time for tasks in Prefect 2 ?
TypeError: task() got an unexpected keyword argument 'timeout_seconds'
Kelvin DeCosta12/21/2022, 9:31 AM
is configured with
, can its result only be consumed via
? Calling such tasks directly, i.e..
, seems to raise the
result = my_task()
Mark NS12/21/2022, 9:48 AM
docker container for execution.
Aleksandr Liadov12/21/2022, 11:15 AM
Jaime Raldua Veuthey12/21/2022, 11:32 AM
Alexandru Anghel12/21/2022, 1:58 PM
For more context, i'll add some code in the thread. Thanks!
Crash detected! Execution was interrupted by an unexpected exception: AttributeError: 'KubeCluster' object has no attribute '__qualname__'
Michał12/21/2022, 2:24 PM
151847.351 | INFO | prefect.agent - Completed submission of flow run '5adc8699-d9a5-4042-85d0-0b072a4873d0' <frozen runpy>128 RuntimeWarning: 'prefect.engine' found in sys.modules after import of package 'prefect', but prior to execution of 'prefect.engine'; this may result in unpredictable behaviour 151848.970 | INFO | Flow run 'fantastic-tamarin' - Hello from demo-flow! 151848.995 | ERROR | Flow run 'fantastic-tamarin' - Crash detected! Execution was interrupted by an unexpected exception: httpx.HTTPStatusError: Client error '422 Unprocessable Entity' for url 'http://prefect-server:4200/api/flow_runs/5adc8699-d9a5-4042-85d0-0b072a4873d0/set_state'
For more information check: <https://httpstatuses.com/422>
Edmund Tian12/21/2022, 3:03 PM
). Anyone know what’s going on? Context: I’m using the
decorators for my Python functions in my app hosted on Google Cloud Run. I’m not using Prefect Cloud. I’m on the following prefect version:
. It’s hard for me to repo because this occurs during my chron job. I haven’t been able to repo locally.
Mark NS12/21/2022, 3:19 PM
David Steiner Sand12/21/2022, 6:12 PM
Jessica Smith12/21/2022, 7:15 PM
Paco Ibañez12/21/2022, 9:46 PM
. What would be the most efficient way of passing the dataframes around?
YD12/21/2022, 9:50 PM
does not work as expected I am trying to see how can I continue with a workflow after a task failed all the allowed retries. Sample code:
I expect this to run
from prefect import task, flow, get_run_logger, allow_failure from time import sleep @task(retries=2, timeout_seconds=3) def test_1(o, test_retries): if test_retries: sleep(5) else: raise IOError('Raising some error') return True @flow(name='test allow_failure') def run_test_flow(): result = test_1.submit(True, True) result = test_1.submit(result, False, wait_for=allow_failure(result)) if __name__ == "__main__": run_test_flow()
, fail twice, since the
is larger than the
, and then run the second
But it does not appear to be working it does not run the second
@User, @User do you have thoughts on this ?
Sam Werbalowsky12/21/2022, 10:24 PM
? Even if the flow is written in a way that there is no dependencies between tasks?
Gows Shaik12/22/2022, 4:37 AM
Jons Cyriac12/22/2022, 5:03 AM
Chris Arderne12/22/2022, 7:56 AM
(more details in thread!) 1. 🏗️ Structuring project: We're using v2 with a GCS Storage block, a KubernetesJob block and a Flow Deployment (all three defined in Python files). Because we're using GCS Storage, the flow files are not
so we can only use relative imports in the flow like
, which can make local development and bugchecking a bit more error-prone. We will probably shift to just using the Docker images local files, but haven't yet experimented with this in Prefect v2. Are there any best-practises around this? (Project structure in thread) 2. 🏃♀️ Running flows: Prefect
from .utils import foo
command in the CLI, which made it possible to run a local flow. Prefect`v2` doesn't seem to have this? We can't add a
prefect run ...
to the Flow file, because it has relative imports so running it doesn't work. As a workaround I've added a
if __name__ == "__main__"
in a parent directory that imports and runs it, but this isn't very ergonomic. In either case if I want to pass parameters, I must create my own CLI for doing so... Have I missed something in the CLI, or is there a plan to add this? Or is there a better pattern I should be using instead?
James Zhang12/22/2022, 10:14 AM