quick question on the Prefect-maintained Orion collections — are there plans to port the 1.0 Jupyter task to Orion?

Hey, quick question, prefect 2.0: I can't figure out how to set time to live or successful/failed job history limits for k8s jobs which were spawned from a kubernetes orion agent. Everything works fine, but I've got hundreds of completed/failed jobs (and corresponding pods) lying around. Is there a way to configure job history limits, or configure automatic cleanup? (Now that I think about it, I could also use a prefect flow to clean up, but maybe that's too much 😄 )

Prefect 2.0 beginner here- Is there a way to retry a subflow in prefect 2.0 upon failure? Or are retries only reserved for tasks?

Hi, its possible naming a schedule of a flow in cloud? I run many schedules of the flow with different parameters but I would like named it to difference each other.

Hello, we’ve been running Prefect for some our jobs for a while now (3 months) and usually runs go through successfully, in our PROD environment every job has been getting this error since yesterday: “No heartbeat detected from the remote task; marking the run as failed.” Any thoughts on how I should go about debugging this?

Hi, trying to run DbtShellTask using command "dbt run" but get such kind of error:

[2022-06-11 08:35:49+0300] ERROR - prefect.CloudTaskRunner | Task 'DbtShellTask': Exception encountered during task execution!
Traceback (most recent call last):
  File "C:\Users\User\PycharmProjects\demo\venv\lib\site-packages\prefect\engine\task_runner.py", line 880, in get_task_run_state
    value = prefect.utilities.executors.run_task_with_timeout(
  File "C:\Users\User\PycharmProjects\demo\venv\lib\site-packages\prefect\utilities\executors.py", line 468, in run_task_with_timeout
    return task.run(*args, **kwargs)  # type: ignore
  File "C:\Users\User\PycharmProjects\demo\venv\lib\site-packages\prefect\utilities\tasks.py", line 456, in method
    return run_method(self, *args, **kwargs)
  File "C:\Users\User\PycharmProjects\demo\venv\lib\site-packages\prefect\tasks\dbt\dbt.py", line 192, in run
    return super(DbtShellTask, self).run(
  File "C:\Users\User\PycharmProjects\demo\venv\lib\site-packages\prefect\utilities\tasks.py", line 456, in method
    return run_method(self, *args, **kwargs)
  File "C:\Users\User\PycharmProjects\demo\venv\lib\site-packages\prefect\tasks\shell.py", line 145, in run
    line = raw_line.decode("utf-8").rstrip()
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xad in position 7: invalid start byte

Any ideas what is the issue?

Hello, i'm stuck in the following issue setting up Prefect + Dask cluster. When the lfow is executed multiple tasks are run on the Dask Worker. When it run i get an exception (see thread)

Hello, I have an async function, which fetches some data from an REST API (using aiohttp). The task returns a pandas DataFrame. A second function saves the pandas DataFrame as a parquet file into a s3 bucket. Running this functions in python is really fast (<1s). However, if I wrap both functions into prefect tasks and place them into a prefect flow, the flow execution takes up to 5 minutes. I do not understand, why the pure python solution os so much fast, compared to when using prefect. I have posted some sample code inside of the thread.

hello -- i have installed prefect-2 but I got this error

Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/prefect/cli/base.py", line 59, in wrapper
    return fn(*args, **kwargs)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/prefect/utilities/asyncio.py", line 120, in wrapper
    return run_async_in_new_loop(async_fn, *args, **kwargs)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/prefect/utilities/asyncio.py", line 67, in run_async_in_new_loop
    return anyio.run(partial(__fn, *args, **kwargs))
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/anyio/_core/_eventloop.py", line 56, in run
    return asynclib.run(func, *args, **backend_options)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/anyio/_backends/_asyncio.py", line 233, in run
    return native_run(wrapper(), debug=debug)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/asyncio/runners.py", line 43, in run
    return loop.run_until_complete(main)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/asyncio/base_events.py", line 584, in run_until_complete
    return future.result()
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/anyio/_backends/_asyncio.py", line 228, in wrapper
    return await func(*args)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/prefect/cli/base.py", line 162, in version
    if is_ephemeral:
UnboundLocalError: local variable 'is_ephemeral' referenced before assignment
An exception occurred.

Hi - just wondering if anyone knows of a way to point studio.apollographql - at the cloud api? It has nice query building and introspection features I would like to use? Not sure how I would do the authentication?

Hello again, do agents work with Azure's Storage Blob Data Reader privilege to pull Flow from Azure Blob Storage?

Could someone help me understand what is happening with this flow_run?

063bd788-bc0c-46f5-85e4-1092c2b19297

Hi all, how can I skip all the tasks in the flow if there is holiday on that day or any particular date? Assuming I already have a list of holidays. What can be the best approach?

Hi all, is there a way I can specify a task to run only once within a flow? I have noticed some of my tasks run concurrently, this is an issue when I am writing to a table

Hi everyone, Whenever you post a question, please let us know whether your question is about Prefect 2.0 or Prefect 1.0. Otherwise, we all have to waste time figuring out which product you are asking about- that's negative engineering - let's eliminate it. Thanks a lot!

Hi all, I'm switching to DockerRun (image on Azure container registry) and GitStorage (github) on Prefect 2.0. I'm running into an ValueError error when attempting to run a flow. It doesn't throw an error on a specific task so I'm not sure where the problem lays. My runs agent on a VM that is authenticated with "docker login myregistry.azurecr.io". The agent shows that, for the run, the image was succesfully pulled. Any pointers where the issue could be? Thanks!

Hi. Unclear if this is the correct forum for this so please correct me if not. I'm evaluating prefect as a long-term solution for my company, and from a cursory glance prefect 1.0 looks like a perfect fit, with the one exception of the dependency on docker which we're not thrilled about. I tested a copy of 2.0 locally and whilst the solution works well the UI leaves a bit to be desired compared to 1.0. Is the plan to completely replace the 1.0 functionality with 2.0 in the future? I understand that DAGs are now out and so no more schematic diagrams and such in the future

By when can we expect to have databricks notebook triggering feature in prefect 2.0 ?

ModuleNotFoundError("No module named '/root/'") , is there anybody meet thish prob. when run. a flow on. k8s agent ?

Can I scale an (1.0) agent to 3x replicas with the same service account / api key? Or does each replica need its own service account and api key?

Apologies if this has already been answered--but in the 'old' Prefect we started up agents with the -p flag so that we can point them to local modules that our flows needed to import. This functionality seems to have gone away with Orion. Is there some analogous way to indicate to the agent the location of local files that need to be imported? For reference, the short stack trace I'm getting is

Flow could not be retrieved from deployment.

Traceback (most recent call last):

File "/tmp/flow-script-calculate-blob-size4cemrmn_.py", line 1, in <module>

ModuleNotFoundError: No module named 'local_module'

This message was deleted.

Hi there - I am running into an issue with the

DbtShellTask

; locally works all as expected. when i deploy to prefect cloud, I am getting a

dbt: command not found

.. requirements include

dbt-redshift

. My flow storage is Docker. when i try to build the Docker image locally, inside the container I also get

dbt: command not found

even though it is installed in my python environment packages. Has anyone run into this before?

Is there a way to make a state handler do something different if a specific task fails? For example, I have a chain of tasks that depend on each other:

a -> b ->c

I want it so that if

a

and

b

fails, it sends a Slack notifications saying “Flow failed”. But if

c

fails, I want it alter the Slack message by adding b’s output

Hi! Is there a difference between placing a Parameter() before and after the flow context?

Follow up to the above thread - we are running into an issue now with connecting dbt to our Redshift warehouse in Cloud... have verified that all the environment variables are working correctly, do we need to install anything explicitly in our Docker image?

01:13:51  Running with dbt=1.1.0
01:13:51  Partial parse save file not found. Starting full parse.
01:13:52  Found 9 models, 75 tests, 0 snapshots, 0 analyses, 196 macros, 0 operations, 0 seed files, 1 source, 0 exposures, 0 metrics
01:13:52
01:13:52  Encountered an error:
Database Error
  could not connect to server: No such file or directory
  	Is the server running locally and accepting
  	connections on Unix domain socket "/var/run/postgresql/.s.PGSQL.5439"?

Hi, I have a question about subclassing the

Result

object. I am doing operations on spark

DataFrames

in

Tasks

and would like to pass a

DataFrame

from

Task

to

Task

via the

Result

object. If the

DataFrame

is too large to fit in memory simply passing a custom serializer won’t work. So I am thinking I could create a subclass of

Result

called

SparkDataFrameResult

where the

read

and

write

methods are overridden to use the spark

load

and

save

methods. 1. Is there any reason this wouldn’t work or wouldn’t be recommended? 2. Are there any best practices for doing something like this?

Hi, How can I execute a map inside a task? Something like this (code in thread):

Hey Prefect team I am using edge function in my prefect work flow here is code