What is the v2 equivalent of a

ShellTask

?

Hi! We are noticing that

build_from_flow

uploads the entire directory to storage rather than just the file that we want to deploy. Is there a workaround for this? We'd like to just upload the single file rather than the entire directory.... Thanks!

Where are the docs for using a docker image for flow storage?

xposting: kube worker is losing connection to server https://prefect-community.slack.com/archives/C048K0MGHNK/p1682211404185689 here are some logs, where the last one is just

ConnectionTerminated error_code:0

Worker 'KubernetesWorker 3fe10498-8e37-4bca-bc38-f1adf48ee842' started!
last_stream_id:19999, additional_data:None>
httpx.RemoteProtocolError: <ConnectionTerminated error_code:0,
raise mapped_exc(message) from exc
line 77, in map_httpcore_exceptions
File "/usr/local/lib/python3.11/site-packages/httpx/_transports/default.py",
self.gen.throw(typ, value, traceback)
File "/usr/local/lib/python3.11/contextlib.py", line 155, in __exit__
with map_httpcore_exceptions():
line 352, in handle_async_request
File "/usr/local/lib/python3.11/site-packages/httpx/_transports/default.py",
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[truncated...]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
runs_response = await self._get_scheduled_flow_runs()
372, in get_and_submit_flow_runs
File "/usr/local/lib/python3.11/site-packages/prefect/workers/base.py", line
await workload()
line 46, in critical_service_loop
File "/usr/local/lib/python3.11/site-packages/prefect/utilities/services.py",
Traceback (most recent call last):
The above exception was the direct cause of the following exception:
last_stream_id:19999, additional_data:None>
httpcore.RemoteProtocolError: <ConnectionTerminated error_code:0,
raise RemoteProtocolError(self._connection_error_event)
139, in handle_async_request
File "/usr/local/lib/python3.11/site-packages/httpcore/_async/http2.py", line
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[truncated...]
File "/usr/local/lib/python3.11/site-packages/httpx/_transports/default.py",
yield
line 60, in map_httpcore_exceptions
File "/usr/local/lib/python3.11/site-packages/httpx/_transports/default.py",
Traceback (most recent call last):
During handling of the above exception, another exception occurred:
state ConnectionState.CLOSED
h2.exceptions.ProtocolError: Invalid input ConnectionInputs.SEND_HEADERS in
raise ProtocolError(
process_input
[truncated...]
During handling of the above exception, another exception occurred:
KeyError: (<ConnectionState.CLOSED: 3>, <ConnectionInputs.SEND_HEADERS: 0>)
~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^
func, target_state = self._transitions[(self.state, input_)]
process_input
File "/usr/local/lib/python3.11/site-packages/h2/connection.py", line 224, in
Traceback (most recent call last):
Examples of recent errors:
Failed the last 3 attempts. Please check your environment and configuration.
Unable to load step function: {fully_qualified_name}. Attempting install of {requires}.

I authenticate via google, but I cannot see my organization. It seems like I have a personal cloud account. How can I join my organization?

Hi everyone! I am working on migrating my prefect cloud 1 workflows to prefect cloud 2. I have an announcement in my account that says my account will be frozen on May 1 but then i can see in the prefect cloud announcements that the freeze will be on may 15nd. Which is the actual date?

Hi, How to use prefect.blocks.notifications to send Microsoft team notification which would display a table of information like artifacts? Please let me know

Is it important to set up pause/resume with non-blocking? Reading the pause/resume docs it says the agent will not be able to pick up other flow runs if the pause/resume is in the default (blocking) configuration. But we have not set up any concurrency limits for our prefect agents, so I'm guessing that means multiple flows can still be running concurrently, even though one is paused/blocking. Is this the correct way to understand this?

I have a question around the prefect cloud deployment build command. so I want to use the --override flag to add s3fs environmental variable, but also add a value for the pod_watch_timeout_seconds. This is what I found in the documentation, ******************************************************************************************************* overrides: List[str] = typer.Option( None, "--override", help=( "One or more optional infrastructure overrides provided as a dot delimited" " path, e.g., `env.env_key=env_value`" ), ******************************************************************************************************* I was wondering if you could provide an example of using multiple arguments in the --override flag because I am getting errors.

@Prefect Hi All is there any Issue at this time that workflows are not processing anymore...appears turning the flow schedule on + off again fixes it. Any idea/Insight.

not to ask a maybe basic question, but where can you see the description of a task on the prefect cloud UI?

hi all, I am hosting prefect cloud on an ubuntu ec2 instance. is it possible to run

prefect agent

in the background? and redirect standard error and output to a log file? something like nohup >> logs

Hello all, I am having problems logging into a workspace I keep getting an error that the code is invalid or expired after requesting a new code. Is anyone else experiencing this?

Hello - we are still experiencing degraded performance and late runs on Prefect Cloud 2.0 even after the issue was said to have been resolved at 22:29 UTC. Is anyone else also experiencing this and is there any update from the Prefect team?

Hi morning. I want to check whether there was an issue with prefect 2 cloud yesterday at around 7 pm London time? All my agent has lost connection to the server and terminated the agent process

Hi, I'm unable to resume my paused flow runs. The api request to

<https://api.prefect.cloud/api/accounts/xxxx/workspaces/yyy/flow_runs/zzz/resume>

is returning a 404

detail	"Not Found"

Moving this post here since this seems more relevant: Has anyone been struggling with getting the agent to now stall on downloading code before running a deployment? I'm downloading a good bit of code, but when I run a manual s3 download on command line, it completes quickly. Also, I can run this code snippet:

from prefect.filesystems import S3
s3_block = S3.load("flow-code")
s3_block.get_directory(from_path="/", local_path=".")

and it also downloads in a couple of seconds however, when I run the prefect-agent, it stalls at the download. It stalls whether I use an S3 storage block or if I try to run it locally. Frustrating because everything was working fine until it just... wasn't anymore. Has anyone else seen this? One difference I see in the code snippet to what is happening in prefect-agent code is that it is being run async, but I didn't think that should be an issue

Hi everyone, i am running databricks jobs from prefect cloud and sometimes (it is completely aleatory) i get this error.

Encountered exception during execution:
Traceback (most recent call last):
  File "/home/ubuntu/.local/lib/python3.10/site-packages/anyio/_core/_sockets.py", line 186, in connect_tcp
    addr_obj = ip_address(remote_host)
  File "/usr/lib/python3.10/ipaddress.py", line 54, in ip_address
    raise ValueError(f'{address!r} does not appear to be an IPv4 or IPv6 address')
ValueError: '<http://dbc-297e31f0-94d8.cloud.databricks.com|dbc-297e31f0-94d8.cloud.databricks.com>' does not appear to be an IPv4 or IPv6 address

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/ubuntu/.local/lib/python3.10/site-packages/httpcore/backends/asyncio.py", line 111, in connect_tcp
    stream: anyio.abc.ByteStream = await anyio.connect_tcp(
  File "/home/ubuntu/.local/lib/python3.10/site-packages/anyio/_core/_sockets.py", line 189, in connect_tcp
    gai_res = await getaddrinfo(
asyncio.exceptions.CancelledError

It is important to say that no matter if the prefect flow run fails or not, the databricks job is successful. Does anyone know what is going on?

ever since this morning, ive been running into issues with scheduled flow runs where not all tasks load. see below for an example of a successful flow run from yesterday vs what is happening today. for whatever reason, only a portion of flow tasks are getting loaded when the flow initiates. its almost like the code isn't getting fully downloaded from local storage. anyone else encountered this?

hey, my worker is dying with a

404, deployment not found

error. and indeed, the deployment does not exist. my question is why is it looking for this deployment in the first place? where does the worker find its list of deployments? it shows as healthy in prefect but keeps crashing in kube

Hi, how do I troubleshoot "late" flow runs? How does one inspect the state of a flow run?

hello prefect, I'm getting this error from the cloud api when trying to kick off a flow run from a deployment:

response: {'detail': 'Orchestration service in maintenance mode. Try again shortly.'}

How can I avoid this?

Hi everyone, i want to start trying this, i want to load some info to snowflake, but i need the outgoing IPs that prefect cloud use to whitelist them, is posible this?

Hi. I ’ve realized about something. When I initialize the project with

docker-git

/

docker-s3

, with

image_name

and

tag

,

prefect.yaml

does have

image_name

and

tag

in null. Is that expected??

Hello, I am very new to prefect. I work in poetry 1.1.8. I installed prefect-aws, and follow the document https://prefecthq.github.io/prefect-aws/, but got error

ModuleNotFoundError: No module named 'prefect.workers'

. I cannot find the solution via google search. Can anyone help?

Hi, I am trying out the method

register_flow

(from

prefect.projects.base

package) and it seems that does nothing at all, because I don’t see any deoloyment created on the workspace (Whatever if it enforces or not the overwritten). Is that what the method has been created for?

Hey all, I still am confused about the

work queue

and

work pool

behavior 🤔 I have created a deployment with work queue

wq1

. I have created a prefect agent with work queue

wq1

. •

Agent started! Looking for work from queue(s): wq1...

When I trigger a flow via API, it is shown as: • work pool:

default-agent-pool

• work queue:

wq1

However, the agent does not pick up that flow run 🤔 Any thoughts or suggestions? PS: I did it similarly before and it worked before, I think. PPS: I redeployed the flow already several times (i.e. whenever I needed to restart the related EC2 instance).

Follow-Up question: I thought that if I deploy a flow locally, the code does not necessarily need to be copied over for an agent to run locally. However, the agent still states

Flow run 'ancient-bat' - Downloading flow code from storage at '/home/ubuntu'

And there even have been download errors. I did not yet find a documentation that clarified what goes on for me 🤔 PS: Downloading takes ages (it is a pretty big conda environment). PPS: It seems like the agent is redownloading the same flow code for each flow run.

Hi, I ’ve been testing

docker-s3

recipe at

prefect project init

command, and I see that the

push-folder

is always the name of the repo where the command runs over (ignoring

folder

field I already provided).