Hi, maybe a stupid question here, I’m new to prefect and have no prior experience with prefect 1.0 - only using the V2 Basically I’m evaluating the product, and have a question Let’s imagine a complex flow composed of 8 tasks (could be subflows) - one of them fails. How do I re-run everything only starting with the failed task/subflow ? Tx

Hi, can I use blocks defined in the cloud for local filesystem runs (developing)? Or do I have to setup all blocks locally, to let it run? I would like to do a local run, but using all blocks from the cloud.

Hi! Not sure if this is the right place for the question. But i am following this tutorial: https://docs.prefect.io/tutorials/deployments/ I have successfully reached the step where i create the deployment with this command:

prefect deployment build ./log_flow.py:log_flow -n log-simple -q test

which generate the yaml file. But it looks slightly different for

infrastructure

where in the tutorial they have:

infrastructure:
  type: process
  env: {}
  labels: {}
  name: null
  command:
  - python
  - -m
  - prefect.engine
  stream_output: true

and i have:

infrastructure:
  type: process
  env: {}
  labels: {}
  name: null
  command: null
  stream_output: true
  block_type_slug: process
  _block_type_slug: process

And when i run

prefect deployment apply log_flow-deployment.yaml

I get this error:

prefect.exceptions.PrefectHTTPStatusError: Client error '403 Forbidden' for url '<prefect_domain>/api/block_types/016c45c9-6346-49c1-a1cb-da3021f915b2'
Response: {'detail': 'protected block types cannot be updated.'}
For more information check: <https://httpstatuses.com/403>
An exception occurred.

Anyone knows what i need to do?

Hi, I have a question regarding queue priorities. I have a requirement that needs flow runs with "high-priority" to jump all other items in the deployment queue - basically switching from FIFO to LIFO. I saw the documentation on the topic but I don't think I would be able to support such use case. My understanding is that I would need two "low-priority" and "high-priority" queues, but if there is any item in the high-priority, no "low-priority" flow run should start, and I don't think it can be done easily. Are priorities within the same queue something that would be supported in the future, or other ways to work it around? Thanks!

Hi guys, when installing Prefect 1.* with the rss extra, Python 3.10.6 and Pip 22.2.2 I am getting

INFO: pip is looking at multiple versions of feedparser to determine which version is compatible with other requirements. This could take a while.

Collecting feedparser>=5.0.1

Using cached feedparser-6.0.10-py3-none-any.whl (81 kB)

Using cached feedparser-6.0.9-py3-none-any.whl (81 kB)

Using cached feedparser-6.0.8-py3-none-any.whl (81 kB)

Using cached feedparser-6.0.7-py3-none-any.whl (81 kB)

Using cached feedparser-6.0.6-py3-none-any.whl (80 kB)

Using cached feedparser-6.0.5-py3-none-any.whl (80 kB)

Using cached feedparser-6.0.4-py3-none-any.whl (80 kB)

INFO: pip is looking at multiple versions of feedparser to determine which version is compatible with other requirements. This could take a while.

Using cached feedparser-6.0.3-py3-none-any.whl (80 kB)

Using cached feedparser-6.0.2-py3-none-any.whl (80 kB)

Using cached feedparser-5.2.1.zip (1.2 MB)

Preparing metadata (setup.py) ... error

error: subprocess-exited-with-error

× python setup.py egg_info did not run successfully.

│ exit code: 1

╰─> [1 lines of output]

error in feedparser setup command: use_2to3 is invalid.

[end of output]

1. How can I solve this? 2. Can’t we upgrade the feedparser requirement?

Hi there. I was wondering about the significance of agents in Orion. In Prefect 1 we use agents for different flow-run configurations, in particular resource availability. We created five agents:

agent-default

,

agent-small

,

agent-medium

,

agent-large

and

agent-huge

, each with different amounts of available RAM and CPU. Now that we with Orion can set

KubernetesJob

-infrastructure blocks in the

Deployment

-object, having different agents or queues for the different resource tiers no longer seems necessary. Does anyone know of examples where agents/queues have been used explicitly? I noticed for examples the documentation mention agent queue priorities

for how long the prefect 1.x cloud remains supported?

Hi, I’m having trouble using cache with tasks that take blocks as parameters. It seems that the

task_input_hash

hashes the block differently on each flow run, and therefore causing cache misses. I’ll attach code snippets needed to reproduce the issue in this thread.

Struggling with Helm, running these commands

helm install --values k8s-agent.yaml prefect-agent prefect/prefect-agent && \
helm install --values k8s-orion.yaml prefect-orion prefect/prefect-orion

Hi everyone! Is there a good way to cancel a flow run using the REST API / prefect client with Prefect 2.0? Is this what the

set_state

flow run endpoint would be used for? https://docs.prefect.io/api-ref/rest-api/#/Flow%20Runs/set_flow_run_state_flow_runs__id__set_state_post

How do I test a task using

.fn

that contains a call to

get_run_logger

. I get a

RuntimeError: There is no active flow or task run context

exception

Hello, can someone please point me to the code that sets the default label on an Agent? Note that I am also not on Prefect 2.0 yet 😕

@Christopher Boyd the same issue again with flows crashing started to happen a few minutes ago:

raise PrefectHTTPStatusError.from_httpx_error(exc) from exc.__cause__
prefect.exceptions.PrefectHTTPStatusError: Client error '403 Forbidden' for url '<https://api.prefect.cloud/api/accounts/43db7ccd-9f39-41f2-8989-000b28747858/workspaces/cedd89e9-9f12-421e-a17b-94045c976a2a/block_types/e91e7544-7ecd-4fa6-b6d7-a53068cb67fc>'
Response: {'detail': 'protected block types cannot be updated.'}
For more information check: <https://httpstatuses.com/403>

this previously happened and a fix was rushed

It is possible to run the agent in 2.0 in a python file instead of doing

prefect agent start  --work-queue blabla

Hello Prefect, I try to use the SMB block, when I try to

get_directory

I got

Exception has occurred: TypeError
object of type 'PosixPath' has no len()

Do you know how to use SMB block ? I also get this issue https://github.com/PrefectHQ/prefect/issues/6971 when I try to

put_directory

Thanks 🙂

Is there anything we need to do to enable the new Slack notification in the

2.4.1

release which include the Flow run URL on Cloud? Our notifications seem to still be coming through in the “old style” i.e.

Hello, I have high level design question: It is advisable to share the flow code between two deployments of the same flow? details in thread

Good afternoon - starting this past Sunday morning (September 25th) we started receiving this error message when running flows. These flows have been running for several months without issue. There have been no changes to the flows, environment, versions, etc. We are running Prefect 1.0, Python 3.7 I have tried this on these images - prefecthq/prefect:0.15.13-python3.7 prefecthq/prefect:latest-python3.7 any thoughts?

I am running an agent through supervisorin a remote server/ I am specifying the path for prefect to run.

command=/home/ubuntu/project/env/bin/prefect agent start --work-queue "my queue"

This works fine but then when i run flows in the UI the worker is using the global python runtime and no the one in the virtual environment, please help

If I wanted to parameterize a flow, how would I do that? I'd like to pass an array of strings (email addresses). The flow below works, but I'm not sure it is the proper way to do things. Please ignore the class name, just trying to test.

from pydantic import BaseModel


class CustomList(BaseModel):
    data: list[str]


@flow(name = "Fidelity Allocations")
def FidelityAllocationsFlow(toEmail: CustomList):
   #code....

if __name__ == "__main__":
    emailAddress = CustomList(data=['<mailto:cgunderson@spiderrockadvisors.com|cgunderson@spiderrockadvisors.com>'])
    FidelityAllocationsFlow(toEmail=emailAddress)

Hey all - having some Twitch API issues so we'll be live on YouTube only: https://www.youtube.com/c/PrefectIO

Hi All, Running prefect server which is to be accessed via load balancer what URLs should be used in config. Should it be load balancer url or host and port. Same question for agent. Thanks

Hey, on prefect 2 cloud I randomly encounter "internal server error"s when the agent wants to communicate with the server (exception see in thread). I have a flow which spawns up to 50 Tasks (the tasks are all done within seconds). I run the flow currently every 5 minutes. Am I running in some sort of api limitation due to too many tasks? Is there something to prevent this internal server error from happening? This error happens approx. every 15th time, all other times, the flow runs just fine.

File "/usr/local/lib/python3.9/site-packages/prefect/client.py", line 226, in raise_for_status
    raise PrefectHTTPStatusError.from_httpx_error(exc) from exc.__cause__
prefect.exceptions.PrefectHTTPStatusError: Server error '500 Internal Server Error' for url '<https://api.prefect.cloud/api/accounts/bd169b15-9cf0-41df-9e46-2233ca3fcfba/workspaces/f507fe51-4c9f-400d-8861-ccfaf33b13e4/task_runs/|https://api.prefect.cloud/api/accounts/bd169b15-9cf0-41df-9e46-2233ca3fcfba/workspaces/f507fe51-4c9f-400d-8861-ccfaf33b13e4/task_runs/>'
Response: {'exception_message': 'Internal Server Error'}

Hello, with several long running flows I get an error like:

prefect.infrastructure.process - Process 'meticulous-manatee' exited with status code: -9

Any guidance?

hi there, in prefect 1.0 we could dynamically assign the task names with input params, something like:

@task(name="extract(ctx.id)")
def extract(ctx):

These would then be displayed in the UI as

extract(4)

for example if our flow was running for ctx.id=4. I haven't been able to find anything like this for 2.0, am I just missing something?

What is the relationship between an agent running on kubernetes and a kubernetesJob defined in infrastructure... • Does the kubernetes agent start the kubernetes job? ◦ if so, do I need a kubeconfig in the infrastructure block when the agent is already on the cluster? • If I run the job in an infrastructure process block from a kubernetes agent, will it run on the kubernetes agent?

Hey team. Can someone help with releasing the latest version of prefect-gcp? https://github.com/PrefectHQ/prefect-gcp/pull/61 The current version has a bug with credentials that I think will be solved with the commit from yesterday by @Andrew Huang

I’m getting the following error:

AttributeError: 'Credentials' object has no attribute 'project'. Did you mean: 'project_id'?

It looks like your PR will fix this @Andrew Huang.

Has anyone here tried running pyspark inside a prefect task? I get

RuntimeError: Java gateway process exited before sending its port number

when I run pyspark methods inside a task.