@Marvin How to run prefect as a script without connecting to the server in prefect 3

@Marvin I set a job watch timeout of 300 second. I then get an error in the dashboard: Job 'test-01': Job did not complete within timeout of 300s. Why didn't the flow stop after this error? I'm on prefect 2.19

Hello, I just have a quick question. Can I use Prefect without spawning a server? I am looking for a tool to run a one-off workflow (think a glorified

make

), and wondering if Prefect is a good fit

I’m wondering if and how prefect could be used to transfer large amounts of data between different servers/clouds. Basically, I’d need to move 10-100 gigabyte csv/Jason files from an SFTP server to ADLS, and later on between other sources and sinks. Moving this amount of data as a one gigantic in memory string between tasks does not seem very sound approach for many reasons. So how would you actually do that?

@Marvin How to set Kubernetes Job resource limits on Prefect3

@Marvin How can I trigger dependent flow runs after a flow run has complete in Prefect 2.X

@Marvin all of my runs are suddenly stuck into "late" status. What's going on and how to debug the issue?

We’re having trouble using Blocks with SecretStr on the latest prefect (2.14.5) with pydantic 2 (2.5.1). When I paste the example from the prefect docs into a ipython session, I get an error:

from typing import Optional

from prefect.blocks.core import Block
from pydantic import SecretStr

class AWSCredentials(Block):
    aws_access_key_id: Optional[str] = None
    aws_secret_access_key: Optional[SecretStr] = None
    aws_session_token: Optional[str] = None
    profile_name: Optional[str] = None
    region_name: Optional[str] = None

RuntimeError: no validator found for <class 'pydantic.types.SecretStr'>, see `arbitrary_types_allowed` in Config

Anyone have any ideas? I know pinning

pydantic<2

will make the error go away, but since Prefect supports pydantic 2 I’m trying to understand if this is a Prefect bug or if we need to do something differently with our blocks on pydantic 2.

@Marvin What might be causing a

Flow could not be submitted due to infrastructure.

error?

I must be missing something simple… I have two production servers … we will call them Primary and Prefect … as you can guess the Prefect server houses my agent and accepts 10% of the web traffic. The Primary server accepts 90% of the web traffic and has at least one

run_deployment

specified in one of it’s routes. When posting to that URL I get an error and it seems like it is looking for the deployment on a local copy of Prefect Server (previously Orion) instead of Prefect Cloud which is what I’m using… Here’s the error without the stack trace…

prefect.exceptions.PrefectHTTPStatusError: Client error '404 Not Found' for url '<http://ephemeral-prefect/api/deployments/name/alert-early-intervention/alert_early_intervention>'

… I have confirmed that the deployment exists in my Prefect Cloud under the name

alert-early-intervention/alert_early_intervention

.