Hi everyone, 👋 I'm participant ongoing data talks/mlops-zoomcamp and we're using prefect for workflow orchestration. I'm aware there new release. Fully readed changelogs and prefect 2.0 guides. But I'm having problem prefect deployment with mlflow artifacts. When I run my training script(with prefect task/flow) artifacts saves without any problem. But when I create deployment and start agent mlflow doesn't saves artifacts. Agent shows my all tasks and flow without any error. Happy new release! 🎉

Hi everyone! Congrats on the release of Prefect Cloud v2 🎉 I have a question about the lifecycle of Prefect Cloud. When is it a time limit (EOL) to use Prefect Cloud v1? I am currently using v1 and would like to move to v2 at some point.

Hi, Not sure if I missed a setting a something in v1, please let me know if that's the case... But, I was wondering if v2 is better (or will be) at managing its database size - in particular in regards to db partitions and retention policies? With about 1,500 flow runs / 100,000 task runs a day, we seem to be accumulating data rather quickly. About 3 months of running on v1 have given us:

+--------------+-------------+------------------+-------------------+
|table_name    |relation_size|indexes_sizes_size|total_relation_size|
+--------------+-------------+------------------+-------------------+
|log           |15 GB        |9060 MB           |24 GB              |
|task_run      |5765 MB      |17 GB             |23 GB              |
|task_run_state|13 GB        |6308 MB           |19 GB              |
|flow_run      |241 MB       |472 MB            |713 MB             |
|flow_run_state|438 MB       |118 MB            |557 MB             |
+--------------+-------------+------------------+-------------------+

I've been searching for a python workflow management and I've found the Prefect website. can I use Prefect 2.0 locally without cloud services for free without any user limitations? Is the paid subscription for using the Prefect cloud?

Morning, quick question does prefect have to be installed where the agent is running from? If so does it not create 2 database instances?. When prefect is installed on host running the webservice the .prefect directory and orion.db instance gets created. And then when installing prefect again on the host from where the flows are being run then another .prefect directory and orion.db instance gets created?

Hi (-: A very quick one. I am running a Local Agent in a Render service:

The agent seems to start successfully and listen to flows on a queue I’ve defined.

the queue name is “work-queue”. I’ve also defined the needed PREFECT_API_KEY and PREFECT_API_URL with my account and workspace in Render’s env.

(if you don’t know Render, it’s an Heroku alternative)

But when I start a flow via Prefect Cloud UI 2.0, the agent won’t receive the flow:

this local agent runs on Render on a web service. Maybe I need to open some ports? or install it on a different type of service in Render? On the other hand if you look at the logs above it dose say “Agent started! Looking for work from queue(s): work-queue...” meaning it might succeeded contacting the Prefect Cloud.

Any help would me much appreciated.

Concerning the latest changes in versions >2.0.4 : previously agents were able to find and place flows into queues by tags. From version > 2.0.4 the doc says queue names are used instead. Specifically here (https://docs.prefect.io/concepts/deployments/) it s said:

A deployment additionally enables you to:
- Assign a work queue name to delegate deployment flow runs to work queues.

But in the deployment section there is no info on how to declare a queue to be associated with a deployment. Could you please clarify?

In future versions it is expected to pass parameters on the command line

prefect deployment run command

? as plain text or reading them from file? thanks in advance

I may have bumped into a new bug: It seems none of my schedules are triggering anymore on 2.1.1, neither ones I've created with the --cron argument nor ones I've created manually in the GUI. The flow runs show up as scheduled and then late. They are never processed by any agents (nothing shows up in their logs) and they do not show up under "upcoming runs" for the work queues. I can run all of my deployments by triggering them manually, it's only the scheduled runes that don't get picked up. Does anyone have working schedules in 2.1.1?

hey people! really need your help 🙏 I am trying to build a

flow

, that should execute some tasks at the same cluster. It means that if the

flow

get a new message, it should use the same workers (cluster) as for previous messages instead of creating new ones. do you have any idea how to solve my case (coz I don't)?

Hey all, I’m currently looking into using Prefect to leverage dbt’s model graph. Ideally, we want to use Prefect’s 2 dynamic flow creation to generate a flow for each node of the dbt graph, however, we’re not being able to launch several flows to run at the same time. We’re definitely missing something, but couldn’t find it in the new docs. Can someone provide some insight? I’m running Prefect locally

Are there any plans for versioning deployments in Prefect 2 similar to how registered flows are versioned in Prefect 1? Currently the previous deployment just gets overridden, and that is including all source files uploaded to storage. Related to this - what about having two deployments of the same flow, but with different schedules. Currently each deployment gets uploaded to the storage, which means duplicate file transfer and 2 versions in S3 if bucket versioning is enabled

Hi folks! Just a quick one - I'm trying to delete some old projects. This is on 0.x. I'm getting GraphQL timeouts when trying to delete them and wondered if there's another way to get them deleted? I've turned off all the schedules, I'd just really like to get rid of them to avoid confusion / UI clutter.

Hi all, I am using prefect 2.1.1 trying to build a deployment through the provided python object [

from prefect.deployments import Deployment

]. I am not sure about how to get

infrastructure block

and

storage block

(created through the UI) added to the Deployment. Is this feature not included in the object?

Prefect 2 question. Remote storage. I'm trying to build & deploy pipeline according to documentation https://docs.prefect.io/concepts/deployments/#build-the-deployment. I run command as explain in doc. And I see that Prefect put in bucket WHOLE folder that I'm in (whole workdir) - take a look at the screen. If I will run command from my $HOME Path it will upload whole HOME path in remote bucket? If think.. by logic, it should upload only flow, or flow dir - path that defined in command, but not all environment around. Should I open issue for that? or it there is any idea under that behavior?

Hello everyone, Has anyone attempted accessing a public s3 bucket using the S3 block or know of a work around? Had a good search around but can't seem to find how you could change the S3 validation.

in prefect cloud 1.0 how can we find the tasks that block a task concurrency limit?

We are hoping to use Parameters to pass in a different value for one of our flows (through the UI and also through the API). Is it best to just have a default parameter value in the flow definition at first so it will change if we pass another value through? I can’t seem to find in the docs how to grab the parameter in the flow that is passed through especially if we create a flow run with the API. We are using prefect 1.0 atm

Hi all, Unsure if this is a common issue or a configuration issue on my side. Getting the following issue:

ValueError: 'api.prefect.cloud' does not appear to be an IPv4 or IPv6 address

Complete error log in comment.

What is the idiomatically Prefect (2.0) way of handling failed tasks? My task raises an exception that bubbles all the way when I run my flow locally. I’d like to just be able to have the flow finish with a failed state and not see the whole stack trace?

Anyone have favorite built in prefect tasks? Just noticed there was an sftp one after i implemented it myself

how to stop the async task runs in flow context? I need to execute a loop in the flow context but I don’t want it to execute the next loop before the first loop is finished. I see the tasks are executed asynchrously. For example:

with Flow(
    name="ETL",
) as flow:
    page = 0
    max_page = 999999
    while page <= max_page:
        page += 1
        dummy_task()