Hi, question about deployments: Currently I build and apply deployments using the CLI. However, there are some things I can't set with CLI (like description), and also, specifying all arguments on a command line (in a bash script) is way less readable than yaml. I would love to use yaml, but the problem with it is that all changes I make to it (like a custom description I add) will get overwritten every time I re-generate the build file. Is there a way to make Prefect take into account existing build file?

We currently run (on Prefect 1) our flows in a custom docker container based off of the prefect image. The image lives in our gitlab container registry. Our codebase, data input and output directories are all exposed to the container as volumes. It seems like the recommended way to achieve this in Prefect 2 is documented here. I'm curious if this is correct route and if so what exactly would be stored in blocks? Why is block storage required?

Help! This is going to be a rare case 😕 Been scratching my head for the past few days on this one: Flows dont run properly if I include a certain custom import. However when I comment it out, it works?? Note this is also only affecting deployed flows. If I run the flow locally, everything's ok...

Hi everyone, asking again. Is there a way to pass pod memory/cpu limits/requests while building deployments using CLI?

Hi! can we use BRIN index for timestamp instead of btree in log table? when the table gets huge, queries are getting real slow.

Hey everyone 🖖, so is Prefect 2 actually under • https://www.prefect.io/legal/prefect-community-license/ • or Apache License 2? As stated here https://docs.prefect.io/faq/ We are basically unsure to what degree we might be able to use (and thus sell) prefect in our client projects. Greetings

Hey team! I was wondering if there are any recommended best practises for how to name flows/deployments? Currently if we have a flow named

test_flow_1

then we’d call our deployment

test_flow_1/test_flow_1_deployment

which is a bit redundant. Is it recommend to have unique deployments names? Will there be any issues if we have many deployments named the same thing

Is it possible to visualize task outputs in Prefect 2? Both Kubeflow and Flyte support rendering HTML / Markup in the steps, so it is possible to render all kinds of reports and have them displayed. Is this also possible in prefect?

Working on migrating to Prefect 2. In prefect 1, we built separate images for each flow and we’d like to tighten that up in our prefect 2 implementation. I really like the idea of github storage, but in our environment, we have a lot of unrelated, disparate flows with different dependencies. We can’t really predict the last 2% of dependencies that might be needed for any give flow. What are some ways to work around that if we used Github storage?

What is the easiest way to share company internal flows? Is it necessary to publish them to Pypi (a private one?) or are there other way?

Hi! For those using Prefect Cloud, where would one find one's ACCOUNT_ID?

If I create write to a file in a flow and then in a sub flow create another file, will these files all be available to until the top level flow finishes? Running on a

kubernetes-job

Hi everyone, I tried adding customization to the deployment (see screenshot for reference). Deployment creates successfully and I am able to run the flow. When I describe the pod, it doesn't show the memory limits/requests I set using customizations. I went through the code at

prefect/src/prefect/agent.py

under

get_infrastructure

overrides are added to an object called data but that's not used afterwards to update the infra instead infra_dict was used which was used to create data in the earlier steps. Can this be a reason of missing customizations I am trying to send via infra_overrides? Or am I looking at it incorrectly.

Hi it seems that with the latest version of prefect 2.7.10

prefect-airbyte

is not working anymore I got his error

trigger_manual_sync_connection
    raise err.AirbyteServerNotHealthyException() from e
prefect_airbyte.exceptions.AirbyteServerNotHealthyException

Even if the airbyte connection is properly triggered in airbyte.

Hi! Maybe someone can help? https://github.com/PrefectHQ/prefect/discussions/8286

Hi! I hope it is well. I have a problem, I need rum a flow in the specifics days 1 of January and 25th of July, but I don't know how can organize a schedule for these 2 days, actually i'm use prefect 2.6.7 this work, but just for a 25 of July:

from prefect.orion.schemas.schedules import CronSchedule

schedule=CronSchedule(cron="0 0 25 6 *", timezone="America/Sao_Paulo"),

Hi Prefect, I have a question with regards to automation. If we have an automation kick-off another deployment to run, how do we get the id of the original deployment that entered the specified state ? Is that a parameter we can get somewhere ? or using the API ? Would appreciate your inputs there Tx !

🥁 We’re so excited to announce that our second Prefect Associate Certification Course (PACC) of 2022 will be held in Washington DC! 🏛️ Come get hands on training from our Prefect engineers Jeff Hale, Peyton Runyan, Taylor Curran, Bianca Hoch,Bill Palombi, and our CEO Jeremiah Lowin 🙌 Register today! P.S. We have upcoming PACCs in New York City, San Francisco, and London.

Hello! I'm encountering errors when passing a non-JSON-serializable object, an AnnData, into my flows or from flow to subflow. If an object isn't serializable as JSON, does that mean it can't be passed to flows/subflows cloud deployments? Anyway, any solutions or workarounds are appreciated!

Was there any update on this ticket? (passing DB connections as parameters between tasks) - https://github.com/PrefectHQ/prefect/issues/5866

Not obvious how I get the current status of a flow run via the REST API

I’m having trouble retrieving flow run history in 1.3. This used to return all the history (many, many objects for a flow), but now it is only returning 1 result. The UI is in a weird state where the run history is also only displaying one row, but if I drill in I see the actual history which has many successful flows

query {
  flow_run(where: {flow: { id: {_eq: "7c5e836d-cedb-4d05-b566-c0fb6d6d9c0a"}},
           state: {_nin: ["Scheduled", "Running", "Cancelling"]}},
           order_by: {scheduled_start_time: desc} limit: 500 ) 
    {   
    id
    name
    state
    start_time
    end_time
    flow { name id }  }
}

image.png

The second run history row above only lists one flow but drilling in will show many succesful flows. I guess the API is returning what this page is.

I’ve been using this same call for months without this issue in prefect 1.3.

I scraped the call from the UI on the flow page that is working and it sends this request:

--data-raw '[{"operationName":"TableFlowRuns","variables":{"limit":15,"name":null,"offset":0,"state":null,"orderBy":{"scheduled_start_time":"desc"},"flow_group_id":"84054e78-22a4-478b-9394-7efad0ebcf9e"},"query":"query TableFlowRuns($name: String, $limit: Int, $offset: Int, $orderBy: [flow_run_order_by!], $flow_group_id: uuid, $flow_id: uuid, $state: [String!]) {\n  flow_run(\n    where: {flow: {flow_group_id: {_eq: $flow_group_id}, id: {_eq: $flow_id}}, name: {_ilike: $name}, state: {_in: $state}}\n    order_by: $orderBy\n    limit: $limit\n    offset: $offset\n  ) {\n    id\n    name\n    end_time\n    scheduled_start_time\n    start_time\n    state\n    __typename\n  }\n}\n"}]'

Any idea why the first call is only returning one result? Which corresponds to the incorrect UI state?

Is there a way to translate the second call to graphql ?

Howdy I'm trying to understand how prefect task are ran on ECS. Does Prefect 2.0 do a topological sort and run the task in parallel if able?

Hi everyone, I am using docker images with KubernetesJob to run my flows. Wanted to know in which cases should I be creating/updating the deployments if I make changes in my code

Hi, I have some thought about versioning and deployment in Prefect and would love to hear some opinions on it: Basically, the usual deployment/versioning I use is like this: to release a new version of it I create a release on Github, this then builds the image of the service and uploads it to repository, using git tag as the version. Then, I update my terraform files for whatever environment I want (dev/staging/prod), and the old service will be replaced by the new one. easy. With prefect, it's a bit more complicated. There are "versions" for flows/deployments. but they are merely attributes/markers. I can not create a Deployment and say "use this flow, with version X.Y", because the version of the flow is pinned down by the Flow object I need to pass in. So the only solution I see here is to bundle the code of the flows into the Agent image (which I need to create anyway due to dependencies), and version this. Then I just update the agent container version in terraform, and I can make sure that the correct version of dpleoyment will be executed. But there is still a problem of how to update deployments easily on prefect server - this unfortunately does not solve it. One more option is to split flow code and deployment code into two different repositories. Then, flow repository can be nicely versioned, and in deployment repo I can create a separate folder for each environment - each environment can then pin down the version of the flows repo it wants to use. It works, but will get messy when I have more flows and would want to version them independently (now I would need a separate repo for each flow, or use some complicated tag scheme). TL;DR: bundling versions/deployment as is done currently in Prefect makes deployment of deployments/flows to multiple environments really clunky.