Regarding DBT Tasks, where should the models be stored? Can they be subdirectories of the flows/ directory where I have my hello_dbt example?

Hi all, I'm looking for some help with the following error: Failed to load and execute flow run: ModuleNotFoundError("No module named 'pyodbc'") We are running an Ec2 instance with the Prefect Docker Agent. --How do we install the Python modules? Is it needed on the Ec2 instance, (which we did) or is there a way to load it into the container? Thanks for your assistance and support.

Hi Everyone. I'm trying to deploy my Prefect project on GCP using Google Cloud Task for BigQuery. Is there any documentation or tutorial with a similar case using GCP and Prefect Orion?

Hi, how can we rename a tenant on Prefect Cloud?

Hello, does anyone have any experience setting up Prefect Orion to be accessed via a web URL (HTTPs)?

Hi All, In Prefect 2.0(server and agent running locally in docker containers) do I still need to create tenant? it seems some commands from Prefect <2.0 doesn't work in Prefect 2.0.

#Prefect 2.0 Any Idea why I'm getting this error while starting prefect agent in docker. Prefect 2.0?

Hello everyone i'm trying prefect for my job and was wondering if someone has a resource that show how i can use the service within AWS please much appreciated.

Hello, i want to overwrite this configuration by using environment variables.So which one is correct? Thanks!

export PREFECT__SERVICES__SERVICES__TOWEL__MAX__SCHEDULED__RUNS__PER__FLOW=1
or
export PREFECT__MAX__SCHEDULED__RUNS__PER__FLOW=1

we installed prefect on Google kubernetes cluster

Hey Yall. So we have The pattern of a Driver flow that calls child flows. Its working wonderfully where we have the create and the the Waitfor... however as the driver grows to control more flows... our waitfors are starting to error saying that the flow didnt start within 15 seconds. Once this errors... all the dependancy flows dont kick off, so its getting rather annoying. We are setting the number of workers... which we had at 10.. now lowering to 4 just to test it out... With unlimitted flow concurrency, shouldnt our flows be starting faster and not hitting that 15 second wall?

Hey all, currently we have some reusable tasks that at the minute are getting repeated alot in the code. I was wondering if it's possible to do something like this (currently doesn't work, but wondering if there is way round it), we're using prefect 1:

def build_full_refresh_base(name, s3_dir_prefix, **flow_kwargs):
    with Flow(name, **flow_kwargs) as flow:

        repeat_task_output = repeat_task()
    return flow, repeat_task_output


@contextmanager
def FullRefreshFlow(name, s3_dir_prefix, **flow_kwargs):
    flow = build_full_refresh_base(name, s3_dir_prefix, **flow_kwargs)
    try:
        yield flow
    finally:
        print("do something")


if __name__ == "__main__":
    @task(log_stdout=True)
    def some_task(repeat_task_output):
        print(repeat_task_output)

    with FullRefreshFlow("hello", "some_dir") as (flow, repeat_task_output):
        some_task(repeat_task_output)

    flow.run()

I need to run and wait on the output of a docker container as part of my Prefect flow. I see that in Prefect 1.x there is an API for running a docker container and waiting on its result. I don't see a similar API in Prefect 2. Is that coming? Should we use

docker

module for Python instead? Or is there a better way?

Hello, I’ve got a bit of an issue that would love some feedback on. There was an initial bug reported in our application where the graphql query to Prefect for a Flow, given the project and flow name, was returning None. Using the Interactive API, I found that the project and flow name for that given Flow existed and got a return. This code has not been touched in months and just recently started failing. Diving further, I noticed our ECS agents have not queried for flows to execute in 3 days, which is when the bug was reported. I restarted our Fargate Services where our ECS agents live, and the agents are still not querying for flows to execute, despite successfully running and ‘waiting for flow runs’. Any idea what’s going on or how to proceed with debugging further?

Which channel is best to ask a newbie question?

This message was deleted.

@Anna Geller or @Kevin Kho Any update on this issue? We’re running into the same issues with no logs appearing when using Prefect with LocalDaskExecutors on a Docker Agent in “processes” mode https://github.com/PrefectHQ/prefect/issues/5769

I feel like I have a basic version problem or something. I created an account in cloud.prefect.io, created an API key and then tried to use it in the CLI on an Ubuntu machine. 'prefect cloud login -k MYKEY' gives me an auth error. Lots of docs say perfect auth login etc but my cli (2.0b6) says auth is not a valid command. Help! What else do you need to know? Am I running the wrong version of CLI?

Hi everyone! I’ve registered a few new flows last week, I can’t seem to be able to trigger a flow run for those through GraphQL. (We’re using a rails application to trigger flow run requests, that app uses graphql and a flow’s version group id. It works fine for a bunch of other, previously created flows.) The requests get sent successfully by our app but nothing happens on prefect cloud. Is there somewhere a place where I can look up all flow run requests a tenant has received? I need some kind of logs to be able to debug this 🤔

Hi team, In Prefect Orion, how to specific postgres connection url with specific schema (instead of using public)? I am trying to use postgres instead of SQLite.

is anyone who uses

AirbyteConnectionTask

seeing a new error starting ~last week? all of my runs are failing with

Error during execution of task: KeyError('schedule')

. haven't changed a thing in my prefect flow configuration. thought maybe it was related to an airbyte upgrade (im running the self-hosted version), but im struggling to troubleshoot. any ideas?

Is there a way to make the

Overview

tab in the GUI for a given flow always display the actual date instead of "Last Sunday" or whatever?

How do Retries work with Dependent Flows? I added retries to the task from the parent flow, but they don't seem to be firing?

Hey, out of nowhere I have started getting errors when trying to run my flow locally from the cli with

prefect run -p flows/harness_racing_victoria/harness_racing_victoria_results.py

error trace in 🧵 Any ideas?

Hello, community 👋 The case is: I have a project contained several flows. Base environment (whereas prefect server is running, local) contains no extra dependencies (just prefect lib installed). Local agent and docker agent are running. • Flow A includes f.e. pandas and it runs in docker image (screenshot 1), docker agent assigned • Flow B includes no extra dependencies When I’m trying to register Flow A (suppose in base environment). error

No module named 'pandas'

occurred. Flow B is ok. So the question is: what is the best practice to register flows with different environments and dependencies? Hope for your support and thank you!

Hi, I was looking into Prefect Orion and have a question: I want my flow to get triggered every time a new file gets uploaded to a blob storage. Also the name of the file should be passed to the flow as a parameter. What would be the best way to solve this? Creating a new deployment every time seems a bit overkill to me.

Is there a direct equivalent of ECS Agent in Prefect Orion?

Hello, all. Is it possible to implement the saga pattern with Prefect? and if so do we have an example? I was wondering if prefect is a good fit for orchestrating services.

Is there a neat way to execute undo's in a flow if one task in the chain fails? For example, with this flow:

TaskA: resultA -> TaskB: resultB -> TaskC: resultC

if task B fails, I want to run:

UndoTaskA (resultA)

if task C fails, I want to run:

UndoTaskB (resultB) -> UndoTaskA (result A)

I thought I got the gist of subflows, but as I read the docs (concepts specifically: https://orion-docs.prefect.io/concepts/flows/#subflows) I am starting to think I am wrong again. So the use case is the following:

@task
def for_loop_B(category):
  for item_i in category.items:
     load_item(item_i)
     store_item(item_i)

@flow
def for_loop_A()
  categories = retrieve_all_categories()
  for category_i in categories:
    for_loop_B(category_i)

Doing the above seemed wrong, because also (by default at least) tasks or flows would fail just because one internal step failed. Additionally, the processing of the categories is fully independent. So I thought,

for_loop_B

should also be a flow, and

load_item

and

store_item

should be tasks for example. However, I got confused because of what the docs mention: “Unlike tasks, subflows will block until completion with all task runners.“. I want the processing of items within a category to happen sequentially, but multiple categories can be processed in parallel, how can I make that happen? Also, can tasks be nested within tasks for that matter?