Hi, we are considering to upgrade our prefect server to 2.0. I see that it’s still not production-ready. What’s the timeline that it’ll be ready for production work? What are the consideration if we still do decide to use 2.0?

Hi Folks, I'm finding myself in a pickle I was in before, where I'm trying to just pull a secret from my cloud setup, and I did this in my python script in the last place I ran prefect, but I can't get it to work. While I was developing, I was able to simply set the use_local_secrets = False, and I could just pull down a secret, but I can't get it to work anymore. I only see one short blurb about getting cloud secrets in the prefect API docs around secrets. Can someone help me with the right commands? I'm getting a message that it keeps looking for it locally

Hi Everyone, we're using Prefect Server on premise V1.2.2 hosted on Kubernetes, and everything is working perfect. However, every now and then we get the occasional error

Error during execution of task: ConnectTimeout(MaxRetryError("HTTPConnectionPool(host='prefect-apollo.prefect', port=4200): Max retries exceeded with url: /graphql (Caused by ConnectTimeoutError(<urllib3.connection.HTTPConnection object at 0x7f32b0d48910>, 'Connection to prefect-apollo.prefect timed out. (connect timeout=60)'))"))

This seems to affect all running tasks at the time, so for example if I have 2 or 3 tasks running, they all fail at the exact same time and give this error

message has been deleted

Hello! I am trying Prefect2.0 for a simple flow from a docker container. I am getting an error and am stuck trying to resolve it - MissingContextError (“No settings context found”) Is there an explicit initial config required? Can I run flows from within a container? Any help would be really appreciated! Thanks!

Hi. I'm looking at Prefect 2.0. The documentation shows support for S3 for storage. By chance, does this S3 support extend to a self-hosted S3-compliant service, such as MinIO? Otherwise, what are the options for a self-hosted environment? The documentation states that remote storage is required when running in k8s or docker. Would the File Storage on a Remote File System option, such as SMB or NFS, work?

One more question, for Task retries, I see that a retry count and a retry interval are supported. Are you looking to add support for exponential back-off or might that capability already exist?

Hi all. Does anyone have a solid guide for getting up and running with Prefect Cloud and Azure Kubernetes Service for agents? I'm finding bits and pieces everywhere, but could really benefit from a comprehensive tutorial. I've found guides specific to using Docker agent on an Azure-based Ubuntu server, but not the same level of detail for a Kubernetes architecture. Thanks!

Hi everyone. I have a problem where my flow is unintentionally exposing secrets in the UI. I am passing secrets to the

env

argument of a class derived from

ShellTask

.

bash = LoggedShellTask(stream_output=True)

# more stuff here ...

with Flow(
    FLOW_NAME, storage=docker_storage, run_config=run_config, result=PrefectResult()
) as flow:

    run_results = bash(
        helper_script=get_helper_script(repo_path),
        command=get_command(ml_script_path, ml_script_arguments, conda_env=conda_env),
        env=dict(
            SNOWFLAKE_USER=snowflake_user,
            SNOWFLAKE_PASS=snowflake_pass,
            SNOWFLAKE_DATABASE=snowflake_database,
            SNOWFLAKE_OUTPUT_SCHEMA=snowflake_output_schema,
        ),
        log_file_path=get_log_file_path(output_dir),
    )

The issue is Prefect automatically creates a

List

and

Dict

task that have

PrefectResults

by default and this exposes the secrets in the UI. A couple of ideas that come to mind: 1. Assign a specific results objects to each task (not at the flow level) 2. Create a wrapper task that receives all the secrets and returns a dict that is passed to the

ShellTask

3. Create a

ShellTask

that accepts each secret as a parameter Any suggestions? One feature I'd like to preserve is the ability to restart the flow and I'd rather not persist the secrets anywhere. Thanks!

Hello, Have any one managed to run Shell Task via SSH ? or submitted a Spark job to a Yarn cluster ?

Regarding DBT Tasks, where should the models be stored? Can they be subdirectories of the flows/ directory where I have my hello_dbt example?

Hi all, I'm looking for some help with the following error: Failed to load and execute flow run: ModuleNotFoundError("No module named 'pyodbc'") We are running an Ec2 instance with the Prefect Docker Agent. --How do we install the Python modules? Is it needed on the Ec2 instance, (which we did) or is there a way to load it into the container? Thanks for your assistance and support.

Hi Everyone. I'm trying to deploy my Prefect project on GCP using Google Cloud Task for BigQuery. Is there any documentation or tutorial with a similar case using GCP and Prefect Orion?

Hi, how can we rename a tenant on Prefect Cloud?

Hello, does anyone have any experience setting up Prefect Orion to be accessed via a web URL (HTTPs)?

Hi All, In Prefect 2.0(server and agent running locally in docker containers) do I still need to create tenant? it seems some commands from Prefect <2.0 doesn't work in Prefect 2.0.

#Prefect 2.0 Any Idea why I'm getting this error while starting prefect agent in docker. Prefect 2.0?

Hello everyone i'm trying prefect for my job and was wondering if someone has a resource that show how i can use the service within AWS please much appreciated.

Hello, i want to overwrite this configuration by using environment variables.So which one is correct? Thanks!

export PREFECT__SERVICES__SERVICES__TOWEL__MAX__SCHEDULED__RUNS__PER__FLOW=1
or
export PREFECT__MAX__SCHEDULED__RUNS__PER__FLOW=1

we installed prefect on Google kubernetes cluster

Hey Yall. So we have The pattern of a Driver flow that calls child flows. Its working wonderfully where we have the create and the the Waitfor... however as the driver grows to control more flows... our waitfors are starting to error saying that the flow didnt start within 15 seconds. Once this errors... all the dependancy flows dont kick off, so its getting rather annoying. We are setting the number of workers... which we had at 10.. now lowering to 4 just to test it out... With unlimitted flow concurrency, shouldnt our flows be starting faster and not hitting that 15 second wall?

Hey all, currently we have some reusable tasks that at the minute are getting repeated alot in the code. I was wondering if it's possible to do something like this (currently doesn't work, but wondering if there is way round it), we're using prefect 1:

def build_full_refresh_base(name, s3_dir_prefix, **flow_kwargs):
    with Flow(name, **flow_kwargs) as flow:

        repeat_task_output = repeat_task()
    return flow, repeat_task_output


@contextmanager
def FullRefreshFlow(name, s3_dir_prefix, **flow_kwargs):
    flow = build_full_refresh_base(name, s3_dir_prefix, **flow_kwargs)
    try:
        yield flow
    finally:
        print("do something")


if __name__ == "__main__":
    @task(log_stdout=True)
    def some_task(repeat_task_output):
        print(repeat_task_output)

    with FullRefreshFlow("hello", "some_dir") as (flow, repeat_task_output):
        some_task(repeat_task_output)

    flow.run()

I need to run and wait on the output of a docker container as part of my Prefect flow. I see that in Prefect 1.x there is an API for running a docker container and waiting on its result. I don't see a similar API in Prefect 2. Is that coming? Should we use

docker

module for Python instead? Or is there a better way?

Hello, I’ve got a bit of an issue that would love some feedback on. There was an initial bug reported in our application where the graphql query to Prefect for a Flow, given the project and flow name, was returning None. Using the Interactive API, I found that the project and flow name for that given Flow existed and got a return. This code has not been touched in months and just recently started failing. Diving further, I noticed our ECS agents have not queried for flows to execute in 3 days, which is when the bug was reported. I restarted our Fargate Services where our ECS agents live, and the agents are still not querying for flows to execute, despite successfully running and ‘waiting for flow runs’. Any idea what’s going on or how to proceed with debugging further?

Which channel is best to ask a newbie question?

This message was deleted.

@Anna Geller or @Kevin Kho Any update on this issue? We’re running into the same issues with no logs appearing when using Prefect with LocalDaskExecutors on a Docker Agent in “processes” mode https://github.com/PrefectHQ/prefect/issues/5769

I feel like I have a basic version problem or something. I created an account in cloud.prefect.io, created an API key and then tried to use it in the CLI on an Ubuntu machine. 'prefect cloud login -k MYKEY' gives me an auth error. Lots of docs say perfect auth login etc but my cli (2.0b6) says auth is not a valid command. Help! What else do you need to know? Am I running the wrong version of CLI?

Hi everyone! I’ve registered a few new flows last week, I can’t seem to be able to trigger a flow run for those through GraphQL. (We’re using a rails application to trigger flow run requests, that app uses graphql and a flow’s version group id. It works fine for a bunch of other, previously created flows.) The requests get sent successfully by our app but nothing happens on prefect cloud. Is there somewhere a place where I can look up all flow run requests a tenant has received? I need some kind of logs to be able to debug this 🤔