Hi everyone! Is there a limit to how many service accounts you can have?? The button to add a new one is greyed out for us 🤔

Hi, if I’m deploying using ECS, do I have to store secrets (such as Snowflake credentials) within Prefect Cloud, or can I somehow keep them within my infrastructure? I’m guessing I could access them using AWS secrets manager, or within the docker image (prob not a great idea), but I’m wondering what the best practice is. Thanks

Sorry to ask multiple questions but I have a flow that is running on ECR which fails with the following message

An error occurred (ClientException) when calling the RegisterTaskDefinition operation: Invalid setting for container 'flow'. At least one of 'memory' or 'memoryReservation' must be specified.

I have set memory (and cpu) in the ECSRun task_definition (both quoted and unquoted) and still get this error. Does anyone know what I might try?

Upgrading to 1.0 (using the nightly version that has the Capacity Provider fix), digging a lot! Trying to test out the Dask Performance Reports - is there a way to dynamically generate a filename for it based on the Run's arguments? (not totally sure what Event Handlers have access to 😅 )

is label the "primitive" to use for achieving some sort of tenancy isolation?

How different is Orion from 11.0? We recently began researching Prefect as a solution, shortly before Orion came out. Is it necessary to also evaluate Orion (in addition to 11.0) to form the most comprehensive view of prefect? Or will Orion co-exist with existing 11.0 because they solve different problems and neither will replace the other? Thanks!

What does

*prefect.core.flow.Flow.run_agent*

do? It's unclear from documentation and src code, which seems to say it runs an Agent from inside a Flow, which might be confusing since I was under the impression that Agent runs Flows, not the other way around.

Are the

services

enumerated on this page still update to date and complete?

What is the standard way to set up prefect code to handle promotion through various environments? (dev -> stage -> prod). I’m using Prefect Cloud and would like to use the Secrets manager.

Hi 🙂 I'm facing an error on an HPC cluster with my hello_world only with prefect 2:

Hi all 😎 I'm going through the Orion docs and noticed that the "states" link under the FAQ > Deployment > How is Orion organized? section leads to a 404 page (w/ URL = "https://orion-docs.prefect.io/faq/concepts/states") Not sure if this is the appropriate place to post this (please redirect me if not!), I believe the appropriate link is to the "states" page is "https://orion-docs.prefect.io/concepts/states/#overview"

Hi there, noob question here, there is an http api in prefect? I wan't to get the prefect task status on my zabbix, with an http request to prefect, so I can create trigger when something goes wrong, didnt find anything about in the docs

Hi everybody, It seems like our team is going to have tons of flows. Is there any ideas how to create a hierarchy of the flows or to arrange them somehow? Thank you.

Hi there, is it possible to use https://docs.prefect.io/api/latest/tasks/files.html file operators with files hosted on SFTP with user/pass? Or another to use prefect to fetch files from SFTP?

Hi guys. We're trying to set our log format to log everything JSON so we can consume it in Datadog and search the logs. We'd targetting the following format:

{
  "flow_name": "",
  "flow_id": "",
  "task_name": "",
  "task_id": "",
  "message": "",
  "customer_id": ""
}

The questions we have are: 1. Is using

PREFECT__LOGGING__FORMAT

the correct place to specify the JSON format? 2. How can we handle this additional attribute

customer_id

. Our intent here is for most tasks that are related to our customers to do something like

<http://logger.info|logger.info>('Something', customer_id)

3. To the previous point, how can we extend this further such that we can add arbitrary parameters or objects to the log,

<http://logger.info|logger.info>('Something', my_object,my_second_object)

and have them show up nsted in the json?

Hello all Is Prefect can be entirely installed on Kubernetes? If yes, is there a guide somewhere? To what I've understood, only some parts of the architecture can run in K8S. Am I wrong? Thank you

See you at 3:15p Eastern today! https://prefect-community.slack.com/archives/C036FRC4KMW/p1649259298561779

Hi everyone! I’m trying to use an S3List task https://docs.prefect.io/api/latest/tasks/aws.html#s3list but it seems like the bucket prefix is not an optional value even though it should be? 🤔 Boto3 docs lists it as optional (https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/s3.html#S3.Client.list_objects), but the task implementation doesn’t (see screenshot). This is what I try to do:

s3_list_task = S3List()

        flow.set_dependencies(
            task=s3_list_task,
            keyword_tasks=dict(
                bucket='my-bucket-name'
            )
        )

And this is the error message:

TypeError: run() missing 1 required positional argument: 'prefix'

Does this mean I can’t use buckets without prefixes together with the S3 prefect tasks?

Total rookie question here -- I'm attempting to install and play around with Prefect 2.0 in PyCharm, having some trouble verifying the install was successful. When I attempt to check the version in PyCharm's terminal I receive the following error messages. I receive similar error messages when attempting to run simple python scripts listed in the First Steps tutorial. I created a python project in PyCharm and installed Prefect 2.0b2 using their settings interface. Any idea where I went wrong? Ive attached screenshots of the error messages and my project's Prefect directory.

We need alerting when a flow is submitted for execution but is never executed. Can we create an automation for this? Basically for every flow, send an email if a flow is submitted but not started after 60 seconds. Is this possible?

hey, someone can help me with a query in graphql api? I wan't to collect all created flows in one query, but I wan't ONLY the LASTEST version of each, how can I do so?

Is possible to debug flows like celery? Example: I have my flows registered and i want to be the one that will pull the flow to run (with debug) in pycharm

PrefectLive is starting in 15 minutes! See you in chat! https://prefect-community.slack.com/archives/C036FRC4KMW/p1649259298561779

Hi. I'm just starting to look at Prefect 2.0. Is there a suggested way of handling a waiting period mid flow? For example if I start a flow, and after an initial task I want to wait for 5 minutes before continuing, do I just sleep for 5 minutes in a task? What about if I wanted to wait for days?

is there a way programatically to get a list of deployments in prefect? i know we can do

prefect deployment ls

but i want to get all my created deployments...1000 of them and run them all. so im assuming i have to programatically execute prefect deployment run deploymenet_name for each deployment in my deployments

Hi, I have two projects in Prefect Cloud: one for dev & one for prod. Each project has its own a Docker agent using DockerRun for running flows in containers (in EC2) built based on images from Dockerhub. These images are being re-built with a CI/CD flow (using a Dockerfile) whenever we push code to our Git repository either in dev or prod (one image tag for dev and one for prod) The flow source code is stored in Github and this is configured as the storage for our flows. For some reason my flow runs in the

prod

project suddenly fail with

Failed to load and execute flow run: ModuleNotFoundError("No module named 'project_name'")

however, all the flows with exactly the same source code in Prefect

dev

project run just fine. When I pull these images and start containers locally the project structure and packages are the same for both dev and prod. What could be the issue here?

Strictly speaking, do Tasks need to run in an environment that has the original "Task" function source code? Cloudpickle/pickle would lead me to think that the "bare minimum" necessary would be just an environment with task's py dependencies, but not the task source code themselves.

What is the recipe for configuring a scheduled Flow's parameters based on the "scheduled time"? I found

parameter_defaults

for Clocks, but they seem to be static w.r.t each clock. For example, I want the parameter be "day of the week",i.e. Mon, Tue....

Hey team - having some troubles running docker tasks in a docker agent on windows (dind) ..