Hey guys, i started getting this error when running my flows on ECS Agent:

An error occurred (ThrottlingException) when calling the RegisterTaskDefinition operation (reached max retries: 2): Rate exceeded

This means that i should increase my Rate limit on my AWS account?

Hi. is it possible in prefect 2.0 to delete all deployments at once? Thanks

Is there an ECS flow runner planned for 2.0? Couldnt find anything in the v2 tagged github issues... Thanks!

Getting this message.

TypeError: cannot unpack non-iterable PrefectFuture object

Do I just need to do

@task(nout=2)

in Prefect 2.0?

One last question for today... Mapping Tasks... is this still TODO on Prefect 2.0?

Is the Cloud RBAC granularity at the tenant level? I.e. I’d like to have everyone be a reader, but some users to only be able to “write” some flows but not others. Separately, I’d also like to have “per-team” agents and some way to limit which agents can a user submit flows to. Am I better sticking to just separate tenants/separate servers for on-prem deployment, than considering the RBAC for this? Is

workspace

the 2.0 cloud analogue to the 1.0 tenant?

Hi, someone know if there is a way of installing just the core of Prefect Orion, without the REST API, UI, CLI, just the python API? Thanks in advance.

Is there a way to force a specific mapped task to run in sequence even though the flow is using a local dask executor? I have a flow with a bunch of mapped tasks that I want to run in parallel but one mapped task that i'd like run in sequence.

Hi folks, I'm trying to schedule a flow but keep getting

Something went wrong. Please wait a few moments and try again

. Flow runs fine locally. Any idea how to troubleshoot this?

Hi everyone. Has somebody successful ran flows that use Dask Dataframes or multiprocessing library on a Windows Local Agent run deployed on Cloud? I am finding it impossible to define a flow and wrap it inside the necessary

if __name__ == ‘__main__’

. It all works fine when running locally with

flow.run()

but it tells me it can't find a Flow when I deploy it to Cloud and run it from there.

Hi there, is it possible to obtain the dask client when running with the `DaskExecutor`/`LocalDaskExecutor` or the

DaskTaskRunner

in Orion?

Morning - where there any issues with cloud this morning? I have a flow that failed (sent no slack message) - error was;

raise ClientError(result["errors"])
prefect.exceptions.ClientError: [{'path': ['secret_value'], 'message': 'An unknown error occurred.', 'extensions': {'code': 'INTERNAL_SERVER_ERROR'}}]

Hey team! With prefect 2 coming soon, would someone be able to point me in the direction of the benefits prefect cloud will have over hosting our own server? I notice the pricing page highlights the support and escalation, I was wondering if there was a similar breakdown to the technical benefits (for example, right now I know prefect cloud supports flow concurrency limits, while prefect server 1.x does not).

Hi all, how do we build docker image from orion branch ?

Executing task: docker build --pull --rm -f "Dockerfile" -t myprefect:latest "." <

Sending build context to Docker daemon  101.5MB
Step 1/38 : ARG PYTHON_VERSION=3.8
Step 2/38 : ARG BUILD_PYTHON_VERSION=3.8
Step 3/38 : ARG NODE_VERSION=14
Step 4/38 : FROM node:${NODE_VERSION}-bullseye-slim as ui-builder
14-bullseye-slim: Pulling from library/node
Digest: sha256:f7137af1e34927cb3251a6e091edff4592c49a89422196680a2f087edcbc6e4d
Status: Image is up to date for node:14-bullseye-slim
 ---> e2eb23c871f2
Step 5/38 : WORKDIR /opt/orion-ui
 ---> Using cache
 ---> 74138fe30a86
Step 6/38 : RUN apt-get update &&     apt-get install --no-install-recommends -y         chromium     && apt-get clean && rm -rf /var/lib/apt/lists/*
 ---> Using cache
 ---> 77731060b4e3
Step 7/38 : RUN npm install -g npm@8
 ---> Using cache
 ---> ac23c1b14bea
Step 8/38 : COPY ./orion-ui/package*.json .
When using COPY with more than one source file, the destination must be a directory and end with a /
The terminal process "/usr/bin/bash '-c', 'docker build --pull --rm -f "Dockerfile" -t myprefect:latest "."'" terminated with exit code: 1.

Hello all, I recently started experimenting with Prefect for one of my projects. I would like to ask whether it is possible to use the asynchronous functionality provided by dask; there are some tasks that involve IO output and as such they could be executed asynchronously (I am using the latest release of Prefect 2.0).

hey everyone, is there a way to use variables that are global to every task like an increment counter ?

Cross-posting here for visibility - this is a fantastic piece of content by Peyton that will: • walk you through getting started writing data pipelines in Python • showing challenges of operationalizing such Python script - the negative engineering part • walk you through how to turn this script onto a Prefect 2.0 flow • show you around the new Prefect 2.0 UI

Hey, was wondering if mapped tasks are able to output multiple values with nout ? i have an error when trying with dummy code:

from asyncio.log import logger
import prefect, pymongo, datetime
from prefect import task, Flow, flatten
from prefect.tasks.secrets import PrefectSecret
from prefect.tasks.mysql.mysql import MySQLFetch
from prefect.storage import GitHub

@task(nout= 3)
def produce_output(input):
    return input*2-1, input*2, ["val"+str(input*2-1), "val"+str(input*2)]

@task
def reduce(a):
    return sum(a)

@task
def retry_post(data):
    logger = prefect.context.get("logger")
    <http://logger.info|logger.info>(data)

with Flow("abcdef") as flow:
    success, fail, to_retry = produce_output.map([1, 2, 3, 4])
    total_success = reduce(success)
    total_fail = reduce(fail)
    retry_post.map(flatten(to_retry))

flow.run()

Hey, have a question: • Using Prefect 1.0 • Using

Local

storage and

Prefect Cloud

I want to use

stored_as_script=True

for my flow (pickle storage is not working with an SQLAlchemy object for some reason), and so I need to specify

path

— but that is giving me an error. Sharing the relevant code snippet and error.

with Flow(
        "eco_flow", 
        schedule=schedule, 
        storage=Local(stored_as_script=True, path='~/.prefect/flows/test-flow.py')
    ) as flow:

    cities = ['mumbai', 'new-delhi']
    
    for city in cities:
        data = get_traffic_data(city)  
        clean_data = clean_traffic_data(data)
        insert_to_db(clean_data)

Error after registering the flow:

Failed to load and execute flow run: ModuleNotFoundError("No module named '~/'")

I guess I am defining the

path

incorrectly?

Trying to use the DaskTaskRunner in Prefect 2.0 and getting this error. I am following the published documentation.

from prefect.task_runner import DaskTaskRunner
ModuleNotFoundError: No module named 'prefect.task_runner'

Is there an extra that needs installed?

OK, So I am trying to use the DaskRunner with a for loop to concurrently run a task over a list.

@flow(name="Subscriptions Flow",
      task_runner=DaskTaskRunner())
def main():
   *snip*

    pages_list = get_pages_list(client, "subscriptions", params)
    for page in pages_list:
        item_list = get_items_list(client, "subscriptions", page)
 

if __name__ == "__main__":
    flow_result = main()
    print(flow_result)

But I get a RuntimeError from the multiprocessing. Can I get some help on what I am doing wrong with the DaskRunner. I want to parallel run over what is in

pages_list

What is the equivalent to this in 2.0?

@task(checkpoint=False)

If there is one currently

What is the best way of restricting some tasks (with "tag2") to not run until all tasks (with "tag1") has succeeded? Does not have to be tag-based (Prefect 1)

How can I have the first task in a flow specify a timestamp that subsequent tasks can access? Basically I want all tasks to be able to read/write data to the same directory, with the directory have a name like 2022_06_09__113826 based on the approximate time the flow started.

Hello, We currently have no running flows however when looking at the "Task Concurrency" page, I have a tag for which it says that 2 task currently running 😮

RUN export PREFECT__CONTEXT__SECRETS__AWS_CREDENTIALS=$(echo `(aws secretsmanager get-secret-value --secret-id ${prefect_aws_secret_name} --region ${region}) | jq -r '.SecretString'`)

I'm having a ton of trouble using ARG/ENV/RUN with a dockerfile, what I want is for PREFECT__CONTEXT__SECRETS_AWS_CREDENTIALS to show up as an env variable in the final image. Does anyone have experience setting env variables in a dockerfile from command output?

Hello, we are trying to run prefect agent behind a proxy and encounter this:

Error downloading Flow from Azure: ('Cannot connect to proxy.', ConnectionResetError(10054, 'An existing connection was forcibly closed by the remote host', None, 10054, None))

Following env vars are set

HTTPS_PROXY=x.x.x.x
AZURE_STORAGE_CONNECTION_STRING=xxxx

We can connect to blob should just fine using

az

. Any pointer?

Hey I'm trying to run a mapped task and it looks like everything was successful but then right after reduce when I try to merge the results I'm getting this error:

Hi All, We've just migrated to Prefect Cloud from Server and are confused as to whether the scheduled times of our flows will remain or will we have to manually re-schedule them ourselves. Thanks

I am trying to use GIT storage with ECS run. The goal is to have a docker container which has all the python packages installed and then just pull the code from github prior to running the flow. This obviously requires that the container should be able to pull code from Github. Then, in

task-definition.yaml

I am providing

GITHUB_ACCESS_TOKEN

through linux ENVIRONMENT variables. But I am getting this error:

'The secret GITHUB_ACCESS_TOKEN was not found.  Please ensure that it was set correctly in your tenant: <https://docs.prefect.io/orchestration/concepts/secrets.html>'`

Am I missing something?