Hey folks, we’re seeing very strange flow registration behaviour when trying to register a long-standing existing flow to Prefect Cloud (1.0). My question is, is there a 2500 max task limit in Prefect Cloud on statically defined DAGs? We’ve been registering this flow for months (and it’s been growing) and we’ve just surpassed 2500 nodes, and it’s now throwing an error, so want to check if you have an internal limit defined somewhere…

Hello. Starting to look at migrating to Prefect 2.0, and I’m a little confused about how to structure storage blocks for flows. Is it best practice to create a single block for all flows, or to create one block per flow? I’m not seeing any way when building the deployment to specify a subfolder, which makes me think option 1 is kind of forced…

Join us for PrefectLive this Wednesday, live on Twitch right now. @Taylor Curran will be showing us the cool stuff you can do with AWS Blocks! 😛arty-parrot:

Hello. I have a question. I want to deploy my flow in local subprocess from my VM hosted in AWS. The flow code is on my local computer with all the dependencies. When I execute the command "prefect deplyment buid ..." in my local computer with s3 as my block , should I normally start the agent on my VM in AWS. How to configure the agent so that before executing my flow on my VM, he will install the dependicies required. I read this article on how to deploy prefect 2 on AWS but it doesn't mentioned how the agant will execute the flow. Does the agent install dependicies before running the task inside the flow ? Here is the link https://discourse.prefect.io/t/how-to-deploy-prefect-2-0-flows-to-aws/1252

Hi, could anyone suggest how do we set up ssh keys in prefect kubernetes agent? we need to connect to a SFTP server that we can only let local development env work but not after register the flow.

What methods are folks using to send custom Kubernetes job templates to a

kubernetes-job

infrastructure block in Prefect 2.0? In 1.0, we could accomplish this by using the

KubernetesRun

module and passing it like so:

KubernetesRun(job_template_path="my/local/path.yaml")

It seems like we can try to mimic the functionality in 2.0 by using some hacks to pipe the output from our job template .yaml file, represent it as a one-line JSON string then force that output into

--override job=

but this doesn’t seem ideal.

This work queue uses a deprecated tag-based approach to matching flow runs; it will continue to work but you can't modify it

I noticed this warning on the Prefect Cloud UI. Is there any documentation on the recommended approach to use?

Hi all, I have some questions about prefect 2.0 versus 1.0. I use to use prefect 1.0 at a previous company, and am trying to get prefect up and running at a new company. I started down the road of prefect 2.0 but didn’t quite realize how much has changed, and am now wondering if I should stick with prefect 1.0. It looks like there may have been a useful discourse post about this topic (as reference here: https://docs-v1.prefect.io/), but that post no longer exists. Are there other resources on this? Specifically I am most worried about the new UI for prefect 2.0 as it doesn’t seem that useful for my use case - I don’t see the option to start flow runs, view and retry tasks and task graphs, etc.

is there a way to use

Parameters

or something analogous as input to a

KubernetesRun

to request a specific amount of resources on the fly? I'm working w/ Fargate and want to give users flexibility to run flows with differing amounts of compute

Hi all why can we not create service accounts in the ui for v1 - we are on the standard plan

im getting intermittent 404s on the service account page…

Hello everyone, I have a prefect 1.0 Server running on K8 cluster with Git as storage. Is it possible for me to run a flow on a remote server with all the scripts and dependencies being on remote server and only using prefect on K8 to track the progress of the flow run.

Hi, bumping this one - maybe someone has an idea if I'm doing something wrong or if there really is a problem in ConcurrentTaskRunner/prefect-snowflake

Hello 🙂 I have a problem with a newly created flow. It is a relative straightforward one and pretty easy but it only works locally (flow.run()) and not on the cloud 😞 The logs don't show any information, just reference task fail. The task is using a custom class for processing and loading the data like this:

@task(log_stdout=True)
def etl(
    connector,
    exec_file,
    environment: Optional[str] = None
):
    master_data = MasterDataSet(connector, exec_file)
    master_data.get_dataset(environment)
    master_data.load_dataset()

My flow looks like this:

.... as flow:
    source = SOME_SOURCE
    destination = SOME_DESTINATION

    file_list = ['SOME/PATH/TO/FILE', 'ANOTHER/PATH/TO/FILE']

    master_dataset = etl.map(
        connector=unmapped(source),
        exec_file=file_list
    )

The cloud logs look like this:

Task 'etl_master_data': Starting task run...
Task 'etl_master_data': Finished task run for task with final state: 'Mapped'
Flow run FAILED: some reference tasks failed.

There is no failed reference task. Does anybody has any idea why this is not working on the cloud but locally? It is configured like all the other flows, even the data handling by class is already implemented for other flows. I really have absolutely no clue why 😄

(Prefect 2.0 question) my flow has a

start_date

parameter. In the flow code the default of this value is set like this

def sync_circleci(start_date: datetime.date = datetime.datetime.today().date()):
   ...

and it will come out in the YYYY-MM-DD format. Now when I deploy the flow, it seems that the default value is not recognized by Prefect UI. When I run this flow "using default param values", the flow fails with this error

prefect.exceptions.SignatureMismatchError: Function expects parameters ['start_date'] but was provided with parameters []

which means the default value is not recognized (?). And when I select a date value using the date picker, looks like the date format is not recognized by my flow code with this error

- start_date: invalid date format

How should I handle this?

Upgraded to 2.1.0, ran locally works fine but deployments don't work anymore as libraries could not be find. Do you have to somehow install libraries now to run the deployments???

apologies if this is a silly question, but I’ve been going around and around for an hour now How can we trigger a flow run from the REST API (Prefect REST API - Prefect 2.0) using a POST

/api/flow_runs/

request from an external source? This was something we’d do regularly with the GraphQL api in Prefect 1.0

2.1.0 seems great, I especially like being able to set schedules when building! However, as far as I can tell, there's no way to set the timezone, unless I'm missing something. If I'm not missing something I'll go ahead and create a github issue, just figured I'd check first.

Hi all, we are running the Prefect 2.1.0 agent on Kubernetes. The API in the deployment has no errors, but the agent throws the following error: prefect.exceptions.PrefectHTTPStatusError: Client error ‘404 Not Found’ for url ‘https://api.prefect.cloud/account/[my accountid]/workspace/[my workspace id]/block_types/’ Can someone please explain what is going on?

I have an issue running deployments in 2.1.0. My dbt flows take two parameters that have default values set: loglevel and n_dbt_threads. For some reason I am now getting the following message when running the deployments: prefect.exceptions.SignatureMismatchError: Function expects parameters ['loglevel', 'n_dbt_threads'] but was provided with parameters []. You can see the parameters listed on my deployment page in the screenshot attached. If I run the deployment with custom parameters, manually setting the values for the two parameters, everything's fine. Has anyone found a workaround for this problem, or am I meant to be doing something different compared to 2.0.4?

Hi 👋 For

KubernetesJob

is it possible to specify a

node-selector

and

tolerations

to be able to assign the worker pod to a specific node pool ? I can’t see it in the doc Thanks for your help

I’ve got a question RE: deployments with

null

(local) storage

Is there a way to stop a Dask cluster's queue to do work when an exception happens?

Are there performance implications between • starting several agents each with their own single work queue • starting one agent with several work queues (e.g. moving all work queues to one agent like

prefect agent start -q queue1 -q queue2 …

within the script below) I am bootstrapping agents within a docker container with this python script:

import prefect
import asyncio
import subprocess
import time
import sys

async def bootstrap_agents():

    # get list of work queues
    pc = prefect.get_client()
    work_queues = await pc.read_work_queues()

    # create agents by ID 
    for wq in work_queues:
        proc = subprocess.Popen(['prefect', 'agent', 'start', str(wq.id)])
        print(f'Starting agent associated with {wq.name}. PID number is {proc.pid}', file=sys.stdout)

if __name__ == '__main__':
    asyncio.run(bootstrap_agents())
    while True:
        time.sleep(1)

If there is a better deployment pattern I’d love to get insight on that.

good morning everybody - i have an "execution environment" question: is it possible to have an agent running on a micro EC2 instance that deploys flow runs as ECS tasks? i know in 1.x we could spin up an ECS agent that would automatically do that, but in 2.x it seems like we would manually have to stitch that piece together? this repo shows how to spin up an agent running on ECS in 2.x which is cool, but the infra in the deployment is still

Process

- which means the flow run will be executed in the same container that the agent is running - am i understanding that correctly?

Looking to get back into prefect now that it is more stable. I have multiple deployments each with their own flow. Can i prevent the same flow from running twice at the same time while using just one work queue? In the early betas the solution was one work queue or agent per deployment. Edit: dang. Just noticed #5623 is still open

I am running a flow in Prefect 1.0 with a

DaskExecutor

with

cluster_class="dask_jobqueue.SLURMCluster"

. Everything works fine, unless SLURM kills the resources (e.g. due to time limit). Then the computation is naturally terminated, but in the Prefect cloud I still see the flow and task running. I did an experiment where SLURM kills the resouces after 2min, but the flow is still shown as running for 1h 38min and counting. I thought that the heartbeat option should kick in at some point to flag the flow as

Failed

. Has anyone experience with a similar setup?

Hi everyone, I'm trying to use Docker Storage with our dockerhub account and I keep getting an error that I'm not given permission to push to our account:

InterruptedError: denied: requested access to the resource is denied

. If I

docker push

outside of prefect it'll push no problem. Maybe it's an incorrect

registry_url

?

flow.storage = Docker(   
    registry_url="<http://registry.hub.docker.com|registry.hub.docker.com>", 
    image_name="atddocker/atd-microstrategy",
    image_tag=ENV,
)

when you call

prefect agent start -q misspelledname

it shouldn’t auto create the queue