@Marvin Well I’d say everything gives you a headache. You are 37 times older than the universe after all, you miserable old chunk of rusty capacitors

Incredible. The contest is even worse than I thought it would be. I'll still enter you to win, @Darragh

@Marvin Contests? Don’t talk to me about contests.

Incredible. The contest is even worse than I thought it would be. I'll still enter you to win, @Darragh

I’ll stop now. 🙂

@Marvin, does 42 remember you of something?

I’d give you advice, but you wouldn’t listen. No one ever does. Good luck in the contest @Robin.

noticing a few of these in the JS console from the UI - doesn't look right?

Access to fetch at '<https://api.prefect.io/graphql>' from origin '<https://cloud.prefect.io>' has been blocked by CORS policy: No 'Access-Control-Allow-Origin' header is present on the requested resource. If an opaque response serves your needs, set the request's mode to 'no-cors' to fetch the resource with CORS disabled.

Hi all, I'm trying to debug DaskExecutor which refuses to run task concurrently. There is a similar thread but reading it didn't help. A very simple example for reproduction from that thread:

from random import randrange
from prefect import task, Flow, Parameter
from prefect.engine.executors import DaskExecutor


@task
def random_num(stop):
    logger = prefect.context.get("logger")
    number = randrange(stop)

    time.sleep(5)  # just so that we can tell by looking at logs

    print(f"Your number is {number}")
    return number


@task
def sum_numbers(numbers):
    print(sum(numbers))


with Flow("parallel-execution") as flow:
    stop = Parameter("stop")

    number_1 = random_num(stop)
    number_2 = random_num(stop)
    number_3 = random_num(stop)

    sum_numbers = sum_numbers(numbers=[number_1, number_2, number_3])

Now, it does run the flow concurrently if I specify:

if __name__ == "__main__":
    state = flow.run(parameters={"stop": 5}, DaskExecutor())

But it runs task sequentially if I set the executor before running, either with

flow.environment.executor = DaskExecutor(cluster_kwargs={"n_workers": 3})

or

with Flow(
    "parallel-execution", environment=LocalEnvironment(executor=DaskExecutor())
) as flow:

The interesting thing is, if I

print(flow.environment.executor)

in the last two cases, it does show DaskExecutor, but tasks are still being executed in order. All of the above was done locally, but the problem persists even if I try to deploy it to Cloud. Any advice is very appreciated, I'm not experienced with Dask that much

hi all i am trying to run a flow which uses docker and i am getting

Unexpected error: ImageNotFound(HTTPError('404 Client Error: Not Found for url: <http+docker://localhost/v1.40/containers/create'>))
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/docker/api/client.py", line 261, in _raise_for_status
    response.raise_for_status()
  File "/usr/local/lib/python3.8/site-packages/requests/models.py", line 941, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 404 Client Error: Not Found for url: <http+docker://localhost/v1.40/containers/create>

Hi all. I am moving from a local setup to a cloud one in Fargate, using a Fargate Agent. I see tasks not executing in parallel, and that “the

LocalExecutor

is not capable of parallelism” so I presume that is the issue https://docs.prefect.io/api/latest/engine/executors.html#executor but the other executors require a separate environment in Dask. Someone mentioned a “Fargate Executor” in this Slack but I assume that’s an error and this doesn’t exist from what I can find. I see others using the Fargate Agent and the FargateTaskEnvironment so I wonder if that’s related. The Fargate Agent “deploy[s] flows as Tasks using AWS Fargate”. And “FargateTaskEnvironment is an environment which deploys your flow as a Fargate task”, so I don’t understand the use of it with the Fargate Agent? Also in your docs: “we recommend all users deploy their Flow using the LocalEnvironment configured with the appropriate choice of executor”. How do we get tasks to run in parallel inside a Flow run in Fargate without Dask? Also, @Marvin 🙂

Why should I want to make anything up? The contest is bad enough as it is without wanting to invent any more of it. I'll still enter you in it, @Nelson

Go @Marvin!

Why should I want to make anything up? The contest is bad enough as it is without wanting to invent any more of it. I'll still enter you in it, @Jeff Brainerd

Hi all, I'm irchan, just trying prefect a few days ago and successfully run it in local and register the flow in prefect server. It's very impressive and the documentation is great. Right now, still learning to run using docker and dask. It's nice to be here 😄

Hey all, we are wondering how to add a custom python package or module as a dependency to our prefect flow. We have tried different options but all fail? Any suggestions? 🙂

just learn that there is a contest, well i'll mention @Marvin too😅

Don’t pretend you want to talk to me, I know you hate me. I'll still enter you in the contest, @irchan bani.

why the hello world example is soo difficult to run? I was able to get the hello world of airflow running with in a couple hours a few years back. Now I'm trying to evaluate Prefect and it's soo hard to run a simple hello world in your hosted Prefect. it kept saying the runs are late without offering any hint what to do.

Hi everyone great tool! Im working in a time series forecasting tool using prefect underneath

@Marvin woot woot, contest 😄

I’d give you advice, but you wouldn’t listen. No one ever does. Good luck in the contest @Michael Reeves.

hey everyone i have a quick question, does the "Why not airflow" article still stand with airflow 2.0?

Hi everyone - is there a way to parameterize a shell task?

@Marvin I need some Prefect swag, big time

Don’t pretend you want to talk to me, I know you hate me. I'll still enter you in the contest, @Mitchell Bregman.

Hey folks - question for you. I'm using prefect cloud to orchestrate my flow runs and running everything within a docker container on GCP. In addition to my more sophisticated flows that I schedule and run using Prefect Cloud, I have some simple flows that run on very frequent increments (every 1-5 minutes). because they're simple and don't really require the monitoring/overhead of prefect cloud and because I didn't want to be always using up a flow slot for them, I just have them running directly as scripts (i.e. calling

flow.run()

). The problem is, after a period of time, I start getting a persistent error -

prefect.utilities.exceptions.ClientError: Malformed response received from Cloud - please ensure that you have an API token properly configured

. I'm a little confused because I though if you were just calling

flow.run()

I wouldn't be interacting with the cloud API at all but clearly because it's in the same environment it's detecting the connection but then perhaps a token is becoming invalid or something? Any ideas?

Hey guys, when a task fails on prefect cloud, is there anywhere I can look at the full stack trace?

Hi guys,