I’d like to be able to run a flow from a set of start tasks. I understand this used to be supported but might not be any more. What’s the best way to go about this?

Is there a way to trigger databricks notebook from prefect 2.0 in the beta version ?

prefect run command: How to I specify text parameter with non alpha numeric characters

Hi.. I used https://gist.github.com/anna-geller/2014180ee5eaec9ea54f4d3f5b98ca93 as a reference to get exception logs.

class MyFlow(prefect.core.Flow):
    def __init__(flow_name, state_handlers, **kwargs):
        state_handlers = [log_stdout, on_flow_start, on_flow_finish]
        super().__init__(flow_name, state_handlers, **kwargs)


def on_flow_finish(flow: Flow, old_state: State, new_state: State) -> Optional[State]:
    if new_state.is_finished() and get_flow_state_name(new_state) == 'Failed':
        exception_trace: str = "".join(traceback.format_list(traceback.extract_tb(list(new_state.result.values())[0].result.__traceback__)))
        ## do something with exception_trace

While running it on prefect cloud, the

new_state.result

dict is empty resulting in a

IndexError

. When we run it locally, the new_result.result comes back with a length of the number of tasks in the flow. What's the right way to trap exceptions from a task in the flow's scope?

Hi, I have been working with the prefect flows terminal state handler. From the terminal atate handler, Its possible to get only the state of every related tasks , but unable to identity the corresponding task details including task name. Is it possible to get all the reference tasks and its corresponding states to prepare a detailed summary of the flow run.

Does anyone know of a way to apply some kind of tagging to the k8s jobs the agent starts so that I can easily use gcp metric explorer to see resource usage per flow_id? Everything is just

prefect-job-foo

not sure how I can isolate one flow from another?

Hi guys, I am running several python scripts once a day. Currently I am using crontab for the scheduling. I´d like to do this with prefect in the future using prefect orion without prefect cloud. Do anybone know of a simple tutorial or how to to get this up and running? Thanks 🙂

hi, i am new to prefect, i was able to run task in parallel using: flow.run(executor=LocalDaskExecutor()), but calling run() is blocking tell all tasks are finished, possible to call run in non-blocking way? so the run()-call returns immediately

any way to programmatically kill a flow that is running for more than a certain period of time? I am aware of SLA automations but want to do it code way. Please suggest. Thanks!

Hola 🙂 I have an issue with a pod not terminating while the flow is finished. ⬇️

Hi! I’m trying to run a flow locally with a prefect server. I have the server running as well as a local agent. When I run

flow.run()

, it errors out with this message:

TypeError: can't pickle _thread.lock objects

Note: I’m running the flow through a build system (bazel)

👋 hullo! curious, in prefect 2 is there a way to set some flow-level context that are accessible by all sub-flows/tasks?

Starting a new thread. Continuing from https://prefect-community.slack.com/archives/CL09KU1K7/p1653578937668889 We are working on running an orion server in an Azure VM. We seem to have gotten the UI accessible via a public IP and also have setup default storage (azure blob), added deployment have successfully run flows via the CLI. When we load the UI via the Public IP, the web page shows no information. Even the settings page fails to show the Orion version. I'm sure there is a setting (or access) somewhere that we are missing. Any thoughts?

Is that possible to use one task in multiple flows in order to follow "don't repeat yourself"? For example I have one task checker that will check difference of db to decide to update or not - that is used many times. So some code reuse maybe handy. Any suggestions? Some simple example, if difference_rows >0 : use task_to_update else : skip all tasks of this flow only

The extra logger that I have defined doesn't seem to work Then also tried overriding the logger defined in the in custom module Something strange happens here: Logs are visible in console but not on cloud or UI

Question regarding prefect server, we are running it in an EC2 Ubuntu instance with 20GB and getting close to using 100% of storage usage, is the server/docker containers storing log files? If so, where can I find out where?

Hi. I was trying to use conditionals instead of python if-else error occured 'reference tasks must be part of the flow'.

Has anyone else experiencing these errors with Prefect today? Capacity is unavailable at this time. Please try again later or in a different availability zone

Hello, everyone, I was getting the "something went wrong when you try to fetch the project". My setup is hosted prefect server on an EC2 box and on my local, I was trying to connect to the dashboard. I can see all the dashboard/menus, I can even list the project from the graghQL port, just can't list or create a project from the dashboard. Any idea where could go wrong?

Hi Prefect Team!! I am trying to execute a deployment using Prefect 2.0 . I have created a default storage as Azure blob Storage and created a deployment. When I try to execute the deployment or run it through work-queue. I am receiving the below error. Currently I am testing this on windows OS. Please note that when I run ' python leo_flow.py ' I do see that the task and the flow is getting executed as expected. Any help or guidance around this would be highly appreciated!!

Can we start using Prefect 2.0 with Prefect Cloud?

Hi, I want to set up a Prefect agent in AWS ECS. I've set my AWS cred in

~/.aws/config

and did

prefect auth login

using my API Token. Then I'm running the

prefect agent ecs start

command, expecting an Agent to be created in Fargate and linked to my Cloud, but I'm getting the following error:

ERROR - agent | Failed to infer default networkConfiguration, please explicitly configure using `--run-task-kwargs`
Traceback (most recent call last):
...
ValueError: Failed to infer default networkConfiguration, please explicitly configure using `--run-task-kwargs`

What am I doing wrong?

Hi, does anyone know whether I can integrate Prefect on-prem, as part of my product, without paying for a subscription?

or is it mandatory to pay per use?

And another unrelated question. Can anyone please explain the typical production integration with Prefect? Concretely speaking, I have a web MERN (MongoDB, Express.js, React, Node.js) stack, and I would like to attach/integrate my JavaScript Node.js code of REST APIs (e.g. POST /employees) with an existing Prefect DAG.

Not sure about the API and the integration protocols

Hi I am starting to randomly see this on flow runs;

12:09:39 ERROR prefect-server.ZombieKiller.TaskRun No heartbeat detected from the remote task; marking the run as failed.
12:22:53 INFO prefect-server.Lazarus.FlowRun Rescheduled by a Lazarus process. This is attempt 1.

I am running on GKE autopilot - any ideas how to debug cause of such events and mitigate against their occurence?

Hi, I'm trying to understand what happens if the agent running a flow is interrupted (more specifically if there is a way to gracefully stop it). From a simple experiment of a 60 seconds long flow, that I interrupt the agent running it, the runs remain in the UI even after re-launching the agent:

Hi There, I'm having a problem with task logging using cloud and dask cluster, no logs are being streamed to the cloud except errors and "Starting task run ..." / "Finished task run for task with final state: ....." messages and my logging statements is not showing. keep in mind that I'm using mapped tasks

Hi:) I am running a flow on seclude and would like to set the flow run name as a template something like: run_my_flow_{time.now} Is there is a way to do that? I want to avoid the auto generated name Thanks:)