Hi all, Is there a way to start a local agent with the LocalDask executor? To clarify: from the CLI Thanks!

i have installed prefect server on a VM in Azure. my code (flow) is in my local machine. can i register that flow with the prefect server running in VM? If yes, can you please let me know the steps? I have once completed the flow register in my local prefect server, but now installing on cloud.

Question for y'all! Can a

@task

be orchestrated as a job/pod on Kube?

i have registered some flows in a remote prefect server. is there a way i can call to run these flows from some other code

I have installed prefect 1.0.0 and was previously running 0.15.2. For both versions I am getting the following error when trying to run my workflow via the following:

python3 workflow.py

My workflow only has one import:

import prefect

Any thoughts on why this might be? Any assistance would be appreciated

Hi All, The prefect tasks which are in resume state are not going through, they remain in resume state. This is affecting our production flow runs as well. Can someone please check if there's any issue with prefect cloud? cc: @Kevin Kho @Christina Lopez @Anna Geller

Hello, I was curious if there was any experience or documentation about self hosting I could look at?

Hello Hope that you are well. I have a parent task I have created and I am able to create a create a series of task runs based on the inputs according to this documentation: https://docs.prefect.io/core/idioms/task-run-names.html. How can I ensure that the children tasks that are created from this are able to run in parallel instead of sequentially? Do I need to set the flow's executor to be the LocalDaskExecutor? Thanks for your help,

hey everyone, is there any documentation on how to run prefect orion server on docker locally?

I am trying to install and run prefect server in Azure Container. This is my docker file. Without the docker run command, I am getting docker compose down error. with the docker run, the error states - executor failed running

FROM prefecthq/prefect:latest

WORKDIR /app

RUN apt-get -y update && \
    apt-get install -y <http://docker.io|docker.io> && \
    apt-get install -y docker-compose

RUN docker run --privileged -d docker:dind

RUN docker --version

COPY docker-compose.yml .

RUN prefect backend server && \
    prefect server start

```

Hello everyone, are there any rate limits on the GraphQL API? Thanks.

i have tasks that call operators/connectors that i’ve defined with their own loggers. but even when i set

log_stdout=True

those logs don’t show. any advice on how to include logs from those other loggers?

Can we apply rbac in prefect server or only on prefect cloud?

Error for some users on "hello-world" code we have an agent running on a host, on user group "a" I am in this user group and can run the hello-world code without a problem. another user on that host, who is on a different user group, tries to run the same "hello-world" code he can run it using the

flow.run()

and it runs OK he can register it and see the flow in the dashboard, but it fails with

Failed to load and execute Flow's environment: ModuleNotFoundError("No module named '/home/<user_name>'",)

any thoughts? does every user group needs a different agent ?

Hi! I was trying to set up a parent flow. It shows run succeeded, which doesn’t appear to be the case, as one of the flows is supposed to write to a database and nothing was written. however, if running one flow (a part of the parent flow), it does write to db. what could have gone wrong here? thanks!

Good morning, Prefectionists! We are sailing. I have successfully deployed two very simple Workflows about a month ago and everything is working beautifully. I'm a DBA for a small non-profit and we handle a ton of data. We decided to implement prefect so I can automate many of the data importing and ETL tasks I perform manually. I wanted to share a couple of things Prefect taught me. 1:Register the flow on the agent its intended to run on. for some reason this resolved all the import statements correctly. 2:Translating your workflow into prefect code helps to refactor the steps of the flow. I would do these in SSIS, and SSIS made the "theory" of the flow easy, but it didn't truly represent the logic of the flow. In doing so, I developed a much more concise workflow, which had a few more tasks. Let this happen: its a good thing. 3: Python is amazing. And since Prefect doesn't get in the way of that, prefect is amazing. It quite literally can do anything. 4: Docker flows: This can get messy. I found about 52 containers that were left over from all my flows being run. This is a really intelligent solution regardless, it just leave a trail. I'm sure there is a setting to automatically remove these that I just haven't found. Don't be alarmed if you have clutter in your docker instance. It well worth it. 5: When deciding to implement a flow in Prefect, you are forced to design the whole thing. No more "Oh, we'll deal with that when we get there..." This is a steep hill to climb, but you once you do, its a never-ending coast down the other side. This forces questions to the beginning of the process and helps get solid answers for them right away. Its possible that a workflow may not be a good candidate for Prefect, or maybe parts of the workflows are candidates. You actually save a lot of time designing end to end. Thanks for reading.

Hi folks, Wondering if there's a simple way to register flows remotely when using prefect server? (no cloud option in this environment, unfortunately)

Hi All , I am having issue with prefect server. Then server is starting on server ip and config.toml is set to apollo_url="http://10.1.x.x:4200/graphql" everything starts normally and no errors during start , the ui shows green connection to API, however 1-2 minutes later it turns yellow and stops working , i turn debug=true but I am not able to find logs are written or how to troubleshoot this problem. Any help would be appreciated (this is local install on ubuntu)

prefect register --project "Project Name" --path ./Prefect/ I am getting error after pointing my install from localhost to 10.1.x.x IP , here is my configuration [server] host = "http://10.1.xx" port = "4200" host_ip = "10.1.x.x" host_port = "4200" endpoint = "${server.host}:${server.port}" [server.ui] host = "http://10.1.x.x" port = "8080" host_port = "8080" host_ip = "10.1.x.x" apollo_url="http://10.1.x.x:4200/graphql" raise InvalidSchema("No connection adapters were found for {!r}".format(url)) requests.exceptions.InvalidSchema: No connection adapters were found for '10.1.x.x:4200'

to run a flow frrom any code, we need to know the flow_id. how can we get the flow id programmatically?

Is there a way to get more debug info from a flow run? I'm trying to use git storage and am getting this error:

Failed to load and execute flow run: NotGitRepository()

is there a way to get

prefect server create-tenant ...

to use a different apollo endpoint via environment variables or cli args? It defaults to

<http://localhost:4200/graphql>

but I'm attempting to run this in a docker-compose file and need to use a different hostname than localhost

is it possible to programmatically create a project (within a tenant) with the cli?

Hi, There's a small issue in the helm chart during upgrade - it logs an incorrect message to the screen. (Details inside)

Is it possible to define async tasks in Python using

async def...

?

Hey all! if a

state_handler

is being used on a

Flow

, and a

@task

has a

default_exception

and the

new_state

is

Failed

... is it possible to get that Exception for inspection in the state handler?

Hi, Wondering what the best practice would be for making secrets available to flow runs picked up by Docker Agents? Specifically looking at the

SLACK_WEBHOOK_URL

at the moment and have tried setting it as an environment variable for the agent, as well as adding it to the config.toml on the machine hosting prefect server but I'm still seeing

ValueError: Local Secret "SLACK_WEBHOOK_URL" was not found.

Hi all, I have a task that returns a dataframe, but instead it's returning a FunctionTask. Could it be something with a task definition?

Hi all, There seems to be some issue with Prefect Cloud heartbeats. Randomly, some mapped tasks show an error:

No heartbeat detected from the remote task; retrying the run.This will be retry 1 of 3.

and then it never actually retries.. I believe the Lazarus process must kick in every 10 minutes and reschedule the task, right? CC: @Christina Lopez @Kevin Kho @Anna Geller

Hi all, We have a running Prefect server (version 0.15.4) on EC2 (8 CPU, 64 GB of RAM) with approx. 15 docker container with specified flows. Each one docker container contains at least 1 running flow (max. 3). Every container has a local agent which is connected via --api parameter to prefect server. We have 6 flows which runs as infinite flow (when flow is finished state handlers starts same flow with create_flow_run() function). The rest of flows are scheduler to only run once an hour. The problem is that once a week or two most of our infinity flows fails on error:

Failed to retrieve task state with error: ReadTimeout(ReadTimeoutError("HTTPConnectionPool(host='apollo', port=4200): Read timed out. (read timeout=15)"))

Sometimes the flow can continue without any problem, because we have infinite flow set to continue even when flow failed:

def never_ending_state_handler(obj, old_state, new_state):

if (old_state.is_running() and new_state.is_failed()):

send_message_to_email(obj)

if (old_state.is_running() and new_state.is_successful()) or (old_state.is_running() and new_state.is_failed()):

time.sleep(5)

create_flow_run.run(flow_name="our_flow", project_name = "our_project", run_name = str(uuid.uuid4()))

return new_state

But when we receive error:

A Lazarus process attempted to reschedule this run 3 times without success. Marking as failed.

we are not able to continue... The flow is failed but state handler is not working anymore to reschedule failed flow. Does anybody know what may cause problem with Read timed out or Lazarus?