Is it possible to run a prefect agent as a service on Windows 10? If so what's the right way to set that up?

is it possible to enable authentication for the UI on the helm chart deployment?

I’m using Helm and Terraform to deploy Prefect on Kubernetes (AWS EKS). Since the service endpoints are generated dynamically on deployment, I can’t configure the UI’s default GraphQL URL to point to the Apollo endpoint when I install the helm chart. Currently, I have to manually go the the Prefect UI and set it - as would any user. I’d rather have the setup completely automated, if possible. Any help is appreciated.

Does the self-hosting include an enterprise support option?

In a flow, I'm looking to collect the result of several different tasks, and then pump that final array of task results into another task (which basically sends the results out via email). I'm having some struggles with this - are we able to add task results to an array within a flow?

Hi, I'm deploying the version 2021.09.02 of the prefect-server helm chart (latest I believe) with ingresses enabled, and when I navigate to the address for the prefect-ui, the page is stuck with a loading bar. (see image below). The apollo url works as expected. Could you point me in the right direction as to why it's stuck loading?

Hi! We are using Prefect with an S3 backend for a few month now and have accumulated >100GB in

.prefect_result

files. Is there an automated way in Prefect to clear all data older than a certain date, or do we have to rely on S3 for that?

Hi ! I have some questions on memory usage in Prefect. We're running our flow using Kubernetes Run with a LocalDaskExecutor. The flow maps a bunch of tasks, with a total of about 50K tasks for it to finish. What we see is that the memory usage of the runner pod steadily increases for all the duration of the flow. Whenever it reaches its predefined memory limit, we see a sudden drop in usage but the flow keeps running properly. Is it something that is to be expected from a flow with that many tasks ? Is there anything we could do to prevent using as much memory ?

Hi, We are have a odd issue where after around 30 minutes of a flow running, the arguments passed to a task appear as NoneTypes. "`"AttributeError: 'NoneType' object has no attribute 'start_date'""` We running prefect server 0.15.6 with the KubernetesRun run config and DaskExecutor. I have verifyed that the time is the issue by implementing a delay in a upstream task - without the delay the flow run completes successfully.

Hi, How can I set Prefect limit on the number of flows an agent can run? I want to make sure the agent only runs 1 flow. I'm using the cloud version with standard option.

Hi, I am facing issue with connecting the UI to graphQL endpoint. I ran Prefect server using Docker in a GCP VM, which doesn’t have any public IP as it sits behind a bastion host. I can connect to UI via ssh tunnelling, however on UI it says

Couldn't connect to *Prefect Server* at *<http://localhost:4200/graphql>*

. It’s mentioned here that the public IP of the VM should be used, but my VM doesn’t have any public IP.

Hi ! I'm trying to run flows using temporary Dask clusters as described here. Using the exact same code as in the article, it does not seem to work as intended. When launching the flow, a job pod is properly created, followed by a dask root pod. It lasts for a few seconds then the dask pod disappears and the job pod terminates. Job pod logs are below. Got any idea what might be going wrong ?

I am running my flows in DockerAgent, the image I want to use is in private gitlab registry, how can I specify username and token required to pull the image ?

Hi everyone 👋 I’m trying to run a Flow locally, with a local agent and can’t get it past the “Submitted for execution” state.. Tried a lot of different stuff: • shutting down the agent (though this seems like something not very well documented, please correct me if I’m wrong) • matching the labels of the agent and the flow • switching things up with the environment variables (and Secret()) In the agent logs this is the error I get. The messed up thing is that everything was running smoothly 2 days ago..

Hey, Is there any documentation on how prefect and Kubernetes work together? It is a little unclear if all flows run get submitted to Kubernetes even if there is no capacity or if prefect queues these and submits the job when capacity exists, If it is Perfect that manages this is it Kubernetes agent that deals with this? TIA

was someone able to run prefect server in local with docker container or something ?

I’m having trouble running prefect server on my local machine (MBP with intel processor). I believe it’s run successfully before, but today I’m getting this error message:

Error response from daemon: Ports are not available: listen tcp 127.0.0.1:5432: bind: address already in use

. I’ve tried restarting the computer, reinstalling docker desktop, and making a fresh conda environment with a fresh installation of prefect, but the error persists. When I run

lsof -i:5432

, I get no output. Any ideas for next troubleshooting steps?

Hi everyone. Being a current (soon to be former) user of Airflow, I have decided to move our infrastructure to Prefect, and I am just starting to test some pipelines. They're so easy to configure. However, I am wondering what would be the cleanest way to make a flow retry an upstream task if a downstream task fails. Is there something like Airflow subdags, allowing to group tasks inside a flow? Thanks a lot

Is it possible to use a github app or a sshkey auth instead of a github access token?

Hello - I am mostly deployed and things are working well - running prefect server with DaskExecutor. I have created a custom wrapper around the Flow class so that I can easily set the run environment for local development vs production flows. I have a function that is creating the dask scheduler/worker through dask gateway - all works, EXCEPT that the dask cluster is also spun up on registration. Is there an easy way to prevent this? I am wondering if I have to make a custom dask class and use that as part of the executor, rather than a custom function that creates it as part of initiating the flow.

Hello, i tried to run prefect server start but i'm getting message: ModuleNotFoundError: No module named 'packaging'. Somebody with same problem ?

Hello, I am using Docker agent to run flows, one of the task inside the flow needs to interact with k8s cluster which requires KUBECONFIG, currently I am making this available with

--volume

flag, not sure if I am doing it correctly here, what is the best practice ?

hey all I am trying to deploy flow runs with a local agent on a local server started with

prefect server start

any full examples of such use cases or best practices? just for additional context running into some issues while working inside of a devcontainer where access to

localhost

is klunky if possible is there a sample flow file that shows how to run and deploy the flow on the server backend?

Hello everyone! Doing some experiments to see if Prefect might be a good fit for us. I've tried running the examples with

server

here: https://docs.prefect.io/core/examples/overview.html#running-with-prefect-cloud-or-server But running the jobs results in

Failed to load and execute Flow's environment: ValueError('Flow is not contained in this Storage')

I'm running everything on a local agent to start. What obvious thing am I missing? Thanks in advance.

is there a recommended approach for using the prefect cli with multiple server installations? I have a dev and prod setup and I'd like to be able to do something like

prefect server create-tenant --profile dev

The prefect helm chart's values.yaml files says that

serverVersionTag

is

used for the apollo, graphql, and towel services

but the graphql image comes from the hasura repo on dockerhub, not the prefecthq one so the versioning is different. Is this a mistake or am I missing something?

Any idea what this means? It's in the

Failed flows

section of the dashboard. I can't find any errors in pod logs

Something went wrong while trying to fetch failed flow information. Please try refreshing your page. If this error persists, return to this page after a few minutes.

Does anyone know of a prefect image that has an ssh client installed? I just need this (from official docs)

FROM prefecthq/prefect:latest
RUN apt update && apt install -y openssh-client

I feel like someone else has to have made this and put it out on dockerhub already. I know, I'm being lazy. 🙂

Hi, for the docker-compose.yaml generated by

prefect server config

, is there any way I can configure the UI service to start in other port than 8080 in the container? I don’t mean port mapping to docker host 8080:8080 here. I mean the port that UI service runs at in the container itself. Perhaps through some environment variables like UI_PORT?