If there was work done to implement tenants for server, would a PR be entertained? Or is that a hard segregation you are trying to maintain at all costs between cloud / server?

Hello Prefect Team, I'm trying to use local secrets with the core server setup to test some things. However, I constantly get an key error even as i set the environment variable via the agent as such: prefect agent start docker --env PREFECT__CONTEXT__SECRETS__VAULT_TOKEN=<REDACT> --api <REDACT> Unexpected error: HTTPError('400 Client Error: Bad Request for url: <REDACT>/graphql') Traceback (most recent call last): File "/usr/local/lib/python3.7/site-packages/prefect/client/secrets.py", line 137, in get value = secrets[self.name] File "/usr/local/lib/python3.7/site-packages/box/box.py", line 335, in getitem raise BoxKeyError(str(err)) from None box.exceptions.BoxKeyError: "'vault_token'" Am i missing something in my setup? use_local_secrets is set to true in the config.

Hi wonder of anyone could help? I have a colleague running windows and they get the following error when they run

prefect server start

- traceback in thread

Hey folks 🙂 I've started experimenting with ECS agent to replace our current Fargate agent, but I'm having issues with AWS permissions. Is there a place where I can find the minimum set of permissions that an ECS agent needs to successfully deploy a flow as an ECS task?

Hi all, I've built an ETL pipeline with prefect locally and know that I need to run the prefect local agent start to pick up scheduled workflows. I am currently doing it on Powershell. My question is if I close the powershell, how do I know if the agent is still running? Is there a way to make sure it is always running so that my scheduled tasks will run on schedule?

I am trying to deploy the prefect pipeline to the window server 2012 r2 but it doesn't work since docker is not compatible with r2. Is there any other workaround or do I have to use linux to run prefect?

Trying to add DaskExecutor to a flow I run on K8s. I'm currently using KubernetesRun and Docker for storage. When I add DaskExecutor I get and error while building the flow image. On the thread the code I'm using

Hey all - I am registering my flows with the following (to a server deployment on k8s);

def register_deploy():
    # pylint: disable=missing-function-docstring
    with prefect.context(
        config={
            "cloud": {"graphql": os.environ["GRPAHQL_ENDPOINT"]},
            "server": {"ui": {"endpoint": os.environ["UI_ENDPOINT"]}},
        }
    ):

        for flow_name, flow in all_flows.items():
            <http://logging.info|logging.info>(f"Registering flow: {flow_name}")
            flow.register(project_name=PROD_PROJECT, labels=[SHARED_LABEL])

This works (in terms of registering) but the stdout is the following;

Result check: OK
Flow URL: <http://localhost:8080/default/flow/a6013647-a31e-4d3c-9e7e-82e69c02f71e>
 └── ID: d78a0fc5-3990-461d-91d7-593a4278b6b2
 └── Project: prod
 └── Labels: ['prefect-agent']

I thought setting

config.server.ui.endpoint

would lead to that endpoint being printed but it seems to not (

localhost:8080

as opposed to

UI_ENDPOINT

) - any ideas?

Hi Again; I am running into the following error when registering flows on CI (works perfectly from my local machine);

Haha Me again! So I am having some trouble with attaching

Result

classes to my flows. If I specify the result class in the same file that the flow is defined via

flow.result = ...

everything works as expected. However if I import the flow def and then attach the result class (same way just outside the flow def) I do not get any stored results. I have put together a minimal repo here -> https://github.com/josh-gree2/minimal-repo/compare/result-in-cli...master (the two branches show what I mean)

Hi everyone

I would like to ask how do I get more information to find the issue with a failed flow. In my logs I juts got

Process PID 266 returned non-zero exit code

is there a way to edit the command on k8s jobs? it's

prefect execute flow-run

but I want to add a command before that

Could anyone help me understand updating flows? Do I always need to re-register all my flows and flow of flows? I built a script that register all my flows and flow of flow. However everytime I make an update to my flows, it takes a long time as if it is running a whole cycle on the backend. Am I doing it right?

Hey Team! i have some general questions about secrets and script storage. When i use Gitlab Storage to register flows, it seems that they still look for secrets i plan to add onto the agent during execution; yet an error is thrown when i'm just registering the flow:

ValueError: Local Secret "VAULT_TOKEN" was not found.

Is this the expected behavior? should all secrets be registered wherever flows are created/registered?

Hello again! So I have run into the following issue - seems to have appeared out of nowhere since don't think it was happening yesterday... I am registering my flows on CI - get the following output;

INFO:root:Registering flow: weather_data_etl
Result check: OK
Flow URL: <http://localhost:8080/default/flow/b366cae3-8200-42d9-87b5-f621711e897a>
 └── ID: 2570985c-efd0-4b29-88f1-96b86862d34a
 └── Project: prod
 └── Labels: ['prefect-agent']
INFO:root:Registering flow: test_flow
Result check: OK
Flow URL: <http://localhost:8080/default/flow/96b48708-c731-4dd7-9ba0-f354b8ef7028>
 └── ID: 1307c478-cc0e-49ee-b4b8-b00a8834eed6
 └── Project: prod
 └── Labels: ['prefect-agent']

So all seems good... However if I go to the UI there are no flows visible at all - that being said if I look into the postgres DB then the flows with the above IDs both exist...

Could anybody give me some idea of sensible default resource limits for the helm chart? They are all unspecified. My apollo server seems to be having trouble with connections and I wonder if its because the deployment is not requesting enough resources? I have a few mapped tasks feeding into each other and all goes well until towards the end when some of the mapped tasks finish in a pending state - this is completely random behaviour some times everything works as expected. The particular error I get in the logs is this in the thread

Hi everyone, I have prefect-server set up via minikube and the official helm charts (prefect-server-2021.03.10) including a Kubernetes Agent set up via helm chart. I want a flow to use the DaskExecutor so I've set the flow.executor to an instance of the DaskExecutor. I'm able to register and run the flow, but when I run the flow a job pod starts, but no other pods do where I would expect dask workers to spin up. Does anyone know what is going on here? The simple task and dask worker definitions can be seen at https://github.com/Adam-D-Lewis/sample-flow if needed.

Quick question: is there a setting or a flag that will not allow prefect to queue a flow that is identical to one that is currently running in the schedule queue?

Hey guys! is there a way to execute a registered flow via the prefect CLI? not using flow.run() but actually doing a "quick run" for example on the UI

Hi all! I'm looking to spin up a second K8 agent in the same cluster as a dev agent, but it does not seem to appear when following the kubectl install options. Any pointers?

Hello, I'm having an issue starting prefect server with persistent storage on Windows 10, any help would be great! I've tried using my own config.toml file where the only different is the UI's Apollo URL. I've also tried a docker-compose file ( however I admit I'm not the best with docker )

Struggling after Update to 0.14.12 to start prefect-server. Hi all 👋 . I just upgraded to prefect 0.14.12 (from 0.14.4) running server on a machine of us (hosting more docker apps, so maybe something changed outside of my control as other people use it to start apps, too). I do connect to the system via VPN which might create more complications? After the update my command:

prefect server start --use-volume -d

is pulling the images, creating the network and creating the parts (tmp_postgres_1, tmp_hasura_1, …). However, it never finishes (no detach) and I do get errors in the logs of tmp_apollo_1:

Trying again in 3 seconds...
Building schema...
FetchError: request to <http://hasura:3001/v1alpha1/graphql> failed, reason: connect ECONNREFUSED 192.168.64.3:3001
    at ClientRequest.<anonymous> (/apollo/node_modules/node-fetch/lib/index.js:1461:11)
    at ClientRequest.emit (events.js:315:20)
    at Socket.socketErrorListener (_http_client.js:469:9)
    at Socket.emit (events.js:315:20)
    at emitErrorNT (internal/streams/destroy.js:106:8)
    at emitErrorCloseNT (internal/streams/destroy.js:74:3)
    at processTicksAndRejections (internal/process/task_queues.js:80:21) {
  type: 'system',
  errno: 'ECONNREFUSED',
  code: 'ECONNREFUSED'
}
Trying again in 3 seconds...

Not sure what’s going on? My config is pretty simple, too:

[server]
    [server.database]
    host = "localhost"
    port = "5432"
    host_port = "5434"

    [server.hasura]
    host = "localhost"
    port = "3000"
    host_port = "3001"

The micro-services seem to be up and running:

(prefect) [root@myserver:~]$ docker ps
CONTAINER ID        IMAGE                           COMMAND                  CREATED             STATUS                    PORTS                              NAMES
aec1a48767a3        prefecthq/ui:core-0.14.12       "/docker-entrypoint.…"   17 minutes ago      Up 17 minutes (healthy)   80/tcp, 0.0.0.0:8080->8080/tcp     tmp_ui_1
2996c1840fe1        prefecthq/apollo:core-0.14.12   "tini -g -- bash -c …"   17 minutes ago      Up 17 minutes (healthy)   0.0.0.0:4200->4200/tcp             tmp_apollo_1
2263ca7701e6        prefecthq/server:core-0.14.12   "tini -g -- python s…"   17 minutes ago      Up 17 minutes                                                tmp_towel_1
ed29fdb1b3b0        prefecthq/server:core-0.14.12   "tini -g -- bash -c …"   17 minutes ago      Up 17 minutes (healthy)   0.0.0.0:4201->4201/tcp             tmp_graphql_1
2a3dffb7d641        hasura/graphql-engine:v1.3.3    "graphql-engine serve"   17 minutes ago      Up 17 minutes (healthy)   0.0.0.0:3001->3000/tcp             tmp_hasura_1
cafbde90dc48        postgres:11                     "docker-entrypoint.s…"   17 minutes ago      Up 17 minutes (healthy)   0.0.0.0:5434->5432/tcp             tmp_postgres_1

Any help would be much appreciated!

Hi everyone, I upgraded my Prefect server from

0.14.6

to

0.14.12

and since then I see even when idle, CPU hovers between 10%-50%.

docker stats --all --format "table {{.ID}}\t{{.Name}}\t{{.CPUPerc}}\t{{.MemUsage}}"

shows the following:

CONTAINER ID        NAME                CPU %               MEM USAGE / LIMIT
c6eda45c6f59        tmp_ui_1            0.00%               4.953MiB / 7.432GiB
2466edd82357        tmp_towel_1         0.00%               52.57MiB / 7.432GiB
e9d0643d134e        tmp_apollo_1        2.69%               62.68MiB / 7.432GiB
ad41dcf8646a        tmp_graphql_1       3.24%               69.5MiB / 7.432GiB
a8f65bcb2efa        tmp_hasura_1        4.62%               153.8MiB / 7.432GiB
bc9742482bd8        tmp_postgres_1      4.64%               27.86MiB / 7.432GiB

I accidentally clicked "Cloud" instead of "Server" when configuring my server's graphql endpoint... now I get the following error when I try to visit my server:

it works in an incogneto tab... is there a way to correct this without wiping all of my browser storage?

Hello everyone! I was thinking about a situation where we might use Prefect and the following unusual? unique? challenge came to mind. Our tasks/flows can include custom code written by customers. It's messy, but a very powerful model which allows them to write specialized tasks. What we would want is for each worker container to have work from only one customer. The scheduler could spin up multiple container per customer, but never multiple customers' work in a single container. Is there any way to implement that in Prefect without falling back to a single container per flow?

Hello everyone 👋. We're hosting prefect server in k8s and also running our flows there using the k8s agent. Everything works great, we have a Slack notification (state handler) setup to notify us when something goes wrong. However, when the pod running the flow is killed externally (e.g. due to OOM), we get no slack notification. This makes sense because the notifications happens from within the flow pod. Is there a way how to send these kind of notifications from other Prefect server components (towel) ? Or should this case be handled on kubernetes cluster with OOMkill alerting?