i am able to see projects under command prefect get projects

but now able to see on UI

how to fix that ?

Hello, I'm very new here so apologies for the question if it sounds newbie-like - I'm currently reviewing some prefect 1.0 code with mind to refactor into prefect 2.0, and one of the functions for prefect that is currently being used is prefect.core.parameter.DateTimeParameter, which doesnt seem to be supported in prefect 2.0. Is that the correct interpretation? If so, what would be prefect 2.0 alternative for this? Thanks in advance

Hello here, I am new to Prefect and run my first flow in my local K8S cluster and use

KubernetesRun

. The flow is trying to download the

py

file from a local git repo in the same local K8S cluster. But I got an

SSL_VERIFICATION

error even though the

url/host

for

git storage

is

HTTP

. Is there anyone having similar issue?

Hello folks, I am trying to run Prefect server on an EC2 instance using the instructions in Prefect Docs and am facing the following error while trying to get it up and running. Has anyone faced a similar issue in hosting Prefect server?

Could not upgrade the database!
graphql_1   | Error: HTTPConnectionPool(host='hasura', port=3000): Max retries exceeded with url: /v1/query (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f25746773d0>: Failed to establish a new connection: [Errno 111] Connection refused'))
apollo_1    | Checking GraphQL service at <http://graphql:4201/health> ...
towel_1     | {"severity": "ERROR", "name": "prefect-server.Lazarus", "message": "Unexpected error: ValueError([{'extensions': {'path': '$.selectionSet.flow_run', 'code': 'validation-failed'}, 'message': 'field \"flow_run\" not found in type: \\'query_root\\''}])", "exc_info": "Traceback (most recent call last):\n  File \"/prefect-server/src/prefect_server/services/loop_service.py\", line 60, in run\n    await self.run_once()\n  File \"/prefect-server/src/prefect_server/services/towel/lazarus.py\", line 37, in run_once\n    return await self.reschedule_flow_runs()\n  File \"/prefect-server/src/prefect_server/services/towel/lazarus.py\", line 89, in reschedule_flow_runs\n    limit=500,\n  File \"/prefect-server/src/prefect_server/database/orm.py\", line 501, in get\n    as_box=not apply_schema,\n  File \"/prefect-server/src/prefect_server/database/hasura.py\", line 85, in execute\n    as_box=as_box,\n  File \"/prefect-server/src/prefect_server/utilities/graphql.py\", line 84, in execute\n    raise ValueError(result[\"errors\"])\nValueError: [{'extensions': {'path': '$.selectionSet.flow_run', 'code': 'validation-failed'}, 'message': 'field \"flow_run\" not found in type: \\'query_root\\''}]"}

Hi folks, wondering if it’s possible to get IP addresses of Prefect Cloud. I’d like to lock down the network rules for my agent running on AWS ECS, so restricting access to a fixed set of IP addresses would be useful. I’ve been googling but couldn’t find anything. Thanks

Hey, is there a way to limit concurrency on a

map

call?

I'm trying to get a DaskExecutor on ECS/Fargate to work. And having issues. The flow hangs on

Creating a new Dask cluster with None.fargate_cluster...

. I'm running on prefect==0.15.6. My code is attached. The trouble is that it just hangs and I have no clue what it's "trying" to do.

Hello! Do you have any tips for monitoring my Prefect flows that fail before they can even begin?

When using

Task.map

, is there a way to customize the repr of the child task besides seeing

Mapped Child n

where n is a number? For example

Task.map(x=list("abc"), ...)

, is there a way to see

Mapped Child a

instead?

Another question on ECS deployment.. what’s the best way to get environment variables set from my task definition? Can I use

os.environ.get

? I have also tried

EnvVarSecret

but that only seems to work within a flow?

Question regarding automations: is there a way to assign an automation at the project level? E.g. I would like to setup a slack notification anytime any flow within a project fails to start at its scheduled time. Currently (I think) I need to manually select all relevant flows; this is fine in the short term but I would like to avoid having to do this each time I add a new flow to the project.

Hey guys do we have a ETA when the 2.0 will be released to the broader audience?

Hi All. Quick question on how to create a Flow that references external files. In my case, I’m going to run a set of perhaps a dozen queries against a database, and I obviously don’t want to embed the SQL. I’d like to store the SQL external in files. Is that something I can do?

I have some weirdness going on with a flow that pulls from Azure secrets into the prefect.config object(?) Anyways, the main issue is that the code gives this error

File "/mnt/data/prefect/venv/lib/python3.8/site-packages/box/box.py", line 490, in __getattr__

raise BoxKeyError(str(err)) from None

box.exceptions.BoxKeyError: "'Config' object has no attribute 'staging_db_constring'"

However if I attempt to print out the config object the code works as expected. I haven't touched the config.toml file and Googling hasn't helped. Any help would be appreciated. Thanks

Greetings all 👋 Is there a way to create a WebhookAction that includes a header in the payload via the Automations UI? If not would anyone be able to share a GraphQL query I could use to do this?

Hi All, Is there way to start a new flow using automations? I am looking to trigger a flow B when flow A has finished.

👋 in a flow that contains subflows in prefect 2.0, is there a way to specify

wait_for

between the subflows? i'll drop a trivial example in the thread

Hi Team , We are trying to automate registering the prefect flows with Jenkins but not able to register it , can you please share a documentation which helps to register the flows

👋 Hey all! PrefectLive is starting in 15 minutes - join DevOps king @Tyler Wanner as he works on a Kubernetes deployment in Cloud 2. You can find us on Twitch, and the replay will be available on our PrefectLive YouTube playlist.

hi everyone 👋 We are interesting in using ECS Agent with Prefect 2.0. Is this something already available ? Cannot find anything related to that in the Orion doc so far. cc @Julian Mehnle

Hey guys, i have a task that can take up to 20m to run (and also have a retry decorator), i started getting

Marked Failed by a Zombie Killer process

how can i increase this tolerante and why this happening? My task only take some time but this not means that is failed

Hey folks. If I register a flow, I get an identifier back. How do I use that identifier to get the flow state information back from the service? I am probably just looking in the wrong place, but I can't find it in the docs.

Hi all, what would you consider best practice for reporting business-case-specific results from a Prefect pipeline, such as a data cleansing task or data quality checkpoint task? Would you report the results directly to the prefect logs, or is it better to keep the prefect logging to task summaries and output detailed reports as a text or html file?

What does it mean when a Task has the

Running

state but also has an

End Time

and the

Duration

isn't ticking up? Example:

Hi, to use Gitlab storage, do I have to provide my personal access token, or can I use a repository specific deploy token? As far as I can see, a personal access token gives read/write access to all of my repositories. Perhaps a project-level access token would work, but I see the docs refer to a personal access token: https://docs.prefect.io/orchestration/flow_config/storage.html#gitlab

We are mapping over a series of requests, to make multiple tasks within the same flow, • We want to set the task state to be failed/pending for each task, after it has run. • Then later we want to update the task state to be succeeded via the graphQL/rest API • We are using Prefect cloud as the API back_end. ---- • Currently what python methods/GraphQL should we use to update the state of individual task runs inside of flow runs?

Or how to make a task that runs after a new blob arrives in a google cloud storage bucket?

Hi, I'm looking to set up some Prefect-agents-as-a-service within my organisation. We're planning to use the hosted Prefect Cloud with some AWS ECS agents, and a GitHub Actions pipeline to publish new flows (probably using S3 as storage). This is all greenfield so we're pretty flexible on the details. Right now we're sticking to Prefect 1.x but if there's a good reason to jump to 2.x I don't think that would be a problem (as long as it's safe/secure we don't mind the occasional hiccough while it's still in beta) Most of that is fine; so far we're just using a local agent deployed on EC2, but I can see there's an ECS agent which we can presumably use easily enough to get any scalability that we'll need, and we can use the

prefect

CLI to push flows from GitHub Actions. But where we're hitting problems is with dependency management (both internal code which is shared between multiple tasks/flows, and external dependencies). From what I've seen, Prefect doesn't really support this at all (flows are expected to be self-contained single files), with the implication being that the agent itself has to have any shared dependencies pre-installed (which in our case would mean that any significant changes require re-building and re-deploying the agent image - a slow process and not very practical if we have long-lived tasks or multiple people testing different flows at the same time). I tried looking around for Python bundlers and found stickytape, but that seems a bit too rough-and-ready for any real use. This seems to be a bit of a known problem: 1, 2 and specifically I see:

V2 supports virtual and conda environment specification per flow run which should help some

And I found some documentation for this (which seems to tie it to the new concept of deployments), but I'm still a bit confused on the details: • would the idea be to create a deployment for every version of every flow we push? Will we need to somehow tidy up the old deployments ourselves? • can deployments be given other internal files (i.e. common internal code), or is it limited to just external dependencies? Relatedly, do deployments live on the server or in the configured Storage? • is there any way to use zipapp bundles? • ideally we want engineers to be able to run flows in 3 ways: entirely locally; on a remote runner triggered from their local machine (with local code, including their latest local dependencies); and entirely remotely (pushed to the cloud server via an automated pipeline and triggered or scheduled - basically "push to production") — I'm not clear on how I should be thinking about deployments vs flows to make these 3 options a reality. I also wonder if I'm going down a complete rabbit hole and there is an easier way to do all of this?