Hey everyone. If I want to have my flow set up in a 'flows' folder, and then have that flow import 2 tasks from a different file, how would I go about that? What kind of import statement would I need to add at the top of my file?

Hi all, I'm looking to get started with Prefect. I've installed Prefect Server on an EC2 instance in a private subnet which I can access via a bastion server. I've successfully started the server and can see it running. I'm trying to access the ui by using a putty tunnel to forward a port on my laptop, via the bastion server, to the <machine with prefect>:8080. I configured firefox to use SOCKS5 with localhost and the port forwarded by putty. Unfortunately I'm not getting the ui to load up. It's a difficult one to debug as there's lots of components involved. Has anyone any experience doing something like this? Any tips on prefect configuration that might be pertinent?

Hey there ! Starting to try out GraphQL (first time 😬) to clean up old flow runs in order to reduce the size of Postgre Database. I manage to get flow run ids before a given date as well as deleting a single flow run from its id using

delete_flow_run

but is there a way to delete multiple flow runs at once ? Thanks !

Hi everyone! I'm new to Prefect (just a couple of months). I've been using it as provided for some local tests without an issue. I was trying to replicate this tests on a remote machine (ssh mediated) but I have runned into some issues. I'm trying to spin up a local prefect server (just testing some flows) on a remote machine. Some data on versions `prefect diagnostics`:

{

"config_overrides": {},

"env_vars": [],

"system_information": {

"platform": "Linux-5.4.0-96-generic-x86_64-with-glibc2.27",

"prefect_backend": "server",

"prefect_version": "0.15.13",

"python_version": "3.8.12"

}

}

I was setting the local server as option and started as usual:

prefect backend server
prefect server start

No special options. Images are pulled and all the services started. However some errors (regarding graphql mainly) start to apper on logs.

graphql_1   | Error applying Hasura metadata from /prefect-server/services/hasura/migrations/metadata.yaml
graphql_1   | 
graphql_1   | Could not upgrade the database!
graphql_1   | Error: Failed to apply Hasura metadata! Error: 503 Server Error: Service Unavailable for url: <http://hasura:3000/v1/query>
apollo_1    | Checking GraphQL service at <http://graphql:4201/health> ...
tmp_graphql_1 exited with code 1

(I have the full trace until stopping them). I also checked (while waiting the spin up - just displaying the failing one)

> docker ps 
7036b92f538d   prefecthq/server:core-0.15.13   "tini -g -- bash -c …"   About a minute ago   Restarting (1) 1 second ago                                      tmp_graphql_

which is continously restarting. I have tryied: • previuos Prefect version (0.15.0) with the same effect. • the same env on my local machine with success (so I suspect from the remote configuration, but I could not figure what is missing) Thanks!

Howdy, Oh man I am struggling to figure out why all my tasks are failing with

Failed to load and execute Flow's environment: ModuleNotFoundError("No module named '/home/magnox/'")

Env: • Local Agent + Local Storage (Running in docker [python:3.10] ... yeah yeah lol) • External Postgres 12 • Prefect Server in Kubernetes (Official HELM) • Django Integrated Code: The Source is located in

/code/

We have added

/code

to the path and Python path on execution. Notes: • Sometimes a single container will work fine (using the same exact image and everything)(Cant seem to figure out why) • I have tried with and without Dask • I tried to symlink the /code -> /home/magnox to no avail

Hello, do we have the ability to delete a tenant?

hello

I am compltely new

to perfect cloud

I have a question

does prefect cloud provide gpus??

Hi, Is there a way to append a string to a flow run name?

Hi, how do we setup prefect to use localstack running locally on minikube?

Hi everyone, quick question on the Airbyte operator. Let's say I have Airbyte running on an EC2 instance. And I have prefect deployed to run on ECS. Will the airbyte operator run the connection on the Airbyte EC2 instance or on the Prefect ECS?

Hi. I'm trying to guage how much Prefect Cloud would cost. Anyone have a GraphQL query (or anything) on hand to see how many billable task runs I have? (per project/flow, and over time, would be nice)

Hi, I'm still evaluating prefect as an alternative to Airflow. The docs aren't totally clear to me whether prefect server is intended for production us. Is it open source software that I can run on my own production infrastructure, or is it more supposed to be a test bed for development, with production usage always through prefect cloud?

Hi! I’m looking at the options for best recording a summary of results (e.g. “2,131 records loaded, 5,000 skipped” etc.) and was wondering if there was an option that would show up in the job tool tips like the screenshot. If not, any other suggestions of where to best/prominently log and display?

Hello, does Prefect server integrate with Okta or any other similar platforms?

What is the correct way to start the agent using an API key? Previously this worked:

prefect agent local start --token [some-token] --api <https://api.prefect.io>

But after generating an api KEY (not a token) and running a similar incantation:

prefect agent local start --token [THE-NEW-KEY] --api <https://api.prefect.io>

it complains with

prefect.utilities.exceptions.ClientError: [{'path': ['register_agent'], 'message': 'Unauthorized: permission error.', 'extensions': {'code': 'FORBIDDEN'}}]

Hey, I have a flow with manual task. I added a state handler, in order to post a Slack Message in a channel, to let know that a task is waiting for approval. Is there a way to print some kind of link to the task is running?

Hello! I'm new to prefect and I'm trying to use the prefect cli to register my a flow. Every time I run it, it returns with "0 registered." I even tried to register one of the tutorial programs (command: prefect register --project "misc" --path 03_parameterized_etl_flow.py) and it did the same thing. I was able to get one flow to register using the flow.register() method in my script, but I can't get the cli to detect my flows.

Hello to all! I have a question. Using checkpoints and targets, I set up the pipeline in such a way that successfully completed tasks are not re-executed. It’s great! But is there any mechanism that allows you to force the task to be reloaded? For example, when you need to force overwrite a file, even if it has already been created.

Is it possible to include a link to a pdf produced as output from a run using the

Artifacts

api? I’m currently using the server backed and a local agent.

The latest Orion announcement has me quite excited for "local" task concurrency limits. It reminded me of the per-agent concurrency work that wasn't possible in Server unfortunately. Any plans to pick back up in in Orion?

Hey all - I’ve been running prefect in production with no issues on

kubernetes

. This morning, no prefect job pods are spinning up - has anyone encountered this?

Hi there, does any one know if there’s a way to disable (turning off) schedule of a running flow via cli ? I’ve been looking through the documentation and there seems to be no option for that.

Hi. I'm sure the answer will be here if I search 😅 Why am I not seeing logs logged using

prefect.context.get("logger").warning()

Surely if the default logging level is

info

, it will hide

debug

, but should still log and show

warning

,

error

, and

critical

?

Hey. Anyone that can point out if there's a gotcha I missed here? I created a flow to get a flow run (by name)'s logs and email them to a user. I figured out the GQL query in the

Interactive API

tab on the Server page. I then moved it to my flow. When running the flow, I'm getting

ReadTimeoutError: HTTPConnectionPool(host='localhost', port=4200): Read timed out. (read timeout=15)

It's strange to me, because I'm using default port/other settings, and expect it to work since the

Interactive API

gives a result almost immediately

Another question, but big issue for us here. Sorry folks. We have deployed Prefect Core on an on-premise virtual machine with 4 CPU and 16 Go of RAM. Some flows regularly fail and are retried. Right now we have 10 flows retrying, and they are overloading the CPUs... Those retrying flows slow down the subsequent ones, that get slower and slower, until no flow can run anymore and the whole VM is virtually stopped. The tasks executed by the flows are not memory-intensive and, when no error is raised, they run smoothly in several seconds. But here, we are facing some critical error problems... One solution would be to reboot the whole application regularly, but it was something advised to do with Airflow and its scheduler issues, and one of the reason we switched from Airflow to Prefect. I have also optimized for the number of retries and the retry_timedelta, but that is not an optimal solution. You can find attached a screenshot of my htop command. Any thoughts on how to solve the issue? Btw we are using Prefect for launching an entirely-automated renewable electricity trading and, except for these bugs that I'm sure we can solve, we are very happy with the solution and would happily collaborate for writing blog posts showing our use-case of your solution or anything! Best, and thanks in advance!

Hello, This may be something simple that I’m just totally overlooking but trying to use the s3.s3Upload.run in my flow in the following way:

with Flow("test") as flow:
    s3.S3Upload.run(data, credentials="AWS_CREDENTIALS", bucket='bucket_name', compression='gzip')

flow.run()

Data is json string. I keep on getting this error

AttributeError: 'str' object has no attribute 'bucket'