prefect-community #prefect-community

Are they suggestions on which dataframe library to use with prefect, other than pandas? dask being one option, but I find the lazy-loading more trouble sometimes. there seem to be many others like modin, vaex, koalas, polar. Looking for something that plays very nicely with prefect -- which has many similar scheduling features.

Volker L

02/21/2023, 9:54 PM

Hi guys, hope someone can help me with my current problem. I have two virtual machines. One is running the Prefect server and I try to deploy a flow from the other one. I have started the Prefect server successfully

Copy code

ubuntu in 🌐 instance-20230102-1221 in ~ via 🅒 prefect took 5m57s 
❯ prefect config set PREFECT_SERVER_API_HOST=0.0.0.0
Set 'PREFECT_SERVER_API_HOST' to '0.0.0.0'.
Updated profile 'default'.

ubuntu in 🌐 instance-20230102-1221 in ~ via 🅒 prefect took 3s 
❯ prefect server  start

 ___ ___ ___ ___ ___ ___ _____ 
| _ \ _ \ __| __| __/ __|_   _| 
|  _/   / _|| _|| _| (__  | |  
|_| |_|_\___|_| |___\___| |_|  

Configure Prefect to communicate with the server with:

    prefect config set PREFECT_API_URL=<http://0.0.0.0:4200/api>

View the API reference documentation at <http://0.0.0.0:4200/docs>

Check out the dashboard at <http://0.0.0.0:4200>

I am able to connect to the Prefect UI in the browser, by replacing

0.0.0.0

with the ip address of the vm. On my second vm, i have done the following:

Copy code

ubuntu in 🌐 instance-20221129-2229 in yfin-db/yfin_db on  main [!?] via 🐍 v3.11.0 via 🅒 main took 6m37s 
❯ prefect config set PREFECT_API_URL=<http://xxx.yyy.zzz.xxx:4200/api>
Set 'PREFECT_API_URL' to '<http://xxx.yyy.zzz.xxx:4200/api>'.
Updated profile 'default'.

ubuntu in 🌐 instance-20221129-2229 in yfin-db/yfin_db on  main [!?] via 🐍 v3.11.0 via 🅒 main took 6s 
❯ prefect deployment apply healthcheck-deployment.yaml 
Successfully loaded 'healthcheck'
Deployment 'healthcheck/healthcheck' successfully created with id '52b59be9-fe11-49d1-acad-3257f3fc758d'.
View Deployment in UI: <http://xxx.yyy.zzz.xxx:4200/deployments/deployment/52b59be9-fe11-49d1-acad-3257f3fc758d>

To execute flow runs from this deployment, start an agent that pulls work from the 'test' work queue:
$ prefect agent start -q 'test'

As you can see, I have changed the

PREFECT_API_URL

to the Prefect server running on my first vm. Than I have applied the deployment and Prefect tells me, that the deployment was successfully created. However, the deployment does not appear in the Prefect UI. Did I miss anything?

✅ 1

Mark Li

02/21/2023, 10:23 PM

Hello All, Have Prefect Orion deployed (version 2.6.4-python3.10). When looking at the Settings section, I noticed the PREFECT_ORION_DATABASE_CONNECTION_URL was not obfuscated. Is there any way to obfuscate that? Or another way to handle the admin/password other than passing it in via the connection string for Helm?

✅ 1

Hamza Naanani

02/22/2023, 12:29 AM

Hello, I just installed a clean prefect 2.8.2 installation on an EC2 machine with a separate venv, but when I try to launch prefect scripts I get the following error :

RuntimeError: Prefect requires sqlite >= 3.24.0 but we found version 3.7.17

Is this something on my end? Thanks

Ankit

02/22/2023, 4:35 AM

Hello everyone, I have been noticing some mysql query timeout issues whereas the same exact query runs in seconds if I run the flow on jupyter. Both jupyter and prefect are in the same k8s cluster using same docker image. Prefect version - 2.7.10

Haotian Li

02/22/2023, 7:12 AM

What's the best practice for creating a flow run of a deployment in the cloud from a local python script without waiting for it to finish?

✅ 1

Nikhil Joseph

02/22/2023, 8:38 AM

any idea where an ecs agent won't pick up flows? all of them are stuck in late. had to stop the existing agent and spin up a new one for new flows to run

David Elliott

02/22/2023, 9:39 AM

Quick one - I believe the docs for state types ought to include the

NotReady

state? Might need updating?

Deceivious

02/22/2023, 10:38 AM

Hi all, I might have found a #bug in implementation of the

task

decorator OR my knowledge of python decorators fails me 😅 While using the prefect

task

decorators for some reason, the kwargs supplied while calling the decorated function are being converted into args. To test this, Ive written a minimum viable script. <<Included in the thread>> Expected use case: result when

USE_PREFECT

is set to False. Error use case: Error raised when

USE_PREFECT

is set to True. If you look over the error trace log, when printing args and kargs. kargs is empty while the data expected in kargs is being sent to args. ENV: python: 3.10.8 prefect: 2.8.0

Slackbot

02/22/2023, 2:38 PM

This message was deleted.

David Elliott

02/22/2023, 2:41 PM

I’m seeing a bug whereby the agent sometimes submits the same flow run twice in quick succession I hit Quick Run via the UI - is this known about? I’d have expected there to be some idempotency controls either in the UI or the agent to ensure the same flow run doesn’t get submitted twice? I’ve had this twice today already, details in 🧵

Alexandre Guitton

02/22/2023, 3:25 PM

Hello 👋 Quick question, if I want to run my flow on Kubernetes, I need to set up a

KubernetesJob block

and use that in my deployment. Now, let's say I have three different flows, which require different python dependencies. Does that mean I need to create my three Docker images, then create three blocks, one for each, and finally associate the right block with the right flow in my deployment?

Tushar Gupta

02/22/2023, 4:03 PM

Hello everyone Quick question, I want to ask please help me out, I am unable to find out. I have a project named ABC in which flow name is let say XYZ now whenever I run my XYZ flow as everyone knows prefect give random names to the runs. Now I want to get the time at which the latest success run occurred using python. Please help me out. Thanks in advance!

Segun Adelowo

02/22/2023, 5:08 PM

@Zanie Pls can anyone help, I added the github and docker blocks to prefect deployment build command, and I get this error on the agent "_alembic_tmp_deployment" when I run a deployment via the UI. sqlalchemy.exc.OperationalError: (sqlite3.OperationalError) table _alembic_tmp_deployment already exists [SQL: CREATE TABLE _alembic_tmp_deployment ....] I am using the latest prefect version. I dont get any error if i remove the blocks from the deployment command.

Charlie Henry

02/22/2023, 7:31 PM

Hey y'all, I have a task where I am running a command in a docker container. This task ends up taking a while and I have a decent amount of logging that I'd like to stream back to the cloud UI. For some reason, the logs only come through after the task is complete. Is there a way to stream this logging back to prefect cloud?

Copy code

@task()
def docker_command(docker_image, environment_variables, command, logger):
    response = (
        docker.from_env()
        .containers.run(
            image=docker_image,
            command=f"python {command}",
            environment=environment_variables,
        )
        .decode("utf-8")
    )
    <http://logger.info|logger.info>(response)

Also, I'm using the docker-py python library for this, rather than a block which might also be a solution?

Ethienne Marcelin

02/22/2023, 8:30 PM

Hello ! hope you're having a good day. I have a simple question, why can't we write custom Filesystems inheriting from

WritableFileSystem

? With this code

Copy code

In [6]: class MyFS(WritableFileSystem):
   ...:     _basepath = ""
   ...:     def read_path(self, path):
   ...:         return
   ...:     def write_path(self, path, content):
   ...:         return
In [7]: MyFS()

I get the following error

Copy code

OSError: source code not available

😢 1

Leon Kozlowski

02/22/2023, 8:30 PM

What is the best way to debug errors in automations that use a pre-baked block? I created an automation that used the opsgenie-notify block and In my automation events page I'm seeing

prefect-cloud.automation.action.failed

with no other info

Paco Ibañez

02/22/2023, 10:29 PM

Hello! when calling

run_deployment

how can I minimize the time that the agent will take to pick up the new flow run?

John Horn

02/22/2023, 11:40 PM

Copy code

# prefect version
23:39:04.743 | DEBUG   | prefect.client - Connecting to API at <https://api.prefect.cloud/api/accounts/57e0246d-d752-4737-b29a-192e8f6eb886/workspaces/6d439078-f534-4a0a-89e0-9c4ced644d6a/>
Version:             2.5.0
API version:         0.8.2
Python version:      3.8.16
Git commit:          eac37918
Built:               Thu, Oct 6, 2022 12:41 PM
OS/Arch:             linux/aarch64
Profile:             default
Server type:         cloud

I am trying to create a deployment that calls a flow with a flow_run_name parameter. I am guessing that my prefect version needs to be updated however I don't know how to upgrade the prefect cloud version. My actual local prefect agent is using the latest 2.8.2 but I'm guessing it is prefect cloud that is stuck on 2.5.0 and I see no documentation on how to update that.

Copy code

@flow(flow_run_name='testing123')
TypeError: flow() got an unexpected keyword argument 'flow_run_name'

Jarvis Stubblefield

02/23/2023, 1:15 AM

Okay… so I’ve been running flows successfully for about 2 months or so … last night I decided to try to create a new one … have run into some issues, etc. Fixed all of that… now I received on what I believed to be my final run of this flow an error on the Prefect Cloud side of things… I don’t know what it means so I’m posting it here for some insight. Do I just retry my flow run or have I hit some limit?

Copy code

prefect.exceptions.PrefectHTTPStatusError: Server error '500 Internal Server Error' for url '<https://api.prefect.cloud/api/accounts/5626ffe9-0140-4e88-babc-4a4fc614bb99/workspaces/ee8a533d-2754-420e-87f2-2d6b084984af/flow_runs/>'
Response: {'exception_message': 'Internal Server Error'}
For more information check: <https://httpstatuses.com/500>

Samuel Hinton

02/23/2023, 2:06 AM

Hi team! I think I may have found a bug or improvement for flow deployments. When defining a schedule, one of the options is using the RRule as per https://docs.prefect.io/api-ref/server/schemas/schedules/#prefect.server.schemas.schedules.RRuleSchedule However, this doesn’t appear compatible with python’s ZoneInfo way of defining timezones:

Copy code

from zoneinfo import ZoneInfo
from dateutil.rrule import MINUTELY, rrule
from prefect import flow
from prefect.deployments import Deployment
from prefect.server.schemas.schedules import RRuleSchedule

london = ZoneInfo("Europe/London")

@flow
def some_flow():
    print("whoa")

Deployment.build_from_flow(
    flow=some_flow, 
    schedule=RRuleSchedule.from_rrule(
        rrule(
            freq=MINUTELY,
            interval=10,
            dtstart=dt(2020, 1, 1, tzinfo=london),
            byhour=range(8, 9),
        )
    ),
)

Naively, I’d expect this to work, but it fails because ZoneInfo doesnt have a name attribute

Copy code

schedule=RRuleSchedule.from_rrule(
  File "/Users/sh/Projects/flows/.venv/lib/python3.9/site-packages/prefect/server/schemas/schedules.py", line 410, in from_rrule
    timezone = rrule._dtstart.tzinfo.name
AttributeError: 'zoneinfo.ZoneInfo' object has no attribute 'name'

Theres an easy workaround here (like setting the timezone name explicitly like is done in schedules.py:413, but thought it might be good to get this working with pythons new standard way of handling timezones, especially because the old

from datetime import timezone

requires specifying an hour delta which actually doesnt get used by prefect2

Samuel Hinton

02/23/2023, 2:26 AM

Another question - what is the best practise about setting PREFECT_API_URL from code?

prefect.settings.PREFECT_API_URL

has getters but not setters and I dont want to just go under the hood. Using

os.environ

doesnt seem to work, because the env var is set after importing prefect (use case is we have dev and prod versions of prefect and we want to loop over envs and upload a common flow to all of them)

Samuel Hinton

02/23/2023, 2:58 AM

Another fun one with multiple deployments in one project. It seems like if we set up an S3 bucket for storage, each deployment uploads the entire project. For example, if I have

flows

folder filled with 20 python files, each with, say, 2 flows inside them, if I try to deploy the 20 flows, every single deployment will reupload all 20 files, which is incredibly slow. Is there a nicer way of doing this, such that I can upload everything just a single time? I see theres a skip_upload
I could use, but it feels a bit of an anti-pattern for me to use a bool flag like “already_uploaded” to toggle the skip_upload flag

Jenia Varavva

02/23/2023, 3:23 AM

Hi, I like the flow run timeline view: it’s slick and useful. However, when a flow goes over a few hundred tasks, it hangs the UI. Is there some option to disable it? Or is the best option to downgrade?

Walter Cavinaw

02/23/2023, 5:11 AM

trying to log into prefect cloud, but a number of errors:

plus one 1

Dmytro Ponomarchuk

02/23/2023, 8:36 AM

Hi there, Could anybody help me to reset an

Unhealthy

Work Queue? It just stopped processing any new flows without any reason. And no errors in the Agent log which is running on AWS ECS infrastructure

Tolga Karahan

02/23/2023, 9:04 AM

Hi guys. In what format base job manifest should be supplied in the UI? I'm trying this one got validation error while deploying: `RuntimeError: Unable to load 'kubernetes-custom-job' of block type None due to failed validation. To load without validation, try loading again with

validate=False

Samuel Bunce

02/23/2023, 9:15 AM

hi prefect, we had the following error last night

Copy code

prefect\client\base.py", line 130, in raise_for_status
    raise PrefectHTTPStatusError.from_httpx_error(exc) from exc.__cause__
prefect.exceptions.PrefectHTTPStatusError: Server error '502 Bad Gateway' for url
'<https://api.prefect.cloud/api/accounts/xxxxx/flow_runs/filter>'
For more information check: <https://httpstatuses.com/502> 
Agent stopped!

which we believe is caused by a failure to communicate with the cloud instance. We have a default argument for number of tries, which is set here to 3 https://github.com/PrefectHQ/prefect/blob/42e5a978390575b8bd6ef4c258ba703f7e28fca2/src/prefect/utilities/services.py#L18 - ideally we would like to be able to reconfigure it or set it to infinite so that when our agents cannot communicate with cloud, they don't all collapse - we had to restart everything this morning. As another point, this obviously generated hundreds of late flows. Why is there no 'select all' function on the UI, or capability to remove all late flows. The best I have been able to do so far is to go through and manually click to remove all late flows. Feel like there should be more functionality for actions on batches of flow runs

Stephen Lloyd

02/23/2023, 11:32 AM

How are folks managing versions of custom utilities modules. We build a separate docker image for every flow. We don't want to rebuild every image for any change to the utilities module. Does anyone have a similar setup and how are you handling changes?

Tolga Karahan

02/23/2023, 2:03 PM

Hi guys. How can I use CLI to create a block from a custom template like doing in Python? https://prefect-community.slack.com/archives/CL09KU1K7/p1677150966414349?thread_ts=1677143084.199779&cid=CL09KU1K7