how are you organizing your prefect server environment and deployments? lets say i have a folder with X projects and i have a folder with respective environments (one for each project). i therefore need at least one deployment py-file for every project, in which i can define the venv to use, etc. from what i understood, i need to call each deployment py file in order to deploy it to the prefect server. in order to not do this manually and to restart the process after a reboot i would define it as a systemd service. but then i would have to define X services and manage them manually, which is cumbersome, e.g. given the amount of projects. in contrast, with airflow, i would only join airflow as systemd service and it would scan for the DAGs and run them accordingly. is there something similar possible in prefect, like a meta-deployer thats capable of handling all the folder/import/venv structures?

Hi all. I am interested in using Prefect to run workflows on an HPC machine with the Slurm job scheduler. However, there isn't much documentation for how to use

prefect-dask

to achieve this. Does anyone have some minimal working examples for how to dispatch individual tasks in a flow as Slurm jobs?

Hello, I am trying to edit the

parameter_openapi_schema

of an existing deployment.

I have an install issue 😅 In 🧵

hello guys. I need help here. so, i have a simple flow that do extract and load data. in the extract step i get a result from cx_oracle basiclly ~70k row and 77 columns. the result is passed to load task (again using cx_oracle). i think the process inside task is success without problem (in prefect and i try a basic python function call on notebook). but the problem is in prefect there is a gap between the end of extract task to the start of load task (will provide an image), the gap time is relativelly big, sometimes even longer than the extract task process. is there some problem in passing data between task or what? the flow running on kubernetes-job and i use on premise prefect server and agent (all in same cluster)

Some of my prefect tasks need to use a database connection. Connections are not serializable, so tasks would have to construct one on run, but this is pricy. If you have a bunch of tasks that require the connection is there a way to provide them with something like a connection pool?

i want to use PostgresExecute - how do i pass the password to this task? it only seems to be possible through the run method?

Hi, I couldn't create a tenant, and got this error when running 'prefect server create-tenant -n default'. raise ClientError(result["errors"]) prefect.exceptions.ClientError: [{'message': "[{'extensions': {'path': '$', 'code': 'validation-failed'}, 'message': 'no mutations exist'}]", 'locations': [{'line': 2, 'column': 5}], 'path': ['create_tenant'], 'extensions': {'code': 'INTERNAL_SERVER_ERROR', 'exception': {'message': "[{'extensions': {'path': '$', 'code': 'validation-failed'}, 'message': 'no mutations exist'}]"}}}]

Hi, is there a way to safely delete Prefect logs from Postgres without messing up scheduled flows? The tables log, task_run and task_run_state can grow rapidly in size, so I want to delete them every X days, but only for the ones that are ran and not scheduled.

Hi all, i am getting this error

ModuleNotFoundError: No module named 'dask.system'

while importing

File "/Users/mkorad/PycharmProjects/altruistic-armadillo/src/**", line 1, in <module>

from prefect import task, Flow, Parameter

Any idea about this error?