In Prefect we can define tasks as functions with

@task

decorator, or as classes by subclassing

Task

. In general when programming in python and presented with these options, I typically reach for the subclassing option. I find I can think about problems at a higher level with OOP, easier to build up complex components from simpler ones, cleaner abstraction, etc. However I do find it’s a bit clunky with at times when building flows. Is it recommended to avoid the subclassing approach and use a functional approach instead when possible?

When I try to use

from pydantic.v1 import SecretStr

as an input arg type to my flow I"m getting:

pydantic.errors.PydanticUserError: The `__modify_schema__` method is not supported in Pydantic v2. Use `__get_pydantic_json_schema__` instead in class `SecretStr`.

Looks like this use case was called out here, but never implemented. Should I file a bug?

@Marvin When I try to start my local prefect server, it fails with the following error: alembic.util.exc.CommandError: Can't locate revision identified by 'c63a0a6dc787' And that is not a revision from my project. How can I fix it?

Hi! quick question, what's the recommended approach to define the KubernetesJob resources (cpu and RAM requests and limits) in a flow deployment to a Kubernetes based Work Pool?

def deploy_hello_flow():
    hello_flow.deploy(
        name="hello_flow_k8s_deployment",
        work_pool_name="default-worker-pool", # k8s work pool
        image="europe-docker.pkg.dev/cambium-earth/prefect-runners-prod/default:latest",
        tags=["prod"],
        schedule=CronSchedule(cron="0 12 1 * *", timezone="UTC"),
    )

@Marvin how many tasks can be created/ran at once without causing the postgres sql connection pool to reach its limit?

@Marvin I get this error when trying to set my Prefect profile. I use Prefect 2.10.3:

Unknown setting 'PREFECT_SERVER_ALLOW_EPHEMERAL_MODE'. (type=value_error)

When I run

prefect config unset PREFECT_SERVER_ALLOW_EPHEMERAL_MODE

I get the same error

@Marvin How do I run .deploy to a process type worker

I do get this warning

RuntimeWarning: coroutine 'run_task_async' was never awaited
  return fn(*args, **kwargs)
RuntimeWarning: Enable tracemalloc to get the object allocation traceback

Probably related?

Using prefect, is it possible to implement tasks in different programming languages, we already have our business login in Nodejs and I couldn't find any docs related to this

I want to ask the Dask Experts whether they can help identify a scheduler bottleneck I have experienced. I have a flow that schedules 22 parallel mapped jobs (each with 65 items) allowing for 1430 parallel jobs. When I schedule the job on an k8s cluster with 375 nodes (each with a worker that runs 4 threads), my scheduler throws messages out like the below. The CPU utilization of the scheduler also goes to 100%. These jobs ultimately die when all the workers get disconnected.

distributed.utils_perf - WARNING - full garbage collections took 10% CPU time recently (threshold: 10%)
...
distributed.core - INFO - Event loop was unresponsive in Scheduler for 7.07s.  This is often caused by long-running GIL-holding functions or moving large chunks of data. This can cause timeouts and instability.
...
distributed.comm.tcp - INFO - Connection closed before handshake completed
...
distributed.scheduler - INFO - Close client connection: Client-worker-3c6d8642-53b5-11eb-800e-32b98c347770

When I scale the job down to 250 nodes and 3 threads per worker, I still get 100% utilization, but it is slightly more stable. where the only warning messages is

distributed.utils_perf - WARNING - full garbage collections took 11% CPU time recently (threshold: 10%)

Thanks for any advice!