Hey, not sure if this the right place to ask, can anyone provide an idiomatic example of using Oracle DB connection across multiple tasks please? I am helping out a colleague with an ETL flow which requires multiple queries done across multiple connections, which would preferably persist throughout the flow. The initial problem we are encountering is that you cannot pass the connection object between tasks as it needs to be serialisable.

@Marvin are there any best practices on how to optimize the self-hosted prefect postgresql database? Opening the "Runs" tab takes minutes, and I feel like I need to actively delete old runs on a schedule. But I would like to know the best practices.

I've been using this Dockerfile for almost a year now without issue, but issues have cropped up since starting to migrate to Prefect 3.

Hi all, I am going in circles with resolving the misconfiguration between prefect and pydantic. I have tried many versions, but currently failing with prefect 3.1.12 and pydantic 2.7.0 Using above config

pydantic.errors.PydanticUserError: If you use `@root_validator` with pre=False (the default) you MUST specify `skip_on_failure=True`. Note that `@root_validator` is deprecated and should be replaced with `@model_validator`.

Tried reverting back to pydantic 1.10.11 as that's what we has used in the past.

And because prefect (3.1.11) depends on pydantic (>=2.7,<2.10.0 || >2.10.0,<3.0.0)
 and prefect (3.1.13) depends on pydantic (>=2.9,<2.10.0 || >2.10.0,<3.0.0), prefect (>=3.0,<3.1.12 || >3.1.12,<3.1.14 || >3.1.14,<4.0) requires pydantic (>=2.7,<3.0.0).
And because prefect (3.1.14) depends on pydantic (>=2.9,<2.10.0 || >2.10.0,<3.0.0)
 and prefect (3.1.12) depends on pydantic (>=2.7,<2.10.0 || >2.10.0,<3.0.0), prefect (>=3.0,<4.0) requires pydantic (>=2.7,<3.0.0).

Reverting to prefect 2.20.X I get the following for any prefect cli or run.

pydantic.v1.error_wrappers.ValidationError: 1 validation error for Profile
settings
  Unknown setting 'PREFECT_SERVER_ALLOW_EPHEMERAL_MODE'. (type=value_error)

@Marvin sometimes tasks log that they have completed and downstream tasks (as defined by wait_for) are kicked off, but the prefect interface shows the task still in a running state (an hour or more later). Not sure if it's relevant but a later task did fail in at least one occurance of this, and although the flow oeverall is marked as failed that failed task (and some of the successfully completed tasks) are stuck in a running state. presumably this is just a bug that will be fixed somewhere by the prefect devs rather than something i've done wrong?

I'm clearly not understanding logging well here is my sample code

~/my_package/tests/test_logging.py

import structlog
import logging
from prefect.logging import get_run_logger
from prefect import flow


test_logger = logging.getLogger(__name__)

def test():
    structlog.get_logger().info("test from structlog", y=1)
    test_logger.info("test from logging")


@flow()
def flow_func():
    test()


if __name__ == "__main__":
    flow_func.serve(
        name="test",
    )

I have set PREFECT_LOGGING_EXTRA_LOGGERS='my_package' When running from a console this I get these logs

13:36:16.218 | INFO    | prefect.flow_runs.runner - Runner 'test' submitting flow run '8b1f6602-d38b-43b0-964e-29c91a279859'
13:36:16.515 | INFO    | prefect.flow_runs.runner - Opening process...
13:36:16.524 | INFO    | prefect.flow_runs.runner - Completed submission of flow run '8b1f6602-d38b-43b0-964e-29c91a279859'
13:36:18.133 | INFO    | Flow run 'pompous-tench' - Downloading flow code from storage at '.'
2024-12-30 13:36:18 [info     ] test from structlog            y=1
13:36:18.419 | INFO    | Flow run 'pompous-tench' - Finished in state Completed()
13:36:18.734 | INFO    | prefect.flow_runs.runner - Process for flow run 'pompous-tench' exited cleanly.

and from the local UI I see none of the logs. Questions: 1. why is only the structlog showing on the console 2. why are none of the logs being propagated to the UI? (prefect 2.20.10)

@Marvin I'm running docker on ECS and got this error with my flow - (new error after pushing a new image). What might cause this? Failed due to a(n)

CannotPullContainerError

error: wrong diff id calculated on extraction

sha256:c0f1022b22a9b36851b358f44e5475e39d166e71a8073cf53c894a299239b1c5

Hi all, I'm just starting with Prefect this weekend, and I can't seem to find the answer to a perhaps simple question: it looks like once I ran

prefect server start

once, I can no longer get the ephemeral server to come up when I test the flow. Upon some debugging, it looks like the presence of ~/.prefect/profiles.toml is causing that problem. Actively set the profile to "ephemeral" doesn't help. What is the right way of getting back to the initial state? Just delete the profiles.toml?

@Marvin Can i create a worker using Prefect docker image ? For example, from this image : prefecthq/prefect:3-python3.12, can I use

prefect worker start --pool "my-docker-pool" --type "docker"

?

@Marvin how do we enable authentication in prefect 3.0 in kubernetes