Fawzi
10/21/2025, 10:29 AMMarvin
10/21/2025, 10:29 AMMarvin
10/21/2025, 10:34 AM# BAD: returns a client
@task
def get_client():
return httpx.Client()
# GOOD: use the client internally; return data
@task
def fetch_data(url):
with httpx.Client() as client:
r = client.get(url)
return r.json()
# GOOD: for Pydantic v2 models
@task
def make_model():
model = MyModel(...)
return model.model_dump() # or model_dump_json()
2) Explicitly configure result persistence on flows/subflows with a JSON serializer and a storage location
- In Prefect 3.x, you can set these on each flow/task so only serializable data is written.
from prefect import flow, task
from prefect.serializers import JSONSerializer
from prefect.filesystems import LocalFileSystem
result_store = LocalFileSystem(basepath="/opt/prefect/results") # or S3/GCS/etc.
@flow(
persist_result=True,
result_serializer=JSONSerializer(),
result_storage=result_store,
)
def parent_flow():
# call subflows/tasks and return serializable data
sub_result = child_flow()
return {"child_summary": sub_result, "status": "ok"}
@flow(
persist_result=True,
result_serializer=JSONSerializer(),
result_storage=result_store,
)
def child_flow():
data = compute_something() # return only JSON-serializable values here
return {"rows": len(data)}
Notes:
- If you have complex objects, you can use a custom object encoder with JSONSerializer to convert them to JSON-safe forms, but you still shouldn’t try to persist live client connections.
- Alternatively, if you truly don’t need Prefect to store the return values, set persist_result=False and manage persistence yourself inside the task/flow. But since you asked to keep results, prefer the JSON approach above.
3) Reduce the “monitoring” OSError noise
- Upgrade to the latest Prefect 3.x worker/runtime — we’ve improved robustness in log streaming and process monitoring over time. See Prefect release notes.
- Ensure the worker’s stdout/stderr are not being closed or redirected to an ephemeral/rotating sink while the run is still streaming logs (common with some container logging setups).
- If the error persists frequently, capturing worker logs and environment details can help us pinpoint the logging backend/source. The message itself does not mark the run failed.
Helpful docs
- Results and persistence: Prefect 3.x Results guideMarvin
10/21/2025, 10:34 AM