https://prefect.io logo
Join Slack
Powered by
<@ULVA73B9P> when calling a subflow I get an error...
# ask-marvin
a
@Marvin when calling a subflow I get an error Response: {'exception_message': 'Invalid request received.', 'exception_detail': [{'type': 'value_error', 'loc': ['body', 'parameters'], 'msg': 'Value error, Flow run parameters must be less than 512KB when serialized.', 'input': {'run': {'_state': {'fields_cache': {'project': {'_state': {'fields_cache': {}, 'adding': False, 'db': 'default'}, 'id': 'bdfd2301-a05f-464f-8903-4067adcfc95d', 'title': 'tests', 'user_id': 1, 'created_at': '2026-03-03T220926.224451+00:00', 'updated_at': '2026-03-03T220926.224466+00:00'}, why does this happen and how can I prevent it from happening?
m
✅ thought for 133.9 seconds
This happens because Prefect stores subflow parameters as serialized API payload data, and Prefect 3.x enforces a 512 KB max serialized parameter size by default. In your error, it looks like you’re passing a very large Python object into the subflow — likely something like a Django model instance, ORM object, or a nested object containing lots of internal state (
_state
, 
fields_cache
, related objects, etc.). Once Prefect tries to JSON-encode that for the child flow run, it exceeds the limit. Why it happens - Calling a subflow creates a new flow run - That new flow run includes the subflow’s parameters - Prefect serializes those parameters and sends them to the API - If the serialized JSON is larger than 512 KB, the API rejects it with: 
Flow run parameters must be less than 512KB when serialized.
From your snippet, this part is the giveaway: 
'_state': {'fields_cache': ... }
That strongly suggests a complex ORM/model object is being passed instead of a small primitive payload. Common cause Something like this:
Copy code
@flow
def child(run):
    ...

@flow
def parent():
    run = SomeDjangoModel.objects.get(...)
    child(run=run)   # too large / too complex
Even if that object “works” in normal Python, Prefect still needs to serialize it for the subflow run request. How to prevent it 1. Pass IDs or small primitives instead of full objects
Copy code
@flow
def child(run_id: str):
    run = MyModel.objects.get(id=run_id)
    ...

@flow
def parent():
    run = MyModel.objects.get(...)
    child(run_id=str(run.id))
2. Pass only the fields you actually need
Copy code
@flow
def child(run_data: dict):
    ...

@flow
def parent():
    run = MyModel.objects.get(...)
    child(run_data={
        "id": str(run.id),
        "title": run.title,
        "user_id": run.user_id,
    })
3. Store large data externally, pass a reference If the child flow needs a large payload: - write it to a DB - store it in S3/GCS/Azure - persist it as a file/object - then pass only the path/key/id Example:
Copy code
@flow
def child(s3_key: str):
    # load the full payload from storage here
    ...

@flow
def parent():
    key = write_large_payload_somewhere(...)
    child(s3_key=key)
4. Avoid passing ORM objects, querysets, response objects, or rich class instances These often serialize badly or become huge because they include: - internal state - relationship caches - metadata - nested related objects How to debug it Before calling the subflow, inspect what you’re passing:
Copy code
import json
from fastapi.encoders import jsonable_encoder

payload = {"run": run}
encoded = jsonable_encoder(payload)
size = len(json.dumps(encoded, separators=(",", ":")).encode())

print(size)
print(type(run))
print(encoded)
If that size is large, you’ve found the issue. Can the limit be changed? Yes, Prefect has a configurable API parameter size limit, but I’d treat that as a last resort. In most cases, the better fix is to pass smaller inputs. The relevant setting is 
server.api.max_parameter_size
. If you truly need it, you can increase it in your Prefect server configuration, but that usually just masks the underlying design issue. Best practice For subflows in Prefect, think of parameters as: - identifiers - config - lightweight metadata Not as: - full records - large datasets - ORM instances - in-memory application objects If you want, paste the subflow call signature and the type of 
run
 you’re passing, and I can help rewrite it into a safe pattern.
a
@Marvin do the return values of a flow also have the same problem, or can these be any size?
m
❌ Error: unhandled errors in a TaskGroup (1 sub-exception)
Sorry, I encountered an error while processing your request. Please try again.
a
@Marvin do the return values of a flow also have the same problem, or can these be any size?
m
❌ Error: unhandled errors in a TaskGroup (1 sub-exception)
Sorry, I encountered an error while processing your request. Please try again.
a
@Marvin do the return values of a flow also have the same problem, or can these be any size?
m
✅ thought for 240.3 seconds
Short answer: no, not the same limit. Flow parameters have the 512 KB API/database limit. Flow return values are handled differently: - if result persistence is enabled, the return value is written to result storage and Prefect stores only metadata/a reference - if result persistence is not enabled, the value may only exist locally/in memory for that run context and won’t be reliably available as a persisted result So in practice: Parameters - sent to the API when creating the flow run - must be small - default limit: 512 KB serialized Return values - are not subject to that same 512 KB parameter check - can be much larger if they are persisted to result storage Important caveat for subflows Even though return values don’t have the same parameter limit, passing a large return value from one flow into another subflow call can still cause the same problem at the next boundary. Example:
Copy code
@flow
def child():
    return huge_object

@flow
def parent():
    x = child()
    grandchild(data=x)  # this can fail, because now x becomes a parameter
So: - returning a huge object: generally okay with persistence - re-passing that huge object as a subflow parameter: can fail Best practice - Use flow/task returns for large data only when result persistence is configured appropriately - For large cross-flow handoffs, prefer: - object storage paths - database IDs - document keys - lightweight metadata references Example safe pattern:
Copy code
@flow
def produce():
    path = write_large_data_to_s3(...)
    return path

@flow
def consume(path: str):
    data = read_large_data_from_s3(path)
If you want, I can also show: 1. how to check whether result persistence is enabled in your setup, or 2. what happens specifically for parent flow -> subflow return passing in your case.
a
@Marvin yes, how do I check for result persistence? as long as the value is available in my local function I don't care about anything else
m
✅ thought for 20.9 seconds
If you only care that the value is available inside the current Python process when one flow calls another, then you usually don’t need to care much about persisted results. A subflow call like:
Copy code
@flow
def child():
    return {"big": "value"}

@flow
def parent():
    x = child()
    print(type(x), x)
will give you the return value locally in 
parent()
as normal Python data. The important distinction is: - local in-process availability: yes, you get the value back - stored/recoverable later by Prefect: that depends on result persistence How to check whether result persistence is enabled Option 1: inspect the flow definition If you explicitly set it, it will look like this:
Copy code
@flow(persist_result=True)
def my_flow():
    ...
or:
Copy code
@task(persist_result=True)
def my_task():
    ...
If you did not set 
persist_result=...
, then Prefect may be using defaults/settings. Option 2: check at runtime with Prefect settings You can inspect the current setting value:
Copy code
from prefect.settings import get_current_settings

settings = get_current_settings()
print(settings.results.persist_by_default)
If that prints 
True
, results are persisted by default unless overridden. If it prints 
False
, they are not persisted by default unless you explicitly enable it. Option 3: check from inside a flow You can do:
Copy code
from prefect import flow
from prefect.settings import get_current_settings

@flow
def my_flow():
    settings = get_current_settings()
    print("persist_by_default:", settings.results.persist_by_default)
What matters for your case Based on what you said:
as long as the value is available in my local function I don't care about anything else
Then the key thing is: Returning a large object from a subflow to its parent is fine for local use in the same run/process. What breaks is when Prefect has to turn that object into parameters for another flow run. So this is usually okay:
Copy code
@flow
def child():
    return huge_object

@flow
def parent():
    x = child()
    print("Got it locally:", x is not None)
But this can fail:
Copy code
@flow
def child():
    return huge_object

@flow
def grandchild(data):
    ...

@flow
def parent():
    x = child()
    grandchild(data=x)   # may hit the 512 KB parameter limit
Rule of thumb - subflow return -> use immediately in Python code: okay - subflow return -> pass into another flow/subflow as a parameter: risky if large - subflow return -> pass into a task in the same flow: generally much less of an issue than subflow parameters, though huge objects can still be inefficient If you want to verify behavior in your environment Try this minimal test:
Copy code
from prefect import flow

@flow
def child():
    return {"data": "x" * 800_000}

@flow
def parent():
    x = child()
    print("Length:", len(x["data"]))
    return len(x["data"])

parent()
If 
parent()
can print the length, then the large return value is available locally just fine. Then test the failure mode:
Copy code
from prefect import flow

@flow
def child():
    return {"data": "x" * 800_000}

@flow
def grandchild(data):
    return len(data["data"])

@flow
def parent():
    x = child()
    return grandchild(x)

parent()
That second version is the one likely to trigger the parameter-size error. If you want, I can also show you the safest pattern for large objects between parent flow, subflow, and tasks so you can keep your current structure without hitting this again.
Open in Slack
PreviousNext
Powered by