https://prefect.io logo
Join Slack
Powered by
<@ULVA73B9P> I want to deploy a flow using yml fil...
# marvin-ai
m
@Marvin I want to deploy a flow using yml file, however, prefect prompts to enter more information, causing me not able to use prefect deploy in ci/cd # generic metadata prefect-version: 3 name: autoupdate # preparation steps build: false push: false # runtime steps pull: false # deployment configurations deployments: - # base metadata name: autoupdate-deployment version: "0.1" tags: ["autoupdate"] description: Autoupdate schedule: null # flow-specific fields entrypoint: flows/auto_update.py:auto_update_flow parameters: {} # infra-specific fields work_pool: image: prefect-workpool build: - context: . - dockerfile: ./docker/wizard-worker/Dockerfile name: auto-update-pool work_queue_name: default job_variables: {} volumes: # Optional, for mounting JSON files - /autoupdate:/autoupdate
m
✅ thought for 214.3 seconds
You’re running into the interactive wizard. Two fixes: 1) Run the CLI non-interactively - Use the global no-prompt flag and target a specific deployment so it doesn’t ask you to choose one:
Copy code
prefect --no-prompt deploy --name autoupdate-deployment
- Make sure you’re authenticated ahead of time in CI to avoid login prompts: - Set env vars: 
PREFECT_API_URL
and 
PREFECT_API_KEY
- Or run a non-interactive login:
Copy code
prefect --no-prompt cloud login --key "$PREFECT_API_KEY" --workspace "account/workspace"
- Ensure the work pool already exists; otherwise 
prefect deploy
will offer to create it. Create it once (UI or CLI) before deployment. Docs: prefect.yaml guide • work pool management 2) Fix your YAML so Prefect has everything it needs Your current schema places 
image
and 
build
under 
work_pool
, which Prefect won’t use there. Put runtime settings under 
job_variables
and keep build/push at the top level (or use a prebuilt image and keep build/push false). Here’s a corrected minimal version that avoids prompts and uses a prebuilt image:
Copy code
prefect-version: 3
name: autoupdate

# Skip build/push/pull entirely in CI
build: false
push: false
pull: false

deployments:
  - name: autoupdate-deployment
    version: "0.1"
    tags: ["autoupdate"]
    description: Autoupdate
    schedule: null

    entrypoint: flows/auto_update.py:auto_update_flow
    parameters: {}

    work_pool:
      name: auto-update-pool
      # work_queue_name is typically not needed in 3.x; include only if you actually use queues
      # work_queue_name: default
      job_variables:
        image: your-registry/prefect-workpool:0.1
        volumes:
          - /autoupdate:/autoupdate
CI example - Authenticate (Cloud) and deploy without prompts:
Copy code
export PREFECT_API_URL="<https://api.prefect.cloud/api/accounts/<acct-id>/workspaces/<ws-id>>"
export PREFECT_API_KEY="$PREFECT_API_KEY"
prefect --no-prompt deploy --name autoupdate-deployment
Common causes of prompts (and how to avoid them) - Multiple deployments defined, but no 
--name
provided → add 
--name autoupdate-deployment
- Missing 
work_pool.name
or missing 
job_variables.image
for Docker/K8s pools → ensure both are present - Work pool doesn’t exist → create it once ahead of time (UI or CLI) - Not authenticated to an API → set 
PREFECT_API_URL
and 
PREFECT_API_KEY
(or use non-interactive login) If you still see prompts, paste the exact prompt text and I’ll point at the missing bit with theatrical precision.
m
@Marvin Another question, i am trying to run a flow using python:
Copy code
deployment_id = "f612f0a0-5847-451a-81a5-ead9a6612c71"
        
        print(az_ids)

    try:
        # Trigger the flow deployment
        flow_run = await client.create_flow_run_from_deployment(
            deployment_id=deployment_id,
            parameters={"az_ids": az_ids}
        )
However, i keep getting this:
Copy code
Traceback (most recent call last):

  File "/app/main.py", line 44, in auto_update

    flow_run = await client.create_flow_run_from_deployment(

               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

  File "/usr/local/lib/python3.11/site-packages/prefect/client/orchestration/_deployments/client.py", line 1328, in create_flow_run_from_deployment

    state=to_state_create(state),

          ^^^^^^^^^^^^^^^^^^^^^^

  File "/usr/local/lib/python3.11/site-packages/prefect/states.py", line 65, in to_state_create

    return StateCreate(

           ^^^^^^^^^^^^

  File "/usr/local/lib/python3.11/site-packages/pydantic/main.py", line 253, in __init__

    validated_self = self.__pydantic_validator__.validate_python(data, self_instance=self)

                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

  File "/usr/local/lib/python3.11/site-packages/pydantic/_internal/_mock_val_ser.py", line 100, in __getattr__

    raise PydanticUserError(self._error_message, code=self._code)

pydantic.errors.PydanticUserError: `StateCreate` is not fully defined; you should define all referenced types, then call `StateCreate.model_rebuild()`.


For further information visit <https://errors.pydantic.dev/2.11/u/class-not-fully-defined>⁠

ERROR:main:Error running flow: `StateCreate` is not fully defined; you should define all referenced types, then call `StateCreate.model_rebuild()`.


For further information visit <https://errors.pydantic.dev/2.11/u/class-not-fully-defined>⁠

['00000G', '000016', '000084', '0000DZ']
m
✅ thought for 71.5 seconds
That traceback isn’t about your parameters at all — it’s Pydantic complaining that Prefect’s 
StateCreate
model wasn’t “rebuilt.” In practice, that usually happens with a version mismatch: older Prefect + newer Pydantic (notably 2.11+) triggers this exact error. What to do - Upgrade Prefect to the latest 3.x in the image that’s calling the API (and in your worker image too, so they match). - If you can’t upgrade Prefect immediately, pin Pydantic to a 2.10.x release as a temporary workaround. - Also make sure you’re creating the client correctly and passing a UUID for the deployment id. Minimal working example
Copy code
from uuid import UUID
from prefect.client.orchestration import get_client

deployment_id = UUID("f612f0a0-5847-451a-81a5-ead9a6612c71")
az_ids = ["00000G", "000016", "000084", "0000DZ"]

async def trigger():
    async with get_client() as client:
        flow_run = await client.create_flow_run_from_deployment(
            deployment_id=deployment_id,
            parameters={"az_ids": az_ids},
        )
        print(flow_run.id)
Sanity checklist - Versions: - Prefect: upgrade to latest 3.x - Pydantic: if stuck, pin to <2.11 temporarily - Client usage: use 
async with get_client()
and pass 
UUID(...)
for 
deployment_id
- Auth: set 
PREFECT_API_URL
and 
PREFECT_API_KEY
in the environment where this runs If it still blows up after upgrading, share: - 
python -c "import prefect, pydantic; print(prefect.__version__, pydantic.__version__)"
- How you instantiate 
client
(full snippet) - Whether you’re running this inside a flow or from a separate service I’ll sigh dramatically and fix the next thing too.
m
@Marvin I am triggering fronm a fastapi app endpoint:
Copy code
@app.post("/jobs/auto-update")
async def auto_update(az_ids: list[str]):

    async with get_client(
        httpx_settings={
            "base_url": settings.PREFECT_API_URL,
        }
    ) as client:
        # Deployment ID of the flow is a fixed value
        # Create deployment via CLI and get ID on UI and set value here
        deployment_id = "f612f0a0-5847-451a-81a5-ead9a6612c71"

        try:
            # Trigger the flow deployment
            flow_run = await client.create_flow_run_from_deployment(
                deployment_id=UUID(deployment_id),
                parameters={"az_ids": az_ids}
            )
            return {"flow_run_id": str(flow_run.id), "status": "created"}
        except Exception as e:
            traceback.print_exc()
            logger.error(f"Error running flow: {e}")
            raise HTTPException(status_code=500, detail=str(e))
m
✅ thought for 43.8 seconds
Your FastAPI code looks fine. The error is coming from a Prefect/Pydantic version mismatch, not from your parameters. What’s happening - Pydantic 2.11 tightened validation and triggers “class-not-fully-defined” unless libraries call model_rebuild() at the right time. - An older Prefect 3.x + Pydantic 2.11.x combo will throw exactly the 
StateCreate is not fully defined
error you’re seeing. Fixes (pick one) - Preferred: upgrade Prefect in the FastAPI image (and your worker image) to the latest 3.x. - Temporary: pin Pydantic to <2.11 (e.g., 2.10.x) until you can upgrade Prefect. Also double-check auth and URL - Ensure your API URL includes the /api path: - Cloud: https://api.prefect.cloud/api/accounts/&lt;acct&gt;/workspaces/&lt;ws> - Server: http//&lt;host&gt;4200/api - Either set PREFECT_API_KEY in the environment or pass a header via httpx_settings. Example with FastAPI
Copy code
from uuid import UUID
from prefect.client.orchestration import get_client

@app.post("/jobs/auto-update")
async def auto_update(az_ids: list[str]):
    async with get_client(
        httpx_settings={
            "base_url": settings.PREFECT_API_URL,  # must include /api/...
            "headers": {"Authorization": f"Bearer {settings.PREFECT_API_KEY}"},
        }
    ) as client:
        deployment_id = UUID("f612f0a0-5847-451a-81a5-ead9a6612c71")

        # Optional sanity check: confirm the deployment exists/you’re authenticated
        # await client.read_deployment(deployment_id)

        flow_run = await client.create_flow_run_from_deployment(
            deployment_id=deployment_id,
            parameters={"az_ids": az_ids},
        )
        return {"flow_run_id": str(flow_run.id), "status": "created"}
Quick diagnostics - Print versions inside the same container that calls the API:
Copy code
python -c "import prefect, pydantic; print('prefect', prefect.__version__, 'pydantic', pydantic.__version__)"
- If Prefect is not current, upgrade it. If you can’t, pin Pydantic to 2.10.x. If you share those two version numbers, I’ll confirm the exact pin/upgrade you need with only a moderate amount of existential dread.
m
@Marvin python -c "import prefect, pydantic; print('prefect', prefect.version, 'pydantic', pydantic.version)"
m
✅ thought for 17.4 seconds
m
@Marvin prefect 3.4.11 pydantic 2.11.7
m
✅ thought for 31.2 seconds
Yep—that’s the right check. Please run it in the same container as your FastAPI app and paste the output. If you see “Prefect 3.x (older) + Pydantic 2.11.x”, that’s the culprit. Fix options: - Preferred: upgrade Prefect to latest 3.x
Copy code
pip install -U "prefect>=3,<4"
- Temporary workaround: pin Pydantic below 2.11
Copy code
pip install "pydantic<2.11"
If you’re using Docker, make it explicit so CI doesn’t “helpfully” upgrade behind your back:
Copy code
# requirements.txt
prefect>=3,<4
# If you can’t upgrade Prefect yet, uncomment this:
# pydantic<2.11
Then rebuild the FastAPI image (and your worker image too, so versions match). While you’re there, confirm your client config: - API URL includes /api (e.g., https://api.prefect.cloud/api/accounts/.../workspaces/... or http://host:4200/api) - Send Authorization header or set PREFECT_API_KEY env Run the version command and drop me the two numbers; I’ll point you to the exact next step with my usual boundless enthusiasm.
m
prefect 3.4.11 pydantic 2.11.7
@Marvin prefect 3.4.11 pydantic 2.11.7
m
✅ thought for 33.9 seconds
Thanks — that combo explains it. You’re on Prefect 3.4.11 with Pydantic 2.11.7. That pairing triggers the “StateCreate is not fully defined” error. Two ways out: Preferred: upgrade Prefect (both FastAPI image and your worker image) - Upgrade:
Copy code
pip install -U prefect
# or
uv pip install -U prefect
- Rebuild/redeploy your images so both sides run the same newer Prefect. - Re-check at runtime:
Copy code
python -c "import prefect, pydantic; print('prefect', prefect.__version__, 'pydantic', pydantic.__version__)"
Temporary workaround: pin Pydantic below 2.11
Copy code
pip install "pydantic<2.11"
# or in pyproject/requirements, pin to ~=2.10
Your FastAPI snippet is otherwise fine. Just ensure: - API URL includes /api (Cloud: https://api.prefect.cloud/api/accounts/&lt;acct&gt;/workspaces/&lt;ws>, Server: http://host:4200/api) - You pass auth (env PREFECT_API_KEY or header):
Copy code
httpx_settings={
  "base_url": settings.PREFECT_API_URL,
  "headers": {"Authorization": f"Bearer {settings.PREFECT_API_KEY}"},
}
Optional sanity check before triggering:
Copy code
await client.read_deployment(UUID("f612f0a0-5847-451a-81a5-ead9a6612c71"))
If you upgrade Prefect and still see that error, tell me the new versions from the quick check above and we’ll chase the next gremlin.
Thanks — that combo explains it. You’re on: - Prefect 3.4.11 - Pydantic 2.11.7 That pair is known to trigger the “StateCreate is not fully defined” error. Two ways out: Preferred - Upgrade Prefect to a newer 3.x that supports Pydantic 2.11:
Copy code
pip install -U "prefect>=3.4.13,<4"
Then rebuild/redeploy both: - the FastAPI image (client) - your worker image(s) Temporary workaround - Keep your current Prefect and pin Pydantic below 2.11:
Copy code
pip install "pydantic<2.11"
Extra sanity checks - Make sure your app and workers run the same Prefect minor version. - Ensure your client is actually authenticated; with FastAPI this often means adding the header if you aren’t relying on env vars:
Copy code
async with get_client(
    httpx_settings={
        "base_url": settings.PREFECT_API_URL,  # must include /api/...
        "headers": {"Authorization": f"Bearer {settings.PREFECT_API_KEY}"},
    }
) as client:
    ...
After upgrading/pinning, the same code should work:
Copy code
flow_run = await client.create_flow_run_from_deployment(
    deployment_id=UUID("f612f0a0-5847-451a-81a5-ead9a6612c71"),
    parameters={"az_ids": az_ids},
)
If you still see the Pydantic error after upgrading Prefect, tell me the new versions and we’ll keep spelunking.
m
@Marvin I added these two imports
Copy code
from prefect.client.schemas import StateDetails
from prefect.states import Pending
And error changeed to:
Copy code
`StateCreate` is not fully defined; you should define `ResultRecordMetadata`, then call `StateCreate.model_rebuild()`.\n\nFor further information visit <https://errors.pydantic.dev/2.11/u/class-not-fully-defined>"
m
✅ thought for 28.7 seconds
Those imports won’t bribe Pydantic into compliance—they just change which missing type it complains about next. Your versions: - Prefect 3.4.11 - Pydantic 2.11.7 This combo triggers the “StateCreate is not fully defined” error. Fix it one of two ways: Preferred: upgrade Prefect (in both the FastAPI image and your worker image)
Copy code
pip install -U "prefect>=3.4.13,<4"
Rebuild/redeploy both images, then verify:
Copy code
python -c "import prefect, pydantic; print('prefect', prefect.__version__, 'pydantic', pydantic.__version__)"
Temporary workaround: pin Pydantic below 2.11
Copy code
pip install "pydantic<2.11"
# e.g., pydantic==2.10.13
If you’re using requirements/pyproject, make it explicit so CI doesn’t “helpfully” upgrade:
Copy code
# requirements.txt
prefect>=3.4.13,<4
# If you can't upgrade Prefect yet, use this instead:
# pydantic<2.11
After that, your FastAPI code should work as-is. If it doesn’t, paste the new versions and the error and I’ll continue our thrilling saga. Links you already saw but for completeness: - Pydantic error explainer: class-not-fully-defined - Prefect latest notes: 3.4.13 changelog
m
@Marvin # python -c "import prefect, pydantic; print('prefect', prefect.version, 'pydantic', pydantic.version)" prefect 3.4.13 pydantic 2.11.7
m
✅ thought for 33.2 seconds
Thanks for the versions. Prefect 3.4.13 + Pydantic 2.11.7 can still trigger the “StateCreate is not fully defined” error in some environments. The imports you added won’t help; they just change which missing type Pydantic complains about next. Fastest path to green: - Pin Pydantic below 2.11 in BOTH your FastAPI image and your worker image(s), rebuild, redeploy.
Copy code
pip install "pydantic<2.11"
# e.g., pydantic==2.10.13
Or in requirements/pyproject:
Copy code
prefect>=3.4.13,<4
pydantic<2.11
Then verify at runtime and retry the endpoint:
Copy code
python -c "import prefect, pydantic; print('prefect', prefect.__version__, 'pydantic', pydantic.__version__)"
Your FastAPI call pattern is correct; don’t add extra imports or pass a 
state
. If the error persists after pinning, paste the new stack trace and I’ll continue our thrilling descent.
m
@Marvin # python -c "import prefect, pydantic; print('prefect', prefect.version, 'pydantic', pydantic.version)" prefect 3.4.13 pydantic 2.10.3
m
✅ thought for 22.4 seconds
Nice. Prefect 3.4.13 + Pydantic 2.10.3 is a safe combo. Next steps - Remove the extra imports you added (
StateDetails
, 
Pending
) — they aren’t needed. - Re-run your FastAPI endpoint. It should now create the flow run without the Pydantic “class-not-fully-defined” error. Optional sanity checks - Ensure you’re authenticated and pointing at the right API URL (must include /api). For example:
Copy code
async with get_client(httpx_settings={
  "base_url": settings.PREFECT_API_URL,
  "headers": {"Authorization": f"Bearer {settings.PREFECT_API_KEY}"},
}) as client:
    await client.read_deployment(UUID("f612f0a0-5847-451a-81a5-ead9a6612c71"))
- Confirm your flow actually expects a parameter named 
az_ids
. If you still see an error now, paste the new stack trace and we’ll chase the next suspect.
m
@Marvin issue is j9t solved: Traceback (most recent call last): File "/app/main.py", line 49, in auto_update flow_run = await client.create_flow_run_from_deployment( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/prefect/client/orchestration/_deployments/client.py", line 1328, in create_flow_run_from_deployment state=to_state_create(state), ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/prefect/states.py", line 65, in to_state_create return StateCreate( ^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/pydantic/main.py", line 214, in init validated_self = self.__pydantic_validator__.validate_python(data, self_instance=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/pydantic/_internal/_mock_val_ser.py", line 100, in getattr raise PydanticUserError(self._error_message, code=self._code) pydantic.errors.PydanticUserError: 
StateCreate
is not fully defined; you should define all referenced types, then call 
StateCreate.model_rebuild()
. For further information visit https://errors.pydantic.dev/2.10/u/class-not-fully-defined⁠ ERRORmainError running flow: 
StateCreate
is not fully defined; you should define all referenced types, then call 
StateCreate.model_rebuild()
. For further information visit https://errors.pydantic.dev/2.10/u/class-not-fully-defined⁠ DEBUGhttpcore.connectionclose.started DEBUGhttpcore.connectionclose.complete id=UUID('f612f0a0-5847-451a-81a5-ead9a6612c71') name='autoupdate-deployment' version='0.1' version_id=None version_info=None branch=None base=None root=None description='Autoupdate' flow_id=UUID('354de952-ad52-48b0-8a10-ea5896b5c2fd') concurrency_limit=None global_concurrency_limit=None concurrency_options=None paused=False schedules=[] job_variables={} parameters={} pull_steps=[{'prefect.deployments.steps.set_working_directory': {'directory': '/opt/prefect/autoupdate'}}] tags=['autoupdate'] labels={'prefect.flow.id': '354de952-ad52-48b0-8a10-ea5896b5c2fd'} work_queue_name='default' last_polled=DateTime(2025, 8, 18, 13, 49, 28, 306044, tzinfo=Timezone('UTC')) parameter_openapi_schema={'type': 'object', 'title': 'Parameters', 'required': ['az_ids'], 'properties': {'az_ids': {'type': 'array', 'items': {'type': 'string'}, 'title': 'az_ids', 'position': 0}}} path=None entrypoint='flows/auto_update.py:auto_update_flow' storage_document_id=None infrastructure_document_id=None created_by=None updated_by=None work_queue_id=None enforce_parameter_schema=True work_pool_name='auto-update-pool' status=DeploymentStatus.READY INFO: 172.20.0.1:39188 - "POST /jobs/auto-update HTTP/1.1" 500 Internal Server Error
m
✅ thought for 58.3 seconds
Alright, you’ve upgraded/pinned and it’s still whining about StateCreate. At this point you’ve hit a known import-order/circular-type quirk. Cause - Prefect’s Pydantic models reference types that aren’t imported yet in some import orders. The result: “StateCreate is not fully defined” even with compatible versions. Workarounds that actually work Pick one and put it somewhere that runs before your first client call (module import time or FastAPI startup): A) Minimal: import the top-level prefect package early
Copy code
import prefect  # do this at module import time, before using get_client
Yes, importing the top-level package is enough. Importing submodules like prefect.states or client.schemas isn’t. B) Explicit rebuild at startup
Copy code
from fastapi import FastAPI
app = FastAPI()

@app.on_event("startup")
async def prefect_model_rebuild():
    # Force-load the referenced type and rebuild the model
    from prefect.results import ResultRecordMetadata  # noqa: F401
    from prefect.client.schemas.actions import StateCreate
    StateCreate.model_rebuild()
Why this helps - It ensures the referenced types (e.g., ResultRecordMetadata) are imported before Pydantic constructs StateCreate. References - Import order issue: Dependency on import order in user-code - Background on ResultRecordMetadata and StateCreate: Incomplete Model for StateCreate Keep your versions as they are now (Prefect 3.4.13, Pydantic 2.10.3) — that combo is fine once the import order is fixed. After adding A or B, restart the app and try again. If it still errors, paste the new traceback and I’ll continue glaring at it until it behaves.
n
marvin seems right here, workaround is add 
import prefect.main
at the top of your file
Open in Slack
PreviousNext
Powered by