Hi there! I was struggling to deploy a simple flow from code using

KubernetesJob

. However, just before posting a question in this channel, I figured I should "give it one last try" and to my surprise it actually worked 🤡 As the post is already written, I figured I might as well share was I experienced and the solutions so that anyone else having simiular issues may have somewhere it start. I was trying to create a deployment using

KubernetesJob

as infrastructure. The deployment looked like this (see code further down in the thread) The container image is built from a dockerfile based on

prefecthq/prefect:2-latest

and copies the script containing the flow to

/opt/prefect/flows/flow.py

. This deploys just fine, and I can see that the kubernetes job is created and the Prefect engine starts, however it does not seem like the flow runst, and I instead get the following error (see traceback further down in the thread) It turns out that the base docker-image I had chosen was the issue, as the prefect version did not match the version the server was running (

2.4.5

vs

2.4.2

). From the release-notes for

2.4.3

-

2.4.5

i see that block protection is an issue, and I speculate that this might be the root cause. Hope it helps 🙂 Edit: Moved code and traceback to the thread

Hi! Thanks for the detailed walkthrough here 🙂 If your server is running on 2.4.5 you shouldn’t have encountered this issue even if the client was on an old version.

Thanks, Thomas. Glad to hear it’s working. 🙂 Could you please add the code and traceback to this thread to keep the main channel tidy?

Deployment code:

deployment = Deployment.build_from_flow(
    flow=flow,
    skip_upload=dry_run,
    entrypoint="/opt/prefect/flows/{}:{}".format(os.path.basename(context["flow_file"]), "main"),
    name=metadata["name"],
    work_queue_name=metadata.get("workQueue", "default"),
    infrastructure=KubernetesJob(
        image="<http://my-registry.com/myimage:latest|my-registry.com/myimage:latest>",
        env={
            "PREFECT_LOGGING_LEVEL": "DEBUG"
        },
        labels={
            "<http://prefect.io/flowName|prefect.io/flowName>": self._flow.name,
            "<http://prefect.io/flowVersion|prefect.io/flowVersion>": self._flow.version,
            "<http://prefect.io/taskRunner|prefect.io/taskRunner>": self._flow.task_runner.name,
        },
        name=f"{self._flow.name}-{self._flow.version}",
        namespace="prefect",
        finished_job_ttl=10,
        customizations=JsonPatch([
            {
                "op": "add",
                "path": "/spec/template/spec/containers/0/resources",
                "value": {
                    "requests": {
                        "cpu": tier.cpu,
                        "memory": tier.mem
                    },
                    "limits": {
                        "cpu": tier.cpu,
                        "memory": tier.mem
                    }
                },
            }
        ]),
        stream_output=True,
    ),
)

deployment.apply()

Traceback error:

12:35:54.284 | DEBUG   | prefect.client - Connecting to API at <http://orion:4200/api/>
12:35:54.311 | DEBUG   | Flow run 'jolly-crane' - Loading flow for deployment 'hello-cloud'...
12:35:54.328 | DEBUG   | Flow run 'jolly-crane' - Starting 'ConcurrentTaskRunner'; submitted tasks will be run concurrently...
12:35:54.329 | DEBUG   | prefect.task_runner.concurrent - Starting task runner...
12:35:54.338 | DEBUG   | prefect.client - Connecting to API at <http://orion:4200/api/>
12:35:54.429 | DEBUG   | prefect.task_runner.concurrent - Shutting down task runner...
12:35:54.429 | ERROR   | Flow run 'jolly-crane' - Crash detected! Execution was interrupted by an unexpected exception.
12:35:54.430 | DEBUG   | Flow run 'jolly-crane' - Crash details:
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/prefect/engine.py", line 1350, in report_flow_run_crashes
    yield
  File "/usr/local/lib/python3.10/site-packages/prefect/engine.py", line 365, in begin_flow_run
    await result_filesystem._save(is_anonymous=True)
  File "/usr/local/lib/python3.10/site-packages/prefect/client/orion.py", line 82, in with_injected_client
    return await fn(*args, **kwargs)
  File "/usr/local/lib/python3.10/site-packages/prefect/blocks/core.py", line 769, in _save
    block_document=self._to_block_document(name=name)
  File "/usr/local/lib/python3.10/site-packages/prefect/blocks/core.py", line 339, in _to_block_document
    raise ValueError(
ValueError: No block schema ID provided, either as an argument or on the block.
12:35:54.451 | DEBUG   | prefect.engine - Reported crashed flow run 'jolly-crane' successfully!
12:35:54.475 | ERROR   | prefect.engine - Engine execution of flow run '05dff8be-c6bb-4d3f-95f8-491074fdf99e' exited with unexpected exception
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/prefect/engine.py", line 1620, in <module>
    enter_flow_run_engine_from_subprocess(flow_run_id)
  File "/usr/local/lib/python3.10/site-packages/prefect/engine.py", line 183, in enter_flow_run_engine_from_subprocess
    return anyio.run(retrieve_flow_then_begin_flow_run, flow_run_id)
  File "/usr/local/lib/python3.10/site-packages/anyio/_core/_eventloop.py", line 70, in run
    return asynclib.run(func, *args, **backend_options)
  File "/usr/local/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 292, in run
    return native_run(wrapper(), debug=debug)
  File "/usr/local/lib/python3.10/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.10/asyncio/base_events.py", line 646, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 287, in wrapper
    return await func(*args)
  File "/usr/local/lib/python3.10/site-packages/prefect/client/orion.py", line 82, in with_injected_client
    return await fn(*args, **kwargs)
  File "/usr/local/lib/python3.10/site-packages/prefect/engine.py", line 306, in retrieve_flow_then_begin_flow_run
    return await begin_flow_run(
  File "/usr/local/lib/python3.10/site-packages/prefect/engine.py", line 365, in begin_flow_run
    await result_filesystem._save(is_anonymous=True)
  File "/usr/local/lib/python3.10/site-packages/prefect/client/orion.py", line 82, in with_injected_client
    return await fn(*args, **kwargs)
  File "/usr/local/lib/python3.10/site-packages/prefect/blocks/core.py", line 769, in _save
    block_document=self._to_block_document(name=name)
  File "/usr/local/lib/python3.10/site-packages/prefect/blocks/core.py", line 339, in _to_block_document
    raise ValueError(
ValueError: No block schema ID provided, either as an argument or on the block.
12:35:54.488 | DEBUG   | prefect.client - Connecting to API at <http://orion:4200/api/>