Hi all, I'm running into an issue when trying to run Prefect via Kubernetes. I have my job template setup so that it mounts a NFS drive (this works correctly). But when I change the securityContext to the UID of the user that will access the NFS mount, then I get this error with the job:

UserWarning: Failed to create the Prefect home directory at /.prefect

I've tried changing the environment variable to

{"PREFECT_HOME": "/tmp"}

, which does stop this issue, but then my job has issues with cloning the repo. Does anyone know how I can correctly setup a job so that it has a NFS mount and the correct userID working?

@Marvin Can I download the logs for a task run?

@Marvin are variable going to be deprecated?

How can I cancel, kill, remove, (whatever you want to call it) all flow runs in Prefect Cloud? Do I need to use some CLI options or run a script? I have an issue where I’m hitting API limits due to the system having gotten bogged down when there was an error present in a flow. I have killed the agent and restarted it, but all the flow runs are still trying to execute, and there’s no easy option in Prefect Cloud to clear them.

is it possible to create a helper function/task that implements both create_flow_run and wait_for_flow_run? I tried:

@task()
def flow_run_helper(flow_name, project_name):
    cfr = create_flow_run(flow_name=flow_name, project_name=project_name)
    wfr = wait_for_flow_run(cfr, stream_logs=True, raise_final_state=True)
    return wfr

but its yelling at me about "ValueError: Could not infer an active Flow context while creating edge". my app does a lot of chaining of flows, so i wanted to streamline this

Is it expected that running a local flow with

PREFECT__FLOWS__CHECKPOINTING=false

but with checkpoint data present in the prefect result directory would read from those results? - I would expect this, but this is the current behaviour (on my machine ...)

Hi Guys, wondering if anyone did work with Spark on Prefect. Is there a way to pass spark dataframes between two tasks? I am trying to run Spark in client mode and pass the dataframes between tasks.

Hi guys Did anyone experience issues when running tests that use

prefect_test_harness

in Circle CI? Locally I’m able to run all of the tests, but in Circle CI I’m getting the following error:

RuntimeError: Timed out while attempting to connect to ephemeral Prefect API server.

Hey @Marvin. Prefect 3 caches the task input parameters. I have a huge class I’m passing to every task, is there a way to avoid that class to be cached, globally ? by adding something in the class itself ?

Hello everyone, I'm trying to get a deployment to a docker worker pool to work, but I'm getting the following error:

prefect.utilities.dockerutils.BuildError: manifest for prefecthq/prefect:3.0.11-python3.11 not found: manifest unknown: manifest unknown

The deploy call looks like this:

my_function.deploy(
        name=deployment_name,
        work_pool_name="my-docker-pool",
        image="my-first-deployment-image:tutorial",
        push=False,
        job_variables={
            "image_pull_policy": "Never",
            "auto_remove": True,
            "env": {
                "PREFECT_API_URL": "<http://host.docker.internal:4200/api>",
                "PREFECT_SERVER_API_HOST": "0.0.0.0",
            },
        },
    )

I can provide further information as required, but I am unsure where to start looking. This was working well with prefect 2. Notice that it is not my custom image that it does not found, but an underlying prefect one