https://prefect.io logo
s

Steve Smith

09/22/2022, 1:34 PM
I'm having trouble running a KubernetesJob deployment with Docker storage when I apply the deployment on Windows. The deployment's entrypoint is getting set with a Windows-style path (e.g.
entrypoint="app\\flows\\my_flow.py:my_flow"
), so when I try to run it the job cannot find the flow source code. My fix is to manually replace "\\" with "/" in the entrypoint to make it a Linux-style path before applying the deployment. Is this a known issue?
r

Rob Freedy

09/22/2022, 2:30 PM
Hey Steve!! What prefect version are you running?
s

Steve Smith

09/22/2022, 2:36 PM
Version 2.4.0
r

Rob Freedy

09/22/2022, 2:40 PM
Can you share your deployment build command (or a simplified version of it)? This may be a related bug to the bug raised here: https://github.com/PrefectHQ/prefect/issues/6276
s

Steve Smith

09/22/2022, 2:46 PM
Copy code
k8s_job_prod = KubernetesJob(
        namespace="default",
        image=stack['ecr_image_uri'],
        image_pull_policy=KubernetesImagePullPolicy.IF_NOT_PRESENT,
        labels=dict(group="group", app="app", env=env),
        service_account_name='service_name',
    )


infra_overrides['customizations'] = [
	{
		"op": "add",
		"path": "/spec/template/spec/containers/0/resources",
		"value": {
			"requests": {
				"cpu": resources.cpu,
				"memory": resources.memory
			}
		},
	}
]

deployment_build = Deployment.build_from_flow(
	flow=my_flow,
	name=full_deployment_name,
	description=description,
	version=version,
	work_queue_name="my-q",
	tags=["group", "app", "env"],
	infrastructure=k8s_job_prod,
	infra_overrides=infra_overrides,
)
# Fix for deployments from Windows, entrypoint needs linux-style path
deployment_build.entrypoint = deployment_build.entrypoint.replace("\\", "/")
deployment_build.apply()
This is an anonymised version of my deployment code. It will run fine when deployed from linux-based systems without the entrypoint fix, but when deployed from Windows, it will not run without the fix (but does run with it)
b

Bianca Hoch

09/22/2022, 3:21 PM
@Marvin open "Windows path not being recognized in docker storage entrypoint"
m

Marvin

09/22/2022, 3:22 PM
r

Rob Freedy

09/22/2022, 3:23 PM
We have opened a github issue for your issue related to the windows path in the entry point for the DockerStorage. Any additional context in the Github issue would be greatly appreciated!
👍 1
a

Andreas Nord

09/22/2022, 4:03 PM
I have the same problem (and did a similar fix)
3 Views