Hi, I am honestly not really happy with the direction Prefect 2.0 is going... Prefect 1.0 was decent, but the current setup really confuses me... I still did not figure out what the best way would be to schedule existing Kubernetes Cronjobs? I know there is prefect-kubernetes, but according to the Medium post you need an external storage? Why not just the Docker container? I mean all files are baked in there, including persistent volumes and everything... Hence my question: What is the best way to schedule existing Kubernetes Jobs/Cronjobs using Prefect, with the background info that the containers use persistent volumes and env vars? example:

# apiVersion: v1
# kind: PersistentVolumeClaim
# metadata:
#   name: composer-bigtechmomentum-v1-pvc
# spec:
#   accessModes:
#     - ReadWriteOnce
#   resources:
#     requests:
#       storage: 1Mi
# ---
apiVersion: batch/v1
kind: CronJob
metadata:
  name: composer-bigtechmomentum-v1
spec:
  schedule: "42 3 * * *"
  suspend: false
  concurrencyPolicy: Forbid
  jobTemplate:
    spec:
      template:
        spec:
          containers:
          - name: composer-bigtechmomentum-v1
            image: guestros/xxxxxxxxxx:latest
            imagePullPolicy: Always
          #   volumeMounts:
          #     - mountPath: /app/persistent/
          #       name: composer-bigtechmomentum-v1-pv

hey @Justin, you can tweak the manifest in anyway you wish using infra_override patches

no no, i would be fine with prefect, i just didn't see a straightforward way to convert my "old" cronjobs to kubernetes... some example would be nice, because as I saw it the medium post does not apply to this case, because it uses an external s3 storage

I'm sure they will get to a few of those things as 2.0 matures

hm, thanks already for your help, but I still don't see how it can be applied to my use case... Because as I said, I do not want to store the flow.py seperately, I just want to run the docker image like a k8s cronjob would do, or even as simple as "docker run guestros/xxxx:latest" As far as I can see, you always need to provide a seperate flow.py, which would require for me to rewrite all docker images, store the flow.py's seperately (in s3), instead of just running the docker image like k8s and docker would do...

I think you can specify storage as none then provide your own docker image - as long as the path is set correctly it should pick it up

alright nice? but i would still need to write a "flow.py" to trigger the start of the image? that sounds like an unnecessary step? wouldn't it be more logical to create a deployment from this k8s-job block?

sorry not sure - I use s3 as storage and keep the flows separate to prevent mass container rebuilds

alright, thx anyways! helped a lot 🙂

and much cheaper compared to say aws airflow

how do you handle imports with the s3 setup? like if i have a "helper.py" which contains a "primenumbers" function? just drop it into the s3 as well?

During deploy I copy a shared function folder below my flow so it gets uploaded