https://prefect.io
Join Slack
<@ULVA73B9P> How can I schedule a deployment using multiple parameter settings on a weekly basis?
m

Mark

12 months ago
@Marvin How can I schedule a deployment using multiple parameter settings on a weekly basis?
m
m
  • 2
  • 6
  • 91
Does anybody have flows that act as infinite loops running the same tasks again and again instead of...
c

Carlos Cueto

about 1 year ago
Does anybody have flows that act as infinite loops running the same tasks again and again instead of scheduling a flow to run every few seconds? Just wondering if there’s any downside to doing that vs. scheduling it.
c
a
c
  • 3
  • 5
  • 91
Is there an easy way to manage work pools as Python code for our CI/CD? I initially created my work ...
n

Niall O'Riordan

about 1 year ago
Is there an easy way to manage work pools as Python code for our CI/CD? I initially created my work via the
publish_as_work_pool
method. I now want to manage this work pool as code so any changes can be easily applied such as the image version etc. The docs specify work pools can be configured via the UI, CLI, REST API or Terraform. Other than using Terraform is there an easy way to have work pools defined as code so any updates will be applied?
n
  • 1
  • 3
  • 91
<@ULVA73B9P> how can make backfill like Airflow
r

Rustam Salavatov

over 1 year ago
@Marvin how can make backfill like Airflow
r
m
  • 2
  • 1
  • 91
Hi team, I've got some questions about Kubernetes Workers, specifically their use of base job templa...
b

Billy McMonagle

almost 2 years ago
Hi team, I've got some questions about Kubernetes Workers, specifically their use of base job templates. I'm adapting the prefect-worker helm chart, which suggests configuring the worker with a base-job-template.json file. However, I understand that work pools have their own job templates, and that these must be actively updated using the
prefect work-pool update
command (it is not updated automatically, say the next time the worker restarts). I intend to implement custom, dynamic job templates, so I want to make sure that my job template is always up-to-date for a given helm release. Is there a recommended way to do this?
b
a
  • 2
  • 10
  • 91
Hi, I’m looking for instructions on installing requirements on deployments, but this guide seems to ...
j

Jonah

about 2 years ago
Hi, I’m looking for instructions on installing requirements on deployments, but this guide seems to be missing: https://docs.prefect.io/latest/guides/deployment/kubernetes
✅ 1
j
c
j
  • 3
  • 10
  • 91
<@ULVA73B9P> I have python script. how do i deploy that script?
s

Sung Kim

about 2 years ago
@Marvin I have python script. how do i deploy that script?
s
n
m
  • 3
  • 6
  • 91
Hello. If I want to run flows in a local subprocess. Do I have to specify the environment in which p...
e

emo loic

about 3 years ago
Hello. If I want to run flows in a local subprocess. Do I have to specify the environment in which prefect agents will execute the tasks? Let's supposed my flow requires some dependencies that I have already installed in a python environment. When the agents will pick up the flow, where do I specify the environment they need to in other to run the flow?
e
o
a
  • 3
  • 10
  • 91
having trouble running my gcs-stored flows on kubernetes. I feel like it probably has something to d...
k

Kevin Grismore

about 3 years ago
having trouble running my gcs-stored flows on kubernetes. I feel like it probably has something to do with how my project is structured:
- project
        └── flows
            └── flow1.py
            └── flow2.py
        └── util
            └── util.py
if I do
some/dir/project> prefect deployment build flows/flow1.py:flow_func -n my-flow -ib kubernetes-job/my-job -sb gcs/my-bucket -t k8s
everything in src ends up in my bucket as expected, but when I run the flow I get:
FileNotFoundError: [Errno 2] No such file or directory: '/opt/prefect/flows/flow1.py'
k
a
+2
  • 4
  • 9
  • 91
I am testing Prefect deployed into a kind kubernetes cluster using the helm chart. I'd like to mount...
j

Jacqueline Riley Garrahan

over 3 years ago
I am testing Prefect deployed into a kind kubernetes cluster using the helm chart. I'd like to mount a folder on my local filesystem to a running job. I've configured this in the job template using:
apiVersion: batch/v1
kind: Job
metadata:
  name: prefect-job
  labels: {}
spec:
  template:
    metadata:
      labels: {}
    spec:
      containers:
        - name: flow
          image: prefecthq/prefect:latest
          imagePullPolicy: IfNotPresent
          ...
          volumeMounts:
            - name: filesystem-dir
              mountPath: /job-files
      volumes:
        - name: filesystem-dir
          hostPath:
            # directory location on host
            path: /path/to/my/dir
            # this field is optional
            type: Directory
I'm unable to mount the path which I believe is a function of the volume not being available on the agent. Is there a straightforward way for me to mount a volume to the agent in the helm chart deployment?
j
a
  • 2
  • 6
  • 91
Previous686970Next

Prefect Community

Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.

Powered by