https://prefect.io
Join Slack
We get a pickle thread lock error in a task when using paramiko. This is probably a problem with pic...
r

Russell Brooks

over 3 years ago
We get a pickle thread lock error in a task when using paramiko. This is probably a problem with pickling ssh stuff. What is the alternative? How are people using paramiko with Prefect?
r
k
j
  • 3
  • 5
  • 75
Hi, I am trying to create a flow based on some dynamic values. These dynamic values tell the tasks w...
s

Shuchita Tripathi

over 3 years ago
Hi, I am trying to create a flow based on some dynamic values. These dynamic values tell the tasks which are required to create that flow. My tasks are saved in separate files. For eg: task1.py
def task1():
    @task
    def run_terraform_lt():
        tf = Terraform(working_dir="law_tf")
        tf.init()
		tf.apply()
task2.py
def task2():
    @task
    def run_terraform_rt():
        tf = Terraform(working_dir="rt_tf")
        tf.init()
		tf.apply()
The input is similar to this:
{
    "task1": {
      "id": "foo",
      "task_name": "task1"
    },
    "task2": {
      "id": "bar",
      "task_name": "task2",
    }
}
I am getting the task name from this dictionary. Based on the value of "task_name", I have to create a flow combining all tasks. I am creating a flow where I am trying to add them, but the tasks are not getting added in the flow. Anyone has any idea on how this can be achieved? Here is the snippet of my flow creation code. It is inside a for loop through which I am extracting the task name and other variables.
s
k
  • 2
  • 22
  • 75
I registered a Flow to the wrong Project. How do I delete it from that Project? The cli seems like...
d

Dan Stoner

about 4 years ago
I registered a Flow to the wrong Project. How do I delete it from that Project? The cli seems like it should but it does not seem to understand anything other than
delete project
.
$ prefect delete --help                                                                                                                                                     
Usage: prefect delete [OPTIONS] COMMAND [ARGS]...

  Delete commands that refer to mutations of Prefect API metadata.

  Usage:
      $ prefect delete [OBJECT]

  Arguments:
      project    Delete projects
...
d
k
  • 2
  • 3
  • 75
Is it appropriate to re-use a task across multiple different flows, similar to how you would import ...
h

Harry Baker

about 4 years ago
Is it appropriate to re-use a task across multiple different flows, similar to how you would import a python function? I am trying to organize my code base by DRYing it out, and I find myself reusing the same code defining tasks to instantiate and return a google sheets API client. Would I be able to define a task in a file, that is then imported in multiple other files to be used across multiple flows? Or does it not work like that
h
k
  • 2
  • 2
  • 75
<@ULVA73B9P> I would like to set CPU, Memory Limit to KubernetesJob block, can you give an example?
a

Avi

about 1 year ago
@Marvin I would like to set CPU, Memory Limit to KubernetesJob block, can you give an example?
a
m
  • 2
  • 7
  • 74
hey <@ULVA73B9P> why my flow don't fail if it have timeout of 30 sec here goes the code """ from pre...
p

Praveen Gupta

over 1 year ago
hey @Marvin why my flow don't fail if it have timeout of 30 sec here goes the code """ from prefect import flow, task import time @flow(name='my-flow-check', timeout_seconds=30, log_prints=True) def greetings(): print("comes into greetings") time.sleep(40) print("greeting from prefect") greetings() """
p
m
  • 2
  • 19
  • 74
is anyone else seeing the following mypy error?
b

Brendan Shanahan

over 2 years ago
is anyone else seeing the following mypy error?
b
t
+2
  • 4
  • 5
  • 74
Would anyone know why an azure function isn't able to connect to the prefect cloud? I set the correc...
g

Guy Altman

over 2 years ago
Would anyone know why an azure function isn't able to connect to the prefect cloud? I set the correct api_url and api_key in the az function config settings and it all worked well about a month ago but now I'm getting an error of what looks to be from prefect that the cloud request isn't returning anything.
g
c
  • 2
  • 38
  • 74
How can we read log of a task or flow/subflow using Orion REST APIs? Also, Do we need to do any spec...
x

Xavier Babu

over 3 years ago
How can we read log of a task or flow/subflow using Orion REST APIs? Also, Do we need to do any specific configuration to achieve it via REST APIs?
x
a
  • 2
  • 12
  • 74
When running child flows from a parent flow, how to pass parent flow run name to child flow? below...
p

Patrick Tan

over 3 years ago
When running child flows from a parent flow, how to pass parent flow run name to child flow? below does not work.
p
a
  • 2
  • 5
  • 74
Previous868788Next

Prefect Community

Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.

Powered by