• n

    Nakul Gowdra

    1 year ago
    Hi there, is there a TTL for inactive agents, or is there any API documentation available to clean up dead agents. Thanks in advance
    n
    1 replies
    Copy to Clipboard
  • Jacob Blanco

    Jacob Blanco

    1 year ago
    Can anyone share their strategy for setting up a Staging-like environment with Prefect Cloud? Given that we only have a single tenant whatever solution we come up with will never truly be totally separate from Production flows, so I was curious what people’s approach is here? I realize that you can do local testing but of course that doesn’t mirror every aspect of the environment when deploying stuff.
    Jacob Blanco
    1 replies
    Copy to Clipboard
  • Simone Cittadini

    Simone Cittadini

    1 year ago
    Hi, probably a dumb question, but just to be sure, tasks returning generators as part of their output, are they always guaranteed to work ? Something like this : ( note, I will never use a Dask runner, this is a special case working on huge files )
    class Process(Task):
    def fu(self, rows):
    for row in rows:
    yield row + 100
    def run(self, data):
    output = {}
    output["meta"] = data["meta"]
    output["data"] = self.fu(data["rows"])
    return output
    @task
    def pull(data):
    for row in data["data"]:
    print(row)
    with Flow("test") as flow:
    input_data = {"rows": [1,2,3,4], "meta": "fixedvalue"}
    process = Process()
    data = process(input_data)
    drain = pull(data)
    Simone Cittadini
    emre
    3 replies
    Copy to Clipboard
  • psimakis

    psimakis

    1 year ago
    Hello everyone! I'm trying to find the right way to send a single slack notification for a group of mapped shell tasks. In my case, the list of mapped task is quite big, therefore, the slack messages are numerous and the slack channel has become really spammy. To be more specific, let's suppose that a group of similar shell commands should be mapped:
    from prefect.tasks.shell import ShellTask
    from prefect import Flow
    
    a = ShellTask()
    commands = [
    	'ls',
    	'ls -l',
    	'invalidcommand'
    ]
    
    with Flow('test') as flow:
    	b = a.map(commands)
    	# send a single slack nofiication
    	# that summarize the states of mapped tasks
    	send_slack_notification()
    The purpose of this slack notification is to summarize the states of mapped tasks by displaying the percentage of successfully mapped tasks. In the case above, this percentage will be 66.6% (the last command will fail). I tried to approach the problem using triggers and state handlers but I couldn't find a clean way to achieve the goal. Have you been in this situation before? Any hint? Thanks in advance!
    psimakis
    1 replies
    Copy to Clipboard
  • Newskooler

    Newskooler

    1 year ago
    Hi 👋 Does anyone know how (if at all possible) to iterate over a
    prefect.Paramter
    ? My question in more detail: https://stackoverflow.com/questions/64155793/is-it-possible-to-loop-over-a-prefect-parameter
    Newskooler
    i
    +1
    10 replies
    Copy to Clipboard
  • k

    Kevin Weiler

    1 year ago
    hey all! Does anyone know how to specify these two configuration parameters as environment variables?https://github.com/PrefectHQ/prefect/blob/master/src/prefect/config.toml#L44-L45
    k
    l
    +3
    47 replies
    Copy to Clipboard
  • Robin

    Robin

    1 year ago
    Dear all, I just now got the following error:
    Unexpected error: OSError("Timed out trying to connect to '<tcp://10.100.0.100:44991>' after 10 s: Timed out trying to connect to '<tcp://10.100.0.100:44991>' after 10 s: connect() didn't finish in time")
    Is this a known issue? The environment is a daskkubernetes cluster running on AWS EKS...
    Robin
    nicholas
    9 replies
    Copy to Clipboard
  • Adam

    Adam

    1 year ago
    Does anyone have a working example of this sentence from the docs: “Package your scripts up into a true Python package. You will most likely need to use the 
    COPY
     instruction to put your package into the image, and then the 
    RUN
     instruction to install it.”
    Adam
    m
    3 replies
    Copy to Clipboard
  • Newskooler

    Newskooler

    1 year ago
    Hi 👋, If I do a
    my_function.map(a, b)
    where
    len(a)
    is 2 and
    len(b)
    is 5, will this result in 10 runs? Currently I think it only runs twice : / 🤔
    Newskooler
    nicholas
    6 replies
    Copy to Clipboard
  • k

    Kevin Weiler

    1 year ago
    hi there - I’m getting an error when trying to run one of my flows:
    '413 Client Error: Request Entity Too Large for url: <http://prefect-api.aq.tc/graphql>'
    k
    nicholas
    17 replies
    Copy to Clipboard