https://prefect.io logo
Docs
Join the conversationJoin Slack
Channels
announcements
ask-marvin
best-practices-coordination-plane
data-ecosystem
data-tricks-and-tips
events
find-a-prefect-job
geo-australia
geo-bay-area
geo-berlin
geo-boston
geo-chicago
geo-colorado
geo-dc
geo-israel
geo-japan
geo-london
geo-nyc
geo-seattle
geo-texas
gratitude
introductions
marvin-in-the-wild
prefect-ai
prefect-aws
prefect-azure
prefect-cloud
prefect-community
prefect-contributors
prefect-dbt
prefect-docker
prefect-gcp
prefect-getting-started
prefect-integrations
prefect-kubernetes
prefect-recipes
prefect-server
prefect-ui
random
show-us-what-you-got
Powered by Linen
prefect-server
  • k

    Krapi Shah

    10/06/2021, 6:45 PM
    Hi We are using prefect Cloud to run various flows. I remember we used to see flow storage, label and schedule info on a box to the left. We do not see that box anymore. Not sure since when we are not seeing it but noticed it today. Any help? Thanks
    n
    6 replies · 2 participants
  • e

    Evan Curtin

    10/06/2021, 7:39 PM
    can prefects UI server be authenticated with okta?
    k
    2 replies · 2 participants
  • e

    Eddie

    10/07/2021, 6:17 PM
    I am having trouble locally testing the parent-child flow mentioned in the "Sub-flow result passing" section of this page. I tried to run it with
    prefect run -m flows.dev.parent_child -n parent
    and got the following exception:
    └── 14:10:26 | ERROR   | Task 'create_flow_run': Exception encountered during task execution!
    Traceback (most recent call last):
      File "/Users/epantridge/Library/Caches/pypoetry/virtualenvs/flows-Lt6Mru_j-py3.9/lib/python3.9/site-packages/prefect/engine/task_runner.py", line 859, in get_task_run_state
        value = prefect.utilities.executors.run_task_with_timeout(
      File "/Users/epantridge/Library/Caches/pypoetry/virtualenvs/flows-Lt6Mru_j-py3.9/lib/python3.9/site-packages/prefect/utilities/executors.py", line 454, in run_task_with_timeout
        return task.run(*args, **kwargs)  # type: ignore
      File "/Users/epantridge/Library/Caches/pypoetry/virtualenvs/flows-Lt6Mru_j-py3.9/lib/python3.9/site-packages/prefect/tasks/prefect/flow_run.py", line 118, in create_flow_run
        flow = FlowView.from_flow_name(flow_name, project_name=project_name)
      File "/Users/epantridge/Library/Caches/pypoetry/virtualenvs/flows-Lt6Mru_j-py3.9/lib/python3.9/site-packages/prefect/backend/flow.py", line 192, in from_flow_name
        flows = cls._query_for_flows(
      File "/Users/epantridge/Library/Caches/pypoetry/virtualenvs/flows-Lt6Mru_j-py3.9/lib/python3.9/site-packages/prefect/backend/flow.py", line 287, in _query_for_flows
        raise ValueError(
    ValueError: No results found while querying for flows where {'name': {'_eq': 'child'}, 'archived': {'_eq': False}}
    Does this imply that the child flow has to be registered before the parent flow can be run? If so, is there a way to register a flow in-process when testing locally?
    k
    5 replies · 2 participants
  • r

    Ramzi A

    10/07/2021, 8:15 PM
    Not sure if this the right place to ask, what is the ideal way to setup multiple flows on a VM for example? right now I am setting up each different flow as its own folder and virutal env.
    k
    2 replies · 2 participants
  • s

    Sylvain Hazard

    10/08/2021, 8:54 AM
    I'm having issues with mapped tasks that run when they should not with the wrong input. I'll detail in the thread so as to not post a huge wall of text.
    k
    23 replies · 2 participants
  • f

    Filip Lindvall

    10/08/2021, 12:36 PM
    We have flows in the prefect cloud that should be running but are just empty, no stat us no nothing. Just shows up as past runns without anything in them.
    k
    z
    +1
    12 replies · 4 participants
  • s

    Sylvain Hazard

    10/08/2021, 2:49 PM
    Got another quick question : It seems that running a method in a ThreadPool makes it loose access to the current context and specifically the Prefect logger. Is there any easy workaround this ?
    🎉 1
    k
    z
    4 replies · 3 participants
  • c

    Cooper Marcus

    10/09/2021, 3:20 PM
    I have a flow run that I think has failed-ish - hop in the thread to check out the Prefect Cloud log message - can you help me understand what the message it telling me?
    k
    m
    19 replies · 3 participants
  • s

    Scarlett King

    10/12/2021, 10:02 AM
    I’m running a local prefect server with Kubernetes agent. I’m using a customised image. When running the flow, I’m getting this error: Pod prefect-job-35418434-nr4n5 failed. Container ‘flow’ state: terminated Exit Code:: 2 Reason: Error. What does this error mean? Is there anyway I could look into what’s happening inside the pod to check for this error?
    a
    4 replies · 2 participants
  • j

    Johnny

    10/12/2021, 4:20 PM
    Hey all, I'm looking at running prefect cloud (managed) and realize in order for my task to access my gcp cloud sql db I would need to whitelist IPs. Does anyone know what are the IPs for prefect cloud?
    k
    6 replies · 2 participants
  • c

    Casey Green

    10/12/2021, 9:55 PM
    Is it possible to query for a flow's idempotency key via graphql api? Pretty sure the answer is "no" bc I don't see it in the flow schema, but just double-checking that there isn't some secret way.
    z
    2 replies · 2 participants
  • m

    Martin Felder

    10/13/2021, 4:31 PM
    Hi, we recently upgraded to Prefect 0.15.6, and now I'm getting these weird errors
    apollo_1   | PayloadTooLargeError: request entity too large
    apollo_1   |     at readStream (/apollo/node_modules/raw-body/index.js:155:17)
    apollo_1   |     at getRawBody (/apollo/node_modules/raw-body/index.js:108:12)
    apollo_1   |     at read (/apollo/node_modules/body-parser/lib/read.js:77:3)
    apollo_1   |     at jsonParser (/apollo/node_modules/body-parser/lib/types/json.js:135:5)
    apollo_1   |     at Layer.handle [as handle_request] (/apollo/node_modules/express/lib/router/layer.js:95:5)
    apollo_1   |     at trim_prefix (/apollo/node_modules/express/lib/router/index.js:317:13)
    apollo_1   |     at /apollo/node_modules/express/lib/router/index.js:284:7
    apollo_1   |     at Function.process_params (/apollo/node_modules/express/lib/router/index.js:335:12)
    apollo_1   |     at next (/apollo/node_modules/express/lib/router/index.js:275:10)
    apollo_1   |     at cors (/apollo/node_modules/cors/lib/index.js:188:7)
    every 2 seconds or so. The server UI also does not seem to be running. Does anyone have an idea what this means?
    m
    z
    14 replies · 3 participants
  • l

    Lon Nix

    10/13/2021, 5:02 PM
    noob question: how do I use the prefect cli with a remote server? I've got the server running in k8s but need to use the CLI to create a tenant
    m
    x
    29 replies · 3 participants
  • j

    Jimmy Borzillieri

    10/14/2021, 8:11 PM
    Hi, I’m looking for a way to read/expose the ip address of a running flow. The objective is to whitelist those flows for an external API. I cannot find any docs on this. Can someone please assist?
    m
    5 replies · 2 participants
  • l

    Lon Nix

    10/14/2021, 8:55 PM
    Is there a command to set config values for the CLI? Something like
    prefect config server.host="<http://blah>"
    prefect config server.port="4200"
    m
    m
    5 replies · 3 participants
  • a

    Alexander van Eck

    10/15/2021, 12:12 PM
    Question about environment variables and context. When I want to trigger a flow from the UI I can set environment variables for the run, when I want to trigger the same flow from the CLI can I also pass those environment variables?
    a
    k
    10 replies · 3 participants
  • l

    Lon Nix

    10/15/2021, 3:11 PM
    I'm running prefect server in k8s so I have pods for all prefects pieces, pods for the k8s agents and then those pods are making pods for the flows. How do I control configuration of the flow pods? I would like to annotate them to grant IAM permissions. Do they get whatever the k8s agent has? Can I pass something from the agent to the flow pods?
    a
    5 replies · 2 participants
  • c

    Casey Green

    10/16/2021, 12:54 PM
    Is it possible to limit concurrency at the flow level without using labels?
    a
    m
    +1
    5 replies · 4 participants
  • s

    Sylvain Hazard

    10/18/2021, 9:20 AM
    Hello there ! I'd like to know about any limitation as to how large a mapped task can be. This does not have to be a fixed limit, an order of magnitude would do. Given an input list of size, let's say 1 million, would Prefect Server be able to map a task correctly ? I am running said flow on a LocalDaskExecutor. I believe it is relevant due to it going for a depth first execution. Thanks a bunch !
    a
    k
    9 replies · 3 participants
  • s

    Sylvain Hazard

    10/18/2021, 11:56 AM
    Got another quick question : what would be the best "prefectonic" way to access every other task from a given task ? For context, I'd like to make a task that recaps everything that happened in the others tasks and sends a Slack message with that content e.g. "Flow ran in X minutes, Y tasks failed, etc."
    a
    m
    8 replies · 3 participants
  • a

    Aleksandr Glushko

    10/19/2021, 8:38 AM
    Hi there! Did anyone try to initialise Ray service from Prefect Service on k8s? Im currently getting an error of missmatched keys:
    Task 'init_ray': Exception encountered during task execution!
    Traceback (most recent call last):
      File "/usr/local/lib/python3.7/site-packages/prefect/engine/task_runner.py", line 865, in get_task_run_state
        logger=self.logger,
      File "/usr/local/lib/python3.7/site-packages/prefect/utilities/executors.py", line 328, in run_task_with_timeout
        return task.run(*args, **kwargs)  # type: ignore
      File "<string>", line 57, in init_ray
      File "/usr/local/lib/python3.7/site-packages/ray/_private/client_mode_hook.py", line 62, in wrapper
        return func(*args, **kwargs)
      File "/usr/local/lib/python3.7/site-packages/ray/worker.py", line 844, in init
        connect_only=True)
      File "/usr/local/lib/python3.7/site-packages/ray/node.py", line 166, in __init__
        self._init_temp(redis_client)
      File "/usr/local/lib/python3.7/site-packages/ray/node.py", line 283, in _init_temp
        session_dir = _get_with_retry(redis_client, "session_dir")
      File "/usr/local/lib/python3.7/site-packages/ray/node.py", line 41, in _get_with_retry
        result = redis_client.get(key)
      File "/usr/local/lib/python3.7/site-packages/redis/client.py", line 1606, in get
        return self.execute_command('GET', name)
      File "/usr/local/lib/python3.7/site-packages/redis/client.py", line 901, in execute_command
        return self.parse_response(conn, command_name, **options)
      File "/usr/local/lib/python3.7/site-packages/redis/client.py", line 915, in parse_response
        response = connection.read_response()
      File "/usr/local/lib/python3.7/site-packages/redis/connection.py", line 756, in read_response
        raise response
    redis.exceptions.ResponseError: WRONGTYPE Operation against a key holding the wrong kind of value
    I can initialize it internally in the Ray service, but there is a problem in connection between the srvices
    k
    4 replies · 2 participants
  • s

    Sylvain Hazard

    10/19/2021, 8:58 AM
    Hello ! Is there a way to filter Failed/Skipped/None mapped tasks in order to limit the number of useless downstream mapped tasks created. Say I have a flow that looks like this :
    a = A()
    b = B.map(a)
    c = C.map(b)
    and I would like to prevent C mapped tasks from being created if the upstream B mapped task is Failed. I have tried using a FilterTask which works great but serves as a kind of reducing task which stops depth first execution from working as I would like it to. Is there another way to do this while still making use of DFE ?
    a
    k
    6 replies · 3 participants
  • r

    Raúl Mansilla

    10/19/2021, 9:58 AM
    Hello team!! I´m trying to write a flow monitor in order to shutdown agents and stop scheduled flows if I get a lot of late flows…how can I query for late flow runs state using graphql api? Thanks!
    a
    k
    51 replies · 3 participants
  • a

    AJ

    10/19/2021, 10:37 AM
    hi all, i have some doubts in local agents. 1.) Does the local agents stop running automatically after any certain period of time without job or it will run for 24*7 in the instance once started ? 2.) When will the local agent become stale? 3.) how to run the agent 24/7 ? Thanks.
    a
    1 reply · 2 participants
  • s

    Sylvain Hazard

    10/19/2021, 2:44 PM
    Fell like I'm being annoying asking multiple question everyday but I can't wrap my head around this one. I've been trying to play around with the FlowRunView API and I'm running into issues with TaskRuns locations. Code and explanation below ⬇️
    k
    5 replies · 2 participants
  • p

    Payam Vaezi

    10/20/2021, 6:02 PM
    Hello team! Trying to run a workflow with local dask executor, it works fine locally with prefect core and distributing over multiple threads or processes in my local machine over mapped tasks, but when I want to run the same workflow over prefect server, the prefect server runs those mapped tasks in sequential manner. Is this is an expected behavior? Or am I doing something wrong?
    k
    6 replies · 2 participants
  • r

    Ritvik Mishra

    10/20/2021, 10:00 PM
    Hi! We use prefect to run databricks tasks, and we've been noticing lately that the status prefect shows for a task doesn't always update in time - prefect frequently thinks something is still running, or hasn't even started, when it already started and finished. So the flow doesn't continue on to the next tasks in the dag and gets stuck. Eventually gets the updated status of the databricks tasks and continues. This seems like some sort of communication issue between prefect and databricks, but I can't figure out how to debug. Would appreciate some help here! Thanks!
    k
    a
    65 replies · 3 participants
  • t

    Tara

    10/21/2021, 11:51 PM
    Hi, Im running a prefect server on a local server and my flow is using Local dask executor (threads). I occasionally ran into this error where the task runner failed to set state:
    Failed to set task state with error: ReadTimeout(ReadTimeoutError("HTTPConnectionPool(host='localhost', port=4200): Read timed out. (read timeout=15)"))
    This has been an ongoing issue, anyone has any idea on how to fix this?
    k
    a
    21 replies · 3 participants
  • a

    Arun Dass

    10/22/2021, 5:20 PM
    is there a way to delete historical flow data from the DB. For example only save flow info for the last week? This is mainly due to large amounts of flow data bogging down the postgres DB
    k
    1 reply · 2 participants
  • g

    Gabriel Milan

    10/22/2021, 6:26 PM
    Hi there! Has anyone already set up prefect server on k8s?
    k
    a
    9 replies · 3 participants
Powered by Linen
Title
g

Gabriel Milan

10/22/2021, 6:26 PM
Hi there! Has anyone already set up prefect server on k8s?
k

Kevin Kho

10/22/2021, 6:28 PM
Hi @Gabriel Milan, the helm chart and README of that can be found here
g

Gabriel Milan

10/22/2021, 6:29 PM
awesome!! I wasn't able to find it before, thank you very much!
is there anything on setting up agents for my server deployment?
the k8s agent, i mean
a

Anna Geller

10/22/2021, 7:45 PM
@Gabriel Milan agents work the same way, e.g. following this documentation page you’ll get a YAML template for the agent pod https://docs.prefect.io/orchestration/agents/kubernetes.html#running-in-cluster
g

Gabriel Milan

10/22/2021, 7:50 PM
just another small thing: how do I set secrets on my server?
k

Kevin Kho

10/22/2021, 7:52 PM
Server doesn’t have support for secrets so you would set your secrets locally like this
g

Gabriel Milan

10/22/2021, 7:57 PM
got it
thanks!
View count: 2