https://prefect.io logo
Docs
Join the conversationJoin Slack
Channels
announcements
ask-marvin
best-practices-coordination-plane
data-ecosystem
data-tricks-and-tips
events
find-a-prefect-job
geo-australia
geo-bay-area
geo-berlin
geo-boston
geo-chicago
geo-colorado
geo-dc
geo-israel
geo-japan
geo-london
geo-nyc
geo-seattle
geo-texas
gratitude
introductions
marvin-in-the-wild
prefect-ai
prefect-aws
prefect-azure
prefect-cloud
prefect-community
prefect-contributors
prefect-dbt
prefect-docker
prefect-gcp
prefect-getting-started
prefect-integrations
prefect-kubernetes
prefect-recipes
prefect-server
prefect-ui
random
show-us-what-you-got
Powered by Linen
prefect-server
  • e

    Enda Peng

    04/22/2021, 4:27 PM
    I find an interesting behavior of the local server. I used to launch server on my host A and created some data. Today I try to start server on host B and access its UI with browser on host A, I find that all my flows/projects etc are visible to host B UI because the resources are loaded as
    <localhost://xxxx>
    , others who access server B UI can’t see it
    n
    6 replies · 2 participants
  • t

    Todd Lindstrom

    04/22/2021, 9:25 PM
    Dear Prefect Community. I am new to prefect and I'm evaluating it vs Airflow and the why-not-airflow paper on the website was very convincing but my first experience was not positive - thus reaching out here because I dont want to give up on a technology that might just make my life easier later by giving up too early. I will paste the startup error in this thread - but the problem is on prefect server start - I am getting connection errors, and then later when I pull up the UI :8080 I get to the getting started screen but the graphql connect fails with more docker error logs scrolling. I can move the UI question to the other room if this is the wrong place.
    m
    z
    28 replies · 3 participants
  • e

    Enda Peng

    04/22/2021, 10:57 PM
    I get so confused by how the
    graphql
    works. Here is my operations • First I try
    prefect server config > origin.yml
    then launch it with docker-compose. App works on
    localhost:8080
    • Under
    getting started
    tab, I put
    <http://localhost:4200/graphql>
    as graphql end point and it works. So far so good • Then I try
    prefect server config --graphql-port 40002 > my-prefect.yml
    , launch with docker composer. App still works on the same address • However, I tried to give ui the graphql endpoint with values
    <http://localhost:4200/graphql>, <http://localhost:40002/graphql>, <http://graphql:40002/>, <http://localhost:40002/>
    with/without last
    /
    . None of them works. I try in browser
    <http://localhost:40002/graphql/>
    , it gives me a console to play. As long as I touched graphql port, I will not be able to set up a workable local sever. Really appreciate if someone can help me take a look
    n
    4 replies · 2 participants
  • e

    Enda Peng

    04/22/2021, 11:45 PM
    Re my message above. Eventually my workaround is: • No touch on
    graphql
    . Only move ui and server port
    prefect server start --ui-port 30001 --server-port 30002 --no-postgres-port --no-hasura-port --no-graphql-port
    • On
    prefect server
    tab on the web application, type in
    <http://localhost:30002/graphql>
    to reflect the change • You will be blocked on next tap when creating tenant, add this to your ~/.prefect/config.toml. Otherwise command
    prefect agent start --name "Default Agent"
    will by default access port
    4200
    instead
    server]
    host = "<http://localhost>"
    port = "4200"
    host_port = "30002"
    endpoint = "${server.host}:${server.host_port}"
    Then everything all set
    😄 1
    n
    1 reply · 2 participants
  • f

    Fabrice Toussaint

    04/23/2021, 11:45 AM
    Hey! Can anyone confirm for me that if a schedule contains more than 10 (concurrent) runs, this is limited to 10 runs only. My schedule contains 20 runs (with different parameters for a flow) where many start at the same time. However, when I deploy this schedule to Prefect only 10 runs are scheduled instead of the 20. This is on Prefect's website: There is no limit to the number of flow runs you can create, even if they start at the exact same time with the exact same parameters: Prefect supports unlimited concurrent execution. This does not seem the case for me.
    m
    20 replies · 2 participants
  • a

    Andy Dyer

    04/23/2021, 4:22 PM
    what is the max size you can give as parameters?
    k
    m
    6 replies · 3 participants
  • i

    Ismail Cenik

    04/23/2021, 8:36 PM
    Hello all, We will have 3 flows for 3 environments (DEV, QA, PROD) ... Those flows are all the same except the AWS resources. Instead of creating 3 different python files, is there a way to pass resource names as a parameter with this command to the python file? prefect register flow --file prefectDemo.py --project \"Data Loading\"
    n
    20 replies · 2 participants
  • v

    Vladislav Bogucharov

    04/25/2021, 2:02 PM
    Hi, Prefect community! I successfully created my first flow and was able to see it in the UI. But I have some questions. 1). At the moment I am connecting to the remote machine via ssh. When Prefect is running, I can access the UI through both localhost and the ip of my remote machine, for example xxx.xx.xxx.xxx:8080 or localhost:8080. But I cannot view my flows from my phone, other computer, etc. What confuses me is that the UI is visible on all devices, but flows are displayed only on my laptop at the specified addresses. What could be the problem? 2). How to start agent in detached mode? Now I am running
    prefect agent local start
    every time.
    n
    j
    +1
    35 replies · 4 participants
  • j

    Justin

    04/27/2021, 8:39 AM
    Hi community, I am planning to deploy to Docker Swarm, is there a more official version than https://github.com/flavienbwk/prefect-docker-compose around?
    f
    b
    +1
    4 replies · 4 participants
  • b

    Ben Collier

    04/28/2021, 11:11 AM
    Hi all! We’re deploying the prefect ecs agent to Fargate. Works fine locally, but on Fargate we’re getting neither a successful health check nor any logs appearing in Cloudwatch. Bear in mind that we can deliberately bork the build and see logs. So I’m wondering if anyone else has had a similar experience. Would it be better to start with the docker version of the agent and try to get that working first?
    c
    k
    +1
    108 replies · 4 participants
  • m

    Matt Camp

    04/28/2021, 12:58 PM
    hello, I'm new to Prefect, just got it running yesterday. I am coming from a history of using Airflow and was curious how Prefect users manage variable and connetions? I saw on the main Prefect docs that secrets aren't supported by the
    prefect-server
    and that you need to use the
    config.yml
    . This seems like an all around disaster waiting to happen. I am not able to use the Prefect Cloud due to where I am setting things up, so are there alternative methods people use? TIA
    🙏 1
    k
    m
    +3
    12 replies · 6 participants
  • m

    Mariusz Olszewski

    05/01/2021, 5:35 AM
    how, how to setup flow for not start if last execution not end yet?
    k
    1 reply · 2 participants
  • r

    Rob Fowler

    05/01/2021, 9:21 AM
    I have this weird problem now with one of my lab prefect servers. I can't run jobs, they are only scheduled for 24 hours later.
    k
    2 replies · 2 participants
  • m

    merlin

    05/02/2021, 8:02 PM
    Last noob question before I actually run a POC I promise "Where is an example of prefect task that just runs a pre-built Docker container." I've read the docs extensively, and I think this is covered with LocalExecutor + DockerRun, but the examples are confusing me -- they seem to all be set up for executing some prefect flow python code from the flow run on various environments. Here is my use case: • I have data transform code written in Julia, in a docker container already built with an environment to execute that Julia code, along with permissions to access s3 for read/write on either end of the execution. • I just want to schedule the running of this container (Fargate, EKS?) • Does Prefect need to be installed on the container? • Does the flow code need to execute some python commands in the container to start it off? This probably seems obvious, but its not to me. I want to use Prefect as an orchestrator for many kinds of jobs (Spark, julialang, shell calls to internal team services), and the API is full of environment-specific tasks like "AWS Tasks", "DBT Tasks", "Shell Tasks", etc. I am confused if generic docker containers can be run or if only the kind of tasks from the API documentation can be run.
    k
    7 replies · 2 participants
  • n

    Nathan Walker

    05/03/2021, 1:53 PM
    Hey folks, We're currently using Server until we can get Cloud approved at a client. There's some GraphQL queries/mutations we're looking at adding to Server (more metadata aggregation, schedule updates, etc. - probably no new resolvers) and we're wondering if adding to the GraphQL Schema will be an option in Cloud as well.
    m
    4 replies · 2 participants
  • i

    Ismail Cenik

    05/05/2021, 2:20 AM
    Hello, we are intermittently getting “ No heartbeat detected from the remote task, marking the run as failed.” When we look at the slack channel, we detect two solutions. 1. Adding the "cluster-autoscaler.kubernetes.io/safe-to-evict": "false." 2. Another solution might be disabling the Lazarus toggle in the Prefect UI for our Plan Data Loading Flow -> Done We applied them but still getting the same error. What is your recommendation?
    c
    3 replies · 2 participants
  • f

    Fabrice Toussaint

    05/05/2021, 8:24 AM
    Hey everyone, Can someone explain to me why I keep getting this error in my flow runs 
    requests.exceptions.ReadTimeout: HTTPConnectionPool(host='prefect-apollo.prefect', port=4200): Read timed out. (read timeout=15)
    ? It does not happen all the time but it happens so often that a lot of tasks fail because of it. Our Prefect server is deployed using Kubernetes on GCP and the pod seems to be getting enough resources.
    k
    a
    38 replies · 3 participants
  • f

    Fabrice Toussaint

    05/06/2021, 7:32 AM
    Hey everyone, Does anyone know why I get this errors (sometimes) with scheduled tasks?
    Failed to retrieve task state with error: ClientError([{'message': 'Expected type UUID!, found ""; Could not parse UUID: ', 'locations': [{'line': 2, 'column': 5}], 'path': ['get_or_create_task_run_info'], 'extensions': {'code': 'INTERNAL_SERVER_ERROR', 'exception': {'message': 'Expected type UUID!, found ""; Could not parse UUID: '}}}])
    Traceback (most recent call last):
      File "/usr/local/lib/python3.8/site-packages/prefect/engine/cloud/task_runner.py", line 154, in initialize_run
        task_run_info = self.client.get_task_run_info(
      File "/usr/local/lib/python3.8/site-packages/prefect/client/client.py", line 1399, in get_task_run_info
        result = self.graphql(mutation)  # type: Any
      File "/usr/local/lib/python3.8/site-packages/prefect/client/client.py", line 319, in graphql
        raise ClientError(result["errors"])
    prefect.utilities.exceptions.ClientError: [{'message': 'Expected type UUID!, found ""; Could not parse UUID: ', 'locations': [{'line': 2, 'column': 5}], 'path': ['get_or_create_task_run_info'], 'extensions': {'code': 'INTERNAL_SERVER_ERROR', 'exception': {'message': 'Expected type UUID!, found ""; Could not parse UUID: '}}}]
    Our Prefect server is deployed using Kubernetes on GCP with autoscaling enabled, maybe it has something to do with that?
    k
    20 replies · 2 participants
  • n

    Niko Usai

    05/07/2021, 8:37 AM
    Hi everyone I've got a prefect agent running on a Kubernetes cluster for a year or so and everything was fine, then it started to give me this error few days ago, without any change in anything:
    [2021-05-07 08:16:03,101] ERROR - agent | Error while deploying flow: ValidationError({'_schema': 'Invalid data type: None'},)
    I've tried to search, but couldn't find any hint on what to look, anyone could help?
    k
    7 replies · 2 participants
  • d

    Devin McCabe

    05/07/2021, 4:42 PM
    Does anyone have any tips on managing environments with Prefect Cloud? Did you set up different projects for dev/qa/prod? Different flows within a single project? I'd love to hear about some examples of successful architectures if anyone has any.
    k
    3 replies · 2 participants
  • a

    Andy Dyer

    05/10/2021, 4:30 PM
    Hi all, Does anyone have a good recipe that is pretty plug&play CI for deploying flows either with dind or mounting the docker sock. (preferably using gitlab.com runners)
  • d

    Devin McCabe

    05/10/2021, 6:21 PM
    Does anyone know if the local Prefect server is supposed to work with
    DockerAgent
    ? I'm trying the example flow from the tutorial:
    from prefect import Flow
    from prefect import task
    from prefect.run_configs import DockerRun
    
    
    @task
    def add(x, y=1):
        return x + y
    
    
    with Flow("My first flow!", run_config=DockerRun()) as flow:
        first_result = add(1, y=2)
        second_result = add(x=first_result, y=100)
    
    flow.register(project_name="test-project")
    I submit the flow using the "Quick Run" button in the UI and start a Docker agent:
    prefect agent docker start --show-flow-logs --log-level DEBUG
    But the run forever stays in the pending state and I only ever see
    Querying for flow runs / No flow runs found
    in the agent logs. I've tried
    DockerRun()
    as above and also using a custom image containing only this:
    FROM python:3.8-buster
    RUN pip install prefect
    The UI shows it can reach my agent, though. Any ideas?
    k
    11 replies · 2 participants
  • m

    Marwan Sarieddine

    05/10/2021, 8:25 PM
    Hi folks, we experienced a flow run failure that seems to stem from prefect cloud given the failure messages we see. Re-running the flow resolves the issue but I thought I would flag it in case it helps.
    k
    z
    12 replies · 3 participants
  • n

    Noah Guilbault

    05/10/2021, 11:11 PM
    Help needed / Question -- I have a local agent running in a docker container that is properly picking up a flow i publish from another container but failing with the following error message. "Failed to load and execute Flow's environment: ValueError('Flow is not contained in this Storage')" The flow is assigned a UniversalRun (i tried a LocalRun with the same effect), a LocalExecutor, LocalStorage, and labels ABC and DEF. The Flow has labels ABC, DEF and is properly picked up by the agent.
    k
    15 replies · 2 participants
  • t

    Talha

    05/11/2021, 2:53 PM
    Hi I am trying to connect to the prefect server on localhost:8080. But i am working in a firm where we use HTTP and HTTPS proxies. The prefect server doesnot connect if i don't bipass the proxy. And when I bypass the proxy the flow does not run, as one of the task is api data fetching which needs Proxies. What is the way around it? You can see in picture I am trying to set the Http and https env in the run config as well. If i don't put the os.environ to empty. Then I get the following error: "error 503: cannot connect to localhost:4200" which is my apollo docker port.
    n
    t
    17 replies · 3 participants
  • j

    jaehoon

    05/12/2021, 2:00 AM
    Hi Prefect family! i want see DEBUG logs in UI I tried
    prefect.config.logging.level='DEBUG'
    prefect.config.debug=True
    and
    os.environ["PREFECT__LOGGING__LEVEL"]='DEBUG'
    However, it doesn't work How can i set DEBUG log level?
    c
    5 replies · 2 participants
  • m

    Martin

    05/12/2021, 7:56 AM
    Hi, I am new to Prefect for task orchestration. And have couple questions here: 1. Understand that we have Prefect Cloud which is a managed service. But could we deploy prefect server as cluster? 2. Is it possible to setup a dependency between "flows" instead of "tasks"?
    f
    n
    +2
    7 replies · 5 participants
  • d

    Diego Alonso Roque Montoya

    05/12/2021, 10:58 PM
    Is there a way to send a SUCCESS signal but continue the process running? For context i have a setup heavy process that ideally has a cleanup but as soon as the data is done, I want to continue processing. However, I don’t want to finish the task and move to another process for cleanup because that’s fairly expensive
    z
    1 reply · 2 participants
  • a

    Amanda Wee

    05/13/2021, 1:14 AM
    My team is running Prefect Server on ECS (apollo, hasura, graphql, and towel as separate containers in a single ECS task; postgres as an aurora cluster; Prefect UI as another ECS task), with a single local Prefect agent also on ECS. From time to time we encounter HTTP 502 errors and/or read timeout errors, and also a flow where we had a large task for which we would get heartbeat timeouts, but after breaking it up, Prefect UI would then report that at one of the broken up tasks, the flow runs "forever". The errors seem pretty inconsistent though, but started to occur a bit more frequency recently. Prefect Server itself has remained up, so I originally thought that there was something finicky about the internal load balancer, yet it does work. Would anyone have any ideas what might be going wrong?
    k
    m
    7 replies · 3 participants
  • m

    Mariusz Olszewski

    05/13/2021, 6:16 PM
    Hi, I implement slack notifications about errors from prefect. how to customize the content of the message here? feg. Include direct link to flow or task which fail?
    k
    1 reply · 2 participants
Powered by Linen
Title
m

Mariusz Olszewski

05/13/2021, 6:16 PM
Hi, I implement slack notifications about errors from prefect. how to customize the content of the message here? feg. Include direct link to flow or task which fail?
k

Kevin Kho

05/13/2021, 6:50 PM
Hi @Mariusz Olszewski , I think in general you want to be using the SlackTask here instead to customize messages
View count: 1