https://prefect.io logo
Docs
Join the conversationJoin Slack
Channels
announcements
ask-marvin
best-practices-coordination-plane
data-ecosystem
data-tricks-and-tips
events
find-a-prefect-job
geo-australia
geo-bay-area
geo-berlin
geo-boston
geo-chicago
geo-colorado
geo-dc
geo-israel
geo-japan
geo-london
geo-nyc
geo-seattle
geo-texas
gratitude
introductions
marvin-in-the-wild
prefect-ai
prefect-aws
prefect-azure
prefect-cloud
prefect-community
prefect-contributors
prefect-dbt
prefect-docker
prefect-gcp
prefect-getting-started
prefect-integrations
prefect-kubernetes
prefect-recipes
prefect-server
prefect-ui
random
show-us-what-you-got
Powered by Linen
prefect-cloud
  • b

    Benoît Linsey-Fazi

    01/20/2023, 1:36 PM
    Hello, I have 2 two questions with regards to pricing : • Does prefect have plans for startups ? • The docs says that the personal edition includes 3 users while the organization only includes one. Isn't it weird ? Thanks !
    ✅ 1
    a
    • 2
    • 2
  • a

    Austin Weisgrau

    01/20/2023, 8:28 PM
    Hi there! I'm working on understanding Prefect Cloud infrastructure. I'm coming from an Airflow background. I'm trying to understand the difference between orchestration infrastructure and task execution infrastructure when using Prefect Cloud. Prefect Cloud manages the orchestration infrastructure for you, but does it also manage the task execution infrastructure or do you have to set that up separately? If we wanted to, for example, run tasks in a docker container or as non-containerized python scripts.
    ✅ 1
    b
    g
    • 3
    • 7
  • r

    Ryan Bermel

    01/20/2023, 8:56 PM
    Hi, I am not able to see any logs in the Cloud IU from any of the flows running on my prefect agents. This worked in the past, but has stopped working.
    k
    • 2
    • 3
  • b

    Ben Muller

    01/25/2023, 10:06 PM
    Anyone else have an issue in the UI on the
    Flows
    page when typing into the input to search for a Flow that the text you type gets automatically deleted for about 10 seconds ?
    m
    • 2
    • 2
  • b

    Ben Muller

    01/25/2023, 11:47 PM
    Is prefect cloud having issues, I am getting this error when trying to deploy my deployments: Note: this was running fine about 15 minutes ago. 🧵
    ✅ 1
    j
    e
    k
    • 4
    • 8
  • c

    Carlos Cueto

    01/29/2023, 1:48 PM
    Hello. Since Prefect 1.0 had the ability to send Automations to a generic Webhook URL, we had an automation set up to a Microsoft Power Automate webhook that we had running. Prefect 2.0 does not have a generic Webhook URL action for Automations that I saw, so I went ahead and created my own by inheriting
    AppriseNotificationBlock
    . I went ahead and registered the custom block I created with the UI and created a block under my Blocks page with the webhook URL. When I
    .load()
    the block from local python client and call
    .notify()
    on it, it works perfectly. I thought it would work just the same if I add it as an Action on the Automations tab, but to my surprise, it fails every time. I see the failed events on the Events tab of the automation I set up. Is there a way to see what is causing the failure?
    w
    • 2
    • 5
  • c

    Carlos Cueto

    01/29/2023, 1:54 PM
    image.png
  • d

    Dmitry Sandovich

    01/30/2023, 6:33 PM
    Hi everyone, I'm exploring Prefect REST API. Authentication request fails with 401 (Unauthorized). Here is my request using curl: curl --request POST 'https://api.prefect.cloud/auth/login' --header 'Content-Type: application/json' --header 'Authorization: Bearer <Service Account API key>' I validated service account api key using cli: prefect auth login --key <Service Account API key> With cli I'm able to login successfully. Does anyone know why my authentication request fails? Thanks, Dmitry
    j
    x
    • 3
    • 9
  • s

    Slackbot

    01/30/2023, 7:27 PM
    This message was deleted.
  • t

    Thet Naing

    02/01/2023, 7:55 PM
    Hi all, I am having trouble working with Prefect GitHub storage blocks in the latest version,
    2.7.10
    . I keep getting this error when attempting to run flows. Any idea what's going on?
    Flow could not be retrieved from deployment.
    Traceback (most recent call last):
      File "/usr/local/lib/python3.10/site-packages/prefect/engine.py", line 262, in retrieve_flow_then_begin_flow_run
        flow = await load_flow_from_flow_run(flow_run, client=client)
      File "/usr/local/lib/python3.10/site-packages/prefect/client/utilities.py", line 47, in with_injected_client
        return await fn(*args, **kwargs)
      File "/usr/local/lib/python3.10/site-packages/prefect/deployments.py", line 166, in load_flow_from_flow_run
        await storage_block.get_directory(from_path=deployment.path, local_path=".")
      File "/usr/local/lib/python3.10/site-packages/prefect/filesystems.py", line 916, in get_directory
        raise OSError(f"Failed to pull from remote:\n {err_stream.read()}")
    OSError: Failed to pull from remote:
     Cloning into '/tmp/tmpabasdfprefect'...
    fatal: could not read Username for '<https://github.com>': No such device or address
    ‼️ 1
    ✅ 1
    m
    a
    • 3
    • 4
  • c

    Carlos Cueto

    02/03/2023, 3:34 PM
    I have a notification action set up on the Automations in Cloud UI and sometimes fails but I don't know the reason. If I give someone internally a screenshot of all the event ID failure message, are you able to pinpoint for me why this action is failing? It's an email notification upon flow failure that sometimes decides to fail, but it's causing large issues in our organization due to missed failure notifications for important workflows.
    w
    • 2
    • 6
  • f

    Flavio Oliveira

    02/06/2023, 11:13 AM
    Anyone using SaturnCloud with Prefect 2? How is the experience? Do they support Prefect 2?
    m
    • 2
    • 1
  • j

    Jonas

    02/06/2023, 11:22 AM
    Hello, I have a weird behaviour of prefect cloud blocks. I want to add email credentials for notifications in a Block. But actually most of the essential blocks are missing (see the screenshots). How am I able to see all blocks in prefect cloud?
  • j

    Joseph Thickpenny Ryan

    02/06/2023, 5:02 PM
    Is there a module in the prefect project (or other places I guess) for interacting with the automations API, or do we have to write http requests?
    w
    • 2
    • 2
  • j

    Joseph Thickpenny Ryan

    02/06/2023, 5:54 PM
    In the same vein of question, is there anywhere that I can get a list of event names that we can make triggers on?
    w
    • 2
    • 11
  • r

    Ryan Bermel

    02/06/2023, 9:09 PM
    Hello, I am struggling to onboard some new flows. Is there somewhere that I can look to see why the Cloud is not pulling these from storage? I am getting to following error when I deploy.
    Block document has schema checksum <omitted checksum> which does not match the schema checksum for class 'RemoteFileSystem'. This indicates the schema has changed and this block may not load.
      return cls._from_block_document(block_document)
    c
    • 2
    • 3
  • j

    Joseph Thickpenny Ryan

    02/08/2023, 2:50 PM
    I've created a deployment in the prefect 2 cloud, but the run button for it is greyed out (ie not clickable), any tips on how to debug this?
    • 1
    • 3
  • t

    Tim-Oliver

    02/09/2023, 10:40 AM
    Hi, In one of my 3 workspaces I can't see the new GitHub Block with the exclude .git directory option. Is there something I have to do on my side?
    ✅ 1
    • 1
    • 1
  • c

    Carlos Cueto

    02/10/2023, 1:54 PM
    Woke up this morning to the Cloud's Prefect 2.8.0 update with work pools. I know that all existing work-queues went into the default work pool, however, flow runs are being tagged with either
    work queue
    or
    work pool
    in what seems to be random. I liked the old way of seeing the work queue because it gave me an idea of the execution environment of said flow. Is this intended?
    :upvote: 1
    👍 1
    m
    • 2
    • 3
  • t

    Tanay Kothari

    02/12/2023, 3:18 AM
    Hey Prefect team,
    Task run '<id>' received abort during orchestration: This run cannot transition to the RUNNING state from the RUNNING state. Task run is in RUNNING state.
    
    prefect.exceptions.MissingResult: State data is missing. Typically, this occurs when result persistence is disabled and the state has been retrieved from the API.
    I’m getting this error. I have a Dask cluster on EKS. When I run the prefect flow locally, it connects to the dask cluster and runs it just fine. When I deploy it to the cloud, my prefect agent on Fargate runs the first task in my DAG and then crashes with this error a few seconds later. I’m using Prefect 2.8.0.
    • 1
    • 1
  • a

    Alex Turek

    02/13/2023, 6:29 PM
    where is Prefect Cloud’s maintenance schedule documented?
    c
    j
    • 3
    • 12
  • s

    Stéphan Taljaard

    02/13/2023, 8:24 PM
    set the channel description: Hi 👋 - post comments & code in 🧵 - ☁️ docs: https://docs.prefect.io/ui/cloud/ - status updates: https://prefect.status.io/
    🙌 1
    :thank-you: 2
    👍 1
  • d

    Data Ops

    02/14/2023, 2:13 PM
    Hi everyone! I need speak with someone from financial department about our plan (Logcomex). Our Contact is Richard Christie
    ✅ 1
    r
    • 2
    • 1
  • j

    Joseph Thickpenny Ryan

    02/14/2023, 4:22 PM
    Is there somewhere I can subscribe to upcoming change notifications for Prefect Cloud? I'm working on a v1 to v2 migration at the moment and was confused when work queues went missing (I've found them now) from the UI since the last time I looked
    ✅ 1
    r
    • 2
    • 1
  • s

    Sidharth Ramalingam

    02/15/2023, 3:36 PM
    Hi community is there a way where I could run a deployment on prefect cloud without keeping my local PC switched on. My question is after creating a deployment and scheduling will the flow run even when my PC is disconnected.
    ✅ 1
    s
    • 2
    • 5
  • j

    Jai P

    02/15/2023, 4:01 PM
    hi there! through some wackiness, i've ended up with a couple of prefect cloud accounts based on the aliases of my e-mail address. Is there a way in prefect cloud that i can request for one of those accounts to be deleted?
  • j

    Joseph Thickpenny Ryan

    02/15/2023, 4:08 PM
    Is the automations limit of 3 per workspace a free tier limit or a forever limit?
    ✅ 1
    w
    • 2
    • 2
  • n

    Nikhil Jain

    02/15/2023, 8:31 PM
    Hi, we are getting some spurious alerts from prefect-cloud: This alert is configured to trigger if a flow-run remains in
    Late
    or
    Pending
    state for over 10 minutes, but we are seeing this even if the flow completed successfully and in less than 10 minutes.
    Prefect flow Late or Pending for over 10 minutes
    Flow run build-meta/spectral-anteater entered state
    Completed
    at 2023-02-15T19:39:53.508606+00:00.
    Flow run URL: https://app.prefect.cloud/account/adee12e2-25d9-428a-9a77-32d8d7ca2ac7/workspace/28562586-0812-4e92-803f-99ce64384cbd/flow-runs/flow-run/81433951-84a0-4bb1-95e9-22a942912d3a
    State message: All states completed.
    w
    c
    i
    • 4
    • 7
  • t

    Thet Naing

    02/16/2023, 1:43 AM
    Since the new waterfall visualization was introduced to Prefect Cloud, we've been experiencing frequent freezing and crashing on the UI when using
    map
    for tasks. Is there a way to simplify the visualization so that it doesn't show every single task mapping on the UI? It also makes it rather difficult to see and understand what's happening in the flow
    s
    • 2
    • 6
  • j

    JP

    02/16/2023, 9:06 AM
    Heya! Last night we received quite many slack notifications of flow being healthy from single Automation over span of roughly 2 hours. All but 2 were with valid payload 😕 The amount of notifications are to be attributed to following config • Trigger Type: Work queue health • _*Work Queues*_: <single production work queue> • _*Work Queue*_: Stays in > Healthy • _*For*_: 10 Seconds Our Slack notification template looks as follows
    Name: {{ work_queue.name }}
    Last polled: {{ work_queue.last_polled }}
    Late run count: {{ work_queue.late_runs_count }}
    URL: {{ work_queue|ui_url }}
    But.. why are all but 2 notifications with no values? (see screenshot below). The two valid notifications are 1. The first one at 21:53 CEST 2. At 22:54 CEST (visible in screenshot) edit: For the record.. We do not intend to run such trigger condition (it was a mis-configuration). We do not really care being notified every 10 seconds about a queue being healthy 😅 Hence the subject of the notifications in screenshot is mis-leading
    ✅ 1
    w
    s
    • 3
    • 10
Powered by Linen
Title
j

JP

02/16/2023, 9:06 AM
Heya! Last night we received quite many slack notifications of flow being healthy from single Automation over span of roughly 2 hours. All but 2 were with valid payload 😕 The amount of notifications are to be attributed to following config • Trigger Type: Work queue health • _*Work Queues*_: <single production work queue> • _*Work Queue*_: Stays in > Healthy • _*For*_: 10 Seconds Our Slack notification template looks as follows
Name: {{ work_queue.name }}
Last polled: {{ work_queue.last_polled }}
Late run count: {{ work_queue.late_runs_count }}
URL: {{ work_queue|ui_url }}
But.. why are all but 2 notifications with no values? (see screenshot below). The two valid notifications are 1. The first one at 21:53 CEST 2. At 22:54 CEST (visible in screenshot) edit: For the record.. We do not intend to run such trigger condition (it was a mis-configuration). We do not really care being notified every 10 seconds about a queue being healthy 😅 Hence the subject of the notifications in screenshot is mis-leading
✅ 1
w

Will Raphaelson

02/16/2023, 3:03 PM
thanks for the flag @JP - I believe this may have been a one time blip as a result of a migration we applied on the back end, sorry about that! I think if you fix any misconfiguration on your side things should work as expected, but please do let me know if this crops up again and we can get to the bottom of it.
j

JP

02/16/2023, 3:05 PM
Hi, @Will Raphaelson! Are you saying the notifications being empty is because of the migration? 🤔 The misconfiguration was creating the many notifications we had received, but the issue is that almost all of them were with no values.
I can try to reproduce this, if you'd like 🤷
w

Will Raphaelson

02/16/2023, 3:12 PM
The frequency was for sure because of the migration, and it may have something to do with the empty parameter values - basically the trigger needs a valid triggering event to materialize those values, and i think they were just missing when these fired. I will try to reproduce too in a few hours and make sure these values are populating, but the more the merrier if you want to file a repro example at it would be helpful and appreciated. https://github.com/PrefectHQ/prefect
:thank-you: 1
j

JP

02/16/2023, 3:15 PM
Ah I see. So the trigger spec
• Trigger Type: Work queue health
• _*Work Queues*_: <single production work queue>
• _*Work Queue*_: Stays in > Healthy
• _*For*_: 10 Seconds
Should not have generated gazillion events? If so, then it makes sense! The two "valid" events we received were exactly 1 hour apart, so I suppose this condition would generate hourly alerts of queue "being healthy" for more than 10 seconds 🤷
w

Will Raphaelson

02/16/2023, 5:04 PM
so how the stays in construction works under the hood is that it watchesfor a healthy events, and then watches for the absence of an unhealthy event within ten seconds, which I believe your conditions triggered. On the parameter values being blank, all of mine are populating except for the “count late runs” field, which i think is a legit bug and i filed it internally to clean up, so thanks for surfacing that.
:thank-you: 1
I think everything is behaving as expected right now, let me know if you encounter any other issues?
s

Stéphan Taljaard

02/16/2023, 7:06 PM
I also had queue health check misfires this week, where I got an alert stating my queue is unhealthy. Then I click the link and find that it is not unhealthy. My suspicion is that time check (unhealthy
for 9 minutes
) was ignored, and as soon my batch of flows were due even if a flow is only seconds late, the alert fires. Should I message in this thread if it happens again?
w

Will Raphaelson

02/16/2023, 7:06 PM
yes please, thanks for that. im also monitoring on our side.
👍 1
s

Stéphan Taljaard

02/24/2023, 8:13 AM
Here's a misfire that just happened
Unhealthy work queue
Name: default
Last polled: 2023-02-24T08🔟25.029779+00:00
Late run count:
URL: https://app.prefect.cloud/account/2a0672be-6e64-4081-bdfd-a4dfded5a802/workspace/b70a86f1-659f-408f-add7-9665c6bfa327/work-queues/work-queue/2aa024db-13b8-4203-a89b-2db6487959ad
Prefect Notifications | Today at 10:10
View count: 1