https://prefect.io logo
Join the conversationJoin Slack
Channels
announcements
ask-marvin
best-practices-coordination-plane
data-ecosystem
data-tricks-and-tips
events
find-a-prefect-job
geo-australia
geo-bay-area
geo-berlin
geo-boston
geo-chicago
geo-colorado
geo-dc
geo-israel
geo-japan
geo-london
geo-nyc
geo-seattle
geo-texas
gratitude
introductions
marvin-in-the-wild
prefect-ai
prefect-aws
prefect-azure
prefect-cloud
prefect-community
prefect-contributors
prefect-dbt
prefect-docker
prefect-gcp
prefect-getting-started
prefect-integrations
prefect-kubernetes
prefect-recipes
prefect-server
prefect-ui
random
show-us-what-you-got
Powered by Linen
prefect-ui
  • b

    bitsofinfo

    08/22/2022, 9:15 PM
    i.e. my
    ui.apolloApiUrl
    is =
    <http://apollo-api.mydomain.net/graphql|apollo-api.mydomain.net/graphql>
    yet its reporting localhost in the error
  • b

    bitsofinfo

    08/22/2022, 9:16 PM
    oh... well this is confusing... am i using the 1x series chart?
  • b

    bitsofinfo

    08/22/2022, 9:29 PM
    looking at the orion helm chart - is there not a way to customize the
    api.ingress
    labels?
    g
    • 2
    • 1
  • h

    harsh patel

    08/23/2022, 3:42 PM
    Do we have a roadmap for upcoming features? Was wondering if we are going to have a UI based workflow creation/editor.
    c
    • 2
    • 1
  • j

    Jonathan Pou

    08/25/2022, 1:16 AM
    Does cloud.prefect.io happen to be down? For the last 20min, I've been trying to access the main page via browser (Firefox and Chrome) on my computer as well as from my phone (Safari) and I happen to be stuck on the loading screen.
    g
    r
    +2
    • 5
    • 5
  • n

    Numline1

    08/25/2022, 7:56 PM
    Hey guys, I'm kinda struggling to set up a Prefect on our company server. So it's running and all that, but I had to setup a reverse proxy (Nginx) as well, so that people can access the UI via standard https at 443 and at a company subdomain (internal that is). The UI itself is accessible, but it's trying to make REST API requests to http://127.0.0.1/api/something which is obviously not working. Is there any env setting I can use to make it understand it should use https://prefect.internal.company.com/api instead? I couldn't find anything on that topic anywhere 😞 Thanks!
    r
    • 2
    • 9
  • Feedback on Radar view
    k

    Keith Hickey

    08/27/2022, 2:12 PM
    BLUF: Am I missing some kind of control flow UI of a DAG, or is Radar the only UI? Will Radar or another UI show dependent-order-of-operations at some point? _*UPDATE: See threaded replies. You can direct control-flow of the UI if you get a
    PrefectFuture
    from a prior task and pass it into `next_task.submit(wait_for=future_for_prior_task)`*_ Hello. Regarding the Prefect 2.0 Radar View available from the orion server ... it's neat, and provides a unique dimension of data-traceability and drill-down/fan-out ... however I feel like it has over-rotated from the Prefect 1.0 Schematic to strictly visualize explicit point-to-point data propagation between tasks, and loses the logical sequence of task execution that is equally if not more important. Put another way: Radar highlights data flow and hides control flow. I would argue control flow is more important, as you can typically infer how data is flowing by tracing the flow of control, but inferring dependency and temporality from the subset of tasks that explicitly exchange Python data in a tightly-coupled way will lose sight of implicit data dependencies among tasks. It would seem that getting away from DAGs (directed, acyclic graphs) is a driving principle of Prefect 2.0 ("Goodbye, DAGs"). But I still find it immensely helpful to know progress of the flow execution, i.e. where it is in the overall control graph, or where it failed. Part of my context is that many of our DAGs / pipelines / ETLs / data processing is using external data stores (databases, files, lakehouse tables on cloud storage, etc.) with very large datasets being processed. Maybe this is more batch-processing vs. record streaming and the Radar UI is not as well suited for that? But it's not feasible to pull those large datasets into memory of a python process to be handed as python objects from task to task, or serialized from process to process. Therefore with only this implicit data-dependency, and not an explicit code-level one in the flow, no relationships is mapped between those very-much-related Tasks in the Radar UI. So questions: 1. Is there a way to code-in (via the Prefect API for tasks/flows/Futures) implicit data-dependencies from task-to-task? ... So that those tasks are laid out and connected on the Radar UI in expanding concentric circles? a. i.e. Task
    do_a
    pre-processes dataset
    A
    in
    DB_A
    ... Task
    do_b
    transports resulting data in
    DB_A
    to external
    API_B
    ... so
    do_a
    must happen before
    do_b
    and yields control to it only after done. 2. Even if there's not an implicit (or explicit) data-dependency, but just a matter of sequential control flow (e.g. as governed by the
    SequentialTaskRunner
    ) can this be expressed via the Python Task/Flow API?
    When using the SequentialTaskRunner, tasks are laid out on the same Radar ring with no obvious order, even though they clearly are run in-sequential-ordering (not concurrent). 3. I_*s there hope for a DAG-like schematic? And if so would that be built further into Radar, or another forthcoming UI?*_ 4. There seemed like some API constructs when Tasks returned PrefectFuture (or Task?) objects, which allowed some explicit functions to chain things together. But the recent shift to strip return objects down to their results makes that less possible. Is that Python API and how tasks can be tied together still under design? I saw a few other posts where people may be struggling with the same conceptual issue, linked below. • https://prefect-community.slack.com/archives/CL09KU1K7/p1649011482174799 • https://prefect-community.slack.com/archives/C0192RWGJQH/p1642702304024400
    ✅ 2
    👀 3
    ❤️ 1
    a
    o
    m
    • 4
    • 10
  • a

    Angel Acosta

    08/30/2022, 8:36 PM
    Hi I was curious if there was anything in the works to kind of nest sub-flows into flows in the flow run ui.. I think if some one has a flow with lots of sub-flows, things get kinda messy pretty quick. Especially if that flow runs multiple times in a short period. In the picture I grouped the flow and sub-flows as an example, along with the normal way flows are shown.
    ✅ 1
    j
    • 2
    • 1
  • p

    Pascal Holy

    08/31/2022, 9:14 AM
    Hey everyone, we are a company in the renewables sector and we rely on environment variables to configure some aspects of our flows. I see we can set a context when we run a flow through the UI but not when setting a schedule. Is there another way or is it still under development?
    ✅ 1
    a
    • 2
    • 3
  • a

    Andy Thurgood

    08/31/2022, 4:52 PM
    Hello, Quick question on the 2.0 UI, specifically the collaborator signup flow. We have added a collaborator, which generates a signup email, but the accept link directs/redirects the user to the login page, with no obvious way to set credentials or accept the signup. The link in the invite points to http://url6711.prefect.io/ls/click with some hefty query params of
    upm
    ,
    data
    ,
    sdata
    and
    reserved.
    Is there a way to debug this at all? Should we be landing at a signup or set credentials type page? Thanks
    ✅ 1
    c
    • 2
    • 3
  • p

    Pablo Beltran

    08/31/2022, 8:24 PM
    Hey there, is there any way to cancel an ongoing flow from the Prefect UI. I see that there is a 'delete flow' but I dont want to delete it just cancel it and any running tasks.
    ✅ 1
    j
    • 2
    • 1
  • y

    Youssef Ben Farhat

    09/01/2022, 3:15 PM
    How can I get this Orion Dashboard? I saw many dashboards on internet and now with the latest version I'm having another dashboard.
    ✅ 1
    j
    t
    • 3
    • 4
  • y

    Yusuf

    09/01/2022, 8:48 PM
    Just wanted to share some honest feedback. I love and support the Prefect project a ton and am super grateful to the community. I just started fiddling with the new Prefect Orion UI. I had been using the Prefect 1 Cloud UI before. I'm a little bit lost on the user experience. I might just be too heavily comparing it to what I was used to seeing before. Here's an example though: When I go to flows, I'm essentially inspecting my data pipelines. When I click on a flow/pipeline, I see the deployments. Which is reasonable. However when I click deployments, I'd expect to see information like a history of runs and states. As well as what used to be the "schematic" tab from the old cloud UI, where I can see the actual flow graph. It just feels like what is essentially your flow/pipline's "homepage" should have more information about the pipeline itself. The visual lineage definition of a flow and run history are the two main things I care about when it comes to an existing pipeline, so its been jarring to have that cut out and put elsewhere.
    e
    t
    • 3
    • 6
  • r

    Rajvir Jhawar

    09/02/2022, 1:34 AM
    Hi Prefect Team, I have encountered a issue with deployments: For some reason parameters are required when you want to update a deployment via the UI. For instance if i want to just change work queue you cannot do that without having default parameters already set. Parameters should in this case always be optional. I only supply parameters via api call hence I don't even need to add default params. You can probably get around this by supplying default values, but that is not an optimal experience. The above issues occur in Prefect 2.3.1 Final issue i had so far when the value of "Debug" is set as false Prefect for some reason leaves the value blank. I haven't re-tested this again in a newer version of prefect (i believe in occurred in 2.1.0) . Just wanted to put that on your radar
  • y

    Yusuf

    09/02/2022, 2:25 AM
    Having a difficult time interpreting how this flow definition turns into this graphic. Can anyone help? How would I make it show that task 2 depends on task 1, and log_task depends on task 1 & task 2, and task 4 depends on log_task? I'm really unclear on what this radar view is trying to tell me
    o
    j
    +7
    • 10
    • 27
  • a

    Adam Brusselback

    09/02/2022, 2:50 PM
    Hey all, I am just trying to run a flow from the UI that works just fine when kicked off from python... I am just using the "Get Github Stars" flow/task from the docs, and wanted to pass in params from the UI. It is not going as expected. When I am trying to pass them in from the UI, I get each character passed to the task instead of each individual string in the parameters. See screenshots in thread. Is this just a formatting issue with the parameters?
    ✅ 1
    j
    j
    • 3
    • 10
  • l

    Lucas Cavalcanti Rodrigues

    09/02/2022, 7:21 PM
    Hey, how can I get a list of all UI registered flow group ids using graphQL? I know I can get the flow ID querying for name using:
    gql(
        '''
            query LatestFlowByName($name: String) {
              flow(
                where: {name: {_eq: $name}},
                order_by: {version: desc},
                limit: 1,
              )
              {
                id
              }
            }
        ''',
    )
    But how can I have all flow ids regardless of the name?
    ✅ 1
    j
    • 2
    • 2
  • a

    Adam Brusselback

    09/02/2022, 8:23 PM
    Believe I found another bug related to the UI: I cannot close out of the flow-run diagram view once it's opened without navigating away to another page and back to the flow run
    e
    • 2
    • 3
  • t

    Tan Li

    09/04/2022, 4:41 PM
    Hi there, I was playing around with the Orion UI, but keep seeing an empty dashboard while I can query something, i.e. actual flow-run records in the DB. Any help on what I could messed up here? Really appreciate it! 🙏
    ✅ 1
    e
    • 2
    • 2
  • a

    Andreas Nord

    09/07/2022, 1:50 PM
    Hi! I try to update a deployments schedule but I keep getting "Failed to update deployment".
    👀 1
    ✅ 1
    b
    • 2
    • 4
  • m

    Marc-Antoine Bélanger

    09/08/2022, 12:41 AM
    I have some issues specifying the parameters on a deployment run. I have defined the following properties:
    {"data": {
                            "title": "data",
                            "default": {},
                            "type": "object",
     },
    "train": {
                            "title": "train",
                            "default": {},
                            "type": "object",
     }
    }
    This is what it is showing in the UI: But the run fails:
    Validation of flow parameters failed with error: ParameterTypeError('Flow run received invalid parameters:\n - data: value is not a valid dict\n - train: value is not a valid dict')
    Traceback (most recent call last):
      File "/usr/local/lib/python3.10/site-packages/prefect/engine.py", line 284, in retrieve_flow_then_begin_flow_run
        parameters = flow.validate_parameters(flow_run.parameters)
      File "/usr/local/lib/python3.10/site-packages/prefect/flows.py", line 269, in validate_parameters
        raise ParameterTypeError.from_validation_error(exc) from None
    prefect.exceptions.ParameterTypeError: Flow run received invalid parameters:
     - data: value is not a valid dict
     - train: value is not a valid dict
    They seem to be interpreted as strings in the Parameters tab:
    {
      "data": "{}",
      "train": "{}"
    }
    Any idea what I am doing wrong?
    r
    • 2
    • 1
  • m

    Marc-Antoine Bélanger

    09/08/2022, 12:43 AM
  • a

    Amelie Froessl

    09/08/2022, 9:58 AM
    hello everyone 🙂 . I am trying to use the parameters to run a Flow from the UI. My config parameters input is a pydantic BaseModel. However I have a more complex data structure in that they types of my elements in the conifg parameters are also pydantic BaseModels (something similar to here ). Or like this:
    class Parameters_1(BaseModel, extra=Extra.forbid):
        a: int
        b: int
        c: str
    
    
    class Parameters_2(BaseModel, extra=Extra.forbid):
        d: int
        e: int
        f: str
    
    
    class Parameters_3(BaseModel, extra=Extra.forbid):
        g: int
        h: int
        i: str
    
    
    class TestConfig(BaseModel, extra=Extra.forbid):
        parameters_1: Parameters_1
        parameters_2: Parameters_2
        parameters_3: Parameters_3
    On the UI it shows up like in the attached screenshot. The input would be a dictionary of the values in the respecitve BaseModel like :
    {"a":1, "b":2, "c": "hello"}
    . ... which is not the nicest I guess. I was wondering if there is a way to nicely have the parameters of a sub BaseModel show up/ editable in the UI? Or if somebody has a suggestion on how this could be solved in a nicer way? Thanks in advance!
    ✅ 1
    a
    • 2
    • 18
  • d

    davzucky

    09/10/2022, 12:42 AM
    Quick comment, It would be nice in the UI of the block detail to add as well the block slug. Making my flow generic to be able to load any filesystem using sluck. However, I had to go on the API to file the block type slug. For example, the block type
    Local File System
    has for slug
    local-file-system
    👍 2
    ✅ 1
    j
    m
    • 3
    • 3
  • t

    Tomás Emilio Silva Ebensperger

    09/12/2022, 8:03 PM
    When I set up parameters for a Deployment in the UI, I specify one of the parameters as type dict, for some reason the UI is saying it is an object and it treats it as a json instead. I can always load the json into a dict in the code, but I was wondering if I am doing something wrong.
    ✅ 1
    m
    • 2
    • 13
  • s

    Steph Clacksman

    09/13/2022, 10:41 AM
    How do I cancel a flow run in prefect 2 cloud? I can see how to delete it, but I want to keep the logs but not have it run any more.
    👍 2
    a
    m
    • 3
    • 8
  • d

    David George

    09/14/2022, 2:14 PM
    I have 7 ghost tasks that are running but with no flows, I can’t stop them and new flow runs are stuck due to us having task concurrency limits. Any ideas how I can go about stopping these task runs. (Prefect 1)
    d
    m
    • 3
    • 10
  • v

    Vaikath Job

    09/17/2022, 2:55 AM
    Is there a way to see what agents are polling a prefect queue for flow runs?
    ✅ 1
    a
    • 2
    • 1
  • v

    vk

    09/19/2022, 8:08 PM
    hi all, migrating to prefect Orion and have some questions/troubles with new ui. something that was good in prefect 1 ui - tasks interface for realtime and the timeline was easy to understand. It was super-clear what task runs when, their duration (longer tasks use more space) etc. how do I "read" the chart in Orion? take a look at the sample, I don't get from it the order of execution and relative time spent. is there a way to see old good chart or understand this? and for some reason it doesn't get updated in realtime the code behind this chart:
    from prefect import flow, task
    from prefect import get_run_logger
    import time
    
    
    @task
    def run_computation():
        logger = get_run_logger()
        <http://logger.info|logger.info>("run some computation")
        time.sleep(30)
        return 123
    
    
    @task
    def dependent_task(x):
        logger = get_run_logger()
        <http://logger.info|logger.info>("run dependent task")
        time.sleep(10)
    
    
    @task
    def independent_task(i):
        logger = get_run_logger()
        <http://logger.info|logger.info>(f"run independent task: {i}")
        time.sleep(3)
    
    
    @flow(name="orion2")
    def f():
        x = run_computation()
        independent_task(0)
        for i in range(1, 10):
            independent_task(i)
        dependent_task(x)
    
    
    if __name__ == '__main__':
        f()
    j
    • 2
    • 6
  • a

    Alix Cook

    09/21/2022, 7:09 PM
    Not sure if this has been reported yet, but in the "block" section of the UI, only up to 200 blocks get shown (and only those blocks are searchable) I did a short search and didn't see anyone else report that but feel free to ignore me if this is known
    👍 2
    :thank-you: 2
    ✅ 2
    j
    • 2
    • 2
Powered by Linen
Title
a

Alix Cook

09/21/2022, 7:09 PM
Not sure if this has been reported yet, but in the "block" section of the UI, only up to 200 blocks get shown (and only those blocks are searchable) I did a short search and didn't see anyone else report that but feel free to ignore me if this is known
👍 2
:thank-you: 2
✅ 2
j

Jeff Hale

09/21/2022, 8:02 PM
Thank you, Alix! I talked to the team and am opening an issue to add scrolling or pagination to the UI.
:thank-you: 1
You can track progress at the issue.
:thank-you: 2
View count: 2