• bitsofinfo

    bitsofinfo

    1 month ago
    looking at the orion helm chart - is there not a way to customize the
    api.ingress
    labels?
    bitsofinfo
    1 replies
    Copy to Clipboard
  • harsh patel

    harsh patel

    1 month ago
    Do we have a roadmap for upcoming features? Was wondering if we are going to have a UI based workflow creation/editor.
    harsh patel
    1 replies
    Copy to Clipboard
  • j

    Jonathan Pou

    1 month ago
    Does cloud.prefect.io happen to be down? For the last 20min, I've been trying to access the main page via browser (Firefox and Chrome) on my computer as well as from my phone (Safari) and I happen to be stuck on the loading screen.
    j
    g
    +3
    5 replies
    Copy to Clipboard
  • n

    Numline1

    1 month ago
    Hey guys, I'm kinda struggling to set up a Prefect on our company server. So it's running and all that, but I had to setup a reverse proxy (Nginx) as well, so that people can access the UI via standard https at 443 and at a company subdomain (internal that is). The UI itself is accessible, but it's trying to make REST API requests to http://127.0.0.1/api/something which is obviously not working. Is there any env setting I can use to make it understand it should use https://prefect.internal.company.com/api instead? I couldn't find anything on that topic anywhere 😞 Thanks!
    n
    Ryan Peden
    9 replies
    Copy to Clipboard
  • k

    Keith Hickey

    4 weeks ago
    BLUF: Am I missing some kind of control flow UI of a DAG, or is Radar the only UI? Will Radar or another UI show dependent-order-of-operations at some point? _*UPDATE: See threaded replies. You can direct control-flow of the UI if you get a
    PrefectFuture
    from a prior task and pass it into next_task.submit(wait_for=future_for_prior_task)*_ Hello. Regarding the Prefect 2.0 Radar View available from the orion server ... it's neat, and provides a unique dimension of data-traceability and drill-down/fan-out ... however I feel like it has over-rotated from the Prefect 1.0 Schematic to strictly visualize explicit point-to-point data propagation between tasks, and loses the logical sequence of task execution that is equally if not more important. Put another way: Radar highlights data flow and hides control flow. I would argue control flow is more important, as you can typically infer how data is flowing by tracing the flow of control, but inferring dependency and temporality from the subset of tasks that explicitly exchange Python data in a tightly-coupled way will lose sight of implicit data dependencies among tasks. It would seem that getting away from DAGs (directed, acyclic graphs) is a driving principle of Prefect 2.0 ("Goodbye, DAGs"). But I still find it immensely helpful to know progress of the flow execution, i.e. where it is in the overall control graph, or where it failed. Part of my context is that many of our DAGs / pipelines / ETLs / data processing is using external data stores (databases, files, lakehouse tables on cloud storage, etc.) with very large datasets being processed. Maybe this is more batch-processing vs. record streaming and the Radar UI is not as well suited for that? But it's not feasible to pull those large datasets into memory of a python process to be handed as python objects from task to task, or serialized from process to process. Therefore with only this implicit data-dependency, and not an explicit code-level one in the flow, no relationships is mapped between those very-much-related Tasks in the Radar UI. So questions: 1. Is there a way to code-in (via the Prefect API for tasks/flows/Futures) implicit data-dependencies from task-to-task? ... So that those tasks are laid out and connected on the Radar UI in expanding concentric circles? a. i.e. Task
    do_a
    pre-processes dataset
    A
    in
    DB_A
    ... Task
    do_b
    transports resulting data in
    DB_A
    to external
    API_B
    ... so
    do_a
    must happen before
    do_b
    and yields control to it only after done. 2. Even if there's not an implicit (or explicit) data-dependency, but just a matter of sequential control flow (e.g. as governed by the
    SequentialTaskRunner
    ) can this be expressed via the Python Task/Flow API?
    When using the SequentialTaskRunner, tasks are laid out on the same Radar ring with no obvious order, even though they clearly are run in-sequential-ordering (not concurrent). 3. I_s there hope for a DAG-like schematic? And if so would that be built further into Radar, or another forthcoming UI?_ 4. There seemed like some API constructs when Tasks returned PrefectFuture (or Task?) objects, which allowed some explicit functions to chain things together. But the recent shift to strip return objects down to their results makes that less possible. Is that Python API and how tasks can be tied together still under design? I saw a few other posts where people may be struggling with the same conceptual issue, linked below. https://prefect-community.slack.com/archives/CL09KU1K7/p1649011482174799 https://prefect-community.slack.com/archives/C0192RWGJQH/p1642702304024400
    k
    Anna Geller
    +2
    10 replies
    Copy to Clipboard
  • Jenny

    Jenny

    3 weeks ago
    Hi @Angel Acosta - thanks for the feedback! I'm going to add your comments to a Github issue we have tracking a similar request. Please feel free to add any more comments in there.
  • p

    Pascal Holy

    3 weeks ago
    Hey everyone, we are a company in the renewables sector and we rely on environment variables to configure some aspects of our flows. I see we can set a context when we run a flow through the UI but not when setting a schedule. Is there another way or is it still under development?
    p
    Anna Geller
    3 replies
    Copy to Clipboard
  • p

    Pablo Beltran

    3 weeks ago
    Hey there, is there any way to cancel an ongoing flow from the Prefect UI. I see that there is a 'delete flow' but I dont want to delete it just cancel it and any running tasks.
    p
    1 replies
    Copy to Clipboard
  • Jenny

    Jenny

    3 weeks ago
    Hi @Youssef Ben Farhat - that's an old version of the 2.0 UI. Is there a particular reason that you're looking for that screen?
    Jenny
    Taylor Curran
    +1
    3 replies
    Copy to Clipboard