• h

    Hugues-Yanis Amanieu

    2 years ago
    Hello the community, trying to get acquainted with Prefect to keep up with my consultants here.
    h
    1 replies
    Copy to Clipboard
  • h

    Hugues-Yanis Amanieu

    2 years ago
    I of course have a question 🙂 . All of my so-called ETLs look the same, but it doesn't match the tempaltes I can find in the doc. • I have a task returning a list of file paths ==> extract • I have a task transforming these files in a list of dict • for each dic I have a task to load I just want to be able to map the transform to each file, then map each returned dic to its load task. However I can't find a way to return only the relevant list to the relevant load task
    h
    Dylan
    9 replies
    Copy to Clipboard
  • m

    Marwan Sarieddine

    2 years ago
    Hi folks - I am facing DiskPressure warnings and dask-worker pods getting evicted when running the flow on EKS
    78s         Warning   Evicted                pod/dask-worker-855c59c78d-qwv5v       The node had condition: [DiskPressure].
    78s         Warning   Evicted                pod/dask-worker-855c59c78d-fvc4q       The node had condition: [DiskPressure].
    I am wondering if someone faced similar issues ?
    m
    Chris White
    3 replies
    Copy to Clipboard
  • Mikael

    Mikael

    2 years ago
    If I am to load 50 tables from 1 db to another. Would you implement that as 1 flow per table or 1 flow mapped over all tables? I guess if I make a loop to register 1 flow per table it would be easier to rerun only one table. And maybe easier with debugging. But it would be more cluttered in the UI.
    Mikael
    j
    3 replies
    Copy to Clipboard
  • m

    Marwan Sarieddine

    2 years ago
    The UI starts crashing with large flows it seems - I wasn't able to mark the flow as failed, and then when I return to the dashboard, I cant access any of the flows - and it shows 0 for flows, and runs, even though I have definitely more than one flow and run ...
    m
    nicholas
    +1
    4 replies
    Copy to Clipboard
  • Adam

    Adam

    2 years ago
    Hello friends, I’m running a
    BigQueryTask
    on Prefect Cloud and it failed with the error “No heartbeat detected from the remote task; marking the run as failed”. Was this Zombie killer? It worked fine locally. How can I ensure this long-running task completes?
    Adam
    nicholas
    8 replies
    Copy to Clipboard
  • bral

    bral

    2 years ago
    Hello! is there an option that allows a flow to execute in only one instance? in my case, the flow has not finished yet, but the scheduler started it again.
    bral
    nicholas
    4 replies
    Copy to Clipboard
  • Will Milner

    Will Milner

    2 years ago
    Hey all. I recently upgraded to version 13 of prefect. Did the config change to set the graphql endpoint? this is what I have in my config.toml
    [server]
      
         [server.ui]
               graphql_url = "<http://10.0.0.300:4200/graphql>"
    But when I connect the UI still attempts to connect to
    localhost:4200/graphql
    Will Milner
    nicholas
    2 replies
    Copy to Clipboard
  • r

    Riley Hun

    2 years ago
    Hello friends - I just wanted to take the time to thank all of you for all the help these past few days. Was able to finish my first flow ingesting external data into our internal systems, and it works beautifully. Looking forward to sharing the framework with my colleagues!
  • bral

    bral

    2 years ago
    Hi again 😛arty-parrot: is there something as condition operator ? If previous task was succeeded continue , else interrupt. Do i need to use Result handler ?
    bral
    nicholas
    4 replies
    Copy to Clipboard