• Raviraja Ganta

    Raviraja Ganta

    1 year ago
    Hello Prefect !! I am Ravi, and I am working as a NLP Engineer at Enterpret. Currently looking to scale the data processing pipelines and have a way to monitor them. I am looking into prefect for the workflow automation. Looking forward to learn and contribute to the community 😃
    Raviraja Ganta
    Chris Reuter
    +3
    7 replies
    Copy to Clipboard
  • Richard Pelgrim

    Richard Pelgrim

    1 year ago
    Hey everyone! I'm Richard, a Data Science Evangelist intern over at Coiled.io. Currently exploring all the ways our product integrates with the wider PyData ecosystem and naturally arrived at Prefect. Looking forward to learning more about the product and community here!
    Richard Pelgrim
    Chris Reuter
    +4
    7 replies
    Copy to Clipboard
  • Mathias Herberts

    Mathias Herberts

    1 year ago
    Hi, I'm Mathias, co-founder and CTO of SenX, maker of Warp 10 (https://warp10.io/), a Geo Time Series platform. We have used various tools for task automation along the years, Luigi then Airflow then Prefect.
    Mathias Herberts
    Noah Holm
    +5
    7 replies
    Copy to Clipboard
  • Cory Jez

    Cory Jez

    1 year ago
    Hi all 👋, I'm Cory from Austin FC 🤓, new to slack but I've been using prefect since April or so... hopefully i can help folks as much as asking annoying questions... I'm using prefect with ❄️ snowflake to load data about soccer games around the world 🌍 daily and run a lot of our internal processes that drive a web app for coaches, scouts, and managers!
    Cory Jez
    Jenny
    +4
    7 replies
    Copy to Clipboard
  • Wieger Opmeer

    Wieger Opmeer

    1 year ago
    Hi all, I'm currently trying to figure out if we should stick with Airflow or byte the bullet and start using something else.
    Wieger Opmeer
    m
    +2
    4 replies
    Copy to Clipboard
  • Prateek Gupta

    Prateek Gupta

    1 year ago
    Hi Everyone. I have been asked to deploy ML models using Prefect.
    Prateek Gupta
    Zach Angell
    +3
    5 replies
    Copy to Clipboard
  • Andrew Maguire

    Andrew Maguire

    1 year ago
    Hello - I'm the Analytics & ML lead at Netdata Cloud and looking at prefect cloud as a potential alternative to some flavor of Airflow (self managed, composer or ideally Astronomer). I'm having a little trouble understanding what exactly agents are or what i would need - kinda feels like agents are almost like airflow workers but not quite. If i wanted to use prefect cloud to manage the orchestration of BigQuery jobs and GCP cloud function based pipelines. It seems like maybe i would still need to bring my own agents with me - is that correct? If so do you have any typical examples of how people might do this? From what i have read it looks like maybe i'd ideally have my own dask or kubernetes cluster to have lots of agents available to prefect cloud? Just wondering how to think of scaling this - how would i know how many agents i would need? Or i may be getting things wrong in my understanding perhaps? p.s. apologies if this not a good place for this question. Oh and hello again 🙂
    Andrew Maguire
    Kyle McChesney
    +3
    8 replies
    Copy to Clipboard
  • Joseph Harrison

    Joseph Harrison

    1 year ago
    Hi, I'm trying to figure out if I can use Prefect for an ETL-type scenario. Basically, we'll be receiving large text files via SFTP/HTTPS and we need to process these files eventually into a data warehouse. The Prefect documentation is a bit confusing to me as it mostly focuses on Cloud. I am primarily interested in an on-premise deployment. Would appreciate if anyone can point me to resources that could help me answer these questions:1. Is Prefect Core = Server + UI? 2. Is there a deployment guide for on-premise that doesn't require Docker/Compose? Thanks!
    Joseph Harrison
    Kevin Kho
    +1
    8 replies
    Copy to Clipboard
  • Lucas Beck

    Lucas Beck

    1 year ago
    Hi all, We are currently experimenting with prefect as to replace our own orchestration tool used for scaling complex compute tasks. 👋
    Lucas Beck
    Dylan
    +1
    3 replies
    Copy to Clipboard
  • b

    Baum

    1 year ago
    Hi all, I’m here to migrate from airflow to prefect because of the latency. I have some questions here. I have 2 layers for the process.1. data processing 2. modeling I have many data processed every minute and also have many models processed every minute. A model have dependencies to some data. i.e) data_A - model_A data_B - data_A - model_B data_C - I tried to make each data processing into a flow -> N flows and also each modeling process into a flow -> M flows Then I had a parent flow setting dependencies between data flows and model flows. However this had long latency (I suspect poll_interval) So I made a huge flow which contains all data processing and modeling process. this made the flow much faster but hard to monitor with the UI. Do you think there are some way to achieve this goal?