https://prefect.io logo
Docs
Join the conversationJoin Slack
Channels
announcements
ask-marvin
best-practices-coordination-plane
data-ecosystem
data-tricks-and-tips
events
find-a-prefect-job
geo-australia
geo-bay-area
geo-berlin
geo-boston
geo-chicago
geo-colorado
geo-dc
geo-israel
geo-japan
geo-london
geo-nyc
geo-seattle
geo-texas
gratitude
introductions
marvin-in-the-wild
prefect-ai
prefect-aws
prefect-azure
prefect-cloud
prefect-community
prefect-contributors
prefect-dbt
prefect-docker
prefect-gcp
prefect-getting-started
prefect-integrations
prefect-kubernetes
prefect-recipes
prefect-server
prefect-ui
random
show-us-what-you-got
Powered by Linen
introductions
  • j

    Jason Graham

    06/07/2022, 5:53 PM
    Hi everyone! I am a Data Architect at Align AI looking into orchestration solutions.
    👋 22
    d
    a
    +10
    12 replies · 13 participants
  • t

    tc_prefect

    06/07/2022, 8:23 PM
    Hi Team 🙂 Just wanted to introduce myself. My name is Taylor and I am a new Sales Engineer at Prefect. I like reading.. 📚 I’m currently in the middle of The Changing World Order and Fall of Giants (when I need a fiction break). I always love getting book recommendations, so send any my way! I am looking forward to learning so much from this community and helping to answer as many questions as I can. :cool-llama:
    👋 25
    :marvin: 4
    j
    j
    +7
    10 replies · 10 participants
  • m

    Massimiliano Fanciulli

    06/12/2022, 10:19 AM
    Hello! My name is Massimiliano Fanciulli and i'm working on building a Prefect + Dask cluster for data processing.
    👋 21
    :marvin: 1
    a
    e
    +8
    11 replies · 11 participants
  • o

    Onkar Ravgan

    06/15/2022, 12:44 PM
    Hello Everyone, My name is Onkar and I’m working as a Software Engineer for OpenMetadata. At OpenMetadata we are implementing a metadata management solution that includes data discovery, governance, data quality, observability, and people collaboration. We are also partnering with Prefect to make OpenMetadata work with the Prefects workflows! Nice to meet everyone.
    🎉 1
    ❤️ 2
    🙌 1
    👋 20
    🎯 1
    a
    k
    +10
    16 replies · 13 participants
  • m

    Matteo Fiorillo

    06/15/2022, 2:24 PM
    Hello everyone, I am Matteo, a data engineer at Laserhub. I decided that prefect will be our data orchestration tool! Nice to meet you!
    :marvin: 4
    👋 19
    k
    d
    +11
    13 replies · 14 participants
  • j

    Jeff Hale

    06/15/2022, 9:43 PM
    Hi all! 👋 Jeff Hale here. I’m excited to be starting as a Developer Advocate at Prefect! I’ll be creating resources to help people learn about Prefect.😛refect-duck:
    :marvin: 11
    👋 20
    🚀 9
    :wizard2: 6
    ❤️ 9
    k
    a
    +7
    9 replies · 10 participants
  • r

    Rohan Chutke

    06/15/2022, 10:31 PM
    Hi All, Rohan here. Data Engineer at Memories Group Limited. I am currently in the process of migrating from prefect server to cloud.
    👋 17
    j
    c
    +9
    12 replies · 12 participants
  • m

    Matt Conger

    06/16/2022, 7:16 PM
    Hi Everyone! My name is Matt Conger and I am exciting to be joining the Prefect team for the summer as a Prefect Community Engineering Intern. Looking forward to getting to learn from and work with the community. 😛refect-duck:
    👋 22
    :wizard2: 1
    ❤️ 3
    :cool-llama: 1
    j
    k
    +6
    9 replies · 9 participants
  • m

    Michael Ng

    06/17/2022, 1:04 PM
    Hi All. My name is Michael Ng. I'm the Director of Data Science at Intrinsic Brands. We're an early stage startup standing up our data stack, and we are starting to experiment with Prefect to orchestrate our flows.
    👋 20
    c
    a
    +12
    14 replies · 15 participants
  • l

    Luis Ovando

    06/17/2022, 7:09 PM
    Howdy, My name is Luis, I’m the VP of Engineering at Tani Salud, an healthtech early startup in Mexico, A few colleges recommends me this tool, and we are triying to orchestrate our data flows
    👋 21
    k
    k
    +11
    14 replies · 14 participants
  • a

    Amit Gupta

    06/18/2022, 8:12 PM
    Hi All, My name is Amit Gupta. I’m cofounder and CTO of TrulyMadly, Leading dating app in India. I found about Perfect while doing research about automating ETL and Data science workflows. It’s quite intriguing. Hoping to migrate our ETL and data science workflow soon to Perfect.
    👋 17
    :marvin: 5
    k
    j
    +8
    10 replies · 11 participants
  • s

    Surawut Jirasaktavee

    06/19/2022, 11:52 AM
    Hi Everyone, My name is Surawut. I’m a Data engineer. I just joined the
    course-mlops-zoomcamp
    and learn about Prefect. hope to use Prefect in the future ML pipeline’s corp.
    👋 17
    a
    j
    +7
    9 replies · 10 participants
  • d

    Dominik Wagner

    06/22/2022, 10:38 AM
    Hey there! 👋 I’m Dominik, the sole Data Engineer for German non-profit organisation
    betterplace
    . I’m in the process of building a data stack that primarily revolves around ELT workflows on GCP with airbyte, dbt and BigQuery. Prefect looks like a nice solution to simply (and cheaply) schedule dbt jobs with the possibility to easily scale out to more complex workflows in the future. An initial question: If I want to deploy prefect(open source) myself on GCP, would the recommendation be to do so all in a kubernetes cluster, or something else? 🤔 🙏
    👋 16
    j
    j
    +9
    11 replies · 12 participants
  • i

    Iulia Danilov

    06/22/2022, 7:25 PM
    Hello! I'm Iulia ( iulia not Lulia xD), and I am an aspiring Java software developer. I have got 3 years of hands-on experience with it, along with some SQL, Spring Boot, Maven, CircleCI, and many others! I love tinkering with both new and old projects. Currently in the process of finding a part-time job or a full-time one for my coop winter term.
    👋 18
    j
    j
    +7
    9 replies · 10 participants
  • a

    Abin Antony

    06/22/2022, 9:40 PM
    Hi All, This is Abin, I am developer from Canada, working for public sector project. New to PREFECT, trying to set up PREFECT on private OpenShift - due to privacy/FOIPPA . Quick Question : if any one has set up PREFECT on Openshift using yaml templates please let me know, it will be great help
    👋 17
    j
    k
    +5
    7 replies · 8 participants
  • r

    redsquare

    06/23/2022, 7:29 AM
    Hey - Steve from sunny Manchester, UK here - I am looking to replace our current airflow implementation with Prefect
    👋 18
    k
    h
    +7
    9 replies · 10 participants
  • z

    Zheng Cheng

    06/23/2022, 1:33 PM
    Hi Everyone, Zheng Cheng from China here, our use case for Prefect is complex data analysis, at the beginning we are using ClickHouse and Superset for data visualization, Our main app is Ruby on Rails, database is PostgreSQL. ClickHouse just copy everything in PostgreSQL. And Superset query the ClickHouse to show graph. As more question asked, our SQL in Superset (aka Virtual Dataset) grow longer and longer, to a point that difficult to manage and reason about. so we use “dbt” to break these ~100 line SQL into multiple “dbt models”. But just SQL is still not enough, We need to write Python to do some “data transformation” work. We want the trigger Python when things happen, so the data visualization on Superset is “near realtime” We need to use Python to read data from ClickHouse, and write something back to ClickHouse. We don’t want to use Flask or Django for that. (negative engineering) After some investigation, we found that Airflow and Luigi is “first generation” tools, they are way too complex. We don’t care about DAG. Dagster is kinda early, Prefect seem nice. that’s why I am here. 🙂 Update: I need to make Python code can be trigger by a HTTP Request, Not sure how to use Prefect to do that
    👋 17
    k
    j
    +5
    8 replies · 8 participants
  • w

    William Clark

    06/23/2022, 4:07 PM
    Hello Everyone! Will from North Carolina here, Prefect is wonderful and so is this community! I have a background in Atmospheric Science and all things "Ops". My team has been using Prefect 1.0 for the 6 months to scale out a deep learning factory within the population health space.
    👋 21
    k
    j
    +4
    6 replies · 7 participants
  • r

    Rodrigo Mont’Alegre

    06/27/2022, 7:06 PM
    Hi, Rodrigo here. I'm currently evaluating whether to use airflow or prefect to orchestrate data pipelines.
    👋 17
    j
    c
    +8
    10 replies · 11 participants
  • t

    Thinam Tamang

    06/28/2022, 2:53 AM
    Hello everyone 👋, I am Thinam Tamang. I am an NLP Developer and I have been learning Prefect 2.0. It's great to connect with you all here.
    👋 20
    n
    j
    +7
    10 replies · 10 participants
  • f

    Fabricio Braz

    06/28/2022, 9:28 AM
    Hello folks! Hello Folks! I'm a researcher, professor, and ML Lead at AI Lab @ Universidade de Brasilia/Brazil. We are reviewing our MLOps stack, and there is a high chance that Prefect is gona replace airflow. It's encouraging to found a very active community. I'm Looking forward to sharing experiences! Best wishes!
    👋 23
    c
    j
    +9
    12 replies · 12 participants
  • d

    Dev Dabke

    06/28/2022, 4:47 PM
    Hi y’all! I’m a (almost finished?) phd student in applied math at princeton 🐯 in the beautiful, industrious state of new jersey. Besides enjoying the best pizza and bagels you can find, I do a lot of data analysis and machine learning work that involves quickly spinning up experiments and firing them off onto various computing clusters. I’m looking to incorporate prefect into my workflow so I don’t have to worry about orchestrating the etl parts of my math research! shoutout to @Nate for the help so far and pointing out this slack!
    🎉 14
    👋 18
    b
    a
    +6
    11 replies · 9 participants
  • b

    BarnBuilder

    06/30/2022, 1:12 AM
    Hello all. I'm a retired engineer turned programming junkie. Read an article on Medium.io today about Prefect and I'm hooked. I've banged my head against the wall for months trying to figure out how to implement AF as a one man band. Took me about 10 minutes to realize the beauty of Prefect.
    👋 18
    💙 3
    ❤️ 5
    j
    e
    +7
    9 replies · 10 participants
  • m

    Marcin Grzybowski

    06/30/2022, 12:19 PM
    Hi all, I have just demonstrated results of my PoC to my team. I have got green light to replace our Talend jobs (they are docker containers) with Prefect2.0 code. So at the beginning we will just replace Talend's containers with Prefect containers, so it's transparent for other parts of our system. After that we will see if / how we can use other Prefect features, like Clound and kubernetes deployment. Thank you all for your help and have a nice day
    🙌 10
    🎉 11
    a
    c
    +7
    9 replies · 10 participants
  • e

    Emil Ordoñez

    06/30/2022, 2:58 PM
    Hello Everyone, My name is Emil Ordoñez. I work as a Data Engineer at Intrinsic Brands. We are currently starting to implement Prefect as our orchestration tool/platform. We're going to be using it to orchestrate a set of tools containing, but not limited to: • Fivetran ingestions • Integrator.io flows • DBT jobs • ML models built both on R and Python
    👍 10
    👋 21
    j
    m
    +8
    12 replies · 11 participants
  • p

    Paul Lucas

    07/04/2022, 10:19 AM
    Hi All I’m a senior DE and am currently in the process of creating a POC with Prefect as I start to look for an orchestration/pipeline framework and have come across Prefect having used Airflow in previous roles. Excited by the initial exploration with it.
    👋 23
    a
    j
    +11
    13 replies · 14 participants
  • v

    Viet Nguyen

    07/06/2022, 2:17 AM
    Hi all, I'm Viet Nguyen, I'm a software engineer for University of Tasmania, Australia (Research division). My team is Integrated Marine Observing System and we're enabled by the Australian Government. We're looking for comprehensive solutions and tech stacks that can help our pipelines of transforming ocean data catalogues into cloud-optimised format. And Prefect looks like an ideal solution, Looking forward to sharing experience. Have a nice day. Cheers.
    👋 19
    k
    c
    +8
    10 replies · 11 participants
  • a

    Andy Hudson

    07/07/2022, 12:45 PM
    Hi all, I'm Andy Hudson. I'm Solution Architect at Vizolution Ltd in Swansea, UK. Starting to investigate Prefect as an alternative to Airflow. Looking forward to learning what it can do.
    👋 19
    a
    r
    +11
    13 replies · 14 participants
  • t

    tallkien

    07/08/2022, 4:31 AM
    Hey there, I'm Lyn and I'm soon to start work as a pipeline architect at a Vfx firm. I'm very impressed with the philosophy of Prefect 2.0 and I'm evaluating the possibility of using it to manage day to day business workflows in conjunction with Kafka.
    👋 15
    a
    j
    +2
    5 replies · 5 participants
  • p

    Pierre-Edouard

    07/08/2022, 7:39 AM
    Hello All, I'm Pierre-Edouard Caron, a french devops engineer working at OVHCloud on the email security team (spam/virus filtering). We really love prefect for data manipulation 🙂 I hope to learn a lot with this community !
    👋 17
    a
    j
    +8
    11 replies · 11 participants
Powered by Linen
Title
p

Pierre-Edouard

07/08/2022, 7:39 AM
Hello All, I'm Pierre-Edouard Caron, a french devops engineer working at OVHCloud on the email security team (spam/virus filtering). We really love prefect for data manipulation 🙂 I hope to learn a lot with this community !
👋 17
a

Anna Geller

07/08/2022, 7:41 AM
Welcome to the community!
🙂 1
j

Jeff Hale

07/08/2022, 8:23 AM
Welcome @Pierre-Edouard! 👋
c

Chris Reuter

07/08/2022, 9:22 AM
There is a growing French contingent here at Prefect - welcome @Pierre-Edouard!
j

jawnsy

07/08/2022, 1:50 PM
Welcome to the community @Pierre-Edouard! 👋
l

Laurie Hindes

07/08/2022, 1:55 PM
Welcome, @Pierre-Edouard! let us know how we can help as you explore Prefect!
k

Kevin Kho

07/08/2022, 2:30 PM
Hi @Pierre-Edouard! Welcome to Prefect
b

Brian Russell

07/08/2022, 3:31 PM
Welcome @Pierre Monico!
m

Matt Conger

07/08/2022, 4:09 PM
Welcome to the Prefect community! @Pierre Monico
p

Pierre Monico

07/08/2022, 4:51 PM
Guys, it's @Pierre-Edouard, not me @Pierre Monico 😂
😅 3
🇫🇷
j

justabill

07/08/2022, 5:16 PM
👋 Glad you're here @Pierre-Edouard
View count: 7