https://prefect.io logo
Docs
Join the conversationJoin Slack
Channels
announcements
ask-marvin
best-practices-coordination-plane
data-ecosystem
data-tricks-and-tips
events
find-a-prefect-job
geo-australia
geo-bay-area
geo-berlin
geo-boston
geo-chicago
geo-colorado
geo-dc
geo-israel
geo-japan
geo-london
geo-nyc
geo-seattle
geo-texas
gratitude
introductions
marvin-in-the-wild
prefect-ai
prefect-aws
prefect-azure
prefect-cloud
prefect-community
prefect-contributors
prefect-dbt
prefect-docker
prefect-gcp
prefect-getting-started
prefect-integrations
prefect-kubernetes
prefect-recipes
prefect-server
prefect-ui
random
show-us-what-you-got
Powered by Linen
introductions
  • m

    Mike Pinna

    09/18/2021, 12:55 PM
    Hi all, I’m a feverishly self-taught Python coder about a year in who stumbled upon Prefect about a week ago and I’m very much inspired by the design and functionality of the codebase. Prefect Core feels like it should be part of the standard Python library. It’s elegant, intuitive, and sets the engineering bar in my mind. Also, the care that goes into their documentation is impressive. Even if my code is working, I’ll see an occasional warning about using pandas in an expensive way or something along those lines. That is a feature I really don’t see too often that shows great care and thoughtfulness. This is the kind of code I want to be able to write someday. It’s obvious Prefect is the new standard, it’s just a matter of time before it’s universally recognized as such. This is a revolution in workflow management that I want to be a part of.
    🙌 8
    🙏 13
    👋 12
    j
    k
    +3
    • 6
    • 5
  • m

    Mike Pinna

    09/20/2021, 5:55 PM
    Wow! Thank you all for such a warm welcome! I've decided to apply to work with your company. It's quite obvious that there is magic happening here and you are onto something big. I hope one day we'll be working together but if not I'm happy to part of this community building a better future together. I know I'll learn a lot from you, thanks again!
    :thank-you: 7
    :marvin: 12
  • c

    Chas

    09/23/2021, 3:15 PM
    Hey! My name is Chas and I’m a data engineer at Storyblocks in Washington, DC. We currently use Airflow but I can’t help looking for an actually intuitive solution to scheduling and job dependencies (and execution times vs. schedule times vs. run times 🙄 ). Excited to try Prefect out!
    👋 14
    k
    d
    +4
    • 7
    • 14
  • c

    christopher chalcraft

    09/23/2021, 5:30 PM
    Hello all. My name is Chris and I work at Databricks with a focus on workflow automation. I will be building out a project that uses perfect, dbt, databricks and mlflow. Just getting started here but definitely reach out if you have any suggestions or example repos on using Databricks Tasks or if you are interested in a colab!
    👋 18
    d
    j
    +6
    • 9
    • 8
  • s

    Sandip Viradiya

    09/23/2021, 11:54 PM
    Hello all. My name is Sandip and I work at Lumi Finance in Sydney, Australia. We currently use Databricks and Mode but we are looking for some orchestration layer which can help us to monitor everything at one place. Trying to connect Fivetra, Databricks, Dbt and Looker, all this will be monitored by Prefect (our new DWH). Facing some issue which I will post in respective channels. I know I'll learn a lot from everyone, thanks again!
    👋 18
    n
    h
    +5
    • 8
    • 7
  • s

    Sridhar

    09/28/2021, 10:09 AM
    Hi all, I work for a FinTech and sustainability Start up based in Sydney. I'm currently working on automating the current manual workflow using perfect cloud and keeping my infra on aws. I've hit a few roadblocks which I've posted on community channel. Looking forward to learn and contribute to prefect community. Thanks for the help in advance!!
    👋 15
    j
    c
    +6
    • 9
    • 9
  • w

    Will

    09/29/2021, 10:26 AM
    Hey all, I work for a construction / architecture technology business based in London / NYC. Currently evaluating Prefect as a way to make our our data platform manageable and maintainable, with the aim to run it on ECS Fargate. Thanks in advance for any questions I might have!
    👋 18
    z
    d
    +5
    • 8
    • 7
  • s

    Sanjog Sharma

    10/01/2021, 9:55 PM
    Hi Folks, My name is Sanjog and I work at Google. I just discovered prefect and I am interested in learning more about prefect and running it in GCP infra. Has anyone use Prefect in GKE autopilot or cloud run(?), I'm interested in learning more.
    👋 13
    d
    a
    +3
    • 6
    • 14
  • a

    Andreas Tsangarides

    10/04/2021, 7:50 AM
    Hi there! I work for an Energy Storage startup in the UK. Setting up data pipelines for ingesting data from the national grid, trading platforms, etc. Using AWS for our data infrastructure so hopefully will learn a lot from the community here and will be able to contribute to prefect tasks for things like updating Glue table partitions, running crawlers, running glue jobs (they are simple boto3 wrapper functions in my repo for now!)
    👋 16
    n
    c
    +5
    • 8
    • 7
  • g

    grandimk

    10/05/2021, 2:29 PM
    Ciao! (“Hello!” in Italian) My name is Marco and I’m a data engineer for an EdTech company based in Switzerland/San Francisco. We’re using Prefect, running on AWS, for our data pipelines and I really like it! I’m joining the community to learn more about Prefect and, hopefully, to give my small contribution.
    🇨🇭 1
    :fondue: 1
    👋 24
    k
    d
    +6
    • 9
    • 8
  • r

    Russell Brooks

    10/05/2021, 5:59 PM
    Ok, so Orion is next up...Some of Prefect’s negative/positive engineering ands the concept of supervisor reminds me of what I liked about the erlang programming language and it’s philosophy to just fail and to supervise. At any rate, I’ve been using it for around 2 months now and find it relatively straight forward. I’d like to see more worked examples and variety of examples. The docs are well written but need more examples. Also I’m having a nightmare to get it off my localhost and “deployed” not clear how and where I need to set everything up, e.g. runconfig vs storage vs server ui and so on. How many moving parts to get a hello world deployed remotely but within my institution’s architecture?
    k
    • 2
    • 8
  • s

    Stefano Cascavilla

    10/06/2021, 1:09 PM
    Hi everybody! 😄 I'm Stefano and I'm a Junior Data Engineer that recently joined the crazy data engineering world! 🙂 I'm joining the community to keep in touch with latest Prefect's features, and to learn more about this cool tool!
    👋 19
    d
    c
    +8
    • 11
    • 10
  • a

    Aldo M

    10/07/2021, 8:35 AM
    Hi everyone! Aldo from France, I currently work for a scale up as a Data engineer. We wanted to use Airflow for our workflow, until we hear about Prefect. Excited to test it for our productions pipelines.
    👏 2
    👋 21
    :party-parrot: 3
    a
    c
    +8
    • 11
    • 10
  • b

    Brian Sorahan

    10/07/2021, 7:46 PM
    Hi y’all! I’m just getting started with prefect. Looking forward to exploring it more
    👋 20
    n
    z
    +6
    • 9
    • 9
  • s

    Sylvain Hazard

    10/08/2021, 8:52 AM
    Hey there ! Cruising from France where I work as a Data Engineer for a startup. I'm currently in the process of migrating from Airflow to Prefect and might ask for help in here. Thanks everyone for the work you put in this project, it's crazy good !
    :party-parrot: 3
    👋 25
    a
    c
    +7
    • 10
    • 10
  • t

    Thomas Furmston

    10/08/2021, 8:58 AM
    Hello, Currently scoping out using prefect for our orchestration. Thanks for the great project. 🙂
    👋 26
    a
    c
    +9
    • 12
    • 11
  • i

    Imen Chaieb

    10/09/2021, 9:35 AM
    Hi, I'm Imen, Junior Data Engineer from Tunisia working for a PropTech startup based in HKG. We've switched from Airflow to Prefect and I heard that the slack community is very active and responsive, so here I am to keep myself in the loop 😄
    👋 24
    c
    a
    +7
    • 10
    • 9
  • t

    Tom Varga

    10/13/2021, 9:45 PM
    Hi everyone. I've only just discovered Prefect. It looks pretty promising. Before I commit a large amount of time to learn everything about it, I'd like to ask a quick question. I've been using pydoit for years and really like it's ability to use file md5 signatures to know if a task is up-to-date and for build-avoidance. Can the same kind of capability be developed with Prefect?
    👋 16
    s
    d
    +4
    • 7
    • 6
  • s

    Shawn Jung

    10/14/2021, 12:09 AM
    Hi team, I just happened to visit the Prefect booth at KubeCon21 and discovered this great product. Thank you Kevin, Darren, and Tyler for the nice convo we had. I hope to learn more from this community and automate more of my data workflows!\
    👋 25
    j
    d
    +8
    • 11
    • 10
  • e

    ek

    10/15/2021, 9:02 PM
    Hi everyone, I got introduced to Prefect as ETL tool sometime last year. It was setup on just one big server to do massive jobs/tasks and I was quite impressed on how easy it was to get ETL workflows up and running. Fast forward, we are now running all of our workflows on Prefect+k8s! I’m looking forward to learn more and connect with all of you. Lastly, thanks to the amazing team at Prefect…you guys rock!!!
    :highfive: 6
    👋 26
    m
    c
    +10
    • 13
    • 12
  • e

    Eric Blom (he/him)

    10/20/2021, 2:55 PM
    Hi all, I work with a lot of scrapers as part of an ETL pipeline. We'd like to make them more robust and easier to diagnose when they have problems. As part of that we are planning to move them to Prefect
    👋 19
    h
    k
    +9
    • 12
    • 11
  • p

    Prasanth Kothuri

    10/21/2021, 5:54 PM
    Hello, We are looking to use Prefect to run our data pipelines for genomic surveillance (e.g covid variants) at wellcome sanger institute ! I setup a local instance, agents and test flows, still a lot to figure out !!
    🙌 1
    🧬 2
    :marvin: 8
    👋 19
    d
    k
    +8
    • 11
    • 10
  • k

    Kelvin Malyar

    10/27/2021, 10:10 PM
    Hi All, I'm Kelvin, I work as a software engineer over in the US . I'm hoping to use Prefect and am trying to prototype a few pipelines. I like the concept a lot!
    👋 15
    k
    m
    +7
    • 10
    • 13
  • j

    Jean-Michel Provencher

    10/28/2021, 3:55 PM
    Hi, I’m Jean-Michel and I work as a Software Engineer in Canada. I’m currently implementing Prefect in our Data Platform as a POC and it’s currently working like a charm! Awesome product
    👋 17
    m
    k
    +9
    • 12
    • 12
  • s

    Simon Gillett

    11/08/2021, 12:49 AM
    👋 I’m Simon Gillett, I own a couple of Amazon businesses, am interested in data / B2B EDI. What’d I miss?
    👋 15
    g
    a
    +5
    • 8
    • 7
  • t

    Tomek Florek

    11/09/2021, 3:51 PM
    Hey All! 👋 I'm Tomek , recent graduate from @Daniel Molnar’s Pipeline Academy where I've learned about Prefect and in my first month as Data Engineer at an EV charging platform plugsurfing.com. Liked it so much, that we're planning to use Prefect+dbt+g_e as the core for our stack and I'm looking for helpful materials/ stories to build upon, please share if you have some 🙂
    👋 19
    🙇 1
    d
    k
    +7
    • 10
    • 9
  • m

    Miguel Sanz

    11/09/2021, 4:49 PM
    Hi there! I am @Tomek Florek’s colleague at plugsurfing.com Looking forward to start this prefect journey!
    👋 19
    e
    k
    +8
    • 11
    • 10
  • i

    Ibtissem Hattab

    11/11/2021, 2:05 PM
    Hi All , I"m Ibtissem and I work as a DevOps Engineer at Instadeep I'm Excited to test Prefect for our productions pipelines
    👋 19
    a
    c
    +6
    • 9
    • 8
  • d

    Damien Dee Coureau

    11/13/2021, 10:18 AM
    Hello, I'm Damien (he/him) but you can call me Dee. I'm Head of Production Technology at Ubisoft Film & TV. Among other things, I'm helpings our partner animation studios designing and implementing CG pipelines & workflows. I think I'm falling in love with Orion and I hope I'll get a chance to leverage it in our upcoming productions :)
    🎥 1
    👋 15
    a
    j
    +6
    • 9
    • 10
  • j

    John

    11/13/2021, 4:48 PM
    Hi, I'm John - VP of Basketball Analytics for the Minnesota Timberwolves. I'm interested in evaluating Prefect for DAG orchestration in order to get better visibility into our data and model pipelines.
    👋 17
    🏀 2
    j
    m
    +9
    • 12
    • 12
Powered by Linen
Title
j

John

11/13/2021, 4:48 PM
Hi, I'm John - VP of Basketball Analytics for the Minnesota Timberwolves. I'm interested in evaluating Prefect for DAG orchestration in order to get better visibility into our data and model pipelines.
👋 17
🏀 2
j

Jeremiah

11/13/2021, 4:50 PM
Welcome @John! Glad to have you here and let us know how we can help!
j

John

11/13/2021, 4:51 PM
Thanks @Jeremiah
👍 1
m

Michael Adkins

11/13/2021, 4:52 PM
Hi John! I'm based in Minneapolis 🙂
h

Hiro Rodriguez

11/13/2021, 4:53 PM
Hi @John 👋🏽 Welcome to Prefect!
z

Zach Angell

11/13/2021, 4:55 PM
Welcome to Prefect @John! 👋
k

Kevin Kho

11/13/2021, 5:09 PM
Hey @John, that’s a cool job! Welcome to Prefect!
t

Tyler Wanner

11/13/2021, 5:14 PM
Hiya John! Welcome to the Prefect community
j

Jenny

11/13/2021, 5:24 PM
Welcome @John!
g

George Coyne

11/13/2021, 5:53 PM
Welcome!
j

Jake Kaplan

11/13/2021, 7:02 PM
Welcome @John !
a

Anna Geller

11/14/2021, 10:46 AM
Welcome @John!
d

Darren

11/16/2021, 5:20 PM
Welcome @John!
View count: 4