Channels
announcements
ask-marvin
best-practices-coordination-plane
data-ecosystem
data-tricks-and-tips
events
find-a-prefect-job
geo-australia
geo-bay-area
geo-berlin
geo-boston
geo-chicago
geo-colorado
geo-dc
geo-israel
geo-japan
geo-london
geo-nyc
geo-seattle
geo-texas
gratitude
introductions
marvin-in-the-wild
prefect-ai
prefect-aws
prefect-azure
prefect-cloud
prefect-community
prefect-contributors
prefect-dbt
prefect-docker
prefect-gcp
prefect-getting-started
prefect-integrations
prefect-kubernetes
prefect-recipes
prefect-server
prefect-ui
random
show-us-what-you-got
introductions
- j
Jonas Sølvsteen
01/27/2021, 7:04 PMHej Prefect people! I am Jonas, head of technology in a satellite remote sensing company in Denmark 🇩🇰 🛰️ dhi-gras.com. We do data flows of terabytes of satellite imagery and machine learning to help understand and preserve the environment all over the world 🌱🦧🌍. Checking out Prefect as replacement for our own K8s DAG compute engine to join an awesome community, learn, and contribute. 🤝🌍 5🤝 3👋 7d- 2
- 1
- b
Brandon
01/30/2021, 7:41 PMDobrý den, I’m Brandon! I’m based out of Cleveland, Ohio and am leading the Data Engineering & Analytics efforts at startup named Deck of Dice Gaming. I use Prefect to facilitate my workflows with Dask & ECS+Fargate on my quest to maximize player experience! If you’re interested in mobile games, and want to try it out -> search “Shake It Up - Dice Poker” on whatever AppStore you use and let me know your feedback!🎲 4👋 6nd- 3
- 3
- s
Samuel Hinton
02/02/2021, 1:52 PMGday folks, I’m an Aussie datascientist. Did my PhD in Astrophysics, moved into renewable energy, and now in a data engineering project where Im trying to assess the suitability of Prefect (over airflow2.0) for scheduled data ingestion!👋 6jd- 3
- 2
- s
Scott Jones
02/02/2021, 6:30 PMHi everybody! I’m a consultant in the Boston area, investigating if Prefect can be used effectively for workflows that have Julia as well as Python steps.👋 3jd- 3
- 3
- d
David Clements
02/05/2021, 1:02 AMHey everyone I am the VP of Eng at a Healthcare startup - doing a backoff between prefect and dagster this week with one of our other engineers. Salesforce Data -> Hit an API -> S3 -> Snowflake Load from Stage ( ideally with tests ) !!!👋 7cb+3- 6
- 5
- p
Praveen
02/06/2021, 5:18 PMHey there!👋 5 - s
Sam Johnson
02/10/2021, 3:51 PMHi everyone! I'm a Senior Engineering Manager at a healthcare company - Rally Health. We use Airflow today for data science and engineering data flows. Just here to learn more about prefect for now👋 6mdd- 4
- 3
- j
JAYRON SOARES
02/11/2021, 12:00 AMHi everyone, I'm a data engineer at GAYA Analytics, and can say that Prefect is simple to use and quite effective. Congratulations. Cheers from Brazil👋 5md- 3
- 2
- v
vish
02/11/2021, 3:44 PMHello everyone, I'm an Engineer at Hackpartners, working on a data science project for Network Rail UK! We have been using Dask (+in-house pipelining framework) extensively for data pipelining. We started using prefect core locally few months ago. ❤️ Love it and looking forward to transforming our infrastructure with Prefect! 🇬🇧🙂 2👋 5jd- 3
- 2
- p
Paul Hudson
02/12/2021, 11:00 AMHey folks, I’m a DataOps guy, currently open sourcing skippr.io. Skippr automates schema discovery and evolution, making it easy to declaratively solve the EL in ELT. For example you might load raw json into a Parquet datalake for your perfect.io workflows to operate on. Looking forward to getting Skippr open sourced and exploring what Skippr features, interfaces and metadata might be useful to Perfect users 😊👋 6nmd- 4
- 5
- c
Craig Wright
02/12/2021, 5:18 PMHowdy! I manage Partner Engineering and Developer Relations at Fivetran. If you don't know Fivetran, we do automated data loading as a service. On the DevRel side of the house, we are working on a Task for Prefect that allows you to manage Fivetran sync jobs from your Prefect flows. Expect more on this very soon. Very happy to be joining this community!👋 7:upvote: 2jnd- 4
- 3
- s
Stephen
02/12/2021, 7:14 PMHey! guess I should say hello 😆 I'm Steve, I work as a data engineer Shell Energy based in the UK. Came across Prefect whilst evaluating orchestration tools. Did my own PoC and what can I say.. loved it 😛 Looking forward to building cool stuff 🥶🚀 6👋 7nd- 3
- 2
- j
Josselin Girault
02/15/2021, 1:27 PMHello there! I'm Josselin, Data Engineer at QuickSign from France. Built some internal tools to run small scale ETLs locally but it got pretty messy with extra scripts. Then started to play around with Prefect, and got a few questions about how to make it scale without breaking our whole infrastructure 😅👋 6d- 2
- 1
- z
Zuhair Ikram
02/17/2021, 3:05 AMHi All! I’m Zuhair, a Director of Data & ML at Nurx (SF Bay Area). We currently use Airflow and were thinking of making the switch to Dagster or Prefect👋 6cd- 3
- 2
- m
Max Wood
02/17/2021, 5:47 PMHi! I'm the founder of Deck, a DC-based company that helps progressive campaigns reach the right voters. We're kicking the tires on Prefect as a replacement for our own DAG orchestrator, which we built as an alternative to Airflow. So far we're really excited about it!👋 7mj+3- 6
- 5
- j
Justin Chavez
02/18/2021, 4:14 PMHello everyone! I am a Full Stack Engineer at Reverie Labs where we are applying machine learning to drug discovery. Currently moving some of our Python scripts to Prefect flows. DC-based as well👋 6nd+2- 5
- 4
- v
Victor
02/18/2021, 9:02 PMHi everyone! I'm Victor, I'm the technical lead of a product that its a PIM (Product Information management) for ecommerce and furthermore its an engine of operations for big companies. I'm very excited with the tool👋 6d- 2
- 1
- t
Thanneermalai Lakshmanan
02/22/2021, 10:30 AMHi Everyone , This is Thanneer, CTO at Analytic Edge . We build AutoML on top of marketing data . We are planning to use Prefect for building our data pipeline / data flow . We plan to run it on Dask cluster as the same will be used for running our modelling pipeline as well👋 5jd- 3
- 2
- t
Thomas Browne
02/22/2021, 3:04 PMHello everyone I'm Tom and I'm building a workflow tool in Prefect that will have DAGs that also have human interaction nodes in them so it's quite a fun project.👋 5jd- 3
- 5
- t
Thomas Browne
02/22/2021, 3:05 PMEvaluating Prefect vs Airflow for this kind of requirement. Client is already invested in Airflow so we have to identify the bits of Prefect that are "game changing" and whether or not it's worth introducing a second data workflow stack. - a
Ananthapadmanabhan P
02/23/2021, 5:21 AMHey everyone! I’m Ananthu, Team lead - Data Science at the industrial analytics firm - VIMANA. We have been using an in-house, tailor-made, R based framework to deploy our production workload for years and are slowly migrating to industry standard tools wherever it is relevant. Trying out Prefect and here to learn more👋 4nd- 3
- 2
- v
Vincent Chéry
02/23/2021, 2:18 PMHi all! 👋 I am Vincent, data scientist currently working for the french government on a cool project related to fishing 🎣. I'm deploying prefect server in production, very excited to get it up and running !👋 3j- 2
- 1
- c
Clément Chastagnol
02/23/2021, 2:45 PMHi there! I’m a senior data scientist working in Paris, France for a big health data company. I’m testing Prefect as a way to better automate internal workflows. Currently using it to publish open data on Covid19 antigen tests, with more to come! 🙂👋 6dm- 3
- 3
- a
Amy Chen
02/23/2021, 8:06 PMHello folks! I’m a Solutions Architect at Fishtown Analytics (makers of dbt). I’m interested in learning more about how people are using Perfect, specifically with dbt. Nice to meet you!🎉 6👋 6dj+2- 5
- 6
- m
Moritz Rocholl
03/01/2021, 4:27 PMHey there! Im a DevOps engineer at Kiteswarms in Germany. Currently testing Prefect for our internal Data Analysis Workflows. Nice to meet you 🙂👋 6d- 2
- 1
- s
Srikrishnan Subramanian
03/03/2021, 4:55 AMHey everyone! I’m Srikrishnan, working in the data science team at VIMANA, an industrial analytics firm. Trying out prefect to convert our internal data science workflows to industrial workflow paradigms, here to learn more! great to meet you all!👋 3d- 2
- 1
- u
Udbhav Agrawal
03/03/2021, 5:35 AMHello everyone! I’m Udbhav, Data Engineer at VIMANA. We are trying out Prefect for building our data pipeline and data flow. Looking forward to learn some cool stuff. Nice to meet you all! 🙂👋 5nd- 3
- 2
- s
Sopha de Lopez
03/03/2021, 2:38 PMHello everyone Eager to test Prefect for our Datalab in France. Great to join this active community👋 6d- 2
- 1
- j
Johan Lundin
03/05/2021, 6:13 AMHi everyone, just joining since we are using prefect at work👋 6nd- 3
- 2
- r
Robin Papa
03/05/2021, 11:37 AMHi all, I'm Robin, an operations manager in the seafood logistics business. I was actually looking for a business process engine framework, but Prefect looks so easy to work with. So I'm approaching Prefect from a business process viewpoint. I hope I'm able to: • use Prefect as a workflow engine where I can retrieve by API what the next task in the flow is • integrate conditional business logic to define what the next task in the flow should be • step by step automate each task (which is more likely Prefect's use case) So, I'm trying to build the backend to make our business futureproof, even though now Prefect does not match the current use case of business processes flows that well. It's used for data flows right? But everything is data nowadays 😉. Curious to hear your ideas about this! I'll do some more experiments.👋 4and- 4
- 4
Title
r
Robin Papa
03/05/2021, 11:37 AMHi all, I'm Robin, an operations manager in the seafood logistics business. I was actually looking for a business process engine framework, but Prefect looks so easy to work with. So I'm approaching Prefect from a business process viewpoint.
I hope I'm able to:
• use Prefect as a workflow engine where I can retrieve by API what the next task in the flow is
• integrate conditional business logic to define what the next task in the flow should be
• step by step automate each task (which is more likely Prefect's use case)
So, I'm trying to build the backend to make our business futureproof, even though now Prefect does not match the current use case of business processes flows that well. It's used for data flows right? But everything is data nowadays 😉. Curious to hear your ideas about this! I'll do some more experiments.
👋 4
a
Amanda Wee
03/05/2021, 12:10 PMWelcome! For the conditional logic, you could look into the
caseswitchifelse:upvote: 2
n
nicholas
03/05/2021, 1:47 PMHi @Robin Papa - welcome to Prefect! It sounds like Prefect will likely work for your use case but let us know over in #prefect-community if you run into trouble 🙂
r
Robin Papa
03/05/2021, 2:02 PMThanks so much Amanda and Nicholas!
d
Dylan
03/05/2021, 4:32 PMWelcome Robin!
View count: 4