https://prefect.io logo
Join the conversationJoin Slack
Channels
announcements
ask-marvin
best-practices-coordination-plane
data-ecosystem
data-tricks-and-tips
events
find-a-prefect-job
geo-australia
geo-bay-area
geo-berlin
geo-boston
geo-chicago
geo-colorado
geo-dc
geo-israel
geo-japan
geo-london
geo-nyc
geo-seattle
geo-texas
gratitude
introductions
marvin-in-the-wild
prefect-ai
prefect-aws
prefect-azure
prefect-cloud
prefect-community
prefect-contributors
prefect-dbt
prefect-docker
prefect-gcp
prefect-getting-started
prefect-integrations
prefect-kubernetes
prefect-recipes
prefect-server
prefect-ui
random
show-us-what-you-got
Powered by Linen
introductions
  • j

    Jorge Severino

    01/05/2022, 1:24 AM
    Hello Prefect Community! I'm Jorge from Chile, and I'm leading a digital transformation in an investment company. We're learning Prefect in order to take control of our new bots force! I'm really happy to be here! ;)
    👋 14
    n
    k
    +7
    • 10
    • 9
  • r

    Roel Schenk

    01/05/2022, 4:38 PM
    Hi Everyone, I'm Roel from the Netherlands and team lead for Solution Consulting withing Minit (Process Mining Vendor), We are very interested in Perfect. And will be investigating if we can use it to orchestrate our Python+DBT pipelines for customer engagement in our team. Any help to get started quickly with some tests in a docker environment will be greatly appreciated. Nice to meet all of you!
    👋 13
    k
    a
    +5
    • 8
    • 8
  • t

    Tony Liberato

    01/07/2022, 3:33 PM
    Good morning, Everyone. My name is Tony and I am the DBA for a non-profit Consumer Advocacy organization. Our goal for the next year is to mature our data flow environment. Prefect seems to meet all of our needs. So at the start of 2022, I have been on-boarding our proving-group platform to pilot prefect as our work-flow orchestrator. Wish me luck. One challenge I am already struggling with is we are a windows shop. I get the feeling that almost all of prefect deployment scenarios are linux based.
    👋 12
    a
    k
    +6
    • 9
    • 9
  • h

    Heeje Cho

    01/08/2022, 9:54 PM
    Hey all, Heeje here. I'm a data engineer at Illumina and my group and I will be evaluating Prefect over the next few weeks so I'm sure I'll have a lot of questions. I come from an Airflow background but I've been hearing a lot about Prefect Core and Orion so i'm excited to explore!
    👋 15
    g
    k
    +7
    • 10
    • 12
  • a

    Allison Shaw

    01/10/2022, 5:56 PM
    Hey everyone 👋 I'm Allison. Love seeing everyone's background and how Prefect is being used. Excited to be in this community!
    👋 17
    k
    n
    +9
    • 12
    • 11
  • i

    Ian Andres Etnyre Mercader

    01/10/2022, 7:38 PM
    Hi everyone, I'm a developer working with @Daniel Kornhauser on web app for skimming life sciences papers quickly for Diligence and R&D.
    👋 17
    k
    a
    +8
    • 11
    • 10
  • a

    Aaron Wormus

    01/15/2022, 3:37 PM
    Hi! 👋 I'm the CTO at SmartX Advisory Solutions - and am evaluating different tools for our data pipelines. Happy to be part of the community.
    👋 12
    k
    n
    +6
    • 9
    • 8
  • c

    Clément Peyrard

    01/17/2022, 8:59 AM
    Hi everyone, I'm Clément, Data Scientist at EarthDaily Agro / GEOSYS (Toulouse, France), and I've been playing with Prefect to see how it could enable the team to deploy python analytics (more) rapidly and interact with our APIs and internal satellite data processing pipelines.
    👋 13
    n
    a
    +7
    • 10
    • 9
  • n

    Nash Taylor

    01/17/2022, 8:52 PM
    Hey everyone, my name is Nash, and I'm a Machine Learning Engineer at Paystone in Vancouver, Canada. I've been keeping an eye on Prefect since about 2018 and have been really impressed by the progress and rapid growth over the years. I'm here partly for selfish reasons -- I've got a problem that I'm about 30% sure Prefect might be able to solve, and I'm trying to understand it better to evaluate that -- and partly because I just wanted to see how things were going here and see if I could catch up on the latest advancements. I've been looking for an excuse to incorporate Prefect into my stack for, I suppose about 4 years now, and I may have one coming up. Hope everyone is happily eliminating their negative engineering! (Is that still the motto around here?)
    👋 14
    a
    k
    +5
    • 8
    • 7
  • a

    Aaron Pickering

    01/18/2022, 11:40 AM
    Hi all, My name is Aaron, I'm the lead data scientist at Fresenius Data Solutions, and as of recently heading the data operation at Seenly.io, a new marketing startup. At the startup, I have the option to choose the tech, so I made the switch to Prefect from Airflow. So far I'm loving the platform, everything is so intuitive and its certainly making my life easier. Looking forward to contributing to the community!
    👋 18
    c
    a
    +7
    • 10
    • 9
  • l

    Luis Aguirre

    01/18/2022, 11:57 PM
    Hi everyone. I'm a developer working on Bumblebee, a no-code data transformation tool. We're currently exploring and trying out Prefect Orion for the data processing part 🙂
    👋 20
    n
    c
    +9
    • 12
    • 11
  • p

    Patrick Wyatt

    01/20/2022, 7:21 AM
    Hello, I'm Patrick Wyatt, an online game developer in the Seattle area, and co-founder of One More Game. I work on data analytics in my "spare time" when I'm not doing all the myriad tasks related to developing & publishing a game. We're pre-launch now so only have a few tens of users playing our game. Our data volume is quite low -- hundreds of records per day, not millions ... yet. I've really enjoyed building a pipeline to export JSON data from our game & backend servers -> AWS Kinesis Firehose -> S3 data lake, and then using Prefect to manage flows that ingest and process the data in Firehose.
    👋 18
    k
    c
    +7
    • 10
    • 13
  • m

    Maximilian Lutz

    01/25/2022, 8:11 AM
    Hi everyone! I am Max, Data Specialist at the german solar power startup Eigensonne ☀️ I will be rebuilding our data pipeline and look forward to learn a lot about prefect here 🙂
    ☀️ 5
    👋 16
    a
    c
    +9
    • 12
    • 11
  • d

    Domenico Di Gangi

    01/30/2022, 4:05 PM
    Hi Everyone, I am Domenico from Italy, currently a graduate research fellow on machine learning at the Italian National Council of Research (CNR) and a part time consultant for a company providing firewall and network monitoring services. I am starting to use prefect in both my research and to orchestrate data pipelines within my consultant role. I am very glad to be here!
    ✅ 1
    👋 10
    l
    a
    +4
    • 7
    • 6
  • k

    Keurcien Luu

    01/31/2022, 2:42 PM
    Hi! I'm Keurcien, I am the head of data at Choose, a French e-commerce company. We're looking for solutions to simplify our data workflows and make them more robust and maintainable. I tried Prefect to replace our old CLI that was designed to copy parquet files and update our Hive Metastore accordingly, and I liked the product, the API, the coding experience and the UI. Looks very promising so I'd like to give it a full try. Thanks for making this!
    👋 14
    :party-parrot: 1
    g
    j
    +6
    • 9
    • 8
  • d

    Dev Dasgupta

    01/31/2022, 5:52 PM
    Hi Everyone, I'm Dev and I am a Sr Data Engineer in a Healthcare company based out of New York. I recently joined my company and they are using Prefect for orchestrating their data pipeline. I am new to prefect (ex Airflow-er) and I'm looking forward to learning more about Prefect.
    👋 12
    k
    d
    +6
    • 9
    • 9
  • f

    frojo

    02/02/2022, 11:24 AM
    Hi everyone!👋 My name is Felix, I'm a data analyst at INVAP SE in Córdoba, Argentina. We've been using Prefect for a couple of months and we're really happy! From satellite images to raw telemetry processing, Prefect is handling all the flows flawlessly🙌.
    👋 11
    a
    c
    +7
    • 10
    • 10
  • c

    Caleb Ejakait

    02/03/2022, 2:53 PM
    Hi, Caleb here, a Data Engineer working at startup based in Nairobi, Kenya. I have used prefect before but starting to get my teeth sunk in trying to adopt it for the company I work for. Admittedly, I have been a lurker on this channel but looking forward to more interactions with the awesome community around Prefect.
    👋 13
    k
    a
    +5
    • 8
    • 7
  • m

    Moises Vera

    02/03/2022, 11:10 PM
    Hello prefect guys!! I'm Moises engineer in Mexico, trying to figure out some specific behaviours of prefect xD I'm automating some ETL processes and really new in data engineering
    👋 9
    k
    j
    +4
    • 7
    • 8
  • e

    Evan Curtin

    02/04/2022, 2:21 PM
    Hey there again! I’ve been in here from time to time but made a new account since I just started a new gig at Relativity as a data scientist!
    👋 11
    c
    k
    +4
    • 7
    • 10
  • a

    Areg Melik-Adamyan

    02/04/2022, 3:14 PM
    Hello! I'm Areg Melik-Adamyan, Principal Engineer and architect at Intel. With whom I can discuss some of the issues of prefect usage in internal project?
    👋 13
    a
    j
    +2
    • 5
    • 6
  • n

    Niels Prins

    02/07/2022, 4:28 PM
    Hi, I'm Niels, data engineer. I work on different projects, ranging from data pipe-lines to full blown data driven apps. All with the aim to give data insights / insights in data. In my projects I work together with engineering teams, data scientists and analysts. I just landed a job where we use prefect for multiple data flows in combination with Jenkins and K8s. I hope to learn from this community, especially about CI/CD in combination with Prefect =D
    👋 16
    k
    d
    +9
    • 12
    • 11
  • f

    Francois van Heerden

    02/09/2022, 8:32 PM
    Hi everyone. My name is Francois and I'm a code system developer at Necsa in South Africa. We are considering Prefect to use as an automation tool for our simulation tool chain. Since we already moved to Python as a unified front end, implementing complex flows in Prefect has been straight forward. Very impressed with the API and UI so far
    👋 14
    k
    j
    +3
    • 6
    • 5
  • d

    Dennis Hinnenkamp

    02/10/2022, 1:42 PM
    Hi! 👋 I'm the Technical Lead and Co-Founder at siːkwəl - Analytics Engineering - a Northern Germany based Data & Analytics Consulting Agency and I evaluate and test different tools for our customers data pipelines. Happy to be part of the community and to support the community productively 😃
    👋 14
    k
    l
    +7
    • 10
    • 9
  • r

    Roger Bosch Mateo

    02/16/2022, 12:11 PM
    Hi! I'm the Data Engineer at Indoorclima. We are currently evaluating options for building our ETL and prefect seems amazing! We do have some questions that couldn't find an answer in the documentation: • Will prefect orion be compatible with flows previously created? • While prefect seems amazing, prefect orion is even better with the possibility of true dynamic flows with "if", "for" and "while". We have seen that it is expected an a stable release in early 2022 and as of now it is not recommended to deploy orion in production. As we want to start developing ASAP is there a more specific ETA for the stable release? • Are there any plans of supporting other databases like MySQL? Thank you all!
    👋 11
    a
    j
    +4
    • 7
    • 8
  • j

    Joshua Greenhalgh

    02/16/2022, 2:36 PM
    Hi, My names Josh - I lurked around here and asked a few questions towards the start of 2021 in a previous job - started at a new place and I am taking on the role of lead Data Engineer - Would love to understand peoples experiences with using Vertex on GCP is this a robust way of using Prefect? In the past tried to run everything myself on k8s but was far to much pain than it was worth!
    👋 11
    j
    k
    +3
    • 6
    • 7
  • d

    Dan Morris

    02/17/2022, 3:04 PM
    Hello, I am one of the Data Engineers at North of England Commissioning Support, an NHS organisation
    👋 12
    🇬🇧 1
    a
    c
    +8
    • 11
    • 11
  • h

    Hui Huang

    02/17/2022, 7:25 PM
    Hi Prefect Builders & Community!~~ Hui here. I am a virtual CTO & founder of TheSparkOne Network. Prefect system and tools are designed & implemented in a modern way. I love it. ❤️👏 Played with it in the morning… got several hello-flow examples successfully. Love to co-learn and co-build data economy together. 😃
    👋 14
    a
    c
    +4
    • 7
    • 6
  • m

    Matthias

    02/21/2022, 10:42 AM
    Re-introducing myself as I switched company (and no longer have access to my previous company email). Been working in the field of ML for 5 years now and following this community for the past year. Where I currently work, we are heavily invested in k8s. So for us, it made sense to chose Argo as our orchestration framework. I must admit that I like it for several reasons: • it is k8s native, so you can treat flows like any other deployment every task runs in a separate container so you can easily isolate tasks that require strict security requirements However, there are several drawbacks too. Most notably: • because every task run in a separate container, your tasks are less atomic not to waste too much time on pod creation etc. • Since it is k8s, it is really hard for devs to use it. Hence the responsibility falls into the hands of the data platform team (who manages the k8s cluster), which is the wrong thing to do imo. For these reasons, I would like to introduce Prefect as an alternative to Argo. Therefore, I really look forward to a production-ready release of Orion! 💪
    👋 14
    👍 2
    j
    a
    +6
    • 9
    • 32
  • r

    Raimundo Pereira De Souza Neto

    02/21/2022, 8:02 PM
    Hello, I'm software developer from Brazil (Manaus, the heart of the Amazon forest), and I loved Prefect 😛refect: .
    👋 13
    d
    j
    +6
    • 9
    • 8
Powered by Linen
Title
r

Raimundo Pereira De Souza Neto

02/21/2022, 8:02 PM
Hello, I'm software developer from Brazil (Manaus, the heart of the Amazon forest), and I loved Prefect 😛refect: .
👋 13
d

Dylan

02/21/2022, 8:03 PM
Welcome! 😄
j

Jeremiah

02/21/2022, 8:04 PM
Hello @Raimundo Pereira De Souza Neto!
a

Anna Geller

02/21/2022, 8:06 PM
Welcome to the community @Raimundo Pereira De Souza Neto! 👋
j

Jenny

02/21/2022, 8:47 PM
Welcome to Prefect! 👋
l

Laurie Hindes

02/21/2022, 9:58 PM
Nice to have you here, @Raimundo Pereira De Souza Neto!
c

Chris Reuter

02/21/2022, 10:47 PM
Welcome @Raimundo Pereira De Souza Neto!
j

justabill

02/22/2022, 2:05 PM
Hey there, @Raimundo Pereira De Souza Neto, welcome!
r

Richard Christie

02/22/2022, 2:13 PM
Welcome
View count: 4