👋 Hello everyone! I am a Data Engineer at Flex a fintech company in New York. Lurking here to learn more about Prefect!

👋 Hi! I’m running a nascent fund automatically trading market-moving news. Want to learn about using Prefect+more to run all our ETL jobs (sync price data, news feeds, retrain ML models). A bit more at predmachine.com. Thanks for a making this community great!

Cheers; I work at a fintech company, and we’re using Prefect for our precompute/research platform. Youtube videos have been useful, thanks for that!

Bonjour! Just checking out Prefect and couldn’t resist the temptation to join another Slack workspace. 😉 Looks like a nice community!

G'day to all. I work at a green energy startup looking at using Prefect to schedule ETL jobs for IoT devices. Been easy to get started with Prefect, I just have some questions about best practices for deployment 🚀

Hello folks, Guillaume from Bandai Namco Mobile here. Trying out Prefect as an alternative solution to Airflow for our data pipelines needs, thanks for creating this community!

Hi all, Gaylord working for Xero in New Zealand. Considering Prefect as an orchestration tool and I wanted to know if this Slack is also a good place to ask questions around Prefect Cloud?

Matt Alhonte again, reintroducing myself a 3rd (4th?) time now that I'm on a new Prefect job as a Machine Learning Engineer with SymphonyRM! 🙂

Hi, I’m Henning. I’m a free-range data clown based in Norway. Trying out Prefect in order to collect scattered lambda functions (across two clouds) into something more sane. By the way, what is the group name for lambdas? It’s a murder of crows, a gaggle og geese, a skulk of foxes, a … of lambdas?

Hi, I'm Donny working and I am a Customer Data Architect at Census. I have been playing around with Prefect and am a former head of data (engineering, science, analysis) at two early-stage companies. I hope to contribute to the community around this awesome product!

👋 - Hi everyone. Ben here from the Great Expectations team. I’ve been hearing so many good things about the Prefect community for so long that I finally figured I should join to check it out! Awesome work. I’m way impressed with what ya’all have accomplished so far. Everyone I talk to that’s used it loves Prefect. Doing something right here! 🎉

Hello folks 👋 , Abhishek here, currently based in India. I am a Python developer and currently working on few data engineering tools and Infra. Apart from my work I also organize community events across India which includes local Python meetups and PyCon India conference. Glad to be a part of this vibrant community! Looking forward to contribute 🙂 Prefect is perfect 🙂

Hello Prefect World! Great to be here. I'm John, currently working on some data pipelines for Altium.com and Octopart.com (PCB design software and electronic components search, respectively). After spending a couple of months with Airflow I tried out Prefect, and rebuilt 2 months of work in a couple of days! Ecstatic 🙂 ❤️ super impressed, and love the community so far.

Hello Prefect! I'm Bryan, VP of Architecture at an Ed Tech company specializing in the data/backend side of things. Evaluating Dagster and Prefect for our ETL needs. Happy to be here!

Hello, I'm here exploring my options other than Airflow. 😄

Hello Prefect! , Im Hari, exploring argo, prefect & temporal. we have used airflow before, but started evaluating these to look for something beyond just scheduling.

Hi. I'm Janne, Head of Data Engineering at CollectiveCrunch. I'm looking into Prefect to run our petabyte -scale geospatial data processing pipelines.

Howdy everyone! I'm Trip, and I work as Healthcare Data Consultant in Virginia. I have about 15 years experience in Healthcare, IT, and Data. Currently I am evaluating a number of solutions that pertain to data workload orchestration. I am curious about prefect, but I will admit I am having a bit of trouble parsing the seemingly nuance differences between Flow Storage and the concept of version control of the Flow .py files themselves. Do we use a traditional git repo with cicd workflows providing change control of the actual code? I know I can store the flows in a number of different ways, but I just can't seem to get it to click as to how a team might collaboratively manage updates to those flows over time.

Hello Prefect !! I am Ravi, and I am working as a NLP Engineer at Enterpret. Currently looking to scale the data processing pipelines and have a way to monitor them. I am looking into prefect for the workflow automation. Looking forward to learn and contribute to the community 😃

Hey everyone! I'm Richard, a Data Science Evangelist intern over at Coiled.io. Currently exploring all the ways our product integrates with the wider PyData ecosystem and naturally arrived at Prefect. Looking forward to learning more about the product and community here!

Hi, I'm Mathias, co-founder and CTO of SenX, maker of Warp 10 (https://warp10.io/), a Geo Time Series platform. We have used various tools for task automation along the years, Luigi then Airflow then Prefect.

Hi all 👋, I'm Cory from Austin FC ⚽🤓, new to slack but I've been using prefect since April or so... hopefully i can help folks as much as asking annoying questions... I'm using prefect with ❄️ snowflake to load data about soccer games around the world 🌍 daily and run a lot of our internal processes that drive a web app for coaches, scouts, and managers!

Hi all, I'm currently trying to figure out if we should stick with Airflow or byte the bullet and start using something else.

Hi Everyone. I have been asked to deploy ML models using Prefect.

Hello - I'm the Analytics & ML lead at Netdata Cloud and looking at prefect cloud as a potential alternative to some flavor of Airflow (self managed, composer or ideally Astronomer). I'm having a little trouble understanding what exactly agents are or what i would need - kinda feels like agents are almost like airflow workers but not quite. If i wanted to use prefect cloud to manage the orchestration of BigQuery jobs and GCP cloud function based pipelines. It seems like maybe i would still need to bring my own agents with me - is that correct? If so do you have any typical examples of how people might do this? From what i have read it looks like maybe i'd ideally have my own dask or kubernetes cluster to have lots of agents available to prefect cloud? Just wondering how to think of scaling this - how would i know how many agents i would need? Or i may be getting things wrong in my understanding perhaps? p.s. apologies if this not a good place for this question. Oh and hello again 🙂

Hi, I'm trying to figure out if I can use Prefect for an ETL-type scenario. Basically, we'll be receiving large text files via SFTP/HTTPS and we need to process these files eventually into a data warehouse. The Prefect documentation is a bit confusing to me as it mostly focuses on Cloud. I am primarily interested in an on-premise deployment. Would appreciate if anyone can point me to resources that could help me answer these questions: 1. Is Prefect Core = Server + UI? 2. Is there a deployment guide for on-premise that doesn't require Docker/Compose? Thanks!

Hi all, We are currently experimenting with prefect as to replace our own orchestration tool used for scaling complex compute tasks. 👋

Hi all, I’m here to migrate from airflow to prefect because of the latency. I have some questions here. I have 2 layers for the process. 1. data processing 2. modeling I have many data processed every minute and also have many models processed every minute. A model have dependencies to some data. i.e) data_A - model_A data_B - data_A - model_B data_C - I tried to make each data processing into a flow -> N flows and also each modeling process into a flow -> M flows Then I had a parent flow setting dependencies between data flows and model flows. However this had long latency (I suspect poll_interval) So I made a huge flow which contains all data processing and modeling process. this made the flow much faster but hard to monitor with the UI. Do you think there are some way to achieve this goal?

Hi there, my name is Bob and I'm currently looking into prefect as orchestration engine for a new data platform I'm setting up at a client. I have a lot of experience with Airflow, so I'm curious to see how Prefect does the job 😉

Hi folks, my name is Dominic and I'm a budding data engineer trying to implement a data pipeline ecosystem at my company for webscraped and collected data. I'm an experienced Alteryx user trying test out more open source python based solutions, and the many pain points I've experienced even trying to start Airflow has led me to try out Prefect instead.