Hello Everyone, i am a AWS DevOps engineer and I really liked MLOps things. so I am just starting perfect with one of the clients.

Hi everyone, I’m an analytics engineer and looking for an orchestration solution on AWS for some Python-based dataflows. Prefect seems like a great option for us on paper either with ECS or EKS implementation on our AWS instance, but my sysops team have an uncertainty about whether we can fairly use it without using paid Prefect Cloud services. I understand myself that Prefect is “open-source” and therefore if we host it ourselves we should be able to spin up what we need on the free tier, no? Is it the case that we would have to use one of the pricing plans to automate with Prefect even if we host ourselves?

Hello all, My name is Dru Clark and I am working to modernize a ‘workflow’ that grew out of compromise. The Rolling Deck to Repository program (rvdata.us) is soon to go into a funding renewal time and we are looking for solutions to streamline the way we manage, process and move data. We are in the initial phase of testing prefect (open source) and expect there will be some questions. thanks

Hello everyone, This is Marco, and I work for a data and analytics company in Italy. I'm currently working on incorporating Prefect into company's processes. I wanted to inform you that I have reached out to the sales team to request a demo and get an idea about pricing. I'm looking forward to their response and hope to hear back from them soon.

Howdy everyone, I’m Sofian, I’m working as Site Reliability Engineer in a cloud computing company in France (OVHcloud). We are using Prefect to create business workflows in order to automate several aspects of our automation for Infrastructure Resiliency. From async job to move data from cluster to cluster, to self-repair and replacement of persistent medium of storage (HDD, SSD, NVME, etc.) to simple automation on-demand. We have lot of steps to perform with different internals APIs to build these flows (we called them robots). We started with Prefect 1.0, then migrated to Prefect 2.0 and now running self-hosted Prefect Clusters Service in

2.9.0

version. So far we run more than 100k flows and tasks and we monitor through Grafana our SLOs regarding tasks/flows failures. It used as a massive Scheduler for us and we push to use it even more nowadays for diverse data operations. Some figures as today: • Started to implement Prefect 1.0 in April 2022 • Migrated to Prefect 2.0 in Nov 2022 and did several versions upgrade (~10) up to current version

2.9.0

• Used it for our internals workflows and as backend scheduler • Total Flow runs 105 997

reported by the DB with SELECT count(*) FROM flow_run;

• Total Task runs 272 308

reported by the DB with SELECT count(*) FROM task_run;

• 15 deployment and 10 different work-queue • 4 Clusters • Some Failures 🙂 and even more of Success 👏 We adopted it because it’s Python based and the learning curve is very slight. Also the dynamics around the project are extremely positives. Happy Positive Engineering 🙂 Just wanted to give you a feedback and Thanks for this workflow framework 🤝

Hello All i am Naveen a Graduate Student from Canada and also currently working as an intern in an organization, joined prefect community to learn and explore more regarding the tool. Happy to be here 🙂

Hello. My name is Mike, working in Python Development with large electron microscopy images. We have an existing Prefect 1 workflow that we are migrating to Prefect 2. For the most part it is going well, but we have some sticking points that I will go into detail on the Discourse page.

Hi everyone! 👋 I'm Jan, I'm currently working as a Data Platform Tech Lead for Ki Insurance, Lloyd's of London first algorithmic underwriting syndicate. We have rolled out Prefect 2 as our workflow management tool over the last quarter and so far I'm extremely happy with it, but we aren't using all of it in anger yet, so I might have the odd question or two every so often.

Hello, I’m Jose from Argentina! I hope to get to know better the community. Thanks!

Hello, I am Samyak from India. Experienced in DevOps and buidling communites and organizing events. Happy to join Prefect community.

You can connect me over LinkedIn : https://www.linkedin.com/in/samyyaakk/

Hi, this is Imcom. I am currently using Prefect for job scheduling in k8s. I am wondering where/which channels I could throw questions for discussion and assistance. Thanks in advance and happy to join the community!

Hey everyone👋! I'm Islam, and I just joined the Platform team in Prefect, a brilliant bunch of people from my interactions so far! I'll be working on Prefect Cloud with a focus on ensuring the scalability of the platform to meet the demand of our customers growth 📈. I'm based in San Diego, CA ☀️, and I was previously at Datadog where I was a part of the Azure integrations team azure. Looking forward to meeting more of you and working with you guys!

Hi, I'm Khoa and I'm a junior data engineer. My team is currently using prefect for data automation on our server. I wonder where should I discuss about problems with prefect. Thank you!!!

Hello, I'm Jacob and I'm a data analytics engineer. I'm currently learning about prefect and hopefully use to automate elt workflow. Happy to connect https://www.linkedin.com/in/jacobukokobili/

Hello everyone! My name is Rafael and I'm a data engineer. I'm from Brazil and recently started using Prefect.

Hi all! I'm Joe, I work as research software developer at an ocean observatory 🦀. I've inherited some prefect 1 pipelines what we would like to migrate to prefect 2, so I'll probably have questions about both versions. So far the migration guide and discourse forum have been very helpful!

👋 Hello World, I'm Collin and I've just recently joined the Prefect team as a fullstack engineer. In a past life, I worked at a large org building a (mostly internal, closed-source 🙈) platform for data scientists which included wrapping and managing heavily modified Airflow - if only we'd had what-is-now Prefect... 🙃 . Since then I've been doing more fullstack product dev and am very excited to hop in and work across the stack to help improve any and all things for y'all! Looking forward to being able to answer at least 1 question here sometime soon 😅

Hey folks 👋 I’m Uriel and I just joined the Prefect team prefect logo as a Software Engineer with a focus on open source 📖! The goal is to make sure our open source projects stay in harmony with the work we’re doing internally ✅ AND with the needs of the community here ✅. I’m still getting ramped up but I hope to be able to help here and directly in our git repos soon! A little bit about me — I’m based in Chicago (hope any locals were safe during the recent heat wave 🥵) and in past roles I’ve worked in Infrastructure and Backend product roles. I love soccer ⚽, biking 🚴‍♂️ and being bad at beach volleyball 🏐. See you around!

Hello everyone. I am a data engineer from Japan. My team is using airflow, but I am interested in prefect and I recently built OSS prefect-server on a kubernetes cluster on my home server. However, the documentation seems to be mostly for the prefect cloud, and I don’t really know how to use the OSS one, so I joined here to see if you guys can help me! I’m sure I’ll have a lot of trouble from now on, but thank you in advance for your help.

Hello! I am in the current YC batch and am looking to use Prefect for the first time (most of my experience with ETL jobs was with ScalaSpark for supply chain forecasting at amazon). Thank you in advance for all the help! If there's anyone I can ping with some quick initial questions that would be very helpful 🙂

Hello ! 👋 I'm a data engineer from France, founder of a data-driven media about French Hip-Hop @RapMinerz (https://www.instagram.com/rapminerz/). I'm currently deploying prefect on my GKE, but i'm quite stuck right now... So let's meet in the #prefect-community room 🫠 Thanks in advance

Hello everyone. I’m new here, I’m from Costa Rica, and I’ve been reviewing the possibility of using Prefect for my projects. One of them would be a flow that has the ability to call other flows asynchronously, depending on the revision of some files that are placed in a shared folder, it will execute the flow that is necessary for that file. The main flow must always be monitoring the shared folder. But I haven’t found enough documentation to do this. Can you guide me???

My intention is to also make video tutorials on YouTube in Spanish of projects made in Prefect to promote the product in Spanish-speaking countries.

Hey everyone! 👋 I’m working on a small PoC to see if Prefect is the tool we need 😄 I’ve had troubles with deployment to Prefect Cloud, where is the best place to ask that?

Hey! I’m from Texas, and I just moved to the Philippines! I built a small Generative AI framework around Marvin and Prefect for my tiny awesome consulting practice

Hi everyone! I'm Will. I'm currently searching for an opportunity as a data engineer in the US. I'm looking to network, meet new people and learn new things. Add me on linkedin! https://www.linkedin.com/in/william-chen-5456261a9/

Howdy Prefect team! I'm Gabe, cofounder of Zenfetch (YC W23) and former ML Eng at Stripe (where we used Airflow 😭). At Zenfetch, we're building an activation layer on top of knowledge bases. Think of us as a way to passively recall the information you store with tools like Pocket or Readwise, directly in your workflows. I'm a BIG fan of marvin and prefect, and wanted to give a quick shoutout to some folks who have gone above and beyond to help my team setup our data ingestion pipelines: @Alexander Azzam @Nate @Chris White Feel free to add me on Linkedin!

Hi! I've joined this community before, but this time I'm here as a Product Advocate at Prefect! prefect logo In my last few years as a Data Engineer, I've moved data pipelines from all sorts of job schedulers and orchestrators to Prefect to improve observability and make working with cloud infrastructure less painful. You may have also seen me trying my best to answer questions and be helpful from time to time here on Slack. I'm looking forward to seeing more of everyone soon!

Hi. I'm Rochdi HASSINE, Devops manager in a Business School. Architect in Data for years. Interested by all the subjects around this domain. I have tested recently Airflow but faced a lot of issues. Just discover Prefect and wanted to learn about it. I'm looking forward to seeing more of everyone soon!