Hi All, I'm working with Prefect for a US government client and see many opportunities for the application within government. Next step is to set it up on Redhat servers... I'll head over to the server channel to ask how

Hello All. Our company is just starting testing Prefect but I though it might be a good idea to join already, since I've heard there are plenty of helpful people here 🙂

Hi, I am an astronomer who is building data management software for a camera on a large telescope (toltec.astro.umass.edu). I just started looking into prefect 2 (orion) and it seems a great choice to replace our existing bare bone celery tasks.

hi folks! i’m tao, a software engineer at intruder. we’re looking at prefect to set up our ETL pipelines, with a vision for bigger orchestration flows in the future. great to be here! 👋

Hi all, I am Yvan, I am discovering Prefect. We are looking for an orchestrator for an invoicing solution developped in Python and Prefect looks promising.

Hi all. My name is John and I am a senior technical architect at VIA. Currently, I'm working on two projects that I believe will benefit from an orchestration tool. I am familiar with Airflow and found myself agreeing with a lot that was written in the Why not airflow post. So far, I've deployed Prefect into our test K8s cluster using the helm chart and pulling in a test flow from Github using a private access token. I have a couple of questions and would like to test some of the approaches I've taken thus far. Which #channel is best to ask questions in?

Hi! I'm Iván, a senior Data Scientist & MLOps discovering the wonders of data engineer tools. I've discovered Prefect as an alternative to Airflow and kubeflow and it looks so much neater and usable, I'll definitely give it a try. However I have a very specific question and I couldn't find the answer anywhere, so I joined this slack with the hopes of asking and see if this is the tool to go for my needs. Thank you everyone =)

Hi , I am Andrei , a dev in python and php . I want to use Perfect to execute remote a python script. I have joined the slack channel to get access to support.

Hi all, I'm Patrick - working as a DevOps Engineer in Austria. Started with Prefect topics a few weeks ago, currently deploying Prefect-Server instances at AKS and Rancher for my company and running my very first workflows. Looking forward to get in touch with you! :)

Hey 👋 I'm Gustavo, working as a Data Engineer in Austria for a few years now, originally from Peru 🙂 Prefect has piqued my interest as I currently use Azure Data Factory for orchestration which is not the nicest thing in the world. Have a nice weekend!

Hello all, I'm Jonas from Bern, Switzerland. I spend a lot of time looking for a tool to do some specific ETL tasks and am very happy I finally found prefect which makes my project very easy. Since I just started to dive into prefect I'm tring to start with the 2.0 beta

Hello everyone. I'm David from Mississauga, Canada. I am using Prefect to automate a number of data capture processes and will then expand to automate the generation of derivative data products. Although I like a lot of what Orion has to offer, I will be using 1.0 for a while to minimize the risks of early adoption.

Hi all, I'm Greg from South Australia. I have been implementing a data pipeline and API framework to facilitate ecommerce to ERP synchronisation and have been using Prefect for the ETL part. I'm a bit of a data n00b and have a .net background so python is new to me too. So far so good with Prefect Cloud though and currently going through UAT with a couple of customer implementations now. Dashboards, alerts and centralised job management have been big winners already as we prep for production rollout soon.

Hi team, I'm Benoit from New-Zealand. I am a Data architect/engineer and Django lover. After many years doing data contracting for corporate and startups (e-com, supermarkets, media) and building web products for a bunch of startups. I am starting with some friends a data integration company. Focusing on custom data migrations, data cleansing, machine learning and BI as a service, because I have a green field, the tech stack will be bleeding edge Serverless Prefect 2.0 + DBT on Google Cloud (BigQuery). I will do my best to have a strong open source culture in the company and hopefully can help improving Prefect 2.0 🙂

Hi all 👋 - just pip installed prefect. Look forward to the days ahead 😄

Hi everyone. I'm Robert from Florida, United States. I work as a Data Engineer using Python and SQL to solve data problems, and I just started to use Prefect to create and automate my ELT/ETL pipelines. I look forward to getting acquainted with all of you! 🙂

hey, Al B here 👋, I'm an open-source author (libpostal, the NLP-based international street address parser), longtime data scientist from Brooklyn, and am part of the leadership at the Florida Rights Restoration Coalition who restored voting rights to 1.4 million people with past convictions. We're starting with Prefect 2.0 (just submitted a pull request) as our orchestrator in a first-of-its-kind data infrastructure built by and for people impacted by mass incarceration to organize and empower our folks to change the system.

Hello 👋 I'm Dominic, I'm currently working as a Data/Systems Engineer from Boston, MA. I've been starting to experiment with Prefect for our data pipeline and look forward to chatting with y'all!

Hi everyone! I'm Nikhil, working as a software engineer in London. Just started using prefect to automate my pipelines. So far everything is going great!

Hi everyone, I am an NLP scientist at Enterpret. Currently we are looking to scale our processes. We are looking for tools that abstract Infra and Compute management. We are evaluating based on the following points: • There are multiple tasks in a single flow. • Data needs to be transferred from one task to another task. • Data that needs to be transferred can be huge. So persistance of data at each step of flow (and / or) at the end of the flow is needed. • Should be able to run each task locally manually (For experimentation / debugging purposes) and on cloud (for scaling purposes) • Should be able to run tasks parallely that are independent to each other in a flow. • Should be able to run flows parallely -> Could be useful for running different experiments at same time • Having the ability to monitor the progress of task in a flow, since tasks can take more time • Compute needed for each task in a flow can be different. Ex:Training needs GPU, Data Gathering can work on CPU • Compute used should scale down when no flow is running. • Each step in a flow can have different dependencies. • Learning curve should be less so that data scientists can feel less overwhelmed. I have evaluated Metaflow. But configuring it took a lot of time. Packaging of custom built code is difficult. I was evaluating AWS Sagemaker but the complexity is too high. Can some one help me in understanding the differences between Prefect and Sagemaker. Pros and Cons of both so that I can explain to my team. Would Prefect covers my usecases? I also write blogs on MLOps. Interested in contributing as well. https://ravirajag.dev/

Hi all 👋, excited to join Prefect community. My name is @Artem Vysotsky. I’m a software engineer passioned about all things data. I’m building my own things with prefect and looking to learn more from the community. Previously People.ai. Originally from Belarus, 8 years ago moved to San Francisco, CA and now living in St Pete, Florida. I’m planning to build a very simple tool: a notification service based on data warehouse. + sql + slack. I want users to be able to run a free form SQL query, use the output of it in a message template and send it to a configured slack webhook. Any feedback/input is appreciated. My LinkedIn profile. Happy to connect.

Hi everyone, glad to join the community. I work with the Research Computing group at the University of Virginia.

I’ve started a new data analysis project and am using it as an opportunity to dive into Prefect. It involves some basic data wrangling with a DL component. At the moment I’m doing some prototyping on my local machine, but the goal is to scale it to run on GPU nodes on our HPC system and/or Kubernetes cluster. It’s been fun, I’m quite excited about the Prefect ecosystem—technology/framework as well as people!

Hi all. My name is Sang and I'm currently just starting as a data scientist in Arenko, in the UK - I hope to be working with prefect infrastructure but come from an airflow background. I hope to try to improve upon the existing infrastructure of prefect pipelines. Very nice to meet you all.

Hi, when spinning up tasks in a flow we see some (spinup?) delays. Is there some documentation on how this delay is minimised? I'd expect that the state reporting to Orion server is offloaded from the “hot loop” to avoid latency incurred by the network?

Hello everyone, Glad to be here! My name is Ralph.

Hello everyone, This is Baris , and excited to be here. 👋 In the mean time I have tested my first flow with KubernetesRun and got an error about SSL verification. Details are on the thread.

Good morning all. I’m Gary and I lead a team of Data Engineers. We have recently implemented a number of workflows using Airflow, but while it functions perfectly, I’m a little concerned about its overall architecture. Having recently stumbled across Prefect, I’m a convert, but first have to invest some time in understanding it in more detail.

Hey, I'm Bernardo, I've worked as a Data Scientist before and now I'm engineering an on-prem MLOps solution (hopefully I can open source it at some point). I've never used any workflow orchestrator before, but Prefect looked perfect, specially after reading Why Not Airflow in the docs 🙂