https://prefect.io logo
Docs
Join the conversationJoin Slack
Channels
announcements
ask-marvin
best-practices-coordination-plane
data-ecosystem
data-tricks-and-tips
events
find-a-prefect-job
geo-australia
geo-bay-area
geo-berlin
geo-boston
geo-chicago
geo-colorado
geo-dc
geo-israel
geo-japan
geo-london
geo-nyc
geo-seattle
geo-texas
gratitude
introductions
marvin-in-the-wild
prefect-ai
prefect-aws
prefect-azure
prefect-cloud
prefect-community
prefect-contributors
prefect-dbt
prefect-docker
prefect-gcp
prefect-getting-started
prefect-integrations
prefect-kubernetes
prefect-recipes
prefect-server
prefect-ui
random
show-us-what-you-got
Powered by Linen
introductions
  • d

    Dylan

    04/28/2020, 4:36 PM
    set the channel description: Channel for welcoming new members of the community
  • j

    joao

    07/14/2020, 8:47 AM
    Hi The tutorials, both the "Running a Flow" and the "Universal Deploy" tutorials, fail. Both on the
    prefect auth login
    step. Prefect tries to connect locally rather than to your cloud:
    requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=4200): Max retries exceeded with url: /graphql (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f5aa8c9f630>: Failed to establish a new connection: [Errno 111] Connection refused',))
    How do I tell Prefect that it should connect to the cloud?
    d
    • 2
    • 1
  • a

    Andrew Madonna

    10/28/2020, 5:29 PM
    Hi all 👋 , I run a small data analytics and engineering consultancy Data Mill (datamill.co / https://www.linkedin.com/company/data-mill/). We started using Prefect for some projects after testing it on some toy pipelines. Really liking it so far compared to Airflow and Luigi.
    👋 1
    d
    j
    • 3
    • 2
  • e

    Edison A

    10/29/2020, 9:29 PM
    Hello World, I'm Edison Abahurire, realtime data processing engineer form Kampala, Uganda. I'm using prefect for FTP data ETLs in staging, and its very promising. I tried Airflow and it was quite a job; it wanted me to re-write my whole project 😒.
    j
    d
    • 3
    • 2
  • m

    Marc Lipoff

    12/08/2020, 7:30 PM
    Hi everyone. We are considering trying out Prefect, and had some questions. We are a healthcare company, and deal with hipaa. 1. Does Prefect Cloud have security processes in place to deal with the complexities of hipaa? 2. Is there a write up on how Prefect Cloud ensure data security for its customers? Also, what is the best way to get a quote for a Enterprise-level Cloud account?
    j
    k
    j
    • 4
    • 19
  • r

    Robert Bastian

    12/15/2020, 2:23 PM
    Hello everybody! I'm Robert Bastian, VP of Research and Innovation for Enverus. We're looking at Prefect as a replacement for Airflow (who isn't?). We love the UI and the documentation is excellent. I love being able to look in the github repo at the code to understand how things work. That said, I have some basic questions on how to do things the "Prefect way" that I'll be asking. Is there a channel for "new to Prefect" folks? Thanks!
    j
    j
    • 3
    • 2
  • m

    Matt Drago

    12/15/2020, 10:34 PM
    Hey folks, I've been lurking around here and have started to post a few replies so I thought it best to do an Intro. I'm Matt Drago and I work for a company with their HQ in Melbourne, Australia called PaperCut. I'm running with a team that is setting up Prefect to help out with our Data Democratisation journey. I'm looking forward to also contributing back to the Prefect code base where I can.
    👋 7
    🎉 5
    n
    d
    +2
    • 5
    • 4
  • k

    Kireeti Kunam

    12/22/2020, 8:52 AM
    how to install prefect on local system using docker
    👀 1
  • k

    Kireeti Kunam

    12/22/2020, 8:53 AM
    and how to run my first code snippet ., somebody please help me
  • k

    Karolína Bzdušek

    12/22/2020, 3:38 PM
    Hey all! I am Karolina, I am Data Engineer at cividi and right now I am responsible to do research for suitable solution for our workflow. You got my attention, so far Perfect looks like a hot candidate! Looking forward to get more info from the community, I'll check other channels 🙂 And hey, merry christmas/ happy holiday to all! 🎄🎉
    👏 4
    ☃️ 3
    👋 4
    d
    m
    • 3
    • 2
  • g

    Giovanni Giacco

    12/24/2020, 9:30 AM
    Hello to everyone. I am Giovanni, I am a technical product manager and in these days I'm looking for a solution to create a data engineering pipeline for geospatial data elaboration. I am experimenting with Prefect core and it looks great. Actually, it's like a battle between Prefect and Dagster. Thank you to all, this is a great community!
    👋 2
    m
    d
    s
    • 4
    • 3
  • g

    Gareth Dwyer

    12/30/2020, 3:54 PM
    Hey all, I'm Gareth - Founder of Ritza.co, a technical publishing company. We produce books, blog posts, technical documentation etc with a focus on machine learning, devops and software engineering. I'm writing some articles that discuss Prefect so would love to discuss some things here as I don't have a ton of first hand experience with it. Specifically regarding the "why not airflow" article and whether Airflow 2.0 adequately addresses any of the concerns about Airflow raised there. https://docs.prefect.io/core/getting_started/why-not-airflow.html Specifically I'd like to add prefect to our article here https://www.datarevenue.com/en-blog/airflow-vs-luigi-vs-argo-vs-mlflow-vs-kubeflow
    👀 1
    👋 3
    d
    c
    • 3
    • 4
  • m

    Michał Dwużnik

    01/03/2021, 7:30 PM
    Hello, Michal, university sysadmin looking to get some automation UIs built
    👏 5
    👋 3
    d
    • 2
    • 1
  • j

    Jeremy Phelps

    01/04/2021, 11:00 PM
    Hello, I'm a software engineer looking to migrate away from Google Cloud Composer.
    👋 4
  • s

    Steven Das

    01/06/2021, 2:12 PM
    Hi all, software architect here exploring whether Prefect might be a good option for running some rather complex flows in a production SaaS product which involve a mix of... 1. Multiple users with different roles, all submitting their own inputs and reacting to the progress of the flow in different ways. 2. Calls to various services (some long-running). 3. Quite a bit of branching logic based on the inputs provided by users and other services, and potentially even the need to repeat, loop, and/or add/remove steps midway through the flow as needed. 4. Potentially multiple different terminal states for the flow (e.g., we can terminate if we have data X and Y, or else if we have data Z, X, and A). 5. The ability to run and A/B test different versions of flows for different users simultaneously. Not sure if something that hairy is abusing Prefect's focus on data engineering, or is even possible in Prefect...but I look forward to finding out and getting to know you all. :)
    👋 2
    m
    w
    • 3
    • 7
  • t

    Tadas

    01/07/2021, 9:02 AM
    Hello, I’m software engineer looking into replacing our Airflow deployments and pipelines with Prefect 🙂 Mainly because we can’t move some of our pipelines to Airflow as it is missing good support for dynamic dags. We need a high degree of dynamism in some of our pipelines.
    👋 5
    d
    m
    j
    • 4
    • 10
  • j

    Jeff Williams

    01/07/2021, 7:06 PM
    Hello All. I am a contractor working for a company that I can't really disclose, not a government entity, due to compliance issues. However, I am trying to set up Prefect in a GCP cloud. I am using the great Medium article as my roadmap - https://medium.com/the-prefect-blog/prefect-server-101-deploying-to-google-cloud-platform-47354b16afe2. I am running into some issues on start up so I am here to see if I can glean some insights into resolving those issues.
    👋 6
  • m

    Maurits de Ruiter

    01/11/2021, 8:57 AM
    Hi everyone! I'm a software developer looking for a great tool to professionalize our data pipelines. Right now we're working with lambda's and S3 triggers, but it's getting too big for that. I started out with Airflow, but it doesn't seem to have support for dynamic dags and that is a main feature we need.
    👋 4
    d
    • 2
    • 1
  • e

    Eric Sawler

    01/11/2021, 12:57 PM
    Good morning! I am evaluating prefect for orchestrating our workflows. I'm starting with the community edition..
    👋 5
    d
    j
    • 3
    • 4
  • l

    Lennard Timm

    01/12/2021, 1:22 PM
    Hi everyone! I'm a data coach and I work with data teams, guiding them and helping them build a mature data practice. I have a strong motivation to not use Airflow (due to its cluttered UI, eccentric scheduler, …) and tried Prefect some time ago. I think its version was 0.6 back them and it didn't feel very ready but I made it work and deployed pipelines, each in a container running on Docker Swarm. I'm really excited to see how Prefect matured and recently reworked the pipelines and set up Prefect Cloud and an agent. I need to distill this setup to a blueprint so I can reproduce that but Prefect already has earned a spot in my Data Engineering Toolkit. Thank you for your work!
    :marvin: 4
    🚀 7
    👋 5
    j
    d
    • 3
    • 3
  • l

    Loic M

    01/13/2021, 9:09 AM
    Hello there, I'm Loïc, data engineer from 🇫🇷 As I made a recent transition from a more data scientist/ML Ops position, I am looking to expand my toolkit 🙂 I already had some experience with Airflow, which I found quite cumberstone: that's why I am impatient to try Prefect in my personal and freelance projects !
    👋 5
    d
    • 2
    • 1
  • l

    Lucas Kjaero-Zhang

    01/14/2021, 5:16 AM
    Hi everyone! I’m coming from a software dev background and am dipping my toes into data. Good to meet you all
    👋 6
    c
    d
    • 3
    • 2
  • b

    Benedikt Maria Beckermann

    01/15/2021, 2:43 PM
    Hi 👋 Data Engineer from the gaming industry here 🙂 Currently investigating prefect to stay up to date with the tech ^^
    👋 5
    n
    d
    m
    • 4
    • 3
  • b

    BK Lau

    01/15/2021, 8:14 PM
    Hi BK from Boston here. Currently investigating Prefect to use in quantum computing workflow
    👋 3
    d
    • 2
    • 4
  • b

    Barbara Gunion

    01/20/2021, 7:39 PM
    Hi, I am BI engineer from Colorado. I am a 6 month newbie user,
    👋 5
    m
    d
    • 3
    • 3
  • t

    Tom Baldwin

    01/22/2021, 2:37 AM
    Hello 👋 I’m a data scientist in Portland, OR. Checking out Prefect as a potential replacement for our existing data pipeline orchestration system.
    👋 5
  • a

    Anze Kravanja

    01/22/2021, 6:03 AM
    Hello 👋 Another DS here, trying to see if Prefect fits our needs.
    👋 6
    n
    j
    • 3
    • 2
  • c

    ciaran

    01/22/2021, 4:53 PM
    Hey folks 👋 Just playing around with Prefect as we look to use it Cloud Engineer @ Development Seed but remote in the UK
    👋 7
    n
    d
    j
    • 4
    • 3
  • m

    Matt Gordon

    01/22/2021, 6:40 PM
    Hi all. I’m a Principal Software Engineer at Kairos Aerospace, where we do greenhouse gas emissions monitoring. We are long-time airflow users, but I am starting to use Prefect to implement some of our new ML workflows.
    👋 6
    n
    d
    j
    • 4
    • 7
  • j

    James Hyde

    01/25/2021, 4:35 PM
    👋 Hi all, I'm a BI Engineer at GoCardless in early stages of discovery to understand how Prefect could help improve our development workflow. Looking forward to learning more.
    👋 7
    d
    • 2
    • 1
Powered by Linen
Title
j

James Hyde

01/25/2021, 4:35 PM
👋 Hi all, I'm a BI Engineer at GoCardless in early stages of discovery to understand how Prefect could help improve our development workflow. Looking forward to learning more.
👋 7
d

Dylan

01/25/2021, 4:36 PM
👋 Welcome James!
View count: 4