https://prefect.io logo
Join the conversationJoin Slack
Channels
announcements
ask-marvin
best-practices-coordination-plane
data-ecosystem
data-tricks-and-tips
events
find-a-prefect-job
geo-australia
geo-bay-area
geo-berlin
geo-boston
geo-chicago
geo-colorado
geo-dc
geo-israel
geo-japan
geo-london
geo-nyc
geo-seattle
geo-texas
gratitude
introductions
marvin-in-the-wild
prefect-ai
prefect-aws
prefect-azure
prefect-cloud
prefect-community
prefect-contributors
prefect-dbt
prefect-docker
prefect-gcp
prefect-getting-started
prefect-integrations
prefect-kubernetes
prefect-recipes
prefect-server
prefect-ui
random
show-us-what-you-got
Powered by Linen
introductions
  • j

    John

    03/07/2021, 6:59 AM
    ๐Ÿ‘‹ hi community! am here kicking the prefect tires on behalf of a digital services team in austin, tx, usa. iโ€™ve got the server running with tasks in docker containers. so far so good ๐Ÿ˜…
    ๐Ÿ‘‹ 7
    d
    • 2
    • 1
  • d

    Diego Estela

    03/08/2021, 1:38 AM
    Hi all. I'm Diego, currently doing some proof of concept using prefect in the logistics business.
    ๐Ÿ‘‹ 6
    m
    d
    • 3
    • 2
  • j

    Jay Sundaram

    03/09/2021, 1:50 PM
    Hi all, I'm Jay and am evaluating Prefect for orchestrating computational pipelines. Can someone please point me to documentation that outlines how Prefect writes the flow's state to some xml or json that can be inspected before, during, after the flow has been run/executed?
    ๐Ÿ‘‹ 3
    m
    m
    d
    • 4
    • 3
  • j

    Jay Sundaram

    03/09/2021, 1:52 PM
    If a task/node fails, but the rest of the flow is still running- after some intervention, how do we restart that one failed task/node while the rest of the flow is still running?
    m
    m
    • 3
    • 4
  • k

    Keith Hinde

    03/10/2021, 11:56 AM
    Greetings all - I'm Keith, doing a POC on Prefect for use in my not-for-profit data company. Long time Airflow and Ni-Fi user, so interested to see what Prefect can bring!
    ๐Ÿ‘‹ 4
  • a

    A P

    03/12/2021, 4:01 PM
    Hi all, First time here, tried running prefect local server,,,( prefect server backend) followed every step of Getting started document ,UI server starts but i continuously fail to launch a local agent. Any hints around what could be possibly wrong.
    n
    • 2
    • 11
  • w

    Warsame Bashir

    03/12/2021, 11:30 PM
    Hi all, looking at getting a basic task to show up on the prefect UI (cloud ? maybe). Should I start an agent
    prefect server start
    and kick off my script?
    n
    • 2
    • 1
  • a

    Azeem

    03/15/2021, 11:14 AM
    Hello all, I am Azeem. Getting perfect in IT security. Master in Active Directory.
    ๐Ÿ‘‹ 5
    n
    d
    • 3
    • 3
  • c

    Carlos Gutierrez

    03/18/2021, 11:03 AM
    Hey everyone! Good to be joining this community. I'm a Software Engineer who is new to Prefect and data flows. I'm just getting started with the tool to see if it can fulfill my company's needs. Hope to learn as much as I can about Prefect and data workflows, and to collaborate in making the community growย ๐Ÿ˜„
    ๐Ÿ‘‹ 6
    d
    j
    • 3
    • 2
  • c

    Carlos Gutierrez

    03/18/2021, 11:05 AM
    Is there any dedicated channel to answer questions and problems for people getting started with Prefect?
    m
    • 2
    • 2
  • p

    Paul Prescod

    03/21/2021, 3:32 PM
    Hello everyone, I'm evaluating Prefect for use in specialized CI systems. Our builds can take 30 minutes before the tests start running, so smart parallelization may be able to bring that number down.
    ๐Ÿ‘‹ 7
    d
    j
    • 3
    • 2
  • f

    Fitra Kacamarga

    03/23/2021, 10:51 AM
    Hi everyone! Iโ€™m a Senior Data Scientist in Singapore. Last year, I did a small project using Prefect for Covid-19 Spread Modeling. Now Iโ€™m looking to adopt Prefect for our Data Platform. I am looking forward to learning more and getting support from this awesome community ๐Ÿ™
    ๐Ÿ‘‹ 8
    j
    d
    • 3
    • 2
  • j

    John Lenard

    03/24/2021, 3:20 AM
    Hi Everyone, This is John, software developer in NJ.
    ๐Ÿ‘‹ 3
    d
    • 2
    • 1
  • r

    Reece Hart

    03/24/2021, 4:08 AM
    Hi. I'm the CTO of MyOme, a small Bay Area genomics startup. I'm considering Prefect for genome sequence analysis and interpretation pipelines. If you have experience with that, I'd love to hear from you. (We're hiring!)
    ๐Ÿ‘‹ 6
    d
    m
    • 3
    • 3
  • r

    Ryan Baker

    03/26/2021, 4:58 PM
    Hi everybody, Iโ€™m a software engineer at Recursion, we already use Prefect Core but we are considering using Prefect Cloud as part of our MLOps platform. Nice to meet you all. ๐Ÿ‘‹
    ๐Ÿ‘‹ 8
    n
    d
    +2
    • 5
    • 6
  • d

    David Hagan

    03/26/2021, 5:07 PM
    Hi everyone, I'm CEO/engineer at a Boston-based air quality sensor startup spun out of MIT. I've been a pretty big Luigi and Snakemake user in the past, but am looking at Prefect to re-work all of our data pipelines and MLOps.
    ๐Ÿ‘‹ 6
    n
    d
    m
    • 4
    • 3
  • z

    Zach Schumacher

    03/30/2021, 2:53 AM
    Hi all, Iโ€™m the leader of the data engineering team over at Simplebet, a sports betting startup whose mission is to make every moment a bettable experience. Looking at prefect as a potential replacement after a disappointing upgrade to airflow 2.0.
    ๐Ÿ‘‹ 10
    c
    n
    +2
    • 5
    • 5
  • j

    Joseph Loss

    03/30/2021, 11:47 AM
    Hi everyone, I'm a quantitative developer at a options portfolio management firm in Chicago, IL. I also work as a freelance algorithm dev on UpWork in my (little) spare time. I've lately taken it upon myself to revamp and modernize our dev ops workflow (VisualCron on Windows Server is a nightmare) and I came across Prefect in my quest ๐Ÿ™‚
    ๐Ÿ‘‹ 8
    z
    j
    d
    • 4
    • 3
  • j

    Joseph Loss

    03/30/2021, 11:49 AM
    I am truly appreciative of what you guys are doing here. I've already converted a few workflows on my local to Prefect Cloud and it is a complete game changer!
    ๐Ÿ˜„ 5
    ๐Ÿ‘ 2
  • n

    Noah Holm

    03/30/2021, 12:08 PM
    Hey all! Iโ€™m the data engineer of Mentimeter, a Stockholm based interactive presentation tool. Super excited to be here as weโ€™ve just decided to start testing Prefect for our growing orchestration needs. ๐Ÿ‘‹
    ๐Ÿ‘‹ 8
    z
    m
    +2
    • 5
    • 4
  • h

    Harper

    03/31/2021, 2:43 PM
    Hey everyone! I'm a Data Solution Architect with Databand.ai, a pipeline observability tool. I'm excited to collaborate with the burgeoning Prefect community by discussing pain points and promoting best practices within the data engineering space ๐Ÿ˜›refect:๐Ÿ˜’uccess-kid:
    ๐Ÿ‘‹ 3
    z
    d
    • 3
    • 2
  • j

    Javier Bosch

    04/09/2021, 2:07 PM
    Hi all, I am a Senior Data Scientist at DRW.com. I have been using kedro for our projects and looking to integrate with Prefect to run and manage our pipelines. I also use Dask and now ray quite heavily. I will be looking into the source of Prefect. I saw Dask is used as an executor. I would love to try to add Ray (ray.io) as an executor.
    ๐Ÿ‘‹ 5
    ๐Ÿ‘ 3
    k
    d
    m
    • 4
    • 5
  • n

    Noonaut

    04/09/2021, 9:14 PM
    Hi All! I'm the new lead of a recently born modeling area, we are part of the Integrated Risk Management office of the Bank. The organization doesn't have any team doing Data Science except for us (6 statistician + me, a developer). I had experience with DW and lots of different technologies but completely new to Data Science in general and the open source tooling around. I'm trying to pick the right stack for us and Prefect seems a great solution for Data Workflows!
    ๐Ÿ‘‹ 8
    k
    d
    • 3
    • 2
  • t

    Tim Chan

    04/13/2021, 11:36 PM
    Hello everyone. I'm the lead data scientist at Statsig.com. We build feature flag and experimentation tools for developers. We just integrated Prefect and it's been working great.
    ๐Ÿ‘‹ 8
    n
    j
    d
    • 4
    • 3
  • j

    Jacob Baruch

    04/17/2021, 9:40 PM
    Hi, I am a senior data analyst from an Israeli startup Hexa3D, we are the worldโ€™s first 3D & Augmented Reality as a service platform. we're starting to examine prefect as our data workflow manager tool, going to use and contribute to this perfect community
    ๐Ÿ‘‹ 7
    k
    • 2
    • 1
  • s

    Sveta Kochkaeva

    04/21/2021, 5:00 PM
    Hi, I am a part of a brave team of UK startup UBIO๐Ÿ˜. We help developers to drive API integration with websites when there is no API๐Ÿ˜ณ. It sounds magical and it's magical. We work with Google, Kayak and many more (Finance, Broadband, Travel). Will be happy to connect with you guys and find solution for our Dataflow automation
    ๐Ÿ‘‹ 6
    d
    • 2
    • 1
  • j

    Jacopo Tagliabue

    04/21/2021, 8:19 PM
    Hi all, I'm Jacopo from Coveo, in NYC. Sorry to bother, but I thought the news would be interesting for members of this community (esp. ML practitioners), especially if you need new shiny excuses to build new data pipelines on Prefect ๐Ÿ˜‰ DATA CHALLENGE AT SIGIR 2021 Tired of models โ€œvalidatedโ€ on the MovieLens dataset that donโ€™t work in practice?ย We are proud to host the Data Challenge at SIGIR eCom 2021. Teams are invited to join the challenge and propose novel ML solutions for recommendation and intent prediction in eCommerce: for the occasion, we reveal a unique dataset, with fine-grained anonymized shopping events (>30M), enriched by linguistic behavior and catalog meta-data. May the best MRR/F1 win! IMPORTANT LINKS Challenge Sign-up:ย https://sigir-ecom.github.io/data-task.htmlย  Github:ย https://github.com/coveooss/SIGIR-ecom-data-challenge Data Challenge Paper (draft):ย https://arxiv.org/abs/2104.09423
    ๐Ÿ™Œ 3
    ๐Ÿ’ฏ 3
  • r

    Richard Cox

    04/22/2021, 3:16 PM
    Hi everyone, Iโ€™m Rick from Exscientia, weโ€™re in the middle of migrating a bunch of automations to Prefect Server and having great fun so far.
    ๐Ÿ‘‹ 5
    k
    d
    • 3
    • 2
  • t

    Todd Lindstrom

    04/22/2021, 9:16 PM
    Hello my name is Todd I work for a relief agency that works with refugees all over the world and our end goal is to end war... I hope to get some help with prefect on a mac.

    https://youtu.be/l_imhWlzlisโ–พ

    ๐Ÿ‘‹ 6
    m
    d
    • 3
    • 4
  • w

    William MacMillan

    04/23/2021, 2:49 PM
    hey everyone, I'm Bill, and I'm just this guy, you know? (consultant at the moment, data scientist, finance quant on occasion, data management guy) I have some questions about the performance of prefect; trying to decide if I want to slip this in a stack or write an app for a moderate scale, but relatively low latency (human scale, that is) data capture. Feel free to slide in on my DMs, I'd love to get some insight on my question. Thanks!
    ๐Ÿ‘‹ 8
    k
    • 2
    • 1
Powered by Linen
Title
w

William MacMillan

04/23/2021, 2:49 PM
hey everyone, I'm Bill, and I'm just this guy, you know? (consultant at the moment, data scientist, finance quant on occasion, data management guy) I have some questions about the performance of prefect; trying to decide if I want to slip this in a stack or write an app for a moderate scale, but relatively low latency (human scale, that is) data capture. Feel free to slide in on my DMs, I'd love to get some insight on my question. Thanks!
๐Ÿ‘‹ 8
k

Kevin Kho

04/23/2021, 3:07 PM
Hi @William MacMillan! Welcome to the Prefect community! Feel free to post any questions you may have in the community channel.
๐Ÿ’ฏ 2
View count: 4