• Al Barrentine

    Al Barrentine

    5 months ago
    hey, Al B here 👋, I'm an open-source author (libpostal, the NLP-based international street address parser), longtime data scientist from Brooklyn, and am part of the leadership at the Florida Rights Restoration Coalition who restored voting rights to 1.4 million people with past convictions. We're starting with Prefect 2.0 (just submitted a pull request) as our orchestrator in a first-of-its-kind data infrastructure built by and for people impacted by mass incarceration to organize and empower our folks to change the system.
    Al Barrentine
    Kevin Kho
    +11
    14 replies
    Copy to Clipboard
  • Dominic Cabral

    Dominic Cabral

    5 months ago
    Hello 👋 I'm Dominic, I'm currently working as a Data/Systems Engineer from Boston, MA. I've been starting to experiment with Prefect for our data pipeline and look forward to chatting with y'all!
    Dominic Cabral
    Kevin Kho
    +7
    9 replies
    Copy to Clipboard
  • Nikhil Joseph

    Nikhil Joseph

    5 months ago
    Hi everyone! I'm Nikhil, working as a software engineer in London. Just started using prefect to automate my pipelines. So far everything is going great!
    Nikhil Joseph
    Michael Adkins
    +8
    10 replies
    Copy to Clipboard
  • Raviraja Ganta

    Raviraja Ganta

    5 months ago
    Hi everyone, I am an NLP scientist at Enterpret. Currently we are looking to scale our processes. We are looking for tools that abstract Infra and Compute management. We are evaluating based on the following points: • There are multiple tasks in a single flow. • Data needs to be transferred from one task to another task. • Data that needs to be transferred can be huge. So persistance of data at each step of flow (and / or) at the end of the flow is needed. • Should be able to run each task locally manually (For experimentation / debugging purposes) and on cloud (for scaling purposes) • Should be able to run tasks parallely that are independent to each other in a flow. • Should be able to run flows parallely -> Could be useful for running different experiments at same time • Having the ability to monitor the progress of task in a flow, since tasks can take more time • Compute needed for each task in a flow can be different. Ex:Training needs GPU, Data Gathering can work on CPU • Compute used should scale down when no flow is running. • Each step in a flow can have different dependencies. • Learning curve should be less so that data scientists can feel less overwhelmed. I have evaluated Metaflow. But configuring it took a lot of time. Packaging of custom built code is difficult. I was evaluating AWS Sagemaker but the complexity is too high. Can some one help me in understanding the differences between Prefect and Sagemaker. Pros and Cons of both so that I can explain to my team. Would Prefect covers my usecases? I also write blogs on MLOps. Interested in contributing as well. https://ravirajag.dev/
    Raviraja Ganta
    Anna Geller
    +5
    8 replies
    Copy to Clipboard
  • Artem Vysotsky

    Artem Vysotsky

    5 months ago
    Hi all 👋, excited to join Prefect community. My name is @Artem Vysotsky. I’m a software engineer passioned about all things data. I’m building my own things with prefect and looking to learn more from the community. Previously People.ai. Originally from Belarus, 8 years ago moved to San Francisco, CA and now living in St Pete, Florida. I’m planning to build a very simple tool: a notification service based on data warehouse. + sql + slack. I want users to be able to run a free form SQL query, use the output of it in a message template and send it to a configured slack webhook. Any feedback/input is appreciated. My LinkedIn profile. Happy to connect.
    Artem Vysotsky
    j
    +9
    12 replies
    Copy to Clipboard
  • Karsten Siller

    Karsten Siller

    5 months ago
    Hi everyone, glad to join the community. I work with the Research Computing group at the University of Virginia.
  • Karsten Siller

    Karsten Siller

    5 months ago
    I’ve started a new data analysis project and am using it as an opportunity to dive into Prefect. It involves some basic data wrangling with a DL component. At the moment I’m doing some prototyping on my local machine, but the goal is to scale it to run on GPU nodes on our HPC system and/or Kubernetes cluster. It’s been fun, I’m quite excited about the Prefect ecosystem—technology/framework as well as people!
    Karsten Siller
    Will Raphaelson
    +6
    8 replies
    Copy to Clipboard
  • Sang Young Noh

    Sang Young Noh

    5 months ago
    Hi all. My name is Sang and I'm currently just starting as a data scientist in Arenko, in the UK - I hope to be working with prefect infrastructure but come from an airflow background. I hope to try to improve upon the existing infrastructure of prefect pipelines. Very nice to meet you all.
    Sang Young Noh
    Jenny
    +7
    10 replies
    Copy to Clipboard
  • s

    Sander

    5 months ago
    Hi, when spinning up tasks in a flow we see some (spinup?) delays. Is there some documentation on how this delay is minimised? I'd expect that the state reporting to Orion server is offloaded from the “hot loop” to avoid latency incurred by the network?
    s
    Kevin Kho
    +1
    11 replies
    Copy to Clipboard
  • Ralph Antoine Vital

    Ralph Antoine Vital

    5 months ago
    Hello everyone, Glad to be here! My name is Ralph.
    Ralph Antoine Vital
    Anna Geller
    +5
    7 replies
    Copy to Clipboard