Hello, I'm Patrick Wyatt, an online game developer in the Seattle area, and co-founder of One More Game. I work on data analytics in my "spare time" when I'm not doing all the myriad tasks related to developing & publishing a game. We're pre-launch now so only have a few tens of users playing our game. Our data volume is quite low -- hundreds of records per day, not millions ... yet. I've really enjoyed building a pipeline to export JSON data from our game & backend servers -> AWS Kinesis Firehose -> S3 data lake, and then using Prefect to manage flows that ingest and process the data in Firehose.

Hi @Patrick Wyatt, that’s quite a stack! What is your current game?

Howdy Kevin -- we haven't announced yet, sorry -- it's still under wraps

Ah ok. No worries!

Welcome to the community @Patrick Wyatt, look forward to playing on announce. You've got a good set of PC gamers here (and one or two console @Zach Angell)

Hey @Patrick Wyatt 👋 I’m one of those PC gamers! Welcome to the community

Welcome @Patrick Wyatt! Looking forward to the official announcement 👋

Welcome @Patrick Wyatt!

Great @Patrick Wyatt, we’d love to learn more about how you’re implementing that tech stack and improvements you’re seeing. Welcome!

Welcome to the community @Patrick Wyatt!

@Andrew Black -- this is a green-field project for my new company, and the last time I built something like this was some years ago, so I guess the biggest improvement is that I was able to utilize solutions like Prefect, Firebolt, Kinesis and S3 instead of building bespoke solutions using on-prem servers! The code I've written is a combination of C#/Kotlin/JavaScript for the game services that need to export data to Kinesis, and Python/SQL for Prefect ingestion and data-processing, of course. The project is deployed/automated using Terraform to configure build pipelines on Buildkite for CI/CD; and deployed in Docker containers running in Nomad. So there's a fair bit of infra work to make this all function. If you have some specific questions I'm glad to try to answer. Cheers!

Very cool

Thanks for sharing @Patrick Wyatt, and welcome to the Prefect community!

Welcome @Patrick Wyatt!