Hey everyone! If you’re interested in learning more about what’s happening in the Prefect/Dask space, or if you just want to see some live action Prefect coding, join me for a livestream with our friends from Coiled! https://twitter.com/markov_gainz/status/1280996022420791296

Hi all! ⬆️ ⬆️ This livestream is happening right now on the ; stop on by if you're interest in learning some stuff about Prefect and Dask! 😄

We’re live again! @Laura Lorenz (she/her) is talking about Prefect and scientific computing for the SciPy 2020 sprints - tune in here:

Hey folks! We want to know which tools you’re using with Prefect so that we can build better experiences and more streamlined integrations. Let us know what you’d like to see! (and thanks to everyone who’s filled out the survey so far) https://forms.gle/FdpQd29KthNEFFMj7

Hi video content fans! If you missed it live you can watch the replay of today’s livestream on Results and Resources (brought to you by the letter “R”!!) on youtube here: . MEGA shoutout to the contributors who have been working on

PandasSerializer

whose work I feature! While you’re watching, see if you can spot the huge invisible sweatdrop on my forehead when I attempt to pronounce

parquet

and secretly realize I don’t know how to say it 😅 unfortunately I have now learned it is the more sophisticated

par-kay

and not

par-kett

which is what I went with 🙃 today I learned!

We have a HUGE announcement today: Prefect Server and Prefect UI are now standalone open-source projects! 🍾🎉 This release of Server and the UI consists all-new code, containing four months of improvements since the last release. Most importantly, they are now shared codebases with Prefect Cloud, which means the pace of maintenance and improvements will be dramatically better. In fact, the UI is the full Cloud UI… which means you can self-host it even if you’re running against the managed Prefect Cloud API. Prefect Core 0.13 is also out with full support for the new open-source backend. Thank you to everyone in this community who helped make this possible over the last few months. We’re excited to continue delivering major improvements to all of you by building on this solid foundation. Happy engineering! https://medium.com/the-prefect-blog/the-all-new-prefect-server-and-ui-f3141b111b36

Hi everyone! Some fun news - following our big open-source announcement, Prefect has been the #2 repo on GitHub for the past few days! I’m sure that comes as absolutely no surprise to anyone here 😉 but as we prepare to welcome many new users to our community, I want to thank everyone here who’s been part of this story for some time. It means an incredible amount to our small team to see our efforts and excitement reciprocated by our users — every time someone has a good experience here in Slack, or opens their first PR, or closes a frustrating bug, or finds one of Marvin’s easter eggs, it compounds in a way that is unique to an open-source community. Thank you for being part of ours.

Did I say #2? Sorry for the typo. Congrats, team!

Hi all, we're experiencing some unexpected downtime with the Cloud API - will update as we know more.

The Cloud API is back to full service - we're monitoring the situation and will continue to update as we know more.

Hi folks - we are seeing an incredibly large amount of Cloud activity today that is causing some issues in our backend. You might see delayed work and delayed state updates in the UI as we try to identify the root cause of the slowdown. I will provide updates as I have them.

Quick update: we released some performance improvements and things appeared to have calmed down for the time being; we’re still actively monitoring the situation and will report back if anything looks off! Thanks for your understanding

Announcing: the 5-for-5k contest! Prefect is about to hit 5,000 GitHub stars (🤯) and we think that’s as good a reason as any to give stuff away, headlined by a 5-user Cloud license but we’ve got lots of great gear, Marvins, coffee… you name it. Entering the contest is really easy - just mention @Marvin here in Slack (he’ll have to be in the channel), or mention

@marvin-robot

in any issue or PR on Prefect’s GitHub. Go on — it’ll work. At least it did in tests! 😅 The contest will stay open until we hit 5,000 stars and then we’ll pick and announce the winners. Good luck and happy engineering!

Hey everyone! We’ve been quietly working on refining many new features over the last few weeks - in @Jenny’s latest blog post, she outlines how to configure manual approval for your tasks; check it out here: https://medium.com/the-prefect-blog/needs-approval-184f2512a3cf

Hello everyone! Just a reminder that the 5k contest looks like it’ll be over in a few days… just mention @Marvin to enter! p.s. we’re so happy that you all love Marvin as much as we do - he runs a TON of Prefect’s internal operations behind the scenes, and we’re going to make him a more permanent fixture here in the community. Prepare to have even MORE delightfully depressing conversations!

Hey folks!  Someone opened up this PR that adds Prefect to a list of awesome python tools: https://github.com/vinta/awesome-python/pull/1589. It’s already been getting some activity, but still has 8 “thumbs up” reactions to go before it can be merged!  If you enjoy using Prefect and would like to see it included on this list, give it a 👍 and hopefully we can gain some more visibility!

Honestly, I don’t think any of us expected to see Prefect get adopted so quickly - we hit 5000 stars on GitHub early this morning. It’s a big milestone for our team, sure, but it’s an even more important marker of how this community has grown. I want to especially thank every person here who’s ever responded, commented, helped, or even just added a slackmoji to someone else’s comment or question. It makes every bit of difference to make sure Prefect works properly - and that we all get to fail successfully. Uh… and for anyone new, failing successfully is a good thing around here. 🙂 Happy engineering - we’ll announce contest winners soon!

p.s. Marvin will return…

Hey everyone - just wanted to let you know that Google is experiencing intermittent outages across their services. Prefect Cloud appears to be operating normally, but wanted to let you all that we’ll be monitoring the situation closely! Update: Google says the issue is resolved (details here: https://status.cloud.google.com//incident/zall/20010). Prefect Cloud was unaffected by the incident

Hi everyone! Cloud users will have just received a notification about our new Public Changelog, a place where we'll begin to publish a durable, long-form changelog of features and updates across the Prefect ecosystem. This first entry details some really great features and improvements we've released of late that we didn't publicize as much as they probably deserved, including things like flow concurrency limits, improved flow run cancellation, configurable schedules, and more! We plan to release these public changelogs roughly every weeks, so stay tuned!

<!here> Hi Everyone! I’m excited to announce the winners to the 5 for 5K Contest! Our Runner Ups! You’ll all receive a 😛refect-duck: and 😛refect: stickers 💯 @Fanilo A. @Hamza Ahmed @Ben Fu @Timothy Byrne Github user

RMKD

(we’ll

@

you on GitHub, don’t worry) @itay livni @Mitchell Bregman @Cole Howard @abhilash.kr @Spencer Our Winners! You’ll all receive a box of swag, including coffee, 😛refect-duck:, and more! @Marko Herkaliuk @Dinu Gherman @Sandeep Gainda @KIRYL BUCHA And our Grand Prize Winner, who will receive 1 Prefect Cloud Enterprise Tier License for 5 Users for 1 Year, is… 🥁 🥁 🥁 🥁 🥁 🥁 🥁 🥁 🥁 🥁 🥁 🥁 🥁 🥁 🥁 🥁 🥁 🥁 🥁 🥁 🥁 🥁 🥁 🥁 🥁 🥁 🥁 🥁 🥁 🥁 🥁 🥁 @Avi A! Congratulations! We will follow up with all of our winners by email to coordinate details. Thank you everyone who entered. We’re so excited to celebrate this milestone with you all ⭐ Just wait ’till you see what we’ve got in store for 10k 😉 Happy Engineering!

Hello everyone! Our very own @josh just published this fantastic blog post on contributing to the Prefect Task Library: https://medium.com/the-prefect-blog/the-prefect-task-library-49a1a1c5b466. If you’ve been considering making a contribution, whether to add an entirely new task or to add more features to an existing one, this post will help you get started!

Hey folks! The Great Expectations team reached out to collaborate on improving Prefect’s integration with the GE library. We plan to use this collaboration as a proving ground for a more general feature that allows you to publish structured task metadata that can be rendered in the UI. If you’re a GE user, or are interested in this, please chime in on the discussion we’ve started here: https://github.com/PrefectHQ/prefect/discussions/3558

💡Spotlight on a new feature💡 Each Friday, I am going to put the spotlight on a particular feature recently released. This is a great week to start because of a really exciting feature now available in Prefect Cloud (and coming soon to Server). Following a year of thinking about the best approach and listening to feedback, our UI team has replaced the Gantt Chart with a new Timeline view. In addition to looking slick, this new minimal design allows you to identify much more readily potential issues in your flows or your infrastructure, how long tasks are taking and just generally more granular detail on timings between the various task run states. Now available on your flow run page. Let us know your feedback and look out for more enhancements soon!

💡Spotlight on a new feature💡 Over the last couple of weeks @Michael Adkins along with members of the community have been working on our Helm chart for deploying server. We have continued to build upon this, adding a Kubernetes agent this week. Hopefully, we'll have sometime in the Core Contributor Cantina today at 4pm Eastern time to go through any questions or topic items you'd like to discuss with @Laura Lorenz and @Michael Adkins. The link to the Catina is here. Let us know your feedback and look out for more enhancements soon!

Hi everyone! I want to take an opportunity to highlight a new channel we’ve created called #find-a-prefect-job that’s, uh, for finding Prefect jobs! Please take a moment to visit, whether you’re hiring or looking for an “almost perfect” opportunity. Thanks to @jack clement and @Robin for getting us kicked off!

💡Spotlight on a new feature💡 Because we were off last Thursday and Friday for the Thanksgiving Holiday here in the U.S., I have two weeks of work to choose from for my spotlight this week. We have found many users that are just starting with Prefect can get confused about deferred computation patterns; in particular, many users would write code like:

@task
def return_two_things():
    return 1, 2
with Flow("example") as flow:
    one, two = return_two_things # this raises an error

This would cause an error, because Prefect has no way of knowing how many values your task returns (because it has not been run yet!). @Jim Crist-Harif crafted an elegant solution that allows you to specify the number of return values so that you can continue using familiar Python syntax.

@task(nout=2)
def return_two_things():
    return 1, 2
with Flow("example") as flow:
    one, two = return_two_things # this now works!

Thank you to you the community that continues to provide feedback, so that we can continue to add enhancements such as these once we have had the chance to observe and choose the best way to provide a solution. For more details, please see the link here. In addition, this may look more like a 💡spotlight💡on @Jim Crist-Harif but he as just published a new deployment tutorial guide, perfect for anyone getting started or more seasoned users wanting to read about our new flow configuration. The link can be found here.

Hey folks! Do you love Great Expectations? Have you always wanted to see your validation results natively in the Prefect UI? Wait no longer! We’ve teamed up with the Great Expectations team on a shiny new task that automatically publishes your validation reports to the UI — and the best part is that you can reuse this hook to create completely custom markdown reports for any other task that you’d like! For more info, check out our blog post: https://medium.com/the-prefect-blog/introducing-the-artifacts-api-b9e5972db043 If you have questions about Great Expectations, join their slack and say hello: https://greatexpectations.io/slack

Hello everyone! Friendly FYI that we will be performing some planned service upgrades this weekend, on December 13th, 12:30AM UTC - 1:00AM UTC. We don’t expect any issues, but feel free to follow along on our status page: https://prefect.status.io/pages/maintenance/5f33ff702715c204c20d6da1/5fd3bb6c1ac27a0536e58d0c