https://prefect.io logo
Title
k

Kevin Systrom

04/29/2020, 12:52 AM
If I run the flow.run block it seems to miss the cache every time... I'm v confused!
c

Chris White

04/29/2020, 12:54 AM
Welcome back! Whenever you run Prefect without a backend, the cache is stored in
prefect.context
. I believe that your call to
with prefect.context():
is effectively clearing the cache between runs
I’d expect that your tasks will begin picking up the cache if you remove that particular line
k

Kevin Systrom

04/29/2020, 1:39 AM
Thanks @Chris White - that did it. I was actually using the context for something but removed it for clarity. So this means that the cache stays around as long as the with block (which is a single run/gets cleared out every time)?
💯 1
when you say 'without a backend' ... can you clarify? thanks
(ah, nm. i see... i'm starting the server now)
c

Chris White

04/29/2020, 1:48 AM
For sure! For your first question: - the cache lives in
prefect.context['caches']
, so if you clear your context the cache will disappear; if you removed the
with prefect.context():
block then you shouldn’t need to worry about the cache disappearing unexpectedly - for the backend, sounds like you figured it out but to provide some more color: Prefect Core alone maintains state only as long as the Python process running your Flow. We have various options for how to run flows in a more stateful way (which includes how task caching is handled!), one of which is Prefect Server (our open source offering) — if you don’t want to deal with setting up the database, services, etc. Prefect Cloud is essentially a managed version of server (with more bells and whistles, e.g., authentication / user management / etc.). Note also that Cloud is designed to keep your code and data private (see, e..g, https://medium.com/the-prefect-blog/the-prefect-hybrid-model-1b70c7fd296) For the best caching experience, I do recommend using one of our backends
k

Kevin Systrom

04/29/2020, 1:50 AM
got it, makes sense thank you
c

Chris White

04/29/2020, 1:51 AM
anytime anytime - let us know if you have any issues configuring server and / or getting caching working properly 👍
k

Kevin Systrom

04/29/2020, 1:51 AM
and one last thing: i noticed that prefect.context.today is in UTC not local
is that normal?
I expected today to be ... today 🙂
it's all good, i just started on this today so still learning the ins/outs
c

Chris White

04/29/2020, 1:52 AM
Haha yea, we found that using local timezones caused more confusion when migrating flows around from local -> production systems, so for consistency we keep everything in UTC
k

Kevin Systrom

04/29/2020, 1:56 AM
got it - ok. thanks!
I appreciate the help
c

Chris White

04/29/2020, 1:57 AM
anytime anytime, that’s what we’re here for