Today’s weird question. Is there a way to confirm what image a k8s run used? I’ve uploaded a new image, with a new tag, and confirmed that my code changes are in it, but that’s definitely not the code that’s executing. I think there used to be a “versions” tab that had this info, but seems to have gone away?

Hi @Brian Mesick - you can still access the same information that was displayed on the Versions tab by selecting the version of your flow from the dropdown at the top of the flow page and navigating to to the flow "Details" tab on the top left most tile:

Aha cool, thank you. I have trouble remembering that panel is scrollable sometimes.

Hm I don't know of any but definitely haven't encountered this specific use case

I can see that the flow used the expected image, I have downloaded the image and can see the expected code in

/usr/local/lib/python3.8/site-packages/

. Maybe something in

__pycache__

?

Yeah my only thought would be some cache of the package in your CI process

We don’t have a CI process 😅 Just running

flow.register

. I can see it building the wheel and everything for our package. So weird.

Hm... did you try clearing that cache and re-registering?

Which cache?

Your local python cache, I'm not even sure how those are cached though, I'm sorry >.<

Ok, thanks. I think it’s running pip on the image and (hopefully) not locally, so I’d guess it’s populating the cache when it first runs for the health check. I’ll just nuke everything and rebuild from scratch to see what happens. Thanks for looking.

Sure thing, let me know what you find! Definitely an interesting use case

It’s been working for us for a long time, but things are always changing

Is your image pull policy

IfNotPresent

or

Always

? It may be using a cached image entirely. The default is

IfNotPresent

on the K8s agent.

Hey Michael! We recently made a change to make it

Always

to work around that kind of issue, though it’s hard to tell whether it’s working. We’ve never had a problem with images being cached when we bumped the image tag, though. Eventually I blew away my entire local environment, all docker images, bumped the image tag, and rebuilt and the changes got picked up. Still have no idea where things went awry.