Smart Caching Of Expensive Objects In Python

I have a directory of images in order. Typically my code will be using data from a sequential subset of images (e.g. images 5-10), and the naive options for accessing these are: C

Solution 1:

You can extend the default dict and use __missing__ method to call a loading function if the key is missing:

classImageDict(dict):
    def__missing__(self, key):
        self[key] = img = self.load(key)
        return img
    defload(self, key):
        # create a queue if not exist (could be moved to __init__)ifnothasattr(self, '_queue'):
            self._queue = []
        # pop the oldest entry in the list and the dictiflen(self._queue) >= 100:
            self.pop(self._queue.pop(0))
        # append this key as a newest entry in the queue
        self._queue.append(key)
        # implement image loading here and return the image instanceprint'loading', key
        return'Image for %s' % key

And the output (the loading happen only when the key doesn't exist yet.)

>>>d = ImageDict()>>>d[3]
loading 3
'Image for 3'
>>>d[3]
'Image for 3'
>>>d['bleh']
loading bleh
'Image for bleh'
>>>d['bleh']
'Image for bleh'

One evolution would be to store only the N last element in the dict, and purge the oldest entries. You can implement it by keeping a list of keys for ordering.

Solution 2:

Weakrefs aren't what you want -- weakrefs are a way to reference an item that allows the garbage collector to collect (i.e. destroy) the referent if only weakrefs to it exist. In other words, if you create and store only weakrefs to some object, it is likely to be garbage collected quickly, and you won't have benefitted from it.

I'd go with your option #1 above. On modern operating systems, the OS maintains an in-memory cache of recently accessed files (or parts thereof), which will mean that you'll have to bear the cost of loading the file from disk once, but after that, subsequent accesses to the file will be as fast (or nearly so) as if it were in memory in your application. The FS cache is usually a LRU-style cache, so frequently-accessed items will tend to stay in memory, while infrequently accessed items will tend to be evicted (and will subsequently be loaded from disk if needed). In most cases, it is sufficient to rely on the operating system's implementation of this sort of logic, rather than writing your own (especially since you don't have to write and maintain code to do it!)