Code Snippets en

From Python
Jump to: navigation, search


Code Snippets (English)

Here you can put short examples of code for community review. If you have some idea or tip which seems useful, but you are not too sure and/or wish to get comments from the community before running off to publish in Python Cookbook, just click Edit, and write it up here.

Or, if you prefer to write it up in hebrew, use the hebrew snippets page.

(To keep things in order, I recommend duplicating the template at the bottom)

cprops: Cached properties using non-data descriptors

I've been using the following decorator/descriptor for some time now, and it has become a frequent pattern in my code. However, I never found the time to document it. I'll start here with the basic idea, which is quite simple, and add the extensions (don't worry, only 2-3) later if/when I find the time.

The idea of a cached-property seems to be quite known. One common use case is when your class has an attribute which takes some effort to calculate, and is not necessarily always used by applications. In such cases you wish to build it in a "lazy" way. The first time it is accessed, some specialized method is called to calculate it, but any later reference to the attribute will return the stored value.

It seems that people usually implement this using the standard "property" descriptor (sometimes even using specialized decorators). However, these implementations seemed a bit convoluted to me, due to the fact that "property" is a data descriptor. This means that the __get__ method takes precedence over the owner instance's __dict__, so one has to explicitly take care of providing the cached value on later references. On the other hand, had we used a non-data descriptor this would be taken care of automatically by the standard attribute access mechanism - all you have to do is set the attribute on the owner instance.

So, let us start with the "vanilla cprop":

class cprop(object):
    def __init__(self, func):
        self.func = func
    def __get__(self, obj, klass):
        val = self.func(obj)
        setattr(obj, self.func.__name__, val)
        return val

class Demo(object):
    def the_answer(self):
        print "calculating..."
        return 42

>>> demo = Demo()
>>> demo.the_answer
>>> demo.the_answer


  • Hi, Amit. This is nice and probably more efficient than using a data property. Notice, though, that you lose the read-only character of the property:
class Demo2(object):
    def the_answer(self):
        return 42

>>> demo = Demo()
>>> demo.the_answer = 6*9
>>> demo.the_answer
>>> demo2 = Demo2()
>>> demo2.the_answer = 6*9
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: can't set attribute

Shai 10:08, 28 August 2009 (UTC)

Good riddance. All the foot-shooting power to the user :-)
Furthermore, it is actually useful---sometimes you have the result from another source (or you just want to debug), and this gives you a simple way to short-cut the computation. --AmitAronovitch 00:22, 26 October 2010 (UTC)
  • References to other places where people do exactly (or almost exactly) this - please add if you find some more:

--AmitAronovitch 20:16, 27 December 2012 (UTC)


Add dictionaries with last-win priorities, possibly limiting the keys. This is particularly useful for simple config management, where you have several information sources.

The limit_keys option is also a useful shortcut by itself (even for using on a single dict).

def _dict_add(d1,d2):
    res = {}
    return res

def _dict_cascade(dicts):
    return reduce(_dict_add, dicts, {})

def dict_cascade(dicts, limit_keys=None):
    if limit_keys is None:
        return _dict_cascade(dicts)
    limit_keys = frozenset(limit_keys)
    res = {}
    for d in dicts:
        for k in limit_keys.intersection(d.keys()):
            res[k] = d[k]
    return res

public comments/responses

  • Write response here

wiki template/example - please dup and edit

what it does

#put code here
# each line should start with blank
import this

Originator's comments and questions

public comments/responses

  • Write response here
Personal tools