open atlas
↑ Back to track
Python for JS/TS developers PY · 03 · 02

Attribute lookup and descriptors: why the instance dict does not always win

Attribute lookup is not instance-first: data descriptors on the type beat the instance __dict__, which beats non-data descriptors. __getattribute__ runs on every access, __getattr__ only on misses. property is a data descriptor; __set_name__ lets ORM fields learn their names.

PY Senior ◷ 19 min
Level
FoundationsJuniorMiddleSenior

The caching layer looked innocent: a __getattr__ on the client class that lazily built expensive sub-clients on first access. It worked for a year. Then someone refactored a typo inside __getattr__ itself — the method now touched self._registry before _registry was assigned in __init__. The symptom was not an AttributeError pointing at the typo. It was a RecursionError: maximum recursion depth exceeded with a thousand-frame traceback, in production, on instantiation. The mechanism: self._registry missed the instance dict, so Python called __getattr__('_registry'), whose body again touched self._registry, which missed again. Every engineer who debugged it learned the same lesson from the descriptor HOWTO that day: attribute access in Python is not “look in a dict”. It is a precise algorithm with two hook methods that fire at different times, and a descriptor protocol sitting above the instance dict — the machinery that makes property, methods, __slots__, ORMs, and pydantic all work.

By the end of this lesson you will know why RecursionError in __init__ is actually an attribute-miss chain — and how the same mechanism powers property, ORM fields, and bound methods.

The real lookup order: object.getattribute

Every obj.x calls type(obj).__getattribute__(obj, "x"), and the default implementation runs this algorithm — order matters and is the source of most “why is my attribute ignored” mysteries. When you next wonder why a property cannot be shadowed or why assigning to self.__dict__['x'] does nothing visible, trace your way through these five steps:

  1. Walk type(obj).__mro__ looking for "x" in each class __dict__.
  2. If found and it is a data descriptor (defines __set__ or __delete__), call its __get__the instance dict never gets a look.
  3. Otherwise, check obj.__dict__ — a plain instance attribute wins here.
  4. Otherwise, if the class attribute found in step 1 is a non-data descriptor (only __get__ — functions are exactly this), call its __get__. Plain class attributes are returned as-is.
  5. Nothing anywhere → raise AttributeError, which triggers __getattr__ if defined.

So “instance shadows class” is only true for non-descriptor and non-data cases. A property defines __set__ (even a read-only one — its __set__ raises AttributeError), making it a data descriptor: that is precisely why you cannot accidentally shadow a property by writing self.x = 1 — the assignment itself routes through property.__set__. And it is why functions, being non-data descriptors, can be shadowed per-instance: obj.method = lambda: ... works. Bound methods exist because step 4 calls function.__get__(obj, cls), which returns the bound method object — methods are not special syntax, they are the descriptor protocol applied to functions.

getattr vs getattribute

__getattribute__ intercepts every attribute access — override it and you pay its cost on every self.anything, and any self.attr inside it recurses unless you delegate with object.__getattribute__(self, name). __getattr__ is the cheap hook: called only after normal lookup raised AttributeError. That is what makes it right for lazy loading and proxies — zero overhead on hits — and what produced the Hook’s recursion: an attribute miss inside __getattr__ re-enters __getattr__. The defensive pattern:

class LazyClient:
    def __init__(self):
        self._cache = {}          # assigned before any miss can happen

    def __getattr__(self, name):  # only called on lookup MISS
        # reach _cache via object.__getattribute__ semantics already done;
        # but guard against misses on internals to avoid recursion
        if name.startswith("_"):
            raise AttributeError(name)
        client = expensive_build(name)
        self._cache[name] = client
        self.__dict__[name] = client   # next access: plain hit, __getattr__ skipped
        return client

Writing the result into self.__dict__ is the trick worth stealing: the second access is a step-3 instance-dict hit and __getattr__ never fires again — lazy on first touch, native speed after.

Quiz

A class defines `x = property(get_x)` and a careless __init__ does `self.__dict__['x'] = 5` directly. What does `obj.x` return?

Writing a descriptor: validated fields and set_name

A descriptor is any object whose class defines __get__/__set__/__delete__ and which lives in a class attribute (instance attributes are never treated as descriptors). The classic production use is a reusable validated field — write the validation once, attach it to many attributes:

class Positive:
    def __set_name__(self, owner, name):     # called at class creation
        self.slot = "_" + name               # learns its own attribute name

    def __get__(self, obj, objtype=None):
        if obj is None:
            return self                      # class access: Account.balance
        return getattr(obj, self.slot)

    def __set__(self, obj, value):           # __set__ makes it a DATA descriptor
        if value <= 0:
            raise ValueError(f"{self.slot[1:]} must be positive, got {value!r}")
        setattr(obj, self.slot, value)

class Account:
    balance = Positive()    # one line per validated field
    limit = Positive()

    def __init__(self, balance, limit):
        self.balance = balance   # routes through Positive.__set__
        self.limit = limit

__set_name__ (Python 3.6+) is the quiet hero: the descriptor learns which attribute it was assigned to, so you stop passing names twice (balance = Positive("balance")). This is exactly the machinery under Django ORM fields, SQLAlchemy columns, and pydantic-style models: a class-level field object intercepts every read and write, so user.email = "x" can validate, mark the instance dirty for the UPDATE statement, or coerce types — and User.email (note obj is None in __get__) returns the field itself so query DSLs can build User.email == "a@b.c" expressions. The tradeoff is honest: every attribute access becomes a Python-level call, roughly 2–5x slower than a plain instance-dict read — which is why ORMs feel slow in tight loops over a million rows and why bulk paths (values_list, .model_dump) bypass the descriptors.

Quiz

Your validated-field descriptor stores the value with `self.value = value` inside __set__ (on the descriptor itself) instead of on obj. Tests with one instance pass. What breaks in production?

Recall before you leave
  1. 01
    Recite the attribute lookup order for obj.x and explain where property, plain attributes, and methods each win or lose.
  2. 02
    Why is __getattr__ the right hook for lazy loading but a recursion hazard, and how do descriptors plus __set_name__ power ORM fields?
Recap

Attribute access is an algorithm, not a dict read. obj.x invokes type(obj).__getattribute__, which scans the type’s MRO first: a data descriptor there — anything defining __set__ or __delete__, like property, validated fields, or __slots__ slots — answers before the instance __dict__ is even consulted, which is why properties cannot be shadowed and why a smuggled __dict__['x'] entry stays unreachable. Only then does the instance dict win, and below it non-data descriptors: functions define just __get__, so step four is where function.__get__ manufactures bound methods, and why per-instance method overrides work. __getattr__ is a different beast from __getattribute__: the latter intercepts every access (override it and delegate via object.__getattribute__ or recurse), the former fires only after a miss — perfect for lazy loading, with the classic recursion failure when the hook’s own body misses an attribute, and the cache-into-__dict__ trick to make later reads native-speed. Writing a descriptor is four methods: __get__ (return self on class access), __set__ (validation — its presence makes you a data descriptor), optional __delete__, and __set_name__ so the field learns its own name at class creation. That is the entire trick under Django fields, SQLAlchemy columns, and pydantic models — at an honest cost of a Python-level call per access, roughly 2–5x a plain read, which bulk APIs deliberately bypass. Now when you see a RecursionError tracing back through __getattr__, or a property that stubbornly ignores a value you just wrote to self.__dict__, you know which layer of the algorithm is answering — and exactly what to do about it.

Practice

Start at the top. Tasks go easiest → hardest: recall a fact, apply it to a case, then a senior-level stretch. Open one, attempt it, then reveal.

recallapplystretch0 of 6 done

Something unclear?

Ask a question about this lesson. Questions are anonymous and go straight to the author to make the lesson better.

shortcuts expand
search
K
prev piece
k
next piece
j
cycle tier
t
this menu
?
sources2
expand
  1. 01
  2. 02

Trademarks belong to their respective owners. Editorial reference only.