Handling missing dict keys, revisited

Source: Dev.to

Summary of previous approaches

• Using setdefault
• Using defaultdict
• Implementing __missing__

Subclassing dict: important nuances

• __contains__ does not call __missing__. Therefore, k in d returns False for keys that are not yet set, even if __missing__ would handle them.
• dict.get does not call __missing__. Consequently, d.get(k) will not use your custom default.

Both behaviors are intentional and often desirable. __missing__ is only invoked by the d[key] indexing operation, not by other dictionary methods. The same rule applies to defaultdict: only d[key] triggers the default factory; methods like .get() and the in operator do not.

Overriding get

Overriding get can be tempting, but a naive implementation fails:

class M(dict):
    def __missing__(self, key):
        value = "my default value"
        self[key] = value
        return value

    def get(self, key, default=None):
        try:
            return self[key]  # triggers __missing__ if key is absent
        except KeyError:
            return default    # dead code, never reached

Why it doesn’t work

• self[key] calls __missing__ instead of raising KeyError, so the except block is never executed.
• The default argument becomes ineffective:

>>> m = M()
>>> m.get("x", "fallback")
'my default value'          # expected "fallback", got __missing__ value instead
>>> m
{'x': 'my default value'}  # side effect: key was set in the dict

There is no clean way to honor both the __missing__ default and the default parameter in get because they operate in opposite directions. It is better to accept that get and __missing__ serve different purposes and leave get unchanged.

Overriding __contains__

Overriding __contains__ is a design decision. If your subclass provides a value for every possible key, returning True for all keys can be reasonable:

def __contains__(self, key):
    return True

Be aware that "x" in m will return True even when m is empty ({}), which can be confusing.