Python “magic” methods – part 2

Let’s continue our exploration of Python’s magic methods in this second part of the series. This part will focus on numbers and containers, i.e., collections. You can read the first part here.

Container-related methods

Python provides the usual containers, e.g., lists, sets, and dictionaries. You can use the following methods when you want to implement your own.

Common methods

Containers have a size. Python defines two methods to implement to return the number of items in a container: object.__len__(self) for the exact size and object.__length_hint__(self) for an approximation. You should use the latter when getting the exact size is computationally expensive.

Item-related methods

Containers contain objects. Some containers offer index-based access, e.g., list(1), while others offer key-based access, e.g., dict(‘mykey’). In both cases, here are the methods to implement:

Method
Functionality

object.__getitem__(self, key)
Get the object

object.__setitem__(self, key, value)
Set the object

object.__delitem__(self, key)
Remove the object

object.__missing__(self, key)
Called when the key is not found by the default get(key) implementation

object.__iter__(self)
Return an iterator over items (or keys) in the container

object.__reversed__(self)
Reverse the objects in the container

object.__contains__(self, item)
Check whether an item is part of the container

Let’s create a simple hash-map-like container for illustration purposes:

class Container:

def __init__(self):
self.items = {}

def __getattribute__(self, name):
raise AttributeError()

def __len__(self):
return len(self.items) #1

def __setitem__(self, key, value):
self.items[key] = value #1

def __getitem__(self, key):
return self.items[key] #1

def __delitem__(self, key):
return self.items.pop(key) #1

def __contains__(self, key):
return key in self.items #2

def __iter__(self):
return iter(self.items.keys()) #3

def __reversed__(self):
return iter(reversed(self.items.keys())) #4

container = Container()
container[‘foo’] = ‘foo’
container[‘bar’] = ‘bar’
print(len(container)) #5
for x in container: #6
print(f'{x}: {container[x]}’)
print(‘—‘)
for x in reversed(container): #7
print(f'{x}: {container[x]}’)
print(‘—‘)
del container[‘foo’] for x in container: #8
print(f'{x}: {container[x]}’)
print(‘—‘)
print(‘foo’ in container) #9

Delegate on the items dictionary
Check if the key belongs to items
Get the keys’ iterator
Get the reversed key’s iterator
Print 2 as the container has two items at this point
Implicitly calls the __iter__() method
Implicitly calls the __reversed__() method
Print bar: bar since the foo key has been deleted
Implicitly calls the __contains__() method

Number-related methods

Just as we can emulate containers, we can emulate numbers as well.

Arithmetic methods

Arithmetic methods abound; it’s easier to summarize them in a table:

Kind
Method
Operator/function
Comment

All

object.__add__(self, other)
+

object.__sub__(self, other)
–

object.__mul__(self, other)
*

object.__matmul__(self, other)
@
Matrix multiplication

object.__truediv__(self, other)
/
Regular division

object.__floordiv__(self, other)
//
Division without the reminder

object.__mod__(self, other)
%
Reminder of the division

object.__divmod__(self, other)
divmod()

object.__pow__(self, other[, modulo])
pow()

object.__lshift__(self, other)
<<

object.__rshift__(self, other)
>>

object.__and__(self, other)
&

object.__xor__(self, other)
^
Exclusive OR

object.__or__(self, other)
|
Inclusive OR

Binary

object.__radd__(self, other)
+

object.__rsub__(self, other)
–

object.__rmul__(self, other)
*

object.__rmatmul__(self, other)
@

object.__rtruediv__(self, other)
/

object.__rfloordiv__(self, other)
//

object.__rmod__(self, other)
%

object.__rdivmod__(self, other)
divmod()

object.__rpow__(self, other[, modulo])
pow()

object.__rlshift__(self, other)
<<

object.__rrshift__(self, other)
>>

object.__rand__(self, other)
&

object.__rxor__(self, other)
^

object.__ror__(self, other)
|

Assignement

object.__iadd__(self, other)
+=

object.__isub__(self, other)
-=

object.__imul__(self, other)
*=

object.__imatmul__(self, other)
@=

object.__itruediv__(self, other)
/=

object.__ifloordiv__(self, other)
//=

object.__imod__(self, other)
%=

object.__ipow__(self, other[, modulo])
pow()=

object.__ilshift__(self, other)
<<=

object.__irshift__(self, other)
>>=

object.__iand__(self, other)
&=

object.__ixor__(self, other)
^=

object.__ior__(self, other)
|=

Unary

object.__neg__(self)
–

object.__pos__(self)
+

object.__abs__(self)
abs()
Absolute value

object.__invert__(self)
~
Bitwise NOT

Imagine an e-commerce site with products and stocks of them dispatched in warehouses. We need to subtract stock levels when someone orders and add stock levels when the stock is replenished. Let’s implement the latter with some of the methods we’ve seen so far:

class Warehouse: #1

def __init__(self, id):
self.id = id

def __eq__(self, other): #2
if not isinstance(other, Warehouse):
return False
return self.id == other.id

def __repr__(self): #3
return f’Warehouse(id={self.id})’

class Product: #1

def __init__(self, id):
self.id = id

def __eq__(self, other): #2
if not isinstance(other, Product):
return False
return self.id == other.id

def __repr__(self): #3
return f’Product(id={self.id})’

class StockLevel:

def __init__(self, product, warehouse, quantity):
self.product = product
self.warehouse = warehouse
self.quantity = quantity

def __add__(self, other): #4
if not isinstance(other, StockLevel):
raise Exception(f'{other} is not a StockLevel’)
if self.warehouse != other.warehouse:
raise Exception(f’Warehouse are not the same {other.warehouse}’)
if self.product != other.product:
raise Exception(f’Product are not the same {other.product}’)
return StockLevel(self.product, self.warehouse,
self.quantity + other.quantity) #5

def __repr__(self):
return f’StockLevel(warehouse={self.warehouse},
product={self.product},quantity={self.quantity})’

warehouse1 = Warehouse(1)
warehouse2 = Warehouse(2)
product = Product(1) #6
product1 = Product(1) #6
stocklevel111 = StockLevel(product, warehouse1, 1) #7
stocklevel112 = StockLevel(product, warehouse1, 2) #7
stocklevel121 = StockLevel(product1, warehouse2, 1) #7

print(stocklevel111 + stocklevel112) #8

stocklevel111 + stocklevel121 #9

Define necessary classes
Override equality to compare ids
Override representation
Implement addition. If the warehouse and product don’t match, raise an exception.
Create a new StockLevel with the same product and warehouse and the quantity as the sum of both quantities
Define two products that point to the same id; it’s the same product for equality purposes
Create new stock-level objects
Print StockLevel(warehouse=Warehouse(id=1),product=Product(id=1),quantity=3)
Raise an exception as warehouses are different, though products are the same

Conversion methods

Conversion methods allow changing an instance to a numeric type, i.e., int, float, or complex.

Method
 Built-in function

object.__complex__(self)
complex()

object.__int__(self)
int()

object.__float__(self)
float()

If no such method is implemented, Python falls back to the object.__index__(self), for example, when using the instance as an index.

The following sample, however irrelevant it is, highlights the above:

class Foo:

def __init__(self, id):
self.id = id

def __index__(self): #1
return self.id

foo = Foo(1)
array = [‘a’, ‘b’, ‘c’] what = array[foo] #2
print(what) #3

Define the fallback method
Coerce foo into an int. We didn’t implement any conversion method; Python falls back to index()
Print b

Other methods

Finally, Python delegates to a magic method when your code calls a specific number-related function.

Method
 Built-in function

object.__round__(self[, ndigits])
round()

object.__trunc__(self)
trunc()

object.__floor__(self)
floor()

object.__ceil__(self)
ceil()

Context managers’ methods

Python’s context managers allow fine-grained control over resources that must be acquired and released. It works with the with keyword. For example, here’s how you open a file to write to:

with open(‘file’, ‘w’) as f: #1
f.write(‘Hello world!’)
#2

Open the file
At this point, Python has closed the file

A context manager is syntactic sugar. The following code is equivalent to the one from above:

f = open(‘file’, ‘w’)
try:
f.write(‘Hello world!’)
finally:
f.close()

To write your context manager requires to implement two methods: one for opening the context and one for closing it, respectively, object.__enter__(self) and object.__exit__(self, exc_type, exc_value, traceback).

Let’s write a context manager to manage a pseudo-connection.

import traceback

class Connection:

def __enter__(self):
self.connection = Connection()
return self.connection

def __exit__(self, exc_type, exc_value, exc_traceback):
self.connection = None
if exc_type is not None:
print(‘An exception happened’)
print(traceback.format_exception(exc_type, exc_value, exc_traceback))
return True

def do_something(self):
pass

with Connection() as connection:
connection.do_something()

Callable objects

I was first exposed to callable objects in Kotlin. A callable object looks like a function but is an object:

hello = Hello()
hello(‘world’)

The method to implement to make the above code run is object.__call__(self[, args…]).

class Hello:

def __call__(self, who):
print(f’Hello {who}!’)

Conclusion

The post concludes our 2-part series on Python “magic” methods. I didn’t mention some of them, though, as they are so many. However, they cover the majority of them.

Happy Python!

To go further:

Special method names
PEP 560 – Core support for typing module and generic types

Originally published at A Java Geek on October 22nd, 2023

The post Python “magic” methods – part 2 appeared first on foojay.