Intermediate Object Oriented Programming in Python
Intermediate Object Oriented Programming in Python
Hi, and welcome back to Object Oriented Python. In this chapter, we'll look at Intermediate Object Oriented Programming in Python.
- Implicit method calls behind syntax
- Operator Overloading
- Special Methods: Attribute functions
- Customizing builtin data types
- Enforcing Encapsulation: The property class
- Naming Convention of Variables/Attributes
- Behind the scenes of the with statement
- @classmethod & @staticmethod Decorators
- Storing keyword arguments dynamically
- Creating random objects of subclasses of a superclass
- Popular Programming Practices
- Exercise: Changing default index behaviour of Python lists
- Helpful Links
- Where to go from here
We have been over the basics of Object Oriented Programming in Python in the earlier two chapters. In this chapter, I'll try to explain a few more important topics. By the end of this chapter, we would have concluded this primer on Object Oriented Programming in Python. Here we go.
Implicit method calls behind syntax§
We have already seen how printing an object invokes the __str__() method of the object, how the __init__() method is called at the time of object creation and so on. There are many more magic methods that are invoked implicitly. For example, when you are adding two integers, the __add__() method is called, and when a list item is being accessed using square brackets ( [ ] ), the __getitem__() method is called. Here's a comprehensive list of these methods (in Python 3) along with the events on which they are called.
IMPLICIT METHOD CALLED CORRESPONDING SYNTAX __add__ a + b __sub__ a - b __mul__ a * b __mod__ a % b __pow__ a ** b __truediv__ a / b __floordiv__ a // b __gt__ a > b __lt__ a < b __ge__ a >= b __le__ a <= b __and__ a & b __or__ a | b __xor__ a ^ b __eq__ a == b __ne__ a != b __lshift__ a << b __rshift__ a >> b __contains__ a in b, a not in b __pos__ +a __neg__ -a __invert__ ~a __abs__ abs(a) __len__ len(a) __getattribute__ objectName.attributeName __setattr__ objectName.attributeName = value __delattr__ del objectName.attributeName __getitem__ a[i] __setitem__ a[i] = v __delitem__ del a[i] __hash__ hash(a) __bool__ bool(a) __divmod__ divmod(a, b) __call__ objectName() __iter__ iter(a) __reversed__ reversed(a) __int__ int(a) __float__ float(a) __index__ operator.index(a) __sizeof__ sys.getsizeof(a) __trunc__ math.trunc(a) __format__ format(a)
Operator Overloading§
Now that we are familiar with the implicit calls, we can take a step further on how to override these magic methods. For example, the + operator adds two objects by default. Consider a situation where you have two lists of 4 elements each.
>>> listOne = [1, 2, 3, 4] >>> listTwo = [5, 6, 7, 8] >>> print(listOne + listTwo) [1, 2, 3, 4, 5, 6, 7, 8] >>> print(listOne.__add__(listTwo)) [1, 2, 3, 4, 5, 6, 7, 8]
By default, the + operator takes 4 elements of the first list and 4 elements of the second list to construct another list of 8 elements. By overriding the __add__() method, we can add corresponding elements of each list. That is [1, 2, 3, 4] + [5, 6, 7, 8] = [6, 8, 1, 12]. This is one case, you can make the __add__() method do whatever you want.
>>> class MyList(object): def __init__(self, listOfThisObject): self.listOfThisObject = listOfThisObject # stores the list passed during creation of a MyList object def __add__(self, anotherMyListObject): tuplesOfCorrespondingElements = zip(self.listOfThisObject, anotherMyListObject.listOfThisObject) # creates an iterator of tuples of corresponding elements: (1, 5), (2, 6), (3, 7), (4, 8) resultantList = [ a + b for a, b in tuplesOfCorrespondingElements ] # a list comprehension adding corresponding elements of two lists return MyList(resultantList) # returns a MyList object with resultant list def __str__(self): return str(self.listOfThisObject) # __str__() must return a string >>> listOne = MyList([1, 2, 3, 4]) >>> listTwo = MyList([5, 6, 7, 8]) >>> print(listOne + listTwo) [6, 8, 10, 12]
The constructor of builtin zip class takes iterables as inputs and returns an iterator of tuples of corresponding elements. The population of the iterator stops when the shortest input iterable is exhausted. Since it returns an iterator, the __next__() method is used fetch the next tuple.
We have overridden the __add__() method here to suit our needs. Similarly, you are free to override the magic methods to get them to do as you please.
Special Methods: Attribute functions§
Let's begin by doing a quick dir() on the class object.
>>> print( "\n".join( [ attribute for attribute in dir(object) ] ) ) __class__ __delattr__ __dir__ __doc__ __eq__ __format__ __ge__ __getattribute__ __gt__ __hash__ __init__ __le__ __lt__ __ne__ __new__ __reduce__ __reduce_ex__ __repr__ __setattr__ __sizeof__ __str__ __subclasshook__
We already know that each class in Python inherits from the class object. These include the basic data types (str, int etc.) and data structures (list, dict etc.). Let's talk about the attribute functions: __getattribute__, __setattr__, __delattr__.
- __getattribute__() is called when an attribute is being accessed. objectName.attributeName calls __getattribute__() in the form objectName.__getattribute__('attributeName') which further calls the builtin function getattr() in the form getattr(objectName, 'attributeName'). That is:
objectName.attributeName [calls] objectName.__getattribute__('attributeName') [calls] getattr(objectName, 'attributeName').
objectName.attributeName = value [calls] objectName.__setattr__('attributeName', 'value') [calls] setattr(objectName, 'attributeName', 'value')
del objectName.attributeName [calls] objectName.__delattr__('attributeName') [calls] delattr(objectName, 'attributeName')
>>> class Person(object):
def __init__(self, name):
self.name = name
>>> ethan = Person('Ethan')
>>> getattr(ethan, 'name')
'Ethan'
>>> setattr(ethan, 'name', 'Ethan Hunt')
>>> ethan.__getattribute__('name')
'Ethan Hunt'
>>> del ethan.name
>>> ethan.name
Traceback (most recent call last):
ethan.name
AttributeError: 'Person' object has no attribute 'name'
>>> setattr(ethan, 'name', 'Ethan')
>>> ethan.name
'Ethan'
>>> ethan.__delattr__('name')
>>> ethan.name
Traceback (most recent call last):
ethan.name
AttributeError: 'Person' object has no attribute 'name'
>>> setattr(ethan, 'name', 'Ethan')
>>> delattr(ethan, 'name')
>>> ethan.name
Traceback (most recent call last):
ethan.name
AttributeError: 'Person' object has no attribute 'name'
Customizing builtin data types§
str, int, dict, tuple, list, float, bool, set all these are Python classes. This means that we can subclass them and devise our own classes which behave the same way as the built-in types, and at the same time, give us the option of customize selected functionality.
>>> class MyInt(int):
def __add__(self, value):
print("Adding...")
return int.__add__(self, value)
>>> a = MyInt(5)
>>> a + 6
Adding...
11
Enforcing Encapsulation: The property class§
We have seen how methods can be used to set and get attributes. However, they don't strictly impose encapsulation, since attributes can be get and set without these getter-setter methods as well. Pythonic convention is that the users are trusted to access the attributes via getters-setters rather than mangling them directly. It is the Pythonic way.
An anti-Pythonic way to enforce encapsulation is to use the builtin property class and its decorator @property. The property class returns a property object. It is used to manage attributes. Its constructor takes four arguments:
- getter method of attribute x (keyword argument 'fget', defaults to None)
- setter method of attribute x (keyword argument 'fset', defaults to None)
- deleter method of attribute x (keyword argument 'fdel', defaults to None)
- docstring (help text) of attribute x (keyword argument 'doc', defaults to None)
The help(property) reveals a handy example. Consider a class C with an attribute 'x':
>>> class C(object): def getx(self): return self._x def setx(self, value): self._x = value def delx(self): del self._x x = property(getx, setx, delx, "I'm the 'x' property.") >>> c = C() >>> c.x = 5 # invokes the setter setx() >>> c.x # invokes the getter getx() 5 >>> del c.x # invokes the deleter delx() >>> help(C) # Among other help information | x | I'm the 'x' property.
The @property decorator offers an easy-on-the-eye way of achieving the above functionality:
>>> class C(object): @property def x(self): "I am the 'x' property." return self._x @x.setter def x(self, value): self._x = value @x.deleter def x(self): del self._x # @property implies that the function declared below (x) is passed to the property constructor as its first argument, return value (a property object) of which is assigned to the function itself i.e. x = property(fget = x). @x.setter translates to x = property(fset = x), the x on the right is the function declared after the @x.setter notation. Similarly, @x.deleter translates to x = property(fdel = x). # The 'doc' keyword argument of the property constructor gets set to the docstring of the function after the @property notation. So, in actuality, the @property gets translated to x = property(fget = x, fset = None, fdel = None, doc = "I am the 'x' property.")
Don't get confused by the attribute x and the property object x. The example in the documentation is ambiguous. The property object provides us with a handle to the attribute x. Here's a much cleaner example:
>>> class C(object): @property def propertyObjectOfAttrX(self): "I'm the 'x' property." return self._x @propertyObjectOfAttrX.setter def propertyObjectOfAttrX(self, value): self._x = value @propertyObjectOfAttrX.deleter def propertyObjectOfAttrX(self): del self._x >>> c = C() >>> c.propertyObjectOfAttrX = 5 >>> c.propertyObjectOfAttrX 5 >>> del c.propertyObjectOfAttrX
Note that in order for the @property decorator to work, the names of additional functions (setter and deleter) must be the same as the name of the property object i.e. the getter propertyObjectOfAttrX. Otherwise, you'll get an Attribute Error(s). I'll leave that to you to explore.
Read-only attribute
To make an attribute read-only, only specify the getter method of the attribute.
>>> class C(object): def __init__(self): self._x = 2 @property def propertyObjectOfAttrX(self): "I'm the 'x' property." return self._x >>> c = C() >>> c.propertyObjectOfAttrX 2 >>> c.propertyObjectOfAttrX = 10 Traceback (most recent call last): c.propertyObjectOfAttrX = 10 AttributeError: can't set attribute
This happens because the @property decorator call translates to propertyObjectOfAttrX = property(fget = propertyObjectOfAttrX, fset = None, fdel = None, doc = "I'm the 'x' property."). Unless the user knows the variable name( i.e. _x), he/she cannot set it. This is how the property objects enforce encapsulation.
Naming Convention of Variables/Attributes§
There are 4 conventions followed while naming variables in Python:
1. Dunder attributes/Magic attributes such as __doc__, __name__ etc. are named with double underscores wrapped around the attribute name. It is general consensus to use the dunder attributes that Python is shipped with, and not create new ones.
2. Variables/Attributes intended for private use by the class or module are prefixed with a single underscore. In the example shown while demonstrating the property class, the attribute _x is a private variable. Note that this is merely a convention, naming an attribute this way does not make it unusable by external classes and modules. It is merely a hint for fellow developers.
3. Variables/Attributes intended for public use should be named in lower case i.e. variable_name. Note that this is also a convention only, you need not follow it strictly. You can opt for camelCasing as well, like I have done in examples all across the djangoSpin website. The important thing is that you know that this is a popular convention, and while writing professional Python code, you should use lower case variable names with underscores rather than camel casing.
4. Variables/Attributes that are intended for private use by the class or module and not open for subclassing are named with double underscores in the beginning i.e. __variable_name. These can be accessed using a special syntax as shown below.
>>> class Man(object):
def __init__(self):
self.var_one = 1 # public variable
self._var_two = 2 # private variable, open for subclassing and external use
self.__var_three = 3 # private variable, not open for subclassing and external use
self.__var_four__ = 4 # dunder variable
>>> ethan = Man()
>>> ethan.__dict__
{'_var_two': 2, '__var_four__': 4, 'var_one': 1, '_Man__var_three': 3}
>>> _Man__var_three
Traceback (most recent call last):
_Man__var_three
NameError: name '_Man__var_three' is not defined
>>> Man._Man__var_three
Traceback (most recent call last):
Man._Man__var_three
AttributeError: type object 'Man' has no attribute '_Man__var_three'
>>> ethan._Man__var_three # correct syntax to access a private variable which is not open for subclassing and external use
3
Behind the scenes of the with statement §
If you have taken Python 101 course or done any sort of file handling, you might be familiar with the with statement. It is used in situation where resources need to be closed, or some clean-up actions need to be undertaken. A common occurrence of this statement is during file handling operations, where files need to be closed after they are no longer needed. Its other uses include closing network connections, handling exceptions & deleting files that are no-longer.
# WITHOUT WITH STATEMENT
>>> fH = open('fileToBeWrittenInto.txt', 'w')
>>> # a few data manipulation statements
>>> fH.closed
False
>>> fH.close()
# USING WITH STATEMENT
>>> with open('fileToBeWrittenInto.txt', 'w') as fH:
# some data manipulation statements
pass
>>> fH.closed
True
Behind the scenes, the with statement uses the __enter__() and __exit__() methods of the file object.
>>> dir(fH) ['_CHUNK_SIZE', '__class__', '__del__', '__delattr__', '__dict__', '__dir__', '__doc__', '__enter__', '__eq__', '__exit__', '__format__', '__ge__', '__getattribute__', '__getstate__', '__gt__', '__hash__', '__init__', '__iter__', '__le__', '__lt__', '__ne__', '__new__', '__next__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__sizeof__', '__str__', '__subclasshook__', '_checkClosed', '_checkReadable', '_checkSeekable', '_checkWritable', '_finalizing', 'buffer', 'close', 'closed', 'detach', 'encoding', 'errors', 'fileno', 'flush', 'isatty', 'line_buffering', 'mode', 'name', 'newlines', 'read', 'readable', 'readline', 'readlines', 'seek', 'seekable', 'tell', 'truncate', 'writable', 'write', 'writelines'] >>> '__enter__' in dir(fH) True >>> '__exit__' in dir(fH) True
Let's see what happens when we try to use a user-defined class in the with statement:
>>> class Man(object):
def growUp(self):
print("Growing up...")
>>> with Man() as ethan:
ethan.growUp()
Traceback (most recent call last):
with Man() as ethan:
AttributeError: __exit__
In order to make your class compatible with the with statement, all you need to do is define these methods in your class in the following way:
>>> class Man(object):
def __enter__(self):
print("Inside the __enter__() method.")
return self
def __exit__(self, exceptionType, exceptionValue, traceback):
print("Inside the __exit__() method.")
def growUp(self):
print("Growing up...")
>>> with Man() as ethan:
ethan.growUp()
Inside the __enter__() method.
Growing up...
Inside the __exit__() method.
The __enter__() gets executed first, then the with block is executed, then the __exit__() gets executed.
Note that the __enter__() method must return the object on which the operations are being performed in the with block i.e. self.
The 3 positional arguments in the __exit__() method are useful in case any exception is raised in the with block. Let's put an error-raising statement in the with block and see what happens:
>>> class Man(object):
def __enter__(self):
print("Inside the __enter__() method.")
return self
def __exit__(self, exceptionType, exceptionValue, traceback):
print("Inside the __exit__() method.")
print("{} | {} | {}".format(exceptionType, exceptionValue, traceback))
def growUp(self):
print("Growing up...")
>>> with Man() as ethan:
ethan.growUp()
print(someUndeclaredAttribute)
print("Printing afterwards....")
Inside the __enter__() method.
Growing up...
Inside the __exit__() method.
<class 'NameError'> | name 'someUndeclaredAttribute' is not defined | <traceback object at 0x02F5E698>
Traceback (most recent call last):
print(someUndeclaredAttribute)
NameError: name 'someUndeclaredAttribute' is not defined
Note how the __exit__() method is triggerred as soon as the exception is raised. The subsequent print statement in the with block is not executed.
The with statement is extremely handy when clean-up actions need to be defined, whether an exception is thrown or not.
@classmethod & @staticmethod Decorators§
Python offers these decorators to explicitly tell the interpreter that the first argument to a method is not necessarily a reference to the instance. These can be done without, but since Python has provided them, let's see how they work.
A regular instance method perceives its first argument as the object on which the method is being called. It can operate on class attributes as well as instance attributes. Let's recapitulate the example we saw in the first chapter.
>>> class Toy:
'''Toy class'''
count = 0
def __init__(self, name, color):
'''sets instance attributes to provided values; increments counter and prints it.'''
self.name = name
self.color = color
Toy.count += 1
print("Toys manufactured so far:", Toy.count)
>>> woody = Toy('Woody', 'Brown')
Toys manufactured so far: 1
>>> buzz = Toy('Buzz Lightyear', 'White & Purple')
Toys manufactured so far: 2
@classmethod
A method following the @classmethod decorator will perceive its first argument to be the class, and not the instance. The @classmethod decorator denotes a method that operates on the class attributes rather than instance attributes. Let's segregate the initializiation and count increment operation into two different methods to demonstrate this.
>>> class Toy:
'''Toy class'''
count = 0
def __init__(self, name, color):
'''sets instance attributes to provided values; increments counter and prints it.'''
self.name = name
self.color = color
self.incrementCount()
@classmethod
def incrementCount(cls):
cls.count += 1
print("Toys manufactured so far:", cls.count)
>>> woody = Toy('Woody', 'Brown')
Toys manufactured so far: 1
>>> buzz = Toy('Buzz Lightyear', 'White & Purple')
Toys manufactured so far: 2
Note that the class method can be invoked by the object as well. We can call it using the Toy.incrementCount() notation as well, but that would defeat the purpose of the example. Also, the argument can be called anything apart from 'cls', it makes more sense to call it something that is synonymous to the word 'class', as class is a keyword.
@staticmethod
A method following the @staticmethod decorator will NOT perceive its first argument to be the class or the instance. Rather, it will take the first argument to mean a regular positional argument. This is used to denote utility methods, which belong in the class code, but don't operate on the instance or the class. In the below example, checkForName() performs a validation, something like a utility method. It does not operate the instance or the class, and hence neither of these needs to be passes as an argument to it.
>>> class Toy:
'''Toy class'''
count = 0
def __init__(self, name, color):
'''sets instance attributes to provided values; increments counter and prints it.'''
self.name = name
self.color = color
self.checkForName(self.name)
@staticmethod
def checkForName(name):
if name.startswith('w') or name.startswith('W'):
print("Hey! This is a random check to see if your name begins with 'W', and it does!")
else:
print("Oh no! Your name does not start with W, you just missed out on a goodie!")
>>> woody = Toy('Woody', 'Brown')
Hey! This is a random check to see if your name begins with 'W', and it does!
>>> buzz = Toy('Buzz Lightyear', 'White & Purple')
Oh no! Your name does not start with W, you just missed out on a goodie!
Storing keyword arguments dynamically §
We can utilize the builtin dictionary of keyword arguments of each object (stored in __dict__ attribute of the object) to store the keyword arguments supplied to it during its creation.
>>> class Person(object):
def __init__(self, **kwargs):
for argument in kwargs.keys():
self.__setattr__(argument, kwargs[argument])
>>> ethan = Person(name = 'Ethan', age = 23, height = 170)
>>> ethan.__dict__
{'age': 23, 'name': 'Ethan', 'height': 170}
Creating random objects of subclasses of a superclass §
Using the random module and generators, we can have Python make random objects of subclasses of a superclass.
>>> import random
>>> class Crop(object):
def sow(self):
print("Sowing...")
def irrigate(self):
print("Irrigating...")
def harvest(self):
print("Harvesting...")
>>> class Wheat(Crop): pass
>>> class Corn(Crop): pass
>>> class Tomato(Crop): pass
>>> def cropGenerator(numberOfInstancesToCreate):
crops = Crop.__subclasses__()
for number in range(numberOfInstancesToCreate):
yield random.choice(crops)()
>>> cropGeneratorObject = cropGenerator(5)
>>> for cropObject in cropGeneratorObject:
print(cropObject)
<__main__.Corn object at 0x02E7E950>
<__main__.Corn object at 0x02E65BB0>
<__main__.Wheat object at 0x02B09EB0>
<__main__.Tomato object at 0x02E65BB0>
<__main__.Corn object at 0x02B09EB0>
Popular Programming Practices§
We are nearing the end of this course on Object Oriented Programming in Python. We have viewed majority of the knowledge there is on the subject, but knowing the concepts isn't enough. If you long to employ Python in a professional setup, you will find that adhering to certain guidelines goes a long way in writing and maintaining code, especially in a team. I am listing a few popular programming practices here, feel free to research some more.
- PEP 8: Python Enhancement Proposals are proposals to make Python better, either by reducing redundant functionality, or adding more, or guideline documents. PEP 0 maintains a list of all PEPs. PEP 8 is titled 'Style Guide for Python Code'. It lists coding conventions used while coding in Python, regarding naming, code layout, comments etc. It is worth checking out. Again, it's worth stressing that these are only guidelines, and if any convention you are currently employing seems better to you, feel at ease to stick to it.
- Docstrings: Docstrings are discussed in PEP 8, but I want to emphasize on it. Docstrings are invoked by using the __doc__ attribute of functions, classes, modules and help to provide a brief statement about the functionality of the entity.
- Clarity: Focus on clarity of your code. Guido van Rossum, the creator of Python, has said that code is read much more often than it is written.
- Performance: The Zen of Python (>>> import this) states "There should be one-- and preferably only one --obvious way to do it". If your solution seems to take more time than you would expect, revisit your problem and try to come up with a faster solution.
- Maintenance: Code should have appropriate comments and should follow naming conventions so that it is easy for anyone to make changes to it. Having said that, do not have comments that contradict the functionality, or which are irrelevant. Make it a habit to change the comments if any change has been made to the code.
- Encapsulation: Python ideologies dictate that the user is supposed to do the right thing. You can trust the user that he will not mangle with the data he is not supposed to access. Even though you can enforce encapsulation with the property class, you can trust the user.
- Pseudo code with pass: While creating a draft version of your solution to a problem, you may use the pass keyword to avoid writing the actual code in functions and classes. I demonstrated this in the DIY section of the first chapter. The pass keyword allows you to design a high level structure of your program, which you can enhance bit by bit.
Exercise: Changing default index behaviour of Python lists§
In this chapter, we covered several handy concepts that will enhance your understanding of Object Oriented Programming in Python. Implicit method calls behind syntax and how to override them, Operator Overloading, Customizing builtin data types, property class, naming convention of attributes, with statement, storing keyword arguments dynamically and popular programming practices. Your end-of-chapter task is a fairly simple one. We are all familiar with how indexing in lists in Python starts from 0. All you need to do is to create your own list class which starts indexing from 1 and not from 0 i.e. YourList[1] should refer to the first element in the list, YourList[2] should refer to the second element and so on. Be sure to follow popular programming practices, they will only make your code better.
Here are two implicit method calls which you must know before you can proceed with the solution:
- a[i] invokes the __getitem__() method of class list. The __getitem__() takes a single argument i.e. index.
- a[i] = v invokes the __setitem__() method of class list. The __setitem__() takes 2 arguments, index and the value to be set at that index.
You need to override these two methods in a class inheriting the class list.
I'm providing the solution below, for you to tally:
>>> class CustomList(list):
def __getitem__(self, index):
if index == 0:
raise IndexError("Oops! Our indexes start from 1.")
if index > 0:
index -= 1
return list.__getitem__(self, index)
def __setitem__(self, index, value):
if index == 0:
raise IndexError("Oops! Our indexes start from 1.")
if index > 0:
index -= 1
list.__setitem__(self, index, value)
>>> aList = CustomList(['One', 'Two', 'Three', 'Four'])
>>> aList[1]
'One'
>>> aList[2] = 'Dos'
>>> aList
['One', 'Dos', 'Three', 'Four']
Helpful Links§
- dir()
- Comprehensions
- Iterators & Generators
- **kwargs
- zip()
- __repr__() & __str__()
- Serialization in Python
- Replacing Functions in Instance or Class
- Exception Handling
Where to go from here§
Object Oriented Programming is an extremly popular programming paradigm, perhaps the most. It helps you modularize your code and store data in an organized fashion. Furthermore, on a professional front, once you get to grips to these concepts, you can apply for a full-time job in Python. You don't need to be a know-it-all to try it, start small and learn along the way. It is very easy to get overwhelmed by the terms used, but if you stay put, you'll reap the benefits that OOP has to offer.
If you are interested to do some more Object Oriented Programming in Python, I suggest you have a look at Design Patterns in Python.
If you have not checked out Python 101, be sure to do so. It is a comprehensive course covering the basics of Python.
If you have any query regarding Python, put it in the comments below and we will get back to you. Additionally, you can refer to Python Mailing Lists, or Python Internet Relay Chat Discussion, and last but not the least, Google.
Bidding adieu. Cheerio!
See also:
- Python 101
- 50+ Handy Standard Library Modules
- 60+ Handy Third Library Modules
- 50+ Tips & Tricks for Python Developers
- 50+ Know-How(s) Every Pythonista Must Know