Page #7
- Using map()
- Using filter()
- Using reduce()
- Adding arguments to the __init__() of superclass: super()
- Using iter(): The Iteration Protocol
- Getting Largest or Smallest n items of an Iterable
- Executing Python statements dynamically with compile(), exec() & eval()
- Maintaining order in dictionaries
- Using builtin class zip()
- Manipulating Files
Using map()
It is fairly common in programming to iterate over an iterable object, send each element to a function which returns a manipulated version of the element. While this can be done explicitly with a for loop, the builtin class in Python 3 called map allows for shorter and cleaner code to achieve this. The constructor of map class returns an iterator of the results of a function applied to every element of an iterable object. Example:
>>> help(map) ... class map(object) | map(func, *iterables) --> map object | | Make an iterator that computes the function using arguments from | each of the iterables. Stops when the shortest iterable is exhausted. ... >>> def addFive(number): return number + 5 >>> map( addFive, [1, 2, 3, 4] ) <map object at 0x03A1EDD0> >>> aMapObject = map( addFive, [1, 2, 3, 4] ) >>> for result in aMapObject: print(result) 6 7 8 9
The map class is designed to be used in conjunction with Lambda Expressions for tighter code. Lambda Expressions are used to create anonymous functions. In a nutshell, these functions do not have the 'def' keyword, instead have a 'lambda' keyword, take any number of arguments and return a single value in the form of an expression. Click here to know more about Lambda Expressions.
>>> aMapObject = map( lambda x: x + 5, [1, 2, 3, 4] ) >>> next(aMapObject) 6 >>> aMapObject = map( lambda x: x + 5, range(6) ) >>> next(aMapObject) 5
The constructor of the map class can actually take more than one iterable object, in the event that the function takes more than arguments.
>>> anotherMapObject = map( lambda x, y: x + y, [1, 2, 3, 4], [10, 20, 30, 40] ) >>> for element in anotherMapObject: print(element) 11 22 33 44
In the event that the length of the iterable objects is uneven, the population of the to-be-returned iterator stops as soon as the shortest out of the iterable objects is fully traversed.
>>> anotherMapObject = map( lambda x, y: x + y, [1, 2, 3, 4, 5, 6], [10, 20, 30, 40] ) >>> for element in anotherMapObject: print(element) 11 22 33 44
You can cast the returned iterator into a list using the builtin list() function. However, as the documentation says, rather than converting an iterator into a list, it is better to just use a list comprehension especially when lambda expressions are being used to generate the resultant elements.
>>> a = list(map( lambda x: x + 5, [1, 2, 3, 4] )) >>> a [6, 7, 8, 9]
In Python 2, map is a builtin function instead of a class. Its usage is same as in Python 3, except for the fact that in Python 2, it returns a list instead of an iterator.
Using filter()
The builtin filter class does what you expect it to. It filters out the elements which do not match a criteria and creates an iterator out of the elements which match it. The constructor of the filter class accepts a function and an iterable object. The function either returns True or False for an input, and the input which gives a True value gets populated in the iterator returned by the filter class.
>>> help(filter) ... class filter(object) | filter(function or None, iterable) --> filter object | | Return an iterator yielding those items of iterable for which function(item) | is true. If function is None, return the items that are true. ...
Say you wanted to filter out the multiples of 7 lying in the first 30 numbers. You can achieve this using a for loop as well, but the filter class makes for cleaner code.
>>> def multiplesOfSeven(number): return number % 7 == 0 >>> aFilterObject = filter( multiplesOfSeven, range(31) ) >>> for element in aFilterObject: print(element) 0 7 14 21 28 >>> aFilterObject = filter( lambda x: x % 7 == 0, range(30) ) >>> next(aFilterObject) 0 >>> next(aFilterObject) 7 >>> for element in filter( lambda x: x % 7 == 0, range(31) ): print(element) 0 7 14 21 28
If the function provided is None, it creates an iterator only out of true elements of the supplied iterable object i.e. elements which are not 0 and are not empty strings ( '' ).
>>> for number in range(5): print(number) 0 1 2 3 4 >>> aFilterObject = filter( None, range(5) ) >>> next(aFilterObject) 1 # Not 0
Here's another example where the filter class extracts words beginning with the letter 'a' out of a list of words.
>>> listOfWords = ['aardvark', 'apple', 'ball', 'cat']
>>> wordsBeginningWithA = filter( lambda x: x.startswith('a'), listOfWords )
>>> for word in wordsBeginningWithA:
print(word)
aardvark
apple
The filter class, like map class, is designed to be used in conjunction with Lambda Expressions for tighter code. Lambda Expressions are used to create anonymous functions. In a nutshell, these functions do not have the def keyword, instead have a lambda keyword, take any number of arguments and return a single value in the form of an expression. Click here to know more about Lambda Expressions.
Like with the map class, you can cast the returned iterator into a list using the builtin list() function. However, as the documentation says, rather than converting an iterator into a list, it is better to just use a list comprehension especially when lambda expressions are being used to generate the resultant elements.
>>> multiplesOfSeven = list( filter( lambda x: x % 7 == 0, range(30) ) ) >>> multiplesOfSeven [0, 7, 14, 21, 28]
In Python 2, filter is a builtin function instead of a class. Just like map, its usage is same as in Python 3, except for the fact that in Python 2, it returns a list instead of an iterator.
Using reduce()
There are times when you have a sequence of values and you want to reduce it to a single value. The reduce() function in module functools applies a function which operates on 2 elements at a time, to the elements of an iterable object, going from left to right.
>>> import functools
>>> help(functools.reduce)
Help on built-in function reduce in module _functools:
reduce(...)
reduce(function, sequence[, initial]) -> value
Apply a function of two arguments cumulatively to the items of a sequence,
from left to right, so as to reduce the sequence to a single value.
For example, reduce(lambda x, y: x+y, [1, 2, 3, 4, 5]) calculates
((((1+2)+3)+4)+5). If initial is present, it is placed before the items
of the sequence in the calculation, and serves as a default when the
sequence is empty.
Say you have a list of numbers [1, 2, 3, 4, 5] and an anonymous function lambda x, y: x + y. It is a lambda expression which takes two arguments and returns their sum. The reduce() function functools.reduce(lambda x, y: x + y, [1, 2, 3, 4, 5]) will evaluate ((((1 + 2) + 3) + 4) + 5) and return a single value 15.
>>> import functools >>> functools.reduce( lambda x, y: x + y, [1, 2, 3, 4, 5] ) 15 >>> functools.reduce( lambda x, y: x + y, range(6) ) 15 # OTHER EXAMPLES # reducing a list of numbers into their product. >>> def product(a, b): return a * b >>> functools.reduce( product, [1, 2, 3, 4, 5] ) 120 # reducing a bunch of strings into a resultant string >>> def concatenate(a, b): return a + b >>> functools.reduce(concatenate, ['Hello', ' ', 'there!']) 'Hello there!' # finding the largest element in an iterable object >>> functools.reduce( lambda x, y: x if ( x > y ) else y, [10,5,15, 22,14] ) 22
In order to verify the order in which the elements of the iterable object are passed to the specified function, let's make our function verbose.
>>> import functools
>>> def addTwoNumbers(x, y):
print("x = {}; y = {}; x + y = {}".format(x, y, x + y))
return x + y
>>> functools.reduce( addTwoNumbers, [1, 2, 3, 4, 5] )
x = 1; y = 2; x + y = 3
x = 3; y = 3; x + y = 6
x = 6; y = 4; x + y = 10
x = 10; y = 5; x + y = 15
15
Just like the map class, the reduce function is designed to be used in conjunction with Lambda Expressions for tighter code. Lambda Expressions are used to create anonymous functions. In a nutshell, these functions do not have the def keyword, instead have a lambda keyword, take any number of arguments and return a single value in the form of an expression. Click here to know more about Lambda Expressions.
There is an optional argument to the reduce() function, the initialisation value. This becomes the first argument to the specified function, and the first element of the iterable object becomes the second argument to the function.
>>> def addTwoNumbers(x, y):
print("x = {}; y = {}; x + y = {}".format(x, y, x + y))
return x + y
>>> functools.reduce( addTwoNumbers, [1, 2, 3, 4, 5] , 20)
x = 20; y = 1; x + y = 21
x = 21; y = 2; x + y = 23
x = 23; y = 3; x + y = 26
x = 26; y = 4; x + y = 30
x = 30; y = 5; x + y = 35
35
In the event that the iterable object is empty, the reduce() function returns this value without trying out the function. If there is no initialisation value provided and the iterable object is empty, Python raises a TypeError 'TypeError: reduce() of empty sequence with no initial value'.
In Python 2, reduce() is a builtin function instead of a function in module functools. The usage is the same as in Python 3. It is worth noting that the Documentation advises against the usage of reduce() function. As it says, an explicit for loop is more readable.
Adding arguments to the __init__() of superclass: super()
Like all other attributes and methods, the __init__() method of the superclass is inherited by the subclasses. If there is no __init__() method in the subclass, then the __init__() method of the superclass is called when the subclass is instantiated. If the subclass also defines an __init__() method, then it is called instead of the __init__() method of the superclass.
>>> class Automobile(object):
def __init__(self, make):
self.make = make
print("Printing make from inside the base class:", make)
def ignition(self):
print("Igniting...")
>>> class Car(Automobile):
def __init__(self, make):
self.make = make
print("Printing make from inside the derived class:", make)
>>> ferrari = Car('Ferrari')
Printing make from inside the derived class: Ferrari
Now, consider a situation where you want the __init__() method of your subclasses to have additional functionality along with the functionality already defined in the __init__() method of the superclass. To achieve this, you can use the builtin super() function.
>>> class Automobile(object):
def __init__(self, make):
self.make = make
print("Doing something with the make")
def ignition(self):
print("Igniting...")
>>> class Car(Automobile):
def __init__(self, make, horsepower):
super(Car, self).__init__(make)
self.horsepower = horsepower
print("Doing something with the horsepower")
>>> ferrari = Car('Ferrari', 3200)
Doing something with the make
Doing something with the horsepower
super(Car, self).__init__(make) calls the __init__() method of the super class of Car passing the instance as an argument to it, in addition to the positional argument make. We could have easily replaced this statement with Automobile.__init__(self, make), but the super() function helps in decoupling the classes. If tomorrow, the superclass of Car changes from Automobile to, say, Cars, then we will have to change this particular statement inside the Car class as well.
Using iter(): The Iteration Protocol
Sequences and file handlers in Python have an __iter__() magic method. When the sequence is used is in conjunction with a for loop, the __iter__() method of the sequence calls the builtin iter() method and returns an iterator object. This is called the 'The Iterator Protocol'. The iterator object thus received, has a __next__() method, which gives the element in the sequence. When there is no element left and the __next__() method is called, the iterator object raises a StopIteration exception.
>>> numbersListIterator = iter([1, 2, 3])
>>> numbersListIterator
<listiterator object at 0x03040830>
>>> numbersListIterator.__next__()
1
>>> numbersListIterator.__next__()
2
>>> numbersListIterator.__next__()
3
>>> numbersListIterator.__next__()
Traceback (most recent call last):
numbersListIterator.__next__()
StopIteration
Once you have an iterator object, you can iterate over the values in the following 3 ways:
- Using the __next__() magic function of the iterator object, as done above. The __next__() is calling the builtin next() method and passing itself (i.e. numbersListIterator) to it.
- Using the builtin next() function explicitly, such as next(numbersListIterator).
- Using a for loop, such as for number in numbersListIterator: print(number).
Getting Largest or Smallest n items of an Iterable
Python's standard library heapq is an implementation of the heap queue algorithm, or the priority queue algorithm. Using its nsmallest() and nlargest() functions, you can obtain the largest or smallest n items of an iterable.
>>> import heapq
>>> smallestFour = heapq.nsmallest(4, range(10))
>>> smallestFour
[0, 1, 2, 3]
>>> largestThree = smallestFour = heapq.nlargest(3, range(10))
>>> largestThree
[9, 8, 7]
# Both nsmallest() and nlargest() functions provide an optional argument called key, with which you can provide the criteria of comparing items. This can be facilitated using lambda expressions or anonymous functions.
>>> records = [
{'name': 'Ethan', 'rollNo': 1},
{'name': 'Matty', 'rollNo': 2},
{'name': 'Jenna', 'rollNo': 3},
{'name': 'Haley', 'rollNo': 4},
{'name': 'Nathan', 'rollNo': 5}
]
>>> topThree = heapq.nsmallest(3, records, key = lambda records: records['rollNo'])
>>> topThree
[{'name': 'Ethan', 'rollNo': 1}, {'name': 'Matty', 'rollNo': 2}, {'name': 'Jenna', 'rollNo': 3}]
>>> lastTwo = heapq.nlargest(2, records, key = lambda records: records['rollNo'])
>>> lastTwo
[{'name': 'Nathan', 'rollNo': 5}, {'name': 'Haley', 'rollNo': 4}]
Executing Python statements dynamically with compile(), exec() & eval()
Python offers two builtin functions to execute pieces of code put together as a string: eval() & exec(). These functions can be used to execute command inputs from a user, like in a custom interpreter. Optionally, these pieces of code which are in the form of a string, can be fed to the builtin compile() function first, to create a code object (Python bytecode), which can then be handed over to eval() & exec() for execution.
The builtin functions eval() and exec() have 2 similarities and 2 differences. Similarities:
- Both the eval() and exec() evaluate Python statements, which can be in the form of a string, or a code object as returned by the compile() function.
- Both the eval() and exec() take two optional arguments: globals and locals.
These functions differ in the following aspects:
- eval() returns the result of the expression, while exec() does not.
- eval() only evaluates a single expression (anything on the right hand side of an assignment operation), whereas the exec() can take code blocks having loops, try/except, def clauses.
Let's take a look at basic usage of these functions:
>>> a = 10
>>> eval('a + 5')
15
>>> a
10
>>> exec('a + 10')
>>> a
10
>>> exec('a = 20')
>>> a
20
Expand the following code snippet for more examples.
# eval() only evaluates an expression and returns the result. An expression is any combination of operators and operands that you can write on the right hand side of an assignment operation.
>>> a = 10
>>> eval('a + 5')
15
>>> eval('a = 20') # does not evaluate anything other than an expression
Traceback (most recent call last):
eval('a = 20')
File "<string>", line 1
a = 20
^
SyntaxError: invalid syntax
>>> eval( 'def anyFunction(): print(50)' ) # does not evaluate anything other than an expression
Traceback (most recent call last):
eval( 'def anyFunction(): print(50)' )
File "<string>", line 1
def anyFunction(): print(50)
^
SyntaxError: invalid syntax
>>> eval('"Hi"') # has a return value
'Hi'
>>> eval('if 1: print("Hi")') # does not evaluate anything other than an expression
Traceback (most recent call last):
eval('if 1: print("Hi")')
File "<string>", line 1
if 1: print("Hi")
^
SyntaxError: invalid syntax
# exec() supports execution of code-blocks, including statements such as assignment. It does not return any value.
>>> exec('a = 20')
>>> a
20
>>> exec( 'def anyFunction(): print(50)' )
>>> anyFunction()
50
>>> exec('print(1) \nprint(2)')
1
2
>>> exec('if 1: print("Hi")')
Hi
>>> exec('"Hi"') # No output; no return value
In addition to the string form, the statements being passed to exec() & eval() can be in the form of a code object. The compile() function compiles the provided module/statement/expression, and creates a code object (contains Python bytecode) which can be passed to exec() & eval(). This is particularly useful when the same piece of code is being evaluated repeatedly. Expand the following code snippet for usage of compile() function.
# compile(source, '<string>', 'eval') returns the code object which would have been executed if you had executed eval(source).
# source can only be a single expression in this mode.
>>> a = 5
>>> evalCodeObject = compile('a + 10', '<string>', 'eval')
>>> evaluatedValueOfa = eval(evalCodeObject)
>>> evaluatedValueOfa
15
# compile(source, '<string>', 'exec') returns the code object which would have been executed if you had executed exec(source).
# source can be code blocks having loops, try/except, def clauses.
>>> execCodeObject = compile('a = 8; a = a + 10; print(a)', '<string>', 'exec')
>>> executeCodeBlock = exec(execCodeObject)
18
# compile(source, '<string>', 'single') offers a limited functionality of the 'exec' mode, by accepting a single statement or multiple statements separated by semi-colon. It is capable of executing a loop, an if-elif-else construct, a try-except-else-finally construct, a function with semi-colon delimited statements.
>>> singleCodeObject = compile('a = 50; print(a + 4); print(a + 10)', '<string>', 'single') # multiple statements delimited by ;
>>> executeSingleCodeObject = exec(singleCodeObject)
54
60
>>> codeBlock = ''' # an if-else construct
if True:
print("TRUE!")
else:
pass
'''
>>> compiledCodeBlock = compile(codeBlock, '<string>', 'single')
>>> exec(compiledCodeBlock)
TRUE!
>>> codeBlock = ''' # INVALID: two constructs
if True:
print("TRUE!")
else:
pass
if True:
print("TRUE AGAIN!")
else:
pass
'''
>>> compiledCodeBlock = compile(codeBlock, '<string>', 'single')
Traceback (most recent call last):
compiledCodeBlock = compile(codeBlock, '<string>', 'single')
File "<string>", line 7 # referring to the second construct
if True:
^
SyntaxError: invalid syntax
>>> codeBlock = ''' # a function definition with semi-colon delimited statements
def functionOne(): print(6); print(12)
'''
>>> compiledCodeBlock = compile(codeBlock, '<string>', 'single')
>>> exec(compiledCodeBlock)
>>> functionOne()
6
12
>>> codeBlock = ''' # INVALID: two constructs
def functionOne(): print(6); print(12)
def functionTwo(): print(18)
'''
>>> compiledCodeBlock = compile(codeBlock, '<string>', 'single')
Traceback (most recent call last):
compiledCodeBlock = compile(codeBlock, '<string>', 'single')
File "<string>", line 3
def functionTwo(): print(18)
^
SyntaxError: invalid syntax
For further details on arguments of the compile() function, check out this post on Executing Python statements dynamically with compile(), exec() & eval().
Maintaining order in dictionaries
The regular behaviour of Python dictionaries is such that, the order in which the key-value pairs are declared is not necessarily the order in which they are stored in memory. For example:
>>> normalDictionary = {2: 'two', 3: 'three', 1: 'one'}
>>> normalDictionary
{1: 'one', 2: 'two', 3: 'three'}
If you wish to have a dictionary which maintains this order, you can use the OrderedDict class of standard library collections.
>>> import collections
>>> orderedDictionary = collections.OrderedDict({2: 'two', 3: 'three', 1: 'one'})
>>> orderedDictionary
OrderedDict([(1, 'one'), (2, 'two'), (3, 'three')])
>>> dict(orderedDictionary)
{1: 'one', 2: 'two', 3: 'three'}
This class was introduced in version 2.7. If you want similar functionality in older versions, I suggest you look at this page.
Using builtin class zip()
The constructor of builtin class zip takes iterables as inputs amd returns an iterator of tuples of corresponding elements. The population of the iterator stops when the shortest input iterable is exhausted. Since it returns an iterator, the __next__() method is used fetch the next tuple. to Let's look at a few examples:
# TWO ITERABLES AS INPUTS
>>> zipOne = zip( [1, 2, 3, 4], [5, 6, 7, 8, 9] )
>>> zipOne.__next__()
(1, 5)
>>> zipOne.__next__()
(2, 6)
>>> zipOne.__next__()
(3, 7)
>>> zipOne.__next__()
(4, 8)
>>> zipOne.__next__()
Traceback (most recent call last):
zipOne.__next__()
StopIteration
>>> zipOne = zip( [1, 2, 3, 4], [5, 6, 7, 8, 9] )
>>> for element in zipOne:
print(element)
(1, 5)
(2, 6)
(3, 7)
(4, 8)
>>> zipOne = zip( range(5), range(5, 10) )
>>> for element in zipOne:
print(element)
(0, 5)
(1, 6)
(2, 7)
(3, 8)
(4, 9)
>>> zipOne = zip( range(5), range(5, 10) )
>>> zipOne.__next__()
(0, 5)
>>> zipOne.__next__()
(1, 6)
>>> zipOne.__next__()
(2, 7)
>>> zipOne.__next__()
(3, 8)
>>> zipOne.__next__()
(4, 9)
>>> zipOne.__next__()
Traceback (most recent call last):
zipOne.__next__()
StopIteration
# FOUR ITERABLES AS INPUT
>>> zipTwo = zip('Have', 'you', 'met', 'Ted')
>>> for element in zipTwo:
print(element)
('H', 'y', 'm', 'T')
('a', 'o', 'e', 'e')
('v', 'u', 't', 'd')
Manipulating Files
The builtin open() function is used for opening files. This function returns a file object, which has methods like read() and write() associated with it, which are self-explanatory. When we are done manipulating the file, its close() method is called to close it.
# Contents of 'fileToBeReadFrom.txt'
Python is an extremely versatile language.
It is not limited to desktop applications or applications on the web.
People who code in Python are often referred to as "Pythonistas" or "Pythoneers ".
# readingFromAFile.py
# situated in the same directory as fileToBeReadFrom.txt
fileHandler = open('fileToBeReadFrom.txt') # or open('fileToBeReadFrom.txt', 'r')
contents = fileHandler.read()
print(contents)
fileHandler.close()
# OUTPUT in Shell
# Run Menu > Run Module
Python is an extremely versatile language.
It is not limited to desktop applications or applications on the web.
People who code in Python are often referred to as "Pythonistas" or "Pythoneers".
To learn about File Manipulation in detail, I suggest you view this chapter of Python 101.
See also: 50+ Tips & Tricks for Python Developers