Iterators in Python

Introduction to Iterators in Python

Python is an Object-Oriented Language, everything in python is considered as an object, including variables, functions, lists, tuples, set, etc. There are some objects which are iterable, which means we can traverse over them and they will return their member value one by one.

Examples of iterable objects are: List, Tuple, Dictionary

Just like all other things, iterators in python are also objects. They are used for iterating over iterable objects. This simply means that we can use iterators on lists, tuples, dictionaries, and strings.

In Python, iterators are everywhere, mostly they are hidden in plain sight.

Iterators are implemented using two special methods in Python which are iter() and next(). They are collectively called iterator protocol.

Let’s learn how to use iterators

Using Python Iterators

The iter() method is used to create the iterator over the iterable object. It returns an iterator for the object you give it.

The next() method returns the next item from the iterator.

We can use both these methods to iterate over a variable.

Here is the code explaining how to use iterators in python.

mySecret = ["I", "Love", "Python"]
 
myIter = iter(mySecret)
print(myIter)
 
print(next(myIter))
print(next(myIter))
print(next(myIter))

Output:

<list_iterator object at 0x00000148B32CE4C0>
I
Love
Python

In this code, we can see that printing myIter returns an iterator object.

Once we have the iterator object we use next() to traverse through its values. In this way, we get all the values that are contained in the python iterator.

If we try to run next() again then we will get a StopIteration exception:

Traceback (most recent call last):
  File "D:\filePath\iterators.py", line 12, in <module>
    print(next(myIter))
StopIteration

If we use a variable that is not iterable in the next method then we get 'TypeError'

a = "hello" #defining a string
print(next(a))

Output:

TypeError: 'str' object is not an iterator

How to Make Custom Iterators in Python?

Just like any other thing in python, making a custom iterator from scratch is super easy. For making our own iterator we have to use two dunder/magic methods provided by Python.

Dunder/Magic methods are methods that have two underscores before and after their name; their invocation generally happens internally from a class or some action.

The two dunder methods we have to use are:

1. __iter__()
2. __next__() These are similar to iter() and next().

__iter__() should return the iterator object. If required, some initialization can be performed.

__next__() should return the next item in the sequence. On reaching the end it should raise StopIteration.

Here is the code showing how to do it

class ModOfTwo:
    def __init__(self, max=0):
        self.max = max
 
    def __iter__(self):
        self.n = 0
        return self
 
    def __next__(self):
        if self.n <= self.max:
            result = self.n % 2
            self.n += 1
            return result
        else:
            raise StopIteration
 
 
# creating an object
numbers = ModOfTwo(3)
 
# creating an iterator object
i = iter(numbers)
 
print(next(i))
print(next(i))
print(next(i))
print(next(i))

Output:

0 #0%2
1 #1%2
0 #2%2
1 #3%2

Let’s understand what is happening here.

We made a class ModOfTwo which will give us the mod of the numbers starting from 0 to the number (which is 3 in our case).

In the init() we defined a maximum number, this is the number till which we want to calculate mods.

In the __iter__()we are setting the value of a variable n to 0. We will return the mod of this variable in the __next__().

In __next__() we are first checking if the value of n is less than max or not, otherwise the iterator will become an infinite iterator. After the check, we are calculating and returning the mod and also incrementing the value of n by 1.

This code will work fine, but if we add one more print statement then we will get this error:

Traceback (most recent call last):
  File "D:\filePath\iterators.py", line 43, in <module>      
    print(next(i))
  File "D:\filePath\iterators.py", line 30, in __next__      
    raise StopIteration
StopIteration

This occurred because in the code we have defined if the n is greater than max then raise a StopIteration exception.

This is how we can make custom iterators in Python.

Some examples where we can use custom iterators are, calculating multiple of a number, power of a number.

Iterator vs Iterable in Python

With so much programming jargon it is easy to forget the difference between an Iterator and Iterable.

Here is a simple explanation.

An iterator is an object which consists of __iter__() and __next__() (collectively these are known as iterator protocol).

An iterable is anything we can loop over using a loop.

This is the easiest explanation that will help you differentiate between the two.

How for loop uses Iterators in Python

for is the most common loop in Python. It can be used to iterate over any iterable. To have a better understanding of iterators let’s see how iterators are used inside of for loop in python.

The syntax of for loop is:

nums = [1, 2, 3, 4, 5]
for i in nums:
    print(i)

This is what happens behind the scenes

nums = [1, 2, 3, 4]
nums_iter_obj = iter(nums)
 
while True:
    try:
        print(next(nums_iter_obj))
    except StopIteration:
        break

Output:

1
2
3
4

In this code, we made an iterator object of the iterable nums.

Then we made an infinite while loop in which there is a try-catch block where we stop the loop if we get StopIteration, else it will continue to print the next element in the iterable.

Infinite Iterators in Python

Iterators iterate over an iterable (lists, tuples, string, etc.) and they are finite or they have a fixed length. But iterators can also be infinite or never ending.

These python iterators are capable of running forever and special care needs to be taken while handling these.

Let’s understand Infinite Iterators by making our own infinite iterator.

This will be very similar to the custom iterator(ModOfTwo) we made earlier.

class InfiniteIterator:
 
    def __iter__(self):
        self.n = 0
        return self
 
    def __next__(self):
        result = self.n * 2
        self.n += 1
        return result
 
multiplesOfTwo = InfiniteIterator()
 
i = iter(multiplesOfTwo)
 
print("Multiples of two are:")
print(next(i))
print(next(i))
print(next(i))
print(next(i))
print(next(i))

Output:

Multiples of two are:
0
2
4
6
8
.
.
.
infinite

This code is very similar to the code we wrote while making our custom iterator. Instead of calculating the mod here, we are calculating the multiples of 2.

The main difference is, we do not have the max variable which we were using in the next(). The max variable there was responsible for raising an exception if all the values were traversed. In this code, we are not performing any such check. Because of the absence of that check, this iterator is infinite. It will keep returning you the multiples of 2 forever.

See Also:

• Max() Function in Python.
• While Loop in Python.
• super() function in python.