### Iterators and Iterables in Python 3

Lately I've been examining the difference between iterators and iterables in Python more closely. There's a subtle but distinct difference between the two that's easy to overlook (or simply ignore); however, most every time you've typed a for-loop, such as for i in [1, 2, 3, 4, 5]: print(i), you've used both an iterable and an iterator.

According to the Python 3.6 documentation, iterator types define special methods __iter__ and __next__. Together these form something in Python called the iterator protocol. Moreover, __iter__ typically just returns self, so we only have to worry about writing a definition for __next__. While collections.abc already contains an abstract Iterator base class (see e.g. collections.abc.Iterator.__abstractmethods__), instead of importing it I'm going to write it as follows.
from abc import abstractmethod
from typing import Any

class Iterator:
# c.f. PEP 234, PEP 322

@property
@abstractmethod
def _state(self) -> None:
raise NotImplementedError

@_state.setter
@abstractmethod
def _state(self, value: Any) -> None:
raise NotImplementedError

def __iter__(self) -> 'Iterator':
return self

@abstractmethod
def __next__(self) -> Any:
raise NotImplementedError

Iterator types are lazy[1] and thus only cough up a value when we ask them to (with the keyword next), so I've added an abstract _state property, to be defined more thoroughly in a subclass (this could just be as simple as a field variable though).

Now, a simple iterator example might look as follows.
class SimpleIterator(Iterator):
def __init__(self, start: int =0, stop: int =10) -> None:
self._state = start
self.stop = stop

@property
def _state(self) -> int:
return self.__state

@_state.setter
def _state(self, value: int) -> None:
self.__state = value

def __next__(self) -> int:
if self._state >= self.stop:
raise StopIteration
else:
val = self._state
self._state += 1
return val

if __name__ == '__main__':
itr = SimpleIterator()
for value in itr:
print(value)

# prints 0 ... 9

Note that, when this iterator type raises StopIteration, it continues to do so; i.e., calling next(itr) at any point in the future will raise StopIteration. We can convince ourselves of this with a simple brute-force unittest.
from unittest import main, TestCase

class TestIterator(TestCase):

def setUp(self) -> None:
self.itr = SimpleIterator()

def test_stop_iteration(self) -> None:
# Run iterator to the end
for item in self.itr: pass

# Check to see if it raises StopIteration a hundred times or so
for _ in range(100):
with self.assertRaises(StopIteration):
next(self.itr)

main()

# ----------------------------------------------------------------------
# Ran 1 test in 0.002s
#
# OK

Also, it's not necessary for an iterator to terminate, so the following is fine too.
class SimpleInfiniteIterator(Iterator):
def __init__(self) -> None:
self._state = 0

@property
def _state(self) -> int:
return self.__state

@_state.setter
def _state(self, value: int) -> None:
self.__state = value

def __next__(self) -> int:
""" Just keep adding one """
val = self._state
self._state += 1
return val

Because a non-terminating iterator never raises a StopIteration, the for-loop continues indefinitely (and hence we don't have to worry about testing that it continues to raise such an exception as it never raises it in the first place).

So that's an iterator. Now what's an iterable? And how is it related to iterators?

From what I've found online, I think the most distinct difference is that iterable types can't be exhausted. In other words, every time a for-loop calls iter(iterable), it gets a fresh new iterator. That's why the following code sample doesn't raise any exceptions.
>>> X = list(range(10))
>>> last = X[-1]
>>> # Loop over X multiple times and print its contents
>>> for _ in range(3):
...     for i in X:
...         print(i, end=', ' if i != last else '\n')
...
0, 1, 2, 3, 4, 5, 6, 7, 8, 9
0, 1, 2, 3, 4, 5, 6, 7, 8, 9
0, 1, 2, 3, 4, 5, 6, 7, 8, 9

In this case, we're looping over a list referenced by X three times. Now, let's try overriding list's __iter__ method so that it cycles over items in the list.
class CyclicIterator(SimpleInfiniteIterator):
""" Another cyclic iterator that shadows itertools.cycle
(inherits from SimpleInfiniteIterator to avoid some boilerplate
code)
"""

def __init__(self, some_list: 'CyclicList') -> None:
SimpleInfiniteIterator.__init__(self)
self.some_list = some_list

def __next__(self) -> int:
val = self._state
self._state += 1
self._state %= len(self.some_list)
return self.some_list[self._state]

class CyclicList(list):

def __iter__(self) -> Iterator:
return CyclicIterator(self)

# X = CyclicList([1, 2, 3, 4, 5])
# for i in X: print(i)
#
# prints 1 ... 5 repeatedly

This seems a little strange, however, because I've passed a reference to the iterator (also, my use of __getitem__ on list to get values).