Iterator Pattern: Sequentially Accessing Elements Without Exposing the Structure

Sometimes you need to traverse a collection (like a list, tree, or database results) without exposing its underlying representation.

The Iterator Pattern provides a standard way to sequentially access elements of a collection without exposing how that collection is structured internally. This makes it easy to change the collection’s implementation later without breaking the code that iterates over it.

---

Real-World Analogy

Think of a remote control for a TV. You don’t need to know the exact wiring inside the TV to move between channels — you just use the channel up/down buttons.

The TV is the collection, and the remote control is the iterator — giving you a consistent way to navigate without exposing internal details.

---

Real-World Backend Example

Let’s say you have a backend system that processes user activity logs stored in chunks (e.g., in batches of 1000 records).

You want a clean way to iterate through these logs without the caller having to know whether the logs come from a database, file, or API.

Concrete Iterator

from collections.abc import Iterator

class LogIterator(Iterator):
    def __init__(self, logs):
        self._logs = logs
        self._index = 0

    def __next__(self):
        if self._index < len(self._logs):
            log = self._logs[self._index]
            self._index += 1
            return log
        raise StopIteration

Concrete Iterable

from collections.abc import Iterable

class LogCollection(Iterable):
    def __init__(self, source):
        # Simulate loading logs from different sources
        self._logs = self._load_logs(source)

    def _load_logs(self, source):
        # This could be replaced with DB queries, API calls, etc.
        if source == "db":
            return [f"DB log {i}" for i in range(5)]
        elif source == "file":
            return [f"File log {i}" for i in range(3)]
        else:
            return []

    def __iter__(self):
        return LogIterator(self._logs)

Example usage

if __name__ == "__main__":
    db_logs = LogCollection("db")
    print("Iterating over DB logs:")
    for log in db_logs:
        print(log)

    file_logs = LogCollection("file")
    print("\nIterating over File logs:")
    for log in file_logs:
        print(log)

Output:

Iterating over DB logs:
DB log 0
DB log 1
DB log 2
DB log 3
DB log 4

Iterating over File logs:
File log 0
File log 1
File log 2

---

Gotchas & Considerations

1. Encapsulation
   • The Iterator hides the internal structure of the collection, but still allows sequential access.
2. Multiple Iterators
   • You can have multiple iterators over the same collection without them interfering with each other.
3. Lazy Evaluation
   • Iterators can load data on demand, which is great for large datasets (e.g., streaming DB results).
4. Python’s Built-in Iterators
   • In Python, any class implementing
     __iter__()
     and
     __next__()
     can be used in a
     for
     loop. You often don’t need to manually create an iterator unless you need custom traversal logic.

---

When to Use Iterator Pattern

• You want a consistent way to traverse different types of collections.
• You want to change the internal structure of collections without affecting the traversal logic.
• You need multiple simultaneous traversals of the same collection.

---

Final Thoughts

The Iterator Pattern is so common in Python that it’s baked into the language via its iterator protocol (

, ).

It gives you decoupling between the traversal logic and the collection’s internal representation, making your code more flexible and reusable.

In backend systems, this pattern is particularly powerful when working with large datasets, streamed results, or abstracting different data sources behind a unified interface.

Used well, it makes consuming data from complex sources feel as simple as looping through a list.