Data Structures22 min read

Memory Efficient Structures

Learn how to reduce memory usage with generators, iterators, __slots__, and streaming patterns so your data handling stays fast and stable.

David Miller
October 17, 2025
5.3k248

Memory matters when data becomes big.

Common memory mistake

Building huge lists:

nums = [i for i in range(10_000_000)]

Better: generator

nums = (i for i in range(10_000_000))
print(next(nums))

Streaming file lines (classic real-world)

def read_lines(path):
    with open(path, "r") as f:
        for line in f:
            yield line.strip()

slots (reduce per-object memory)

class Person:
    __slots__ = ["name", "age"]

    def __init__(self, name, age):
        self.name = name
        self.age = age

Graph: list vs generator

flowchart LR
  A[List] --> B[Allocates all values]
  C[Generator] --> D[Produces one value at a time]

Remember

  • generators are best for large sequences
  • streaming avoids memory spikes
  • slots helps when creating many objects
#Python#Advanced#Memory