Python Multiprocessing

Multiprocessing is for CPU-heavy tasks such as:
- image processing
- video encoding
- large computations
- machine learning preprocessing

      Threads are limited in Python for CPU work (because of the GIL), but processes bypass that because each process has its own interpreter.
      
      ## Basic Process
      
      ```python
      from multiprocessing import Process
      
      def worker(name):
          result = sum(i * i for i in range(1_000_000))
          print(f"{name} finished with result {result}")
      
      if __name__ == "__main__":
          p1 = Process(target=worker, args=("Worker-1",))
          p2 = Process(target=worker, args=("Worker-2",))
      
          p1.start()
          p2.start()
      
          p1.join()
          p2.join()
      
          print("All processes done!")
      ```
      
      ## Why the if __name__ == "__main__" guard matters
      
      On Windows and some environments, without this guard, your process can restart itself infinitely.
      
      Always include it.
      
      ## Using Pool for many items
      
      ```python
      from multiprocessing import Pool
      
      def square(n):
          return n * n
      
      if __name__ == "__main__":
          with Pool(4) as pool:
              results = pool.map(square, range(10))
          print(results)
      ```
      
      ## Sharing results using Queue
      
      ```python
      from multiprocessing import Process, Queue
      
      def worker(q, name):
          q.put(f"Done by {name}")
      
      if __name__ == "__main__":
          q = Queue()
          procs = []
      
          for i in range(3):
              p = Process(target=worker, args=(q, f"P{i}"))
              p.start()
              procs.append(p)
      
          for p in procs:
              p.join()
      
          while not q.empty():
              print(q.get())
      ```
      
      ## Simple performance mental model
      
      ```mermaid
      flowchart LR
        A[CPU Work] --> B[Threading]
        B --> C[GIL limitation]
        A --> D[Multiprocessing]
        D --> E[Multiple CPU cores]
      ```
      
      ## Key points
      
      - Use multiprocessing for CPU-heavy tasks
      - Use async for I/O-heavy tasks
      - Always use main guard
      - Pool is great for batch processing