An Introduction to Python Concurrency

Copyright (C) 2009, All Rights Reserved
David Beazley
http://www.dabeaz.com

Presented at USENIX Technical Conference, June 15, 2009

Introduction

This tutorial is an overview of concurrent programming idioms in Python.

Requirements and Support Data Files

This tutorial requires the use of Python 2.6 or newer. No third party modules are required. Examples have been tested on both Unix and Windows XP.

The following file contains some supporting data files that are used by the various code samples. Download this to your machine to work with the examples that appear in the tutorial.

This download also includes a PDF of the lecture slides.

Code Samples

Here are various code samples from the course. You can either cut and paste these from the browser or simply work with them directly in the "Concurrency" directory. The order in which files are listed follow the course material.

Part 3 : Python Thread Programming

  • countdown.py. A very simple thread (a first example).

  • countdown2.py. Launching a function in a thread.

  • race.py. Example of a race condition. Run multiple times to see different results.

Part 4 : Thread Synchronization Primitives

  • lock_ex.py. An example of using a lock to synchronize access to shared data.

  • lock_with.py. An example of using a lock with the Python 2.6 context-manager feature.

  • rlock_ex.py. An example of code-based locking using a reentrant lock.

  • sema_signal.py. An example of using a semaphore for signaling between threads.

  • event_barrier.py. Using an event to implement a synchronization barrier.

  • event_notify.py. Using an event and semaphore to implement a kind of notification mechanism.

  • cond.py. An example of setting up a producer-consumer problem with a condition variable.

Part 5 : Threads and Queues

  • pc_queue.py. A producer-consumer problem using threads and queues.

Part 6 : The Problem with Threads

  • perf.py. A program that illustrates some performance problems with threads.

Part 9 : Processes and Messages

  • channel.py. An example of setting up a message channel between processes using the subprocess module.

Part 10 : Multiprocessing

  • countdownp.py. A very example of launching a process.

  • perfmp.py. A simple performance test of CPU-bound processes. Compare to the thread example earlier.

  • pipemp.py. An example of a multiprocessing producer/consumer with a pipe.

  • queuemp.py. Using multiprocessing queues.

  • poolmp.py. An example of using a multiprocessing pool.

Part 11 : Alternatives to Threads and Processes

  • generator.py. A very simple example of using generators to implement a form of cooperative multitasking.