This course is for more experienced Python programmers who like a
challenge and who want to take their Python skills to a whole new
level. Attendees are expected to already be familiar with the core
Python language and common library modules such
as pickle, os, sys, etc. Having some basic
knowledge of network and systems programming is strongly advised.
Dates: March 19-22, 2012.
[ Register |
More Information | FAQ ]
Python Concurrency and Distributed Computing Workshop
[4 days] This one-of-kind workshop, designed for more experienced
Python programmers, provides an in-depth examination of concurrent
programming idioms and standard library modules. The aim of this
workshop is to go far beyond the information found in the manual and
to gain a deeper understanding of how different approaches work, their
performance characteristics, limitations, and types of real-world
problems for which they are suited. Course topics include, but are not
limited to the following:
- Interpreter execution model
- Thread programming
- The global interpreter lock
- Interprocess communication
- Message queues and message passing
- Asynchronous and non-blocking I/O
- Distributed computing
In addition to the above, this course is taught exclusively using
Python 3. Although all of the topics also apply to earlier Python versions,
this is a perfect opportunity to see what's new and try it out.
Do you have to be some kind of freaky expert to attend? Absolutely
not--the whole point of the workshop is to learn. You'll walk away
from the workshop looking at various Python libraries and frameworks in a
whole new light. Plus, the whole topic is just cool.
Here is a general outline for the workshop.
- Introduction. A brief introduction to concurrency and distributed computing topics.
- Thread Programming. Everything you wanted to know about
Python threads, but were afraid to ask. Includes the absolute basics
of using the threading module and effective ways to use
threads in a way that won't make your code (or your head) explode.
Also includes detailed information on the Global Interpreter Lock (GIL),
tuning parameters, and the interaction between threads and C/C++
- Message Passing and Data Serialization. Message
passing is a core component of distributed computation. This
section provides an in-depth look at different interprocess
communication mechanisms, their performance characteristics,
and tuning options. Topics include the subprocess module, named
pipes, network sockets, memory mapped regions, pickle, marshal,
structure packing, and binary I/O. New for 2011: we'll also talk about
using Python with 0MQ.
- Multiprocessing. A tour of features provide by the
multiprocessing library that allow you to take advantage of multiple
CPU cores. Topics will primarily focus on multiprocessing queues and
- Distributed Programming. An in-depth tour of different distributed
programming techniques. Topics include programming with actors, client-server
computing, REST, remote procedure call, and distributed objects.
- Advanced I/O handling. A look at different I/O handling
techniques including blocking, non-blocking, asynchronous, and event-driven
I/O. The primary goal of this section is to better understand the
I/O handling using by different libraries and frameworks such as asyncore, Twisted, etc.
- Generators and Coroutines. An overview of concurrent
programming using generators and coroutines. The major focus of
this section is on using generators to implement user-level task switching
and to better understand libraries based on microthreads, tasklets, green-threads,
and similarly named entities.
The workshop consists of highly detailed presentation slides and
nearly 50 hands on exercises where you will get to
experiment--approximately half the time will be spent working on
programs. For a sample of presentation materials, see my talk on
the Python GIL. If
you are familiar with some of my
past PyCON tutorials,
you should know that the concurrency workshop has almost no overlap.
With the exception of the GIL talk, almost all of the material has not
been presented in any other venue.