Dave Beazley's mondo computer blog. [ homepage ]

Sunday, August 09, 2009


Essential Misconceptions

A few days ago, Mike Riley posted a great review of the new "Python Essential Reference, 4th Edition" on Dr. Dobb's CodeTalk. In that review, he writes:

"While the author could have taken the easy path of regurgitating the online documentation, he has instead reworked the explanation for each class and function call in the Python core library with commendable clarity, frequently accompanying these detailed examinations with extremely useful and meaningful code examples. The book is also very well designed and organized, making it a snap to find information within a matter of seconds."

This is a reviewer who really gets what this book is about. However, for every great review like this, I also encounter comments that simply dismiss the book out-of-hand saying it "offers nothing" over Python's online documentation. With all due respect to Python's fine documentation, I beg to differ.

First and foremost, I've always viewed the Python Essential Reference as a serious programming reference for myself (yes, I always have a copy next to my desk and I use it regularly). Although, I will admit that Python certainly has a lot of online documentation, it's also missing a lot of essential details. For example, I can't count the number of times I've looked at the online documentation for something only to have to go out and do some kind of extended Google search to fill in a missing detail (or worse, having to load the source code for some module and look through it).

Let's look at an example. Suppose you're writing some networking code with the socket module and you want to use the recv(bufsize [, flags]) method of a socket. If you head off to the online documentation you will certainly find some information.

"Receive data from the socket. The return value is a string representing the data received. The maximum amount of data to be received at once is specified by bufsize. See the Unix manual page recv(2) for the meaning of the optional argument flags; it defaults to zero."

Yes, this is all very useful. Especially that part about having to refer to a Unix man page. I'm sure the Windows programmers find that especially useful. If you turn to the Essential Reference p. 483, you'll not only find a description, but you will also get a complete table showing you exactly what can be given for flags along with a brief description of each option. This approach is found throughout the book--with few exceptions are readers simply referred to other documentation. As another example, I would challenge anyone to effectively use something like the setsockopt() or getsockopt() methods of a socket using nothing by Python's online docs.

The other thing that I've tried to do in the book is answer all sorts of questions about tricky interactions between different parts of Python. Take, for example, this question: Can a separate execution thread safely close a generator/coroutine function by invoking the generator's close() method? Sure, that's not the kind of question that comes up every day, but if you know a thing or two about generators and coroutines, you'll know that they are often used in the context of concurrent programming, just like threads. Not only that, threads and generators might be used together (for example, using threads to carry out blocking operations). Thus, it is reasonable to assume that programmers working with both threads and generators in the same program might start to wonder about their possible interaction. I know I did.

If you try to find an answer to this question using the online documentation, you will be searching for some time and probably come up with nothing. Although there is plenty of discussion about generators, the yield statement, and other matters, you really don't find much about generators and threads mixed together. Even PEP 342, the official specification that introduced the generator close() method says nothing on this matter.

Now, let's look at the Essential Reference. First, if you turn to the index and look up "Threads", you will find about a half-page of subentries. In fact, there is even an entry labeled "Threads: close() method of generators, p. 104." If you turn to p. 104, you will find a sentence "if a program is currently iterating on a generator, you should not call close() asynchronously on that generator from a separate thread of execution or from a signal handler."

This is certainly not the only example, but there are a wide variety of similar questions that I try to address. For example, can you use a decorator with a recursive function? (p. 113). Or what is the interaction between the __slots__ feature of a class and inheritance? (p. 133). Or, does the name mangling of private attributes (e.g., __foo) in a class introduce a runtime performance penalty? (p. 128). All of these questions fall into a general category of issues related to the "side-effects" of using various Python features. Although you can find some of this in the online docs, it is often scattered and incomplete. I've tried to fix that.

Finally, I've really tried to make the Essential Reference a kind of programming "cookbook" of sorts. Although its primary goal is to be a reference, I have also incorporated a wide variety of practical examples from the Python training courses that I run. For instance, if you know about the Generators or Coroutines tutorials I presented at PyCON, you'll find similar information. I also include examples that explore tricky interactions and customization features of certain library modules. For example, how do I customize an XML-RPC server to only accept connections from known IP addresses? (p. 494). Or how do I use the ssl module to implement a secure server? (p. 489). Many of these examples are related to things that I've had to figure out once before, but can never quite remember on a day-to-day basis. By putting them in the book, it helps me remember how to do a variety of tricky things.

So, that's about it. I hope people find the book to be useful. If so, tell your friends. If not, feel free to use it for propping up some uneven furniture. Just don't say that it's the same as the online docs.

Labels: ,

I can't believe people are saying this is the same as the Python doc.

Most the folks I've heard talk about it refer to it as the best Python book, hands down.

I could be wrong, but I always refer non-Python programmers who need to work with Python to this book (one of the four editions). It tells you exactly what you need to know without telling you how to think or what to do. The strongest language used is something like, "this usage is discouraged" and it usually has to do with security or bug avoidance.

Anyway, even if it is your book, you're right. It's a fantastic reference. I'm relieved to hear you use it too. All this time I thought you were just an uebergenius that kept it all in your head ;-)

Carl T.
Hi Dave,

I have every revision of this book. I pre-ordered this fourth copy and read it while on vacation last week. It, as the others did/does not disappoint. To simply state that this is regurgitation is ludicrous. More importantly this book is a guide that is detailed but not overly telling. It's "just right" in terms of details and quantity.

My only wish is that I could buy the pdf version of it..

Nice work!!

David, you've convinced me to get a copy of the 4th edition! I'm impressed you've found a way to cover both Python 2 and 3 without the book getting any fatter.

The 2nd edition was key to my learning Python. I found the writing beautifully clear and concise, and this clarity is matched by the design of the book. The only thing I'd have liked would have been a way to separate the language reference and the library reference.
I thought that I would not need this book; but I'm convinced now from the opposite :-)
Hi David,

I have always been an O'Reilly fan, but also enjoy Addison-Wesley books. I purchased Python Essential Reference/4e yesterday for the Kindle. Unlike most techies I know, I've started moving away from paper books for material that tends to be updated frequently, to reduce the dead trees piling up in my office :)

While I really like the content so far, the formatting and features of the Kindle edition of this book are horrible. I'm not reading this on a Kindle proper, but on the PC software and iPhone app. Both do a horrible job of letting you search the book, and neither let you jump directly to the table of contents via the built-in mechanisms; I had to instead jump back to the front of the book, page forward until reaching the table of contents in order to jump somewhere in the book. Also, it only seems to allow you to jump to "location", which I can only presume is paragraph or section number.

I know this isn't your fault, but I'd appreciate it if you could take the feedback back to your editor so it can go up the ladder to the appropriate folks at AW. This is my first Kindle tech book and it has been a terrible experience. I much prefer the PDFs that Apress, Packt, and Manning put out, and O'Reilly is the standard bearer with it's multiple versions (PDF, Mobi and ePub). The ePub editions are far more usable in Stanza than what I've experienced with the Kindle.

Sorry for the rant. Again, I know that you are not directly responsible for the formatting of these new digital formats, but I could see it having a negative impact on future sales. I'll continue to look into this, and if I find that the problem actually exists on the Kindle side, I'll be sure to post a follow-up.

That is really interesting feedback on the Kindle edition. For what it's worth, I have never been contacted by Pearson/AW regarding any aspect of the Kindle edition. I've never seen it, never proofread it, or ever been asked anything about it one way or the other.

I would actually be really interested to know what other authors are experiencing with this? Is there publisher working with them on Kindle preparation? Or are publishers just doing it all on their own?

My inclination on the Essential Reference is that there would be numerous problems syncing the table of contents and index entries to electronic content--mainly because so many different tools get used in production (original text in Word, typesetting in Quark, index created through some other means, etc.). Ugh.

In any case, I'm sorry to hear that it's not the best experience.
Anyone who compares your excellent book with the official online docs needs to schedule an appointment with a cluestick.

I have always recommended it as the best book for programmers transitioning to python from other languages. The only reason I don't have the 4th edition is because I've grown fond of my autographed 3rd edition. ;)

I did some more research and found that it was indeed just this edition that had the problems with Table of Contents. I downloaded samples of 2 other A/W books and an Apress book and had no issues with the ToC link working. You might want to talk to your editor about having someone fix that.

In the mean time, I got a refund on the Kindle edition and purchased a DRM-free PDF from InformIT for a few dollars more. To me, it's worth it to be able to search, to have a working table of contents, and... did I mention no DRM? I like not being treated like a criminal by default!

Will you be at PyCon this year? Maybe I can print out my cover page from the PDF and have you sign it ;)
Post a Comment

<< Home


08/01/2009 - 09/01/2009   09/01/2009 - 10/01/2009   10/01/2009 - 11/01/2009   11/01/2009 - 12/01/2009   12/01/2009 - 01/01/2010   01/01/2010 - 02/01/2010   02/01/2010 - 03/01/2010   04/01/2010 - 05/01/2010  

This page is powered by Blogger. Isn't yours?

Subscribe to Posts [Atom]