08 May 2010

I just finished "Bursts" by Albert-Laszlo Barabasi. Like a lot of non-fiction books I've read lately, it promises a lot more than it delivers, although it's still a fairly enjoyable ride.

The complaints about this book are only two, but both are pretty serious for this sort of weighty non-fiction book:

  • The entire research-based content book can be summarized in one dry sentence: when you measure the rate of various human activities, you will find that they usually conform to a power law, and only occasionally to a gaussian or more rarely Poisson distribution. OK, that's fine, but it's a far cry from the subtitle of the book, "The Hidden Pattern Behind Everything We Do." Barabasi spends the book ruminating on the implications of this finding for predicting human behavior, and of course in a statistical sense it can. However, as has been observed of the weather, I don't want to know in a statistical sense whether it will rain this afternoon, and in exactly the same way Barabasi never shows you can predict individual humans on individual days -- only that you can predict percentages based on groups of humans on average days.

  • Barabasi intertwines the tale of this recent research into human behavior patterns with the 14th century tale of György Székely, a Transylvanian who by an unlikely turn of events became the leader of a Hungarian revolt. Now, I'm a sucker for history, and this story definitely made me want to read some more about Hungarian history (they're got quite a bit over there). Furthermore, he has an excellent motivation for including this particular story -- an ancestor of his figures into it.

    However, the problem is that the analogy around which he's trying to build the parallel between the two narratives is that in the historical story, a noble named Telegdi made a speech in which he seemed to forsee exactly the unlikely turn of events that came about. See? Even in the 14th century they could predict human events, and now we're using digital technology to recreate this.

    Except that no, that doesn't hold any water. Barabasi is clearly a smart guy whose research is reasonably interesting, but in the desire to create a cohesive theme for his book he reaches much farther than his logic supports. As even he points out, fortune tellers are right part of the time not because they're predicting the future but because they make a lot of predictions. Yes, after the fact you can often find someone who foretold the outcome. However, as anyone who's made a risky move and had it pay off knows, those people are wrong as often as they're right. In fact, of all the players in the historical drama, most felt differently, meaning that there was absolutely no systematic way of predicting anything. Unfortunately, the intellectual justification for including the (fascinating) history in the book is incorrect.

Really, the key questions Barabasi's research (and thus the book indirectly) raises are about this question of how much we can predict the future actions of a population of humans. Surprisingly, Barabasi doesn't talk about what to me is the glaring counterpoint: chaos theory. Barabasi makes reference to having read chaos theory, but in the book he never discusses the very salient implication of it for his main theme. Chaos theory shows that in many cases (including among other historically significant activities elections and wars), the outcome is dramatically sensitive to initial conditions and thus even given statistical bounds on the input, the outcome is not subject to meaningful prediction. To me, this is a happy thing (I don't want to know the outcome of an election before I've voted in it) but I wish Barabasi had dealt with this - he may well have an answer to it, but he didn't include it in the book.

I don't mean to say I hate the book; like Freakonomics it does present some interesting research on human behavior patterns to read about, but unlike Freakonomics, which absolutely admits it has no unifying theme, this book tries to have a unifying theme but in my opinion fails to find a meaningful one.

05 May 2010

LDAP, she is a beach

Several techy posts coming out in the near future.

Yesterday I finally cracked an annoying problem I've been working on off and on since I came to Japan -- how to have you web app authenticate against ActiveDirectory. It wouldn't have been so hard except that our SysAdmin insisted it was absolutely impossible, would require buying another piece of hardware or two etc. etc. Of course, that's all totally untrue -- every AD server is an LDAP server. But man, LDAP, despite the "L" standing for "Lightweight", is the last of the dinosaur protocols. Everything is wrapped in five levels of obscurity and indirection with a verbose syntax. If you ever had any exposure to the OSI protocol stack or X.400 mail protocols you'll feel right at home. Should you be cursed to figure this out in your own network environment, start by downloading the Apache Directory Studio, it will save lots of time.

Expressions are the best part. When building a filter expression, you need to express a search query, not unlike SQL. Their solution is indeed logical but produces the oddest text since full-on LISP. It's a prefix notation where every level of the epxression tree is enclosed in parentheses. Thus, for "object representing a user whose account name is mary or bob and whose account is not expired" we get


SQL has never looked so good.