Hello?

So, I just got an email from JR East (the train system here) with an "important notification" about Eki-net (the online system for buying domestic train tickets). The notification was that you will no longer be able to use a password that's the same as your user id. Yikes, they just figured out that's a good idea? It's a good thing most of the Russian cybercriminals can't read Japanese...

Trying to use Trac...

So, in part because at work we have a need for task/issue management software, I've tried to get educated on the various open-source options in that area. It seems like there are two main choices: Trac or Redmine (I also ran across one with a beautiful user interface called Integria IMS but because of various internationalization issues I didn't explore it much).

Trac is the standard issue tracker in the Python open source world, and since I'm blessed to be able to spend virtually all of my time writing code in Python these days, it was the first thing I looked at. Naturally, like almost all issue/bug/task trackers, Trac has all the standard features like entering a ticket, having types, statuses, and so forth. From the fact it's widely used and the fact that it's implemented in Python, it was definitely my default choice. Whatever system we use at work we'll probably spend a lot of time customizing, so the Python thing is big.

Then a co-worker here in Japan mentioned Redmine. Redmine is newer than Trac but has a lot of the same features (as do the others). In general the interface to Redmine feels a lot more up-to-date than Trac's (it's not that Trac's interface is bad, it's just a pre-Web-2.0 kind of feeling, whereas Redmine used a lot of more interactive widgets in it's UI).

When I started looking realistically at what we would need to do in order to adopt either one of these company-wide for managing the tasks of 300 people, I realized that a lot of the things we needed were related to scale. While open source projects can have scale in terms of users, I don't think either project is really set up for creating thousands of tasks per month. In particular, we clearly need to have:

• Ability to create new tasks by importing from spreadsheet (it just doesn't make sense for anyone to have to click "Create New Task" 1200 times)
  
• Ability to batch edit tasks (when the above goes wrong, you have 600 tasks to change from "New" to "Assigned")
  
• GANTT charts (i.e., a graphical representation of tasks over time). Open source projects are notoriously lax about schedules. We work in production, and our production managers need good ways to see exactly what's going on.
  
• Task dependencies. If you know about about CG production, it's very pipelined: the animation can't start until the layout is done, etc., so you want a way to declare that Task F depends on Task C.
  

None of the above are particularly specialized; they're certainly all features that have been implemented in project management software (e.g., Microsoft Project or FastTrack Scheduler) for decades. But, surprisingly, Trac has zero of the above features (Redmine by default has all except the import).

To be fair to Trac, Trac pushes a plug-in philosophy, and there are sure enough plugins for all of the above (more about that in a minute). I read through several discussions on the Trac site where every one of the above features was pooh-poohed by a Trac developer as "just not important enough to be in core." There are two big problems I had with that.

Issue Management Should Have Scale

Seriously? Ability to multi-select items is considered "not core"?

Maybe it's just because Trac comes from the world of tracking open-source projects; it's fair to say that they probably rarely need to create 600 tickets at once, or update the status of 150 of them. But that's not actually an unusual thing to want to do if you're tracking short-term tasks for a group of 300 people. Those basic, simple software features like multi-select/batch edit or the ability to establish relationships are absolutely central.

I'm totally understanding of the fact that time and attention are limited, and if it was simply a case of "we haven't gotten to that yet" I would have no problem. But a lot of the Trac core developers seem to think they *shouldn't* actually have standardized way to establish a relationship between two issues, which is insane.

Plug-ins Are a Terrible Way to Standardize Architecture

None of that would really matter if Trac's plug-in approach worked well enough, to be honest. OK, sure, you have to download some plug-ins, but hey, all the functionality is there, right?

There are a lot of cases where that would be true. But it's not true in real life, because the core architecture of a system needs to provide the base data items for the plug-ins to manipulate.

Here are the specific ways Trac fails to provide enough common infrastructure to actually make plug-ins to support core features useful:

• For batch edit, the most common batch edit plug-in only works with custom queries. If you just go to the normal "Show me the isssues" default query you can't do a batch edit. This is because there's no extensibility of the individual item UIs.
  
• For issue dependencies, Trac doesn't provide the plugins a standard way to store relationships between issues. Because of this, the plugins try to work around this by using the custom field support which Trac does have. Unfortunately, the custom field support doesn't include a type system, it's easy to fail data integrity (your list of dependent tasks is being stored as a comma-separated text field; there's no way to use the real relational data model).
  
• For the spreadsheet-import feature, again, because Trac doesn't provide a type mechanism for custom fields, while the import plug-in does its best to try and support custom fields, there's no way for it to know what the type of the fields added by other plug-ins is. Thus, when you try to import a spreadsheet which records that your task is dependent on Task #1, the text field "1" gets interpreted as a float and becomes "1.0" which then fails the referential integrity check. Your bug tracker is now broken and cannot be repaired from the UI; it's MySQL Query Browser time for you.
  
• A group here in Japan called Shibuya Trac wrote a GANTT chart generator for Trac. Unfortunately, it's out-of-date for versions so you'll be Googling for awhile if you want to install it, and for a longer while if you don't read Japanese, the native language of the developers.
  

Oh and, It's not based on a real ORM

When I actually started looking through the Trac database schema and source code to see why some of the above things were true, I got much more depressed. I realized that just because of history, the Trac project is old enough that it had to grow its own solutions to a lot of problems. They have their own ORM, their own templating system, their own database abstraction layer, etc.

I've spent a lot of time writing apps against Django, the absolutely awesome Python database toolkit. Especially when used with South, Django is a complete Python equivalent to Ruby on Rails. The ORM problem (how to map Python classes to a relational database), the database schema migration problem, and the database backend problem are completely and throughly solved by Django (as they are by Ruby on Rails). And the templating problem is pretty well laid out too. For Trac, they were building an app, so they've solved all those same problems but not as elegantly or cleanly as Django. Their database adapter isn't quite as complete, the ORM is a little more manual, and of course none of those subsystems are as well documented ("Look at the code" is frequently mentioned in the Trac developer docs; Django has awesome documentations even without looking at the code, which I'm now convinced is one of the best signs of quality in a piece of open source software.

Looks like Redmine from here

While Redmine doesn't have spreadsheet import built in, they in general have taken a much more expansive view of what is "core": they treat multi-select as a core UI ability, they treat the ability to declare relationships between issues as a core data structure item (although the core package doesn't specify what type of relationship it is); and they have always had time-oriented views into the data.

I really wish I could use Trac, among other reasons because I don't have any particular desire to use Ruby (nothing against it, I just already know Python very well). But the well-integrated easy-to-support features in Redmine and the lack of same in Trac mean it's just not practical. Either Trac needs to make a much more realistic set of decisions about what features are "core" to their stated mission, or they need to significantly beef up their core data infrastructure so that independently developer plug-ins can truly build on each other without dozens of referential integrity problems cropping up. Given that Trac is a relatively successful project, whose core developers seems pretty happy with the way it works now, I guess I'll have to learn a little Ruby.

15 Albums That Will Stick With Me

My friend Michelle Coulter started this thread on Facebook. My contribution got kind of long, so it migrated here.

Windham Hill - Legacy, a Collection of American Folk
Windham Hill has faded from memory pretty quickly, but not only did they totally evoke the Northern California feeling for me, they also found and promoted a lot of truly talented musicians.  Whereas most of WH was new-age-y light instrumentals, this collection was gritty folk music by unknown performers from all over the country.  The quality of the songs and performances were so good that several of the songs from this collection are still stuck in my mind decades later (I even learned to play "Dodging the Blues" by John Gorka on guitar). The interests sparked by this album led me eventually to Butch Hancock/Joe Ely/and the Austin New Folk movement on one hand, and Uncle Bonsai and other eclectic west coast folk on the other.  Listening to guitarists like Kaki King always reminds me of Windham Hill's early days.

Bangles, All Over the Place
The Bangles are of course most famous for the "A Different Light" album.  I loved that album too, but the best thing about it was that it made me go back and find this album, which much more accurately represents the hard-rocking LA girls they started out as.  While "Walk like an Egyptian" from "All over the Place" is definitely a good pop song, "Hero Takes a Fall" from this album is a great *rock* song.  This album got my car going way too fast down the 101 far too many times.    Among other reasons to like the Bangles, their bass player, Michelle Steele, used to be in the iconic band The Runaways.  This album also has the Bangles covering "Going Down to Liverpool", which was written by and would eventually be released by Katrina and the Waves.  I've always wondered how an up-and-coming LA band heard about a B-side from an (at the time) obscure Cambridge, England band in order to cover it.

Chicago II
Probably most people remember Chicago as the schmaltzy balladeers they indeed became. But the formation of the band was as a jazz ensemble with some rock influence, and if you go back to those albums (basically Chiacgo I and Chicago II) they're anything but pop music. 11-minute long jams, extended jazz arrangements, and intense inner monologues fill out those first few albums. There are a fair number of bands that can and do perform this stuff live in concert, but there weren't many that made it their debut album. It's the opposite of internet-friendly.

The Beatles, 1962-1966
My sister was an original Beatlemaniac, and while I liked the Beatles jut fine growing up, it wasn't until I got this compilation album many years later that I felt like I really became a Beatles fan.  The sheer quantity of infectious, well-written, pop music they put out in these productive years is unique in music history, even without counting what they would go on to produce in the following albums. Many years later I'm learning more and more Beatles songs on guitar.

Matthew Sweet, Girlfriend
Hee-hee, compared to the rest of this list this is the guilty pleasure zone. Matthew Sweet writes unapologetically pop songs in an undeniably narcissistic way (he performs almost every instrument on his albums). When I first heard this album (like The Bangles, it frequently powered my highway driving) I hoped that kind of hard-driving pop music would be a category I could fill out my record collection with. But in the end, I had to content myself with other Matthew Sweet albums (of which there are quite a few). Sadly, it didn't seem to lead anywhere for Matthew Sweet either: I haven't heard anything interesting from him in a long time, but I keep hoping he'll have some awesome late-career comeback.

Not an Album, but a place: KFJC, 89.7FM
I can't draw up a list of "musical influences" like this without talking about KFJC, even though it's a radio station rather than an album. Specifically, it's the college radio station at Foothill College in Los Altos Hills, California. Now, college radio, while it can bring you some new music you've never heard, is more often unlistenable dreck. What makes KFJC special is that many of the DJs, rather than being this semester's freshmen, are actually Silicon Valley employees who love music and who have been doing their weekly show on KFJC for 20 years. They're not only volunteers, they pay for the privilege (I believe they have to register for one class at the college to be eligible). The result is shows run by a DJ with an encyclopedic knowledge of some sub-sub-sub-genre. Phil Dirt's instrumental surf hour, The Norman Bates Memorial Soundtrack show, the reggae show… there's so much musical knowledge here available to you, all for flicking your dial (on the Peninsula) or getting your URL correct (anywhere in the world via http://www.kfjc.org/) that you have no excuse not to partake.

Talking Heads, More Songs About Buildings and Food
Talking Heads, Stop Making Sense
Into the dinosaur environment of 70s album-oriented rock came four Rhode Island School of Design students (OK, three RISD students and Jerry Harrison) who proved that rock could not only be intelligent and interesting, it could be intelligent, interesting, and danceable! Who knew! The visual and musical amalgam the band provided was a godsend. In some sense, these two albums are the beginning and end: More Songs… was the first time I heard about the band and started listening to them; and Stop Making Sense was really their swan song as a band, after which individual projects would take off and the general rise of Alternative Rock meant that their approach was no longer as unique. Stop Making Sense has a particularly fond place in my heart because the old Varsity Theatre in Palo Alto, a classic single-screen movie house, used to have midnight showings of Stop Making Sense. Only 50-60 people would come, and since the Varsity actually used to host plays, there was a large stage where the majority of audience would end up right in front of the screen dancing for the entire movie. Alas, not so possible at the modern multiplex. The Varsity, meanwhile, is now a Border's bookstore.

Juluka, Scatterlings of Africa
I actually don't remember anymore how I first happened to listen to Johnny Clegg (Juluka was a partnership between him and Sipho Mchunu), but I was instantly hooked. Juluka was extremely well-known in South Africa as a biracial band in the apartheid era, and as a result of that fact, had to play underground and couldn't release their albums on the official South African radio or music channels. They became a world-of-mouth hit with their combination of rock influences, zulu guitar style, and explicitly political anti-apartheid lyrics. What you wouldn't know from the albums are the extent to which their performances are organized around Zulu dance as well as zulu guitar. Both times I've seen Johnny Clegg live the band did one exhausting 1:45 set in which they continually sang, played, and danced (those folks are *in shape*) without a break; there's some good tracks on YouTube. For me, not only does Juluka's music hold up well, it was my gateway to discovering the whole range of African music (the South African compilation series "All the Hits 2xxx" which is issued each year is especially recommended if you run across it).

Music Man, Original Broadway Cast Album
My parent's taste in music ran mostly to Rachmaninoff and Debussy, whereas even within classical I'm more partial to Mozart or Bach.  However, the one place where our tastes enthusiastically overlapped was in musicals.  As a kid, the album cabinet built into our (wall-sized) "hi-fi" was stocked with the soundtrack albums for every big broadway production -- Camelot, Oklahoma, Sound of Music, and eventually A Chorus Line.  But really, I didn't spend as much time with any of them as with Music Man.  The combination of the good-old-Iowa setting (my mom's home) plus a bunch of great songs for the genre meant hours and hours in my room singing along to "Seventy-six Trombones."  To this day I can still recite "Trouble in River City" from memory.  

The death of naive entertainment made musicals a lot harder to sustain.  The modern equivalents in "Rent" or "Chicago" are great entertainment but somehow don't give rise to memorable songs in the same way (can you hum the tune to any song from "Rent"?).  

Bonnie Raitt, Luck of the Draw
This is a good album, but that's not why it's listed here.  It's listed here because I went to see Bonnie Raitt in Santa Cruz at the Catalyst just before Nick of Time and this album were released.  Bonnie Raitt was a musician well-known as a touring musician but not someone who had ever achieved massive success at that point.  But, she had been touring and playing music for her whole life (she was probably around 40 at the time).  Seeing her live, I was absolutely dumbstruck by her skill as an entertainer.  I had the epiphany that all live musicians who've had a long career are extraordinarily appealing to see live -- they have a ineffable quality of being relaxed and welcoming to the audience.  Bonnie Raitt had just recently recovered from drug and alcohol abuse problems, and you could see that history.  But the sheer humanity of her performance was incredibly moving.  Thus, when Nick of Time and Luck of the Draw came out, it felt very satisfying that her years of toil finally paid off for her (and they have some catchy songs, too).

It's also good to have this in the list as a reminder that there are tons of musicians who spend their whole career without ever getting their "Nick of Time": they've done great work and been successful enough to stay in the business, but never really had their breakthrough hit. John Hiatt (who appears on Raitt's albums), Richard and Linda Thompson, Allison Kraus, Blues Traveler, J.J. Cale, Joe Craven…

Miles Davis, Kind of Blue
When I first moved to Northern California, I started studying piano (I didn't play any instruments as a kid) and I was lucky enough to attend the ensemble workshops at Stanford Jazz Camp.  This is just an unbelievable resource for people lucky enough to live in the Bay Area who also happen to like Jazz; musicians of any level are welcome to enter the ensemble classes.  We met in the Braun Music Hall's "small auditorium" where the piano I practiced on was a 1927 Steinway grand, and so I learned why Steinways are, well, Steinways.  

But the music we did for the workshop was mostly Miles Davis.  And why not?  His classic recording occurred in a time when he was legendary not only for his playing, but for his scouting of great musicians to play with.  Every tune on Kind of Blue is a classic, one that jazz musicians should and do study and reprise.  I had listened to Kind of Blue before the Jazz Workshop, but of course after learning and playing the tunes, my appreciation of the musicianship recorded on the album was of a different order.

Grateful Dead, Touch of Grey
This is probably the Dead's best studio album, but let's face it -- I'm really just putting it here as a stand-in for the experience of attending a Dead concert.  Fortunately, I got to attend three Grateful Dead concerts.  They evoked a feeling of communal happiness, goodwill, and freedom from worry more than any other group experience I've had (Burning Man is close, but Burning Man has an intellectual edge which Grateful Dead concerts distinctly did not).  I don't use drugs, so that's not where I was getting the feeling at Dead concerts (although the same can't be said of all my fellow concert-goers).  The combination of the band and the Deadheads who followed them around created a true group good vibration; swaying back and forth to the music seemed like the only reasonable choice, and everybody did it.  It didn't hurt that I always saw the Dead outdoors in the summer -- at Frost Amphitheater and Laguna Seca.  I hear the modern equivalents are Phish, so even if the studio albums don't sound great to you, catch a concert sometime.

Fleetwood Mac, Rumors
While I think objectively this is one of the highlights of the album-oriented rock era, it doesn't date all that well (I doubt anyone will become a new fan of Fleetwood Mac by reading this). For me, though, it's a shoo-in for this list because of the role the song "Go Your Own Way" (talk about a paean to 70s individualism) played in my high school life. I was a nerdy shy and overweight kid for most of my childhood. Early in high school, my mother passed away from cancer and I started trying much more actively to get attention at school. By the beginning of my senior year, I had finally lost weight, bought some non-polyester clothes, and knew enough people to be comfortable. Then, the honor society got a presentation by a woman from the local Muscular Dystrophy Association about a new type of benefit event: the Dance-a-thon. You had music playing all night and got pledges of a certain amount of money for each hour you danced. The promotional video MDA made for this showed inspirational scenes of MD victims being helped to get their lives in motion again, and the soundtrack was "Go Your Own Way" from "Rumors".

Somehow, my friend Si Steinberg and I immediately took to this idea. We blitzed the Honor Society into being the sponsor, convinced our principal into letting us have an all-school assembly to pitch the idea to the students, and roped virtually every other student organization into playing some part in the production: the Student Council was in charge of soliciting for prizes, the Home Ec club made and sold food all night, the DECA (business) club handled the accounting, etc. It was like something out of a Andy Hardy movie. The event went off, we raised IIRC $7000K+ for MDA, and everybody had a good time and went home exhausted.

In the course of pitching, organizing, and running that event, I must have watched the "Go Your Own Way" promo video over a hundred times. It indelibly imprinted Fleetwood Mac on me and kept me inspired and excited over those hectic months. Most everyone has probably forgotten the Dance Marathon, but it and Fleetwood Mac had a huge part in my life.

Jackson Browne, Running on Empty
After all the personal stories on this page, really, the reason for "Running on Empty" being here is because it's a great road album. Musicians write about being on the road because they spend a lot of their life there, and I still don't think anybody's bested this album as a record of that combination of boredom, tedium, and exhilaration. And certainly Jackson Browne has every right to pen the classic road album: he joined Nitty Grity Dirt Band at age 18 and has been a performing musician continuously since then.

• 19:29 Leo is at work #

Automatically shipped by LoudTwitter

I just finished "Bursts" by Albert-Laszlo Barabasi. Like a lot of non-fiction books I've read lately, it promises a lot more than it delivers, although it's still a fairly enjoyable ride.

The complaints about this book are only two, but both are pretty serious for this sort of weighty non-fiction book:

• The entire research-based content book can be summarized in one dry sentence: when you measure the rate of various human activities, you will find that they usually conform to a power law, and only occasionally to a gaussian or more rarely Poisson distribution. OK, that's fine, but it's a far cry from the subtitle of the book, "The Hidden Pattern Behind Everything We Do." Barabasi spends the book ruminating on the implications of this finding for predicting human behavior, and of course in a statistical sense it can. However, as has been observed of the weather, I don't want to know in a statistical sense whether it will rain this afternoon, and in exactly the same way Barabasi never shows you can predict individual humans on individual days -- only that you can predict percentages based on groups of humans on average days.


• Barabasi intertwines the tale of this recent research into human behavior patterns with the 14th century tale of György Székely, a Transylvanian who by an unlikely turn of events became the leader of a Hungarian revolt. Now, I'm a sucker for history, and this story definitely made me want to read some more about Hungarian history (they're got quite a bit over there). Furthermore, he has an excellent motivation for including this particular story -- an ancestor of his figures into it.
  
  However, the problem is that the analogy around which he's trying to build the parallel between the two narratives is that in the historical story, a noble named Telegdi made a speech in which he seemed to forsee exactly the unlikely turn of events that came about. See? Even in the 14th century they could predict human events, and now we're using digital technology to recreate this.
  
  Except that no, that doesn't hold any water. Barabasi is clearly a smart guy whose research is reasonably interesting, but in the desire to create a cohesive theme for his book he reaches much farther than his logic supports. As even he points out, fortune tellers are right part of the time not because they're predicting the future but because they make a lot of predictions. Yes, after the fact you can often find someone who foretold the outcome. However, as anyone who's made a risky move and had it pay off knows, those people are wrong as often as they're right. In fact, of all the players in the historical drama, most felt differently, meaning that there was absolutely no systematic way of predicting anything. Unfortunately, the intellectual justification for including the (fascinating) history in the book is incorrect.

Really, the key questions Barabasi's research (and thus the book indirectly) raises are about this question of how much we can predict the future actions of a population of humans. Surprisingly, Barabasi doesn't talk about what to me is the glaring counterpoint: chaos theory. Barabasi makes reference to having read chaos theory, but in the book he never discusses the very salient implication of it for his main theme. Chaos theory shows that in many cases (including among other historically significant activities elections and wars), the outcome is dramatically sensitive to initial conditions and thus even given statistical bounds on the input, the outcome is not subject to meaningful prediction. To me, this is a happy thing (I don't want to know the outcome of an election before I've voted in it) but I wish Barabasi had dealt with this - he may well have an answer to it, but he didn't include it in the book.

I don't mean to say I hate the book; like Freakonomics it does present some interesting research on human behavior patterns to read about, but unlike Freakonomics, which absolutely admits it has no unifying theme, this book tries to have a unifying theme but in my opinion fails to find a meaningful one.

LDAP, she is a beach

Several techy posts coming out in the near future.

Yesterday I finally cracked an annoying problem I've been working on off and on since I came to Japan -- how to have you web app authenticate against ActiveDirectory. It wouldn't have been so hard except that our SysAdmin insisted it was absolutely impossible, would require buying another piece of hardware or two etc. etc. Of course, that's all totally untrue -- every AD server is an LDAP server. But man, LDAP, despite the "L" standing for "Lightweight", is the last of the dinosaur protocols. Everything is wrapped in five levels of obscurity and indirection with a verbose syntax. If you ever had any exposure to the OSI protocol stack or X.400 mail protocols you'll feel right at home. Should you be cursed to figure this out in your own network environment, start by downloading the Apache Directory Studio, it will save lots of time.

Expressions are the best part. When building a filter expression, you need to express a search query, not unlike SQL. Their solution is indeed logical but produces the oddest text since full-on LISP. It's a prefix notation where every level of the epxression tree is enclosed in parentheses. Thus, for "object representing a user whose account name is mary or bob and whose account is not expired" we get

(&(objectType=user)(|(aASAccountName=mary)(aASAccountName=bob))(!expire=*))

SQL has never looked so good.

Lost in Translation redux

Remember that scene in Lost in Translation where Bill Murray asks whether to hold the glass to the right or the left, and the commerical director goes off on a 5-minute tirade about something in Japanese (his philosophy of directing, as it turns out), and afterwards the translator translates the entire 5-minute speech as, "to the left?"

The actor who plays the commercial director is actually an 80s rock'n'roll star and actor who goes by "Diamond Yukai." Last weekend I met his manager, and he's still gigging! He had a show yesterday in Shibuya at O West.

Japanese Health Care FTW

Today I had an experience which, while it's normal here, is inconceivable in the US. For those of you who still hold the delusion that the US has a functional health care system, wake up.

Last night I realized I needed to get some more contacts because I was almost out. So today I went to the contact store (no appointment, no calling ahead). They asked me about my order and then, because this was only the second time I had bought from them, told me I had to get my eyes re-checked (normally, you only need to get your eyes re-checked once a year, similar to the US).

Now, if I was in the US, first of all the eye doctor would probably be somewhere else than the contact lens store, and secondly, getting an appointment at my eye doctor in the US meant calling a week ahead -- or three weeks ahead if you want a Saturday appointment, since today was a Saturday.

However, here in Japan, they just said, "Go to the 5th floor." I went to the 5th floor where the nice receptionists took a look at my insurance card (health insurance is mandatory in Japan btw), and asked me to wait. I waited about 3 minutes, had my vision checked by an assistant, took my contacts out, waited another three minutes, had the actual eye doctor check my retinas and other things, waited another about 5 minutes (while watching the Olympics) and got a new pair of contacts. Total time: 20 minutes; co-pay: 380 yen (about 4 dollars).

Then, back downstairs to buy the contacts. Now, you would think the price for all this convenience would be that the actual contacts are more expensive. And you would be wrong. My contacts cost about 20% less than 1-800-contacts did back in the US.

My fellow Americans: your health care system is f*&%ed. Please get on with fixing it. It doesn't have to be like this, and the solution *does* have to involve the government.

Twentieth Century Boys

Based on high recommendations from my friend Gilles, I just rented and watched 20th Century Boys (Pt. 1) last night.

The basic idea of the story is that a group of school friends make a secret clubhouse out of grass one summer to have a place to look at manga, listen to Jim Morrison, Jimi Hendrix, and so forth on the radio, and share what passes for porn in junior high school. As part of passing the time, they make up a symbol for their group (an up-pointing finger over an eyeball) and create a group sketchbook that shows a series of attacks on the world that the friends come together to defeat. 25 years later, world events seem to be eerily echoing the sketchbook from their childhood, and they're forced to come together again to deal with this fact.

It's an interesting take on storytelling, since the story spans the time from the early 70s through 2015. It pops back and forth in time a lot, although they're pretty consistent about using a supertitled date to tell you where you are. That was absolutely essential towards the end of the movie, when they started jumping around by a few days instead of 20 years. The basic idea is interesting, and in the story the characters try to access 25-year-old memories to attempt to reconstruct the details of that long-ago summer, so cutting back to those scenes lets the audience learn the memories as the characters relive them. Also, the casting is awesome; they did a fantastic job of getting child actors who all plausibly look like the child versions of the adult actors.

They picked well for the central character to tell the story through: Kenji was a close to a leader as the group had back in the day, with personal style that eventually led him into a career as a rock'n'roller -- which in turn caused him to drink his way down out of society. Now he's a convenience store clerk (the initial scenes of the manager exhorting him to practice the chain's greeting more sincerely is just one of the many little societal commentaries in the film) who's raising his niece as a single parent.

Unfortunately, I think the weakest point of the story is the one widely shared among Manga adaptations, whether animated or live action -- the storytelling is frequently elided. That is to say, there are just scenes you need to have that aren't there. Towards the end, the enemy (ironically named "Friend", he heads up a cult/political party called "Friend Party" / 友民党) has sent a giant robot into Tokyo and our group gets together and heads out to stop him. Cut to, shot of one of the group tackling Friend from behind on the top of a building. How did they get there? How did he know where Friend was? etc. is never explained. Another member of the group, for equally unknowable reasons, happens to be watching from the next rooftop.

This shortcut storytelling is so widespread among Manga adaptations it must be a fairly conscious choice (if you watch standard Japanese TV dramas or live action movies, they don't suffer from this problem, so it's not like storytelling is unknown here). I've always assumed that it comes from the fact that the main audience in Japan for a Manga adaptation is people who are already fans of the Manga -- thus, the filmmakers assume the audience already basically knows the story, and wants to get to the "good parts", i.e., the scenes where the characters' emotions are clearly on display and/or the action scenes, rather than actually clearly retell a story the audience already knows. In terms of the domestic market for a film like this, that makes perfect sense; unfortunately, and to the great detriment of the industry here, it severely limits the market for the film outside the Manga's pre-existing fanbase. Most worldwide audiences do prefer their stories clearly told rather than needing to guess at the missing pieces.

Please get wasted at home

From Tokyo

There's a public service campaign going on right now about not putting on your makeup, drinking soda, etc. while riding the train -- the tagline is always, "Please do it at home." The latest poster addresses a different behavior to be practiced in the home, though.

Note for those not living in Japan: this is not a joke, these posters are really up all over Tokyo right now.

How to Attend the Electronic Theater at SIGGRAPH Asia 2009

こんにちは、
（English follows the Japanese)

僕は今年の横浜のシーグラフアジアの
コンピューターアニメーションフェスティバルチェアーで,
フェスティバルの主要なエベントはエレクトロニックシアターです。

エレクトロニックシアターの上映の予定は、
１２月の１７日の木曜日の夕方のプレミアショーと１８日の金曜日の夕方のショーを行いますが、
１９日の土曜日の上映も二つ予定されます。午後の 16:15 も夕方の 19:00 もあります。
エレクトロニックシアターは２時間ぐらい掛かります。
エレクトロニックシアターの内容とは、下記のリンクを参照して
「エレクトロニックシアター」にクリックしてください。
エレクトロニックシアターは横浜パシフィコのメインシアターでやって、
ソニーマーケティング日本に上がって頂いた４K のディジタルシネマプロジェクターを使います。
フルHD の変種の作業はイマジカでやっています。

CAF ET Program

他のシーグラフのエベントに参加しなくてもエレクトロニックシアターに出席したい方は、
下記のリンクを参照するとクレジットカードでチケットの何枚を買えます。

ET Tickets

エレクトロニックシアターに上にシーグラフアジアの登録したい場合は、
一般的なシーグラフアジアの日本語のサイトを参照して下さい。

SIGGRAPH Asia Registration

そのメッセージは他の興味がある方に自由に転送すると幸いです。
質問または不明な点になればぜひ私にご連絡ください。

宜しくお願いします。

レオ

Hi all,

I'm the chair for this year's SIGGRAPH Asia Computer Animation Festival in Yokohama, and the main event of the festival is the Electronic Theater.

The schedule for screenings for the Electronic Theater includes a premiere show on the evening of Thursday, December 17th as well as a show on the evening of Friday the 18th, but there are also shows scheduled on Saturday the 19th, at both 4:15pm and 7pm. The Electronic Theater lasts about two hours. For the contents of the Electronic Theater, check the web page below and click "Electronic Theater". The Electronic Theater will be presented via a 4K Digital Cinema projector donated by Sony Marketing (Japan) in the Main Theater at Pacifico Yokohama. The full-resolution HD edit is being carried out at Imagica.

CAF ET Program

For people who don't want to participate in other SIGGRAPH events but just want to go to the Electronic Theater, please go to the link below, and you can purchase any number of tickets via credit card.

CAF ET Tickets

If you want to go to other SIGGRAPH Asia events besides the Electronic Theater, you can of course visit the main SIGGRAPH Asia site:

SIGGRAPH Asia Registration

Please feel free to forward this message to any other interested people, and if you have any questions, or if anything is unclear, please get in touch.

Yours,

Leo

Disappointing Books #1, "Castles, Battles, and Bombs"

I've read two disappointing books recently, so in the interest of helping others avoid the mistake, I'll share.

"Castles, Battles, and Bombs" by Jurgen Brauer and Hubert van Tuyll has a very clear concept: the authors attempt to study military history through the tools of standard (aka neoclassical) economics. They choose six periods of history, pair them with six principles of economics (opportunity cost, substitution effects, marginal cost vs. marginal beenfit, etc.) and attempt to use the economics principle to understand the period of history.

It's a neat idea; too bad it doesn't add much to the understanding of history. Now, I'm not saying the authors are wrong in their observations. Yes, clearly, building castles in the High Middle Ages reduced the funds kings had available to devote to maintaining field armies. But what was true in pretty much every one of the six chapter of this book is that the economic principle, while at least somewhat applicable, is not key to understanding the period in question.

For instance, they study Renaissance Italy via the economic question known as the principal-agent problem, which is to say, when you hire something to do something, how do you know they (your agent) are doing what you (the principal) want? Since Italy was constantly at war through most of this period, and that war was primarily conducted via hired mercenaries, the condottori (literally meaning contractors), it's easy to see why they matched the two.

While the book goes through the considerable lengths the Italian city-states went to in order to mitigate the principal-agent problem, all of which make sense (essentially, an auditing corps of city employees were used to keep track of the activities of the condottieri, as well as a tendancy for longer-term relationships), it's not believable that the principal-agent problem was a cause for the commencement or ending of this era. The principal-agent problem has to be managed any time you engage in a contract, yes; but since the problem existed throughout the era in study, and roughly equally for all participants, I felt like I learned nothing new about the era by studying these. The real reasons why the mercenary system existed during that era, why the city-states chose it, and why they eventually chose otherwise, all lie outside the area of study.

Every chapter suffers from this failing, in that while the principle does apply, it doesn't offer key insights. In the fifth chapter of the book, they go over the well-trodden data showing that (non-nuclear) strategic bombing is not economically justifiable. That is to say, the total economic costs to an aggressor country of building and operating a combined-arms air force capable of conducting a strategic bombing campaign is greater than the economic damage done to the enemy by said campaign (factories, surprisingly, turn out to be pretty easy to repair). They use the extensive data on WWII's air campaign against Germany to show this. Unfortunately, as they acknowledge, even the post-war analysis within the US military reached the same conclusion about cost-benfits 60 years ago. They claim this is a lesson in marginal cost vs. marginal benefit, but in fact what the data show is that it's simply a bad military investment in the first place, you don't have to think about marginal cost vs. marginal benefit. Allocating those resources to tactical air support of operations instead of conducting strategic bombing at all is the conclusion supported by the data.

The worst case of this is the chapter on castle-building vs. field army maintenance in the High Middle Ages. They attempt to pitch the preference for castle-building as causing a lost opportunity, in that the monarchs couldn't also maintain a field army. But what their data shows is that the cost of castle-building, even for aggressive castle builders like Edward I in England, pales beside the costs of war. Their own data shows Edward spent £90,000 on Welsh castles, but £1,400,000 on fielding armies for his wars against Wales, Scotland, and France. Castles are so cheap relative to armies that it's distorting the data to say they represented a lost opportunity.

In the end, this book does explore its thesis. Unfortunately, I think the book shows that the thesis isn't worth taking much farther. Is it helpful for a historian to understand the basic principles of economics? Yes. But will they be a primary guide to understanding past actions? Based on these examples, no.

The Death and Rebirth of a Laptop

Chronology of An Ordeal

Over the last 2 1/2 weeks, I've dealt with dropping my MacBook Pro and the various fallout from that. I'm writing down the full experience, for that cathartic "getting over it" benefit -- that way, this can be my last post on the topic!

Aug 11 afternoon

While seeing "Up" at the Kabuki Theater in SF, my MacBook falls out of my backpack from a height of about 3 feet. The person behind me squealed as it was happenning, so I turned around in time to see it happen. It was very depressing. In order to avoid thinking about it during the movie, I didn't test it at all at that point.

Aug 11 evening

Got back to hotel, verified something was Very Very Wrong. The machine was still awake and I could activate various apps, but they all went into wait cursor state.

Aug 11 evening 2

In order to connect to the rest of the world while the Mac was on hiatus, I went to Best Buy in San Francisco and bought a netbook (Eee PC in particular). For $300. Incredibly cheap. Since planned dinner didn't happen, I spent much of the evening installing Firefox/Pidgin/etc. The trackpad's buttons are *incredibly* irritating -- the left mouse button sticks unless you imemdiately push a little on the other side.

Aug. 13

On the plane back to Tokyo, I realize that since I bought this MacBook Pro at SIGGRAPH 2006 at Boston, it is exactly two weeks out of extended warranty. Hah hah hah, isn't that funny.

Aug. 14

evening Make appointment with Apple Store for Sunday morning.

Aug. 16

Off to Labi in Shibuya, bought a Bluetooth adapter for the Netbook. This thing is hilarious, it's literally the size of your thumbnail. It worked flawlessly: XP recognized it immediately, and I could then pair the EeePC with the wireless keyboard and mouse I used with my laptop. Using the mouse meant I didn't need to use the incredibly annoying trackpad button on the EeePC, eliminating the biggest reason to hate the thing.

Aug. 16

The Geniuses at the Apple Store confirm that indeed my hard drive is dead. They have two pieces of good news: the first-level diagnostic at least say everything else is fine; and, hard drive replacement is only Y35170 (about U$350). There's also a kind of ironic piece of news: my hard drive was 80GB, and Apple Service doesn't stock hard drives that small anymore. Would it be OK if I got a 200GB hard drive instead? Yes, that would be just fine. They took my number and said they'll call when it's done, which should be about a week.

Aug. 16-28th

An intense two weeks of catching up from SIGGRAPH and working frantically on the SIGGRAPH Animation Festival, all while holding down my regular job as well. After two weeks with the EeePC, I have pretty much all the software I would use on a Windows machine installed: Firefox w/ various plugins; OpenOffice; FileZilla; Pidgin; QuickTime/iTunes; Putty; etc. I actually tried to install XAMPP but it didn't work. Some extra things I learned about the EeePC in this time:

• OpenOffice feels sluggish on it, although that's kind of true on any computer
  
• Firefox 3.5 runs just absolutely fine. You can't tell the speed difference at all.
  
• Not all videos I want to see play OK. YouTube at standard resolution does, but even downloaded videos don't play if they're bigger than half-size.
  
• The small screen is OK, but does feel a little crowded in Google Docs Spreadsheet or Gmail. The fact that web apps don't use screen real estate quite as efficiently as local apps matters in those cases.
  
• Speaking of Gmail, of course during this time I switched entirely to Gmail since I didn't want to store mail locally on the EeePC. I doubt I'll ever switch back, this having-all-your-email-online thing is pretty handy.

Really, I'm impressed with the EeePC. It cost me less -- even with the Bluetooth adapter -- than the repair on my MacBook, and it's a perfectly convenient way to connect to the internet. It's also really light & small. After this experience, I suspect that when I'm traveling, it'll usually be the Netbook and not the MacBook that's in the backpack, although for things like my class lectures, video editing, or 3D Apps the laptop still has a place in my computing life.

Aug. 28

Afternoon finally get a chance to call and ask what's happening with my laptop since they haven't called me. This leads to the single highly frustrating experience I had with Apple in this process: Call Center from Hell. I call the Apple Shibuya store, try to navigate the Japanese-language call system. It clearly says repair is 3, then Mac is 2, then repair is 3 again. 10 minutes' waiting on the phone (at least they gave an estimate; unfortunately the estimate isn't updated after the initial one). The guy finally gets on the phone and asks for my name.

Now, in Japan, being asked for my name on the phone is a problem. My name isn't an easy one even in English, and it's Japanese it's almost impossible to spell for someone over the phone. Japanese aren't that good at recognizing English letters on the phone to start with, and worse yet, my name has the dreaded letter 'v' in it. Japanese, even today, are not used to the letter 'v' (it's a complicated history based on how they usually imported English words into Japanese; under that system, all 'v's got changed, usually to 'b's). So, trying to get someone to recognize the letter 'v' in a spelled English word is just an exercise in futility. We tried three different times, and he was never able to find me in the database (undoubtedly because he was hearing one or two letters wrong).

After ten minutes of this, he thankfully suggested we switch to serial number instead. It's mixed letters and number, but no 'v', so it went much better. He then confirmed various different things with me, then announced he didn't have the information there, but would try to call and get it; would I please stay on the line? Another wait wait wait.

He came back with the piece de resistance. He couldn't find out what the status of my laptop was. To find out, I would have to, according to him, call the Apple Store in Shibuya.

I was so dumbfounded at this bit of call center idiocy that I had a hard time coming up with the appropriate Japanese for "I did, that's how I was lucky enough to be transferred to you." Oh, he said, he doesn't work at the Apple Shibuya Store. I had figured that out already, but telling me to start the process over again to get back to him wasn't going to help any, especially if it took another half an hour. I was pretty unhappy with all call center operators at this point even if they speak perfect keigo (Japanese polite speech).

Finally he put me on hold again (I wondered if he was going to transfer me to the manager for upset gaijin). When he came back, he told me to call the Apple Shibuya Store, and instead of choosing 3 for repairs, choose 5 for Other. Oh, that's obvious -- to call and ask about a repair, instead of picking the option for Repair, I should choose Other (admittedley, "keep hitting 0 until you're speaking to a person" is a good first order rule for call center interactions).

OK, I reluctantly hung up, called the exact same phone number again, and hit 5. About forty times. The nice person from the store came on, asked for my service ticket number (thankfully not my name) and confirmed in 5 minutes that my laptop was ready.

Aug. 28 evening

I go into the Apple Store to pick up my laptop (you don't need an appointment for pickups). It appears to be working fine, and I ask how to do the restore from my TimeCapsule backup. The guys shows me how one of the default screens in the install lets me migrate data from a backup. Sweet! (I think at the time) I have to work late that night on Animation Festival stuff, so no change to start the restore.

Aug. 29

I spend much of the day trying to get the restore to work. The process, which I ultimately repeated about five times, is:

• Boot Mac
  
• Wait for annoying marketing "welcome to Mac" movie to finish (you can't skip to end if you want the menu)
  
• Click through choosing language, keyboard, etc.
  
• Aha! Finally! Choose "transfer information from a backup"
  
• At this point, you plug in/connect the TimeCapsule to the MacBook.
  
• A list of one item appears; you click on the entry for your TimeCapsule
  
• A wait of about one hour ensues while the screen is hung on "Checking contents of TimeCapsule"
  
• During this wait, you have to keep coming by and poking the keyboard to prevent the Mac from going to sleep. If it goes to sleep, it concludes it can't use the backup, and you have to force power-off and repeat from Step 1.
  
• Finally, the check is done and you can choose "Transfer". A dialog appears that has four item (they're something like "Users", "Files and Folders", something like that). It prints the size of the first two, then display "Calculating Size" next to the third and fourth.

I got to that point, I think, four times. I waited about 90 minutes the first time, and the bar still said 0%. I kept trying minor variations -- nothing changed. Finally, since I had a party to go to Saturday night, I set it off again about 8pm and left. Even when I got home, I just let it run.

Aug. 30 Morning

After 17 hours, it still said "0%". I concluded this just wasn't going to work. I didn't really want to go back to the Apple Store with my MacBook and my TimeCapsule, so I tried another tack.

• Boot system
  
• Insert OS install CD
  
• Hold down option to force CD boot
  
• Wait for screen to come up (about a minute)
  
• Choose language, keyboard again
  
• Ooh, look, among the options in the utilities menu is "restore disk from backup". Let's try that.
  
• This utility has a quite different menu for picking a backup. It correctly displays that there are two backup images on the TimeCapsule. A little poking around lets me verify which one has the most recent (July 30) backup.
  
• Pick that. It says, "Calculating size of backup". Uh-oh, I think here we go again.
  
• But no, after about ten minutes it displays the size and offers a "Restore" button!
  
• I press that. A progress bar appears. It's slow, but only expectedly so -- it claims the restore will take about 4 1/2 hours.
  
• The restore actually takes only about 3 hr 45 min.
  
• Exciting! Reboot, and we're done, right?

August 30 Afternoon

OK, I hit reboot. The good news is, it reboots! And, when it comes up, it even recognizably is my desktop, with my files on it! Hallelujah!

...but not quite. The system is "unresponsive" as in, clicking the Apple menu produces a 15 second wait before the menu actually appears. Every action is infinitely slow. It's so painful I can't even get to Activity Monitor.app to see what's going on, but after ten teeth-pulling minutes I get to a Terminal window and type top. A minute or so later a display appears. From the intermittant updates on the display, it looks like the machine is 60% idle, but something called 'mdserver' is taking up a fair amount of the time getting used.

Back to the "trusty Netbook" as I was now calling it. Google for mdserver, and find a hilarious site called "the great crusade against mdserver." It turns out it's the server process that indexes your disk "in the background" (that's funny) to enable Spotlight searches. One user described almost the same phenomenon I was seeing, and a snarky know-it-all replied, "Well, if you've changed a lot of files lately, this is expected." Hello, Apple? Locking up my machine after a restore... is not expected.

I was glad I looked things up before just killing the process, not because killing the process would hurt anything, but because the mdserver processes, like zombies, just keep coming back even if you kill them. That information saved me some extra frustration. OK, looks like the machine belongs to effing Spotlight for the next while... time to go to the gym.

August 30 Evening

When I got back from the gym a couple hours later, then played a few soothing rounds of Flower on the PS3 and taken a shower, in fact, Spotlight had decided it had consumed enough of my computer time and quiesced. But then GDClent-something was active. Oh right, once upon a time (when Spotlight really didn't work well at all) I installed Google Desktop Search. Off to hunt through System Preferences.app for the place to turn that off completely (only two places needed in the end), and finally my machine is responsive.

The network is still kind of slow, but that's not Apple's problem -- it happens whenever it's rainy here in Tokyo (we have a typhoon right now).

The Bottom Line

In the end, I got off pretty lightly. The fact my backpack was unzipped that day cost me:

• $300 for a netbook
  
• $15 for a netbook Bluetooth adapter
  
• $350 for the disk drive replacement (but it's bigger now)
  

My backup as of July 30 ultimately worked, despite the above trevails.

The Good Things

• Actually, the MacBook Pro is pretty tough. None of the electronics were bothered by the drop.
  
• Time Machine is a really really good thing. All Mac users should be using it.
  
• Apple Repair Service is effective, and not that terribly expensive. I've partially dissembled my MacBook before, and like most laptops I didn't want to go any farther. But call them, don't wait for them to call you.
  
• The Netbook is really pretty effective. I doubt I'll take the laptop on the road much anymore.
  
• I have learned to live by Gmail, and love it. The days of Mail.app are pretty much over for me.

The Bad Things

• Apple's Voice Mail menus suck, just like every other large corporation's customer support.
  
• The restore option in the default installer.... not so good. Anything that takes 4-8 hours per trial -> Boooooo.

Some people never learn

And now, off to install Snow Leopard!

Cool tricks with the Django admin interface

At work we've been doing lots with the Django toolkit. One of the things we've been looking at lately is whether we can (without overriding lots of code) have a different admin interface for inserting an object versus updating an object. The answer: yes, trivially.

I don't have space for a full Django tutorial here, so I'll assume you know all the Django lingo. This example assumes you have a project set up, and within that you have an application called AssetVersions, and within that you have a class called Asset (all of which, I have, in fact). What if you wanted the insert interface to have the field 'name' and 'project', but wanted the update interface to have all fields except 'project'? (this makes 'project' an insert-time-only property).

• In your admin.py for AssetVersions, create a separate AdminModel class for Asset, and add it to a separate instance of the AdminSite class. We'll stick that instance on admin.create_site for convenience:
  

  
  import models
  from django.contrib import admin
  from django import forms
  
  class AssetAdmin(admin.ModelAdmin):
      exclude = ( 'project', )
  admin.site.register(models.Asset,AssetAdmin)
  
  admin.create_site = admin.AdminSite()
  
  class AssetAdmin(admin.ModelAdmin):
      fields = ( 'name', 'project', )
  admin.create_site.register(models.Asset,AssetAdmin)



• Then, in the urls.py for your project, override the url for the specific application and Model class to access the alternate AdminSite:
  
  

  from django.conf.urls.defaults import *
  from django.contrib import admin
  admin.autodiscover()
  
  
  urlpatterns = patterns('',
      (r'^', include('AssetVersions.urls')),
      url(r'^admin/(AssetVersions/asset/add/)',  admin.create_site.root),
      url(r'^admin/(.*)',                        admin.site.root),
  )

Voila, the create and edit interfaces are now decoupled, and yet each AdminSite is pretty much largely ignorant of the other.

"If I want to see the future of gaming, I go to South Korea. If I want to see the past, I go to Japan"

Very interesting interview with Rich Hilleman. Although his current title is Chief Creative Officer, he's in fact a die-hard programmer who used to run the training program for programmers at EA. He has been developing games for a long time, and is a hardcore gamer himself.

http://games.venturebeat.com/2009/08/26/eas-chief-creative-officer-describes-game-industrys-re-engineering/

I thought he nails a lot of the things that are changing in the business right now.