Japanese Health Care FTW

Today I had an experience which, while it's normal here, is inconceivable in the US. For those of you who still hold the delusion that the US has a functional health care system, wake up.

Last night I realized I needed to get some more contacts because I was almost out. So today I went to the contact store (no appointment, no calling ahead). They asked me about my order and then, because this was only the second time I had bought from them, told me I had to get my eyes re-checked (normally, you only need to get your eyes re-checked once a year, similar to the US).

Now, if I was in the US, first of all the eye doctor would probably be somewhere else than the contact lens store, and secondly, getting an appointment at my eye doctor in the US meant calling a week ahead -- or three weeks ahead if you want a Saturday appointment, since today was a Saturday.

However, here in Japan, they just said, "Go to the 5th floor." I went to the 5th floor where the nice receptionists took a look at my insurance card (health insurance is mandatory in Japan btw), and asked me to wait. I waited about 3 minutes, had my vision checked by an assistant, took my contacts out, waited another three minutes, had the actual eye doctor check my retinas and other things, waited another about 5 minutes (while watching the Olympics) and got a new pair of contacts. Total time: 20 minutes; co-pay: 380 yen (about 4 dollars).

Then, back downstairs to buy the contacts. Now, you would think the price for all this convenience would be that the actual contacts are more expensive. And you would be wrong. My contacts cost about 20% less than 1-800-contacts did back in the US.

My fellow Americans: your health care system is f*&%ed. Please get on with fixing it. It doesn't have to be like this, and the solution *does* have to involve the government.

Twentieth Century Boys

Based on high recommendations from my friend Gilles, I just rented and watched 20th Century Boys (Pt. 1) last night.

The basic idea of the story is that a group of school friends make a secret clubhouse out of grass one summer to have a place to look at manga, listen to Jim Morrison, Jimi Hendrix, and so forth on the radio, and share what passes for porn in junior high school. As part of passing the time, they make up a symbol for their group (an up-pointing finger over an eyeball) and create a group sketchbook that shows a series of attacks on the world that the friends come together to defeat. 25 years later, world events seem to be eerily echoing the sketchbook from their childhood, and they're forced to come together again to deal with this fact.

It's an interesting take on storytelling, since the story spans the time from the early 70s through 2015. It pops back and forth in time a lot, although they're pretty consistent about using a supertitled date to tell you where you are. That was absolutely essential towards the end of the movie, when they started jumping around by a few days instead of 20 years. The basic idea is interesting, and in the story the characters try to access 25-year-old memories to attempt to reconstruct the details of that long-ago summer, so cutting back to those scenes lets the audience learn the memories as the characters relive them. Also, the casting is awesome; they did a fantastic job of getting child actors who all plausibly look like the child versions of the adult actors.

They picked well for the central character to tell the story through: Kenji was a close to a leader as the group had back in the day, with personal style that eventually led him into a career as a rock'n'roller -- which in turn caused him to drink his way down out of society. Now he's a convenience store clerk (the initial scenes of the manager exhorting him to practice the chain's greeting more sincerely is just one of the many little societal commentaries in the film) who's raising his niece as a single parent.

Unfortunately, I think the weakest point of the story is the one widely shared among Manga adaptations, whether animated or live action -- the storytelling is frequently elided. That is to say, there are just scenes you need to have that aren't there. Towards the end, the enemy (ironically named "Friend", he heads up a cult/political party called "Friend Party" / 友民党) has sent a giant robot into Tokyo and our group gets together and heads out to stop him. Cut to, shot of one of the group tackling Friend from behind on the top of a building. How did they get there? How did he know where Friend was? etc. is never explained. Another member of the group, for equally unknowable reasons, happens to be watching from the next rooftop.

This shortcut storytelling is so widespread among Manga adaptations it must be a fairly conscious choice (if you watch standard Japanese TV dramas or live action movies, they don't suffer from this problem, so it's not like storytelling is unknown here). I've always assumed that it comes from the fact that the main audience in Japan for a Manga adaptation is people who are already fans of the Manga -- thus, the filmmakers assume the audience already basically knows the story, and wants to get to the "good parts", i.e., the scenes where the characters' emotions are clearly on display and/or the action scenes, rather than actually clearly retell a story the audience already knows. In terms of the domestic market for a film like this, that makes perfect sense; unfortunately, and to the great detriment of the industry here, it severely limits the market for the film outside the Manga's pre-existing fanbase. Most worldwide audiences do prefer their stories clearly told rather than needing to guess at the missing pieces.

Please get wasted at home

From Tokyo

There's a public service campaign going on right now about not putting on your makeup, drinking soda, etc. while riding the train -- the tagline is always, "Please do it at home." The latest poster addresses a different behavior to be practiced in the home, though.

Note for those not living in Japan: this is not a joke, these posters are really up all over Tokyo right now.

How to Attend the Electronic Theater at SIGGRAPH Asia 2009

こんにちは、
（English follows the Japanese)

僕は今年の横浜のシーグラフアジアの
コンピューターアニメーションフェスティバルチェアーで,
フェスティバルの主要なエベントはエレクトロニックシアターです。

エレクトロニックシアターの上映の予定は、
１２月の１７日の木曜日の夕方のプレミアショーと１８日の金曜日の夕方のショーを行いますが、
１９日の土曜日の上映も二つ予定されます。午後の 16:15 も夕方の 19:00 もあります。
エレクトロニックシアターは２時間ぐらい掛かります。
エレクトロニックシアターの内容とは、下記のリンクを参照して
「エレクトロニックシアター」にクリックしてください。
エレクトロニックシアターは横浜パシフィコのメインシアターでやって、
ソニーマーケティング日本に上がって頂いた４K のディジタルシネマプロジェクターを使います。
フルHD の変種の作業はイマジカでやっています。

CAF ET Program

他のシーグラフのエベントに参加しなくてもエレクトロニックシアターに出席したい方は、
下記のリンクを参照するとクレジットカードでチケットの何枚を買えます。

ET Tickets

エレクトロニックシアターに上にシーグラフアジアの登録したい場合は、
一般的なシーグラフアジアの日本語のサイトを参照して下さい。

SIGGRAPH Asia Registration

そのメッセージは他の興味がある方に自由に転送すると幸いです。
質問または不明な点になればぜひ私にご連絡ください。

宜しくお願いします。

レオ

Hi all,

I'm the chair for this year's SIGGRAPH Asia Computer Animation Festival in Yokohama, and the main event of the festival is the Electronic Theater.

The schedule for screenings for the Electronic Theater includes a premiere show on the evening of Thursday, December 17th as well as a show on the evening of Friday the 18th, but there are also shows scheduled on Saturday the 19th, at both 4:15pm and 7pm. The Electronic Theater lasts about two hours. For the contents of the Electronic Theater, check the web page below and click "Electronic Theater". The Electronic Theater will be presented via a 4K Digital Cinema projector donated by Sony Marketing (Japan) in the Main Theater at Pacifico Yokohama. The full-resolution HD edit is being carried out at Imagica.

CAF ET Program

For people who don't want to participate in other SIGGRAPH events but just want to go to the Electronic Theater, please go to the link below, and you can purchase any number of tickets via credit card.

CAF ET Tickets

If you want to go to other SIGGRAPH Asia events besides the Electronic Theater, you can of course visit the main SIGGRAPH Asia site:

SIGGRAPH Asia Registration

Please feel free to forward this message to any other interested people, and if you have any questions, or if anything is unclear, please get in touch.

Yours,

Leo

Disappointing Books #1, "Castles, Battles, and Bombs"

I've read two disappointing books recently, so in the interest of helping others avoid the mistake, I'll share.

"Castles, Battles, and Bombs" by Jurgen Brauer and Hubert van Tuyll has a very clear concept: the authors attempt to study military history through the tools of standard (aka neoclassical) economics. They choose six periods of history, pair them with six principles of economics (opportunity cost, substitution effects, marginal cost vs. marginal beenfit, etc.) and attempt to use the economics principle to understand the period of history.

It's a neat idea; too bad it doesn't add much to the understanding of history. Now, I'm not saying the authors are wrong in their observations. Yes, clearly, building castles in the High Middle Ages reduced the funds kings had available to devote to maintaining field armies. But what was true in pretty much every one of the six chapter of this book is that the economic principle, while at least somewhat applicable, is not key to understanding the period in question.

For instance, they study Renaissance Italy via the economic question known as the principal-agent problem, which is to say, when you hire something to do something, how do you know they (your agent) are doing what you (the principal) want? Since Italy was constantly at war through most of this period, and that war was primarily conducted via hired mercenaries, the condottori (literally meaning contractors), it's easy to see why they matched the two.

While the book goes through the considerable lengths the Italian city-states went to in order to mitigate the principal-agent problem, all of which make sense (essentially, an auditing corps of city employees were used to keep track of the activities of the condottieri, as well as a tendancy for longer-term relationships), it's not believable that the principal-agent problem was a cause for the commencement or ending of this era. The principal-agent problem has to be managed any time you engage in a contract, yes; but since the problem existed throughout the era in study, and roughly equally for all participants, I felt like I learned nothing new about the era by studying these. The real reasons why the mercenary system existed during that era, why the city-states chose it, and why they eventually chose otherwise, all lie outside the area of study.

Every chapter suffers from this failing, in that while the principle does apply, it doesn't offer key insights. In the fifth chapter of the book, they go over the well-trodden data showing that (non-nuclear) strategic bombing is not economically justifiable. That is to say, the total economic costs to an aggressor country of building and operating a combined-arms air force capable of conducting a strategic bombing campaign is greater than the economic damage done to the enemy by said campaign (factories, surprisingly, turn out to be pretty easy to repair). They use the extensive data on WWII's air campaign against Germany to show this. Unfortunately, as they acknowledge, even the post-war analysis within the US military reached the same conclusion about cost-benfits 60 years ago. They claim this is a lesson in marginal cost vs. marginal benefit, but in fact what the data show is that it's simply a bad military investment in the first place, you don't have to think about marginal cost vs. marginal benefit. Allocating those resources to tactical air support of operations instead of conducting strategic bombing at all is the conclusion supported by the data.

The worst case of this is the chapter on castle-building vs. field army maintenance in the High Middle Ages. They attempt to pitch the preference for castle-building as causing a lost opportunity, in that the monarchs couldn't also maintain a field army. But what their data shows is that the cost of castle-building, even for aggressive castle builders like Edward I in England, pales beside the costs of war. Their own data shows Edward spent £90,000 on Welsh castles, but £1,400,000 on fielding armies for his wars against Wales, Scotland, and France. Castles are so cheap relative to armies that it's distorting the data to say they represented a lost opportunity.

In the end, this book does explore its thesis. Unfortunately, I think the book shows that the thesis isn't worth taking much farther. Is it helpful for a historian to understand the basic principles of economics? Yes. But will they be a primary guide to understanding past actions? Based on these examples, no.

The Death and Rebirth of a Laptop

Chronology of An Ordeal

Over the last 2 1/2 weeks, I've dealt with dropping my MacBook Pro and the various fallout from that. I'm writing down the full experience, for that cathartic "getting over it" benefit -- that way, this can be my last post on the topic!

Aug 11 afternoon

While seeing "Up" at the Kabuki Theater in SF, my MacBook falls out of my backpack from a height of about 3 feet. The person behind me squealed as it was happenning, so I turned around in time to see it happen. It was very depressing. In order to avoid thinking about it during the movie, I didn't test it at all at that point.

Aug 11 evening

Got back to hotel, verified something was Very Very Wrong. The machine was still awake and I could activate various apps, but they all went into wait cursor state.

Aug 11 evening 2

In order to connect to the rest of the world while the Mac was on hiatus, I went to Best Buy in San Francisco and bought a netbook (Eee PC in particular). For $300. Incredibly cheap. Since planned dinner didn't happen, I spent much of the evening installing Firefox/Pidgin/etc. The trackpad's buttons are *incredibly* irritating -- the left mouse button sticks unless you imemdiately push a little on the other side.

Aug. 13

On the plane back to Tokyo, I realize that since I bought this MacBook Pro at SIGGRAPH 2006 at Boston, it is exactly two weeks out of extended warranty. Hah hah hah, isn't that funny.

Aug. 14

evening Make appointment with Apple Store for Sunday morning.

Aug. 16

Off to Labi in Shibuya, bought a Bluetooth adapter for the Netbook. This thing is hilarious, it's literally the size of your thumbnail. It worked flawlessly: XP recognized it immediately, and I could then pair the EeePC with the wireless keyboard and mouse I used with my laptop. Using the mouse meant I didn't need to use the incredibly annoying trackpad button on the EeePC, eliminating the biggest reason to hate the thing.

Aug. 16

The Geniuses at the Apple Store confirm that indeed my hard drive is dead. They have two pieces of good news: the first-level diagnostic at least say everything else is fine; and, hard drive replacement is only Y35170 (about U$350). There's also a kind of ironic piece of news: my hard drive was 80GB, and Apple Service doesn't stock hard drives that small anymore. Would it be OK if I got a 200GB hard drive instead? Yes, that would be just fine. They took my number and said they'll call when it's done, which should be about a week.

Aug. 16-28th

An intense two weeks of catching up from SIGGRAPH and working frantically on the SIGGRAPH Animation Festival, all while holding down my regular job as well. After two weeks with the EeePC, I have pretty much all the software I would use on a Windows machine installed: Firefox w/ various plugins; OpenOffice; FileZilla; Pidgin; QuickTime/iTunes; Putty; etc. I actually tried to install XAMPP but it didn't work. Some extra things I learned about the EeePC in this time:

• OpenOffice feels sluggish on it, although that's kind of true on any computer
  
• Firefox 3.5 runs just absolutely fine. You can't tell the speed difference at all.
  
• Not all videos I want to see play OK. YouTube at standard resolution does, but even downloaded videos don't play if they're bigger than half-size.
  
• The small screen is OK, but does feel a little crowded in Google Docs Spreadsheet or Gmail. The fact that web apps don't use screen real estate quite as efficiently as local apps matters in those cases.
  
• Speaking of Gmail, of course during this time I switched entirely to Gmail since I didn't want to store mail locally on the EeePC. I doubt I'll ever switch back, this having-all-your-email-online thing is pretty handy.

Really, I'm impressed with the EeePC. It cost me less -- even with the Bluetooth adapter -- than the repair on my MacBook, and it's a perfectly convenient way to connect to the internet. It's also really light & small. After this experience, I suspect that when I'm traveling, it'll usually be the Netbook and not the MacBook that's in the backpack, although for things like my class lectures, video editing, or 3D Apps the laptop still has a place in my computing life.

Aug. 28

Afternoon finally get a chance to call and ask what's happening with my laptop since they haven't called me. This leads to the single highly frustrating experience I had with Apple in this process: Call Center from Hell. I call the Apple Shibuya store, try to navigate the Japanese-language call system. It clearly says repair is 3, then Mac is 2, then repair is 3 again. 10 minutes' waiting on the phone (at least they gave an estimate; unfortunately the estimate isn't updated after the initial one). The guy finally gets on the phone and asks for my name.

Now, in Japan, being asked for my name on the phone is a problem. My name isn't an easy one even in English, and it's Japanese it's almost impossible to spell for someone over the phone. Japanese aren't that good at recognizing English letters on the phone to start with, and worse yet, my name has the dreaded letter 'v' in it. Japanese, even today, are not used to the letter 'v' (it's a complicated history based on how they usually imported English words into Japanese; under that system, all 'v's got changed, usually to 'b's). So, trying to get someone to recognize the letter 'v' in a spelled English word is just an exercise in futility. We tried three different times, and he was never able to find me in the database (undoubtedly because he was hearing one or two letters wrong).

After ten minutes of this, he thankfully suggested we switch to serial number instead. It's mixed letters and number, but no 'v', so it went much better. He then confirmed various different things with me, then announced he didn't have the information there, but would try to call and get it; would I please stay on the line? Another wait wait wait.

He came back with the piece de resistance. He couldn't find out what the status of my laptop was. To find out, I would have to, according to him, call the Apple Store in Shibuya.

I was so dumbfounded at this bit of call center idiocy that I had a hard time coming up with the appropriate Japanese for "I did, that's how I was lucky enough to be transferred to you." Oh, he said, he doesn't work at the Apple Shibuya Store. I had figured that out already, but telling me to start the process over again to get back to him wasn't going to help any, especially if it took another half an hour. I was pretty unhappy with all call center operators at this point even if they speak perfect keigo (Japanese polite speech).

Finally he put me on hold again (I wondered if he was going to transfer me to the manager for upset gaijin). When he came back, he told me to call the Apple Shibuya Store, and instead of choosing 3 for repairs, choose 5 for Other. Oh, that's obvious -- to call and ask about a repair, instead of picking the option for Repair, I should choose Other (admittedley, "keep hitting 0 until you're speaking to a person" is a good first order rule for call center interactions).

OK, I reluctantly hung up, called the exact same phone number again, and hit 5. About forty times. The nice person from the store came on, asked for my service ticket number (thankfully not my name) and confirmed in 5 minutes that my laptop was ready.

Aug. 28 evening

I go into the Apple Store to pick up my laptop (you don't need an appointment for pickups). It appears to be working fine, and I ask how to do the restore from my TimeCapsule backup. The guys shows me how one of the default screens in the install lets me migrate data from a backup. Sweet! (I think at the time) I have to work late that night on Animation Festival stuff, so no change to start the restore.

Aug. 29

I spend much of the day trying to get the restore to work. The process, which I ultimately repeated about five times, is:

• Boot Mac
  
• Wait for annoying marketing "welcome to Mac" movie to finish (you can't skip to end if you want the menu)
  
• Click through choosing language, keyboard, etc.
  
• Aha! Finally! Choose "transfer information from a backup"
  
• At this point, you plug in/connect the TimeCapsule to the MacBook.
  
• A list of one item appears; you click on the entry for your TimeCapsule
  
• A wait of about one hour ensues while the screen is hung on "Checking contents of TimeCapsule"
  
• During this wait, you have to keep coming by and poking the keyboard to prevent the Mac from going to sleep. If it goes to sleep, it concludes it can't use the backup, and you have to force power-off and repeat from Step 1.
  
• Finally, the check is done and you can choose "Transfer". A dialog appears that has four item (they're something like "Users", "Files and Folders", something like that). It prints the size of the first two, then display "Calculating Size" next to the third and fourth.

I got to that point, I think, four times. I waited about 90 minutes the first time, and the bar still said 0%. I kept trying minor variations -- nothing changed. Finally, since I had a party to go to Saturday night, I set it off again about 8pm and left. Even when I got home, I just let it run.

Aug. 30 Morning

After 17 hours, it still said "0%". I concluded this just wasn't going to work. I didn't really want to go back to the Apple Store with my MacBook and my TimeCapsule, so I tried another tack.

• Boot system
  
• Insert OS install CD
  
• Hold down option to force CD boot
  
• Wait for screen to come up (about a minute)
  
• Choose language, keyboard again
  
• Ooh, look, among the options in the utilities menu is "restore disk from backup". Let's try that.
  
• This utility has a quite different menu for picking a backup. It correctly displays that there are two backup images on the TimeCapsule. A little poking around lets me verify which one has the most recent (July 30) backup.
  
• Pick that. It says, "Calculating size of backup". Uh-oh, I think here we go again.
  
• But no, after about ten minutes it displays the size and offers a "Restore" button!
  
• I press that. A progress bar appears. It's slow, but only expectedly so -- it claims the restore will take about 4 1/2 hours.
  
• The restore actually takes only about 3 hr 45 min.
  
• Exciting! Reboot, and we're done, right?

August 30 Afternoon

OK, I hit reboot. The good news is, it reboots! And, when it comes up, it even recognizably is my desktop, with my files on it! Hallelujah!

...but not quite. The system is "unresponsive" as in, clicking the Apple menu produces a 15 second wait before the menu actually appears. Every action is infinitely slow. It's so painful I can't even get to Activity Monitor.app to see what's going on, but after ten teeth-pulling minutes I get to a Terminal window and type top. A minute or so later a display appears. From the intermittant updates on the display, it looks like the machine is 60% idle, but something called 'mdserver' is taking up a fair amount of the time getting used.

Back to the "trusty Netbook" as I was now calling it. Google for mdserver, and find a hilarious site called "the great crusade against mdserver." It turns out it's the server process that indexes your disk "in the background" (that's funny) to enable Spotlight searches. One user described almost the same phenomenon I was seeing, and a snarky know-it-all replied, "Well, if you've changed a lot of files lately, this is expected." Hello, Apple? Locking up my machine after a restore... is not expected.

I was glad I looked things up before just killing the process, not because killing the process would hurt anything, but because the mdserver processes, like zombies, just keep coming back even if you kill them. That information saved me some extra frustration. OK, looks like the machine belongs to effing Spotlight for the next while... time to go to the gym.

August 30 Evening

When I got back from the gym a couple hours later, then played a few soothing rounds of Flower on the PS3 and taken a shower, in fact, Spotlight had decided it had consumed enough of my computer time and quiesced. But then GDClent-something was active. Oh right, once upon a time (when Spotlight really didn't work well at all) I installed Google Desktop Search. Off to hunt through System Preferences.app for the place to turn that off completely (only two places needed in the end), and finally my machine is responsive.

The network is still kind of slow, but that's not Apple's problem -- it happens whenever it's rainy here in Tokyo (we have a typhoon right now).

The Bottom Line

In the end, I got off pretty lightly. The fact my backpack was unzipped that day cost me:

• $300 for a netbook
  
• $15 for a netbook Bluetooth adapter
  
• $350 for the disk drive replacement (but it's bigger now)
  

My backup as of July 30 ultimately worked, despite the above trevails.

The Good Things

• Actually, the MacBook Pro is pretty tough. None of the electronics were bothered by the drop.
  
• Time Machine is a really really good thing. All Mac users should be using it.
  
• Apple Repair Service is effective, and not that terribly expensive. I've partially dissembled my MacBook before, and like most laptops I didn't want to go any farther. But call them, don't wait for them to call you.
  
• The Netbook is really pretty effective. I doubt I'll take the laptop on the road much anymore.
  
• I have learned to live by Gmail, and love it. The days of Mail.app are pretty much over for me.

The Bad Things

• Apple's Voice Mail menus suck, just like every other large corporation's customer support.
  
• The restore option in the default installer.... not so good. Anything that takes 4-8 hours per trial -> Boooooo.

Some people never learn

And now, off to install Snow Leopard!

Cool tricks with the Django admin interface

At work we've been doing lots with the Django toolkit. One of the things we've been looking at lately is whether we can (without overriding lots of code) have a different admin interface for inserting an object versus updating an object. The answer: yes, trivially.

I don't have space for a full Django tutorial here, so I'll assume you know all the Django lingo. This example assumes you have a project set up, and within that you have an application called AssetVersions, and within that you have a class called Asset (all of which, I have, in fact). What if you wanted the insert interface to have the field 'name' and 'project', but wanted the update interface to have all fields except 'project'? (this makes 'project' an insert-time-only property).

• In your admin.py for AssetVersions, create a separate AdminModel class for Asset, and add it to a separate instance of the AdminSite class. We'll stick that instance on admin.create_site for convenience:
  

  
  import models
  from django.contrib import admin
  from django import forms
  
  class AssetAdmin(admin.ModelAdmin):
      exclude = ( 'project', )
  admin.site.register(models.Asset,AssetAdmin)
  
  admin.create_site = admin.AdminSite()
  
  class AssetAdmin(admin.ModelAdmin):
      fields = ( 'name', 'project', )
  admin.create_site.register(models.Asset,AssetAdmin)



• Then, in the urls.py for your project, override the url for the specific application and Model class to access the alternate AdminSite:
  
  

  from django.conf.urls.defaults import *
  from django.contrib import admin
  admin.autodiscover()
  
  
  urlpatterns = patterns('',
      (r'^', include('AssetVersions.urls')),
      url(r'^admin/(AssetVersions/asset/add/)',  admin.create_site.root),
      url(r'^admin/(.*)',                        admin.site.root),
  )

Voila, the create and edit interfaces are now decoupled, and yet each AdminSite is pretty much largely ignorant of the other.

"If I want to see the future of gaming, I go to South Korea. If I want to see the past, I go to Japan"

Very interesting interview with Rich Hilleman. Although his current title is Chief Creative Officer, he's in fact a die-hard programmer who used to run the training program for programmers at EA. He has been developing games for a long time, and is a hardcore gamer himself.

http://games.venturebeat.com/2009/08/26/eas-chief-creative-officer-describes-game-industrys-re-engineering/

I thought he nails a lot of the things that are changing in the business right now.

Saw "Up"

Before I left the US last week, I saw Up in 3D at the Sundance Kabuki. It was awesome. That said, I now understand this post much better (warning: language and lots of it).

Awesome interface at SIGGRAPH 2009

I just tried this about an hour ago, and it's totally awesome (and a little bit freaky):

I know it's popular in America but still...

...the sign in Dallas airport that said, "Y'all roll on in for some Fresh Sushi!" still surprised me.

...fresh sushi in Dallas comes from where exactly?

Best Web UI Feature ever

Yelp totally takes the cake for my favorite little bit of Web UI. In their site, they have a map widget in the right-hand column which autoscrolls to stay on the screen. I always find these autoscrolling web widgets annoying -- they slow down the page, and make the UI jumpy. Apparently I'm not alone, because Yelp's widget has a special checkbox on it:



Now that is appropriate UI text!

Discussion about 3D tool development and mainstream development

In the course of a discussion on a SIGGRAPH alias about something else (Plone), someone asked the question below, and my response got kind of long. While I'm sure most people's eyes will simply glaze over (a perfectly justifiable response), it reminded me of many a Sunday morning conversation on a ridgetop above Marin, so I'm forwarding here.

------------------------

Gustavo's question:

This is Gustavo, I am working on the migration to Plone 3. Quick question, reading your post I found this line quite intriguing:

"develop web apps in Django all day at this point -- computer animation pipeline programming is converging with mainstream development frighteningly quickly"

I am quite intrigued about that, what sort of applications are you talking about? Do you have any pointers or links to pages that talk about it? I am really interested on it

Thanks,

Gustavo

---------------------
My answer:

Hi Gustavo,

Sure, here's what I was referring to. None of it is exactly public, so I can't point you to web pages or anything, but it's not exactly secret either.

The previous generation of CG pipeline tools was largely implemented inside of the DCC (Digital Content Creation) tools (Maya, Max, XSi, Modo, etc.). Those tools all have pretty robust scripting environments nowadays, so you can implement tools in some high-level scripting language with some UI toolkit available to you. And I mean, really pretty robust environments -- we use Django as a object-relational mapper for our code that runs in Python inside of Maya.

That's all good, but when you think about it, that's fundamentally implementing tools in the client/server approach that dominated in the 1980s and 1990s. Those tools (including what Polygon was using up until this year, and the tools I implemented at EA before that):

• connect to the data source via a LAN connection
  
• use a storage-level protocol (usually SQL supplemented by file system access via CIFS/NFS)
  
• require that the code be propagated out to the client machine

For sure there are huge advantages to the DCC integration, and it's still what I recommend for artist-facing tools. However, it has some problems that are becoming especially apparent as the computer animation production world becomes more outsourced and more collaboration-oriented:

• the storage-level protocols are inappropriate to extend outside the company (you don't expose your MySQL or Samba/NFS server to the public net these days)
  
• for a lot of the legitimate stakeholders in your project (notably project managers) the Maya interface is not a preferred means of use
  
• updates to the code require that the updated code be pushed to the clients
  
• those tools are not well-connected to the rest of the computing environment -- notably Excel. Getting data in or out required custom development in every case
  
• the tools are very dependent on the client OS/environment

So, increasingly, we're putting a lot (not all) of our tools into a web-facing mode. While the files themselves are still saved via the standard Maya interface, we're moving our status-tracking apps (that keep track of the status of every model, animation, and rendered shot) to be web-facing. This means:

• extending the app to be accessible from other companies is straightforward -- the security model for web-facing applications is very well-understood
  
• you get to use modern development tools. There's nothing running inside any DCC that compares to the productivity of developing with Django/Ruby on Rails/ASP.NET inside Kodomo/Eclipse/Visual Studio
  
• users have a lot more ad hoc means of data integration (when we haven't had time to code a custom query report for you, just copy-and-paste out of the HTML table into Excel)
  
• apart from that, it's easy to integrate with community software (wiki/forum/etc) to pick up the benefits thereof
  
• printing, etc. is free (I've never seen an in-Maya application that supported printing)
  
• we're out of the software installation business, notably including not caring whether you run Windows or Mac OS X or Linux (different studios have different answers), which version of those OSes you're running, and largely not caring even which version of Maya you run

In a CG production environment, web-facing apps still have some drawbacks, notably:

• you cannot send the actual 3D data over http -- it's just too heavy (a few hundred megabytes for a set is not unusual). So the metadata travels over http but the actual data can't
  
• UI development on the web is harder than locally (IMHO)
  
• in return for getting out of the software installation business, we have to deal with the browser wars. If it wasn't for toolkits like jQuery or YUI, it almost wouldn't be viable to do this

What made me comment on the convergence of application development is that most of the above comments (maybe absent the data volume) apply to almost any kind of custom tool development. I'm very firmly convinced that except for very data-heavy applications (video, 3D scenes) web-facing application development is a huge win.

Regarding the data-heavy stuff, we did something I think is pretty interesting this year. When you have multiple companies participating in CG production, you have to share the model assets, and as I said, those are fundamentally heavy, so you won't be uploading them over http. The traditional solution to this is production managers doing a lot of FTP or running around carrying hard drives between locations; the high-end solutions (which we couldn't afford) are the companies that sell you high-bandwidth shared workspaces that you can use online.

I implemented an interesting solution this year for version control of these massive data assets -- it's like a hybrid between CVS and git. There's a shared server holding the assets (accessible via SFTP) and a web-facing status-tracking application. It's like CVS in the sense that checkout is a global operation -- to be able to edit an asset, you need to check it out on the global web-facing database, either via the web interface or via a web services call from the in-DCC tool. However, it's like git in that the actual asset is assumed to take considerable time to propagate from the committing party to all parties. The artist uses an in-Maya tool to actually check in; that tool both copies the asset to the local file server, and updates the checked-out status on the global web-facing database. Then, it kicks off an asynchronous transfer of the asset data from the local file server to the shared SFTP server. Thus, while the metadata about checkin is instantly propagated via the central server, the actual asset transfer happens later. In the current implementation, downloading from the central server to the local file server is a periodic task kicked off once a day at each company. Of course, the checkout tool enforces the constraint that you can't check out an asset unless your local file server currently holds the version known to be the latest version, and no one else has that asset checked out on the central web-facing server.

Implementation-wise, by making some assumptions about the directory structure, we created a protocol that is atomic for both upload and download even when using nothing but standard FTP operations.

Again, putting this on the public internet is very straightforward: you're only dealing with http and SFTP security (we didn't even bother to use https although it wouldn't affect the architecture at all obviously). For the same reason, you can deploy this on any generic web server, it doesn't take a fussy installation or full virtual server -- just Apache and SFTP. For users, all of their file reads/writes are happening to the local file server, so they get a very high level of service. It uses a lot of disk storage -- but that's the cheapest element of the equation these days.

Sorry, that's probably way more answer than you wanted!

Take care Gustavo,

Leo

From some poking around I did for work: Animation Feature Market 2009

All data from public sources as cited.

Digital Animation Feature Film U.S. Box Office Revenue (and Market Share) by Year
http://www.the-numbers.com/market/ProductionMethods/DigitalAnimation.php

Note that 2009 is incomplete.

• 2005 $553M
  
• 2006 $1,082M
  
• 2007 $1,179M
  
• 2008 $1,032M
  
• 2009 $463M
  

If I try to make an estimate for 2009 overall, it looks like it's going to be about the same as last year:

• Monster vs. Aliens $195M (actual)
  
• Coraline $75M (actual)
  
• Up $300M (actual $263M to date)
  
• Bolt (released in 2008) $4M (actual)
  
• The Battle for Terra $1M (actual)
  
• The Tale of Despereaux $7M (actual)
  
• Ice Age 3 $170M (actual $68M in first weekend)
  
• Ponyo $10M
  
• Sita Sings the Blues $2M
  
• Cloudy with a Chance of Meatballs $100M
  
• 9 $20M
  
• Fantastic Mr. Fox $30M
  
• Astro Boy $60M
  
• Planet 51 $35M
  
• The Princess and the Frog $100M
  
• Total $1109M
  

Traditional animation is excluded from the above but is trivially small. Admittedley, Coraline's category is open to interpretation.

There's really nothing there that would allow one to analyze the effect of 3D screens across the whole market, but for individual films it appears to generate the majority of their revenue. Here are some quotes from www.boxofficemojo.com about some particular films and weekends:

Coraline conjured $18.9 million over the four-day weekend... Coraline's 3D gross was down only seven percent, accounting for an estimated $14 million of the four-day weekend at 1,060 sites. The tally stands at $39.1 million in eleven days, around 72 percent of which coming from 3D presentations. [February 17]

Ice Age: Dawn of the Dinosaurs gathered $67.5 million in five days (55 percent of which from 1,606 3D venues). [July 5]

... with the release of Up. The adventure comedy swooped in with a $68.1 million start... the [3D] format accounted for $35.4 million of Up's gross. [June 1]

Animated Features Released to Theatres in the United States

by Jerry Beck (well-known animation historian)

• 2005 11 films
  
• 2006 21 films
  
• 2007 13 films
  
• 2008 20 films
  
• 2009 13 films known at this point (includes anticipated releases for remainder of year)

from http://www.cartoonresearch.com/movies2005.html.

• 12:20 Today is the last day for SIGGRAPH Asia 2009 Computer Animation Festival submissions... watching the counts, answering emails... #

Automatically shipped by LoudTwitter