Best Web UI Feature ever

Yelp totally takes the cake for my favorite little bit of Web UI. In their site, they have a map widget in the right-hand column which autoscrolls to stay on the screen. I always find these autoscrolling web widgets annoying -- they slow down the page, and make the UI jumpy. Apparently I'm not alone, because Yelp's widget has a special checkbox on it:



Now that is appropriate UI text!

Discussion about 3D tool development and mainstream development

In the course of a discussion on a SIGGRAPH alias about something else (Plone), someone asked the question below, and my response got kind of long. While I'm sure most people's eyes will simply glaze over (a perfectly justifiable response), it reminded me of many a Sunday morning conversation on a ridgetop above Marin, so I'm forwarding here.

------------------------

Gustavo's question:

This is Gustavo, I am working on the migration to Plone 3. Quick question, reading your post I found this line quite intriguing:

"develop web apps in Django all day at this point -- computer animation pipeline programming is converging with mainstream development frighteningly quickly"

I am quite intrigued about that, what sort of applications are you talking about? Do you have any pointers or links to pages that talk about it? I am really interested on it

Thanks,

Gustavo

---------------------
My answer:

Hi Gustavo,

Sure, here's what I was referring to. None of it is exactly public, so I can't point you to web pages or anything, but it's not exactly secret either.

The previous generation of CG pipeline tools was largely implemented inside of the DCC (Digital Content Creation) tools (Maya, Max, XSi, Modo, etc.). Those tools all have pretty robust scripting environments nowadays, so you can implement tools in some high-level scripting language with some UI toolkit available to you. And I mean, really pretty robust environments -- we use Django as a object-relational mapper for our code that runs in Python inside of Maya.

That's all good, but when you think about it, that's fundamentally implementing tools in the client/server approach that dominated in the 1980s and 1990s. Those tools (including what Polygon was using up until this year, and the tools I implemented at EA before that):

• connect to the data source via a LAN connection
  
• use a storage-level protocol (usually SQL supplemented by file system access via CIFS/NFS)
  
• require that the code be propagated out to the client machine

For sure there are huge advantages to the DCC integration, and it's still what I recommend for artist-facing tools. However, it has some problems that are becoming especially apparent as the computer animation production world becomes more outsourced and more collaboration-oriented:

• the storage-level protocols are inappropriate to extend outside the company (you don't expose your MySQL or Samba/NFS server to the public net these days)
  
• for a lot of the legitimate stakeholders in your project (notably project managers) the Maya interface is not a preferred means of use
  
• updates to the code require that the updated code be pushed to the clients
  
• those tools are not well-connected to the rest of the computing environment -- notably Excel. Getting data in or out required custom development in every case
  
• the tools are very dependent on the client OS/environment

So, increasingly, we're putting a lot (not all) of our tools into a web-facing mode. While the files themselves are still saved via the standard Maya interface, we're moving our status-tracking apps (that keep track of the status of every model, animation, and rendered shot) to be web-facing. This means:

• extending the app to be accessible from other companies is straightforward -- the security model for web-facing applications is very well-understood
  
• you get to use modern development tools. There's nothing running inside any DCC that compares to the productivity of developing with Django/Ruby on Rails/ASP.NET inside Kodomo/Eclipse/Visual Studio
  
• users have a lot more ad hoc means of data integration (when we haven't had time to code a custom query report for you, just copy-and-paste out of the HTML table into Excel)
  
• apart from that, it's easy to integrate with community software (wiki/forum/etc) to pick up the benefits thereof
  
• printing, etc. is free (I've never seen an in-Maya application that supported printing)
  
• we're out of the software installation business, notably including not caring whether you run Windows or Mac OS X or Linux (different studios have different answers), which version of those OSes you're running, and largely not caring even which version of Maya you run

In a CG production environment, web-facing apps still have some drawbacks, notably:

• you cannot send the actual 3D data over http -- it's just too heavy (a few hundred megabytes for a set is not unusual). So the metadata travels over http but the actual data can't
  
• UI development on the web is harder than locally (IMHO)
  
• in return for getting out of the software installation business, we have to deal with the browser wars. If it wasn't for toolkits like jQuery or YUI, it almost wouldn't be viable to do this

What made me comment on the convergence of application development is that most of the above comments (maybe absent the data volume) apply to almost any kind of custom tool development. I'm very firmly convinced that except for very data-heavy applications (video, 3D scenes) web-facing application development is a huge win.

Regarding the data-heavy stuff, we did something I think is pretty interesting this year. When you have multiple companies participating in CG production, you have to share the model assets, and as I said, those are fundamentally heavy, so you won't be uploading them over http. The traditional solution to this is production managers doing a lot of FTP or running around carrying hard drives between locations; the high-end solutions (which we couldn't afford) are the companies that sell you high-bandwidth shared workspaces that you can use online.

I implemented an interesting solution this year for version control of these massive data assets -- it's like a hybrid between CVS and git. There's a shared server holding the assets (accessible via SFTP) and a web-facing status-tracking application. It's like CVS in the sense that checkout is a global operation -- to be able to edit an asset, you need to check it out on the global web-facing database, either via the web interface or via a web services call from the in-DCC tool. However, it's like git in that the actual asset is assumed to take considerable time to propagate from the committing party to all parties. The artist uses an in-Maya tool to actually check in; that tool both copies the asset to the local file server, and updates the checked-out status on the global web-facing database. Then, it kicks off an asynchronous transfer of the asset data from the local file server to the shared SFTP server. Thus, while the metadata about checkin is instantly propagated via the central server, the actual asset transfer happens later. In the current implementation, downloading from the central server to the local file server is a periodic task kicked off once a day at each company. Of course, the checkout tool enforces the constraint that you can't check out an asset unless your local file server currently holds the version known to be the latest version, and no one else has that asset checked out on the central web-facing server.

Implementation-wise, by making some assumptions about the directory structure, we created a protocol that is atomic for both upload and download even when using nothing but standard FTP operations.

Again, putting this on the public internet is very straightforward: you're only dealing with http and SFTP security (we didn't even bother to use https although it wouldn't affect the architecture at all obviously). For the same reason, you can deploy this on any generic web server, it doesn't take a fussy installation or full virtual server -- just Apache and SFTP. For users, all of their file reads/writes are happening to the local file server, so they get a very high level of service. It uses a lot of disk storage -- but that's the cheapest element of the equation these days.

Sorry, that's probably way more answer than you wanted!

Take care Gustavo,

Leo

From some poking around I did for work: Animation Feature Market 2009

All data from public sources as cited.

Digital Animation Feature Film U.S. Box Office Revenue (and Market Share) by Year
http://www.the-numbers.com/market/ProductionMethods/DigitalAnimation.php

Note that 2009 is incomplete.

• 2005 $553M
  
• 2006 $1,082M
  
• 2007 $1,179M
  
• 2008 $1,032M
  
• 2009 $463M
  

If I try to make an estimate for 2009 overall, it looks like it's going to be about the same as last year:

• Monster vs. Aliens $195M (actual)
  
• Coraline $75M (actual)
  
• Up $300M (actual $263M to date)
  
• Bolt (released in 2008) $4M (actual)
  
• The Battle for Terra $1M (actual)
  
• The Tale of Despereaux $7M (actual)
  
• Ice Age 3 $170M (actual $68M in first weekend)
  
• Ponyo $10M
  
• Sita Sings the Blues $2M
  
• Cloudy with a Chance of Meatballs $100M
  
• 9 $20M
  
• Fantastic Mr. Fox $30M
  
• Astro Boy $60M
  
• Planet 51 $35M
  
• The Princess and the Frog $100M
  
• Total $1109M
  

Traditional animation is excluded from the above but is trivially small. Admittedley, Coraline's category is open to interpretation.

There's really nothing there that would allow one to analyze the effect of 3D screens across the whole market, but for individual films it appears to generate the majority of their revenue. Here are some quotes from www.boxofficemojo.com about some particular films and weekends:

Coraline conjured $18.9 million over the four-day weekend... Coraline's 3D gross was down only seven percent, accounting for an estimated $14 million of the four-day weekend at 1,060 sites. The tally stands at $39.1 million in eleven days, around 72 percent of which coming from 3D presentations. [February 17]

Ice Age: Dawn of the Dinosaurs gathered $67.5 million in five days (55 percent of which from 1,606 3D venues). [July 5]

... with the release of Up. The adventure comedy swooped in with a $68.1 million start... the [3D] format accounted for $35.4 million of Up's gross. [June 1]

Animated Features Released to Theatres in the United States

by Jerry Beck (well-known animation historian)

• 2005 11 films
  
• 2006 21 films
  
• 2007 13 films
  
• 2008 20 films
  
• 2009 13 films known at this point (includes anticipated releases for remainder of year)

from http://www.cartoonresearch.com/movies2005.html.