The Distributed Computing Fallacy

I’ve found that sometimes in the VFX industry, technology is marketed to sound great in theory but ends up failing in practice. There are times where I hear people casually mention how simple things would be if they used their idea for how technology should be organized at a facility. Now I usually tend to stay out of those conversations because they end up feeling like a pissing contest but allow me to give some examples.

Usually during the craziness of a show diskspace will start filling up and artists are asked to submit files for backup and removal.  I always find myself sitting next to an artist who gripes:

Oh my God. What the hell is wrong with this place. Diskspace is so cheap. Why can’t they just buy more. Hell I should bring my 1 TB hard drive from home.

Again it’s an absurd thing to say but you always run into these casual technological opinions to major problems in the VFX industry. As one commenter said:

Thanks to silicon valley’s massive push for investment in world wide internet bandwidth during the dot com boom in the late 90s, outsourcing knowledge jobs to local people doesn’t make business sense. (Share holders want growth) Knowledge jobs are moving to the cloud. The only jobs that can’t be outsourced are the ones that can’t be done online. This results in a massive re-balancing of world wide knowledge labour markets.

Ah yes it’s so simple with cloud computing! I’ve had a few encounters with cloud computing. A company I worked for announced they had these new render procs in some other part of the country. While these procs were very fast, they had to have my file and all the geometry and textures associated with it transfered to the location of where the procs exist. Say hello to data latency: The amount of time delay for the file to make it to the location of the proc. I’ve found that this sometimes took longer than the render! It’s like having a powerful gun with a complicated reloading system.

You’d be surprised to know that many companies in the VFX industry are divided over cloud computing:

It seems like a no brainer — but not everyone is biting.

Cloud computing critics argue that many of the security systems aren’t enough to keep a smart, motivated data thief out of a client’s files. They also believe data latency — the time it takes to access off-site files, especially when those files are large — slows down the work flow too much for off-site cloud computing to be practical.

I think it’s great to see networks getting faster but what we miss is that the file sizes are getting larger also.

Fallacies of Distributed Computing

I came across this paper about many of the fallacies involved on distributed computing that should be read by every person in the VFX industry. Here are the 8 common fallacies of distributed computing:

1. The network is reliable.
2. Latency is zero.
3. Bandwidth is infinite.
4. The network is secure.
5. Topology doesn’t change.
6. There is one administrator.
7. Transport cost is zero.
8. The network is homogeneous.

I think some of us see something work in the simple test case and then just assume it will work for a vfx production. Well if you were able to do it for 20 shots why can’t you do it for 1500 right? If you think that’s possible your money will probably end up in the same place as your data: In the clouds.

Soldier On.

8 Responses to The Distributed Computing Fallacy

  1. notgonnatellya says:

    you’re really grasping at straws in your quest to bitch as much as possible about the industry, aren’t you? Why don’t you just quit if you hate everything so much.

    • VFX Soldier says:

      Not exactly. The purpose of this post was to show that people mistakenly think that technology is easy to implement and even some of the biggest vfx facilities question distributed computing.

    • Shootsy says:

      Why do you keep on reading then?!

      You probably bow down to everything that’s thrown at you and probably a fan of Itunes who doesn’t even know what a record store is…geez I’m digressing!

    • Jason Macza says:

      Wouldn’t it be great if through out history those that seek to better things bent to this critique.
      “If you don’t like it quit!?

      “Hey louis pasteur! If you don’t like how we treat disease around here you should quit this penicillin thing you are all about.”

      “Hey Mother Jones! If you don’t like that we send children into coal mines you can leave Alabama.”

      Thanks for your work VFXSoldier

      • VFX Soldier says:

        Hey thanks to everyone that reads the blog…

        If we all agreed on everything what use is it for me to blog about anything?

        I thought this was one of my more tame posts! Apparently not.

  2. Carlos says:

    You bring up some very valid points. But the problem isn’t so much about the technology, but how a business plans to use it.

    From the examples you give, sounds like they failed to plan, especially about the network side. Cloud will fail miserably if the network can’t support it.

  3. […] example, in reaction to my piece about the failures of distributed computing I get this: you’re really grasping at straws in your quest to bitch as much as possible about the industry, […]

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: