I read about it on CNet. What you see above is Lucasfilm's data center. Its network features 300 10-gig ports and 1,500 1-gig ports. Every digital artist within the company has a desktop with gigabit connectivity.

Lucasfilm uses Verari servers with dual core, dual Opteron processors and 16 GB RAM as well as bunches of legacy machines. IT director Kevin Clark says it can take as little as 6-7 months before new equipment turns legacy. For storage, Kevin has 300 TB on a NetApp NAS. (BTW, check out this Infoworld article about NetApp's Data Ontap GX virtualization software, which allowed Lucasfilm to maintain 200 TB across 20 servers while having storage capacity appear as one single 200 TB disk. Lucasfilm has been testing the software for the past year, and will be moving to a GX cluster soon.)

As with Google, Lucasfilms says power utilization is an important factor in its data center equipment selection. The company's website also points out that its facility is LEED certified by the US Green Building Council. I've been hearing a lot about LEED these days. It's mentioned in this SearchDataCenters article, and it will be on the agenda at Data Center World. If green IT is a priority for corporate data center managers, it will likely become an increasingly important topic in your sales discussions with enterprise customers.

Jon Price from ISPCON is worried about running out of space on his Salesforce.com account. According to Salesforce's edition comparison data sheet (PDF), customers with "professional" and "enterprise" subscriptions only get 1 GB of total storage, or 20 MB per user, whichever is greater. Even "unlimited" accounts include just 120 MB per seat.

But Jon wants his CRM system to be a rich repository of data on customer interactions, with "hundreds of proposals, ppt slides, scanned documents, copies of contracts and whatnot all jammed into that same database and assigned to activities, opportunities and customer records." 120MB certainly won't accommodate this level of usage. I'll bet people will want to save screencasts and product videos as well before long.

The obvious answer, Jon says, is Amazon's S3. Why couldn't Salesforce use Amazon's web services API to build an applet for saving customer data to S3? Better yet, why not offer their own pay-per-GB storage using the servers they maintain at Rackspace?

(I didn't know that Salesforce has servers at Rackspace, but this sounds like yet another reason why Rackspace should develop an on-demand storage platform?)

Adam Gross, Salesforce's SVP of Developer Marketing, told Jon that Salesforce/S3 integration IS possible, but it doesn't seem like they have any plans to build a solution. I wouldn't be surprised to see a third party mashup in the near future. I totally agree with Jon about the tremendous PR buzz it would generate: on-demand apps + on-demand storage, what could be better? (Probably not leasing storage capacity by the server and installing/managing one's own software?)

PS - Jon also uses Webmail.us for email, and he has a Typepad-powered blog. It seems his choices agree with IDC's survey finding that IT users prefer having someone else maintain their applications?

Everyone's talking about Microsoft's soon-to-be-released Windows Home Server ("WHS"). It's a new operating system that's based on Windows Server 2003 (future editions will be based on the upcoming Longhorn Server). Microsoft's current plans are to offer it on an OEM-only basis. HP and others will start offering hardware packages later this year.

Bill Gates unveiled the WHS during his keynote at the Consumer Electronics Show. He envisions a server in every living room, so that Microsoft can "give you connected experiences 24-hours a day". It will monitor security patches/virus definitions/drive health/etc on all Windows PCs on your home network, and back up all of their data. It also offers Zune and XBox connectivity, so you can use it to store all of your music/photos/videos for easy access from any Microsoft-powered device. Wired calls it total convergence.

And of course, content stored on the WHS will be available over the Internet! The product preview mentions a "free customizable Windows Live website", and CNet reports that customers will get "personalized Internet addresses with no monthly fees". In other words, why sign up for a hosting account - to which you have to go through the trouble of uploading your data - when you can selectively web-enable whatever home-hosted content you wish?

The WHS' storage methodology is pretty nifty, BTW. Ars Technica reports that users won't see a C:\ drive - just a single, expandable storage pool. The capacity of any internal, external or USB drives on any computer can be added to the central store. WHS duplicates all data across two or more disks for redundancy. In addition to automated, incremental file backups, WHS takes scheduled snapshots of client systems, with which customers can perform full restores.

Amazon's S3 storage service had a bit of a hiccup last Thursday and Friday. I came across this discussion thread on Amazon's developer forum via CNet. Rich Miller and Dan Farber have also mentioned the incident.

As frustrated as customers must have been by two days of much higher than usual latency and unpredictable errors, their reactions were surprisingly mild. And once Amazon pinpointed the problem (a batch of defective new hardware), the complaints stopped. In addition, three posters thanked Amazon for being transparent about sharing specific details.

In contrast, I've also had the misfortune of putting customers on new hardware that turned out to be faulty, and I most certainly didn't have the benefit of such patience and understanding. Why? The answer is in this video:

Click To Play

If you were making a 5 minute presentation on your company, how would you spend that time? You'd show lots of data center photos. You'd talk about 24/7 tech support. You might fill a slide or two with customer logos - but you wouldn't focus on what cool apps your users have built. Because chances are, you have no clue.

Jeff Barr's speech shows that Amazon is different. Have you ever heard of TV Mojo, he asks? They rock. What about SmugMug? They're so awesome that Jeff himself stores thousands of photos on their site. He succeeds in portraying Amazon as not just an ordinary vendor, but a collaborator in one exciting adventure after another. People sign up for S3 and EC2 not just for hosting, but to be where the action is. As Motorola CEO Ed Zander puts it, what customers want to buy are "cool experiences".

So don't let any technical problems Amazon might have convince you that you offer better web hosting. You're not on a two-dimensional playing field where "no glitches" is all it takes to be "better". Amazon will learn from any technical issues it comes across; it will get better. You may already have solved those challenges; you've been in the hosting market for much longer. But do your services have half the sizzle of theirs? And what are you doing about that??

I've been reading "Why Not?", a book about innovation. One of its key concepts is to ask yourself what Croesus (the ancient rich king) would do. In other words, if you could throw unlimited amounts of money at a problem, what solution would you pick?

Web-hosting-wise (unless you're Google), maybe a more practical approach is to ask what Robert Marsh would do. As founder/Head Surfer of EV1, Robert single-handedly created the discount dedicated servers market. He also popularized complex hosting among EV1 customers by introducing private racks, and rolled out VPSes with a BIG party, complete with fireworks.

He blogged (sort of) before blogging was fashionable (by making a personal soapbox out of his customer forum). He saw potential in APC's Mobile Data Center before Sun made a splash with the Blackbox. He wanted to travel regularly to cities with high customer density - much like what Amazon Web Services evangelists are doing. And he put some work into developing a beyond-the-box hosting environment, which so many hosting companies began offering last year.

I've compiled a quick collection of facts and stats that I think will define web hosting in 2007. (If you're reading in RSS and the slides don't show, click here.) If Robert were still in the hosting business, what would he do in today's market?

Would he build his own S3? At least two Rackspace customers have traded their managed storage for Amazon's pay-per-GB solution - and just this morning my new friend Santosh asked which hosting companies offer S3-like shared storage. What should I tell him??

Might he take Tier 1 Research analyst Dan Golding's advice and snap up a start-up CDN? Buy a shipyard and offer container colo for Sun Blackbox owners?

He'd have another Birthday Bash, that's for sure. Or several parties in different cities - he'd drive his mobile data center right up to the entrance of not just HostingCon, but Salesforce.com's and VMWare's user conferences. Did you know that 7000+ people attend each? And they work for companies that could become your customers!

The most interesting figure (on slide #18), BTW, comes from Vlad Miloushev of 3tera. Vlad thinks up to 90% of web servers are hosted in-house. Robert wanted to go after this market with not just private racks, but private suites.

What else? Who knows. If I were you, I'd take the guy to Fleming's the next time you're in Houston. He might have some insights for you!

TechCrunch reports that Smartsheet.com, a web-based project management service, is about to release new Amazon S3-powered storage capabilities:

"Previously, the company's server was located in Dallas, Texas but with the new capability to share files and send attachments, Smartsheet needed horsepower from Amazon. S3 gives developers access to storage to run their own global network, which Smartsheet needed for its December release. The new version allows users to attach documents from a hard drive or server, and send emails updates about any changes made to a document."

Where in Dallas, you ask? I did a traceroute, which terminated at Rackspace. (Just last week, I read that Webmail.us, another Rackspace customer, is also using S3 for storage.) According to Smartsheet developer Todd Fasulo:

"To clarify, we have integrated the S3 service for doc storage to our existing SAS-70 certified environment in Dallas, TX. The S3 service is used to store documents attached to Smartsheets. (It's really fast...)"

Bill Boebel from Webmail.us says he loves Rackspace. I'm sure Todd from Smartsheet does too. I was once a Rackspace customer, and their service is totally outstanding. But is Fanatical Support enough? Bill insists that his use of S3 demonstrates Amazon's success but absolutely not Rackspace's failure. On the other hand, he acknowleges in the same blog post that "yes, our use of Amazon S3 displaced our use of Rackspace's managed backups".

Yes, Bill signed up with Amazon mainly for its web services stack, an offering that's not within Rackspace's core competency. In which case, should Rackspace partner with Amazon and get a cut on these deals?

PS - Another TechCrunch reader commented that "I wish someone would start web hosting based on Amazon S3. The same pricing with CPanel, FTP, etc." That actually isn't possible. S3 offers only storage with no processing power. But apparently utility pricing does have appeal. So might it make sense for Mosso to offer pay-per-use pricing on "the system"?

Bill Boebel, CTO of Webmail.us, is a happy Rackspace customer. But he recently moved his backup system to Amazon. Jason Hoffman from Joyent asked what this means:

Is it a success for Amazon or a failure of Rackspace? Or both? Will Amazon's offering mature to the point where it would make sense to run everything there? Will Rackspace wise up and begin to offer comparable services? Will we begin to do Grid Peering relationships where say our users or Rackspace's users could have network access from our servers to Amazon's without incurring a bandwidth charge?

Bill's response was, there's more to Amazon Web Services than its hosting infrastructure.

Bill was unhappy with his previous backup solution NOT because it's hosted at Rackspace. Off-the-shelf backup systems just aren't very efficient for maildir, where file names change frequently (to track read/replied/flagged status), causing the same email message to be backed up multiple times. Bill wanted to write a homegrown backup system. He did so at Amazon rather than Rackspace because:

At S3, we were able to just develop the maildir backup logic and some data cleanup logic. We skipped developing the backup storage system altogether. We coded the storage client, not the storage server. Initially we had planned on building both. But when S3 came out our thoughts quickly shifted...

Bill concluded that:

We're always looking for ways to build new stuff faster. In some cases this will mean building on top of services hosted by other companies, such as Amazon. In other cases it will mean building on top of open source software and hosted it on servers at Rackspace.

So if you share Doug Erwin's ambition of winning and keeping 100% of each of your customers' hosting business, it's going to take more than the fastest hardware and the bestest service. You'd also have to match Amazon's Web Services Stack, as illustrated by Read/Write Web:

I spent most of this afternoon walking around the exhibit hall. The three most talked-about products seem to be:

1. 3Tera's AppLogic Grid OS (disclaimer: I recently joined the company's advisory board)

2. Righteous Software's backup software (disclaimer: Founder and CEO David Wartell has been a long-time friend)

3. Cleversafe's dispersed storage solution (disclaimer: I haven't looked into Cleversafe at all, but John Martis from Hostway and Will Charnock from The Planet both said it's "just the kind of thing I'd be into".)

These solutions have two things in common.

First, the share the goal of making the web hosting world a safer place. 3tera eliminates downtime - whether unplanned hardware-failure-associated outages or scheduled maintenance for infrastructure upgrades. Righteous Software makes data loss obsolete with affordable, nearly-continuous incremental backups, even on open files. And Cleversafe offer dispersed storage across not one, not two, but 11 different locations.

Second, all three are software solutions. As British Telecom CTO Matt Bross puts it, we're in a software world; we've got to stop thinking hardware. Nicholas Carr is even more blunt; he says software kills hardware:

Consider the telephone answering machine. It began as a bulky analogue box running spools of tape. It turned into a small digital box, often incorporated into a phone. And finally it disappeared altogether, turning into pure software running out somewhere on a phone company's network. Once you bought an answering machine. Now you buy an answering service. And so it goes.

3tera CEO Vlad Miloushev said during his panel today that more than 80% of all servers in the world are maintained by in-house IT departments. This means the web hosting industry has enormous expansion potential, particularly since AFCOM's research shows that 50% of corporate data centers will become obsolete within the next 5 years. At the same time, increasing adoption of hosted versus desktop software will drive just about every software developer in the world into the hosting market.

We'll get all that business, and we'll service it securely, reliably and seamlessly - with innovative software solutions rather than bigger and costlier gear. That's what I see as the next big thing.

PS - There was actually a 4th thing that I found super impressive. Ivaylo Lenkov from SiteKreator mentioned quite casually that he's hosting 20,000 to 30,000 sites per Dual Xeon server. Wow - now that's a profitable operation!

A little over a year ago, when I worked at EV1Servers, I got a call from an event producer who needed 200 Mbps of bandwidth capacity for a three-day webcast. I told him I could set up a dedicated gigabit switch with unmetered connectivity, but he'd have to place his order a week ahead of time, and keep the system for at least 30 days. He said I was out of my mind, and I thought he was being unrealistic. What I offered was a highly flexible solution that had served hundreds of other customers well.

But I was wrong.

Earlier this week Second Life (SL)(which traceroutes to Internap) came out with a new version of its client software. Half way through release day, the file was moved to Amazon's Simple Storage Service (S3). According to SL's official blog:

"For the tail 8 hours of the download rush, we averaged roughly 70 gigabytes of viewer download per hour. [That's about 160 Mbps] Then it settled down to a relatively steady stream of about 20 to 30 gigabytes per hour. In the last 23 hours we've transferred a total of ~900 gigabytes so far - which I'd estimate to be around 30,000 to 38,000 downloads."

S3 costs $0.15 per GB for distributed, redundant storage and $0.20 per GB for bandwidth. Second Life now owes Amazon $180.15. In contrast, the solution I quoted the would-be customer cost at least 20x more. And that's for one single server on Cogent-only bandwidth. Yikes.

By the way, SL had also considered Akamai, but...

"It just turned out that the S3 solution was ready for deployment immediately, where akamai requires more negotiation. In other words, we already had an amazon S3 account where I was test something out, and then when we noticed the bandwidth was pegged, we made a fast decision to speed up our plans to put our viewer elsewhere, and chose S3."

Here's what I think is the moral of the story: the web hosting industry is evolving very, very quickly. If you can't meet a customer's needs - no matter how crazy it sounds - someone else will.

Which reminds me of a conversation I had with my friend Patrick. He's an attorney now, but I'm expecting him to become a Web 2.0 CEO any day now.

Him: If I order a web hosting plan, they'll back up my data continuously and automatically, right?

Me: Which planet are you from?

But based on what David Wartell at Righteous Software has been telling me, quite a few web hosting companies will be able to deliver what Patrick's looking for very soon. The question is, will you be one of them?