Liebert has denied any problems with design and they might be right, but we thought it worth discussing.  The Liebert units force air under the raised floor using a blower fan assembly mounted on a drive shaft (could be one fan or two fans depending on your model number) assembly.  The drive shaft rides on two bearings bolted to the frame.  On the left side shaft is a pulley.  This pulley and its counterpart connected to the electric drive motor is what spins the blower assembly.  See figure 1A for an idea of how the unit is designed.

Figure 1A – Inside the Liebert Chilled Water series CRAC unit.

It is to the best of our understanding that Liebert ships these units with a two belt pulley system that is adjustable and variable.  This two belt system allows some slippage on motor start-up.  However, this slippage also contributes to a belt-dust problem.  Liebert knows about the belt-dust problem and sells what they deem a “low dust” belt for the computer room environments.  See figure 2A for a close up of this pulley and belt system.

Figure 2A – Liebert pulley and belt system for chilled water CRAC units.

I say “it is our understanding” for two reasons.  Liebert has so far refused to comment on this directly (as well as TechPlan, who provides parts) even after several requests.  Our HVAC company was able to get some information from their support resources, but nothing so far in writing.  Feel free to comment below if you have more in-depth knowledge about this issue.

Again, it is our understanding that the systems is designed to allow slippage of the belt.  In late 2006 our building maintenance staff (who managed the CRACs at the time) decided to eliminate this belt dust problem by using a three belt pulley on the Liebert-CWs.  This three belt system drastically reduced the slippage of the belts resulting in a cleaner compute environment.  All was well until March of 2010.  We had two separate Liebert unit’s drive shafts break just hours apart.

Our HVAC company was able to determine according to Techplan that the increased start stop torque provided by the three belt system caused them to weaken and snap over time.  The Liebert shafts are hollow core foam filled shafts, and therefore not designed for this type of torque without a variable speed start relay.

Our building maintenance team continues to stand by their statement that Liebert not only authorized but requested we move to a three belt system in order to limit the dust; “Belt Dust” – which is an entirely different issue we will discuss in a future post.

We have seven Liebert CRACs in our data center and after further investigation all seven shaft were showing signs of fracturing.  We quickly went into prevention mode and replaced the shaft in every remaining unit.  The shafts cost slightly less than $300 from TechPlan, plus a hundred bucks or so for installation.  I like to be safe rather than sorry when it comes to data center environmental services.  It would be nice to know whether this is a design flaw or a mistake on our part, but alas we may never know.

We were offered the option of upgrading the Lieberts to variable start relays, but the cost would have been several thousand dollars.  We may break a shaft every 5 years or so, but that’s a small cost compared to the cost of converting to variable start.  You might be asking why we don’t just go back to the two belt system.  Good question and we will discuss in a future post.  The short answer: Belt Dust is really bad for your computers, tapes, fans, disk drives and more!

See the gallery of pictures below for more Liebert snapped shafts.

The Lakeside Technology Center (350 East Cermak) is a 1.1 million square foot multi-tenant data center. Owned and operated by Digital Realty Trust it is one of the world’s largest carrier hotels, housing data centers for financial companies and Chicago’s commodity markets due to the abundance of peering providers available. This data center, which has over 70 tenants, started out life as a home to printing presses for the Sears Catalog and Yellow Book. It was developed by the R.R. Donnelley Company before being converted to data center space in 1999.

350 East Cermak is currently the second-largest power customer for Commonwealth Edison, trailing only Chicago’s O’Hare Airport. The industrial strength infrastructure includes four fiber vaults and three electric power feeds, which provide the building with more than 100 megawatts of power. Grid power is supported by more than 50 generators throughout the building, which are fueled by multiple 30,000 gallon tanks of diesel fuel.

One of the most distinctive features of the facility is its cooling system, which is supported by an 8.5 million gallon tank of a refrigerated brine-like liquid (seen at left in the top photo). The huge tank serves as thermal energy storage for the Metropolitan Pier and Exposition Authority (MPEA), including the nearby McCormick Place Exposition Center and Hyatt Regency Hotel as well as 350 East Cermak. Thermal energy storage can reduce costs by running chillers during off-peak hours when power rates are lower.

This hallway provides a glimpse of the Gothic architecture that provides a sense of atmosphere not seen at most data center buildings.

The building may have its exterior flourishes, but underneath it’s industrial-strength ifnrastructure. This view of one of the four main risers provides a sense of the power and fiber infrastructure.

3PARTV — March 08, 2010 — 3PAR announces its storage tiering software, providing autonomic data movement between different storage tiers.

3PARTV — February 04, 2010 — We spoke with Jordan Gross and Michael Shanks of Ultraspeed about their business and their use of 3PAR storage

3PARTV— January 26, 2010 — Marc Farley of 3PAR interviews Nick de Beer from SYSDBA.

3PARTV — January 25, 2010 — 3PAR just announced new VMware Server management plug-ins for provisioning, snapshots and rapid data VM recovery.

3PARTV — January 19, 2010 — 3PAR has added a new level for Cloud Agile partners and expanded the program internationally.

Here’s a sampling of some of the comments I found:

“You may want to check the TOS/AUP. I would think that hosting on FIOS is not allowed.”

“The FIOS service is designed for broadband access. It is not for people hosting or using the bandwidth in a commercial/wholesale manner. Generally the telcos call this DIA, Dedicated Internet Access, and you will pay at least $100/Mbps.”

“You’ll need to pay extra for a business plan to do this [and also get a static IP]. I personally do use this for personal stuff. It works fine. I wouldn’t use it for anything critical, however. With that said I’ve had 100% uptime on the fios in the past year.”

And the rest of the comments were very similar in nature.  The bottom line consensus of the user community is “hosting on FiOS is a bad idea, rent a server or pull a DS-3″.  Let’s discuss further and find out the truth.

Hosting on a FiOS connection

The first thing I want to do is point out two VERY important things:

1. Hosting on FiOS is an absolutely fine thing to do, and is fully supported by Verizon.
2. There is a major difference between Verizon’s Consumer FiOS and Verizon’s Business FiOS.

Consumer FiOS is generally delivered over PPPoE (Point to Point Protocol over Ethernet).  Consumer FiOS is also in most territories full of restrictions on usage, including bandwidth caps and blocked ports (25, 80, 443 for example).  Consumer FiOS also prohibits hosting servers of any kind in its Terms of Service.  Hosting on Consumer FiOS is a bad idea, period.  Don’t do it.

However, and lucky for us, Verizon also offers a Business FiOS package with static IPs (up to 256), no blocked ports, no PPPoE, and no ToS restrictions for hosting in the contract.  This service is designed as a replacement for legacy dedicated services such as DS-1, DS-3 and Metro Area Ethernet services.  Business FiOS comes in many speeds such as 15/5, 20/20, 25/15, and 50/50 (at the time of this writing) with 100/100 service planned for later this year.  The only caveat to Business FiOS is that it usually costs around $120/month compared to Consumer FiOS that starts at $39/month.  With Business FiOS you can also use your own router (such as a Cisco).

There are things to consider when hosting yourself.  The most common gotchas are space for the servers, cooling, and power redundancy.  Also, remember, if you host your own servers on Business FiOS connection then you’re limiting yourself to a single point of failure.  If your FiOS goes down, you’re down.  When hosting in a colocation data center there are usually at least three data providers in the center.  You’re going to also need some sort of back up power (a large UPS might be enough for you).

Real World Hosting on Verizon FiOS

Now that I’ve taken you through some of the caveats of hosting on FiOS and dispelled some of the myths, let’s talk about a real world example of a company running it’s entire operation from a Business FiOS connection.

That would be me.  That’s right.  This website, and over 40 other websites of mine are running on a 20/20 symmetrical FiOS connection.  I have two servers (one Windows 2003 server running mail, and some .NET applications and another Ubuntu 64 server running my WordPress and PHP applications).  It’s all fronted by a Cisco 871W router connected directly to Verizon.  Surprised?  You shouldn’t be.

I have a very large APC UPS that gives me about six hours of runtime in the case of a power outage.  This UPS also backs up the Verizon ONT.  If the ONT loses power, it drops Internet services and maintains only the telephone service.  It’s vital to provide backup power to the ONT whether hosting at home, or at your business location.

With 20 mb/s symmetrical FiOS and my 40+ websites I receive over 20,000 unique hits per day.  The FiOS circuit is monitored by Cacti (through the Cisco router) and varies from 1 to 3 mb/s most of the day, with spikes ranging from 5 to 10 mb/s when my sites make it into Digg or Slashdot for example.  Speaking of Slashdot… I’ve been Slashdotted three times since running on FiOS and not once did my website(s) become unavailable or slow to respond.  The FiOS circuit hit 100% utilization for over six hours on one such occurrence.  I immediately called friends across town on other services (AT&T and Comcast) and asked them to check my sites performance.  They said they could tell it was a little slower, but more than acceptable performance.

Why did I decide to Host on FiOS?

That’s a great question.  I’ll tell you.  It’s affordable, convenient, and rock solid reliable.

• Since I started hosting on FiOS, back in 2006 it has literally not been down for more than a few minutes here or there.
• I pay a little more than $120/month for service.
• Having my servers here locally makes maintenance and backups simple.

And there you have it.  Hosting a server (or servers) on Verizon FiOS is simple, easy, affordable and makes perfect sense.  It’s not against Verizon’s ToS (on business class service) and oh… I almost forgot… FAST.

3PARTV – January 20, 2010 — Very easy & straightforward. Start fat and move to thin, efficient storage on a 3PAR F or T Class system.