Power IT Down Day

Today (Aug. 27th) is "Power IT down day".... a marketing/awareness-building event by HP, Intel and Citrix to promote installing and activating PC power management software. By doing so, companies can reduce power during off-hours and weekends.

Little do they know, this is both (a) a great idea, and (b) missing-the-boat on a huge additional opportunity.

Here's why: the event is definitely a great move to build awareness that IT doesn't need to be on 100% of the time... this is just a myth. And shutting down a PC/monitor overnight can save close to 200W. But the messaging the initiative is missing is that this doesn't need to be confined to desktops and monitors. Data Center Servers, the most power-hungry (and wasteful) equipment can be turned off, too.

At the time of this writing, the official website at HP showed over 2,700 participants, saving an estimated 35,000 KWh. But here's a sobering statistic: At a recent Silicon Valley Leadership Group Energy Summit, Cassatt piloted Server power management software. The organization using the software operated a number of its own data centers -- and the case study findings showed that if this software were used enterprise-wide, the annual savings could be 9,100KWh for this enterprise alone.

Maybe we'll try to expand the scope of Turn IT Off Day in 2009...

A day with the California ISO

Earlier today I was able to spend time with folks at the California Independent System Operator (ISO). This organization literally operates California's power grid, ensuring that supply meets demand, and that problems get worked-around seamlessly. They also work with utilities, public policy makers and technology providers to promote solutions that permanently (as well as on-demand) reduce electrical load on the power grid.

I got a tour of their demonstration lab, which is focused on providing efficiency technology, specifically "Demand Response" (DR) for commercial and residential installation. DR is a concept that is designed to reduce end-user demand for electricity during shortages (hot summer weekday afternoons). That's when electricity is most expensive, and when brown-outs are most likely. By consistently curbing peak consumption during peak hours, the ISO (and utilities like PG&E that offer DR incentives & programs) helps reduce the need to build new generating plants that can cost millions. Consider the fact that, during the summer, power consumption roughly doubles from night-to-day. That's alot of expensive peak generation that's needed only a fraction of the time. And that's why participating in DR programs can generate cool, hard $ for end-users.

On exhibit in the lab were a number of "smart" residential & commercial technologies including programmable thermostats, appliances and building control systems that curb electrical consumption during DR events. Picture a smart building that dims perimeter lighting and reduces air conditioning when commanded; picture a home thermostat that reduces air conditioning by a few degrees when it receives a signal from its utility; picture a washer/dryer that delay their cycle until electric rates drop. All of these technologies are available today, but many are awaiting legislative approvals for utility programs that incentivize their use.

Now consider this: The "energy density" of an office building is under 10W per square foot. The energy density of a data center is over 100x of that. What would the possibilities be of bringing DR to a data center? Even a 5% reduction of a data center load could outstrip the DR savings of an entire office building. Think that this isn't possible? Between Dev/test/staging servers and failover servers, there are almost certainly 5% of servers that are "non-critical" and could be gracefully powered-down during the brief DR events.

What are the possibilities there?

Bringing Cloud Computing "Inside"

Just a quick note & pointer to a great article just published by Bill Coleman, Cassatt's CEO. (You'll recognize his name as the "B" from BEA Systems).

The article is in On Demand Enterprise, and is titled "Bring Cloud Computing Inside: Internal Clouds Give You Efficiency and Experience While Avoiding the Stormier Problems". An excerpt:

What if you could construct your own internal cloud within your own four walls, to maximize the benefits of the concept without being held up by its limitations? And what if you didn't have to change your current applications or data while still being able to leverage external cloud resources where they best fit your needs? For many of us, this is, in fact, the ideal application of cloud computing. The internal cloud negates the limitations of current cloud providers, and enhances the cost and availability benefits. Here's your chance to be a little selfish, take advantage of cloud-style architecture for yourself. Set up a cloud within your own shop and you are experiencing, as they say, the best of the both worlds -- and maybe getting a jump on the future, to boot.

Yes, the term "internal cloud" might offend some IT folks' sense of accuracy. But once you get past the terminology, the concept of the architecture's simplicity, the incredible economies-of-scale, and the preservation of invested capital are pretty compelling.

Creating a Generic (Internal) Cloud Architecture

I've been taken aback lately by the tacit assumption that cloud-like (IaaS and PaaS) services have to be provided by folks like Amazon, Terremark and others. It's as if these providers do some black magic that enterprises can't touch or replicate.

However, history's taught the IT industry that what starts in the external domain eventually makes its way into the enterprise, and vice-versa. Consider Google beginning with internet search, and later offering an enterprise search appliance. Then, there's the reverse: An application, say a CRM system, leaves the enterprise to be hosted externally as SaaS, such as SalesForce.com. But even in this case, the first example then recurs -- as SalesForce.com begins providing internal Salesforce.com appliances back to its large enterprise customers!

I am simply trying to challenge the belief that cloud-like architectures have to remain external to the enterprise. They don't. I believe it's inevitable that they will soon find their way into the enterprise, and become a revolutionary paradigm of how *internal* IT infrastructure is operated and managed.

With each IT management conversation I've had, the concept that I recently put forward is becoming clearer and more inevitable. That an "internal cloud" (call it a cloud architecture or utility computing) will penetrate enterprise datacenters.

Limitations of "external" cloud computing architectures

Already, a number of authorities have pretty clearly outlined the pros and cons of using external service providers as "cloud" providers. For reference, there is the excellent "10 reasons enterprises aren't ready to trust the cloud" by Stacey Higginbotham of GigaOM, as well as a piece by Mike Walker of MSDN regarding "Challenges of moving to the cloud”. So it stands that innovation will work around these limitations, borrowing from the positive aspects of external service providers, omitting the negatives, and offering the result to IT Ops.

Is an "internal" cloud architecture possible and repeatable?

So here is my main thesis: that there are software IT management products available today (and more to come) that will operate *existing* infrastructure in a manner identical to the operation of IaaS and PaaS. Let me say that again -- you don't have to outsource to an "external" cloud provider as long as you already own legacy infrastructure that can be re-purposed for this new architecture.

This statement -- and associated enabling software technologies -- is beginning to spell the beginning of the final commoditization of compute hardware. (BTW, I find it amazing that some vendors continue to tout that their hardware is optimized for cloud computing. That is a real oxymoron)

As time passes, cloud-computing infrastructures (ok, Utility Computing architectures if you must) coupled with the trend toward architecture standardization, will continue to push the importance of specialized HW out of the picture. Hardware margins will continue to be squeezed. (BTW, you can read about the "cheap revolution" in Forbes, featuring our CEO Bill Coleman).

As the VINF blog also observed, regarding cloud-based architectures:

You can build your own cloud, and be choosy about what you give to others. Building your own cloud makes a lot of sense, it’s not always cheap but its the kind of thing you can scale up (or down..) with a bit of up-front investment, in this article I’ll look at some of the practical; and more infrastructure focused ways in which you can do so.

Your “cloud platform” is essentially an internal shared services system where you can actually and practically implement a “platform” team that operates and capacity plans for the cloud platform; they manage its availability and maintenance day-day and expansion/contraction.

Even back in February, Mike Nygard observed reasons and benefits for this trend:

Why should a company build its own cloud, instead of going to one of the providers?

On the positive side, an IT manager running a cloud can finally do real chargebacks to the business units that drive demand. Some do today, but on a larger-grained level... whole servers. With a private cloud, the IT manager could charge by the compute-hour, or by the megabit of bandwidth. He could charge for storage by the gigabyte, and with tiered rates for different availability/continuity guarantees. Even better, he could allow the business units to do the kind of self-service that I can do today with a credit card and The Planet. (OK, The Planet isn't a cloud provider, but I bet they're thinking about it. Plus, I like them.)

We are seeing the beginning of an inflection point in the way IT is managed, brought on by (1) the interest (though not yet adoption) of cloud architectures, (2) the increasing willingness to accept shared IT assets (thanks to VMware and others), and (3) the budding availability of software that allows “cloud-like” operation of existing infrastructure, but in a whole new way.

How might these "internal clouds" first be used?

Let's be real: there are precious few green-field opportunities where enterprises will simply decide to change their entire IT architecture and operations into this "internal cloud" -- i.e. implement a Utility Computing model out-of-the-gate. But there are some interesting starting points that are beginning to emerge:

• Creating a single-service utility: by this mean that an entire service tier (such as a web farm, application server farm, etc.) moves to being managed in a "cloud" infrastructure, where resources ebb-and-flow as needed by user demand.
• Power-managing servers: using utility computing IT management automation to control power states of machines that are temporarily idle, but NOT actually dynamically provisioning software onto servers. Firms are getting used to the idea of using policy-governed control to save on IT power consumption as they get comfortable with utility-computing principles. They can then selectively activate the dynamic provisioning features as they see fit.
• Using utility computing management/automation to govern virtualized environments: it's clear that once firms virtualize/consolidate, they later realize that there are more objects to manage (virtual sprawl) , rather than fewer; plus, they've created "virtual silos", distinct from the non-virtualized infrastructure they own. Firms will migrate toward an automated management approach to virtualization where -- on the fly -- applications are virtualized, hosts are created, apps are deployed/scaled, failed hosts are automatically re-created, etc. etc. Essentially a services cloud.
  

It is inevitable that the simplicity, economics, and scalability of externally-provided "clouds" will make their way into the enterprise. The question isn't if, but when.

Upcoming Data Center Energy Efficiency Seminar

I just got a preview of what should be a pretty comprehensive video seminar hosted by Cisco, EMC, VMware and APC - due to air Thurs. Aug 21. Registration and agenda are on the Cisco site. Here's a (moderately hyped) preview:


Cisco's "Green" mastermind is Paul Marcoux, VP of Sustainable Engineering, who I had a chance to chat with at a recent seminar. They're looking at all aspects of being green, not just data center power... but sourcing materials and power, as well as equipment & waste disposal, and even employee carbon footprints (think: shorter commutes).

More vendors supporting Server Power Management

Bit-by-bit, it's becoming clearer that a major path to decreasing IT energy consumption has to do with Efficient Operation of equipment, not just with installing efficient equipment in-and-of-itself. Organizations that buy efficient servers, but don’t take intelligent operation into account, are the moral equivalent of buying compact fluorescent bulbs for your house, but never using the switches to turn off the lights.

In response, a number of vendors are now investigating the “efficient operations” issue. The concept has largely begun with desktop PC power management with companies like 1E with their NightWatchman product, and Verdiem with their Surveyor product. The companies are claiming over 500,000 seats under power control. Further, Verdiem announced Edison, a free consumer version.

But of more interest -- because more megawatts are involved -- are the products that power control servers in the data center. According to an EPA study based on IDC data, the average volume server consumes ~200W, mid-range server consumes 520W, and high-end server consuming a whopping 6,400W apiece. Finding ways to reduce consumption by even 5% yields a pretty big return.

Follows is a brief survey of the market for server power management products, where each fit, and my (admittedly biased) score.

• Cassatt: Offering Active Response, Standard Edition. This is a policy-based approach to power-controlling just about any vendor make of server, including external PDUs; policies can be composed based on Time, Events, Capacity and server demand/load. Plus, server power-up/down is sequenced based on interdependencies & priorities. There is also a feature to allow it to interface to utility company "demand response" programs. This technology and a case study was displayed at a recent SVLG conference in Santa Clara, CA. Grade: A Why? Because it’s vendor neutral, works with virtual & physical apps, and provides any number of policies for energy savings
  
  
• HP: Their Insight Power Manager is available for their ProLiant and Integrity servers. While specific to their hardware and management software, the product allows for real-time monitoring, power capping and CPU throttling. Grade: C+ Why? While their goal is noble, like most vendors they’re technology-specific, and provide limited control/management of total power. Try to use it to power manage your IBM bladecenter...
  

• IBM: Much like HP, their PowerExecutive product. It’s available only for specific IBM BladeCenter and System x servers, and hooks into IBM Director. While it’s hardware-specific, it allows for real-time monitoring of power consumption. Then, their is their Active Energy Manager. While also IBM hardware specific, this allows for monitoring, capping, and to a certain degree, CPU power throttling. Grade: C+ Why? There’s really only limited capability here since you can only use it with enabled IBM systems, (sorry, HP) and even then, you have limited control over power consumption.

• PowerAssure: This Silicon Valley startup has an offering that begins as a services engagements. Power Assure’s approach optimizes utilization by load shifting and load shedding, knowledge of power trends/costs and association of business and technical processes back to actual server use. I’ve not seen too much about how the service works or is priced, what technology they use, or what platforms they work with; but they’ve just received a new round of financing. Grade: Incomplete

• Virtual Iron: This past week, VI announced its intent to release it’s “experimental” LivePower feature as part of its VM management package (keeping up with the Jones’… see VMware below). With LivePower, users can set polices that will shut down physical machines when there are no virtual machines running on the hardware. At the same time, if there is excessive CPU capacity, LivePower will be able to consolidate virtual machines onto fewer physical ones. Apparently, though, it has to use the Intel Node Manager, and it’s not clear (yet) how power will be physically controlled… probably with Wake on LAN. Grade: B Why? This is clearly an important feature, but it will be limited to one approach to power management (CPU utilization) and be available only for machines running VI’s suite.

• VMware: For the past 6 months or so, VMware has also touted its “experimental” Distributed Power Management (DPM). Their execution is essentially identical to Virtual Iron’s, above (and, if imitation if the finest form of flattery, VMware should be flattered, given Virtual Iron’s product, above). VMware uses Wake on LAN to do its power control of idle hosts. Grade: B. Why? Same as above… what do you do if you have physical nodes not running VI3?

I'm pretty pleased that vendors are realizing that powering up/down servers is a smart/simple way to saving on wasted power, especially in environment with even a little cyclicality. And, as Virtual Iron pointed out in a recent release, even if you've already virtualized, you might be able to save another 25% off your power bill if you shut-down idle hosts. Food for thought, guys.

The next question: To initiate a consolidation project first, or a power-management project first?

Data Center Efficiency Resources: Seminars, Webinars & Tutorials

My Inbox today had a number of really great resources to help get started with assessing and increasing data center efficiency.

First, two seminars from the Uptime Institute:

How to Apply the Four Metrics to Achieve Data Center Greenness Online Seminar

The seminar presents your executives a model for understanding data center energy consumption and points the way to strategies that can effectively curb it. It will demonstrate how a green tune-up can yield up to 50% savings. Most important, it will present four metrics for measuring data center greenness that can help your organization benchmark and control its energy consumption and greenhouse gas emissions. Dates and Registration

Revolutionizing Data Center Energy Efficiency

The Revolutionizing Data Center Energy Efficiency Seminar will prepare your executive team to improve bottom-line performance, avert costly and unnecessary data center expansions or builds, improve whole-data center utilization, and contain CapEx, OpEx, energy consumption, and green house gas emissions. Dates and Registration

Next, I received a newsletter from the U.S. Department of Energy's Industrial Technologies Program (ITP). It was chock-full of great info and reports, as well as a few resources open to all. Two resources of interest included:

Save Energy Now - Data Center Assessments Offered at No Cost

There's still time to apply for a 2008 Save Energy Now energy assessment of your facility to discover new and cost-effective ways to save energy. To date, U.S. manufacturing facilities have identified energy cost savings of more than $807 million through Save Energy Now.

Webinar to Highlight Save Energy Now Resources for Utilities

If you're with a utility, then on August 20, 2008, a Webinar entitled "Exploring Options for Keeping Industrial Customers Competitive" will take place from 12:00 p.m. to 1:30 p.m. MDT. This is the first in a series of energy management Webinars sponsored by ITP, Western Area Power Administration, and American Public Power Association. Register Here.

DOE's ITP program also listed a number of other upcoming events.

Subscribe to this DOE newsletter -- this is one of those rare occasions where the US government is really taking the lead in creating and distributing useful information and resources.

A peek into the (dire) world of Managed Service Providers

The other day I was able to have a conversation with the COO as well as the SVP of Technology for a mid-market MSP/Hosting provider. Their situation was probably typical and telling of the problems most IT operators are facing.

They have a mid-sized data center; perhaps a third of their business is co-location, with the remaining business being hosted applications. They seem to have healthy growth, but have hit the limit on their UPS -- and have no more power capacity.

Naturally, I asked the question of whether they've virtualized/consolidated. No, they said; can't do it for the Co-Lo applications, and their hosted apps don't lend well to virtualization.

Next, I asked if they knew how much unused capacity they had on existing machines; perhaps idle machines could be power managed, freeing-up power capacity for other machines. They didn't know (but would get back to me)

Finally, I asked if they would consider a utility-computing infrastructure for their hosted-service machines (which makes the best possible use of capital and energy). No, they said, they use local storage, and believed that use of shared (i.e. SAN) storage was "too expensive".

Wow. Here I had a really great set of technologists, a growing business, and architectures that are mainstream -- but it's resulted in a siloed, inflexible and un-sharable infrastructure, where even the global utilization is unknown. They are very likely operating far below optimal energy efficiency (not to mention operational efficiency), and there they are - hitting the limits of their UPS. This is a perfect illustration of "when that next app costs you $million" because you need to build a new data center to hold it.

My take-aways from this experience:

• Use more shared storage - it will ultimately give you a whole lotta flexibility
• Measure and monitor your server use, as well as server utilization - look for patterns where you can share power and/or hardware
  
• Look at your global capacity -- if your total capital efficiency is below 50-75% then you should consider sharing (using virtualization and/or a Utility Computing infrastructure)
• Look at your PUE - if your overhead equipment (i.e. cooling) is inefficient, you can re-claim power for your IT equipment by reducing the consumption of overhead/facilities.