BladeGuy

Guest Blog by Jeff Allen

2011-08-09T10:45:00.001-04:00

I had tweeted about a blog from Jeff Allen, then realized it was on the intranet. For all those that were interested, I reposted it here.

IBM Bladecenter, Dell M1000e, and HP Bladesystem - my quick thoughts

November 14, 2008 ·

My name is Jeff Allen. I have been with HP for over 10 years and have been completely focused on our BladeSystem technology for the past 5 years. I am based in Atlanta, GA and am part of HP’s presales group. Specifically, I’m an Infrastructure Architect, but you can think of that as a customer facing technical consultant.

My thoughts here are my own, not HP’s. I say that to try and give myself some wiggle room if HR tries to fire me for something I say. It may not work, but it should give me a few days more pay while they sort it out.

Anyway, I love what I do and I love working for HP. We make the world’s best blade products built by the world’s best engineers - period. If you don’t believe me, just look at Dell’s M1000e. Ever wonder how they came out with a 10u chassis that accepts 16 blades (with a mix of Half height and Full height blades), and an LCD on the front. Where did they ever dream up such a design? I’ll give you one guess.

Well, we know it wasn’t from IBM. Speaking of IBM, now there’s a Blade division in trouble. I think they have now turned to youtube as their main marketing tool – and it’s not working. Ol’ IBMTOM is busy posting videos and then not allowing anyone to comment on them. Why Tom? If our relationship is going to work at all, you’re going to have to allow me to speak. How about this, I promise that at the end of this blog, you can comment all you want if you let me respond to your videos. Speaking of IBM videos, where’s Doug Balog? I miss the videos of him trying to stuff a pClass switch into an eClass enclosure and wondering why it didn’t fit. His frequent youtube appearances were an almost daily event until HP released cClass. Oh yeah, that was about the time that HP knocked Bladecenter off its high horse and IBM has been losing market share ever since. Don’t worry Doug, as I write this, HP has over 5000 jobs open – one could be yours and we’ll show you how that switch fits.

One thing IBM is very proud of is their backward compatibility. There are caveats to their own story though – not everything works everywhere backwards and forward, but we’ll pretend that it does for this discussion. They have a chassis and blade design built around technology that existed at about the turn of the century. They seem to think that they built a chassis that would never need improvement – it was perfect from day one. Using this theory, all of your datacenters would be full of non-standard size servers all oddly stacked on top of one another. But someone came along and decided that a 19 inch wide rack would suit most vendors’ needs and we all agreed. To that end, as much as IBM has tried to make the industry standardize on their design, their units shipped and revenue generated numbers keep falling – quarter after quarter after quarter. Mark my words on IBM – they are busy designing a chassis that is not Bladecenter as we know it. My guess is that another design is well underway that may or may not allow today’s blades to operate. Here’s their choice – allow old blades and substantially raise the cost of the enclosure design, or make a clean break and enjoy the big bowls of crow that the industry is waiting to serve them. HP made the decision to not allow pClass blades to go forward into cClass. It was a painful decision to make, but when you sit down and think about it, it really does make the most sense. The technology built 7 years ago is bigger and slower and uses more power. To make the stuff run in today’s environment costs more money – and it certainly costs more to build something new to run that old stuff. And then within a year, no one really cares because they are all buying new anyway – so you never recover your development costs. And when a company isn’t recovering their costs, they pass that on to the consumer, and the server that could have cost $2200 now costs $2400. Not a pretty picture. To know that we are on the right track, we didn’t have to look very far. Did Cisco 4000 series switch blades fit in the 5000 series chassis? What about 5000 series blades in their 6000 series chassis? No and No. Cisco already knew what we discovered.

Another thing IBM is very proud of it their “redundant midplane connectors” and the fact that HP doesn’t have them on our half height blades. They’re right, we don’t. So let’s discuss what IBM does have (those who live in glass houses should not hurl stones, excuse me while I pick up this rock). IBM chose an “active” midplane design for Bladecenter. That means that they put eeproms, resistors, capacitors, and other components with a proven record of failure onto the main board that all the blades, interconnects, power, and cooling rely on in order to function. Then they brag about that fact their blades are redundantly connected to this destined-to-fail midplane. By contrast, HP chose a “passive” midplane where all those things that can fail are moved off the midplane and put into a part that can me hot-swapped with no server downtime. We aren’t alone – Cisco, Brocade, Extreme, Nortel, 3Com, and Juniper all chose passive designs in their respective chassis designs. We all did it for the same reasons IBM’s storage division did – “very high levels of reliability of the unit as a whole”. Read it for yourself in IBM’s “Designing and Optimizing an IBM Storage Area Network” Redbook. They go on to describe that an ideal midplane would have no components and should be just a circuit board with sockets and conductor tracks. Sounds oddly like HP cClass. IBM can’t win this midplane battle but they keep trying – but we keep finding stones everywhere we look. Please IBM, for your own health, stop shooting yourself on this one. We like competition – we need to have you around to motivate us to keep building better products.

So what will this blog be about? Anything and everything blade. I work in the technical arena so I plan to focus a lot of my time there. But I’m not a complete nerd and Hp does allow me to go out and see customers – so you’ll get a good balance of blade “business” babble as well like we did today. My immediate thought for my next entry is on BladeSystem Administration where I will spend some time showing all the tips and tricks I use all the time that customers often say “I didn’t know you could do that”. I’ve been keeping track of them and will put them on paper for everyone as soon as I can find another block of time. As I often tell my co-workers “I have an infant at home” which, while true, is pretty much the excuse I give for everything that didn’t get done. So bear with me while I figure out how to best “blog”.

Your comments are always welcome here.

This is not the 10Gb network you’re looking for (Part 2)

2010-06-01T14:53:00.002-04:00

We had a spirited debate started at http://bit.ly/bCIOy6. HP’s blog site is locked for a couple of days during the transition to a new software platform. So I’ve continued the conversation here, please refer to the link above for the rest of the conversation.

The point isn’t whether FCoE is good or bad, or if end to end FCoE is required to get the benefits. As I stated in my original post, the place to start with FCoE is at the edge. Everyone agrees on that. The question is should we be placing million dollar bets that today’s 10Gb core switch offerings are adequate for tomorrow converged networks?

Plenty of vendors and industry organizations seem to think congestion management is important. IEEE would have never worked on 802.1Qau if it wasn’t important. IBM states congestion notification is ”needed to support FC convergence with Ethernet at the data center level“ on page 14 of http://bit.ly/d7T8uA . SNIA thinks it’s important to manage end to end congestion, see page 19 of http://bit.ly/bAUMp5. That paper was co-authored by Errol Roberts of Cisco. Netapp, Nuova and Qlogic stated “a large network requires end-to-end congestion management” on page 2 of http://bit.ly/aHncwI. Nuova is where Cisco acquired much of the FCoE technology used in UCS.

Cisco even states they authored the spec for QCN (802.1Qau) on http://bit.ly/9gI1le. That page goes on to state “This approach maintains the integrity of the network's core and affects only the parts of the network causing the congestion, close to the source”. If we relied only on PFC and ETS as Brad suggested above, every link in the effected data path would be impacted by the congestion.

So I’ll restate my original assertions;

· The place to start with FCoE is at the network edge
· End to end congestion management(QCN) is critical to large scale FCoE deployments
· I am not aware of any Gen1 switch/FEX products that have the hardware capabilities to support QCN
· Customers considering large scale FCoE deployments in the future could be making a million dollar mistake in buying today’s core switching products.

Network Philosophy 101

2010-02-28T22:33:00.005-05:00

In this blog I’d like to review my take on the philosophical difference between Cisco and HP. Any technology involves a number of design choices. These choices create both advantages and tradeoffs. Let’s look at the advantages and tradeoffs in c-Class and UCS network designs.

Cisco wants to move all switching towards the network core while HP wants to move the switching towards the network edge. By moving switching towards the core, Cisco makes it easier to monitor and manage the traffic. Enabling advanced networking features like packet sniffing, QOS and PVLANs are difficult or impossible in a typical virtual switch.

HP lets you use the switching available at the network edge, decreasing management effort, over-subscription and latency. But, virtual switches at the edge lack advanced networking.

As with most challenges it is possible to engineer solutions to accommodate the shortcomings of the technology employed. For UCS you can add more switches, ports and cables to handle the additional bandwidth requirements and more network engineers to design and configure a complicated network. To be sure all the QOS, PVLANs and port mirroring that UCS is optimized for has a significant management cost.

The vast majority of the customers I’ve worked with do not require the advanced networking features at the server edge that Cisco has designed UCS for. Those that do, it’s usually for a limited number of applications. Basic network connectivity meets most application requirements, and it’s simple, cost effective and easy to manage. The total available bandwidth in the datacenter is greatly increased by using the inherent switching capabilities at the edge where appropriate, and only forcing the switching upstream when required.

For users that need more control, Virtual Connect can often be configured with a combination of Vnets and Private networks to meet the need. If more control is required, HP offers ProCurve and even Cisco Catalyst switches for the c-Class enclosure to meet the need.

So where Cisco forces customers to buy more of their stuff to work around their design choices, HP gives customers a number of options (including Cisco switches) to design c-Class solutions that meet differing needs.

But even this isn’t enough. HP has been leading the effort within IEEE to define a new standard referred to as VEPA. http://www.ieee802.org/1/files/public/docs2009/new-evb-congdon-vepa-modular-0709-v01.pdf When these new technologies come on line users will have the best of both worlds; cheap local switching close to the edge when appropriate and enhanced functionality of an enterprise switch when required. All with an industry standard protocol instead of the black box protocols Cisco is using to drive the FEX.

So whether you need simple, cost effective connectivity, maximum network bandwidth, or advanced networking features, c-Class blades provide users with the greatest degree of choice.

Ken

People, Power and Cables; oh my

2010-02-25T09:59:00.000-05:00

As she started down the yellow brick road Dorothy was worried about lions, tigers and bears. In trying to run an efficient and reliable data center there’s a lot more than three things to worry about, but let’s start with the three that are best addressed by blade servers; people, power and cables.

People

People in the data center reduce reliability. People trip over cables, power off the wrong equipment and knock plugs and connectors loose. Since day one blade servers have reduced the number of times people need to enter the data center. Instead of having to bolt new equipment into a rack every time a server is added, you can add one enclosure and support 8, 16, even 20 or more servers. Instead of having to move cables administrators can rezone a SAN or change the VLAN on a switch port or more with I/O virtualization technologies. Using built in management processors the administrator can see the console or load removable media over the network, eliminating the need to be in the data center.

Power

There’s a misconception that blade servers are power hungry. While blades create higher power density, they are usually 20 – 40% more efficient than an equivalent rack mount server. To be clear, the exact same processor, memory and I/O configuration will be significantly more efficient in a blade form factor. A few large fans are more efficient than many small fans, fewer larger power supplies are more efficient than several smaller power supplies. You still need more than one of each to allow for balanced power, and air flow, but a number of servers in a blade enclosure will have less of these than a the same number of stand alone servers.

Speaking of balance, we not only need to have things balanced in the servers and enclosures, things must be balanced across the datacenter. Most servers, including blades use single phase power supplies. Most large datacenters use 3 phase power. The facilities manager has to make sure all three phases are balanced. If a blade enclosure has power supplies in multiples of three, it will be easier to balance the power used by the blades.

Fans can be another area to tune for maximum power efficiency. The power used by a fan goes up as a square of the speed it spins. In addition to the power saved directly by slowing the speed of the fans, it can also make the air conditioning more efficient. A large volume of hot air coming from the back of the rack at high speeds will find its way to places you don’t want it, like the front of the rack. By reducing the speed and volume of the exiting hot air, natural convection will help keep the hot air from being a problem in a well designed datacenter.

Cables

Cables can be the Achilles heel of a datacenter. They’re expensive to buy, manage and decommission. Every connector, bend and enclosure door is a potential failure point. You can often tell a great deal about the state of an IT organization by looking at their cable plant.

It’s not just the cables themselves. The cables need something to plug in to. The NICs, HBAs, switches, PDUs, power panels that go with the cables all and expense and potential failure points to the data center.

Blade servers have been reducing cables since they were first introduced. Let’s take a rather typical example in today’s datacenter, a VMware ESX host. It’s typical to have network connections for Console, vMotion and production, add a SAN connection and a power connection and you’ll have five cables. Double that for redundancy, and add a connection for a management processor, and you can easily have 11 or more cabled per ESX host.

Let’s compare that to a blade enclosure. I can run an enclosure containing sixteen servers on ten cables; two 10Gb Ethernet, four 8Gb Fibre channel, two 3 phase power and two management processor.

So we started with 11 cables for a standalone ESX host. Now multiply 11 cables by 16 servers to match the number in the c-Class, and you have 176 cables. Moving from a group of standalone systems to a blade solution saved over 94% of the cables required. This is not an extreme hypothetical solution. This is a real world deployment scenario.

There are many ways that moving to a blade form factor can make your infrastructure more efficient and reliable, people, power and cables are three of the most obvious. How have blades helped you?