Archive for June, 2009

Can you guess what it is yet?

I’m nearly half way through my UCS training, in my first week at Cisco.

Boy, they like their acronyms here, don’t they? So much so, in fact, they often name the same component multiple times (UCSM=SAM=CAM, or how about FEX=IOM?) Anyway, I digress (perhaps I’ll publish and maintain a dictionary/lexicon for UCS later – interested?). What this post is about is a realization I had today at the end of day 2.

One thing you have to realize about the UCS is that it has hardware virtualization in it. On top of this hardware virtualization you have to deploy vSphere which is…another layer of virtualization. You could deploy simple , non-virtual blades into UCS that just expose their basic NICs and HBAs (well, at least those NICs and HBAs exposed by the CNA/Mezzanine/Adaptor) – but where’s the fun, or ROI, in that?

You will want to extract every ounce of power and dollar of value from UCS and that means putting vSphere on it for maximum compute density, maximum reliability and maximum agility (vMotion between data centers anyone?).

So, (vSphere) virtualization on top of (UCS) virtualization: get some new whiteboard pens and get Visio loaded because the best way you are going to understand, architect, deploy and operate this fantastic solution on time and within budget (ie. not f***ing it up) is by visualizing it – or you could ask Cisco to send someone like me from Advanced Services and I would be delighted to add a supercharger to your engine!

At one point today in our class we discovered that there is such a thing as a vNIC in UCS, and this UCS Blade CNA vNIC has nothing to do with a vSphere virtual machine’s vNIC. Unfortunate namespace clash. Perhaps vSphere’s vNIC needs to be renamed vmNIC, or perhaps the UCS vNIC needs to be renamed uNIC: I’m not in marketing (anymore), so what do I know, other than watch out for confusion.

But the name isn’t the only confusing thing: how does a VM vNIC work with a UCS vNIC? And how does that route through the UCS through the fabric extenders (FEX) in the mid-plane to the Fabric Interconnects (running NXOS and UCSM) and north out of the UCS to the network-at-large?

Think about the components you need to get your head around:

1. VM and it’s vNIC
2. vSwitch – normal, vDS or Nexus 1000v
3. Port Groups and all their settings, like Teaming
4. Up to 128 vNICs or the 2 pNICs on the CNA (depends on which CNA you use, and in which mode)
5. Server Profiles
6. Fabric Extender (FEX), of which there are two on the mid-plane, should be “invisible” as their job is to connect the CNAs to the Fabric Interconnect (NXOS and UCSM).
7. 1-20 ports on the Fabric Interconnect, plus expansion ports.

Looking at this in a list is no problem. Looking at it drawn visually is even better. Sat at the GUI or Console looking at ports and MAC addresses trying to work out the architecture is Engineer Hell. Don’t do it: step away from the laptop, close your eyes, take a deep breath, then turn to face the whiteboard…

I’m a simple guy, so to help me make sense of the UCS basics I drew up this little diagram that helps me understand traffic from a VM (ping command) to the network-at-large. This is a simplification, of course, but it’s a start on top of which I would draw more complex configurations like server profiles using vNICs..

One thing that immediately strikes me is how more complex the logical diagram is compared to the physical. That’s because the logical shows (some of) the virtualization which is taking the burden of the complexity away from the physical – physical is now much, much more simple. In the (bad) old days, the alternative would have been true with the logical would have been simpler than the physical (one-to-one mapping between OS and hardware), which would have push the complexity to physical in a Spaghetti Junction of blades, cards, ports and cables.

So it seems we have swapped on piece of complexity (physical) for another (virtual/logical). THAT’S FANTASTIC NEWS!

If the complexity is logical, and remember you can make UCS really simple by not exploiting the virtualization (why?!), then it is so much easier to manage: click, drag, CLI – all logical human-computer interactions from the comfort of your office/home instead of time in the cold, remote, lonely, noisy data center. Even better – the less people moving cables in the data center = less mistakes = less outages = higher availability AS LONG AS you use good process and Role Based Access Controls (RBAC) on your now-mostly-logical operations. If you leave everything open as default, then you might as well unlock the doors of your data center and put “Come in and toy with me” on your production systems.

But most important of all (and if you know anything about me, you can probably guess what’s coming next), because it is logical it lends itself to efficient, effective and well governed operations – if used right! If the managed configuration (ie. the bits the engineers manipulate) is logical, it can be manipulated in a repeatable, predictable manner by removing repetitive human manual labour with aggregated, automated procedures. An example: the reconfigure/deployment of blades in UCS using server profiles… it’s all there in XML: replay that codified XML config to the UCSM in the Fabric Interconnect in production, and you have a much higher chance of success (because you are repeating a successful change on a known entity – unless your production system is a victim of poor change/governance – don’t go there!).

So, the moral of the story is this: pick up your pen and stand at the whiteboard with your VMware, EMC and Cisco colleagues and really understand how each component interacts, and how that affects the total system (e.g. how would a vSwitch team of IP-Hash work with vNICs and Pinning in the UCS?). Work it out on the board and on paper, do paper test runs, then when you finally get to the console, please start simple with the most simple configuration (no virtualization) and get used to it. Then build up the (logical) complexity – remember, “wire once and walk away” is the tagline for UCS and it’s true. Plug it in, and walk out of the data center: everything else can be done via UCSM. Let me repeat one piece of advice again: (a) understand each component’s interaction, but (b) understand the behavior of the total system. <– an example is, if you use UCS pinning, what is the impact on virtual switches?

And if you get it wrong, which we all do in the pursuit of knowledge and the best system, you can start again (because it’s logical). I’m going to work out some more pictures and post them this week. I’m loving UCS already, and it’s only been two days. GAME ON!