Let me be probably not the first to say that OpenSolaris != Solaris.. I have been using Solaris 10 since it was in beta, and OpenSolaris through me for a couple of loops…

First are some of the cool things I liked:

1. The interface, it is updated and seemed a lot faster.

2. The ease of “patching” only took about half an hour to do a pkg image update.

3. Zfs root made it easy to roll back changes..

Now the parts that i had problems with and did not like too well.

1. I had to download a driver for my ethernet card as the one Sun delivers (sk98sol) is still too old and did not support my card which is one built on to a 3+ year old motherboard.

2. To create a zone, you MUST have a network connection (and at least to the internet for the time being). This really made me mad as I sometimes don’t have access to the Internet, and if I need to create a zone, I don’t want to have to wait for it to download 200+ Mb of packages, that are already on the machine in the first place.

3. No more “full root zones”, I created a zone in the hopes of installing the Comms Suite in it, only to find out that it was not a full root zone and stuff that is required by the Comms Installer to be there wasn’t and therefor I could not install it… Such simple things like unzip and perl are missing from the newly created zone.

In the end, I ended up reinstalling the box with Solaris 10 05/08, which was a task in itself. See when you install OpenSolaris it makes the root drive zfs, and did  some weird things to the VTOC. Therefore when I went in to do the install of the “older” Solaris 10 05/08, the installer would show me the disk, let me “carve” it up like I wanted in the gui and via command line, but when the install went to go on, the installer always came back saying that there was not enough disk to install Solaris. What I ended up having to do was go and do a “format -e” and then fdisk and delete the Solaris partition that was made by OpenSolaris, and let the Solaris installer create it’s own fdisk partition again.

So after finally getting Solaris 10 installed and the latest Recommended/Security/Sun Alert patches put on, I called it a night and left the Comms install for next weekend.

Overall I think OpenSolaris is going in the right direction, but there needs to be a lot of things fixed in it.. The biggest is the zones, there should be an option for “cloning” the already installed OS, since it is already on a ZFS pool. The second is that there should be an option when creating the zone as to what kind of zone it should be, whether a full (which would load every package, so you don’t have to try and do it  your self), sparse or maybe a new one called Jail which has everything in it read only.

Well that could not be good. So I ended up going in to the office. Tried unplugging the Sun Ray and plugging it back in. This is when I saw the 9 D error icon. Nice little icon with a picture of a USB connector and a yellow triangle. So I unplugged it and disconnected the keyboard and mouse and then plugged it back in. Still got the same error. The funny thing about the error is, it is listed as this in the docs:

This is an over current condition on the USB bus, i.e., the total number of devices draws too much current . Consider using a powered hub.

So now I ended up swaping it out with one that was in my office and rebuilding the multi-head group, and they were all set. The interesting thing about it is that the status LED stayed green, instead of turning amber. So the next morning I tried it on a different server (the original server it was attached to is running SRSS 2.0 still) that was running SRSS 3.1, this time nothing showed up in the log files, but the Sun Ray still showed the USB 9 icon and the keyboard and mouse did not work. So I ended up calling it in for replacement. It is nice that the Sun Ray’s have a long warrenty period. This one was bought 2 or 3 years ago.

In an unrelated note, I have to go in early to get a power backplane replaced in one of our V890′s because we have went through three power supplies in the PS0 slot in under a month. The bad part about this is the 890 has 11 zones on it and 1TB of disk, so we are going to have some services out while Sun replaces the backplane and power supply. Hopefully this will fix it though.

For some people MS Windows is the only thing they know, but for others if you use Solaris and ZFS, things go much faster and are much better.

Now that I have a ZFS file system, I can clone my zones instead of having to install each one. I really like zfs!

One note, that I came across creating a zfs file system. One of the disks previously had a ufs file system on it and it would not let me create the pool. to get around it I did this:

The error that caused me to do this was:

The numbers he mentioned: They had assumed that Linux on white boxes would be their best option and cheapest option because of a total open source stack of software. These are fully loaded costs per server for 36 months, including electricity, space and support:

- Intel whitebox hardware + Linux software: $10,350
- AMD whitebox hardware + Linux software: $9,180
- Non-Sun AMD (whitebox) hardware + Solaris: $5,700
- Sun’s AMD hardware + Solaris: $4,760

Pay close attention to the end of the video when he talks about Linux Support costs.

First off, for what ever reason IBM/Tivoli decided that the config files (dsm.sys/dsm.opt) should go in /usr/bin. Why I don’t have a clue but that is not a place where they should go. What is even worse is that when you install the client it puts symlinks to /usr/bin/dsm.[opt|sys] in the /opt/tivoli/tsm/client/ba/bin directory.

What is even better is how they make the symlinks… Any ways to get TSM to work in zones, what I did was change the order the symlinks are. I put the actual config files in /opt/tivoli/tsm/client/ba/bin and then did a symlink in /usr/bin to the /opt/tivoli/tsm/client/ba/bin directory in the global zone (as the /usr filesystem in the non global zones are read only), so now I have this:

This way each zone can have their own dsm.[opt|sys].. There is one “gotcha” with this method, make sure you back up your files before you upgrade the client. I am not sure at the moment whether they would be removed if you upgrade the client or not. Technically the files (dsm.[sys|opt]) should go in /etc and then symlinks from /usr/bin and /opt/tivoli/tsm/client/ba/bin to them.

N.B. This is probably unsupported by IBM and you use at your own risk.

Technorati Tags: Tivoli, TSM, Solaris, Containers, Zones

On a side bar, I decided to see exactly how fast this domain was. (Wish I would have got one of the trial T2000 machines to compare againest, but that is another story).. So my test was to see how long it would take to compile Apache + PHP on the machine. Here are the results:

Compiling PHP 5.0.4

Single thread using make and gcc

real 2m27.49s
user 2m10.31s
sys 0m7.35s

Single thread using gmake and gcc
real 2m26.58s
user 2m10.26s
sys 0m7.02s

48 threads using gmake and gcc
real 0m16.00s
user 2m14.58s
sys 0m12.58s

Apache+PHP
Single thread using make and gcc
real 0m14.63s
user 0m10.38s
sys 0m1.15s

Single thread using gmake and gcc
real 0m14.33s
user 0m10.33s
sys 0m1.08s

48 threads with gmake and gcc
real 0m4.02s
user 0m10.47s
sys 0m1.94s

Needless to say it is pretty darn fast. Need to find something a little larger to play with and maybe use the Sun Studio Compilers to see how much of a difference that makes.

Output from creating one zone on the E25K:

1. Power Requirements: Said vendor requires 4 x 2000watt power supplies to support 14 blades. (8000 Watts) They say that they are getting ready to up them to 3000 watt power supplies, so now we are up to 12,000 watts for 14 servers.
2. No hot swap drives: If you want hot swap drives you have to buy a hot swap drive blade, so if you wanted every blade to have hot swapable drives, you in turn only get 7 blades in the box instead of 14. Effectively giving you 7 1U servers now.
3. Shared backplane: Granted they have 2 backplanes in the chassis, but the problem is that they are both tied to the same support, and to replace either you have to shut down all the blades in the box. Not a very good idea if you have to shut down 14 servers to fix one little problem.

One of the biggest problems I have with blade centers is the power and heat they throw off. For example the vendor said that they could put 6 of their blade centers in a 42 U rack. So we take 6 x 12,000 watts, and you get 72,000 watts of power in 1 rack, in comparison the SunFire E25K we have running is using about 26,423 watts total of power right now. Or you could take 42 of the SunFire X2100 servers and they would only use about 12,600 watts.

Now if we look at the BTU’s generated by each:

If we assume the following: 1 watthour = 3.412 Btu
We see the following:
1 Rack with 6 Blade Centers uses 72,000 Watts or 245,664 BTU’s
1 Rack with 42 X2100 Servers uses 12,600 Watts or 42,991 BTU’s
1 E25K 26,423 Watts or 89,838 BTU’s

Now to find out Ton’s of AC required:
1 ton = 12,000 BTU
1 Rack with 6 Blade Centers: 20.47 Ton
1 Rack with 42 X2100: 3.58 Ton
1 E25K: 7.48 Ton

Now the “average” Data Center AC unit has around 30 to 40 Ton capacity depending on whether you used compressorized system (10 to 30 tons) or a chilled water system (10 to 65 Ton).

So what does that mean? You almost need 1 AC unit for each full rack of Blade Centers.

When you factor all this together, are blade centers really saving you anything. Not in my opinion.

The other “selling” point they try to tell you is better is that you don’t have to run as many cables to your rack. This may be true, but now instead of maybe one large enterprise switch you have to manage, you now have 12 switches that need to be connected back to one or two central switches. Same with the SAN.

So lets see how many cables there really are (if the systems were configured to be redundent where possible):

So yes, you will reduce the number of cables running to a rack, but that is about it for the cable “benefit”.

Now what is the real use of blade servers? Is it to “virtualize” your environment? Don’t think so. Is it to reduce the footprint of what the servers use? Maybe. But in the end, you still have X numbers of servers to manage. If you were to use the 14 blades per chassis and had 6 chassis per rack, that is 84 individual servers to manage.

So what would I do? Not use blade servers, and utilize some really cool features of Solaris 10, aka Containers/Zones. From most of what I have seen most people use blade servers for very light duty services, i.e. web servers, or part of a grid, etc. Web services can easily be moved in to a zone, as well as grid services on under utilitized servers. Especially if you use the new CoolThreads servers that Sun has. In this day where every one is trying to consolidate services and hardware, why would any one want to buy a blade system to increase the number of servers they have to manage? I am 100% about consolidating anything and everything I can. (I have 1 SunFire V890 with about 10 zones on it, and another with 5 on it. These were all seperate machines that I had to manage and patch, and now 15 has gone down to 2.)

Now the only reason I could see using a Blade server is if you could make the chassis act like a mini E25K, and make each blade be able to be part of a domain, which would allow you to tie multiple blades in to 1 logical server. Then I might be able to see a use for it, but for the most part I would stick to small 1 or 2 U servers for small applications IFF (if and only if) they would not run in a Zone.

For what’s it worth.

Technorati Tags: Blade Center, Solaris, Zones, Containers