Archive

Posts Tagged ‘Zones/Containers’

TSM and Solaris containers

April 11th, 2006
Comments Off

Since i have not had a chance yet to look at the new 5.3.3 client to see if/how they fixed the zones problem I will post how I did it.

First off, for what ever reason IBM/Tivoli decided that the config files (dsm.sys/dsm.opt) should go in /usr/bin. Why I don’t have a clue but that is not a place where they should go. What is even worse is that when you install the client it puts symlinks to /usr/bin/dsm.[opt|sys] in the /opt/tivoli/tsm/client/ba/bin directory.

/opt/tivoli/tsm/client/ba/bin> ls -la dsm.*
lrwxrwxrwx 1 root bin 33 Dec 21 08:21 dsm.opt -> ../../../../../../usr/bin/dsm.opt
-r–r–r– 1 root bin 782 May 18 2005 dsm.opt.smp
lrwxrwxrwx 1 root bin 33 Dec 21 08:21 dsm.sys -> ../../../../../../usr/bin/dsm.sys
-r–r–r– 1 root bin 971 May 18 2005 dsm.sys.smp

What is even better is how they make the symlinks… Any ways to get TSM to work in zones, what I did was change the order the symlinks are. I put the actual config files in /opt/tivoli/tsm/client/ba/bin and then did a symlink in /usr/bin to the /opt/tivoli/tsm/client/ba/bin directory in the global zone (as the /usr filesystem in the non global zones are read only), so now I have this:

/usr/bin> ls -la dsm*
lrwxrwxrwx 1 root root 37 Jul 22 2005 dsm.opt -> /opt/tivoli/tsm/client/ba/bin/dsm.opt
lrwxrwxrwx 1 root root 37 Jul 22 2005 dsm.sys -> /opt/tivoli/tsm/client/ba/bin/dsm.sys

This way each zone can have their own dsm.[opt|sys].. There is one “gotcha” with this method, make sure you back up your files before you upgrade the client. I am not sure at the moment whether they would be removed if you upgrade the client or not. Technically the files (dsm.[sys|opt]) should go in /etc and then symlinks from /usr/bin and /opt/tivoli/tsm/client/ba/bin to them.

N.B. This is probably unsupported by IBM and you use at your own risk.

Technorati Tags: , , , ,

IBM, Solaris, Tivoli, Zones/Containers , , ,

bigboy domain quirks

March 27th, 2006
Comments Off

Well I had 120+ zones created and then the script stopped.. I ran out of inodes on my 1.1TB file system. So I deleted the zones and recreated the file system with the -T option to see if that will help any..

On a side bar, I decided to see exactly how fast this domain was. (Wish I would have got one of the trial T2000 machines to compare againest, but that is another story).. So my test was to see how long it would take to compile Apache + PHP on the machine. Here are the results:

Compiling PHP 5.0.4

Single thread using make and gcc

real 2m27.49s
user 2m10.31s
sys 0m7.35s

Single thread using gmake and gcc
real 2m26.58s
user 2m10.26s
sys 0m7.02s

48 threads using gmake and gcc
real 0m16.00s
user 2m14.58s
sys 0m12.58s

Apache+PHP
Single thread using make and gcc
real 0m14.63s
user 0m10.38s
sys 0m1.15s

Single thread using gmake and gcc
real 0m14.33s
user 0m10.33s
sys 0m1.08s

48 threads with gmake and gcc
real 0m4.02s
user 0m10.47s
sys 0m1.94s

Needless to say it is pretty darn fast. Need to find something a little larger to play with and maybe use the Sun Studio Compilers to see how much of a difference that makes.

Interesting, Solaris, Zones/Containers , ,

Overhead of Software raid.

March 25th, 2006
Comments Off

Well as I posted the other day about the E25K domain and creating zones. I have started to create some zones on the X2100 tonight. With 2 SATA drives mirrored, and the machine idle it takes 6:32 minutes to create a zone. So almost twice as long as the E25K machine with the 1.1TB file system stripped across 17 drives and 6 SCSI controllers. Compared to the V890 which has a 3 disk raid 5 array, it would appear that using the SVM software raid is a real performance hit. The drives in the V890 are even fibre channel drives, but it still takes 30 minutes to create a zone on that machine. So how do we speed it up? not sure at the moment. On the one v890 I have Solaris 10 on, I have to quit using Raid 5 for the zones file system and went to Raid 10 (1+0) to get it to speed up, but unfortunatly I did not have enough drives in the second V890 to do that. If any one has any good tips on how to speed up SVM Raid 5 array’s let me know.

Solaris, Sun, X2100, Zones/Containers , , ,

Large Sun Domain

March 23rd, 2006
Comments Off

We installed some new UltraSparc IV+ boards in our E25K yesterday, and to “test” the new boards, we decided to make one huge domain. (it is not every day I get to play with a large domain like this, usually they are 8 or 16 way). Well the domain has 24 cores, and 98GB of ram, and 1.1 TeraBytes of disk. What better way to test it then install Solaris 10 and create as many zones as I can. I started the process this morning, and each zone took about 3.5 minutes to create. (I was doing one at a time, assigning 254 zones per ethernet card, and with 12 ethernet cards, I figured I could get about 3048 zones on the machine. After 7 hours I have 120 zones on the machine, but unfortunatly I have to shut the domain down as it appears one of our System Controllers (SC1) fried it self, and we can’t turn it off or on, so Sun will be in tonight and we are going to shut the whole box down and replace the bad SC and then boot it back up. I think once that is done, I will start the script back up again and let it run all night, 3 of them at a time. Needless to say this box “spanked” my v890 with 8 x 1.2GHZ UltraVI cores with 16 gig of ram. I created a zone on it today and it took about 30 minutes to do on that machine.

Output from creating one zone on the E25K:

# zonecfg -z oscar -f oscar.cfg
# time zoneadm -z oscar install
Preparing to install zone .
Creating list of files to copy from the global zone.
Copying <2573> files to the zone.
Initializing zone product registry.
Determining zone package initialization order.
Preparing to initialize <917> packages on the zone.
Initialized <917> packages on zone.
Zone
is initialized.
The file contains a log of the zone installation.

real 3:29.7
user 57.9
sys 1:18.3

Interesting, Solaris, Work, Zones/Containers , , ,

What is the deal with Blade servers?

March 16th, 2006

On wednesday a vendor came in to give their marketing pitch on why we should be running blade servers. My personal opinion is that blade servers are a waste of time and money. Why do I say they are a waste, well here are my points:

  1. Power Requirements: Said vendor requires 4 x 2000watt power supplies to support 14 blades. (8000 Watts) They say that they are getting ready to up them to 3000 watt power supplies, so now we are up to 12,000 watts for 14 servers.
  2. No hot swap drives: If you want hot swap drives you have to buy a hot swap drive blade, so if you wanted every blade to have hot swapable drives, you in turn only get 7 blades in the box instead of 14. Effectively giving you 7 1U servers now.
  3. Shared backplane: Granted they have 2 backplanes in the chassis, but the problem is that they are both tied to the same support, and to replace either you have to shut down all the blades in the box. Not a very good idea if you have to shut down 14 servers to fix one little problem.

One of the biggest problems I have with blade centers is the power and heat they throw off. For example the vendor said that they could put 6 of their blade centers in a 42 U rack. So we take 6 x 12,000 watts, and you get 72,000 watts of power in 1 rack, in comparison the SunFire E25K we have running is using about 26,423 watts total of power right now. Or you could take 42 of the SunFire X2100 servers and they would only use about 12,600 watts.

Now if we look at the BTU’s generated by each:

If we assume the following: 1 watthour = 3.412 Btu
We see the following:
1 Rack with 6 Blade Centers uses 72,000 Watts or 245,664 BTU’s
1 Rack with 42 X2100 Servers uses 12,600 Watts or 42,991 BTU’s
1 E25K 26,423 Watts or 89,838 BTU’s

Now to find out Ton’s of AC required:
1 ton = 12,000 BTU
1 Rack with 6 Blade Centers: 20.47 Ton
1 Rack with 42 X2100: 3.58 Ton
1 E25K: 7.48 Ton

Now the “average” Data Center AC unit has around 30 to 40 Ton capacity depending on whether you used compressorized system (10 to 30 tons) or a chilled water system (10 to 65 Ton).

So what does that mean? You almost need 1 AC unit for each full rack of Blade Centers.

When you factor all this together, are blade centers really saving you anything. Not in my opinion.

The other “selling” point they try to tell you is better is that you don’t have to run as many cables to your rack. This may be true, but now instead of maybe one large enterprise switch you have to manage, you now have 12 switches that need to be connected back to one or two central switches. Same with the SAN.

So lets see how many cables there really are (if the systems were configured to be redundent where possible):

System Power Cables Network San Total cables
1 Blade Chassis 4 at least 2 at least 2 8
6 Blade Chassis’ 24 at least 12 at least 12 48
1 X2100 1 2 2 5
42 X2100 42 84 84 210

So yes, you will reduce the number of cables running to a rack, but that is about it for the cable “benefit”.

Now what is the real use of blade servers? Is it to “virtualize” your environment? Don’t think so. Is it to reduce the footprint of what the servers use? Maybe. But in the end, you still have X numbers of servers to manage. If you were to use the 14 blades per chassis and had 6 chassis per rack, that is 84 individual servers to manage.

So what would I do? Not use blade servers, and utilize some really cool features of Solaris 10, aka Containers/Zones. From most of what I have seen most people use blade servers for very light duty services, i.e. web servers, or part of a grid, etc. Web services can easily be moved in to a zone, as well as grid services on under utilitized servers. Especially if you use the new CoolThreads servers that Sun has. In this day where every one is trying to consolidate services and hardware, why would any one want to buy a blade system to increase the number of servers they have to manage? I am 100% about consolidating anything and everything I can. (I have 1 SunFire V890 with about 10 zones on it, and another with 5 on it. These were all seperate machines that I had to manage and patch, and now 15 has gone down to 2.)

Now the only reason I could see using a Blade server is if you could make the chassis act like a mini E25K, and make each blade be able to be part of a domain, which would allow you to tie multiple blades in to 1 logical server. Then I might be able to see a use for it, but for the most part I would stick to small 1 or 2 U servers for small applications IFF (if and only if) they would not run in a Zone.

For what’s it worth.

Technorati Tags: , , ,

Interesting, Rant, Solaris, Work, Zones/Containers , , , ,