Category: Solaris

Posted on

Splunk 7.0 the good and bad

So I will preface this post by saying I love Splunk, it is the best log aggregation application out there.

So on with the post, and it must be a good one right? Anyways, Splunk released version 7.0 of their Splunk Enterprise product last week during their .Conf 2017 conference, which I was an FTR at. There were a few new features in it that were amazing, such as the new metrics index type which was blazingly fast. So like all “fanboys” of anything I decided to update my home server on Thursday night after the conference was over. This is where the fun began.

First when I started using Splunk years ago, it supported a myriad of operating systems for the servers. If you wanted Solaris, FreeBSD, AIX, HP-UX, MacOSX, Windows or Linux you were golden. However over the years that list has been pared back to now just Linux and Windows. (MacOSX is supported, but only for the free and trial editions. Basically used for development and home use, not for enterprise use.)

So now that Solaris is no longer supported, I needed to switch my home system from OpenIndiana (aka OpenSolaris) over to Linux. With that I spun up a new CentOS 7 VM on my home server, and copied over all my Splunk data from the Solaris one to the Linux one. I then removed the bin and lib directories (I use the tar installs and that is they only place machine specific binaries exist.) With that done, I untarred the Linux Splunk 7.0 over top my current directory and started it up. So far everything was good, until I tried to do a search. If it was a search for like the last 15 minutes it worked, but anything over that was dead because one of the hot buckets was corrupted. I am not sure if it happened during the transit or what. So off to the fsck commmand to try to fix them. An hour or so later it couldn’t fix some of them, so it was getting late and I just went to bed.

The next day when I returned home I tried to log in to my Splunk instance to see how it was doing, to my surprise I couldn’t even log in to it. It appeared that the linux host had crashed. I was dumbfounded as I hadn’t seen an actual kernel panic like that in a while. So I restarted the machine and started splunk back up and everything was working again.

A few days past and I went to check on it again, and once again it was dead. So now I am really curious. I ended up installing the crash utilities on the host and started going through the vmcore files. Yup each time it crashed it was splunkd that caused it. Unfortunately I don’t know much more than that as to what is actually causing it to happen. It appears to happen at random times.

The output of crash shows this:

KERNEL: /usr/lib/debug/lib/modules/3.10.0-693.2.2.el7.x86_64/vmlinux
DUMPFILE: /var/crash/127.0.0.1-2017-10-01-12:26:50/vmcore [PARTIAL DUMP]
CPUS: 8
DATE: Sun Oct 1 12:26:43 2017
UPTIME: 1 days, 17:12:55
LOAD AVERAGE: 0.00, 0.01, 0.05
TASKS: 330
NODENAME: splunk
RELEASE: 3.10.0-693.2.2.el7.x86_64
VERSION: #1 SMP Tue Sep 12 22:26:13 UTC 2017
MACHINE: x86_64 (3399 Mhz)
MEMORY: 3 GB
PANIC: "double fault: 0000 [#1] SMP "
PID: 1420
COMMAND: "splunkd"
TASK: ffff8800bae91fa0 [THREAD_INFO: ffff880000120000]
CPU: 3
STATE: TASK_RUNNING (PANIC)

First few lines of the “bt” output from crash:
PID: 1420 TASK: ffff8800bae91fa0 CPU: 3 COMMAND: "splunkd"
#0 [ffff8800bfac4d88] machine_kexec at ffffffff8105c4cb
#1 [ffff8800bfac4de8] __crash_kexec at ffffffff81104a32
#2 [ffff8800bfac4eb8] crash_kexec at ffffffff81104b20
#3 [ffff8800bfac4ed0] oops_end at ffffffff816ad2b8
#4 [ffff8800bfac4ef8] die at ffffffff8102e97b
#5 [ffff8800bfac4f28] do_double_fault at ffffffff8102b6e2
#6 [ffff8800bfac4f50] double_fault at ffffffff816b6908
[exception RIP: page_fault+13]
RIP: ffffffff816ac52d RSP: ffff880000122fc8 RFLAGS: 00010092
RAX: 0000000000000ff8 RBX: 0000000000000000 RCX: ffffffff816ac2ac
RDX: 00001fffffffffff RSI: ffffffff81a73118 RDI: 0000000000000000
RBP: ffff880000123098 R8: ffffffff81911167 R9: ffffea00002e7b80
R10: ffffea00002e7b80 R11: 0000000000000000 R12: ffffffff81a73118
R13: ffffffff81a73118 R14: ffff880000120000 R15: ffff88008665a580
ORIG_RAX: ffffffffffffffff CS: 0010 SS: 0000
--- ---
#7 [ffff880000122fc8] page_fault at ffffffff816ac52d
#8 [ffff880000123048] spurious_fault at ffffffff816afd8e
#9 [ffff8800001230a0] __do_page_fault at ffffffff816b01ae
#10 [ffff880000123100] do_page_fault at ffffffff816b0325
#11 [ffff880000123130] page_fault at ffffffff816ac548
[exception RIP: spurious_fault+48]
RIP: ffffffff816afd8e RSP: ffff8800001231e8 RFLAGS: 00010002
RAX: 0000000000000ff8 RBX: 0000000000000000 RCX: ffffffff816ac2ac
RDX: 00001fffffffffff RSI: ffffffff81a73118 RDI: 0000000000000000
RBP: ffff880000123208 R8: ffffffff81911167 R9: ffffea00002e7b80
R10: ffffea00002e7b80 R11: 0000000000000000 R12: ffffffff81a73118
R13: ffffffff81a73118 R14: ffff880000120000 R15: ffff88008665a580
ORIG_RAX: ffffffffffffffff CS: 0010 SS: 0000
#12 [ffff880000123210] __do_page_fault at ffffffff816b01ae
#13 [ffff880000123270] do_page_fault at ffffffff816b0325
#14 [ffff8800001232a0] page_fault at ffffffff816ac548
[exception RIP: spurious_fault+48]
RIP: ffffffff816afd8e RSP: ffff880000123358 RFLAGS: 00010002
RAX: 0000000000000ff8 RBX: 0000000000000000 RCX: ffffffff816ac2ac
RDX: 00001fffffffffff RSI: ffffffff81a73118 RDI: 0000000000000000
RBP: ffff880000123378 R8: ffffffff81911167 R9: ffffea00002e7b80
R10: ffffea00002e7b80 R11: 0000000000000000 R12: ffffffff81a73118
R13: ffffffff81a73118 R14: ffff880000120000 R15: ffff88008665a580
ORIG_RAX: ffffffffffffffff CS: 0010 SS: 0000

Output from the “vm” command:
PID: 1420 TASK: ffff8800bae91fa0 CPU: 3 COMMAND: "splunkd"
MM PGD RSS TOTAL_VM
ffff88008665a580 ffff8800b87f4000 324076k 864244k
VMA START END FLAGS FILE
ffff8800a5efe5e8 563d22303000 563d248b7000 8000875 /splunk/splunk/bin/splunkd
ffff8800a5efe438 563d248b7000 563d24964000 8100873 /splunk/splunk/bin/splunkd
ffff8800a5efe288 563d24964000 563d249db000 8100073
ffff880097e90438 7f4f8d800000 7f4f8ea00000 8200073
ffff8800a1f6cd80 7f4f90400000 7f4f91c00000 8200073
ffff8800a1f6ca20 7f4f923f7000 7f4f923f8000 8100070
ffff8800a1f6c948 7f4f923f8000 7f4f925f8000 8100073
ffff8800a1f6c6c0 7f4f925f8000 7f4f925f9000 8100070
ffff8800a1f6c798 7f4f925f9000 7f4f927f9000 8100073
ffff8800a1f6cca8 7f4f927f9000 7f4f927fa000 8100070
ffff8800a1f6c870 7f4f927fa000 7f4f929fa000 8100073
ffff88009ce66948 7f4f935ff000 7f4f93600000 8100070
ffff88009ce66a20 7f4f93600000 7f4f93a00000 8100073
ffff88009466aca8 7f4f93a00000 7f4f94800000 8200073
ffff88009ca98e58 7f4f949cd000 7f4f949e3000 8000075 /usr/lib64/libresolv-2.17.so
ffff88009ca98d80 7f4f949e3000 7f4f94be3000 8000070 /usr/lib64/libresolv-2.17.so
ffff88009ca99008 7f4f94be3000 7f4f94be4000 8100071 /usr/lib64/libresolv-2.17.so
ffff88009ca990e0 7f4f94be4000 7f4f94be5000 8100073 /usr/lib64/libresolv-2.17.so
ffff88009ca98f30 7f4f94be5000 7f4f94be7000 8100073
ffff88009ca98bd0 7f4f94be7000 7f4f94bec000 8000075 /usr/lib64/libnss_dns-2.17.so
ffff88009ca98af8 7f4f94bec000 7f4f94deb000 8000070 /usr/lib64/libnss_dns-2.17.so
ffff88009ca98ca8 7f4f94deb000 7f4f94dec000 8100071 /usr/lib64/libnss_dns-2.17.so
ffff88009ca991b8 7f4f94dec000 7f4f94ded000 8100073 /usr/lib64/libnss_dns-2.17.so
ffff88009ca98798 7f4f94ded000 7f4f94df9000 8000075 /usr/lib64/libnss_files-2.17.so
ffff88009ca986c0 7f4f94df9000 7f4f94ff8000 8000070 /usr/lib64/libnss_files-2.17.so
ffff88009ca98948 7f4f94ff8000 7f4f94ff9000 8100071 /usr/lib64/libnss_files-2.17.so
ffff88009ca98a20 7f4f94ff9000 7f4f94ffa000 8100073 /usr/lib64/libnss_files-2.17.so
ffff88009ca98870 7f4f94ffa000 7f4f95000000 8100073
ffff8800a1f6ce58 7f4f95000000 7f4f95600000 8200073
ffff880036735cb0 7f4f957ef000 7f4f957f0000 8100070
ffff880036735878 7f4f957f0000 7f4f959f0000 8100073
ffff880036735a28 7f4f959f0000 7f4f959f1000 8100070
ffff880036734af8 7f4f959f1000 7f4f95bf1000 8100073
ffff880036735d88 7f4f95bf1000 7f4f95bf2000 8100070

So now that means I will definitely hold off upgrading my production servers as if this is happening on my personal one, then I can only imagine what would happen to larger instances. It could also be a result of me being a fanboy and installing the .0 release of software, which any good admin will tell you “just say no to .0”.

Posted on

OpenSSL and Solaris 10

So if you are still running Solaris 10 and haven’t looked at the patches recently, Oracle bundled in OpenSSL 1.0.1 as a patch. While this was awesome to see an updated version, now that everyone should only be running TLSv1.2 on their websites, there are some issues. The main issue is that the GCC version that is supplied with Solaris 10 for what ever reason has /usr/sfw/include and /usr/sfw/lib hardcoded in to the gcc binary as the first Include and Library path. While I can understand why it was probably done to begin with, it makes compiling any software that needs an updated version of OpenSSL completely impossible.

For example say you wanted to compile a new *AMP stack, and wanted the latest version of Apache with SSL enabled. Well, it won’t work. It will by default use the old 0.9.7 OpenSSL libraries and include files. I spent days on this, even compiled my own version of OpenSSL and tried to link against it, still wouldn’t work and kept linking against the 0.9.7 ones..

So how to fix this? Well you could build your own version of GCC, which in and of itself is not an easy task. You also can’t remove the OpenSSL 0.9.7 libraries as there is a lot that depends on it. So if you read the patch notes for the new OpenSSL they give some “recipes” on ways to maybe possibly fix it during compile time. Which for me did not work. So what I ended up doing was this:

 

  1. In /usr/sfw/include I moved the openssl directory to .openssl. The new 1.0.1 includes are in /usr/include/openssl.
  2. In /usr/sfw/lib I removed the symlink from libssl.so and libcrypto.so. Also did this in /usr/sfw/lib/amd64. This allows those apps that are using the actual libssl.so.0.9.7 to still run, but compile time stuff can’t find them.

Once I did that, I went back to the apache directory and did the compile and yippie, it was against the /usr/lib/libcrypto.so.1.0.1 and /usr/lib/libssl.so.1.0.1, which also meant that I could limit Apache to only use TLSv1.2.

 

So If this helps just one person, great.. This was something that took me a few �weeks to figure out. I also should have noted that if I had fully read the readme in the patch that introduced OpenSSL it would have probably went a little faster… If you are worried about breaking stuff, once you get your compile done, you can put the symlinks back and move the openssl directory back to it’s original place.

N.B. I am not sure if the gcc in Solaris 11 has the same quirk.

Posted on

Subversion on Solaris

So I have been trying to find the “definitive” guide on compiling and installing Subverison on Solaris. There are random sites over the interwebs that have a spattering of different tips, so I thought I would write one how how I did it and what all was done. When you finish this, you will have a basic Subversion system up and running to which you can then further lock down… Requirements: I downloaded the following:

  1. Subversion 1.8.10 (http://mirror.metrocast.net/apache/subversion/subversion-1.8.10.tar.gz)
  2. APR 1.5.1 (http://mirror.metrocast.net/apache/apr/apr-1.5.1.tar.gz)
  3. APR Util 1.5.3 (http://mirror.metrocast.net/apache/apr/apr-util-1.5.3.tar.gz)
  4. scons 2.3.0 (http://prdownloads.sourceforge.net/scons/scons-local-2.3.0.tar.gz)
  5. Serf 1.3.7 (http://serf.googlecode.com/svn/src_releases/serf-1.3.7.tar.bz2)
  6. Apache HTTPD 2.2.27 (http://mirror.metrocast.net/apache/httpd/httpd-2.2.27.tar.bz2)
  7. SQLite 3.8.6 (http://www.sqlite.org/2014/sqlite-autoconf-3080600.tar.gz)
  8. ViewVC 1.1.22 (http://viewvc.tigris.org/files/documents/3330/49347/viewvc-1.1.22.tar.gz)
  9. diffutils 3.2 (http://ftp.gnu.org/gnu/diffutils/diffutils-3.2.tar.gz) [ Needed for ViewVC to work ]

Next up is compiling the software. This is the order I did things:

  1. Apache HTTP Server
  2. APR
  3. APR-Util
  4. SQLite
  5. scons
  6. serf
  7. subversion
  8. viewvc
  9. diffutils

I put all the tar balls in a directory called svn in my home directory. So all the instructions below are relative to it.

 

Apache HTTP Server 

cd httpd-2.2.27
./configure --prefix=/opt/svnweb --with-ssl=/usr/sfw --with-ldap --enable-mods-shared="ssl deflate rewrite ldap authnz-ldap dav dav-fs dav-lock"
make
make install

 

APR

cd apr-1.5.1
./configure --prefix=/opt/sungeek
make
make install

 

APR-Util

cd apr-util-1.5.3
./configure --prefix=/opt/sungeek --with-apr=/opt/sungeek
make
make install

 

SQLite

cd sqlite-autoconf-3080500
./configure --prefix=/opt/sungeek
make
make install

 

scons


mkdir /home/unixwiz/scons
cd scons
tar -xvf ../scons-local-2.3.0.tar
ln -s /home/unixwiz/scons/scons.py /home/unixwiz/bin/scons

(made sure the link points to a directory in your path)

 

serf

At line 251 of SConstruct add the following (this is needed to get it to work on Solaris):

env['PLATFORM'] = 'posix'

(it should be directly below the line that says env.Append(LIBS=’m’) in the sunos if statement)

cd serf-1.3.7
vi SConstruct   (edit as above noted)
scons APR=/opt/sungeek APU=/opt/sungeek OPENSSL=/usr PREFIX=/opt/sungeek CC=/usr/sfw/bin/gcc CFLAGS=-D__EXTENSIONS__
scons install

The CC and CFLAGS needs to be set otherwise it will try to use CC and will give you some errors about APR_PATH_MAX.

 

Subversion

cd subversion-1.8.10
export CFLAGS=-D__EXTENSIONS__
./configure --prefix=/opt/sungeek --with-apr=/opt/sungeek --with-apr-util=/opt/sungeek --with-serf=/opt/sungeek --with-apxs=/opt/svnweb/bin --with-openssl --with-sqlite=/opt/sungeek
make
make install
cd /opt/sungeek/libexec
cp mod* /opt/svnweb/modules

Next edit the httpd.conf and add the “LoadModule dav_svn_module modules/mod_dav_svn.so” line after the rewrite_module line.

At the bottom of the httpd.conf add the following: (assuming that /svn is the location of your svn repository.)

<Location /svn/repos>
DAV svn
SVNPath /svn>
</Location>

Then change the User/Group from daemon to webservd. Also make sure to change the file systems permissions on /svn to be owned by webservd:webservd.

 

swig-py

cd subversion-1.8.10
make swig-py
make install-swig-py
echo /opt/sungeek/lib/svn-python > /usr/lib/python2.6/site-packages/subversion.pth

 

ViewVC

 
cd viewvc-1.1.22
./viewvc-install

Installation path: /opt/sungeek/viewvc-1.1.22
DESTDIR path: empth

Edit the /opt/sungeek/viewvc-1.1.22/viewvc.conf and change the following:
svn_roots = svnrepos: /svn
default_root = svn_roots
mime_types_files = /opt/svnweb/conf/mime.types
diff = /opt/sungeek/bin/diff

 

Next copy the files form /opt/sungeek/viewvc-1.1.22/bin/cgi/*.cgi to /opt/svnweb/cgi-bin

Add the following to the bottom of the httpd.conf

<Directory /opt/sungeek/viewvc-1.1.22>
Order Allow, Deny
Allow from All
</Directory>

diffutils

cd diffutils-3.2
./configure --prefix=/opt/sungeek
make
make install

 

Posted on

New server

So the server that I bought back in April of 2006 to host this site died Wednesday September 18th, 2013.. I am not sure exactly what happened, �but found it unresponsive around 22:00. I went over to where it was hosted and it was still running, but the ethernet card lights were both on solid. After trying to get it to boot and show something on both the video card or the serial port for about an hour, I finally turned it off and got a screw driver out and removed it from the rack.

I had been expecting this day for a while, since the server was 7+ years old. So I brought it home and left it on the floor. The next night I tried to boot it and see if I could get in to it. No go, something was hosed in it. As soon as I plugged in the power the fans all went to 100% and no output on the video again. Great… So I pulled one of the drives out, and attached it to a SATA/USB adapter and mounted it to a Solaris VM on my Mac. Awesome, all the data was still there. After spending close to 8 hours copying the data off, there was a hunt for a new place to host my site.

The three “ideas” I had were the following:

  1. Joyent – They run servers running SmartOS (nee Solaris). So this would be my primary choice, cause hey, I love Solaris, and really hate Linux.
  2. Amazon Web Services – They only support Linux and Windows. So I would have to switch to Linux or Windows (not really wanting to do that)
  3. Host it at home and upgrade my cable modem to a business class one.

 

So I set out to look at the cost. Both the Joyent and AWS were pretty close for the “same” amount of “hardware”. Comcast Business class was going to be WAY more than hosting it some place else.. Now it was between Joyent and AWS.

Free Trials Away….

Amazon Web Services will let you use a one of their “micro” instances for free for a year. So I decided to set one up and see how it would go. I chose to do a SUSE Linux instances, since they didn’t support Solaris. About 15 minutes after clicking the “go” button, I had a SUSE “VM” on the Internet and root access to it.

While the Amazon VM was being provisioned I went to Joyent.com and decided to sign up for one of their free 2 month trials. Unfortunately it wasn’t as smooth as the Amazon sign up. While doing the registration process, it requires a phone to call to give you a PIN number to type in to finish the registration (I assume to stop hackers from spawning machines automagically). Well I put in the phone number and it called, but it only rang barely once and then hung up. It then changed the status page to an “invalid account” and locked it so I couldn’t do anything.

I tried calling them, and they said I had to submit a support request through the Internet. I did and some emails went back and forth, and then it was time to go to bed. The next day I received an email saying that the account had been updated and to try to log in. I also received an email from an account exec asking how it was going. (More than I received from Amazon…)

After work I logged in and tried to create my first “SmartMachine”. Well that sort of failed since I had not finished the registration part the night before. So I added my CC number to the billing info, but it still would not let me create one as it said I had no billing info set… Ha! I logged out and back in and it was much better, it let me pick the size of machine I wanted to create and a way it went. About 10 minutes later I had a root account on a zone on the machine.

So the work began on trying to get my site back up and running between the AWS SUSE VM and the Joyent SmartOS Zone. Surprisingly the SmartOS machine I had picked, had Apache, PHP, MySQL, etc already installed. BUT PHP did not appear to have been compiled with MySQL support. So I just decided to do my own compilation of Apache+PHP+MySQL.

As you can see, it is all up and working now.

So here is my quick comparison of Joyent (standard 64) and AWS (micro T1) given the 1 day of use now:

  1. Easy of signing up:
    • Amazon: Pretty painless. No issues that I had to contact some one for.
    • Joyent: Minor issue, and it may not be their fault, but it did take extra time to get it fixed
  2. OS Selection:
    • Amazon: They have a variety of Linux (7 different Distro’s) and Windows (2003,2008,2012) instances. However neither would be my first choice of OS for my site. Decided on SUSE Linux in the end.
    • Joyent: They offer 3 different OS’. Linux, Windows and SmartOS. SmartOS is a fork of Solaris when it was “open sourced”. Therefore I chose SmartOS, as I would much prefer it over Linux.
  3. Speed of provisioning
    • Amazon: Roughly 15 minutes from start to when I had root access
    • Joyent: Roughly 15 minutes from start to when I had root access.
  4. Processors:
    • Amazon: 1 Processor (Intel Xeon CPU E5-2650 @ 2.00GHz)
    • Joyent: 24 Running at 2.4GHz, 1vCPU
  5. Memory:
    • Amazon: 658Mb
    • Joyent: 2GB
  6. Disk Space:
    • Amazon: 10GB
    • Joyent: 66GB
  7. Networking:
    • Amazon: 15GB out
    • Joyent: First GB out free, each additional up to 10TB is $0.120

 

Right now the cost of both of these VPS’s is roughly around $47 a month. But will see how that works out with the network costs..

I will update in a month after seeing how they both perform.

Posted on

set default DISK$DATA:[OS.EOL]

Seeing the news the other day of HP is discontinuing OpenVMS brought back some memories. Mostly of all the different operating system’s I have used that are no longer around or have changed a lot. Back in my undergrad days, one of the first OS’ we used to program on was OpenVMS on a VAX. It was for an engineering class and we had to use Fortran 77. I remember our quota’s used to be about 5MB in size, which at the time was “huge”.

So to list some of the other OS’ I have seen gone by the way side (and other computer related items that had a huge effect on where I am at today.)

1. OpenVMS (used it between 1994 and 1999)

2. VM/ESA (not really gone, now called zOS, but haven’t touched it since about 2001)

3. Gopher (this is the “original” web…)

4. IRIX (SGI’s UNIX platform. I still have 2 SGI Indy’s and a copy of 5.3 and I believe 6.5, but haven’t had them on in years, maybe a vacation project some time.)

5. SunOS (not Solaris, but the old BSD based SunOS 4.x) my things have changed in the 19 years that I have been doing Solaris work

6. mSQL (mini sql). Not really gone, but surpassed by other’s (mysql, mariadb, etc). I used msql as my first PHP/FI + DB + Apache installation on a Solaris 2.6 box. I wrote a network management application that controlled DNS, DHCP, etc for university dorm connection management.

7. Trumpet Winsock, for the good old Windows 3.1 days when you needed a way to do TCP/IP over modem or ethernet.

8. NCSA Mosaic, the web browser that is credited with popularizing the WWW. Used to use this on some old SGI and DEC machines.

9. ULTRIX, DEC’s version of UNIX. It was on a lot of DECstations in the Engineering department and one computer in the CS department. Used to have a teacher that made us make sure everything compiled on it vs the Solaris or Linux hosts.

10. AltaVista, Search engine to use before Google came around. Now it is just a “front end” to Yahoo search 🙁

11. Atari 400, used to have one of these at the grandparents house to tinker on.

12. Commodore 64, used to have a couple of these when I lived at home. We I learned some BASIC programming. (Later went on to try Visual Basic programming on Windows 3.11 on a 80486 DX4-100 AMD PC.)

13. BeOS, was a really neat idea, excellent media support, unfortunately it was around the time of the PC vs Mac battle so getting buy in was hard.

 

This all also brings back memories how of rudimentary computers were back then and the lack of security. There was no SSH, everything on the VM, OpenVMS and UNIX machines was done through telnet. There was no SSL, and people didn’t think twice about typing in a credit card number on a web site.

I also remember doing web surfing with Lynx on various UNIX systems. And what goes along with Web browsing then email, the first GUI email client I remember using was Pegasus Mail on a Novell Netware based mail system. Once people started doing POP3 mail, people switched over to Eudora Mail. Which I used for a while, but not a lot. I for some reason stuck with Pine a text based mail reader, mostly because I used it on the server that received all the mail.  (And to totally geek out, there were times were I would telnet in to the POP3 port on the mainframe and read my mail by issuing the pop commands by hand.)

As for personal computers, I have had quite a few since my first one. My first computer only had a 40MB hard drive in it. It was a KLH brand 80386 SX 16 that I bought from Phar-Mor. I think I had it maxed out a 4MB of Ram which at the time was huge. I remember trying to play some game on it (I keep thinking it was SimCity, but may be wrong) and it needed more Video RAM cause it only came with 128K of video ram. So I had to buy more to up it to like I think 384K.

As a list of what I have had or still have, here goes:

  1. KLH 80386SX 16MHz – First, no longer have it, came with a 40MB hd, and a EGA 15inch monitor.
  2. AMD 80486DX4 100MHz – Used this to run Windows 3.11, Linux and later Solaris 2.6. It came with a 320MB hard drive. I later paid close to $300 for a 1.6GB hard drive for it. It had a VESA Local Bus video card and a Sound Blaster 16 sound card. No longer have this computer.
  3. Intel Pentium II 266MHz – Bought this in 1997 from a company called Vektron (who later went out of business, like all fly by night computer places back in the early days). It had 32MB of ram and a 500MB hard drive. It ran Windows 95, Windows NT, BeOS, Solaris and Linux. (I had bought bigger and more hard drives later, just can’t remember what all was in it.) I actually still have this machine, it’s most recent use was as a router for my home network running Solaris 10 with 3 NIC’s (one on Comcast, one on Verizon and one on my home network). The hard drive died in it a couple of years ago, so I turned it off, it is still sitting in a rack thought.
  4. Sun SPARCstation 2 – This was my first “workstation”. I got it second hand from a friend’s company. It was where I cut my teeth on Solaris. It ran Solaris 2.5 when I got it, and over the years I upgraded it to Solaris 7. Ironically it only had a 40MHz processor and 64 MB of ram. It had 2 huge external 800MB disk packs and a freakishly heavy 17 inch Sony monitor that used 13W3 connector with BNC ends. I still have this one, but the disk packs both died, so it hasn’t been on in years.
  5. Sun Ultra5 – 360MHz, 128MB of ram. One of the first “IDE” based lower end workstations from Sun. I still have this, but I think the power supply is bad, as I can’t get it to turn on :(. When it ran, I had Solaris 9 on it.
  6. SGI Indy – 2 of these 133MHz with 96MB of ram. One of the coolest “workstations” I ever owned. I believe they both still run, but haven’t been on in years. One ran IRIX 5.3 and the other ran IRIX 6.5
  7. Dual Intel Pentium III 933MHz – Bought this in probably 2001 I think. It is huge, it was a full tower with onboard IDE raid (which only works with Windows because of driver issues.). Right now it has 1.5GB of ram in it, ~2TB of disk and runs Solaris 10 with 7 zones running on it.
  8. IBM Thinkpad i1100, Celeron 500MHz. This one was given to me as a result of work being done for a company. It was my first laptop, and I still have it today. However it’s stats are very underwhelming by today’s point of view. The monitor is an LCD one, but not TFT, so that means there are all kinds of shadows and the picture isn’t crisp. It also only had a 5GB hard drive in it. Which means after installing Windows 2000 on it, there was only maybe a gig free. It also had no floppy drive, and no network ports. So I bought a Linksys WAP11 back in the day (probably in 2002 when I got this) for upwards of $300 so I could have wireless internet on it.
  9. ThinkPad A22p – 900MHz Pentium III. I bought this one as a replacement of the first. Side by side this one is HUGE, as it has a 15 inch display that runs at 1600×1200. It also had a 30GB hard drive (which was split in to 3 10GB chunks, one for Windows XP NTFS, One for Solaris 10 and one for FAT 32 to share files between the two OS’).
  10. AMD 3600+ – Got this one in 2005. It currently runs a combination of Windows XP and Windows 7. Has about 2.5 TB of disk on it.
  11. Sun X2100 – This server. Currently running Solaris 10, with a surprisingly small 160GB of disk with 4 zones on it.
  12. Apple MacBook Pro 2.0GHZ – This was one of the first Intel based Mac’s that was released in 2006. It had a Dual Core 2.0 GHz processor, 2GB of ram an a 100GB hard drive. It did have it’s issues (mostly battery and power adapter ones), but it ran solid for about 5 years. In the fall of 2011 the logic board “died” and it will no longer run in full “user” mode. (I think it is the graphics part of the board.) Still have it hoping for a price drop of replacement boards some day.
  13. Apple Mac Pro – Dual Quad Xeon 2.8GHz with 10 GB of ram. This is the best desktop I have ever had. It is fast and quiet. Right now I think I have close to 13GB of disk on it (both internal and external). I also dual boot it with MacOSX 10.8 and Windows 7 (for a couple of games)
  14. Apple MacBook Pro 2.8GHz iCore7 – the replacement for the one that died above. It is hands down probably 4 to 8 times faster than the 2.0 one that I had before.
  15. Sun V20z – Used to run VMware ESX 3.5 with a Sun T3 fibre connected Disk array. The V20z is fully loaded with processor (2) and ram (16GB). One loud machine…
  16. IBM X3550 – Dual Quad Xeon with 8GB of ram. Used to run VMware vSphere 5.0. Used it to play around with doing virtualization of my house servers. Unfortunately it is too loud to leave running 24×7, so it is only on when needed.
  17. HP XW8600 workstation – Dual Quad Xeon with 16GB of ram. This is my “production” VMware server at  home. It has 3 TB of disk it in and runs probably 11VM’s all the time. It was used to replace the noisy IBM one, and it is super quiet.

As for a list of operating systems I keep current with, it is many and with VMware it is possible to have “test” versions of everything sitting around which helps a lot. Basically the following is what I keep running:

  1. MacOSX 10.7 and 10.8
  2. Windows XP, 7, 8, 2008, 2008R2, 2012
  3. CentOS 6.3
  4. Solaris 10, 11
  5. OpenIndiana 151
  6. pfSense (freebsd)
  7. OpenBSD
  8. Ubuntu Linux

Well that is about enough nostalgia for tonight. Trying to think of other things to put back on the blog to start updating it more often. If you have any idea’s leave a comment (open for 30 days only to keep the spammers away..)