unixwiz anything dealing with *NIX or what ever I want to write about


Cash for Cache

I decided to build a new VMware host for my home "lab" last week to replace the HP workstation I had been using. (The real motive was to turn the HP workstation in to a large NAS since it has 12 SATA ports on it, but more on that later.) So off to part out my new server. What I ended up purchasing was the following (Prices as of 3/24/2015 in USD):


The plan was to set this system up with VMware vSphere 6 and then migrate everything from my VMware 5.1 system to this. So I began building it as the parts arrived last friday night. Everything was going swimmingly until I forgot that the LSI2308 SAS/SATA RAID card doesn't have any cache. What I found was that the 2 480GB SSD drives in a RAID 1 on that card were fast, extremely fast, as in I could boot a Windows 7 or Windows 2012R2 VM in about 3 seconds. However the 2 2TB SATA drives that I made a RAID 1 on there were slow as hell. (Same as the issue I was having with the HPXW8600 system.) I had originally thought it was just the RAID rebuilding, so I left it at the RAID bios over night rebuilding the array.

Well after leaving it at 51% completed and going to bed, waking up 8 hours later and it was only at 63%, I knew that I would never be able to use the SATA drives as a hardware mirror on that device. So I powered it down and disconnected them from the LSI2308 and moved them over to the Intel SATA side of the motherboard. This is where things get interesting, as I really wanted to have a large 2TB mirrored datastore for some of my test vm's that I didn't run 24x7 (the ones I do are on the SSD RAID 1.) In order to achieve this I had to do some virtualization of my storage...

The easiest way I could get the "mirrored" datastore to work was to do the following:

  1. Install FreeNAS vm on the SSD drive (pretty simple a small 8GB disk with 8GB of ram, which would leave me 24GB of ram for my other VM's.)
  2. On each of the 2TB disks, create a VMware datastore, I called them nas-1 and nas-2, but it can be anything you want.
  3. Next create a VMDK that takes up nearly the full 2TB  (or smaller in my case, I created two 980GB VMDK's per each 2TB disk.)
  4. Now present the VMDK's to the FreeNAS VM.
  5. Next create a new RAID 1 volume in FreeNAS using the 2 disks (or 4 in my case) presented to it.
  6. Create a new iSCSI share of the new RAID 1 volume.

Now comes the part that gets a little funky. Because I didn't want the iSCSI traffic to affect my physical 1GB on the motherboard I created a new vSwitch but didn't assign any physical adapters to it. I then created a VMkernel Port on it and assigned the local vSphere host to it with a new IP in a different subnet. I then added another ethernet (e1000) card to the FreeNAS VM and placed it in that same vSwitch and assigned it an IP in the same subnet as the vSphere host.

With the networking "done", it is now time to add the iSCSI software adapter:

  1. In the vSphere Client, click on the vSphere host, and then configuration
  2. Under Hardware, select Storage Adapter, then click Add in the upper right.
  3. The select the iSCSI adapter and hit ok. You should now have another adapter called iSCSI Software Adapter, in my case it was called vmhba38.
  4. Click on the new adapter and then click Properties
  5. Next I clicked on the Dynamic Discovery tab and clicked Add.
  6. In the iSCSI Server address I ended the IP address I made on the FreeNAS box on the second interface (the one on the "internal vSwitch")
  7. Click ok (assuming you didn't change the port from 3260)
  8. Now if you go back and click Rescan All at the top, you should see your iSCSI device.
  9. Now we just need to make a datastore out of it, so click on Storage under the Hardware box
  10. Then Add Storage...
  11. Then follow through adding the Disk/LUN and the naming stuff.

You should now have a new iSCSI datastore on the 2 disks that were not able to be "hardware" mirrored. Using HD Tune in a Windows 7 VM on that datastore I got this:

HD Tune running in Windows 7

As you can see, the left side of the huge spike was actually the writing portion of the test, which got drowned out by the read side of the test. Needless to say the cache on the FreeNAS makes it read extremely fast. As an example a cold boot of this Windows 7 VM took about 45 seconds to get to the login screen from power on. However a reboot is about 15 seconds or less..

Now on the FreeNAS side here is what the CPU utilization looked like during the test:

FreeNAS CPU usage

You can see that is barely touched the CPU's while the test was running. So lets look at the disk's to see how they dealt with it:

FreeNAS disks

It looks like the writes were averaging around 17MB/s, which for a SATA/6Gbps drive is a little slow, but we are also doing a software raid, with cacheing being handled in memory on the FreeNAS side. The reads looked to be about double the writes, which is expected in a RAID 1 config.

The final graph I have from the FreeNAS is the internal network card:

FreeNAS Network

Here we can see the transfer rates appear to be pretty close to that of the disk side. This is however on the e1000 card. I have yet to try it with the VMXNET3 driver to see if I get any faster speeds or not.

While the above may not show very "high" transfer speeds, the real test was when I was transferring the VM's from the HP box to the new one. Before I created the iSCSI datastore and was just using the straight LSI2308 RAID1 on the 2x 2TB disks, the write speed was so bad that it was going to take hours to move a simple 10GB VM. After making the switch, it was down to minutes. In fact the largest one I moved, was 123GB in size and took 138 minutes to copy using the ovftool method.

So why did I title this post Cash for Cache, quite simple, if I had more cash to spend on a RAID controller that actually had a lot of cache on it, and a BBU, I wouldn't have had to go the virtualized FreeNAS route. I should also mention that I would NEVER recommend some one doing this in a production environment as their is a HUGE catch 22. If you only have one vSphere host and no shared storage, when you power off the vSphere side (and consequently the FreeNAS VM) you will lose the iSCSI datastore (which would be expected). The problem is when you power it back on, you have to go and rescan to find the iSCSI datastore(s) after  you boot the FreeNAS vm back up. Sure you could have the FreeNAS boot automatically, but I have not tested that yet and to see if vSphere will automatically scan the iSCSI again to find the FreeNAS share.


Looking to the future, if SSD's drop in price to where they are about equal to current spindle disks, I will likely replace all the SATA hard drives with SSD drives and then this would be the fastest VMware server ever.



Raspberry Pi’ing

Splunk temp monitor

Recently I decided I needed a better way to monitor the temperature and humidity in various parts of my house. The main reason was the thermostat for the house is located in a hallway that is more closed in than anything. So while the thermostat may show that it is 75 degrees in the house the rest of the house my only have been 70 degrees or less. After the winter we have had, I also needed a good way to monitor the humidity in the house. The only was I was able to do it was with a little Oregon Scientific thermometer I bought at Target. But the problem with this was it was only for one room, didn't seem to be very accurate and I had no way of logging the values over a time period.

In comes the Raspberry Pi, along with a DHT22 temperature/humidity sensor and Splunk, I can now monitor, record and graph in realtime the temp and relative humidity in various parts of the house (and the outside).

What I got was this:

  1. 3 x Raspberry Pi 2 Canakits from Amazon.com
  2. 5 x DHT22 Digital Sensors from Amazon.com
  3. 1 x DHT11 Digital Sensor from Amazon.com

Now the DHT11, was what I purchased in the first round along with just one of the Raspberry Pi's. It is not as sensitive as the DHT22's, but since it was just for the original test it was ok for what I needed. The second round I bought the other two Raspberry Pi's and the 5 DHT22 sensors.

What I intend to do is use some of the pre-existing CAT5 runs through the house to wire the DHT22's in to and then have the other end of the CAT5 runs connect in to a Raspberry Pi in the Garage. This way I can do multiple sensors on one device versus having a device in every room.


Some of the benefits of getting the Raspberry Pi Canakits I got are:

  1. A clear case is included with the correct cut outs for the raspberry pi.
  2. A USB wifi dongle is included, and the drivers are pre-loaded in the OS.
  3. It comes with a pre-loaded 8GB microSD card.
  4. It comes with a miniature breadboard with a 40 pin cable and breakout board that plugs perfectly in to the breadboard.
  5. Comes with various resistors and led's and pushbuttons.
  6. Has a HDMI cable included, which made it easy to hook in to my monitor
  7. Various jumper cables for the breadboard


Overall, I would say that the total time to get a base monitor up and running is a few minutes. But this is based of me already having Splunk, the network, dhcp, dns, etc already set up. So I am going to detail the basic steps I used to get it up and running:

  1. Unbox the raspberry pi, place the heatsinks on the two "large" chips on the top side, and then place it in the clear case.
  2. Hook up the HDMI, keyboard, mouse, and WIFI dongle.
  3. Insert to the microSD card
  4. Hook up the USB power cable and watch it boot NOOBS.
  5. Once it is booted, select the Raspbian to install. This probably takes the longest of all the steps to do, as it is expanding the operating system on to the microSD card.
  6. Once this is done, it will reboot and bring up a text based config. I set the hostname, enable ssh, set the timezone and finally set the locale.
  7. At the login prompt, you can log in with the userid pi and the password of raspberry.
  8. Next to set up the network, if you are using the ethernet, then it should already have an IP address if you have DHCP running on your network. If you are using the WiFi dongle, then edit the /etc/wpa_supplicant/wpa_supplicant.conf  file as root and put the following in it:

    Where YOURWIRELESSSSID is the SSID of the AP you want to connect to and the PSK value is the password for that SSID/AP. (If you are doing MAC filtering, you can get the MAC address by running ifconfig -a as root and look at the wlan0.

  9. Once you save the file in the item above, issue the following commands:
    wpa_action wlan0 stop
    ifup wlan0
    ifconfig -a
  10. By now if everything is working correctly you should have a IP address and network connectivity. You can use wpa_cli status to verify the network connectivity.
  11. Now that the network is up and running I needed to download some software:
    sudo su -
    apt-get update
    apt-get upgrade
    apt-get install python-dev
    git clone git://github.com/adafruit/Adafruit-Raspberry-Pi-Python-Code.git
    wget http://www.airspayce.com/mikem/bcm2835/bcm2835-1.42.tar.gz
  12. Now that we have the software downloaded it is time to do some little compiling:
    tar -zxvf bcm2835-1.42.tar.gz
    cd bcm2835-1.42
    make install

    That should install the driver for the bcm2835 chip.

  13. Next we need to do the python code setup:
    cd Adafruit-Raspberry-Pi-Python-Code
    cd Adafruit_DHT_Driver_Python
    python ./setup.py install
  14. At this point the code should be done. You can now power down (shutdown -h now) the Raspberry Pi and hook in the DHT22 sensors. (Make sure to disconnect the power before connecting the 40 pin cable.
  15. The way I hooked the sensor in for testing was to connect the 40 pin cable to the Raspberry Pi and the other in to the breakout board which was attached to the mini breadboard. Once that was done I hooked a jumper from 3.3 V to the first pin on the DHT22. Then placed a 10K resistor between another 3.3V connection and the second pin. In addition a jumper was ran from GPIO4 to the second pin of the DHT22. The third pin is left unconnected and the forth pin is connected to Ground. I will post a picture later.
  16. Once everything is connected, power the Pi back up and log in and switch to the root account.
  17. Next to see if everything is working change in to the Adafruit-Raspberry-Pi-Python-Code/Adafruit_DHT_Driver_Python directory.
  18. Then run python ./Adafruit_DHT.py 22 4. The 22 is the type of the sensor, so if you are using a DHT11 use 11, if a DHT22 use the 22. The number 4 is the GPIO port that the sensors data pin is connected to. Once you run it you should see something like this:
    root@rpi2:~/Adafruit-Raspberry-Pi-Python-Code/Adafruit_DHT_Driver_Python# python ./Adafruit_DHT.py 22 4
    using pin #4
    Temp = 20.2999992371 *C, Hum = 40.4000015259 %
  19. In the above, we can see that the Temp is 20.29C and the Humidity is 40.40%. If you want the Temp outputted as Fahrenheit, like I did, make a copy of the Adafruit_DHT.py file (for a backup) and then add a new line at line 37 with the following:
    tf = (( t * 9 ) / 5.0 ) +32;

    Then on line 39, you will want to change the *C to *F, and then in the format(t,h) you will want to change the t to a tf, so the line would look like this now:

    print("Temp = {0} *F, Hum = {1} %".format(tf,h))
  20. Now if you re-run, it will look like this:
    root@rpi2:~/Adafruit-Raspberry-Pi-Python-Code/Adafruit_DHT_Driver_Python# python Adafruit_DHT-f.py 22 4
    using pin #4
    Temp = 68.1800006866 *F, Hum = 40.0 %
  21. Now that we have the data being output in the format we like, the only thing left was to log it. What I did was create a shell script that is run by cron every minute (* * * * *) and it outputs the values to a log file called /var/log/temp+humid.log. This log file is then pulled in by Splunk for graphing and other fun stuff that will be another post.
  22. The script I wrote looks like this:
    export PATH
    RESULTS="`python /root/TempLogger/Adafruit_DHT-f.py 22 4 | grep Temp `"
    TEMP="`echo ${RESULTS} | awk '{print $3}'`"
    HUMID="`echo ${RESULTS} | awk '{print $7}'`"
    DATE="`date \"+%Y-%m-%d %H:%M:%S\"`"
    echo "${DATE} ROOM=FamilyRoom TEMP=${TEMP} RH=${HUMID}" >> /var/log/temp+humid.log
  23. The output that gets logged looks like this:
    2015-03-17 21:44:02 ROOM=FamilyRoom TEMP=68.1800006866 RH=39.7000007629


Some times, and I haven't figured out why yet, it will log null values for the TEMP and RH. I need to add some more checking in to the script to make it more robust, but for now it is working.

In the next post I will cover what I do with the data in Splunk, and how I get the outside temps from the local airport and add them to Splunk as well.


Subversion on Solaris

So I have been trying to find the "definitive" guide on compiling and installing Subverison on Solaris. There are random sites over the interwebs that have a spattering of different tips, so I thought I would write one how how I did it and what all was done. When you finish this, you will have a basic Subversion system up and running to which you can then further lock down... Requirements: I downloaded the following:

  1. Subversion 1.8.10 (http://mirror.metrocast.net/apache/subversion/subversion-1.8.10.tar.gz)
  2. APR 1.5.1 (http://mirror.metrocast.net/apache/apr/apr-1.5.1.tar.gz)
  3. APR Util 1.5.3 (http://mirror.metrocast.net/apache/apr/apr-util-1.5.3.tar.gz)
  4. scons 2.3.0 (http://prdownloads.sourceforge.net/scons/scons-local-2.3.0.tar.gz)
  5. Serf 1.3.7 (http://serf.googlecode.com/svn/src_releases/serf-1.3.7.tar.bz2)
  6. Apache HTTPD 2.2.27 (http://mirror.metrocast.net/apache/httpd/httpd-2.2.27.tar.bz2)
  7. SQLite 3.8.6 (http://www.sqlite.org/2014/sqlite-autoconf-3080600.tar.gz)
  8. ViewVC 1.1.22 (http://viewvc.tigris.org/files/documents/3330/49347/viewvc-1.1.22.tar.gz)
  9. diffutils 3.2 (http://ftp.gnu.org/gnu/diffutils/diffutils-3.2.tar.gz) [ Needed for ViewVC to work ]

Next up is compiling the software. This is the order I did things:

  1. Apache HTTP Server
  2. APR
  3. APR-Util
  4. SQLite
  5. scons
  6. serf
  7. subversion
  8. viewvc
  9. diffutils

I put all the tar balls in a directory called svn in my home directory. So all the instructions below are relative to it.


Apache HTTP Server

cd httpd-2.2.27
./configure --prefix=/opt/svnweb --with-ssl=/usr/sfw --with-ldap --enable-mods-shared="ssl deflate rewrite ldap authnz-ldap dav dav-fs dav-lock"
make install



cd apr-1.5.1
./configure --prefix=/opt/sungeek
make install



cd apr-util-1.5.3
./configure --prefix=/opt/sungeek --with-apr=/opt/sungeek
make install



cd sqlite-autoconf-3080500
./configure --prefix=/opt/sungeek
make install



mkdir /home/unixwiz/scons
cd scons
tar -xvf ../scons-local-2.3.0.tar
ln -s /home/unixwiz/scons/scons.py /home/unixwiz/bin/scons

(made sure the link points to a directory in your path)



At line 251 of SConstruct add the following (this is needed to get it to work on Solaris):

env['PLATFORM'] = 'posix'

(it should be directly below the line that says env.Append(LIBS='m') in the sunos if statement)

cd serf-1.3.7
vi SConstruct   (edit as above noted)
scons APR=/opt/sungeek APU=/opt/sungeek OPENSSL=/usr PREFIX=/opt/sungeek CC=/usr/sfw/bin/gcc CFLAGS=-D__EXTENSIONS__
scons install

The CC and CFLAGS needs to be set otherwise it will try to use CC and will give you some errors about APR_PATH_MAX.



cd subversion-1.8.10
./configure --prefix=/opt/sungeek --with-apr=/opt/sungeek --with-apr-util=/opt/sungeek --with-serf=/opt/sungeek --with-apxs=/opt/svnweb/bin --with-openssl --with-sqlite=/opt/sungeek
make install
cd /opt/sungeek/libexec
cp mod* /opt/svnweb/modules

Next edit the httpd.conf and add the "LoadModule dav_svn_module modules/mod_dav_svn.so" line after the rewrite_module line.

At the bottom of the httpd.conf add the following: (assuming that /svn is the location of your svn repository.)

<Location /svn/repos>
DAV svn
SVNPath /svn>

Then change the User/Group from daemon to webservd. Also make sure to change the file systems permissions on /svn to be owned by webservd:webservd.



cd subversion-1.8.10
make swig-py
make install-swig-py
echo /opt/sungeek/lib/svn-python &gt; /usr/lib/python2.6/site-packages/subversion.pth



cd viewvc-1.1.22

Installation path: /opt/sungeek/viewvc-1.1.22
DESTDIR path: empth

Edit the /opt/sungeek/viewvc-1.1.22/viewvc.conf and change the following:
svn_roots = svnrepos: /svn
default_root = svn_roots
mime_types_files = /opt/svnweb/conf/mime.types
diff = /opt/sungeek/bin/diff


Next copy the files form /opt/sungeek/viewvc-1.1.22/bin/cgi/*.cgi to /opt/svnweb/cgi-bin

Add the following to the bottom of the httpd.conf

<Directory /opt/sungeek/viewvc-1.1.22>
Order Allow, Deny
Allow from All


cd diffutils-3.2
./configure --prefix=/opt/sungeek
make install



So you want to be an IT Superstar?

Today is one of those days that I have to wonder why I took a career in Information Technology (IT)... You see, I have been doing IT for almost 20 years now and it is not like how the commercials on ITT Tech, or any of those other "tech" trade schools. The commercials make it look like it is just a easy 9 to 5 job, where everything is so cool and collect.

What I am going to tell you is it is the exact opposite. You will work all types of hours, some times days on end with out sleep when something dies. You will have unrealistic expectations assigned to your projects by people who more than likely have never even touched a computer or know how anything works on it, other than to send an email or do an Excel spread sheet. You will also probably give up one weekend a month for the famous "patching day" which can be at any time your management decides they want to be. And because they love to do it, it is usually at like 1AM on a sunday morning, which means you lose the entire weekend because you are trying to get sleep and rested up to work that one 8 hour shift that is not your normal work time.

Once you get past all that stuff, unless you are eager to learn on your own time, you can probably kiss any further training to the sky. In the days now of tight budgets and very high work loads, your best bet at training is some computer based training of "what's new in Windows 7", or something totally unrelated to your actual job.

So now that we have talked about that, what provoked me to say this stuff? Well one company, Microsoft. Today was one of those days where I needed to patch some Windows 2008 Servers because of the monthly release of "security" patches because Microsoft and other vendors are in this mode of getting shit out as fast as possible and not checking the code. So as normal, I approved the 7 or 8 patches for the July cycle in WSUS, so far so good. The part that blows is that the patches applied and the servers said, hey I need to reboot. This was no big surprise because how often have you applied a Windows patch and not had to reboot? So off to reboot the servers, and this is where this shit hit the fan. All of the sudden the server went in to a boot loop. In the off chance that you can catch the blue screen of death in the fraction of a second that it was on the screen, you would see that it mentioned something about an error 0x000007b and that you may have a virus.

Well, I can guarantee you that the machines don't have virus' on them. So investigating the error further it appears that the 0x7b is an error that says that the OS can't find the hard drive. Which is ironic because it has booted off of it to get that far. This then starts the oh-shit moment. Luckily this was only 1 of 2 Active Directory servers. I spent a while trying to get it to boot buy following all these different articles. To no avail I could not get it to boot up.

The biggest thing that pissed me off was Microsoft used to have a boot mode where you could step through each driver as it was loading and say whether to load it or not. Unfortunately, I can't find that any where in the F8 menu or any of the other google foo searches. So I tried each of the safe mode options, which each BSoD. I tried Debug Mode, BSoD. I tried to have it log the startup to the ntbtlog.txt, nope, doesn't even write to it. So now I am extremely pissed, to the point where I just said F@#K it, and started a reinstall of Windows 2008R2 (the environment this was in I could do it). But before I did it I tested the other AD server, yup, it bit the dust too.

Luckily reinstalling W2K8 doesn't take terribly long.  However it is a pain in the ass getting an entire environment set back up because one patch blew up your servers. So while I was reinstalling these two servers, I decided to test another less critical server on a different network. Guess what it died too with the same error. So now I am thinking about how bad this could have been if I were doing some heavily used servers.  (Once again this stuff isn't shown in the "tech school" commercials.)

So how do you go forward from this, well there are 2 different type of "tech" people. Those who go home, and start testing every single possibility in their own private lab. Then there are those who don't give a F and wait for other people to fix their problems as they don't have the first clue how to fix stuff if a reboot doesn't fix it.

Can you guess which type of a tech person I am? If you guessed the former, you are correct. First thing I did when I got home from work is created a new W2K8R2 VM and started the OS installing and trying to get it up to the patch level I had the machines at work. But because this is windows that takes FOREVER with all the reboots and waiting for it to "see" the patches offered to it.

The group in the later (those who don't care and wait for others to fix it) really start to make me mad now days. Now I can say that I spend a lot of my own free time doing a lot of stuff to teach my self practically everything I know about IT, as when I went through school, none of this stuff was taught (Shit, I am a UNIX person, but bought a Microsoft TechNet subscription just to learn as much as I can about Windows Server, etc). But some "IT" people seem to get pissed when I make the notion that they need to learn this stuff on their own at home. It is almost the "how dare you ask me to do something on my free time to better my self when I can sit here and do nothing." Well that is the only way you are going to better your self, and learn from your mistakes with out affecting something at your work that may affect something with your pay ...


As I said at the beginning I have been doing IT for close to 20 years now. In that time I have had my hands on the following:

  • Every version of SunOS/Solaris from 4.1.1 up to the current (11)
  • Every version of Microsoft Windows from 3.11 through Server 2012
  • IBM AIX 3.1.2 through 6
  • VM/ESA
  • OpenVMS
  • Various distributions of Linux (and this is one of my huge pet peeves, but that is for another post)
  • Every version of MacOS from 7 through the current 10.9
  • Practically every version of VMware from the original VMware workstation 1.0 on Linux, to vSphere 5.1 to VMware fusion 6.
  • BeOS
  • OS2/Warp
  • Novell Netware

And that is just Operating systems, some of which don't even exist any more. The hardware side is so numerous that is hard to even keep track of, but lets just say I got in to computers when an 80286 8MHz was considered fast and bleeding edge, not to mention a Commodore 64, and Atari 800.


So what is the moral of this post? Really think if you want to get in to IT, and do you have the thirst for learning and teaching yourself. If you don't have that and don't want to spend some times hours a night learning how stuff works, or if spending an entire weekend at work on a nice summer day doing patches is not your thing, please don't take that type of job. IT is almost like a dedication and devotion, if you don't have the time to do it, you probably shouldn't start it.


Joyent SmartOS network monitoring

My free trial period of my Smart Machine ended, so now I was trying to find a way to monitor my bandwidth usage on my Smart machine. There isn't a "easy" way of doing (like logging in to the portal to look at your account) so I devised a way to do it on my own.

The first part of it will be discussed in this post, and I will do another about how to actually view the results.

First off the easiest way I have found to "watch" network traffic is using the kstat command. On my SmartMachine, I have 2 network interfaces, one that has the public interface on it, and one that has the private interface on it. For my purposes I am only currently watching "net1" which is the external interface.

So the small script I have runs every 10 minutes, and logs the information in to a MySQL table. That table is defined like this:

CREATE TABLE `vmnet` (
`interface` char(10) DEFAULT NULL,
`time` bigint(20) DEFAULT NULL,
`obytes` bigint(20) DEFAULT NULL,
`rbytes` bigint(20) DEFAULT NULL,
`htime` datetime DEFAULT NULL,
KEY `tidx` (`time`)


The columns are as follows:

  • interface: which interface we are getting the stats from, right now everything just says net1. But if I were to add net0 it would fit right in.
  • time: time in seconds since the epoch
  • obytes: bytes leaving the interface
  • rbytes: bytes received on the interface
  • htime: human readable time. (Yes i realize I am storing the time twice, and that I can do everything with just time, but what the heck, it is just an extra little storage ;-)...


Now that the table in the DB is defined, set the permissions on it. In my case I created a database just for the "netstats"  and there is just the one table in it called vmnet. I created 2 users that have access to the vmnet table. One just for writing the data in from the script, and another for reading the data for part 2 of this.


Now for the script, it is pretty simplistic:

#Use kstat to grab interface stats
#Define the interface to look at:
VALUES="`kstat -c net -n ${INTF} | egrep \"(obytes64|rbytes64)\"`"
SNAPTIME="`perl -e \"print(time());\"`"
OBYTES="`echo ${VALUES} | grep obytes64 | awk '{print $2}'`"
RBYTES="`echo ${VALUES} | grep rbytes64 | awk '{print $4}'`"
echo "insert into vmnet values ('${INTF}',${SNAPTIME},${OBYTES},${RBYTES},NOW());" | /opt/local/bin/mysql -uUUUUUU -pPPPPPPPPP netstats


In the most simplest form, the script runs the kstat command on the requested interface ${INTF} and then uses egrep to grab the obytes64 and rbytes64. It then takes those to values and creates a sql insert and piles that in to mysql command where UUUUUU is the username and PPPPPPPP is the password for the insert use on the netstats database.

I then run this every 10 minutes. And what you end up with is data in the table that looks like this:

| interface | time       | obytes     | rbytes     | htime               |
| net1      | 1388373702 | 3123241114 | 3977125001 | 2013-12-29 22:21:42 |
| net1      | 1388374200 | 3123381303 | 3977326242 | 2013-12-29 22:30:00 |
| net1      | 1388374457 | 3140146411 | 3977725426 | 2013-12-29 22:34:17 |
| net1      | 1388374800 | 3140170245 | 3977843340 | 2013-12-29 22:40:00 |
| net1      | 1388375400 | 3140526526 | 3978051264 | 2013-12-29 22:50:00 |

Next time I will show how to take the data and make something out of it:

graph of network traffic

netstat output