Surprising is that IBM put’s one of their own OS’s near the bottom of the list. Some of my opinions are :

1. No one uses AIX that much, so no one looks for holes in the code.
2. Any one who uses AIX, doesn’t have it directly connected to the Internet.
3. It is so cost prohibitive to use, that people are looking at Solaris/Linux or Windows to run their business on.

But the funniest thing about this is the last I used AIX the following were still done on install by IBM:
1. telnet enabled
2. root logins allowed remotely
3. no ssh comes with the OS, you have to install a crappy “linux toolkit”, and then install another 10 different packages to get SSH enabled.
4. No RBAC
5. Syslog configuration does not exist
6. Root does not even have a password on install

Seems to me that IBM needs to fix some fundamental issues with their OWN OS before they can say it is not one of the “Most Vulnerable Operating Systems”.

The funniest issue with this is for MacOSX to be listed at the top, all most all of those require some one to actually run something on the machine with administrative privileges.

1. Setup Sun’s JES Directory Server on a Solaris 10 machine
2. Configure the DS to have the AIX schema and objectclasses needed for user and group info
3. Configure a AIX test machine to authenticate against the Sun LDAP Server
4. Eventually move all AIX machines from the IBM DS to Sun DS, thereby having one set of servers that control all users/passwords for all Sun/Linux/AIX machines

Step 1: AIX Schema
The first thing I had to overcome is how AIX has 4 different ways of using LDAP for authentication. AIX 4.3.3 and AIX 5.1 used a non RFC2307 compliant schema. AIX 5.2 and AIX 5.3 can use this old schema or RFC2307, or a new one called RFC2307AIX, which combines the old with the new (there is also another one that I will not cover). When I originally setup the IBM LDAP I used the RFC2307AIX because it allows me to store ulimits and other info about AIX accounts in LDAP and not on each individual machine. But this also makes it harder to port these things over to Sun’s LDAP. Which lead me to yesterday’s afternoon adventure, creating a schema file that would work. I will link to my final copy below.

Step 2: AIX ObjectClasses
Second up was to create the objectclasses required for AIX Authentication, which consisted of creating the eAccount, AIXAccount,AIXaccessGroup, ibm-SecurityIdentities, container, and account. Some of these may not be needed for a fresh install, but I am trying to move entries from IBM’s LDAP to Sun’s LDAP with the least amount of editing an extremely huge ldap ldif export file. I will link to my final copy of this file below as well.

Step 3: Install JES DS
The third step was to download and install the DS 5.2.P4 on my fresh install of Solaris 10 U3 running on a Sparc machine. I did the custom install as I wanted to change the location of where it was installed to. The other thing I did is not load any sample data. Once the install was done, I ran the /usr/lib/ldap/idsconfig script to setup the DS. This is sort of how it went (copied from another doc):

Unfortunately the “directoryserver” command does not exist in Solaris 10, so i did the following:

(I installed the ldap server to /ldapserver)

Now that the indexes are created we can go on to the next step of modifying the schema.

Step 4: Importing new schema
Now I can import the AIXAttributes.ldif and the AIXObjectClasses.ldif files to my fresh newly installed server:

All should go well on the above imports. There were a couple of quirks that I found when creating the files, like how IBM uses one OID for a value when Sun uses a different one.

Step 5: Creating an OU for AIX data
By this time I have fired up the Console for the LDAP server and was doing things through the gui. The first thing I did was create a new OU for my aix data, i.e. ou=aixdata,dc=example,dc=com. This OU is where all my AIX stuff will be, the ou=people,dc=example,dc=com will be where my Sun users will go. (Can’t combine them yet because of massive amounts of differing UIDS between the AIX users and Sun Users).

After creating this, create 2 more OU’s under the aixdata. The first one will be for all users, ou=aixuser,ou=aixdata,dc=example,dc=com. The second will be for group information, ou=aixgroup,ou=aixdata,dc=example,dc=com.

Step 6: Create and AIX LDAP Admin Account
Because of how the AIX servers need to connect and get/put info into ldap, you will need to create an account that has read/write access to the 2 new ou’s you created. (This account is sort of similar to the Sun proxyagent account created with the idsconfig). I created my user, uid=aixldap,ou=aixdata,dc=example,dc=com. Also set the password for it to never expire (if you make it expire, you will have to update every AIX server every time the password expires). Once this user is created give it full read/write access to the ou=aixuser and ou=aixgroup with ACI’s.

Step 7: Export info from IBM LDAP
Since I am planning on moving from IBM LDAP to Sun LDAP I need to export the data from my IBM LDAP to an ldif format:

Do the above on the IBM LDAP server, it will put a file called /tmp/ldapexport.ldif. But now I need to “clean” some stuff out of it. Some of the stuff I need to remove is anything that is not User and Group related. For example I had IBM LDAP Replication setup so there are a ton of entries for that. (so the only thing you should have in your ldif file are entries for the dn’s like username=*,ou=aixuser,ou=aixdata,dc=example,dc=com and groupname=*,ou=aixgroup,dc=example,dc=com
)

Step 8: Import users and groups
Next I imported the cleaned file:

Now all your users and groups should be in the new ldap server.

Step 9: Misc config in Sun LDAP
Some other stuff I did was add a couple indexes to the non-standard attributes that AIX uses:
groupname
username
hostsallowedlogin

The hostsallowedlogin allows us to put an entry in the persons LDAP entry to say which hosts they can log in to. If the attribute does not exist they can log in to any host that is served by this LDAP server. But if they have a value in this attribute, they can only log in to those host(s). (there is also a hostsdeniedlogin, which is the opposite of the hostsallowedlogin, if you want them to log in to every machine but one you can just populate that single host in to the hostsdeniedlogin).

Step 10: Configure AIX to talk to ldap server
One of the fall backs I don’t like about LDAP on AIX is that you have to have a local ldap user, and the ldap client software does not come with the base os. So you will have to install the ldap.client.adt and ldap.client.rte (probably don’t need the adt, but I install it anyways). During this install it usually creates the ldap user and group, which is never where we want it so either create an ldap group and ldap user before you install it, or after it is installed do the following, change the userid and groupid to what you want it to be in /etc/passwd and /etc/group and then run:

Now we can run the mksecldap command:

The above is all on one line. You must make sure the -t is set to 0, if it is not then you will get some weirdness that I will talk about later.

Once this is done, then you need to tell AIX that it is to look in LDAP for it’s info, you will need to edit another file /etc/security/user:

in the “default:” stanza, you will need to change the SYSTEM variable to LDAP, and add a registry variable with the value of LDAP.

I.E.:

Now you should be able to do an id username, and see some results:

Next up try logging in remotely (you may have to create your home directory first). It should work.

Some Notes

1. If you do not set the cachetimeout to be 0, if you change a users password as root, the user will get thrown in to a loop of changing their passwords. I.E. They will never be able to login as everytime they login it will say their password has expired and make them change it again and then it will kick them off (ie close their ssh session). Lather/rinse/repeat.
2. After importing the users from an existing IBM server, you can delete the IBM-ENTRYUUID and the control attributes, they are for IBM DS only and have no use in Sun’s LDAP.
3. Any problems on the import/etc, make sure to look at the ldap server error logs (/ldapserver/slapd-ldap2/logs/error_log) it should tell you exactly what is wrong.
4. If you replicate this LDAP server, you should probably do a Master/Master relationship because of how AIX always stores info about last logins/etc in LDAP. I have not tested yet if it can follow referrals yet.

Here are the two files I was talking about:

AIXAttributes.ldif
AIXObjectClasses.ldif

Some other links:
IBM’s Redbook : Integrating AIX into Heterogeneous LDAP Environments, which I found was missing some stuff

Sun’s Cookbook for Solaris 8 client with Directory Server 5.1/Solaris 9 which I follow some times if I need to connect Solaris 8 machines in.

Hope this helps some one, if it does leave me a message.

Technorati Tags: IBM, Sun, LDAP

The hint to finding the files is given in the first line of the recommended actions. Use the fuser command. The actual command is:

What this will show you is something similar to this:

The column we are really interested in is the last one. That is the column that contains the Process ID of the process that has a particular file open in that file system. So for example process id 1671280 has inode 78 in the /tmp file system open, and that file is using ~1.4MB of space. So if you “kill 1671280″ you would recover the 1.4MB of space. (But make sure you kill it in the proper way, i.e. find out what the process really is and shut it down nicely.)

The other magazine I got is the “IBM Systems Magainze”. It is funny reading through this “magazine” where everything is Pro-IBM and how much better it is than any thing else. One of the articles was “new” features to AIX 5.3, here are some of the things they list:

• An environment variable LD_LIBRARY_PATH for runtime library path has been added for compatibility with SVR4 bases systems.

Interesting, that they say that as I have seen our DBA’s “misusing it” for years now. Doesn’t seem new to me..

Next up, in the base commands and libraries

• A new flag has been added to the tar command, which would specifiy the list of files and/or directories to be excludede from the tar file being created,extracted or listed.
• Text-processiong commands ed/vi/ex support unlimited line lengths
• vi and ex can handle files up to 2GB
• Support for unlimited number of fields and size of a line for awk
• Cimand buffer sizes are unlimited, allowing an increased number of arguments o a command line

Seems AIX tar is a little behind in the game.. (Not to mention that I am still not sure if it can handle largerfiles. And the problem it has with it’s own userid nobody which is something to large to fit in uid_t in a tar archive.)

That is all cool about VI being able to handle 2Gb files, but how often do normal SA’s want to edit a 2gb file. (DBA’s, maybe all the time, but I do not try to ever edit a file that big)..

The unlimited line length is a definate plus, but wonder if there is a buffer overflow in that.

And the funniest of all, (you probably have to be an AIX administrator to get this, but it sounds funny if you are not).:

• The PP size starts at 1MB and goes up to 128GB.

If you read it aloud around people who are not AIX adminstrators, they will probably start laughing. (For those who are not AIX administrators a PP is a physical partition size. Each logical volume in a volume group is made up of physical partitions. )

Here is a quick AIX LVM overview:

Basicly, we have a volume group called “testvg” that has 4 harddrives in it, (hdisk1, 2,3,4). In this volume group there are 3 Logical volumes created that are mounted to /filesystem1, /fs2 and /fs3. In this example /filesystem1 is made up of 4 PP’s, /fs2 is made up of 3 PP’s, and /fs3 is made up of 2 PP’s. They do not have to be in sequential order and if in fact when you change the size of file systems (usually increasing) it will grab the next free PP assuming there is one in the file system. The PP size is the “minimum” size a file system can be made and increased. So if I were to say that the PP size of each of the PP’s in the testvg were 512MB, then /filesystem1 would be roughly 2 GB, /fs2 would be 1.5 gb, and /fs3 would be 1 gig. This volume group is also not mirrored so the LV’s are just allocated in sequence. Maybe later I will do a more indepth picture out different options for AIX Volume Groups.

The “enhancements” to aix 53 for LVM include this:

AIX 5L Version 5.2 offers a new volume group type called scalable volume group (VG), which can accommodate up to 1,204 physical volumes,(N.B. old regular VG’s are limited to 32, Big VG’s it was increased but not sure to what) 4,096 logical volumes (LV’s) and 2,048K physical partitions (PPs). )

Needless to say you can make some pretty big volume groups with some pretty big file systems, but in my personal experience, the larger they are the longer they take to fix etc when a file system becomes corrupted, which for some reason seems to happen more with JFS2 than with JFS. Have not figured out what the exact cause is though.

All in all, you may end up paying a little more for a Ultra45, but then again, it is a way better machine than the pSeries 185. It also runs Solaris which I feel is far superior to AIX. AIX is cool and all, but has too many quirks that just does not make to much sense. There are things in Solaris that are done so much easier and faster than in AIX that just make me laugh when I have to answer how to do something in AIX vs Solaris.

Here are some of my pet quirks about AIX:

1. Disk numbering scheme: all disks in AIX are named in the form of hdisk#. To find out exactly where they are at you have to do either a “lsdev -Cc disk” or “lsattr -El hdisk#” to find the actual controller and slot it is connected to.
2. ODM: Seems too much like the windows registry to me. Screw it up, and your machine does not boot right
3. The “dumbing” of Sysadmins by their dependence on SMIT. Take a AIX admin and put them in front of Solaris/Linux/etc and have them try to do any administrative tasks, and it is a complete loss with out smit. But take a Solaris/Linux/etc admin and put them on AIX, and they can accomplish most of the same administrative tasks with out touching SMIT
4. NIM, Nim is AIX’s equivelant of Jumpstart on Solaris. Jumpstart can be setup in probably under 10 minutes and be booting and installing machines. Nim on the other hand is an all day affair. I kid you not, I spent 8+ hours one day configuring an NIM environment to boot 1 machine. And even then, it did not install all the needed software. It also takes forever to copy 8 CD’s of AIX install media, plus the “Linux ToolKit”, Plus the expansion pack just to get SSH installed on AIX when NIM is used to install a system. If I could only get AIX to boot from a jumpstart server I would be set.
5. Missing core software that should be installed no matter what type of install you do. For example SSH. What operating system besides Microsoft Windows now days does not come installed with SSH? AIX, yup, you have to have 3 different cd’s to install it, and you better be using the OpenSSH supplied by IBM, or they will refuse to talk to you about any problems. (Yes they actually had me verify every part of the version of SSH before they would talk to me.
6. Root allowed to log in remotely. By default when you install AIX, root can log in remotely. MMM Bad Mkay…..
7. Default Open Relay: Last time I checked Sendmail on AIX is still configured by default to be an open relay.
8. Syslog: AIX likes to put stuff in its proprietory errpt. Which means to get the information to log to a central syslog server, you have to modify the ODM to run a script to grab output from the errpt command to send to syslog. Why can’t it send to syslog by default?

And the list could go on for ever, but right now it is time to go to bed.

For those who are interested in knowing how to access a volume group that has quorum checking turned on but not enough disks… This is what I did on a IBM B80, that lost a disk that was in a volume group that held paging space and was not mirrored for what ever reason, and as a result it crashed the machine.

#Force the varyon of the volume group in to systems maintenance mode

#remove the disk from the volume group that had died, you need the PVID

#Show what disks are in the vg, should only be one now

#Varyoff and on to make sure it works right

#add the replaced disk in to the vg

#show that it was added

After this was done, I then added the paging space back to hdisk 2 and then mirrored the volume group. The reason the machine crashed was from a couple of different things.

1. The volume group was not mirrored, and when hdisk2 died, it took a paging space with it.

2. quorum checking was left on. If there are only 2 disks in a volume group, make sure that quorum checking is off, otherwise you will never be able to access the volume group with out doing the above steps.

Technorati Tags: AIX

And comment it out. It “should” have a comment like such above it:

Once you comment it out, all the mail will now be sent directly to the host defined in the DS entry. You may have to stop and restart sendmail for it to take effect.

Technorati Tags: AIX, Sendmail

1. Drop the use of DB2
2. Drop the requirement of needing 2 seperate userid’s just to get it installed.
3. Get the documentation on your web site updated, and take down the white papers that are referencing information that is 2 years old.
4. Call Sun and learn how to do it right.