unixwiz

a while ago I wrote about getting AIX authenticating to a SUN LDAP because of problems with AIX’s LDAP replication that I was having.. I forgot to mention that I “fixed” the AIX LDAP Replication problem. The solution was to put the FQDN of each “master” server in the /etc/hosts file with the IP address associated with it. It seems that IBM’s LDAP server is completely brain dead when it comes to DNS and LDAP, and would not replicate correctly unless I put the master server’s IP’s in the /etc/hosts.. So if you are having problems with getting replication to work with IBM’s LDAP server, try putting the entries in /etc/hosts and I bet it will work..

I have been having some problems getting IBM’s Tivoli Directory Server to replicate properly. So I decided to try and see what it would take to move the IBM AIX machines from using IBM’s LDAP Server to using Sun’s LDAP Server. This is what I hope to accomplish:

1. Setup Sun’s JES Directory Server on a Solaris 10 machine
2. Configure the DS to have the AIX schema and objectclasses needed for user and group info
3. Configure a AIX test machine to authenticate against the Sun LDAP Server
4. Eventually move all AIX machines from the IBM DS to Sun DS, thereby having one set of servers that control all users/passwords for all Sun/Linux/AIX machines

Step 1: AIX Schema
The first thing I had to overcome is how AIX has 4 different ways of using LDAP for authentication. AIX 4.3.3 and AIX 5.1 used a non RFC2307 compliant schema. AIX 5.2 and AIX 5.3 can use this old schema or RFC2307, or a new one called RFC2307AIX, which combines the old with the new (there is also another one that I will not cover). When I originally setup the IBM LDAP I used the RFC2307AIX because it allows me to store ulimits and other info about AIX accounts in LDAP and not on each individual machine. But this also makes it harder to port these things over to Sun’s LDAP. Which lead me to yesterday’s afternoon adventure, creating a schema file that would work. I will link to my final copy below.

Step 2: AIX ObjectClasses
Second up was to create the objectclasses required for AIX Authentication, which consisted of creating the eAccount, AIXAccount,AIXaccessGroup, ibm-SecurityIdentities, container, and account. Some of these may not be needed for a fresh install, but I am trying to move entries from IBM’s LDAP to Sun’s LDAP with the least amount of editing an extremely huge ldap ldif export file. I will link to my final copy of this file below as well.

Step 3: Install JES DS
The third step was to download and install the DS 5.2.P4 on my fresh install of Solaris 10 U3 running on a Sparc machine. I did the custom install as I wanted to change the location of where it was installed to. The other thing I did is not load any sample data. Once the install was done, I ran the /usr/lib/ldap/idsconfig script to setup the DS. This is sort of how it went (copied from another doc):

Unfortunately the “directoryserver” command does not exist in Solaris 10, so i did the following:

(I installed the ldap server to /ldapserver)

Now that the indexes are created we can go on to the next step of modifying the schema.

Step 4: Importing new schema
Now I can import the AIXAttributes.ldif and the AIXObjectClasses.ldif files to my fresh newly installed server:

All should go well on the above imports. There were a couple of quirks that I found when creating the files, like how IBM uses one OID for a value when Sun uses a different one.

Step 5: Creating an OU for AIX data
By this time I have fired up the Console for the LDAP server and was doing things through the gui. The first thing I did was create a new OU for my aix data, i.e. ou=aixdata,dc=example,dc=com. This OU is where all my AIX stuff will be, the ou=people,dc=example,dc=com will be where my Sun users will go. (Can’t combine them yet because of massive amounts of differing UIDS between the AIX users and Sun Users).

After creating this, create 2 more OU’s under the aixdata. The first one will be for all users, ou=aixuser,ou=aixdata,dc=example,dc=com. The second will be for group information, ou=aixgroup,ou=aixdata,dc=example,dc=com.

Step 6: Create and AIX LDAP Admin Account
Because of how the AIX servers need to connect and get/put info into ldap, you will need to create an account that has read/write access to the 2 new ou’s you created. (This account is sort of similar to the Sun proxyagent account created with the idsconfig). I created my user, uid=aixldap,ou=aixdata,dc=example,dc=com. Also set the password for it to never expire (if you make it expire, you will have to update every AIX server every time the password expires). Once this user is created give it full read/write access to the ou=aixuser and ou=aixgroup with ACI’s.

Step 7: Export info from IBM LDAP
Since I am planning on moving from IBM LDAP to Sun LDAP I need to export the data from my IBM LDAP to an ldif format:

Do the above on the IBM LDAP server, it will put a file called /tmp/ldapexport.ldif. But now I need to “clean” some stuff out of it. Some of the stuff I need to remove is anything that is not User and Group related. For example I had IBM LDAP Replication setup so there are a ton of entries for that. (so the only thing you should have in your ldif file are entries for the dn’s like username=*,ou=aixuser,ou=aixdata,dc=example,dc=com and groupname=*,ou=aixgroup,dc=example,dc=com
)

Step 8: Import users and groups
Next I imported the cleaned file:

Now all your users and groups should be in the new ldap server.

Step 9: Misc config in Sun LDAP
Some other stuff I did was add a couple indexes to the non-standard attributes that AIX uses:
groupname
username
hostsallowedlogin

The hostsallowedlogin allows us to put an entry in the persons LDAP entry to say which hosts they can log in to. If the attribute does not exist they can log in to any host that is served by this LDAP server. But if they have a value in this attribute, they can only log in to those host(s). (there is also a hostsdeniedlogin, which is the opposite of the hostsallowedlogin, if you want them to log in to every machine but one you can just populate that single host in to the hostsdeniedlogin).

Step 10: Configure AIX to talk to ldap server
One of the fall backs I don’t like about LDAP on AIX is that you have to have a local ldap user, and the ldap client software does not come with the base os. So you will have to install the ldap.client.adt and ldap.client.rte (probably don’t need the adt, but I install it anyways). During this install it usually creates the ldap user and group, which is never where we want it so either create an ldap group and ldap user before you install it, or after it is installed do the following, change the userid and groupid to what you want it to be in /etc/passwd and /etc/group and then run:

Now we can run the mksecldap command:

The above is all on one line. You must make sure the -t is set to 0, if it is not then you will get some weirdness that I will talk about later.

Once this is done, then you need to tell AIX that it is to look in LDAP for it’s info, you will need to edit another file /etc/security/user:

in the “default:” stanza, you will need to change the SYSTEM variable to LDAP, and add a registry variable with the value of LDAP.

I.E.:

Now you should be able to do an id username, and see some results:

Next up try logging in remotely (you may have to create your home directory first). It should work.

Some Notes

1. If you do not set the cachetimeout to be 0, if you change a users password as root, the user will get thrown in to a loop of changing their passwords. I.E. They will never be able to login as everytime they login it will say their password has expired and make them change it again and then it will kick them off (ie close their ssh session). Lather/rinse/repeat.
2. After importing the users from an existing IBM server, you can delete the IBM-ENTRYUUID and the control attributes, they are for IBM DS only and have no use in Sun’s LDAP.
3. Any problems on the import/etc, make sure to look at the ldap server error logs (/ldapserver/slapd-ldap2/logs/error_log) it should tell you exactly what is wrong.
4. If you replicate this LDAP server, you should probably do a Master/Master relationship because of how AIX always stores info about last logins/etc in LDAP. I have not tested yet if it can follow referrals yet.

Here are the two files I was talking about:

AIXAttributes.ldif
AIXObjectClasses.ldif

Some other links:
IBM’s Redbook : Integrating AIX into Heterogeneous LDAP Environments, which I found was missing some stuff

Sun’s Cookbook for Solaris 8 client with Directory Server 5.1/Solaris 9 which I follow some times if I need to connect Solaris 8 machines in.

Hope this helps some one, if it does leave me a message.

Technorati Tags: IBM, Sun, LDAP

Ever wonder were all the space went in a file system? Does du and df show different results on the same file system? Chances are some one deleted a file that was open in a particular file system. For example the /tmp file system on one of our machines has been filling up and then some one was deleting files from it. But the space is never recovered. In the AIX errpt you may see something like this:

The hint to finding the files is given in the first line of the recommended actions. Use the fuser command. The actual command is:

What this will show you is something similar to this:

The column we are really interested in is the last one. That is the column that contains the Process ID of the process that has a particular file open in that file system. So for example process id 1671280 has inode 78 in the /tmp file system open, and that file is using ~1.4MB of space. So if you “kill 1671280″ you would recover the 1.4MB of space. (But make sure you kill it in the proper way, i.e. find out what the process really is and shut it down nicely.)

Got a couple of magazines in the mail today. One is SysAdmin. Every once in a while they bundle something with it, todays bundle was a CDROM from IBM on “Migrating to Linux or AIX from Solaris OS”. It was one of their “RedBook” cdrom’s. Funny part to me is why would any one want to migrate from Solaris to Linux or AIX. I have been using AIX since about 1995, and to be honest, I would much rather use Solaris any day over AIX. The _only_ thing on AIX I “like” better is maybe printing. As for Linux, I have used it in the past, but in my own opinion it has fallen to it’s own success. Yeah a lot of people have learned how *NIX “works” by using it, but every day there is a distro coming out, and a new way of doing something. Basicly every distro has it’s own way of doing things and most of the time they don’t work on other distros. So for me I definately will not be migrating from Solaris to AIX or Linux, if any it would be the opposite.

The other magazine I got is the “IBM Systems Magainze”. It is funny reading through this “magazine” where everything is Pro-IBM and how much better it is than any thing else. One of the articles was “new” features to AIX 5.3, here are some of the things they list:

• An environment variable LD_LIBRARY_PATH for runtime library path has been added for compatibility with SVR4 bases systems.

Interesting, that they say that as I have seen our DBA’s “misusing it” for years now. Doesn’t seem new to me..

Next up, in the base commands and libraries

• A new flag has been added to the tar command, which would specifiy the list of files and/or directories to be excludede from the tar file being created,extracted or listed.
• Text-processiong commands ed/vi/ex support unlimited line lengths
• vi and ex can handle files up to 2GB
• Support for unlimited number of fields and size of a line for awk
• Cimand buffer sizes are unlimited, allowing an increased number of arguments o a command line

Seems AIX tar is a little behind in the game.. (Not to mention that I am still not sure if it can handle largerfiles. And the problem it has with it’s own userid nobody which is something to large to fit in uid_t in a tar archive.)

That is all cool about VI being able to handle 2Gb files, but how often do normal SA’s want to edit a 2gb file. (DBA’s, maybe all the time, but I do not try to ever edit a file that big)..

The unlimited line length is a definate plus, but wonder if there is a buffer overflow in that.

And the funniest of all, (you probably have to be an AIX administrator to get this, but it sounds funny if you are not).:

• The PP size starts at 1MB and goes up to 128GB.

If you read it aloud around people who are not AIX adminstrators, they will probably start laughing. (For those who are not AIX administrators a PP is a physical partition size. Each logical volume in a volume group is made up of physical partitions. )

Here is a quick AIX LVM overview:

Basicly, we have a volume group called “testvg” that has 4 harddrives in it, (hdisk1, 2,3,4). In this volume group there are 3 Logical volumes created that are mounted to /filesystem1, /fs2 and /fs3. In this example /filesystem1 is made up of 4 PP’s, /fs2 is made up of 3 PP’s, and /fs3 is made up of 2 PP’s. They do not have to be in sequential order and if in fact when you change the size of file systems (usually increasing) it will grab the next free PP assuming there is one in the file system. The PP size is the “minimum” size a file system can be made and increased. So if I were to say that the PP size of each of the PP’s in the testvg were 512MB, then /filesystem1 would be roughly 2 GB, /fs2 would be 1.5 gb, and /fs3 would be 1 gig. This volume group is also not mirrored so the LV’s are just allocated in sequence. Maybe later I will do a more indepth picture out different options for AIX Volume Groups.

The “enhancements” to aix 53 for LVM include this:

AIX 5L Version 5.2 offers a new volume group type called scalable volume group (VG), which can accommodate up to 1,204 physical volumes,(N.B. old regular VG’s are limited to 32, Big VG’s it was increased but not sure to what) 4,096 logical volumes (LV’s) and 2,048K physical partitions (PPs). )

Needless to say you can make some pretty big volume groups with some pretty big file systems, but in my personal experience, the larger they are the longer they take to fix etc when a file system becomes corrupted, which for some reason seems to happen more with JFS2 than with JFS. Have not figured out what the exact cause is though.

Thought this was intresting. If IBM wants people to use their PowerPC chips on the desktop then need to go a lot lower than $4,000 for a “desktop” workstation. I can get an entry level Sun Blade 150 machine for under $1,000. Granted it does not have the same specs as the IBM one does, but for some one starting out on a platform, I would much rather spend less money and get a slightly slower machine than a lot to find out that something does not work right. One thing that is not mentioned on that page, but on the IBM page is that it is really a “server”. Which means this little desktop machine weighs in a 55 pounds. Now to compare this machine versus a Sun Ultra 45, just on specs from each vendors web site lets see both maxed out:

All in all, you may end up paying a little more for a Ultra45, but then again, it is a way better machine than the pSeries 185. It also runs Solaris which I feel is far superior to AIX. AIX is cool and all, but has too many quirks that just does not make to much sense. There are things in Solaris that are done so much easier and faster than in AIX that just make me laugh when I have to answer how to do something in AIX vs Solaris.

Here are some of my pet quirks about AIX:

1. Disk numbering scheme: all disks in AIX are named in the form of hdisk#. To find out exactly where they are at you have to do either a “lsdev -Cc disk” or “lsattr -El hdisk#” to find the actual controller and slot it is connected to.
2. ODM: Seems too much like the windows registry to me. Screw it up, and your machine does not boot right
3. The “dumbing” of Sysadmins by their dependence on SMIT. Take a AIX admin and put them in front of Solaris/Linux/etc and have them try to do any administrative tasks, and it is a complete loss with out smit. But take a Solaris/Linux/etc admin and put them on AIX, and they can accomplish most of the same administrative tasks with out touching SMIT
4. NIM, Nim is AIX’s equivelant of Jumpstart on Solaris. Jumpstart can be setup in probably under 10 minutes and be booting and installing machines. Nim on the other hand is an all day affair. I kid you not, I spent 8+ hours one day configuring an NIM environment to boot 1 machine. And even then, it did not install all the needed software. It also takes forever to copy 8 CD’s of AIX install media, plus the “Linux ToolKit”, Plus the expansion pack just to get SSH installed on AIX when NIM is used to install a system. If I could only get AIX to boot from a jumpstart server I would be set.
5. Missing core software that should be installed no matter what type of install you do. For example SSH. What operating system besides Microsoft Windows now days does not come installed with SSH? AIX, yup, you have to have 3 different cd’s to install it, and you better be using the OpenSSH supplied by IBM, or they will refuse to talk to you about any problems. (Yes they actually had me verify every part of the version of SSH before they would talk to me.
6. Root allowed to log in remotely. By default when you install AIX, root can log in remotely. MMM Bad Mkay…..
7. Default Open Relay: Last time I checked Sendmail on AIX is still configured by default to be an open relay.
8. Syslog: AIX likes to put stuff in its proprietory errpt. Which means to get the information to log to a central syslog server, you have to modify the ODM to run a script to grab output from the errpt command to send to syslog. Why can’t it send to syslog by default?

And the list could go on for ever, but right now it is time to go to bed.