OpenSSL and Solaris 10

So if you are still running Solaris 10 and haven’t looked at the patches recently, Oracle bundled in OpenSSL 1.0.1 as a patch. While this was awesome to see an updated version, now that everyone should only be running TLSv1.2 on their websites, there are some issues. The main issue is that the GCC version that is supplied with Solaris 10 for what ever reason has /usr/sfw/include and /usr/sfw/lib hardcoded in to the gcc binary as the first Include and Library path. While I can understand why it was probably done to begin with, it makes compiling any software that needs an updated version of OpenSSL completely impossible.

For example say you wanted to compile a new *AMP stack, and wanted the latest version of Apache with SSL enabled. Well, it won’t work. It will by default use the old 0.9.7 OpenSSL libraries and include files. I spent days on this, even compiled my own version of OpenSSL and tried to link against it, still wouldn’t work and kept linking against the 0.9.7 ones..

So how to fix this? Well you could build your own version of GCC, which in and of itself is not an easy task. You also can’t remove the OpenSSL 0.9.7 libraries as there is a lot that depends on it. So if you read the patch notes for the new OpenSSL they give some “recipes” on ways to maybe possibly fix it during compile time. Which for me did not work. So what I ended up doing was this:

 

  1. In /usr/sfw/include I moved the openssl directory to .openssl. The new 1.0.1 includes are in /usr/include/openssl.
  2. In /usr/sfw/lib I removed the symlink from libssl.so and libcrypto.so. Also did this in /usr/sfw/lib/amd64. This allows those apps that are using the actual libssl.so.0.9.7 to still run, but compile time stuff can’t find them.

Once I did that, I went back to the apache directory and did the compile and yippie, it was against the /usr/lib/libcrypto.so.1.0.1 and /usr/lib/libssl.so.1.0.1, which also meant that I could limit Apache to only use TLSv1.2.

 

So If this helps just one person, great.. This was something that took me a few  weeks to figure out. I also should have noted that if I had fully read the readme in the patch that introduced OpenSSL it would have probably went a little faster… If you are worried about breaking stuff, once you get your compile done, you can put the symlinks back and move the openssl directory back to it’s original place.

N.B. I am not sure if the gcc in Solaris 11 has the same quirk.