The Benchmarks
This is where things get interesting. With not a single Linux based review at HEXUS so far, and the availability of standard Linux benchmarks about as common as a barbecue in Antarctica, we had to get creative. After failing to score Anandtech's SQL based benchmarks for obvious reasons (sending CD's of their entire database content would be a no-no if I was Anand, so we can understand that), I decided to plagiarise the idea of SQL tests and create my own. Here's how.I took a snapshot of my own online database that shows peoples PC configurations online. Being a small database, 4 tables for the data, a users table for account management and only 80 or so rows per table, one per user, it needed beefing up before it was able to tax the systems we were testing.
Since MySQL doesn't have any kind of transaction logging or tracing, I had to improvise. All my database interaction from the XMLRPC that feeds the database and the PHP that queries it, goes through a single database object written in PHP. Adding a simple function to dump whatever query was being executed (along with some timestamp data) to a file was trivial and I now had transaction logging.
Now the database needed beefing up. A simple PHP script that created a random user name using PHP's mt_rand function and a seed function was written to generate user names. Using those random usernames, new users were added to a new clone of the current database layout. The script seeded the new users with data from the initial 75 rows of the initial database snapshot.
The Athlon XP3200+ system used to create the new database was run with the max_execution_time and memory_limit parameters tweaked so the PHP instance could use as much memory and take as much time as needed.
With the initial database snapshot of 75 users fitting easily in a small amount of memory (it's a mere 140KB), seeding the new database from memory for 30,000 rows was easy enough, just time consuming.
With the 75 initial users having pretty much unique data, over a large number of recorded data fields, it was more than enough to keep the database random enough over 30,000 rows. Some simple checking on the first 1000 rows of all tables showed no identical rows. Good enough for our tests anyway.
The new database and PHP front end was then transplanted to the Opteron server and basically spammed to death using a simple request script running on my XP3200+ test box. Fed the new username list and asked to request as many pages from the PHP front end as possible, the transaction logging captured all the requests on the database to a file.
With each full PHP page pull doing 5 SELECT's on the database, a request of 20,000 pages from the 30,000 users left running overnight was enough to generate a massive trace file with 100,000 statements.
That's the benchmark. MySQL is then used to essentially playback that trace file as fast as it possibly can. Given its size, splitting it in to seperate files with 5 thousand SELECT chunks and playing them back sequentially gives load scaling at each 5K chunk of SELECT's. Using that, there are 20 seperate files from 5K, 10K, 15K....95K, 100K.
The HTTP request script also gave me a decent reproducible benchmark to run. Timing how long it took each computer to serve up 5,000, 10,000 and 20,000 dynamic PHP pages using the MySQL database gives a figure in served requests per second, the higher the better.
So a pair of custom benchmarks, one purely DB oriented, one DB + HTTP/PHP. The other benchmarks were a simple set of LMBench runs on each machine, giving some useful qualitative benchmarks on the underlying hardware.
It's not your usual HEXUS benchmark suite and I'd have loved to included runs from some more stuff, like contest, but the results obtained didn't fit with what was expected. It seems some tuning of x86-64 software is needed, as with all new architectures.
Here's some details on the test software setup.
Software Setup
[*] SuSE Linux Enterprise Server 8 (x86-46 and i686 versions)[*] MySQL 3.23.52 compiled for x86-64 and i686
[*] Apache 1.3.26 compiled for x86-64 and i686
[*] mod_php4 compiled for x86-64 and i686
[*] Intel Xeon kernel version - SuSE Linux 2.4.19-64GB-SMP
[*] AMD Opteron kernel version - SuSE Linux 2.4.19-NUMA
[*] Intel Xeon C compiler - gcc version 3.2
[*] AMD Opteron C compiler - gcc version 3.2.2 (SuSE Linux)
[*] LMBench 3.0
LMBench was compiled from source using the C compilers in the above list, the Linux kernels, MySQL, Apache and mod_php4 were all from SuSE .rpm's and precompiled to binary format. I didn't trust my own x86-64 builds of them, plus SuSE compiles them for you for a reason.
You'll notice that the Opteron kernel has NUMA support for efficient use of connected CPU memory resources in our dual CPU Opteron configuration. The Intel Xeon kernel is also compiled with its support for 36-bit PAE and PSE 'hacks', allowing processes and the CPU access to 64GB of memory. That may account for a small performance hit on machines with less than 4GB of installed memory, but not enough to put us off using that kernel.
Mismatched compilers isn't a cause for concern, the binaries produced for LMBench and nbench were essentially identical across the two compilers, as far as a cursory glance at the machine code for the main program loops was concerned. I can't read assembly language half as well as I'd like, but it seemed to be a fair fight, as you'd expect from a simple .2 revision difference. Binary compatibility is a big deal after all, between revisions of the same branch of gcc.
On to the graphs, finally!
