The IBM POWER8 Review: Challenging the Intel Xeon
by Johan De Gelas on November 6, 2015 8:00 AM EST- Posted in
- IT Computing
- CPUs
- Enterprise
- Enterprise CPUs
- IBM
- POWER
- POWER8
Software Issues
So although the S824 is IBM's benchmark flagship for the scale-out range, the S812L and S822L are the servers that have the best chance at converting the kinds of users currently opt for x86 Xeons:
- Support for Little Endian data
- Best Linux support (Suse, Redhat & Ubuntu)
- (Somewhat) lower power
- 2U form factor which offers decent performance per U
- and probably the most important reason of all: Affordable! ($10k-25k instead of $30-60k)
So yes, the S822L looks like the first worthy alternative since 2010 for the dual Xeon servers. But the S822L did not inherit all the strong points of the typical "Big Blue" servers. The clockspeeds are a bit lower to keep the power consumption in check, and more importantly the LE Linux support is still very young. Sure, POWERLinux has been around for ages, but the software ecosystem was mostly supporting a few Big Endian applications like heavy duty Java servers and SAP.
Let's make the issue at hand a bit more tangible. IBM offers a migration advisor that helps developers to port their applications. That is definitely a good thing, but it also clearly illustrates that building a software ecosystem is a lot more cumbersome than the POWERPoint slides let you believe. In case of IBM's LE Linux, porting the rich x86 Linux software ecosystem to OpenPOWER is not that straightforward:
- Some code has inline x86 assembly such as thread resource locking code.
- Some code has x86 specific APIs
- No support for POWER in the make files which makes recompiling not straight forward
- POWER is 64 bit only.
We have experienced ourselves that this was more than just theory.
Case in point: for X86-64 we simply installed well tuned, ready to run, pre compiled binaries. Benchmarking is pretty easy here with a minor scripting effort.
The story was very different on the IBM S822L. We installed Ubuntu 15.04 (3.19.0-15 - ppc64le). To satisfy our curiosity we did a quick benchmark run with Linux-Bench, an automated benchmarking tool that Ian also likes to use. The benchmark did almost nothing on our POWER system despite the fact that most of the software had some form of support for POWER based systems.
The same was true for most software out there: We had to port most of the software by delving deep in all kinds of config, Readme, and make files. In many cases, we had to search around for alternative libraries that did support OpenPOWER.
Although a lot of software had an entry for "IBM POWER" in the make files, we encountered a lot of trouble. The server nor IBM is to blame: it is simply a fact that most developers - especially those with HPC software - have put a lot more effort in optimizing and validating their Intel x86 version of their software than the more "exotic" platforms.
Linux Ecosystem Not at Full Throttle.. Yet
It is clear to us that the OpenPOWER Linux ecosytem is still young and as a result does not offer the same performance as the older PowerVM and AIX platforms. There is still quite a bit of performance headroom.
A good example is the crypto acceleration. The IBM POWER8 has a dedicated cryptographic unit supporting new POWER ISA instructions to accelerate AES (Encryption), SHA (Hashing), and CRC (Cyclic Redundancy Check) codes. A similar encryption unit was already available in the POWER7+ . We found out that an nx-crypto driver was available and part of the Linux 3.5 kernel. However, even though Ubuntu 15.04 LE for OpenPOWER is based upon the Linux kernel 3.19, the nx-crypto driver was nowhere to be found. You could argue that the same is true for Intel as they introduce new instructions, but as far as we could see, there was no encryption acceleration whatsoever possible, not even based upon the older POWER7+.
A few days after we have finished testing, we found out the vmx-crypto driver will be available in distributions using the Kernel 4.1 and later and will be enabled in OpenSSL 1.0.2 (currently 1.0.1f in the standard repositories). The slide below - found in a presentation given this month - show how fast the ecosystem is expanding but also that it is still in flux.
OpenPOWER gained traction in 2014, the POWER8 is the first POWER chip with LE support and the number of Linux servers on top of OpenPOWER systems is still very small compared to x86. It is pretty simple: it is a much smaller community than the x86 linux server community. According to "the platform", IBM claims that "scale-out POWER8 machines have seen double digit revenue growth in the first half of 2015" but those growth numbers are "against a very small base". That tells us a lot: it is indeed a very small community, but a quickly growing one.
Facebook
Twitter
RSS
146 Comments
View All Comments
JohanAnandtech - Wednesday, November 11, 2015 - link
I did not know you used 7500x3500, my testing was inspired on what the rest of the benchmarking community (Phoronix, Serverthehome) was using (obviously, 1024x768 is too small for current servers).http://www.anandtech.com/show/9567/the-power-8-rev...
This answers your question about threads, right?
JohanAnandtech - Wednesday, November 11, 2015 - link
Oh yes, changed the link. Thanks for the feedback!mapesdhs - Thursday, November 12, 2015 - link
Most welcome! And I really should move the more complex tests to the top of the page...Oh, my wording about threads was not what I'd intended. What I meant was, the no. of threads being larger than the supported no. of hardware threads. Thus, for a 12-core Power8 with 8 threads per core, try using 192 or 384 threads, instead of just the nominal 96 one might assume would make sense.
Ian.
MB13 - Wednesday, November 11, 2015 - link
POWER8 is full of innovation and brings change! An S812LC only costs $6,595 from IBM's external website! http://www-03.ibm.com/systems/power/hardware/s812l...The Power scale out boxes will save on your running and software costs as you can reduce your software licensing and server footprint.
With the OpenPOWER Foundation, you now have companies such as Tyan and Wistron who also create their own POWER8 servers and sell them independently of IBM. If you have not looked at The OpenPOWER Foundation and the innovation it brings through community and collaboration, your missing out big time!
There is change! Don't get left behind!
MB13 - Wednesday, November 11, 2015 - link
and don't forget - POWER8 runs Little Endian and support the latest versions of RedHat, SUSE and Ubuntu!The OpenPOWER servers are Linux only!
Gasaraki88 - Wednesday, November 11, 2015 - link
It's funny how this article is trying to "sell" me the system but I'm still not impressed. Costs more, less performance, and uses more power at idle and load than the Intel system.nils_ - Thursday, November 12, 2015 - link
What I found the most off putting is that you have to do a lot of work to get some things running with Linux. That's a big cost factor.nils_ - Thursday, November 12, 2015 - link
Having a lot of software that isn't really well ported is probably going to remain a problem for Power8 for years to come since so few people have access to these kinds of systems and the cost is prohibitive. The great thing with x86 and ARM is that you can use it at home/work pretty easily without shelling out a lot of money. On x86 you can be sure if your software builds locally and runs locally it will also run on your server.svj - Thursday, November 12, 2015 - link
Well written articles.1. I submit that the headline is misleading. Intel x86 does not compete with POWER at the high end. POWER L & LC line of servers are comparable to x86 based servers. IBM POWER is taking the battle to Intel's home turf.
2. The analysis leaves out cost of SW. Many organizations use commercial software which are priced per core. If POWER can do with 10 cores what Intel does with 18 cores, that means HUGE savings.
3. OPEN POWER is a huge move. I think the market will start seeing the results soon.
alpha754293 - Thursday, November 12, 2015 - link
An excellent review as always Johan. (haha...to zeeBomb. It is my understanding that Johan doesn't post probably as often as he might have otherwise like to because testing servers/enterprise computing solutions takes a LOT longer than testing/benching consumer-level systems. Some of the HPC applications that I run takes hours to days for each run, so when you're running it, you're running those tests over and over again, and before you know it, a month has gone by (or you've ran out of time with the system) or you have to purposely cut it short so that you can test a variety of software.)It's unfortunate that IBM never ported AIX to x86 (unlike Solaris.) I think that there would be more people trying to get into it if the cost of entry (even just to learn) isn't so high. I've looked at getting an old POWER4 system before for that purpose, but by then, the systems are so old and slow that it's like "what's the point?" I think that IBM is literally pricing themselves into extinction (along with their entire hardware/software ecosystem). Unfortunately for many businesses, AIX POWER servers still run their mainframe/backend which means that if you want to get paid $100k+ outta college - go learn AIX on POWER. As the current generation of sysadmins are starting to age and retire out, and they're going to have a hard time finding qualified people, the only way eventually would be that they would have to pay top dollar just to attract people into the field. (Unless they decide to move everything over to the x86/Xeon/Linux world. But for some mainframes (like financial institutions), that's DEFINITELY easier said than don).