The Intel Haswell-E X99 Motherboard Roundup with ASUS, GIGABYTE, ASRock and MSI

The launch of Haswell-E ushered in a triumvirate of new technology – a new CPU line, a new motherboard chipset and DDR4 memory. Today we focus on the new consumer motherboard chipset, X99, with motherboards from all four major manufacturers: the ASUS X99-Deluxe, the GIGABYTE X99-UD7 WiFi, the ASRock X99 WS and the MSI X99S SLI Plus. X99 represents the upgrade over the previous extreme chipset generation, X79, in several key areas in order to align itself better with the mainstream Z97 and Z87 platforms.

The X99 Chipset

Intel’s processor strategy at the high performance end of the spectrum ensures that the same line of motherboards with the same CPU socket and chipset are kept for two generations of CPUs. With the Nehalem and Westmere CPUs there was socket LGA1366 and the X58 chipset, and with Sandy Bridge-E/Ivy Bridge-E CPUs we had LGA2011-0 and the X79 chipset. This new line of Haswell-E processors (along with the future Broadwell-E ones) will use the LGA2011-3 socket and the X99 chipset. This means that users investing in the new Haswell-E processors must also invest in a new motherboard. This has several upsides in the levels of functionality.

For the first half of 2014, the zeitgeist of the X79 platform was centered on its age and the comparison to the newer Z87 and Z97 platforms aimed at the mainstream processor lines. The Z87 and Z97 chipsets afforded more native SATA 6 Gbps ports, more native USB 3.0 ports, PCIe based storage in the form of SATA Express and M.2 and more flexibility from the point of view of the manufacturers – the chipset had a certain amount of configurable input/output from the chipset to the motherboards could be designed with specific users in mind.

With X99, we get some significant upgrades all around. The chipset now supports ten SATA 6 Gbps ports natively, with six of those being RAID capable (more on this later). There is also up to six USB 3.0 ports native, direct Thunderbolt support and more possible PCIe configurations. The new PCIe storage options (SATA Express and M.2 via PCIe) are also both supported. The platform requires DDR4 compared to the older DDR3, but still retains forty PCIe 3.0 lanes from the CPU and eight PCIe 2.0 lanes from the chipset (in 8x1, 4x2 or 2x4 configurations).

PCIe Storage and Lane Allocation

The inclusion of PCIe storage makes more sense for a platform like X99 as there seems to be PCIe bandwidth to spare. For each SATA Express connection, two PCIe lanes are required. For M.2 in PCIe mode, motherboard manufacturers can use two lanes or four lanes depending on the bandwidth they wish to support. With 40 lanes of CPU bandwidth, manufacturers need not use a PCIe 2.0 x4 slot from the chipset for three-way Crossfire support, giving extra free lanes to the chipset for these new storage technologies. Alternatively, some manufacturers are taking advantage of the CPU lanes and using four of these for an M.2 slot, as shown in this MSI chipset diagram:

Here, four way GPU support is given by x8/x8/x16/x8, as it will be with most X99 motherboards, but the fourth PCIe slot can switch between a PCIe 3.0 x8 slot or a PCIe 3.0 x4 M.2 slot. This means that if the M.2 slot is used in PCIe 3.0 x4 mode, only 3x graphics can be used. But, to double up some potential confusion, the system can use M.2 slot in PCIe 2.0 x2 mode from the chipset, allowing four-way GPU configurations again.

This may seem complicated, but at the end of the day the extreme enthusiast can have either:

- 3x GPUs + PCIe 3.0 x4 M.2, or
- 4x GPUs + PCIe 2.0 x2 M.2.

This is such an excruciating decision to make (-sarcasm!). Joking aside, it is a clever solution to the PCIe storage issue if these drives ever get to market in significant number.

With regards to the SATA port situation, Intel is enabling ten SATA ports total but only six of them for RAID. While this is an update over X79, it seems more of a fudge. One would assume that it could be a bandwidth issue, because the link between the CPU and chipset (PCH) is at 20 Gbit/s, or 2.5 GBps which would be saturated by a 5-6 disk RAID-0 array. However, it seems silly to not have RAID on four of those ports. Due to Intel’s previous flex-IO arrangement on Z97, this seems more of an integrated SATA 6 Gbps hub, splitting the bandwidth into four ports. Due to the hub it would limit throughput in RAID so it makes sense to disable it completely for those ports. Intel is telling us that this is due to two separate AHCI controllers in the chipset, with only one of the AHCI controllers enabled for RAID.

For motherboard manufacturers, this gives several options. If the chipset is using Flex-IO which we believe it is (we are still awaiting confirmation), then the amount of SATA 6 Gbps/USB 3.0/PCIe 2.0 lanes is slightly flexible similar to Z97. By providing 10 possible SATA 6 Gbps ports (6+4 of RAID/non-RAID), the motherboard manufacturer could implement a 4+4 arrangement to give more PCIe lanes, or shift around the PCIe lanes in that 4+4 to give a full six USB 3.0 ports. The truth of the matter is that there are very few users who require a six-drive RAID, and so motherboard manufacturers can target different orientations of motherboards for different user segments. The non-RAID possible ports are still fine for optical storage or hard drives with a software RAID-1 applied over the top.

The PCIe Arrangement – 40 or 28 PCIe lanes?

In Intel’s chipset platform overview above, it states that for up to 40 lane CPUs, multi-card configurations are available as 2x16 + 1x8 or 5x8 configurations. There are two interesting points to note here, and the first is the implementation of 5x8.

In the bottom of this slide it shows the following:

In the MSI chipset diagram above, we see that in order to get x8/x8/x16/x8, an additional switch is used such that one of the x16 can be split into x8/x8. Although GIGABYTE claim that it is not that simple, and without an external clock generator (as mentioned in the image above) motherboards may be limited to x8/x8/x8/x8:

Given the pricing of X99 motherboards ($240-$500), one would hope that all manufacturers who are aiming at a four-way GPU design specifically for gaming or PCIe limited compute would implement the single full x16 alongside the trio of x8 slots for quad-PCIe devices.

The second issue to arise from PCIe lane allocation derives from the CPUs themselves. While some X99 motherboards will support Xeons, all the LGA2011-3 oriented CPUs will feature 40 PCIe 3.0 lanes except the Core i7-5820K, which will only support 28. As a result, instead of x16/x16 gaming for two-way GPUs, 5820K users will have x16/x8. As we tested in our CPU review of the Haswell-E consumer processors, for two-way SLI gaming at 1080p, this has almost zero effect on actual frame rates. The 5820K is also limited in three-way GPU gaming, offering x8/x8/x8 rather than x16/x16/x8, and cannot be used for four-way SLI gaming without the addition of a PLX chip. On the motherboard side, this means that manufacturers have to cater for both 28 PCIe lane processors and 40 PCIe lane ones as well.

Normally, the 40 PCIe lane processors will use the PCIe slots in typical fashion, however users are recommended to look into the manual for three-way implementation. For example, the ASRock X99 WS in this review gives the following:

This gives both orientations the same sequence of adding in graphics cards, but users should look in the manual to find out the best way. For our X99 reviews, we will try to aid this in our board specifications tables. As a feature, ASUS has added a series of LEDs connected to an ‘SLI/CFX Switch’ that light up the LEDs next to the PCIe slots to use for 2x and 3x GPU configurations:

The DDR4 Conundrum

As we will explain in our DDR4 coverage, this new platform is a boon for DRAM manufacturers and resellers. Each new computer needs a new memory kit and cannot simply recycle the old DDR3 into a new build.

Almost all motherboards will support either one or two DIMMs per channel, giving a maximum of 64GB of DRAM using 8 GB modules – this does not change anything from X79 then, except the DDR4 itself being lower voltage and different sub-timings. X99 also puts in extra provisions for internal testing – on the first POST with new memory detected, or a change in the DRAM speed, the system will spend a few seconds stress testing the memory in order to find if the settings are out of step. The only potential spanner in the works regarding compatibility is the timing of the launch.

The original date for the release of Haswell-E and X99 was during IDF, on the 9-11^th of September. Everyone had their schedules locked in for those dates, but when Intel bought the release date to two weeks earlier than expected, panic (almost) ensued. The motherboard manufacturers were there or there about with their products, as typically the last few weeks are dedicated to compatibility testing with memory. However the memory manufacturers were still organizing their product lines and module categories.

As a result, the motherboard manufacturers were not as prepared as usual to qualify all the memory ready to go on sale (this has been fixed now with the latest BIOS updates) and memory manufacturers are still putting kits together. Combined with the high price of DDR4 and the initial limited quantity, prices for DDR4 were particularly high: $250 for 16GB as we noted at Haswell-E launch.

Memory manufacturers are still preparing kits, and motherboard manufacturers are continuously updating their qualified lists. With all the memory kits I have in to test, I have not had a motherboard not POST because of it, however in order to get the best performance out of that new high end memory kit (2666 MHz+), BIOS updates should be the number one priority. Most manufacturers have software that can do this (MSI’s Live Update 6) or users can download and update manually. Navigating to a BIOS update via the motherboard manufacturer website might show something like:

X99-DELUXE BIOS 0801: 3. Improve XMP settings
X99-UD7 WiFi BIOS F7b: 2. Improve DDR4 3000&2800MHz 4Gx4 compatibility

This happens throughout the lifetime of the product, and keeping up to date if you purchase a DDR4-3000 kit is vital.

I want to touch on voltages here as well, because although 1.2 volts is the standard for DDR4, there are two other voltage modes for memory on the platform:

Low voltage modules will be placed at 1.05 volts in order to draw less power, but might be limited on speed. High performance modules (typically DDR4-2666 or higher) will run at 1.35 volts, which is an identical swing in the other direction. 1.35 volts for a high end memory kit is reasonable and should not affect longevity of the platform.

The Motherboard Market

As it stands, almost all the motherboard manufacturers announced a number of X99 models, with about half ready to go on day one, and the rest to trickle through over the next few months. These will be the core motherboards through Haswell-E and Broadwell-E, although we might see a partial update nearer Broadwell-E launch.

The following have been announced or are on offer:

Prices range from $500+ (ASUS Rampage V Extreme) to $210 (ASRock X99 Extreme3) although we might see one or two higher than this (ASRock X99 Extreme11, maybe) depending on the final configurations. Note that this is cheaper than Nehalem with X58 when a motherboard reached $700 or with X79 when the Extreme11 was around $600.

I Saw a Thing On Burning Motherboards

Two review websites have reported their X99 motherboards failing since the launch.

Nathan at Legit Reviews was testing high performance memory above the recommended voltage range (1.50 volts) on an ASUS X99 Deluxe and upon changing memory kits it seems that the power delivery to the CPU failed. The power delivery is solely for the VCIN to the FIVR, and thus unconnected to the memory, and it would seem that a large amount of heat caused a burning smell and some solder to move on the motherboard.

Michael at Phoronix had built a system (5960X + MSI X99S SLI Plus) from the ground up and at the point where the machine was first turned on, the chipset failed with a spark and a second issue with the power delivery occurred.

There are a couple of issues worth mentioning here. Nathan at LR was using his review sample that came direct from ASUS, whereas Michael at Phoronix was using a purchased motherboard from Newegg. Both of these would have been from the initial batches of motherboards that come from their respective factories, and while failures are uncommon they do happen: perhaps more so with the first batch out of the factory.

Nathan’s sample seems to have passed through a lot of testing already, whereas Michael’s sample went kaput on turn-on, suggesting more of a failed component rather than anything systemic. It seems unfortunate for the platform that two review websites have both had failures in a short space of time, and substantial failures at that.

I have had failures in the past (Bluetooth adaptor shorting out, DRAM or PCIe slots not working, PSU going BANG… twice) however these tend to be the exception rather than the rule. As with any product, there is a failure rate. I would be more concerned with Nathan’s issue, although it may just have been a sample with a lower weak point than most. When Rajinder was testing motherboards at AnandTech under liquid nitrogen, it particularly showed some of these issues.

Both of these X99 failures are nothing compared to the P55 socket burn issue that occurred back in 2009.

One potential explanation afforded for the failures has been the relatively high current draw for Haswell-E CPUs. When overclocked these CPUs will draw 25 amps from the 8-pin CPU power connector, and thus a good power supply is recommended. Even though that represents an overclocked system, at stock the draw is still high which could lead to unexpected spikes. Nathan noted that the over-current protection (OCP) was being triggered during his failure, and other users are pointing to the single 8-pin CPU connector to blame, rather than having two.

We have both the ASUS X99 Deluxe and the MSI X99S SLI Plus in this review, from those initial batches that were manufactured, and have experienced none of the issues raised. Our ASUS X99 Deluxe has also been through some high speed memory testing as well as CPU overclocking and 160W Xeon testing without issue. Our one data point is invariably anecdotal as best, although if this is a more prominent issue I would assume we would have seen more failures in a consistent circumstance. If something does arise, we will post something about it.

This Review

Today we focus on some of the more exciting motherboards out in the market in the first of our X99 coverage articles. Due to the way the motherboard manufacturers were sampling for X99, we were unable to align several motherboards of a similar price. However one motherboard is a high end part, one is aimed at workstations and another strikes at a nice low price point to provide a contrast. The final one sits in the middle of the price bracket but also at the top of its product stack. We have in to test:

ASUS X99-Deluxe: $400
GIGABYTE X99-UD7 WiFi: $310
ASRock X99 WS: $324
MSI X99S SLI Plus: $229

The Deluxe and UD7 are typically the top of the range for the ‘channel’ line of motherboards (compared to the Gaming/OC ranges), and ASRock’s WS nomenclature is representative of the workstation-like attitude they have taken to this model. MSI’s name of ‘X99S’ indicates SATA Express connectivity, while the SLI Plus itself is oriented towards gamers but in a budget build.