Thermal Management on Stacked Silicon

With a standard processor design, there is a single piece of silicon doing all the work and generating the heat – it’s bonded to the package (which doesn’t do any work) and then depending on the implementation, there’s some adhesive to either a cooler or a headspreader then a cooler. When moving to a stacked chiplet design, it gets a bit more complicated.

Having two bits of silicon that ‘do work’, even if one is the heavy compute die and the other is an active interposer taking care of USB and audio and things, does mean that there’s a thermal gradient between the silicon, and depending on the bonding, potential for thermal hotspots and build-up. Lakefield makes it even more complex, by having an additional DRAM package placed on top but not directly bonded.

We can take each of these issues independently. For the case of die-on-die interaction, there is a lot of research going into this area. Discussions and development about fluidic channels between two hot silicon dies have been going on for a decade or longer in academia, and Intel has mentioned it a number of times, especially when relating to a potential solution of its new die-to-die stacking technology.

They key here is hot dies, with thermal hotspots. As with a standard silicon design, ideally it is best to keep two high-powered areas separate, as it gives a number of benefits with power delivery, cooling, and signal integrity. With a stacked die, it is best to not have hotspots directly on top of each other, for similar reasons. Despite Intel using its leading edge 10+ process node for the compute die, the base die is using 22FFL, which is Intel’s low power implementation of its 14nm process. Not only that, but the base die is only dealing with IO, such as USB and PCIe 3.0, which is essentially fixed bandwidth and energy costs. What we have here is a high-powered die on top of a low powered die, and as such thermal issues between the two silicon die, especially in a low TDP device like Lakefield (7W TDP), are not an issue.

What is an issue is how the compute die gets rid of the heat. On the bottom it can do convection by being bonded to more silicon, but the top is ultimately blocked by that DRAM die. As you can see in the image above, there’s a big air gap between the two.

As part of the Lakefield design, Intel had to add in a number of design changes in order to make the thermals work. A lot of work can be done with the silicon design itself, such as matching up hotspots in the right area, using suitable thickness of metals in various layers, and rearranging the floorplan to reduce localized power density. Ultimately both increasing the thermal mass and the potential dissipation becomes high priorities.

Lakefield CPUs have a sustained power limit of 7 watts – this is defined in the specifications. Intel also has another limit, known as the turbo power limit. At Intel’s Architecture Day, the company stated that the turbo power limit was 27 watts, however in the recent product briefing, we were told is set at 9.5 W. Historically Intel will let its OEM partners (Samsung, Lenovo, Microsoft) choose its own values for these based on how well the design implements its cooling – passive vs active and heatsink mass and things like this. Intel also has another factor of turbo time, essentially a measure of how long the turbo power can be sustained for.

When we initially asked Intel for this value, they refused to tell us, stating that it is proprietary information. After I asked again after a group call on the product, I got the same answer, despite the fact that I informed the Lakefield team that Intel has historically given this information out. Later on, I found out through my European peers that in a separate briefing, they gave the value of 28 seconds, to which Intel emailed me this several hours afterwards. This value can also be set by OEMs.

Then I subsequently found one of Intel’s ISSCC slides.

This slide shows that a basic implementation would only allow sustained power for 2.5 seconds. Adding in an adhesive between the top die and the DRAM moves up to 12.4 seconds, and then improving the system cooling goes up to 20 seconds. The rest of the improvements work below the compute die: a sizeable improvement comes from increasing the die-to-die metal density, and then an optimized power floor plan which in total gives sustained power support for 150+ seconds.

Lakefield: Top Die to Bottom Die Hybrid CPUs: Sunny Cove and Tremont
POST A COMMENT

224 Comments

View All Comments

  • ProDigit - Friday, July 3, 2020 - link

    It actually starts making sense, once you start doubling up on the cores. Sure, dual or quad core atom processors aren't really a thing. But at 95W tdp, you could be running a 30 to 60 core cpu. And those numbers are for 14nm. Not 10.
    That could make it interesting!
    Reply
  • Alexvrb - Saturday, July 4, 2020 - link

    Heck toss in 200W of these and it can be used to generate polygons and stuff! Oh wait... they already tried that. Seriously, a pile of slower cores might be OK for a secondary chip (accelerator) but it's not ideal for a main CPU and even less so for a consumer use case. Even a fast quad core would beat the stuffing out of a 60 core Atom for the overwhelming majority of consumer workloads.

    Actually, even as an accelerator for professional use there are often better solutions - GPUs and/or purpose-built accelerators, depending on your workload. That's why Intel shifted gears in that realm too.
    Reply
  • LiKenun - Thursday, July 2, 2020 - link

    The way I understand the implications for programmers… sometimes a program will do a one-time check for a particular processor feature (e.g.: if Avx2.IsSupported == true) and load optimized code at startup or optimize byte code compilation to remove unused branches. Then the program uses the loaded implementations for its entire running lifetime. There’s going to be a lot of work needed to undo these assumptions about processor feature sets not changing while the program is running. Reply
  • Lucky Stripes 99 - Saturday, July 4, 2020 - link

    It would really be nice if Windows became better about ISA and API version controls. Instead of having to do a bunch of run time checks to avoid arcane system errors, I'd like to be able to set some minimum versions in the header of the EXE so that things will gracefully fail at startup. From a scheduler standpoint, this could allow you to have cores with different ISA versions and it would know which cores to avoid. Reply
  • ProDigit - Friday, July 3, 2020 - link

    Yeah? Tell me where you can actually use avx512 or avx2? Most home users don't need it. Intel makes chips for businesses and for most home users.
    Not for those that occasionally need to run programs that are made for servers or performance machines.
    Avx2/512 makes no sense, and has no home in a laptop.
    Reply
  • Cullinaire - Friday, July 3, 2020 - link

    Don't mind him, he's always harping about avx512 every chance he gets even though it's pointless. Reply
  • lefty2 - Friday, July 3, 2020 - link

    Yes, home users do need it. AVX512 is rarely used, but AVX2 is almost universal. Reply
  • eastcoast_pete - Friday, July 3, 2020 - link

    So, I guess you don't use Microsoft Teams or other video conferencing software on your laptop? Because those use AVX or AVX2 for virtual backgrounds, amongst other features.
    Regarding "no place in laptops", I vaguely remember hearing that about SSEs way back.
    Reply
  • dotjaz - Saturday, July 4, 2020 - link

    Well, AVX512 is truly pointless on a laptop, and possibly on any general consumer parts. That much is true. It is not energy efficient at all. On top of that, there's the mess of subsets.

    But AVX2 does provide sizable benefits over SSE4 even for optimal code. AVX alone is probably not worth it.
    Reply
  • eastcoast_pete - Sunday, July 5, 2020 - link

    With regard to AVX512, it's also a chicken-or-egg issue; as long as software makers can (correctly) assume that most of their customers don't have CPUs that have it, they won't use it, even where it would speed things up over AVX2. That's why AMD starting to have their own implementation of AVX512 is so important; it'll make it more of a mainstream feature that programmers can assume to be available for use by their software. That's one of the reasons this boneheaded move by Intel ticks me off. Reply

Log in

Don't have an account? Sign up now