A Messy Transition: Practical Problems With 32bit Addressing In Windows
by Ryan Smith on July 12, 2007 12:00 PM EST- Posted in
- Software
Except in a few cases where 64-bit code is clearly faster, the primary purpose for Vista x64's existence is to resolve the problems of 32-bit addressing space, and we're just not at the point yet where even most enthusiasts are pushing that limit. Once applications begin to push the 2GB addressing space limitation of Win32 (something we expect to hit very soon with games) or total systems need more than 4GB of RAM, then Vista x64 in its current incarnation would be a good choice.
For some time now we have been mentioning the potential problems that are likely to result from the switchover from 32bit(x86) Windows to 64bit(x64) Windows. Due to a multitude of issues, including Windows' memory management, the basic design of the PC architecture, and consumer support issues, there is no easy path for mass migration from 32bit Windows to 64bit Windows. As a result we have been expecting problems as consumers begin to make the messy transition.
We published the above mentioned guide on February 1st, expecting the fall/winter 2007 games to be the ones to push the 2GB addressing space limitation of Windows, and it turns out we were wrong. It turns out that two weeks after we published the above article, THQ published Supreme Commander, a RTS with a massive appetite for resources. It can be simultaneously GPU limited and CPU limited, which is why it's a standard benchmark here for our performance articles, it's also memory limited in more than one way: it's hitting the 2GB barrier of 32bit Windows.
An artifact of the design of 32bit processors and the 32bit API for Windows, the 2GB barrier is a cap on how much addressing space (related to but not equivalent to memory usage) a single application can use. This isn't a bug but rather the result of how hardware and software was created so many years ago, and while everyone has known this barrier will inevitably be hit, as we'll see there are several reasons why it can't simply be moved or bypassed. Meanwhile hitting it involves affected applications crashing for what can appear to be no good reason, and understanding why the 2GB barrier exists and what can be done will be important for resolving those crashes.
On a personal note, I am a semi-casual player of real time strategy(RTS) games and I've been playing Supreme Commander lately. This is a different kind of article, it's a record and the result of my own efforts to resolve why I was having crashing issues with Supreme Commander. With no intended disrespect towards THQ or the game's developers (Gas Powered Games) we could have not possibly asked for a better example of the 2GB barrier in action. It's exactly the experience we believe many people will have as they hit the 2GB barrier, mainly those power users who use large monolithic applications such as games or multimedia tools. This is an article on what the problem with the 2GB barrier is, what kind of experiences a user may expect when hitting it, and what can be done to fix it.
But before we get too far ahead of ourselves, let's discuss memory management in Windows. Understanding the problem with Supreme Commander requires understanding what the 2GB barrier is, why it's there, and what makes it so problematic.
Facebook
Twitter
RSS
69 Comments
View All Comments
miahallen - Saturday, July 14, 2007 - link
What I found extremely interesting is that you had problems modifying your boot.ini file for a lsrger than 2.6GB app space. I have an almost identical configuration, and have modded almost all my games headers, and my boot.ini is set to 3GB app space. When I modded the boot.ini, I was unaware of the possible problems, but since it didn't cause any, I've been perfectly happy with it.A8N-SLI Delux
Opty 165 @ 2.5GHz
2GB DDR400
8800GTX (768MB VRAM)
Vista x86
I've modded the headers for:
STALKER
C&C3
CoH
Dark Messiah M&M
DiRT
FSX
Silent Hunter 4
TDU
Thanks Ryan, for the great article!
MadBoris - Sunday, July 15, 2007 - link
This should be made clear to people.
If an application is not Large Address Aware there is usually very good reason the developers did not include it that way. There maybe things you break in the game or cause stability problems by adding it.
So people should not be just adding this to all their games, especially considering many games don't come close to the 2GB ceiling in the first place, so their is no benefit only potential negatives.
ChristopherO - Monday, July 16, 2007 - link
This is true... One shouldn't go setting the binary flag because they *can*.I will however confirm that C&C3 and FSX run into the 2GB barrier. FSX is fine once it is patched.
However, C&C3 is so poorly written that it can easily pass 2GB and run into the revised 3GB barrier on any multiplayer map with the maximum number of AI and/or opponents. Detail settings as low as the "medium" preset do nothing to alleviate this problem.
Unfortunately C&C apparently has absolutely no code to purge and re-use memory as I've been in huge games, on the downward slope in terms of remaning opponents/units, and the app still hits a revised 3GB memory space and crashes.
As a developer myself, memory management is *not* that complicated. It takes some forethought, but the EA people apparently never bothered. This is a huge letdown since I've been a C&C fan since the first game came out my first year of college.
miahallen - Saturday, July 14, 2007 - link
I have to add one more thing about my system, may be important.....most of my games I play at 2048x1536, as opposed to you test rig at 1600x1200. That and you were using Vista Ultimate, I'm using Home Premium. Do you think this is what is effecting the results? If so, Home Premium (or even Home Basic) is the x86 OS of choice for gamers, not Ultimate!miahallen - Sunday, July 15, 2007 - link
I was hoping you would comment on this Ryan, do you think the difference in our boot.ini mod was due to the version of Vista we run? Any thoughts?sheh - Thursday, July 12, 2007 - link
...and disturbingly it doesn't set off any sort of multiplayer cheat detection in the game in spite of the fact that we have modified the executable in a very visible way.The game probably checks the code (and maybe data) section(s) in memory, and not the actual EXE file (makes sense considering you can use memory patchers). The header might not be important for cheat prevention.
MadBoris - Thursday, July 12, 2007 - link
Yeah, I remember thinking that the first day I did it, actually in those days it was Securom protected which I was actually more suprised about bypassing. But seriously the exe header should not be something that cheat protection should look for. I can only say glad it didn't or crashing would be a constant problem. ;)
atlr - Thursday, July 12, 2007 - link
I thoroughly enjoyed this article. Good job.atlr - Thursday, July 12, 2007 - link
A lot of good comments have helped edit and tune this article too. (meh, not this one though) Yayyy, community.JetBlack69 - Thursday, July 12, 2007 - link
From a programming perspective, what is a programmer to do? I assume this can happen in a program when "new" returns NULL because the program is out of memory, but what can a program do gracefully?I assume it could just display an error message, but if it were during a game, how could it handle it gracefully and not crash or give an error message? Would it lower the texture detail or remove unneeded objects on the screen?