Need greater than 2GB memory allocation to a program in XP 32-bit

[2Cold Scorpio]

I've come across an interesting issue. Recently, I've been trying to play FakeFactory's Cinimatic Mod 6 for Half-Life 2 on my computer under Windows XP pro 32-bit. The problem is its intended for 64-bit systems because it uses binaries that make the mod utilize greater than 2GBs of RAM. Since I'm in 32-bit, it seems I can't do this. I played the early parts of the mod fine, with all the settings at the highest, but I eventually got to a point that it always crashes (likely due to insufficient allocated memory, seeing as the rest of my system, hardware-wise, has plenty of power to run the mod. I can avoid the problem if I turn down the graphics specs, but this is something I'm trying to avoid.

So my question is...is there a method to get XP to allocate more than 2 GBs of memory to one application (I have 4 GBs of memory, and even with 32-bit XP's limitations of 4 GBs total memory, I should still have plenty of RAM to run the mod. If you want my exact specs, I have it listed to the left.

If this cannot be done and I must use a 64-bit OS to properly run this mod, I have another question: I *do* have Vista Ultimate 32-bit, but not a 64-bit disk (I bought it from my college with student pricing, and its an upgrade disk, however since I run XP anyways, this is acceptable). My question is how can I get a 64-bit disk? I plan on completely replacing my Vista 32-bit installation with a 64-bit one (if needed, as in the above question about XP is impossible). In addition to obtaining a 64-bit disk, could I use my same CD key (seeing as I have Ultimate anyways, and am simply replacing my install with a 64-bit version)? I've tried obtaining one via Windows Marketplace (using the method for getting a 64-bit disk for versions of Vista other than Ultiamte, but since my product key is for Ultimate, it keeps saying I already have the disk, which I don't (I only recieved one DVD for the 32-bit version).

Any help would be greatly appriciated; I'd prefer being able to modify XP to allocate more memory to HL2, but if its impossible, I need help with my Vista 64 questions. Thanks in advance!

[directhex]

a 32-bit application must be compiled to support memory spaces over 2GiB, and it comes with a performance hit. HL2 is not compiled with this.

[2Cold Scorpio]

Normally, HL2 doesn't, true. But this mod does, as it uses custom binaries (even overriding those that Steam tries to use). Here's a few snippits of the ReadMe:

32 BIT WARNING:

This mod was developed and tested on 64 Bit Windows. It is NOT supported on
32 Bit Windows, because of the limited Userspace with 32 Bit systems. You
can try it on 32 Bit, but prepare for crashes.

On some 32 Bit systems it is possible to overcome this limitation with some
boot.ini switches for raising the userspace limit. I'm not going into
detail here, because those tweaks are dangerous and should only be done by
people, who are knowing exactly, what they do.

However, many, many users do use this in 32-bit OSes, using either lower detail settings, the above-mentioned boot.ini tweaks (which I'm *very* interested in), or with no changes at all. This disclaimer is primarily due, other than the limited userspace, because the author created it in a 64-bit OS and even admitted he did not test it in 32-bit.

Here's the interesting stuff:

TECHNICAL INFO:

This mod's binaries are modified for memory allocation beyond 2 GB. To
prevent STEAM from overwriting this binaries with its own copies on each start,
the mod ist using a little launcher, that reroutes STEAM's original binaries
to >NIL.

Start this mod only with the shortcuts in your start menu (that were
created during installation)!


HARDWARE REQUIREMENTS:

- DX9 capable graphics hardware with minimum 256 MB VRAM. (512 MB recommended)
- 2.5 GB RAM (4 GB recommended)


SOFTWARE REQUIREMENTS:

-Fully installed STEAM Client; You can't start the mod without STEAM running.
-Fully installed and unlocked HL2 / EP1 / EP2; Make sure, you have started those games in the past.

My question is does anyone know anything about these boot-ini changes? If not, or if its not what I need, can someone answer my Vista questions stated in the second half of my first post? Thanks

[2Cold Scorpio]

Ok, I found this, which is related to the boot.ini tweaks I mentioned before:

How to use the /userva switch with the /3GB switch to tune the User-mode space to a value between 2 GB and 3 GB

My question is...does this work in Windows XP SP2?

[Paul Adams]

This KB article would imply you can:
Windows XP SP1 May Not Start with the /3GB or /USERVA Switch

However, rather than edit the existing line in BOOT.INI it may be safer to copy & paste it then add /3GB to the second entry and change the description to reflect which is which.

e.g.
If you start with this:

...
[operating systems]
multi(0)disk(0)rdisk(0)partition(1)\WINDOWS="Windows XP" /fastdetect

then you can edit it to make it look like this:

...
[operating systems]
multi(0)disk(0)rdisk(0)partition(1)\WINDOWS="Windows XP" /fastdetect
multi(0)disk(0)rdisk(0)partition(1)\WINDOWS="Windows XP 3GB-enabled" /fastdetect /3GB

in case of problems with /3GB not leaving enough PTEs for the kernel, you can try a third boot option which disables PAE (by turning off DEP):

...
[operating systems]
multi(0)disk(0)rdisk(0)partition(1)\WINDOWS="Windows XP" /fastdetect
multi(0)disk(0)rdisk(0)partition(1)\WINDOWS="Windows XP 3GB-enabled" /fastdetect /3GB
multi(0)disk(0)rdisk(0)partition(1)\WINDOWS="Windows XP 3GB-enabled PAE-disabled" /fastdetect /3GB /noexecute=alwaysoff

Should you have problems booting, or shortly afterwards, then you can use the top menu option (which should default after 30 seconds anyway) to boot normally.

Taking all the above into consideration, bear in mind that the OS is affected by this boot switch, and you are halving the amount of virtual address space for the kernel by implementing it - but other than maybe making this 1 application "work better" you won't gain anything else from it.

[2Cold Scorpio]

Ok, I've tried it, and so far that mod hasn't crashed (nor has XP seemed to suffer any problems). However, I'm still trying to get the multi-option boots for XP you listed (as I had the same idea), but editing boot.ini doesn't cause these options to appear. The problem, it seems, it because Vista is installed as well, and therefor I'm using it's bootloader. When I boot up, even with two versions of XP set inside of boot.ini, the Vista bootloader only shows the original entry (vanilla XP Pro SP2). How do I get these choices to appear?

Also, if i may ask, my default XP entry already had a form of /noexecute entered (I believe its something from SP2). The default before I modified it, looks like this:

multi(0)disk(0)rdisk(0)partition(1)\WINDOWS="Microsoft Windows XP Professional" /NOEXECUTE=OPTIN /FASTDETECT

Does that mean DEP is off already (and what is DEP and PAE anyways?)?

So far, I'm running with the 3GB-enabled with no problems, but i haven't tried any OS-intensive tasks yet, either, hence why I'd like the choice between the tweaked version and the vanilla version of my install (because I have other apps that could benefit, at least a little, from this...like Crysis ).

Currently, my boot.ini looks like this:

;
;Warning: Boot.ini is used on Windows XP and earlier operating systems.
;Warning: Use BCDEDIT.exe to modify Windows Vista boot options.
;
[boot loader]
timeout=30
default=multi(0)disk(0)rdisk(0)partition(1)\WINDOWS
[operating systems]
multi(0)disk(0)rdisk(0)partition(1)\WINDOWS="Microsoft Windows XP Professional - Tweaked" /NOEXECUTE=OPTIN /FASTDETECT /3GB /Userva=2800

When I boot up my computer, I get the choice of:
Windows XP Professional
Windows Vista Ultimate

However, as I mentioned before, additional entries into my boot.ini file fail to show; Vista's bootloader still just shows XP (normally the original vanilla version, but I'm currently running it modified) and Vista. Choosing XP doesn't give any additional options for other entries in boot.ini (when they were there; I took them back out in the meantime). What do I do?

Luckily, I am working with a safety net...if XP fails to boot or load at some point, I can still boot Vista to edit the boot.ini file.

[Paul Adams]

When I boot up, even with two versions of XP set inside of boot.ini, the Vista bootloader only shows the original entry (vanilla XP Pro SP2). How do I get these choices to appear?

Probably more effort than it's worth - check in XP though that the option to "show boot options for N seconds" is ticked, if you edited BOOT.INI and it took effect then it's not Vista's boot loader ignoring it.

Also, if i may ask, my default XP entry already had a form of /noexecute entered (I believe its something from SP2).

Does that mean DEP is off already (and what is DEP and PAE anyways?)?

No:
AlwaysOff = Disabled
OptIn = Enabled for Windows processes only
OptOut = Enabled for ALL processes

DEP = Data Execution Prevention, a security method to prevent data sections of memory having code inserted into them and executed (a la "buffer overflow" or "injection" techniques).

PAE = Physical Address Extension, memory addresses use 36 instead of 32 bits to allow addressing of physical memory above 4GB (also permitting the "gap" between 3.25GB and 4GB to be viewable by 32-bit OS's).

DEP requires PAE, so if the first is enabled then so is the second.
36-bit addressing increases the size of each PTE, so you get less of them.
/3GB reduces the kernel resources by roughly half, so you get less PTEs.
Hence, using PAE and /3GB is the worst combination in terms of system PTEs - run out of those and your system cannot describe all of physical memory even if it is "free".

[2Cold Scorpio]

THanks, I understand now. Things are running smooth so far, but I did change the /noexecute to AlwaysOff, just to help avoid any potential problems.

The Vista bootloader actually has never given me any special options for the XP partition for soem reason (example: no "Press F8..." to give a choice for Safe Mode, etc., but *does* for Vista), and the options in XP are checked to show support boot options for 30 seconds, so I don't know what the problem is. I've given up on multiple entries; it seems redundant anyways seeing as things seem to be running fine and its simple enough to boot into Vista and edit boot.ini if needed.

However, one question remains: as you can see above, I'm using /Userva=2800 in conjunction with /3GB; is this a good idea, and is that the best number to use or should I set it higher (I have 4 GBs of RAM, and my video card RAM is 512MBs; you could get an idea of how much 'usable RAM' I have as far as XP is concerned...)?

[Paul Adams]

However, one question remains: as you can see above, I'm using /Userva=2800 in conjunction with /3GB; is this a good idea, and is that the best number to use or should I set it higher (I have 4 GBs of RAM, and my video card RAM is 512MBs; you could get an idea of how much 'usable RAM' I have as far as XP is concerned...)?

/USERVA is a tweak specifically for /3GB for fine-tuning.

/3GB by itself gives 1024KB to the kernel and 3072KB to user-mode processes

/USERVA=xxxx as an additional switch indicates the more fine-grained value for user-mode processes, and 4096-xxxx is what the kernel now has

So by using a value lower than 3072 you are giving resources back to the kernel which you previously stole - unless your user-mode applications are running out of virtual address space (still) then the lower the better for xxxx.

[2Cold Scorpio]

Thanks for all your help. So far, so good with 2800 (I had read in that MS that 2800 is good for most cases, and 2900 tends to eliminate most, if not all, problems). However, for this particular case, where I was really just needing about 2.5 GBs or so for the one app, 2800 seems to be working fine. The limited time I've spent with the mod so far this weekend (I've been moving...we all know how fun *that* is! ) was crash-free, even with all settings turned back up to Max. And I can't think of a scenario where the OS would need that memory back (seriously, shouldn't the XP kernal run fine even with just one gig allocated to it? I used to run it with just 256 MBs or RAM!). At any rate, using my backup copy of boot.ini, its fairly simple to turn the tweak on and off as needed...

Thanks again for all your help, sir. I hope somebody else can find some use in this (perhaps some hellow HL2 player who wants their game to look much better but isn't running a 64-bit OS... ).

[Paul Adams]

And I can't think of a scenario where the OS would need that memory back (seriously, shouldn't the XP kernal run fine even with just one gig allocated to it? I used to run it with just 256 MBs or RAM!).

Windows uses a virtual memory model, so the 32-bit version has 4GB (2^32) of virtual address space, regardless of physical memory installed.
Windows XP x86 with 256MB installed still has 2GB of virtual memory for the kernel (without /3GB used).

It's actually not so much the kernel needs all of 2GB, but the system resources which are a proportional subset of that space - it's those that can get exhausted and cause issues.

[2Cold Scorpio]

Ah, I understand now. Well, I'll likely only keep the tweak enabled for the duration of the time I'm playing around with the Cinimatic Mod. I've got backup copies of both the modified and unmodified boot.ini, so its easy enough to change (a simple copy/paste and reboot. ). You, sir, have been a great help, thank you for yoru time and patience.