So, why after nearly 23 years is Windows STILL rubbish at memory management?

[badass]

Have a read through this
Windows Vista with 8 GB : Vista Workshop – Performance Boost with 8GB of RAM
Yes, I know its Toms hardware and there is a fair bit of crap in there, but with 4 GB of RAM, I thought I'd do what they did with 8 and disable the page file. My system is definately more responsive with it off. 99.99% of the time, I have used nowhere near 4GB of memory, so why in gods name does this dumb OS feel the need to store ANYTHING AT ALL in the pagefile?
The pagefile should be used ONLY when the system memore is very low or has ran out. No need for it to be used any other time.
All I want is the security of having a pagefile enabled should I need it but windows Vista X64 to only use it when it needs to.
Is that really too much to ask of Microsoft?

[Lucio]

Ok, so MS doesn't always get things 100% but surely there's a reason for them to manage memory in the way that they do?

For example, the paging file could be used because you can't guarentee the RAM retains memory through various changes,

[badass]

Ok, so MS doesn't always get things 100% but surely there's a reason for them to manage memory in the way that they do?

For example, the paging file could be used because you can't guarentee the RAM retains memory through various changes,

Every one else seems perfectly capable of doing this for the past 20 odd years. What ever their reasons, they are not good ones.

[Fraz]

I can't think of any modern OSes that don't uses page files. The idea behind them is sound - maybe Microsoft's implementation is somewhat lacking.

[Bluecube]

Because Windows is a cr*p and clunky OS with far too much legacy code within it even when rewritten from the "ground up" like Vista was meant to be.

[TiG]

Its because its had to deal with all the legacy crap.

If you could make a brand new OS without the requirement to support everything that had gone before you would have a much better system.

Fraid its not ever going to happen tho.

Think of it as building a pyramid upside down, we're about 20 stories up now. We need to take it right back to the first floor and rebuild the ENTIRE thing from the ground up.

Vista was supposed to be that, but sadly it never happened, even after those 5+ years of development and a large amount of shelved code its now back to something much closer to a revision to windows than a replacement.

I think we will sadly see a more apple approach to OSs by microsoft soon tho, which i think will make things even worse.....

TiG

[[GSV]Trig]

I think we will sadly see a more apple approach to OSs by microsoft soon tho, which i think will make things even worse.....

TiG

Oh god no, can you imagine how even more intollerable and pig headed the Mac Brigade would become?

[TheAnimus]

you can't not constantly keep the pagefile uptodate.

What happens if you've got 4 gig of RAM and then suddenly need an extra 2gig, you would have to page 2 gig immedidately.

this is why you constantly page.

you can't have your cake and eat it too.

however flash SSDs might change this a lot, their random access nature, inherent tending towards high scaleability, really should shake things up.

Readyboost might be useless for desktops in its current shape, but with SSDs connected via the south bridge, or even sata.... its another issue.

I'd question what it is you think it should do?

if you can come up with a huristic, that ensures when an app is open, its not slowed by 'needless' paging, but that when you suddenly need another big block its not going to have too choose something to page. Also do all this without too much CPU time.

that would be a good PhD in my mind.

[peterb]

Just to get the terminology right - we are talking about the SWAP file - temporary storage on disk - rather than PAGING which is a memory translation technique which is a legacy requirement and part of the Intel architecture.

I have looked at this with some interest and I will state here that it is from a Linux perspective - but this is not intended to be a Linux v Windows debate,

However I have noticed improvements in memory useage with successive Linux core releases.

Using a 32 bit 2.4 Kernel on a system with 1G Memory and a 2G swapfile, I would regularly be using 65% to 75% of the available swapfile with about 85% to 90% RAM utilisation. Switching to the 2.6 Kernel saw a dramatic improvement in memory use efficiency - RAM useage at about 75% to 80%, but swapfile use rarely above 15% - so there was some work done by the Kernel developers to minimise swapfile use - and it shows!

If you are interested in the detail of how it happens, "Understanding the Linux Kernel" by Bovet and Cesati gives a very good description (I wish I understood it all!). I don't know if a similar description is available for Windows - given the closed source nature of the cde, I suspect not.

But basically, as The Animus says, if an application suddenly calls for a large amount of RAM, the system has three choices: crash, terminate processes that are using that memory, deny the request or park the inuse memory to the swap file and re-allocate it to the requesting process.

[dangel]

What's the fuss about? Windows gives you the option to turn off paging if you want to (and understand the risks). I can't see why it's 'rubbish' at memory management when all it's doing is providing a virtual memory fall-back when you run out of physical RAM. As TheAnimus says, what do you think it should do? Wave it's magical wand and predict you'll never (ever) use more than 8GB of memory (or use a 32bit process that wants more than 4gb)?

[TheAnimus]

Without dragging this thread off topic too much....

The company i work for atm prides itself on its developers, it often referes to itself as a "technology driven hedge fund" we are encoraged to debate the finer points of computer science. So the argument i was having (me, who'd of thought it) was over the OutOfMemoryException. In .Net when any memory allocation operation fails, this is what you get. A lot of 'Front Office' apps are really quickly nocked together code, often running on 32bit. As such you can easily see these if you mess up a little.

The question is what do you do, i was arguing that there is no 'right' thing to do in all cases, only that you musn't re-package the exception and rethrow it (as doing so would require more memory). He was arguing that you need too let things higher up the call stack have some way of logging it.

Whilst i know my statement is nasty, its the only practical way. (enless you have some kind of class designed to waste memory, so it can free it when you try to invoke it, purely to let it know your out of memory...... but that gets messy).

You might not like it, but enless you've got any better ideas, don't slag it off too much.

[Paul Adams]

TheAnimus has basically answered the question

Windows memory management is a balancing act - yes having everything loaded into physical RAM would be great for performance, but then if you were to request something else, what is the response from the OS?
1. Reject the request - "Sorry, no RAM. Unload something."
2. Page "something" out to make room

If you disable the pagefile you could encounter the first response, because you have just run out of virtual memory (which happens to equal physical memory).

If you go with the second choice, how to decide which "something" to page, and then you'll have 2 delays: first when the paging out occurs, and second when the requested load occurs


The memory manager pages out parts of the virtual address space for all processes which get "aged" and are not locked in physical memory as part of a regular housekeeping operation.
The principle being "if you've not used it in a while, I can make better use of that limited resource".

Before Vista, this turned into "available" memory.
From Vista onwards, this gets used by the system for cache - an attempt to predict what files you might want to load from disk and have them pre-cached.

The Vista method does not mean the system has to do more work if memory is needed - cache is volatile and is simply "dropped" if the space is required, so in essence it IS "available"

So Vista's memory manager is trying to give the best of both worlds - physical RAM as soon as it is needed, but in the meantime pre-cached files that you can access without disk I/O needed.

(People see Vista's "free" memory counter in Task Manager shrinking as time goes by when they are doing nothing, and think it is actually "gone" - you have to add free and cache together to get "available" physical memory.)

[peterb]

Thank you Paul - a briliantly clear explanation (as usual).

I think that is the principle used in the Linux Cores, but even then there has been an improvement in the algorithms used, and I'm sure that Windows will evolve too.

I would also guess that there are different optimum settings depending on the nature of the applications being run - so (say) a mailserver may benefit from different settings (such as page age) from those on a machine used as a file server. Again I think the Linux kernel can be tuned in that way - not sure about Windows. (And again to stress this is NOT a Linux is better than Windows comment, but to point out that different operating systems all face the same basic issues - such as the ones described by The Animus)

[TheAnimus]

AFAIK linux has no support for this volatile pre-caching?

I've not got any virtual's handy but i'm sure Paul or some other windows bod can conferm, when you switch the optomize this workstation for background service, isn't this ment to make the paging of running applications less agressive, enless that app allocates (ie, decrease teh amount reserved for un-opened apps).

[directhex]

AFAIK linux has no support for this volatile pre-caching?

not in a smart manner. it tends to cache files already accessed, on the assumption they'll be accessed again

[TheAnimus]

I wouldn't call the vista way 'smart' persay, more exhuastive than smart.

I really do wounder how SSDs with their random access taking over will rock the boat.