As i understand it, Windows sends stuff out to page file that it doesn't need, but has to keep loaded. This means there is more physical ram for things that do need it and for dynamic caching. This cache won't appear in Task Manager.
Apparently Vista is far more aggressive in the amount of memory it uses for caching. It basically tries to use every last piece of memory it can get it's grubby little hands on. The memory manager it pretty sophisticated so it will decide it is more important to cache a certain block of data than to have this other piece of data in memory. It can't unload that other data, so it sends it to the page file.
perhaps that is over simplifying things, but as i said, that's how i understand it. If anyone else has a better (or indeed more accurate) description, then feel free to correct me
