Trust me, decent quality gear sounds very different. I've recently left the budget arena by getting some B&W DM640i speakers, and they sound brilliant. Although day in day out I get to use Genelec and ADAM monitors worth well over £10k each and they sound so much better! As it gets more expensive you do spend more in order to get higher quality but depends how much you want to spend or what you want to use it for.
One of my lecturers swears by decent cables (interconnects at least) and he said he'd lend me a pair of Nordorst interconnets, so I shall be able to judge for myself if they really make much difference