Idle power draw is very worth worrying about for systems that spend most of their time idle or under low load (home server, HTPC, office PC) . It might not be for your gaming desktop but for the rest of the world it's a concern and where performance is already good enough it's far better to save the Watts rather than gain more CPU speed you have no need for.
Consider a system that averages 45W rather than 50W (i.e. 10% less) and is on 24/7 for a whole year:
50/1000 * 24 * 365 = 438 kWh
45/1000 * 24 * 365 = 394.2kWh
5W saves you ~44 kWh a year, which at 12p is around £5.25. I know you're thinking that doesn't sound like much, so multiple that by 1000 for a large company, and then consider that's for maybe 5 years over the PC lifetime, suddenly thats £26,250, factor in rising energy prices, add on the corresponding savings to the office air-con or the money saved because you could increase staff and equipment density without overwhelming that air-con (top London office space can be several thousand quid per sq ft annually).
Power savings matter, even little ones that sound like nothing when that translates to wider savings where a few % can mean big money...