PC not starting - Gets as far as LED's coming on.

Yeah, so I bought a new case and PSU the other day. Fitted everything in and tried to start the lil baby up. All my component LEDs, case LED and keyboard LEDs came on, and the fans started turning, for...like...under a second. Then nothing. I had to turn the power of the PSU off for a few seconds and then on again before it would let me try and boot again. After doing so the same thing would happen. So yeah, I figured I'd been sent a faulty PSU, so I tried my old PSU. Same thing happened. I've tried a bunch of stuff, including taking the power out of everything, taking all my PCI/AGP cards out, checked all the case->mobo connections and can't see anything that's amiss. I've also tried both PSUs outside of the case to no joy.

Any ideas? Thanks a lot.

EDIT---
Yeah, it gets weirder. Leaving my old PSU on the drive bay covery bit had the same result, but resting it on the painted exterior of the case got it booting, so could it be earthing or insulation or something?
EDIT2--
RE: Last edit: This did not happen with my new PSU only my old one. Crazy stuff.

What case, psu, & components?

sounds like motherboard / psu is shorting on the case. take MB out of case and try boot up that way. if it works then you know the board is touching the MB tray.

are you using stand offs to mount the board?

Argh. Yeah. Was using stand offs, but when I read what you said it made me think...yeah, totally put them on wrong in a fit of retardation. Thanks a lot, and all is sorted.

Yeah, this should be put in a big sticky somewhere. If the fans come on and then immediately cut out you have a short. Probably the most common Snafu out there these days.