Gah think I'm just going to have to get the motherboard replaced. Hopefully I'll either get my money back or get the cost taken off a better one.
I tried the trick with the rubber and it didn't help. So the issue must be that the CPU itself is getting squashed somehow. And after I took the extra rubber off, I had trouble even getting it to start again at all. It only started once the heatsink was so loose that it was barely touching and then it overheated before it could even get in to windows. So for now I am just running it with the stock Intel Heatsink&Fan and even that wouldn't start at first, I had to pull on it so it eased the pressure on the chip and then it would boot in to windows.
Hopefully that is enough to justify a new motherboard.