Depends on what processor & RAM you currently have, and whether there are any suitable motherboards on the market for those parts. If your motherboard is genuinely "ancient" then it's unlikely you'll be able to buy a new motherboard that will work with your existing CPU and RAM, and there's a good chance you'll have to replace all three components - essentially performing a complete rebuild of your computer.
At that point you simply need to make sure that you choose a CPU and motherboard based on the same socket, then get the RAM that the chosen motherboard supports. Most motherboard manufacturers will publish lists of the CPUs that have been tested with their motherboards if you want to have an extra reassurance that your choices are compatible.
Best thing is to tell us what you already have, then prepare to be swamped with recommendations of what to replace it with