Unless Oxford (and maybe Cambridge) have changed a LOT (which, I admit, could be), you've been unlucky. I've met some very bright people from Oxbridge and, my conclusion is that most Oxbridge grads are still going to be at least among the best .... though there's plenty from other uni's that can say the same.
My problem with some Oxbridge grands .... in fact, with a proportion of all grads though it's a bit more prevalent from the "better" uni's is that though they might be good, they think they're a lot better than they are. They might be well-educated in a subject, but they often haven't yet worked out that subject knowledge is only half the game, and that they still start at the bottom and work up, because they still have a lot of real-world experience to gain, and often people skills too. The way it works in a cloistered academic environment and the way it works out in the big, bad world are usually very different, and some remarkably bright individuals can also be shockingly naive .... and sometimes, remarkably arrogant too.
These are generalisations and I don't label all Oxbridge grads that way, not by a large margin ... but a good number? Oh yeah.
And Animus is right about a bit of real world experience before uni being valuable too. Personally, I think it strengthens the actual education you get while at uni, as it gives a bit of perspective about that big bad world, and leaves a better rounded individual coming out. Aiming at a minimum of 21-ish is too much, though. Even a mandatory year out (doing something useful, not just dossing around the world) is a delay in starting a career ... but IMHO, a generally worthwhile one.