CPU, virtualization, and overclocking advice

[Profesora]

I am looking into putting together a system that will be heavily making use of virtualization... by heavily I mean up to 20 virtual machines at a time, each one having one or two CPU demanding processes. I was wondering if someone could advise me on the best CPU for such a task. So far I was planning to get a Q6600 and overclock it as much as it will go, and have up to 8GB RAM.

Now it is occurring to me that overclocking and virtualization might not like each other for some reason. Does someone have experience with that? Also I was wondering if the Q6600 is the most suitable CPU for that task. My budget for a CPU is up to 150 quid. Any info will be greatly appreciated

[Webby]

With a CPU budget of £150 and a number of virtual machines you only really have 3 choices, Q6600, Q8200 and Phenom.

Assuming we can discount the Phenoms straight away due to have to buying a new motherboard as well as CPU, Im assuming since you asked about the Q6600 that you already have an Intel board.

That leaves the Q6600 and the Q8200, since overclocking is the order of the day (no reason it should adversly effect anything as long as it is stable!) then you really only have one choice and that is the Q6600.

If you can wait and save up a bit more money (well a lot more money) an i7 system would be more suited to a large number of virtual machines, with 8 threads it will out perform the Q6600 (4 threads) by a large margin in your situation will cost you ~£600 for CPU, motherboard and RAM (6GB) though.

Oh and to save some money check the Scan Shop Offers page, the Q6600 is £120 on there rather than £140 on the normal listings http://www.scan.co.uk/ShopOffers/Index.aspx

[badass]

I am looking into putting together a system that will be heavily making use of virtualization... by heavily I mean up to 20 virtual machines at a time, each one having one or two CPU demanding processes. I was wondering if someone could advise me on the best CPU for such a task. So far I was planning to get a Q6600 and overclock it as much as it will go, and have up to 8GB RAM.

Now it is occurring to me that overclocking and virtualization might not like each other for some reason. Does someone have experience with that? Also I was wondering if the Q6600 is the most suitable CPU for that task. My budget for a CPU is up to 150 quid. Any info will be greatly appreciated

Depends on how much you are planing to spend TBH. Core i7 will without a doubt be the fastest, but mega bucks. The Q6600 is good but not great for virtualisation. I expec the Q9450/9550 to be a lot better due to more virtualisation optimisation and 50% larger L2 caches.
I beleive the Q8200 cannot do 64 bit virtualisation.
I know that in 2 CPU's or more, the Quad core opteron is better than any intel non Nehalem based Quad, however I expect the gap to be smaller when comparing a single Q6600 to a single Phenom.

[shaithis]

Have done a far amount of virtualisation with both the Q6600 and the Q9550, both fairly heavily overclocked.

Overclocking makes no difference, as long as you are sure your overclock is stable to begin with (max all cores with something heavy for 24 hours before declaring your overclock stable)

Depending on the tasks running on each VM, you might find some benefit from having extra disks in the machine and creating your virtual machines (or at least their virtual disks) on different drives.

As for Q6600 v Q9550, I honestly can't say I saw a difference between them but then I wasn't timing any hugely CPU-intensive tasks.

[Moby-Dick]

if you want to run 20VM's you are going to need more than 1 CPU

Memory is much more likly to be the bottleneck as even though you can over commit cpu's by about 400% for an avarage workload , the fact that you are wanting to run 20 CPU intensive processes means you are going to need to really think about what you are doing.

What OS are you going to use for virtualisation ? I'd go with something like ESX 3i.

How much memory do you want each VM to have ? what exactly are they doing ?

Disk wise I'd not use local disk at all unless you have a lot of space for them and multiple local disk controllers.

FYI I do actually know what I'm talking about on this one

[shaithis]

FYI I do actually know what I'm talking about on this one

Although judging from the OP, his budget isn't going to reach anywhere near what your advocating

[Moby-Dick]

I suspect not , but then again I'm not sure how you are going to virtualise 20 CPU intensive workloads without spending a lot of money, then again it depends on your definition of CPU intensive.

[Vimeous]

I'm with Moby on this one - 20 VM's on one quad, no matter how overclocked, is asking for trouble if they're all running simultanious CPU intensive tasks. You'll definitely maximise you hardware use that's for sure!

If you're talking non-concurrant, burst-load tasks 20 will probably work out fine.


To get you as close as possible I suggest:

1. Use free ESX 3.5 as the core OS.
2. Overload on RAM because you'll be needing 512Mb minimum specced per VM (regardless of OS IMO) and even with efficient sharing I doubt 4Gb would cut it. - Right now PC6400 DDR2 is priced to suit you perfectly.
3. If you can, get a CPU with 64-bit VM extensions (I forget if the early Core 2 Quad have).

ESXi is possible but Moby is that free or cost only?

For quick reference I just peeked at one of our ESX 3.5 Update 2 boxes.
It's running 3x Ubuntu Server 64-bit, 1x 2k3 Svr, 1x XP Pro, and 1x 2k8 Svr Ent - non are constantly loaded.
CPU is averaging 15% on a pair of of x5355 quad Xeons.
RAM is averaging 35% on 16Gb.

As has already been mentioned you need to consider your usage.
If you are running multiple machines are they operating purely in RAM? If they are writing to disk is your storage going to be the performance bottle-neck?

Lots to consider but satisfying when in use!

[Profesora]

Once again I am speechless about the knowledge one can absorb from these forums!

Would you believe that I have a BSc and an MSc and I even soon started a PhD in computer science, and frankly said I do need to look up a lot of the terms when you guys are talking about these matters... not speaking well about my university education

Anyway, more to the point:
When I said CPU intensive, I did not really mean CPU intensive... sorry about that confusion. It was early morning and I did not think carefully what I am saying. More like running custom Java programs that do some heavy processing for 0.3secs, then sleep for 0.7secs, then process again etc. So I would like to have 40 of these. All computation can be entirely done in memory, without access to a hard drive. I am planning to run tiny XP in each of the VMs, as I do need windows, hence I would expect 256MB per VM to be sufficient (maybe even 128 will do). Each VM is nothing fancy really, that's why I am optimistic at having 20 on a single quad core CPU.

Budget: up to 700 quid including keyboard, mouse, and delivery (no monitor), that's why the core i7 and multi CPU systems are out of the question for now. I also have not bought any hardware yet, so I could just as well buy a Phenom with the appropriate motherboard, given it will be better than a Q6600 (althought badass suggested the gap would be small)

Basically, I am glad to hear that overclocking does not really interfere with virtualization. The Q6600 has Intel® EM64T and Intel® Virtualization Technology (I guess this is the vanderpool virtualization technology, whatever this is), so I hope it has "64-bit VM extensions" as vimeous suggested. I will read into the VMware ESX Server products and could just as well use the suggested OSs. If the performance gap will be small, I would tend to go to the Intel CPU side, just because I am a bit more familiar with them and I am already overwhelmed with information.

Hopefully I have shed some light into what exactly I am trying to do. Any other suggestions to the best way to accomplish the task will be more than welcome. I will soon put together the specs of a sample system and would be posting it on these forums for further advice.

[Moby-Dick]

ESX3i is the free one - its 3.5 thats going to cost you - as its a single machine you wont need any virtualcenter type stuff .

https://www.vmware.com/tryvmware/?p=esxi

check the HCL for 3i when it comes to choosing things like raid controllers.

if you have got a lightweight XP build on there with a nice small memory print then you'll do fine.

There will be a bit of trial and error required - I suggest starting out with only a few VM's and then scale up and make sure that you are not affecting overall performance.

[Profesora]

Thanks for the advice. ESX3i seems like the perfect option for me now. The lightweight XP build that I am considering is tinyXP that has been circulating for quite some time, and I guess you all know well (http://digg.com/software/TinyXP_55Mb_WinXP_client)

[Vimeous]

Thanks for clearing that up Moby, we run a small mix of both 3i and 3.5 so which was the free option hadn't really entered into my mind TBH

I agree with the start small and see how you get on. Your modest OS footprint is less than I expected so I guess you've got a decent chance of achieving your aims.

You say the processes are planned to be a 0.3secs. Is this a minimum?
Even with well-planned overlap you'll have a minimum of 7 VM's requesting intensive CPU use at all times, or worse all 20 at once. Logically if each VM is given a single core and wants 100% of 0.3 of each second you'll run the machine at full speed 90% of the time with just 12 VM's.


When it comes to OC'ing everything I've read so far suggests a q6600 has the most headroom on air of all the CPU's mentioned so far. Of course with your intended use I wonder if it'll be necessary at all.
I'd suggest buy retail and keep back £20 for modest cooler if you really need it.


As for PC I suggest:
Intel Q9450 - for the extra CPU cache
Asus P5Q - good quality and should OC enough if needed - use the on-board hardware raid initially.
4Gb Corsair PC6400 DMX - this is great value atm - go PC8500 if you're serious about OC'ing without breaking your budget

I'm a bit woolly on datastorage configurations for VM's TBH. If ESXi sits on a USB stick (in our case inside the server) then a single fast drive to store your VM's and say a RAID0 pair for write speed for your calculation results. If the numbers aren't big I'd suggest Western Digital 320Gb drives. However Samsungs bigger F1's (750/1000Gb) are damn fast and stonking vfm making the machine far more reusable in the future.

Cases like the Antec 300 or Coolermaster CM590 should suit with space for extra cooling should you need to OC. That said you may need look elsewhere if you want to watercool.
Graphics - well anything that'll display 1024x768 is a start! An integrated gfx mboard would do nicely!

Hmm it's late and my brain is/has faded ......


Happy VM'ing!

[Profesora]

Thanks a lot for the replies! I will adopt the start small approach and see when I start degrading overall performance, as vimeous's reasoning about loads makes perfect sense. Now I have another question

As vimeous mentioned also the graphics card, another question just popped into my mind that I had not addressed before. If I get a decent graphics card, say ATI HD 4670 (just ~60 pounds), would I be able to make use of it inside a windows VM, for playing games let's say I mean I will build quite a good system, and I would not mind playing some good quality games. As the core OS will be ESXi though, would the VMs be able to take full advantage of the graphics card?