This is a comparison of everything from single, dual, triple and quadruple 650 to quadruple 690. I have done calculations with Norwegian Kroners, but the ratios should carry over to dollars and pounds and euros with acceptable margin of error. To anyone whom already have computers to put GPUs into, this isn't much help, but to everyone who want to buy a computer or more just for the purpose of trying to live longer (speeding up medical research), this will hopefully be helpful.

The trend is that a dual/triple/quadruple setup of GTX 690 is cheaper, though dual/triple/quadruple 660Ti isn't much more expensive. Not taking electricity into account, as any ovens on temperature control will automatically turn themselves off more often if computers heat up the rooms more. Anyone in cold places, like Norway, will only require cooling a few days every year, so northern BOINCers can run 24/365 with virtually no change to electricity bill.

Here are the charts, they work best if you copy and paste it into notepad (it should line most of it neatly):

To make it easier to read, here's a sum up of the data in the first code bit:

I see there is some on and off discussion of the benefits of single or multiple GPU's, single and dual GPU cards, and other relevant discussions about what hardware to buy.
I think this data is still relevant. It shows multiple GPU's are more financially viable than high-end single GPU's simply because of the GPU to total hardware ratio. The more GPU you have to each PSU, motherboard, CPU, RAM and harddrive, the more value you get for your money. With a strict regime of not spending extra money on CPU, RAM and motherboard, 2-4 GPU's seem the better option, even if you must buy a new motherboard.
I recommend the cheaper 8x, 8x, 8x, 8x, or 16x, 8x, 8x, motherboards, that fit your existing CPU (or cheap triple/quad core CPU, most GPU's don't require more than around 2-2.5ghz per GPU, so dual cores can work for quad GPU's, but then it costs so much 1 dollar extra buys a triple core CPU). And I recommend either 660Ti (cheap buy-in price), or 680 (more long-term work will be done per dollar you spend on it). Remember to get one with 2 or 3 fans on, not the stock 1 fan cooler (MSI Twin Frozr are recommended). triple or quad 690 can work, and is very financially efficient, if you are prepared to spend a lot of time with the software part, but it'll either make so much noise you'll burn it down yourself, or it will catch fire and burn down the building, but it will do so quietly.

I imagine AMD GPU's have roughly the same price range and give the same result.

I almost agree. What I would read out of your data:
- higher end single GPUs provide better bang for the buck
- the more GPUs, the more price-efficient

The 2nd point is true for any GPU performance class, since - as you already mentioned - the fixed cost per PC becomes less of an issue the more GPUs you drive from each PC. There should also be power savings in one PC with multiple GPUs versus the same GPUs in individual PCs. Of course, cooling & noise become a serious issue then, which might require water cooling and lead in hotter running GPUs, which increases their power consumption.

An important issue with multi-GPU setups is the PCIe speed and line width. Intuitively you'd want maximum bandwidth for each GPU. However, on Intels mainstream platform "only" 16 PCIe 3 lanes are routed directly to the CPU. Anything more has to pass through the chipset and possibly a PLX PCIe extender chip. The chipset communicates with the CPU over DMI 2.0 at 20 Gbit/s, i.e. approximately 4x PCIe 2 speed. If you buy an expensive board with several real 16x slots the GPUs will be able to communicate faster with each other, which is nice for games, but will all have to share the limited bandwidth to the CPU (which is the only reason we care for PCIe speed running BOINC). AMD should have similar limitations, whereas socket 2011 fares better.. although this gets one into a whole new cost region. And still doesn't provide official PCIe 3 support, as far as I know.

Anyway, thanks for the massive amount of work you'Ve put into this!

MrS

4 way GTX 690 won´t work. 2x590+2x690 yes; 3x690+1x680 maybe, with some bios trick (i did once, but heat is prohibitive). Won´t talk about PSU needs.
4x690 is a no way due to lack of mobo resources (mostly irqs, it seems).
Tryed by some SETI folks and myself several times.

Ah! I thought we were talking strictly about GPUGrid.

That's pretty amazing about POEM needing that much bandwidth.
That has to be one of the few things that can saturate 16 lanes of PCIE2 let alone PCIE3.

I am pleased that you appreciate the time I put into it.

I recommended 8x, 8x, 8x, 8x, or 16x, 8x, 8x, motherboards assuming it would be used for GPUgrid only. And my example computers only used AMD motherboards (one or two exceptions I think), where the top-end motherboard at the time had 8x, 8x, 8x, 8x when all four were in use and 16x, 8x, 8x with three in use, and 16x, 16x with two in use. And the top-end motherboard in question was cheap compared with all the rest of the hardware put together.

I have not taken xeon or opteron or the most high-end i7 motherboards into account, but over time they will probably do more work than the cheaper stuff for every dollar put in (with or without power use taken into account). But since hardware has to be updated every X years buying the very top-end stuff will be too expensive, even if the heat from the hardware is used to heat the whole house.

I hear Gigabyte is going to make motherboards where the heat can be transferred off the chip more efficiently, so the heat can be used for heating or even run gas turbines that produce electricity, but that's a couple of years out. I gave them the concept drawings only a few months back.

Edit: I would like to add that buying a high-end PSU so big that you only use as much power as the PSU can deliver most efficiently, is a bigger power-saver than energy efficient light-bulbs. But PSU manufacturers to my knowledge aren't easily distributing this information.

I can agree that the gain is often small, but a high up-front price is better than a high recurring price. I'm not saying its THE right thing to do when it comes to PSU's, but I like to lower recurring fees, because you never know what month you'll need that recurring fee for something else. While it is in a year cheaper to pay 10 dollars and have 10 dollars of annual recurring fees than to pay 20 dollars and have 9 dollars of annual recurring fees, the second one makes more sense to me, since I know I can afford the 20 dollars now, and don't know for sure that I can afford the recurring fee in a year.

I agree that it depends on the actual numbers. But for me at the time it made sense to go big on the PSU front. My computers heat the majority of the house and are on 99% of the time. The more expensive PSU is almost worth it entirely because of a more expensive and long-lived fan. But a more expensive PSU also helps ensure a more accurate and healthy waveform in the voltages (the DC current is not actually smooth, its converted from an oscillating 50hertz AC current and still keeps the peaks and valleys to some extent). My computers are also on expensive surge protecting hardware with insurance payoff if something goes wrong from the outside.
But I agree, its not for everyone.

I view it this way: if I couldn't afford 10€ per year instead of 9€ per year, then I really shouldn't put my money into running BOINC at all (or retire the oldest rig / GPU).

Hi, GPUGrid Folks:

How long will I still be able to use my GTX650Ti GPUs on GPUGrid? There appears to be a push to the newer cards....

John

Hi, Stefan:

That's OK- as long as I can use them to process short WUs for some time I'm happy.

Thank you,

John

For 3x690 Host or more, use nothing less than a X79 MB you will need the extra lines sooner or latter.

Just and update 4x690 realy does not work for crunching, at least on windows, so the actual limits is 3x690+1x590, or 3x690+xxx where xxx could be any GPU besides the 690 itself, actualy the limitation is only with 4x690, and is not related to the MB capacity itself. Any onther combiantion of 4 GPUs works, even 4x590 for example.

Of course you need to have the PSU and a way to handle the heat problem if you realy want to "play" with this kind of monsters.
____________

What's wrong with 4x690? Is is a Windows or firmware problem?

MJH