I was using a few different cards a few years back, to work on deep learning. Including 2 x GTX 580 and a few others. A checklist:
1. motherboard form factor.
2. cooling.
3. power supply.
0. memory.
It is important to choose a motherboard with the right form factor which would actually fit your cards physically. A fact that the motherboard has 3xPCIe 16 doesn't necessarily mean that it would fit your two(!) cards. Nothing is more frustrating than not being able to fit the cards. Cooling. Can not be overstated. Also note that the box makes a lot of noise during its operation. Ideally you'd want to put it far away from your workplace. Power supply. Note that the spec that you read on the power supply is usually overrated. If you have 4x200W cards an advise is a 2kW PSU. And GPU memory, if you can fit your dataset in, rather than loading/unloading it in batches that will save you a lot of time and efforts. Well worth the money.
These are some good points. I heard from another person that he had problems with the form factor and I will add that to the post tomorrow. I think a 2kW PSU is overkill, but you are right that more is better for PSUs.
If you want memory a good option will be to wait for the GTX Titan X which will be released in the next weeks: 12 GB RAM and it will be the fastest card by far. Overall however, I think the GTX 980 will be better for many cases still – it is just very cost effective.
