I think the thread rapper 5990x is better, the mainboard you can uses supermicro server motherboard, I have a computer using amd 5950x and a 3090ti, the mainboard is x570, if you use 4x 3090, it is best to use a server mainboard, for stable and performance.😉

I read your shopping list, there's has two problems, the motherboard and power supply, the power of one 3090 is 350 watts, and the power of four is 1400 watts, so your power supply should be at least 2000 watts (The specific calculation will be made after the CPU is determined), the problem with the motherboard is that the b650 does not support four 3090 video cards, and it only has two videos card slots.😉

Yeah, there's no way to add two PSUs on the pcparpicker, so that's mean to be 2 of the 1000W ones.

The b650 supports 4. It has enough ports. The blocking is not an issue because I'm going to be using GPU risers to fit the 4 GPUs.

To respond to your first comment, the thread ripper is also very expensive, and I'm waiting until Sept 2023 when threadripper 7 comes out and drops prices for thread rippers.

Well, I’m not sure how feasible the method of two power supplies for one host is😂, and it’s still a problem with the motherboard. I can’t comment on the stability of the method of connecting four 3090s using graphics card expansion (because I haven’t done so yet), I think you should carefully consider your plan, the cost of trial and error is not low.

What is the intended use case for the GPUs? I presume you intend to train networks, but which kind and at what scale? Many small models, or one big model at a time?

Or if you are doing inference, what types of models do you intend to run.

The configuration you suggested is really only good for training / inferencing many small models in parallel, and will not be performant for anything that uses more than 2 GPUs via NVLink.
Also don't forget about system RAM... depending on the models, you may need ~1.5x the total VRAM capacity in system RAM, and deepspeed requires a lot more than that (upwards of 4x) - I would probably go with at least 128GB for the setup you described.

> - with 4 gpus that's 4 PCIe4 lanes per gpu

With the Asus PRIME B650M-A AX you are looking at 16 lanes for 1 GPU and 1 lane each for the other 3 GPUs.

PCIEX16_2, PCIEX16_3 and PCIEX16_4 slots are electrically connected in x1 mode.

Bifurcation is only supported via PCIEX16_1 slot.

[removed]

I actually works very well with ADD2PSU connector (used like 5 PSUs for one 14x3090 rig). He should actually think more of getting 1600W HIGH QUALITY PSU.

Corsair RM series IS NOT SUITABLE for workload you are looking for. Use preferrably AXi series or HXi if you really want to cheap out. We are talking about really abusing those PSUs. AX1600i is still unmatched for this usecase.

Just get a single 3090 and the 7950x or 13 series intel, then use the rest that you would have spent on renting bigger GPUs in the cloud.

Damn, 1 lane each for the other 3 isn't enough for my needs. The bifurcation kinda sucks here. Thanks for the advice.

Each time someone asks this question, someone repeats this misinformed answer.

This is incorrect, NVLink doesn't make much difference.

https://www.pugetsystems.com/labs/hpc/rtx-2080ti-with-nvlink-tensorflow-performance-includes-comparison-with-gtx-1080ti-rtx-2070-2080-2080ti-and-titan-v-1267/

I plan on using this frequently, and compared to even renting a similar configuration online, I would break even after a year.

I don't know if that's going to work well to have 16 PCIe lane, everyone here I've seen making 4 GPUs machines uses the CPUs that have 48 or 16 PCIe lanes.

Also you'll need a lot of watts to power that monster, not to mention you need a 10-20% margin if you don't want fry the PSU.

Thanks for the advice, can you elaborate more about how the Corsair RM series is not suitable for the workload? My rationale was that because it's an open air mining frame instead of a case, I wanted the RM series which supposedly is quieter?

Yeah, that's what I've basically discovered too. The mobo with the 16 PCIe lanes isn't going to work out. Changed my build to threadripper. Any advice or suggestions for a PSU that can handle the workload?

That information is very outdated, and also not very relevant...
The 3090 is an Ampere card with 2x faster NVLink, which has a significant advantage in speed compared to the older GPUs. I'm not aware of any benchmarks that explicitly tested this though.

Also, Puget benchmarked what I would consider "small" models. If the model is small enough, then the interconnect won't really matter all that much as you're going to spend more time in com setup than transfer.
But for the bigger models, you better bet it matter!
Although to be fair, my original statement is based on a node with 4x A6000 GPUs, configured in a pair-wise NVLink configuration. When you jump from 2 paired GPUs over to 4 GPUs with batch-parallelism, the training time (for big models - ones which barely fit in the 3090) will only increase by about 20% rather than the expected 80%.
It's possible that the same scaling will not be seen on 3090s, but I would expect the scaling to be worse in the system described by the OP, since the 4x system allocated a full 16 lanes to each GPU via dual sockets.

Note that this is why I asked about the type of training being done, since if the models are small enough (like ResNet-50), then it won't matter - though ResNet-50 training is pretty quick and won't really benefit that much from multiple GPUs in the grand scheme of things.

Right now mostly CNNs, RNNs, and playing around with style transfers with GANs. Future plans include running computer vision models trained on videos and testing inferencing, but still researching how demanding that would be.

Those are all going to be pretty small models (under 200M parameters), so what I said probably won't apply to you. Although, I would still recommend parallel training rather than trying to link them together (4 GPUs means you can run 4 experiments in parallel - or 8 if you double up on a single GPU).

Regarding RAM speed, it has an effect, but it probably won't be all that significant given your planned workload. I recently changed the memory on one of my nodes so that it could train GPT-J (reduced the RAM speed so that I could increase the capacity), the speed difference for other tasks is probably within 5%, which I don't think matters (when you expect to run month long experiments, an extra day is irrelevant).

Under full load, AXi series is basically silent. But main reason is that PSU is not of high enough quality to actually sustain that load (even higher grade PSUs like EVGA P2 series has problems with infamous 3090 under DL task load) . Also take a look at my big comment on this reddit post.

Well, in just the most basic tasks - like plain resnet100 training (classification) by using nvlink - there is a huge difference.

that also depends on input image size though...

Could you elaborate on what you mean by that?
The advantage of NVLink is gradient / weight communication, which is independent of image size.

Sorry, I referred explicitly to the the last paragraph of yours (that it's quick for small models)

Hmm, the Axi1600 might not be enough for me. This is my new build: https://ca.pcpartpicker.com/user/sixartsdragon/saved/DCh6Q7 and I'm looking at 1821W, so realistically I'm looking for a 2K PSU. I've chosen a Super Flower Leadex 2000 for now? What do you think?

Do you have some numbers or a link because all benchmarks I've seen point to the contrary? I'm happy to update my opinion if things have changed and there's data to support it.

I suppose one PSU will take care of motherboard + CPU + some GPUs and the other one will take care of remaining GPUs.

So if you get 4x 3090, that's 350W x4 = 1400W just for GPUs, +300 watts for CPU, +powering the rest of the components, drives, etc... So let's say we round that up to 2000W, then add at least 10% margin, that's 2200W total.

So maybe 1600W PSU for mobo and some GPUs, and another 1000W or more for the remaining GPUs. Note, if you go with 3090TI, it's more like 450-500W per card, so you have to do the maths.

Or if you want to go future proof, just put two 1600W PSUs, and then you can just swap your 3090 with 4090 in the future and not worry about upgrading PSUs.

I'm seeing the argument for 2 1600 PSUs. It's fine for the mining rig case frame, but it's baiscly confirming to me that this is never going to fit in a case lol.

Then I agree. If you are doing ResNet inference on 8K images, then it will probably be quite slow. However 8K segmentation will probably be even slower (the point of comparison that I was thinking of).
Also, when you get to large images, I suspect the PCIe will become a bottleneck (sending data to the GPUs), which will not be helped by the setup described by the OP.

[deleted]

Buy 2× 4090

Performance is only 1.9 times the 3090 for deep learning and price is more than double, just bad value rn.

You save a lot electricity cost. Much beter value if you plan to use it a lot. It"s much easier to manage 2 cards than 4 cards.

3090 is not a good card, running at high temperatures, high noise, excessively high VRAM temperatures.

For sure that's the upgrade path in the future, but rn my electricity is free so it's not too much of an issue.

It's a budgeting issue, if I could do 4 4090s, I would.

I actually assumed you will be having 2 PSUs. For least problems, buy 2xAX1600i, for cheaper option buy 2xAX1200i. One PSU is actually the worst case, but yeah you can try with a single SFL 2000.

What do you mean least problems and why is one worst case?

Actually I've been sort of looking at ML computers (kind of of browsing and dreaming one day I would have one, but it's always going to be out of my means and needs anyway). Anyway, they can put two PSUs in a box, obviously it's made by companies, so the total cost is twice or 3 times the cost of the parts alone (e.g. building yourself would be 2-3x cheaper) but it could inspire you for picking your parts.https://bizon-tech.com/amd-ryzen-threadripper-up-to-64-cores-workstation-pc

https://shop.lambdalabs.com/gpu-workstations/vector/customize