Hi, I bought two old Infiniband adapters pulled from old servers and I wanted to use them on my ProArt X670E Creator Wifi system. The cards' model is Mellanox CX455A (Connect-X 4 100Gb single port).
However, I can't get either of the cards to work on the motherboard. I tried multiple setups, including:
- Putting the adapter as the only PCIE device on the system, on PCIE slot 1 with x16 mode
- Putting the adapter together with GPU, on PCIE slot 1 and slot 2 with x8 x8 mode
but when the adapter card is in, the BIOS will stuck in POST and it will take 6-7 minutes before it can proceed with anything. After it finally got into OS the card just cannot be detected anywhere (tried both Windows and Linux through live USB media).
But, when I put the card onto the PCIE slot 3 with x2 mode (connected via chipset), it could properly POST and show up in my Linux system. Unfortunately, the card is useless in this slot due to it only supporting PCIE 3.0 thus only having a pathetic 2 GB/s bandwidth, only around 20% of its full capacity.
To make sure the cards are good, I also tried them out on an old ASRock Z370 motherboard. Both of the cards work fine there with a GPU in x8 x8 mode, but I just can't make them work on my X670E's CPU PCIE slots.
What's worse is that when I was tweaking around pulling and plugging the card in various slots, I noticed that the card had become extremely hot when plugged into X670E CPU PCIE slots, much hotter than when it was put into Z370 and in a working state. After trying to debug for a whole night, both of the cards completely failed after being plugged into X670E CPU PCIE slots for prolonged times. I could no longer get them detected on any slot of any motherboard, and the chip would remain completely cold plugged in.
Now that I understand it would be a bit too far-stretched to accuse ASUS motherboard of frying my adapters, considering these are old parts and may be prone to fail. But I did run one of them really fine on Z370 for at least 30 minutes and it only failed when I moved it over to X670E. I would rather believe it is just the adapters were bad but this fact is just disturbing.
After the adapters were dead, I tried a few other PCIE cards I had at hand to see if the slots were bad/defective. However all my other PCIE devices, two GPUs, and one USB controller card, all work fine.
I contacted the server part dealer and they were willing to do a replacement. I am waiting for them to come in so that I may try if any of the BIOS settings impacted this. But without the root cause of this identified, there is no guarantee that I won't fry them again. I am just so frustrated at this situation. I am also considering raising this to ASUS support next Monday, but any extra help and suggestions from this community will be much appreciated.
