R5E - QCODE D6, 95 or 90 upon restart after wake from sleep with > 6 DIMMS (96+ GB)

[vacaloca]

Hi all,

This has been driving me a bit crazy today. No overclock on RAM or CPU, BIOS settings are set to optimized (stock)

reproducibility: always
issue: R5E fails to boot after restart with QCODE D6, 95 or 90 (depending on memory slots populated past 6 DIMMS) after a wake from sleep (QCODE 30) from Ubuntu 14.04.4 flash drive OR Win7 x64 on a 960GB Crucial M500 SSD

Note that the system will boot just fine from a shut down state, and will restart happily many times. The problem comes after a restart is made after waking up from sleep.

Components:

MB: R5E, BIOS 1701 (also tried 2001)
CPU: Intel Core i7-5930K
RAM: G.SKILL Ripjaws 4 Series 128GB (8 x 16GB) 288-Pin DDR4 SDRAM DDR4 2400 (PC4 19200)
Video: GeForce GTX 745 or GeForce GT 635 (tried both because of the D6 code)
PS: SeaSonic Platinum-1200(SS-1200XP3) 1200W
SSD: 960GB Crucial M500 SSD
case: Nanoxia Deep Silence 5 case (tried w/ USB 2, USB 3 and FP AUDIO header disconnected)

RAM slots designation (for reference, in order of manual): A1 A2 B1 B2 || D2 D1 C2 C1

Test procedure: Boot into Windows 7 x64 or Ubuntu 14.04.4, place system to sleep, wake it up from sleep, restart, QCODE will manifest before monitor output if more than 6 DIMM slots populated (A1 A2 B1 B2 || __ D1__ C1)

To debug, I started out with just 1 16GB DIMM on A1, and that worked. I proceeded until I populated the configuration above and tested it about 4 times with no issues. After this point, if I add another DIMM or 2 (7 or 8), no matter how I swap the DIMMs in the slots the system will hang on the QCODES mentioned after a restart is done after coming back from sleep (QCODE 30)

I also tried to connect the 4pin power connector in addition to the 8pin, in case it was some sort of power issue, but no change. Also tried the test procedure with USB drive w/o SSD connected, same thing.

Any further troubleshooting appreciated, as I've been at this for nearly 12 hours today and am on the verge of scrapping the build and returning all the components.

Edit: Also tried a Thermaltake TR2 600W power supply, and same behavior persists.

[QuinValen]

Did you "MEM OK" with each addition of population your RAM?
Did the BLCK freq change between slot populations if so?
and finally is this a full RAM kit, or separate kits pieces?
Had problems with 4 separate 2X8GB kits... had to get the 8x8GB kit to stay stable.

[vacaloca]

QuinValen wrote:
Did you "MEM OK" with each addition of population your RAM?
Did the BLCK freq change between slot populations if so?
and finally is this a full RAM kit, or separate kits pieces?
Had problems with 4 separate 2X8GB kits... had to get the 8x8GB kit to stay stable.

Full RAM kit purchased on newegg. BCLK set to auto and is at 100 MHz for 6,7,8 DIMMs. Just did MemOK for 6,7,8 DIMMs and says it's fine, but issue of QCODEs halting boot after restart is made after waking up from sleep

I'm thinking CPU IMC might not be the best:
Before I narrowed the restart after sleep issue down, I was able to get 2400 XMP with a manual 1.21V voltage, 110% current, and high BCLK amplitude (usually hanging between switch from AE to AA), but I was still having issues when restarting after sleep, I just hadn't realized that was the exact cause yet.

[Chino]

The symptom that you're experiencing is caused by system instability. In your case, your memory is unstable. What I would do is update the motherboard to the latest BIOS as the newer BIOS has better DRAM overclocking. If your system is still unstable, then you might need to tweak your System Agent voltage.

[vacaloca]

Chino wrote:
The symptom that you're experiencing is caused by system instability. In your case, your memory is unstable. What I would do is update the motherboard to the latest BIOS as the newer BIOS has better DRAM overclocking. If your system is still unstable, then you might need to tweak your System Agent voltage.

Have tried latest 2001 BIOS and same issue persists. From stock settings, I just tried to up the SA voltage from 0.856 V to 0.875, 0.885, 0.915 V, same behavior. Do I need to go higher?

I forget the combination of DIMMs, but at one point when testing with Win7 x64, I was able to consistently generate a BSOD after a reboot after sleep wake that reads:

"The BIOS in this system is not fully ACPI compliant. Please contact your system vendor for an updated BIOS"
STOP: 0X000000A5

[QuinValen]

Did you perhaps try each of the power management settings in the BIOS? (My crossfire hates one of the APM settings)
Since it happens across OSs perhaps there's a residual bit of code in the BIOS from when the RAM max was at 64GB?
I haven't read anything for greater than 64GB at this time, much less 128GB.

[vacaloca]

QuinValen wrote:
Did you perhaps try each of the power management settings in the BIOS? (My crossfire hates one of the APM settings)
Since it happens across OSs perhaps there's a residual bit of code in the BIOS from when the RAM max was at 64GB?
I haven't read anything for greater than 64GB at this time, much less 128GB.

Was thinking it might have to do with 64G+, but the issue doesn't happen with 96G -- that's the most I've been able to add without running into the QCODEs halting boot after restart is made after waking up from sleep.

I just tried raising VCCSA to 0.94, disabling C-states (only relevant power savings setting that I saw that wasn't disabled by default) and same deal... tried with 2001 BIOS.

[QuinValen]

Have you tried 1801?
Also, seen talk of 1901 being out, but haven't seen it on the US server site yet.

Does it happen after sleep and a shutdown, or just a fast restart? I am interpreting hot restarts /fast rebooting is this correct?

Have you tried disabling fast boot? (Thinking more than likely you have.)

[Chino]

Continue in baby steps until you find the sweet spot.

[Qwinn]

I'm not suggesting this to avoid the problem, if there's instability you should resolve it, but once you have, I recommend just disabling all forms of sleep and hibernation, including Fast Startup which is enabled by default in Windows 10. The fairly significant cons outweigh the negligible pros where desktops are concerned.