Troubles with the AVX Negative Offset

[Wolfdale]

Hello guys, I have the Strix z370-F board and I need a bit of help regarding the AVX Negative offset option.

I'm able to pass a 6-hours-long Handbrake encoding on a 8K video by using the h.265 encoder (AVX enabled). The voltage set is 1.30v (on Manual, not Adaptive or Offset) and the CPU frequency is 5000MHz. LLC-7.

However if I try to run the same test (same 8K video, same encoder, same everything) at 1.29v but with using AVX negative offset "1" (reducing the frequency to 4900MHz) the system crashes in about an hour or so. I have tried "AVX negative offset" 2 and 3 (gearing down to 4800MHz and 4700MHz respectively) but the system is again unstable at 1.29v. Sometimes it just freezes, sometimes there's a BSOD, sometimes the Handbrake encoder just crashes gracefully with a Windows error message. Obviously if 5000MHz at 1.30v is stable, 4900MHz at 1.29v (or even 4800MHz at 1.29v) should also be stable. However for some reason it seems that even that the AVX negative offset gears down the frequency, the voltage needed to maintain stability remains the same as the one needed for 5000MHz. That kind of makes the AVX negative offset useless. Can someone here at ASUS take a look at this behavior and maybe attempt a fix or something?

TL;DR
5000MHz AVX offset 0 @ 1.30v - stable
5000MHz AVX offset 1 @ 1.29v - unstable
5000MHz AVX offset 2 @ 1.29v - unstable
5000MHz AVX offset 3 @ 1.29v - unstable

So AVX offset isn't wokring good enough as I still need the same voltage as the one needed for AVX offset 0.

[Raja]

Set a ratio of 47X at 1.29V and see if you can pass the AVX stress test. Don't use AVX offset until you know what the CPU can manage under an AVX workload. The gap in stability and required voltages under AVX workloads can be considerable, which is why the option was introduced. A mere 10mV difference and 100MHz gap would negate most of its purpose. LLC 7 is quite severe, too. I tend to use a value of 4-5 on these boards, as it compliments the VRM well.

[Wolfdale]

Raja@ASUS wrote:
Set a ratio of 47X at 1.29V and see if you can pass the AVX stress test. Don't use AVX offset until you know what the CPU can manage under an AVX workload. The gap in stability and required voltages under AVX workloads can be considerable, which is why the option was introduced. A mere 10mV difference and 100MHz gap would negate most of its purpose. LLC 7 is quite severe, too. I tend to use a value of 4-5 on these boards, as it compliments the VRM well.

Raja, maybe I haven't explained the situation good enough. In all the cases I'm always using AVX instructions. 🙂 I start with x50|1.30v|AVX-negative-offset 0 and I run the AVX enabled video encoding. Perfectly stable, always. I've tried this 5 times at least, it always gets the job done without problems.

However when I go back to UEFI and set "negative offset" to "1" and voltage to "1.29", the system is no longer stable. You see, both cases are exactly the same workload. Both are with AVX, I'm using the same video file, same encoder settings. The only difference is that in the first case I'm running 5000 at 1.30 (negative offset 0) and in the second case I'm running 4900 at 1.29v (negative offset 1). Furthermore I have already tried setting the "negative offset" to 2 and 3 (4800MHz or 4700MHz respectively) and it's still unstable at 1.29v. It's like at some point the AVX negative offset "stops working" for a split second, the CPU jumps back to 5000MHz instantaneously and given the fact it only has 1.29v it eventually crashes. At least that's what I'm thinking.

Once again here is it as a quick summary:

5000MHz AVX offset 0 @ 1.30v - running AVX-enabled video encoding stable.
5000MHz AVX offset 1 @ 1.29v - same AVX-enabled video encoding results in a crash.
5000MHz AVX offset 2 @ 1.29v - same AVX-enabled video encoding results in a crash.
5000MHz AVX offset 3 @ 1.29v - same AVX-enabled video encoding results in a crash.

[Raja]

Wolfdale wrote:
Raja, maybe I haven't explained the situation good enough. In all the cases I'm always using AVX instructions. 🙂 I start with x50|1.30v|AVX-negative-offset 0 and I run the AVX enabled video encoding. Perfectly stable, always. I've tried this 5 times at least, it always gets the job done without problems.

However when I go back to UEFI and set "negative offset" to "1" and voltage to "1.29", the system is no longer stable. You see, both cases are exactly the same workload. Both are with AVX, I'm using the same video file, same encoder settings. The only difference is that in the first case I'm running 5000 at 1.30 (negative offset 0) and in the second case I'm running 4900 at 1.29v (negative offset 1). Furthermore I have already tried setting the "negative offset" to 2 and 3 (4800MHz or 4700MHz respectively) and it's still unstable at 1.29v. It's like at some point the AVX negative offset "stops working" for a split second, the CPU jumps back to 5000MHz instantaneously and given the fact it only has 1.29v it eventually crashes. At least that's what I'm thinking.

Once again here is it as a quick summary:

5000MHz AVX offset 0 @ 1.30v - running AVX-enabled video encoding stable.
5000MHz AVX offset 1 @ 1.29v - same AVX-enabled video encoding results in a crash.
5000MHz AVX offset 2 @ 1.29v - same AVX-enabled video encoding results in a crash.
5000MHz AVX offset 3 @ 1.29v - same AVX-enabled video encoding results in a crash.

Try setting the target load voltage using a different LLC (7 is quite aggressive). You may wish to try using Adaptive voltage, too. And for sake of bing thorough, I'd try higher voltage levels to see how far the gap is. 10mV leeway might not be enough for load modulation. The workload likely modulates a bit, which you can't see.

[Wolfdale]

Raja@ASUS wrote:
Try setting the target load voltage using a different LLC (7 is quite aggressive). You may wish to try using Adaptive voltage, too. And for sake of bing thorough, I'd try higher voltage levels to see how far the gap is. 10mV leeway might not be enough for load modulation. The workload likely modulates a bit, which you can't see.

Sorry, I did not understand that fully. Here's my current STABLE setup. 1.300v set in BIOS by using LLC-7 results in 1.296-1.328v depending on the load state (idle/load, it fluctuates a bit). This is fully stable (6+ hours using AVX load).

With LLC-6 and the same 1.300v set in BIOS my actual voltage in HWiNFO64 drops to 1.280-1.312v (link) and is no longer stable. Do you suggest setting ~1.32-1.33v in BIOS with LLC-6 until I get similar stability to setting 1.300v and LLC-7? And if it's stable with 1.33v/LLC-6 you want me to test if 1.32/LLC-6 is stable with negative offset 1 (resulting in 4900MHz) frequency?


I'm just trying to set as low Voltage as possible for my non-AVX workloads. Ideally I want to have something like 1.27v for 5000MHz non-AVX loads with AVX negative offset 3 (4700MHz), but so far it seems this setup wouldn't be stable. 4700MHz at 1.27v would be perfectly stable if that's the max frequency (with avx negative offset = 0), but when I'm using avx negative offset = 3 (and the frequency is again 4700mhz) the system won't be stable for some reason.

[Praz]

Wolfdale wrote:
I'm just trying to set as low Voltage as possible for my non-AVX workloads. Ideally I want to have something like 1.27v for 5000MHz non-AVX loads with AVX negative offset 3 (4700MHz), but so far it seems this setup wouldn't be stable. 4700MHz at 1.27v would be perfectly stable if that's the max frequency (with avx negative offset = 0), but when I'm using avx negative offset = 3 (and the frequency is again 4700mhz) the system won't be stable for some reason.

Hello

The applied voltage is not the same for AVX and non-AVX loads. Intel has programmed the VID table differently depending on AVX usage so this is not as cut and dried as one would think. I will have examples and measurement posted up in the next couple of day that demonstrate this.

[Wolfdale]

Praz wrote:
Hello

The applied voltage is not the same for AVX and non-AVX loads. Intel has programmed the VID table differently depending on AVX usage so this is not as cut and dried as one would think. I will have examples and measurement posted up in the next couple of day that demonstrate this.

But I'm not using offset or adaptive voltage here, I'm using static (Manual) voltage set to 1.300. In theory this should not be related to the VID at all since it's a static (fixed) voltage applied regardless of the multiplier and load type. Furthermore, let's agree that if I'm running stable at 5000MHz/1.3v doing something, I should be at least this stable doing the same thing at 4900MH/1.29v. And that's the problem here - I don't believe the "avx negative offset" really caps the CPU as expected. In theory with 5000MHz non-AVX target and "negative offset 1" the CPU would throttle down to 4900MHz when executing AVX code. However the instabilities I'm seeing suggest that either the 1.29v drops a lot more under AVX compared to 1.30v (making it unable to sustain even 4900MHz - frankly that's unlikely) OR most probably the CPU is not really capped at 4900MHz during the execution of AVX instructions. The way I see it - while the CPU is running AVX instructions every now and then for some reason the frequency spikes back to the original 5000MHz while at the same time it's still running AVX insctructions at 1.29v. Which of course leads to instabilities. For the sake of testing I've just tried to put the AVX negative offset to 10! Non-AVX multiplier is 50, the AVX multiplier is 40. Voltage was set again at 1.29v and guess what - despite the fact that HWiNFO64 is showing that I'm running at 4000MHz at ~1.2800v during the video encoding, it STILL crashed after half an hour. You see, that's just... impossible. That sort of voltage is magnitutes higher than the one needed to sustain 4000MHz. The only reasonable explanation is that at some times (maybe fast enough so the software can't even register that) the CPU frequency jumps straight at 5GHz at 1.29v with AVX... which results in a crash.

Btw running LLC-5 is basically a no-go for me. So far it seems that I need to set CPU Core/Cache voltage at 1.36v with LLC-5 to obtain the same stability I have with 1.30v at LLC-7. That difference is huge and I don't want such high voltages at idle, really. Also, is there even a difference for the VRMs at all? It's true that LLC-7 is a lot more aggressive, but on the other hand 1.30v is a lot lower compared to 1.36. Isn't it the same after all since the voltage under load is the same? I would expect that it makes no difference for the VRM temperatues whether it handles 1.30v at LLC-7 or 1.36v at LLC-5. At the end both settings result in the same voltage under load (so I expect that the VRM temperatures are also identical).

I don't know... I just don't think the negative offset option really works as intended. Right now it seems that it lowers the frequency (as expected) but maybe it's not holding it stable all the time and ultimately you are still not stable which defeats the purpose completely.

[Raja]

When changing LLC, one has to increase VID to get the same load voltage. I think the fact you're pulling the voltages down to their absolute minimum values might be one of the problems. You need to establish exactly how much extra voltage you need for AVX offset to hold the ratios perfectly stable. Until you do that, there's not much else to say or think about.

[Raja]

Higher levels of llc have a lot more overshoot. So your cpu gets a lot more voltage than you set. You can't see it in software as the duration is in the microsecond region. Voltage means nothing without current, so idle voltage isn't the issue. The line sags as soon as appreciable current is drawn. On the other hand, when you set a high llc, your cpu sees far higher voltages as it transitions between load states. That situation is more harmful.

The same goes for AVX. The workload modulates faster than what monitoring tools can see. So while you think the load is continuous, it isn't.

As I said in my previous post, do some testing to find out how much more voltage you need for AVX offset and then work out what you want to do. It will take no longer than you are spending on these posts.

[Wolfdale]

Okay, Raja, I finally managed to get my system running the way I want. I had to increase CPU current capability to 140% and set Power Phase Control and Power Duty Control both to 'Extreme'. I switched to LLC-6 with manually setting Vcore to 1.33 in UEFI. X50 multi selected for non-AVX target multiplier, AVX Negative Offset set to '1' too. Got my system perfectly stable. AVX-Enabled LinX 0.7 is kicking happily at 4900MHz producing ~ 385GFlops. My AVX-enabled 8K h.265 video encoding test seems to also be running smooth as silk at 4900MHz and completed successfully in about 6 hours.


Click the image for larger view^^

Those chips are insanely good once put on custom water cooling and de-lidded. 😮
Thanks for all the help, I guess we can close the thread.