Black4, first, I have to state that i never tried that card... I have though the regular GTX560 Ti DCII TOP... - and I pushed that as hard as could...
😄 Also, have to say that I'm not gaming, so my OCs have to be stable only for benching purposes - which may not be enough for games...
These said, here are some of my experiences:
For some reason, the Asus GPU Tweak is not working nice on my systems... I'm using instead EVGA's PrecisionX... Others are using the MSI Afterburner with success too...
In addition to what HiVizMan said above, be sure that your system's CPU and memory is stable before you start tweaking the graphics card! You have to play with all three hardware to find a stable stage... - for gaming, usually the system's memory is not so important (again, just from my readings), so just leave it a little bit looser (lower frequency and higher timings).
After you set up the CPU and RAM, run Memtest86+!
Also, you have to get the best graphics driver... - you do have to read around the internet to find out which is the best for your card... for start I would try the newest one...
Before you start any tweaking on the VGA, set bios to defaults (F5), boot into OS and uninstall the graphics driver and all Nvidia stuff (will require at least two restarts). Then boot into the OS and use Driver Sweaper to clean up the mess those Nvidia stuff left. Then boot normally, download the driver you like from the Nvidia's driver page and do a clean install... after finished (again, restart will be required), immediately open the Nvidia control panel and stop the automatic updates!
After these, you can start tweaking your system...