Dual Monitor Iffy with 1080ti and 10.12.6

Heya,
Well a few weeks back I was rocking a GTX970 with no issues, running 10.12.4 I think and the old nVidia Web drivers.
I installed the new GTX1080ti to replace the 970 and updated to 10.12.6 with the latest web drivers (378.05.05.25f01) and ever since my dual monitors have been acting weird. Like, my main monitor always works but the second one either gives me weird lines on the screen (gibberish) OR has a black screen OR 10% of the times displays desktop normally. To solve this I either reboot several times or plug off and plug in once again the cables in the back.
First monitor is connected via display port directly. It's a DELL 27" the 144hz one. Second monitor is a 4K Samsung TV connected via HDMI but with a Display port -> HDMI adapter for HDR etc. So basically using two Display ports on the 1080ti.
Has anyone experienced anything like this? Any ideas on what logs should I look for? Or maybe a display enumerator tool to get the ID or something of the second monitor and start looking for that string on logs or any ideas whatsoever?
Also I updated clover to the latest version (wasn't able to boot into 10.12.6 with the old one), still using same old 14,2 definitions AND I obviously have Web drivers enabled with the "new" flag ("new" because it's not new anymore and I hope nVidia injection soon will be a thing of museums since we've been rocking the new flag since circa 2016).
Also this dual monitor setup works wonders on Win10 obviously. Never had a single issue with it.