Oh, I think you are trying to get 594 MHz (HDMI 2.0 timing) to a display that only supports 533.25 MHz (CVT-RB timing). Have you tried HDMI 2.0 timing over DisplayPort to the display from PC to see if the display supports 594 MHz?
HDMI 2.0 timing is like this:
3840×[email protected] 135.000kHz 594.00MHz h(176 88 296 +) v(8 10 72 +)
(front porch, sync width, back porch, polarity)
CVT.RB timing is like this:
3840×[email protected] 133.313kHz 533.25MHz h(48 32 80 +) v(3 5 54 -)
I don’t know if there’s a way to fix this if there’s no way to make a custom timing in Xbox. A modern 4K display supporting more than 4K60 may be the best solution.