People often claim that x11 doesn’t support HDR but that is not true. HDR can be enabled by setting x11 color depth to 30 (10-bits per color component) and software running under x11 can detect 10-bit mode and display HDR content.
You can set color depth to 30 in x11 by creating /etc/X11/xorg.conf.d/30-hdr.conf (location may differ on your distro) with the content:

Section "Screen"
  Identifier    "Default Screen"
  Monitor        "Configured Monitor"
  Device         "Configured Video Device"
  #               24 for 8-bit or 30 for 10-bit
  DefaultDepth    30

The DefaultDepth 30 part enables HDR support.
10-bit mode can also by enabled with nvidia-settings gui if you are using a nvidia gpu:

Software support

X11 itself supports 16-bit per color component, which is more than enough for HDR. In opengl programs you can use glXChooseVisual with GLX_BUFFER_SIZE set to 30 and GLX_RED_SIZE, GLX_GREEN_SIZE and GLX_BLUE_SIZE set to 10. If these are not set then it will automatically choose the best visual, which is 10-bit mode if you are running under 10-bit mode and 8-bit mode if you are running under 8-bit mode.
Some applications do not support 10-bit mode correctly, such as gimp which has artifacts like these:

which is not an issue with x11 itself. Its an issue with gimp. Some other applications such as chromium have good HDR support under x11 and also supports HDR for videos, as seen on youtube: