Gold can be used in electronic components, and diamonds are so hard they can be used for cutting tools.

Historically most of their value has just been being shiny. And with how diamond mining works they just choke the supply to keep the prices artificially High