Consider a simple graph $G$ with vertex set $V = \{v_1, v_2, \ldots, v_n\}$ and edge set $E$. The diminished Sombor matrix $M_{DS}(G)$ is constructed such that its $(i, j)$ entry is $\frac{\sqrt{d_i^2+d_j^2}}{d_i+d_j}$ if vertices $v_iv_j \in E$, and $0$ otherwise, where $d_i$ represents the degree of vertex $v_i$. In this paper, we establish sharp bounds for the spectral radius, and energy of the Sombor matrix of graphs and identify the graphs that attain these extremal values.