The non-Gorenstein locus of stable set rings of finite simple perfect graphs is studied. We describe combinatorially those perfect graphs whose stable set rings are Gorenstein on the punctured spectrum. In addition, we show that, in general, for Cohen--Macaulay graded algebras, their Cohen--Macaulay type and residue are largely independent.