For a prime $p\equiv 3\pmod 4$ and a positive integer $t$, let $q=p^{2t}$. The Peisert graph of order $q$ is the graph with vertex set $\mathbb{F}_q$ such that $ab$ is an edge if $a-b\in\langle g^4\rangle\cup g\langle g^4\rangle$, where $g$ is a primitive element of $\mathbb{F}_q$. In this paper, we construct a similar graph with vertex set as the commutative ring $\mathbb{Z}_n$ for suitable $n$, which we call \textit{Peisert-like} graph and denote by $G^\ast(n)$. Owing to the need for cyclicity of the group of units of $\mathbb{Z}_n$, we consider $n=p^α$ or $2p^α$, where $p\equiv 1\pmod 4$ is a prime and $α$ is a positive integer. For primes $p\equiv 1\pmod 8$, we compute the number of triangles in the graph $G^\ast(p^α)$ by evaluating certain character sums. Next, we study cliques of order 4 in $G^\ast(p^α)$. To find the number of cliques of order $4$ in $G^\ast(p^α)$, we first introduce hypergeometric functions containing Dirichlet characters as arguments, and then express the number of cliques of order $4$ in $G^\ast(p^α)$ in terms of these hypergeometric functions.