A graph is $(t_1, t_2)$-Ramsey if any red-blue coloring of its edges contains either a red copy of $K_{t_1}$ or a blue copy of $K_{t_2}$. The size Ramsey number is the minimum number of edges contained in a $(t_1,t_2)$-Ramsey graph. Generalizing the notion of size Ramsey numbers, the $F$-Ramsey number $r_F(t_1, t_2)$ is defined to be the minimum number of copies of $F$ in a $(t_1,t_2)$-Ramsey graph. It is easy to see that $r_{K_s}(t_1,t_2)\le \binom{r(t_1,t_2)}{s}$. Recently, Fox, Tidor, and Zhang showed that equality holds in this bound when $s=3$ and $t_1=t_2$, i.e. $r_{K_3}(t,t) = \binom{r(t,t)}{3}$. They further conjectured that $r_{K_s}(t,t)=\binom{r(t,t)}{s}$ for all $s\le t$, in response to a question of Spiro. In this work, we study the off-diagonal variant of this conjecture: is it true that $r_{K_s}(t_1,t_2)=\binom{r(t_1,t_2)}{s}$ whenever $s\le \max(t_1,t_2)$? Harnessing the constructions used in the recent breakthrough work of Mattheus and Verstraëte on the asymptotics of $r(4,t)$, we show that when $t_1$ is $3$ or $4$, the above equality holds up to a lower order term in the exponent.