Let $(X, \mathcal{B},μ,T)$ be an ergodic measure preserving system, $A \in \mathcal{B}$ and $ε>0$. We study the largeness of sets of the form \begin{equation*} \begin{split} S = \left\{ n\in\mathbb{N}\colonμ(A\cap T^{-f_1(n)}A\cap T^{-f_2(n)}A\cap\ldots\cap T^{-f_k(n)}A)> μ(A)^{k+1} - ε\right\} \end{split} \end{equation*} for various families $\{f_1,\dots,f_k\}$ of sequences $f_i\colon \mathbb{N} \to \mathbb{N}$. For $k \leq 3$ and $f_{i}(n)=i f(n)$, we show that $S$ has positive density if $f(n)=q(p_n)$ where $q \in \mathbb{Z}[x]$ satisfies $q(1)$ or $q(-1) =0$ and $p_n$ denotes the $n$-th prime; or when $f$ is a certain Hardy field sequence. If $T^q$ is ergodic for some $q \in \mathbb{N}$, then for all $r \in \mathbb{Z}$, $S$ is syndetic if $f(n) = qn + r$. For $f_{i}(n)=a_{i}n$, where $a_{i}$ are distinct integers, we show that $S$ can be empty for $k\geq 4$, and for $k = 3$ we found an interesting relation between the largeness of $S$ and the abundance of solutions to certain linear equations in sparse sets of integers. We also provide some partial results when the $f_{i}$ are distinct polynomials.