Let $M=(m_{ij})$ be an $n\times n$ matrix. The second immanant of matrix $M$ is defined by \begin{eqnarray*} d_{2}(M)=\sum_{σ\in S_{n}}χ_{2}(σ)\prod_{s=1}^{n}m_{sσ(s)}, \end{eqnarray*} where $χ_{2}$ is the irreducible character of $S_{n}$ corresponding to the partition $(2^{1},1^{n-2})$. The polynomial $d_{2}(xI-M)$ is called the second immanantal polynomial of matrix $M$. Denote by $D(G)$ (resp. $D(\overrightarrow{G})$) and $A(G)$ (resp. $A(\overrightarrow{G})$) the diagonal matrix of vertex degrees and the adjacency matrix of undirected graph $G$ (resp. digraph $\overrightarrow{G}$), respectively. In this article, we prove that $d_{2}(xI-A(G))$ (resp. $d_{2}(xI-A(\overrightarrow{G}))$) can be reconstructed from the second immanantal polynomials of the adjacency matrix of all subgraphs in $\{G-uv,G-u-v|uv\in E(G)\}$ (resp. $\{\overrightarrow{G}-e|e\in E(\overrightarrow{G})\}$). Furthermore, the polynomial $d_{2}(xI-D(\overrightarrow{G})\pm A(\overrightarrow{G}))$ can also be reconstructed by the second immanantal polynomials of the (signless) Laplacian matrixs of all subgraphs in $\{\overrightarrow{G}-e|e\in E(\overrightarrow{G})\}$, respectively.