We study the rank of the adjacency matrix $A$ of a random Erdos Renyi graph $G\sim \mathbb{G}(n,p)$. It is well known that when $p = (\log(n) - ω(1))/n$, with high probability, $A$ is singular. We prove that when $p = ω(1/n)$, with high probability, the corank of $A$ is equal to the number of isolated vertices remaining in $G$ after the Karp-Sipser leaf-removal process, which removes vertices of degree one and their unique neighbor. We prove a similar result for the random matrix $B$, where all entries are independent Bernoulli random variables with parameter $p$. Namely, we show that if $H$ is the bipartite graph with bi-adjacency matrix $B$, then the corank of $B$ is with high probability equal to the max of the number of left isolated vertices and the number of right isolated vertices remaining after the Karp-Sipser leaf-removal process on $H$. Additionally, we show that with high probability, the $k$-core of $\mathbb{G}(n, p)$ is full rank for any $k \geq 3$ and $p = ω(1/n)$. This partially resolves a conjecture of Van Vu for $p = ω(1/n)$. Finally, we give an application of the techniques in this paper to gradient coding, a problem in distributed computing.