The Robinson-Schensted correspondence can be viewed as a map from permutations to partitions. In this work, we study the number of inversions of permutations corresponding to a fixed partition $λ$ under this map. Hohlweg characterized permutations having shape $λ$ with the minimum number of inversions. Here, we give the first results in this direction for higher numbers of inversions. We give explicit conjectures for both the structure and the number of permutations associated to $λ$ where the extra number of inversions is less than the length of the smallest column of $λ$. We prove the result when $λ$ has two columns.