In many bi-manual robotic tasks, like peg-in-a-hole assembly, the success of the task execution depends on the error in achieving the desired relative pose between the peg and the hole in a pre-insertion configuration. Random actuation errors in the joint space usually prevent the two arms from reaching their desired task space poses, which in turn results in a random error in relative pose between the two hands. This random error varies from trial to trial, and thus depending on the tolerance between the peg and the hole, the outcome of the assembly task may be random (sometimes the task execution succeeds and sometimes it fails). In general, since the relative pose has $6$ degrees-of-freedom, there are infinite numbers of joint space solutions for the two arms that correspond to the same task space relative pose. However, in the presence of actuation errors, the joint space solutions are not all identical since they map the joint space error sets differently to the task space. Thus, the goal of this paper is to develop a methodical approach to compute a joint space solution such that the maximum task space error is below a (specified) threshold with high probability. Such a solution is called a robust inverse kinematics solution for the bi-manual robot. Our proposed method also allows the robot to self-evaluate whether it can perform a given bi-manual task reliably. We use a square peg-in-a-hole assembly scenario on the dual-arm Baxter robot for numerical simulations that shows the utility of our approach.