For a half-translation surface (S,q), the associated saddle connection complex A(S,q) is the simplicial complex where vertices are the saddle connections on (S,q), with simplices spanned by sets of pairwise disjoint saddle connections. This complex can be naturally regarded as an induced subcomplex of the arc complex. We prove that any simplicial isomorphism $φ: A(S,q) \to A(S',q')$ between saddle connection complexes is induced by an affine diffeomorphism $F : (S,q) \to (S',q')$. In particular, this shows that the saddle connection complex is a complete invariant of affine equivalence classes of half-translation surfaces. Throughout our proof, we develop several combinatorial criteria of independent interest for detecting various geometric objects on a half-translation surface.