We consider semiparametric location-scatter models for which the $p$-variate observation is obtained as $X=ΛZ+μ$, where $μ$ is a $p$-vector, $Λ$ is a full-rank $p\times p$ matrix and the (unobserved) random $p$-vector $Z$ has marginals that are centered and mutually independent but are otherwise unspecified. As in blind source separation and independent component analysis (ICA), the parameter of interest throughout the paper is $Λ$. On the basis of $n$ i.i.d. copies of $X$, we develop, under a symmetry assumption on $Z$, signed-rank one-sample testing and estimation procedures for $Λ$. We exploit the uniform local and asymptotic normality (ULAN) of the model to define signed-rank procedures that are semiparametrically efficient under correctly specified densities. Yet, as is usual in rank-based inference, the proposed procedures remain valid (correct asymptotic size under the null, for hypothesis testing, and root-$n$ consistency, for point estimation) under a very broad range of densities. We derive the asymptotic properties of the proposed procedures and investigate their finite-sample behavior through simulations.