Detecting the presence of target subspace signals with unknown clutters is a well-known hard problem encountered in various signal processing applications. Traditional methods fails to solve this problem because prior knowledge of clutter subspace is required, which can not be obtained when target and clutter are intimately mixed. In this paper, we propose a novel subspace detector that can detect target signal buried in clutter without knowledge of clutter subspace. This detector makes use of the geometrical relation between target and clutter subspaces and is derived based upon the calculation of volume of high dimensional geometrical objects. Moreover, the proposed detector can accomplish the detection simultaneously with the learning processes of clutter, a property called "detecting while learning". The performance of detector was showed by theoretical analysis and numerical simulation.