In this paper, we propose a new secure distributed matrix multiplication (SDMM) scheme using the inner product partitioning. We construct a scheme with a minimal number of workers and no redundancy, and another scheme with redundancy against stragglers. Unlike previous constructions in the literature, we do not utilize algebraic methods such as locally repairable codes or algebraic geometry codes. Our construction, which is based on generalized Reed-Solomon codes, improves the flexibility of the field size as it does not assume any divisibility constraints among the different parameters. We achieve a minimal number of workers by efficiently canceling all interference terms with a suitable orthogonal decoding vector. Finally, we discuss how the MDS conjecture impacts the smallest achievable field size for SDMM schemes and show that our construction almost achieves the bound given by the conjecture.