We present an extension to active learning routines in which non-constant costs are explicitly considered. This work considers both known and unknown costs and introduces the term ε-frugal for learners that do not only consider minimizing total costs but are also able to explore high cost regions of the sample space. We demonstrate our extension on a well-known machine learning dataset and find that out ε-frugal learners outperform both learners with known costs and random sampling.