This work presents a practical solution to the problem of call center agent malpractice. A semi-supervised framework comprising of non-linear power transformation, neural feature learning and k-means clustering is outlined. We put these building blocks together and tune the parameters so that the best performance was obtained. The data used in the experiments is obtained from our in-house call center. It is made up of recorded agent-customer conversations which have been annotated using a convolutional neural network based segmenter. The methods provided a means of tuning the parameters of the neural network to achieve a desirable result. We show that, using our proposed framework, it is possible to significantly reduce the malpractice classification error of a k-means-only clustering model which would serve the same purpose. Additionally, by presenting the amount of silence per call as a key performance indicator, we show that the proposed system has enhanced agents performance at our call center since deployment.