Pistachio nut is mainly consumed as raw, salted or roasted because of its high nutritional properties and favorable taste. Pistachio nuts with shell and kernel defects, besides not being acceptable for a consumer, are also prone to insects damage, mold decay, and aflatoxin contamination. In this research, a deep learning-based imaging algorithm was developed to improve the sorting of nuts with shell and kernel defects that indicate the risk of aflatoxin contamination, such as dark stains, oily stains, adhering hull, fungal decay and Aspergillus molds. This paper presents an unsupervised learning method to classify defective and unpleasant pistachios based on deep Auto-encoder neural networks. The testing of the designed neural network on a validation dataset showed that nuts having dark stain, oily stain or adhering hull with an accuracy of 80.3% can be distinguished from normal nuts. Due to the limited memory available in the HPC of university, the results are reasonable and justifiable.