In this study, we provided a scheduling procedure which is a combination of machine learning and mathematical programming that minimizes the waiting time of higher priority outpatients. Outpatients who request for appointment in healthcare facilities have different priorities. Determining the priority of outpatients and allocating the capacity based on the priority classes are important concepts that have to be considered in the scheduling of outpatients. In this study, two stages are defined for scheduling an incoming outpatient. In the initial stage, we employed and evaluated four distinct clustering techniques; K-means clustering, agglomerative hierarchical clustering, DBSCAN, and OPTICS clustering to classify outpatients into priority classes and suggested the best pattern to cluster the outpatients. In the second stage, we modeled the scheduling problem as a Markov Decision Process (MDP) problem since the arrivals are uncertain and the decisions are taken at the end of each day after observing total requests. Due to the curse of dimensionality, we used the fluid approximation method to estimate the optimal solution of the MDP. our methodology is employed in a data set of Shaheed Rajaei Medical and Research Center, and we represented how our model works in prioritizing and scheduling outpatients.