To maximize palm oil yield and quality, it is essential to harvest palm fruit at the optimal maturity stage. This project aims to develop an automated computer vision system capable of accurately classifying palm fruit images into five ripeness levels. We employ deep Convolutional Neural Networks (CNNs) to classify palm fruit images based on their maturity stage. A shallow CNN serves as the baseline model, while transfer learning and fine-tuning are applied to pre-trained ResNet50 and InceptionV3 architectures. The study utilizes a publicly available dataset of over 8,000 images with significant variations, which is split into 80\% for training and 20\% for testing. The proposed deep CNN models achieve test accuracies exceeding 85\% in classifying palm fruit maturity stages. This research highlights the potential of deep learning for automating palm fruit ripeness assessment, which can contribute to optimizing harvesting decisions and improving palm oil production efficiency.