As Monolithic applications evolve, they become increasingly difficult to maintain and improve, leading to scaling and organizational issues. The Microservices architecture, known for its modularity, flexibility and scalability, offers a solution for large-scale applications allowing them to adapt and meet the demand on an ever increasing user base. Despite its advantages, migrating from a monolithic to a microservices architecture is often costly and complex, with the decomposition step being a significant challenge. This research addresses this issue by introducing MonoEmbed, a Language Model based approach for automating the decomposition process. MonoEmbed leverages state-of-the-art Large Language Models (LLMs) and representation learning techniques to generate representation vectors for monolithic components, which are then clustered to form microservices. By evaluating various pre-trained models and applying fine-tuning techniques such as Contrastive Learning and Low Rank Adaptation (LoRA), MonoEmbed aims to optimize these representations for microservice partitioning. The evaluation of the fine-tuned models showcases that they were able to significantly improve the quality of the representation vectors when compared with pre-trained models and traditional representations. The proposed approach was benchmarked against existing decomposition methods, demonstrating superior performance in generating cohesive and balanced microservices for monolithic applications with varying scales.