Indeed found that training the model to respond using few-shot-prompting was effective at increasing the relevant “why” in explanations, but at Indeed's massive scale, this was resulting in high token consumption. To increase efficiency, OpenAI and Indeed worked together to fine-tune a smaller GPT model that was able to deliver similarly performing results but with 60% fewer tokens.

As part of the fine-tuning process, they harnessed GPT‑4 for data augmentation, and built out specific content guidelines. The team also built out an annotation operation, which involved labeling LLM-generated output, creating ground truths for facilitating automated evaluation, and adding context to help the models understand nuances.

Further testing validated the value of personalization in job recommendations, increasing the pool of qualified candidates. To scale personalized job recommendations, Indeed worked with OpenAI to adopt dedicated instances which were provisioned in January 2024. The Indeed engineering team successfully deployed the fine-tuned GPT model to these instances, allowing them to personalize employment  opportunities to millions more job seekers.

“We have invested in our own AI matching technology for decades to help connect job seekers and employers. Regardless of how good our matching may be, explainability is key to any successful recommendation system. Combining OpenAI's GPT explanations with Indeed's own proprietary AI and vast marketplace data allows us to connect more people to jobs, faster—a win for job seekers, employers, and society,” says Chris Hyams, CEO of Indeed.