The Internet of Vehicles (IoV) has great potential to transform transportation systems by enhancing road safety, reducing traffic congestion, and improving user experience through onboard infotainment applications. Decentralized data sharing can improve security, privacy, reliability, and facilitate infotainment data sharing in IoVs. However, decentralized data sharing may not achieve the expected efficiency if there are IoV users who only want to consume the shared data but are not willing to contribute their own data to the community, resulting in incomplete information observed by other vehicles and infrastructure, which can introduce additional transmission latency. Therefore, in this article, by modeling the data sharing ecosystem as a data trading market, we propose a decentralized data-sharing incentive mechanism based on multi-intelligent reinforcement learning to learn the supply-demand balance in markets and minimize transmission latency. Our proposed mechanism takes into account the dynamic nature of IoV markets, which can experience frequent fluctuations in supply and demand. We propose a time-sensitive Key-Policy Attribute-Based Encryption (KP-ABE) mechanism coupled with Named Data Networking (NDN) to protect data in IoVs, which adds a layer of security to our proposed solution. Additionally, we design a decentralized market for efficient data sharing in IoVs, where continuous double auctions are adopted. The proposed mechanism based on multi-agent deep reinforcement learning can learn the supply-demand equilibrium in markets, thus improving the efficiency and sustainability of markets. Theoretical analysis and experimental results show that our proposed learning-based incentive mechanism outperforms baselines by 10% in determining the equilibrium of supply and demand while reducing transmission latency by 20%.