Encrypted data deduplication is an important technique for saving storage space and network bandwidth, which has been widely used in cloud storage. Recently, a number of schemes that solve the problem of data deduplication with dynamic ownership management have been proposed. However, these schemes suffer from low efficiency when the dynamic ownership changes a lot. To this end, in this paper, we propose a novel server-side deduplication scheme for encrypted data in a hybrid cloud architecture, where a public cloud (Pub-CSP) manages the storage and a private cloud (Pri-CSP) plays a role as the data owner to perform deduplication and dynamic ownership management. Further, to reduce the communication overhead we use an initial uploader check mechanism to ensure only the first uploader needs to perform encryption, and adopt an access control technique that verifies the validity of the data users before they download data. Our security analysis and performance evaluation demonstrate that our proposed server-side deduplication scheme has better performance in terms of security, effectiveness, and practicability compared with previous schemes. Meanwhile, our method can efficiently resist collusion attacks and duplicate faking attacks.