Computational privacy is a property of cryptographic system that ensures the privacy of data being processed at an untrusted server. Fully Homomorphic Encryption Schemes (FHE) promise to provide such property. Contemporary FHE schemes are suited for applications that have single user and server. In reality many of the cloud applications involve multiple users with various degrees of trust and the server need not necessarily be aware of it too. We present a Complementary Key Pairs technique and protocols based on that to scale any generic FHE schemes to multi user scenarios. We also use such technique along with FHE to show how attribute based access control can be achieved while server being oblivious of the same. We analyze the protocols and their security. Our protocols don't make any assumptions on how FHE scheme itself works.