Our main result is a quantum public-key encryption scheme based on the Extrapolated Dihedral Coset problem (EDCP) which is equivalent, under quantum polynomial-time reductions, to the Learning With Errors (LWE) problem. For limited number of public keys (roughly linear in the security parameter), the proposed scheme is information-theoretically secure. For polynomial number of public keys, breaking the scheme is as hard as solving the LWE problem. The public keys in our scheme are quantum states of size $\tilde{O}(n)$ qubits. The key generation and decryption algorithms require $\tilde{O}(n)$ qubit operations while the encryption algorithm takes $O(1)$ qubit operations.