In this paper, we introduce a new attack called controller manipulation attack (CMA) on a Reconfigurable Intelligent Surface (RIS) assisted communication system between a transmitter and a receiver. An attacker has the capability to manipulate the RIS controller and modify the phase shift induced by the RIS elements. The goal of the attacker is to minimize the data rate at the receiver, subject to a constraint on the attack detection probability at the receiver. We consider a number of attack detection models: (i) composite hypothesis testing based attack detection in a given fading block for known channel gains, (ii) sequential quickest detection of CMA in a given fading block for known channel gains, (iii) nonparametric hypothesis test to detect CMA for unknown channel gains over a fading block, and (iv) signal-to-noise-ratio (SNR) moment based detection over possibly multiple fading blocks. In the first case, a simple energy detector turns out to be uniformly most powerful (UMP). In the second case, simplification of the standard CUSUM test and its performance bounds are obtained. In the third case, non-parametric Kolmogorov-Smirnov test is further simplified to a simple per-sample double threshold test. The attack against these three detectors are designed via novel optimization formulations and semidefinite relaxation based solutions. In the fourth case, we consider threshold detection using moments of SNR; various SNR moments under no attack are obtained analytically for large RIS and then used to formulate the attack design problem as a linear program. Finally, numerical results illustrate the performance and trade-offs associated with the attack schemes, and also demonstrate their efficacy.