We consider an amplify-and-forward relay network composed of a source, two relays, and a destination. In this network, the two relays are untrusted in the sense that they may perform Byzantine attacks by forwarding altered symbols to the destination. Note that every symbol received by the destination may be altered, and hence no clean reference observation is available to the destination. For this network, we identify a large family of Byzantine attacks that can be detected in the physical layer. We further investigate how the channel conditions impact the detection against this family of attacks. In particular, we prove that all Byzantine attacks in this family can be detected with asymptotically small miss detection and false alarm probabilities by using a sufficiently large number of channel observations \emph{if and only if} the network satisfies a non-manipulability condition. No pre-shared secret or secret transmission is needed for the detection of these attacks, demonstrating the value of this physical-layer security technique for counteracting Byzantine attacks.