Cyber-Physical Systems (CPS) play a significant role in our critical infrastructure networks from power-distribution to utility networks. The emerging smart-grid concept is a compelling critical CPS infrastructure that relies on two-way communications between smart devices to increase efficiency, enhance reliability, and reduce costs. However, compromised devices in the smart grid poses several security challenges. Consequences of propagating fake data or stealing sensitive smart grid information via compromised devices are costly. Hence, early behavioral detection of compromised devices is critical for protecting the smart grid's components and data. To address these concerns, in this paper, we introduce a novel and configurable system-level framework to identify compromised smart grid devices. The framework combines system and function call tracing techniques with signal processing and statistical analysis to detect compromised devices based on their behavioral characteristics. We measure the efficacy of our framework with a realistic smart grid substation testbed that includes both resource-limited and resource-rich devices. In total, using our framework, we analyze six different types of compromised device scenarios with different resources and attack payloads. To the best of our knowledge, the proposed framework is the first in detecting compromised CPS smart grid devices with system and function-level call tracing techniques. The experimental results reveal an excellent rate for the detection of compromised devices. Specifically, performance metrics include accuracy values between 95% and 99% for the different attack scenarios. Finally, the performance analysis demonstrates that the use of the proposed framework has minimal overhead on the smart grid devices' computing resources.