Personal item tracking devices are popular for locating lost items such as keys, wallets, and suitcases. Originally created to help users find personal items quickly, these devices are now being abused by stalkers and domestic abusers to track their victims' location over time. Some device manufacturers created `anti-stalking features' in response, and later improved on them after criticism that they were insufficient. We analyse the effectiveness of the anti-stalking features with five brands of tracking devices through a gamified naturalistic quasi-experiment in collaboration with the Assassins' Guild student society. Despite participants knowing they might be tracked, and being incentivised to detect and remove the tracker, the anti-stalking features were not useful and were rarely used. We also identify additional issues with feature availability, usability, and effectiveness. These failures combined imply a need to greatly improve the presence of anti-stalking features to prevent trackers being abused.