GrabYourWallet, #boycottNRA and other online boycott campaigns have attracted substantial public interest in recent months. However, a number of significant challenges are preventing online boycotts from reaching their potential. In particular, complex webs of brands and subsidiaries can make it difficult for participants to conform to the goals of a boycott. Similarly, participants and organizers have limited visibility into a boycott's progress. This affects their ability to use sociotechnical innovations from social computing to incentivize participation. To address these challenges, this paper makes a system contribution: a new boycott tool called Out of Site. Out of Site uses lightweight automation to remove obstacles to successful online boycotts. We describe the design challenges associated with Out of Site and report results from two phases of deployment with the GrabYourWallet and Stop Animal Testing boycott communities. Our findings highlight the potential of boycott-assisting technologies and inform the design of this new class of technologies. Finally, like is the case for many systems in social computing, while we designed Out of Site for pro-social uses, there are a number of easily predictable ways in which the system can be leveraged for anti-social purposes (e.g. exacerbating filter bubble issues, empowering boycotts of businesses owned by racial, ethnic, and religious minorities). As such, we developed for this project a new, very straightforward design approach that treats preventing these anti-social uses as a top-tier design concern. This approach stands in contrast to the status quo of ignoring potential anti-social uses and/or considering them to be a secondary design priority. We discuss how our simple approach may help other research projects reduce their potential negative impacts with minimal burden.