The performance of particle advection-based flow visualization techniques is complex, since computational work can vary based on many factors, including number of particles, duration, and mesh type. Further, while many approaches have been introduced to optimize performance, the efficacy of a given approach can be similarly complex. In this work, we seek to establish a guide for particle advection performance by conducting a comprehensive survey of the area. We begin by identifying the building blocks for particle advection and establishing a simple cost model incorporating these building blocks. We then survey existing optimizations for particle advection, using two high-level categories: algorithmic optimizations and hardware efficiency. The sub-categories of algorithmic optimizations include solvers, cell locators, I/O efficiency, and precomputation, while the sub-categories of hardware efficiency all involve parallelism: shared-memory, distributed-memory, and hybrid. Finally, we conclude the survey by identifying current gaps in particle advection performance, and in particular on achieving a workflow for predicting performance under various optimizations.