Visualization design study research methodologies emphasize the need for reflection to generate knowledge. And yet, there is very little guidance in the literature specifying what reflection in the context of design studies actually involves. We initiated a community discussion on this topic through a panel at the 2017 IEEE VIS Conference - this report documents the panel discussion. We analyze the panel content through the lense of our own reflective experiences and propose several priorities for ongoing thinking on reflection in applied visualization research.