Inclusion and accessibility in visualization research have gained increasing attention in recent years. However, many challenges still remain to be solved on the road toward a more inclusive, shared-experience-driven visualization design and evaluation process. In this position paper, we discuss challenges and speculate about potential solutions, based on related work, our own research, as well as personal experiences. The goal of this paper is to start discussions on the role of accessibility and inclusion in visualization design and evaluation.