Neural networks are a prevalent and effective machine learning component, and their application is leading to significant scientific progress in many domains. As the field of neural network systems is fast growing, it is important to understand how advances are communicated. Diagrams are key to this, appearing in almost all papers describing novel systems. This paper reports on a study into the use of neural network system diagrams, through interviews, card sorting, and qualitative feedback structured around ecologically-derived examples. We find high diversity of usage, perception and preference in both creation and interpretation of diagrams, examining this in the context of existing design, information visualisation, and user experience guidelines. This interview study is used to derive a framework for improving existing diagrams. This framework is evaluated through a mixed-methods experimental study, and a ``corpus-based'' approach examining properties of published diagrams linking the framework to citations. The studies suggest that the framework captures aspects relating to communicative efficacy of scholarly NN diagrams, and provides simple steps for their implementation.