CUNY Graduate Center brought together five DHers to discuss “Evaluating, valuing, and promoting digital scholarship.” Where is the place of digital scholarship in the Teaching-Research-Service Triangle? Although everyone added to the discussion from their unique disciplinary perspectives, not all were able to propose methods for evaluating digital scholarship?
Steven Jones answered the question by pointing to projects he has been apart of that publish digitally. He began with the Board of Nines (nines.org), an aggregator of peer-reviewed 19th century scholarly work, and Romantic Circles, an online venue for digital peer-reviewed scholarship. How is the peer review done in this online space? Jones says that by replicating standard peer review plus taking into account the medium in which is was created and innovative features in the work. “The key is in the markup.” But even he seemed stumped as to how proper evaluation and academic equivalencies could be given to the Villa Diodati MOO, a chat room that led way to a interactive game experience.
Sonia K. Gonzales presented her ongoing research comparing information retention between her public health websites. It is very interesting to hear input on the conversation from the perspective of health sciences. The question left unanswered was how the digital component of the survey research, the web page she designed, would be preserved for later use or evaluation.
Amongst the panelists, there was a divide between those who saw digital scholarship as a supplemental component to traditional scholarship those who want the digital scholarship to replace traditional scholarship. To look at the example provided by Elizabeth Macaulay-Lewis, she had Archaeology students from one class create digital object reports and then create condensed lessons for Khan Academy. By creating these videos to be shared online, they created digital scholarship. But does the 2-minute video and lesson equate to the long-form object reports or is it extra (perhaps superfluous) work only to complement the reports? To faculty, would this work be teaching? service? Another example Macaulay-Lewis used to further the question was the Manar Al-Athar online gallery of the archaeological site. In this case, a poster was presented, but otherwise there is ambiguity on how digital projects like a gallery would be critiqued.
There are protocols for how to evaluate virtual work. The Archaeological Institute of America published a white-paper to guide committees on reviewing digital scholarly work. Chris Alen Sula also pointed to the Modern Languages Association and the American History Association and their guidelines to digital evaluation. Even with variance between the disciplines, there is good advice that could apply to any dimension of the humanities. Amanda Visconti, a recent graduate whose dissertation is a digital project, says to document your progress thoroughly through a program like Github, Basecamp, and weekly e-mails to advisors.
What digital humanists must think about in the early stages of their works is how format will affect (and enhance!) the project. The DH community should follow these practices to ensure best evaluation of their work:
- Choose the appropriate format (as authors)
- Respect the medium (as reviewers)
- Document everything (as authors)
- Clarify expectations and rewards (as authors and reviewers)
Humanities has not always been concerned with replicability in its work. With digital scholarship, it is especially important that work created digitally can be replicated/accessed/viewed. Digital scholarship also creates a demand for humanists to have data literacy skills. Because this scholarship typically extends outside disciplines, those with technical expertise should be tapped to join review boards in the humanities. An alternative model suggested is a self-evaluative model where researchers are transparent in their decisions and documentation. In a way this already exists online, but outside of academia. The major barrier to digital work is not proper evaluation, but ensuring the evaluation comes from a respected source.