On peer reviewing: how to nourish an author’s mind and win a JLDHE editor’s heart

Editors and publishers of scholarly journals rarely agree on what makes for a good publication; they do, however, agree on the need for a robust peer review process as a crucial means to judge the merits of potential publications. While fraught with issues and inefficiencies, a critical and supportive peer review is not only what editors rely on when assessing scholarship presented for publication but also what authors hope for in order to improve their work. Understanding how peer review may best serve all parties involved – authors, editors, and reviewers – is thus at the heart of this article. The analysis offered here is based on a session the Journal for Learning Development in Higher Education editors gave at the 2020 LD@3 seminar series, entitled ‘The art of reviewing’. It explores the different aspects of the peer review process while formulating recommendations regarding best practices and outlining JLDHE initiatives for supporting reviewers’ vital work.


Introduction
As editors at the Journal for Learning Development in Higher Education (JLDHE), we annually process scores of submissions, and when it comes to providing a streamlined publishing experience for all the parties involved, we recognise that we achieve varied levels of success. The difficulties are often the result of time constraints and unforeseeable delays; in an ideal system, intrepid editors would communicate most efficiently, determined reviewers respond to requests promptly, and inspired authors revise and resubmit their papers enthusiastically. Alas, in reality, this editorial operation rarely goes to plan. The publishing process, more often than not, is a series of hiccups and delays, not to mention a lot ofat times futilewaiting. In view of its built-in peculiarities and impediments, why maintain the system of editorial oversight and peer review at all?
Since the mid-twentieth century, peer review has been an integral part of the journal publication process. The review's role is, firstly, to provide an editor with an expert opinion on whether a submission should be published (with or without revisions); and secondly, to offer the author constructive criticism on how their work can be revised and improved. The majority of all journal submissions are subject to various degrees of revision, and reviewer input is widely seen as essential to ensuring the quality of the pieces published (Fytton, 2002). In essence, reviewers are an intrinsic element of the publishing process and identifying means by which they can be best supported to play their role effectively has been at the forefront of our mission at the JLDHE.
Based on a workshop delivered by our editorial team of as part of the LD@3 series (JLDHE, 2020a), this article will explore the role reviewers play in the journal publication process and how publishing protocols can be fine-tuned in order to not only increase efficiency in the journal but also energise the entire community. It will probe what constitutes good reviewing practice in the field of learning development and how reviewers, authors, and editors can work together in the most productive and rewarding way. In this spirit, we will also briefly outline planned developments in the support available to reviewers through the JLDHE and explain the priorities that form our ethos and the mission of the journal. Loughlin, Syska, Sedghi and On peer-reviewing: how to nourish Howell-Richardson an author's mind and win a JLDHE editor's heart Journal of Learning Development in Higher Education, Issue 19: December 2020 3

The role of reviewers in the publication process
Being essential to the publication process, reviewers make an appearance at its various stages. Even before they get involved, however, the editor first needs to decide whether a submission should be sent for review at all. A JLDHE editor will thus examine the submitted piece carefully and, if necessary, undertake preliminary discussions with the authors, for example when their submission does not fully meet the journal's requirements or requires additional work to bring it to publishable standard. While not a modus operandi of most academic journals, this stage is important to us at the JLDHE as it provides more scope to investigate cutting-edge topics and more opportunities to increase writers' confidence, especially that of novice authors. This stage of the process is shown as the opening phase of our peer review process in the flowchart below:

Selection of reviewers
In most journals, once an editor deems the submission ready to be seen by a peer reviewer, they usually recruit two or three potential reviewers (Fytton, 2002), based on these reviewers' knowledge and experience with regard to both the remit of the journal and the specific focus of the submission. For example, for subject-specific journals, an editor will seek reviewers with knowledge or expertise in the particular branch of the discipline covered in the submission (Badar, 2006). For journals focused on the research process, the editor may concentrate more on identifying reviewers familiar with particular methodologies and methods (Donmoyer, 1996). Many journals, like the JLDHE, have a broad remit and the reviewer selection criteria can vary depending on the subject and format of the submission, which in the case of the JLDHE can range from articles based on empirical research to critical commentaries and reflections (JLDHE, 2020b). The extent to which editors are able to 'match' reviewers with authors is debateable, given the diversity of viewpoints even within the most specialist fields (Female Science Professor, 2011). Also, given that a piece worthy of publication should be accessible to the diverse readership of a journal, a reviewer with knowledge of the broader context or a related, rather than the exact same field, is often equally well placed to review a piece. In other words, the editor is looking to identify reviewers able to evaluate not only if a submission is a good example of research or investigation in the field but also whether it is appropriate for the remit of the journal (Donmoyer, 1996).
The ways in which journals recruit reviewers vary greatly, with most editors directly approaching known experts, often with a solid record of publishing. This strategy can ensure both subject specific knowledge and familiarity with the literary conventions in the field. Knowledge of experts in a field means an editor can recruit a range of reviewers including not only those who they believe will be sympathetic to a piece but also those whose portfolio suggests they might offer a useful critical or alternative perspective to help develop the writer's argument. Such an approach will garner a wide range of opinions, of particular value to an editor who might be uncertain about the scope or suitability of a submission (Donmoyer, 1996). technologists, senior management, and specialists in educational and learning development. Evaluation of subject matter and of the strength of an argument across disciplinary boundaries is challenging (Becher and Trowler, 2001). Epistemic and intercultural differences combine with differences in academic writing conventions, which can result in failure to recognise an author's argument or their contribution to knowledge (Lea and Street, 1999), particularly so if the paper represents a divergence from established and prevalent discourses (Gallo et al., 2016).
It is evident to most editors, however, that reviewers' expertise is not a guarantee of a smooth peer review process and the traditional method of selection produces mixed results. After contacting potential reviewers, editors tend to wait for two or three weeks for the review, or even just a confirmation that the reviewer will undertake it, which regrettably does not always arrive. In addition, even if such designated reviewers respond promptly, the quickly assembled opinions sometimes fall short of the developmental and supportive reviews the authors hope for. Communication failures, busy schedules, and endless other impediments contribute to this inefficient system that editors at the JLDHE decided to address. In order to respect the availability and flexibility of reviewers as well as to enhance the engagement of potential and existing reviewers, a new method for reviewer recruitment has been initiated in our journal, whereby an open call for reviewers is sent out via the three main JISC mail lists for educational and learning development (LDHEN, SEDA, and EATAW). The call includes the submission abstract and a request for potential reviewers to provide an outline of their interest and expertise in the subject. As such, these calls appeal to the particular interests of the reviewers in the community while also encouraging new members to engage as reviewers, adding an element of communitybuilding. At the same time, for an editor, this method produces a degree of uncertainty as the volume and type of responses can vary greatly. This uncertainty is heightened if the potential reviewer is not an active author themselves.
Indeed, one question that editors often grapple with, and which was also raised in the LD@3 workshop, is whether it is necessary for a reviewer to be a published author (JLDHE, 2020a). Although for some journals this can be a criterion, and being able to advise novice reviewers to draw on their experience of being recipients of reviews can be useful for editors (Female Science Professor, 2011), for fields where the proportion of published authors is lower, this would only serve to limit the number of potential reviewers. In the case of the JLDHE, it should also be noted that as learning developers or those working in associated fields, a high proportion of the journal's reviewers have the advantage of being aware of what constitutes good feedback in terms of both content and tone, and are able to apply this in their approach to reviewinga factor very much evident from the quality and depth of the discussion in the LD@3 workshop.
The issue of authorship aside, there are tremendous advantages to the self-selection process practised by our journal. As already indicated, in the model where editors select and approach potential reviewers, a reviewer may agree, through a sense of obligation, to review an article that they have little interest in. This can result in a superficial or cursory review. In contrast, those responding to an open call are expressing a keen interest in the topic, which often results in a prompt and fastidious review. This in turns makes a world of difference to editors, who are dependent on reviewers submitting reviews by agreed deadlines (Fytton, 2002). As outlined in the flowchart above, review is a multi-stage process and sticking to a timescale is important in ensuring the timely publication of articles and the continued engagement of authors who may suspect there are problems with their submission if they do not receive reviewers' comments by the expected time.
Because of the importance of this, a reviewer's track record of meeting deadlines is often taken into account by editors when selecting reviewers (Badar, 2006). This is, however, less likely to be an issue when potential reviewers are responding to an open call as their capacity to engage with the process is a factor they will take into account before volunteering for the task. Indeed, responders to the JLDHE open calls often include details of their availability, particularly if it is limited or intermittent. Finally, a further challenge faced by editors who approach reviewers is building up a pool of potential reviewers (Badar, 2006) and in particular, maintaining it with up-to-date information. In contrast, an open call can reach a wider group of potential reviewers and is not reliant on an editor's personal networks or their awareness of those currently working in what might be very specific fields.

Journal guidance for reviewers
Whether it is via a covering letter from the editor, information provided on the journal's website, or a list of specific questions reviewers are asked to engage with, an important part of ensuring good reviews is the editor providing clear guidance about the remit of the journal and the areas the reviewers should consider (Weller, 2001;Fytton, 2002). These are often presented as thematically grouped questions, as is the case with the JLDHE, where the groupings are: 1. Contribution to the field (including relevance to the journal).
3. Methods and methodology (where appropriate for the type of submission).

Style and language issues.
The JLDHE review forms, customised to different types of submissions, are made available to reviewers as an electronic form on the OJS platform with a series of prompts, each followed by free text boxes (see Appendix for list of JLDHE themes and prompts).
The thematic questions are followed by a further request for reviewers to provide any other comments to support the author, and an option to share private comments with the editor.
The former provides an opportunity for a reviewer to add summary comments or prioritise the revisions that they believe need to be made, while the latter is a chance to open a dialogue with the editor if they have concerns or questions. The reviewer is also asked to indicate whether they think the submission should be accepted, with or without revisions, or rejected. An alternative to providing reviewers with a standard form is for an editor to give specific guidance through a tailored covering letter sent to reviewers. Where editors employ this method, they often highlight particular areas or questions they would like the reviewer to focus on. Although this can be very useful in helping the reviewer frame their response, the editor can run the risk of being overly directive or prescriptive (Donmoyer, 1996).
Even when asked to comment on particular areas or themes, reviewers may still have questions about their role. These are often around the level of detail and tone of the response required. For example, a reviewer concerned about formatting or language may wonder about the extent to which they should address this in the review without slipping into the role of copyeditor. An effective way to address reviewers' concerns about their role and promote good practice is through training focused on the publishing process (Weller, 2001 2019). This approach was at the heart of JLDHE editors' effort to provide a supportive workshop, although given the time constraint of the LD@3 session, an alternative format was used (JLDHE, 2020a). The editors mapped three fictional reviews onto the thematic sections of the JLDHE review process (JLDHE, 2020b), in the form of a table that allowed for easy comparison of the quality of responses as these fictional reviews varied in tone, level of detail, and length. The participants were divided into small groups and asked to discuss the reviews, identifying the strengths and weaknesses of the examples. From this, they then explored what makes a review useful for both an editor and an author. Following small group discussions, the individual groups shared key points with the rest of the participants. The following sections expand on the themes that emerged from this activity, focusing on what constitutes a good review, followed by a discussion of the value of peer reviews to both our authors and reviewers, as well as to us as editors.

What constitutes a good review?
A review can be a powerful and intimidating instrument in the publishing process and many authors, particularly early career or novice authors, can find it challenging to engage with and to respond to reviewers' comments (Female Science Professor, 2011). When asked to sum up their perceptions of reviews from an author's point of view using a single word or short phrase, participants in the LD@3 workshop proffered 'scary', 'demoralising', 'worrying', 'confusing', and 'mountain to climb' (JLDHE, 2020a). It has further been suggested in the literature that some authors may perceive a request for revisions accompanied by highly critical and negative reviewer's feedback to be at best discouraging and at worst a thinly veiled rejection, resulting in authors not re-submitting revised versions of texts (Martín et al., 2014). It is therefore not surprising that how to respond to reviews is a theme often discussed in advice for academic authors and has been shown to be an area that should be incorporated into training programmes for academic writing (Farley, 2019;Martín et al., 2014).
For an author, the article they submit for publication can represent the culmination of a reviewers' comments has shown that when an author engages, a submission is improved (Weller, 2001;Fytton, 2002). Indeed, a review can have a wider impact on an author than just supporting them in revising a particular article. In reporting on the strategies used by multilingual academics to learn how to write academic articles in English, Martín et al. (2014) found that one of the most commonly cited strategies was to engage with comments and feedback from editors and peer reviewers. This emphasises the importance of constructive feedback not only in identifying specific problems with content and methodology of an article, but also in enabling greater understanding of expectations and linguistic conventions in the field.
In the workshop, the importance of a reviewer being impartial and not connected to the author was also highlighted as a concern of authors. The double-blind review process at the JLDHE is aimed at ensuring the anonymity of both the author and reviewer, and enables both to make judgements based purely on the text of the submission and review.

The value of reviews to authors
The quality of a journal to a large extent depends on the quality of reviews offered in response to the submitted manuscripts, a premise rarely questioned by the parties involved in the process, including authors, editors, and reviewers. In fact, three-quarters of researchers consider a robust peer review process more important than a journal's general reputation when deciding where to publish, and 90% see peer reviews as essential to their professional development (Nicholas et al., 2014). Being a crucial means to improve an article by giving an author the necessary support and guidance regarding potential revisions, a good review contributes vitally to the quality of research, scholarship, and practice in higher education.
Working from the perspective of an author, the participants in the LD@3 workshop identified a set of characteristics of a good review, which largely align with the good practices recognised in the literature. In particular, it was highlighted that authors need constructive criticism delivered in a supportive rather than dismissive or judgemental tone.
The style and tone of the feedback must be respectful and developmental; a patronising tone will not encourage authors to enhance their article. Reviewers act as consultants and coaches who put their own expertise into practice and advise authors professionally as to Loughlin, Syska, Sedghi and On peer-reviewing: how to nourish Howell-Richardson an author's mind and win a JLDHE editor's heart Journal of Learning Development in Higher Education, Issue 19: December 2020 11 how their article qualifies for publication (Rochitte and Mesquita, 2018). Participants agreed that a good review provides authors with expert advice and they rely on the reviewer to critically assess an article, identify the areas of improvement, and provide the author with professional suggestions as to how to advance the manuscript. Feedback should cover both overarching issues, for example, the structure, argument, clarity or progression of ideas, as well as subject-specific themes or sections, in a detailed and coherent way.
In a discussion of the examples provided in the workshop, participants were able to focus on the level and types of detail that should be included in a review and concluded that a good peer review should be sufficient to enable the author to consider ways forward. This may involve referring to specific areas of the text or a particular example of the type of improvements to be made. Some reviewers may even decide to include an annotated text to indicate which sections they are referring to and which need further attention. How much detail a review should contain depends on the type and quality of the article; the key aim and intention is to be specific and clear about what the author needs to do to improve the manuscript.

The value of reviews to editors
Good reviews are essential to maintaining high editorial standards and editors rely on reviewers' expertise and recommendations to help them make a decision about whether to accept or reject a submission; indeed, the literature strongly suggests that this decision should 'be constrained by reviewers' judgments' (Donmoyer, 1996, p.23). If reviewers are unanimous in their verdict and this aligns with the view of the editor, then this is a given.
When reviewers disagree, however, or their verdict goes against editorial opinion, the editor is faced with a challenge. Although Donmoyer recommends always closely following reviewers' advice, at the point of conflict, the quality of the reviews is of paramount importance. Whether supportive or critical of the submission, the reviewer who has provided the most detailed argument with reference to particular areas of the text will be the one the editor will likely follow; on the other hand, when weak reviews do not offer enough meaningful argument, making the final decision will remain the editor's prerogative (Donmoyer, 1996). The editor may also consider discussing the issue with the author, thus offering them an opportunity to respond to conflicting reviews and strengthen their case for publishing the work.
A good review can also help clarify and refresh the editor's thinking around not only the particular submission but also the criteria used to evaluate further submissions (Donmoyer, 1996). At the JLDHE, editors constantly refine their approach to the different types of research published and fine-tune the editorial process to best serve both authors and readers.

Benefits for reviewers
Reviewing is often seen as a thankless job and not all participants in the process are aware of the amount of work, the time pressures, and commitment involved. Being a reviewer for journals that employ open peer review (as opposed to blind peer review) can serve to increase respect in the field and raise one's professional profile (Female Science Professor, 2011). As noted in the LD@3 workshop, this incentive of professional recognition is absent in journals where reviewers remain anonymous to both authors and readers (JLDHE, 2020a). In such circumstances the incentive to be a reviewer is limited to the opportunity to learn, both about academic writing and current work in the field, and to make a positive impact on the literature in the field (Fytton, 2002;Female Science Professor, 2011). For more experienced authors, reviewing is on a quid pro quo basis as they in turn are recipients of reviews (Fytton, 2002), while for others the personal reward comes from being able to support colleagues in the sector.
Having reflected on the most rewarding approaches, we changed our processes at the JLDHE, incorporating a range of initiatives to support our reviewers and reinforce their critical role in the success of the journal. As an example, we include lists of reviewers in our acknowledgements section to recognise their contribution to the quality of papers we publish. Once the peer review process is complete, we also share the individual reviews among all reviewers involved in the particular submission, so less experienced reviewers can learn from the expert knowledge of their colleagues. Finally, we support our reviewers through webinars such as the one we are discussing in this article, as well as developing further resources.

Conclusion: a way forward
The LD@3 webinar and the discussions it generated reinforced for us the need to support both authors and reviewers by providing continuous professional training and best practice guidance regarding peer review. As one of the participants wrote in response to the session: 'I appreciated being able to talk about some of my experiences; I didn't think as a first-time author I'd have much to contribute . . . it would be useful to do another one'.
Clearly, the webinar identified the benefits of community conversations that demystify the publishing process, and offering more such sessions and resources for reviewers has become our key objective, particularly in light of developing the reviewer's zone on the JLDHE website.
We acknowledge that writing a good review can be challenging for anyone. A published author may lack confidence in their ability to review (Female Science Professor, 2011) and even an experienced reviewer may struggle with how to respond to a particular submission. In providing training and support to reviewers it is therefore important to recognise that both experienced and novice reviewers, regardless of their experiences and expertise, can benefit from such support.
The JLDHE editorial team aims to address this by building on the model employed in the LD@3 workshop to develop online training (subsequently available as webinars) for those new to reviewing. Among the numerous training models available, we see most benefits in the two-stage model, which divides training into an orientation session, followed by a practical workshop. The orientation session serves as an introduction to the publishing process, focussing on understanding the sequence of events from article submission to publication, buttressed by a discussion of the reviewer's role and some of the dos and don'ts of reviewing. After this initial session, participants are ready to engage with the review process itself. Thus they are provided with two short articles to review in preparation for the second workshop, which focuses on the reviews they have completed and explores the importance of providing constructive and encouraging feedback.
Adopting this model will allow participants of our workshops to gain an understanding of the role of a reviewer in the publishing process, alongside acquiring firsthand experience of reviewing in a 'safe' environment where mistakes are not only expected but also provide the necessary fodder for productive discussion. In order to reinforce the good practices learned in this training, participants will be offered the opportunity to apply their skills by serving as the third reviewer on a JLDHE article.
Here is where our practice of sharing complete anonymised reviews among reviewers will be particularly beneficial to the trainee reviewers, as it provides an opportunity for further insights to be gained from the process. In addition, the editor looking after the submission will be available to answer questions and provide feedback on their review.
A dedicated area on the JLDHE website will highlight not only the training and support available to novice reviewers but also the fact that the editors are able to provide support and feedback to all reviewers. It is hoped that through enhanced support and training, more potential reviewers will feel that they have a way into becoming community reviewers, while experienced reviewers will be reinvigorated to engage with the process as a developmental activity, rather than just a chore.
While the peer review process is not infallible and can be problematic due to issues such as reviewer bias or limited effectiveness in identifying all innacuracies in the content, it has also been shown to constitute the most critical pillar in publishing, when it comes to the trustworthiness and reliability of scholarly journals (Nicholas et al., 2014). Peer review is currently the most crucial indicator of quality, intellectual rigour, and authority of a journal, and the JLDHE will strive to ensure that our peer review practices, bolstered by robust support for reviewers, continue to build that trust within the discipline of learning development and its professional community.