Tag Archive for: assessments

MONTHLY BLOG 81, RESPONDING TO ANONYMOUS ACADEMIC ASSESSMENTS

If citing, please kindly acknowledge copyright © Penelope J. Corfield (2017)

(*) This BLOG follows its matching BLOG/80 (Aug. 2017)
on ‘Writing Anonymous Academic Assessments’

2017-08 No3 handshake diagram

The first arrival of anonymous assessments of one’s own research is almost invariably annoying. There’s something about the format which gives the author-less verdict a quality of Olympian majesty. And, even if the verdict is favourable, there’s a lurking feeling that one is a mere minnow, being condescended to by a remote and all-wise deity. Ouch!

However, after recovering from one’s initial fury, it’s best to rally and to view the whole exercise as a free consultation. Instead of rushing into print, and getting a stinker of a review, the stinker is delivered in the form of an anonymous assessment before publication. The anonymous critic is, in fact, the best friend, lurking in disguise.

As well as writing constant assessments, academics also read one anothers’ work in typescript. But, as researchers say, ‘good criticism is hard to get’. Many friends just respond loyally: ‘Darling, it’s wonderful; but there’s a typo on page 33’. Such a reaction is not much use. In the case of an anonymous assessment, by contrast, someone has gone to a lot of trouble to identify all your faults. And, what’s more, to give you a chance of remedying them before publication.

On balance, I would say that 80% of all the anonymous advice, which I’ve received over the years, has been invaluable. Another 10% is comparatively trivial, meaning either that the assessor has been sleeping on the job or (rarely) that there’s nothing major to criticise or discuss. But 10% of responses are positively unhelpful, either through being too crushing – or simply irrelevant.

One example of off-the-wall and unusable reflections concerned my editorial introduction to a book of essays entitled: Language, History & Class (1991).1 The anonymous assessor said firmly that I was wrong; and offered, at some length, his/her own philosophical alternative historical/linguistic theory as a variant. In one way, it was a very generous piece of writing. But, on the other hand, it was entirely wasted. I couldn’t use the alternative view, because I disagreed with it – and anyway, it wouldn’t be either right or politic to take someone else’s original thesis as my own, whether I agreed or disagreed. Something in my text had apparently rapped the assessor’s intellectual funny-bone, causing him/her to get distracted into inventing a new theory rather than reviewing a book proposal. The alternative approach was so off-the-wall that I never saw it appear anywhere in print. It was an intellectual kite that never flew.

2017-09 No2 kite in trees

Generally, however, after the first moment of silent fury at reading the anonymous assessor, I buckle down and enjoy the chance to revise in the light of a really in-depth analysis. Often, rewriting helps to strengthen my arguments, giving me a chance to rebut criticisms explicitly. And, simultaneously, the rewrite allows scope for clarification, if ideas were poorly or incompletely expressed first time round. Sometimes points have been made out of their logical order and need reshuffling. And finally, I sometimes (not too often!) change my mind, in the light of criticisms; and the process of rewriting allows me to push my argument into new directions.

In reporting subsequently to the publishers or editors, who have commissioned the anonymous assessment, there is one golden rule. The criticisms do not have to be adopted wholesale. But they must be acknowledged, not simply dismissed. I remember one former PhD student, when editing her first essay for a learned journal, miserably wondering whether she had to ditch her entire argument, in the light of a critical assessment. I was horrified at the prospect. Of course, she had to stand by her new interpretation. (She did). The essay would appear under her name and must therefore represent her considered views. An adverse anonymous assessment does not have the status of a royal command. Instead, the hostile cross-fire gives authors a chance, pre-publication, to decide whether to strengthen or to adapt their arguments.

Then it’s up to editors to decide. Usually they appreciate the chance to get new views into print, with the prospect of opening up further debates. But editors do like to be reassured that the revisionist piece has been submitted knowingly, with a full awareness of the potential controversies to follow, and that the study is well argued and substantiated. In comparatively rare cases, when challenging new views are rejected by one journal, there’s a reasonable chance that the ‘new look’ can find a home elsewhere. Since historical research relies upon debate and disagreement, it’s not such a big deal to find one (temporarily) prevalent view coming up for critique and/or complete refutation.

Only in very rare cases are anonymous assessors unduly harsh or vitriolic. I’ve had plenty of negative responses myself but never anything without some constructive aim or intention. One hostile case, however, occurred in response to a former student of mine who had written an excellent essay on the social history of nineteenth-century Sussex. Some element of the argument had apparently infuriated the anonymous assessor. He/she basically argued that the essay should not have been written. There was nothing constructive upon which the author could build. Fortunately in this case, the journal editor had asked for two anonymous assessments. The second was much more positive, enabling my former student to revise the essay into a stylish contribution. However, I advised her to write to the editor, explaining calmly that she had considered the negative assessment carefully before disregarding it. The fact that the angry assessor’s report had mis-named ‘Sussex’ throughout as ‘Suffolk’ suggested that the tirade was not based upon a very close reading. The editor took this strong hint on board; and the revised essay successfully appeared in print.2

These examples indicate the intricacies of peer review and the publication process. They are socially imbedded – and far from purely impartial. But they strive for an interactive collegial process, which seeks to iron out individual rancour or prejudice. Personally, I take anonymous academic assessments of my embryonic work as seriously as I expect my own anonymous academic assessments to be taken by the anonymous recipients. The veil of secrecy strives to make the exchange of ideas a ‘pure’ intellectual exercise, without the formal courtesies and pleasantries. (Actually, if one wants, it’s usually possible to make a stab at identifying the critics, using one’s research-honed powers. But in my experience, that’s an unproductive distraction).

Scholars who are published in peer-reviewed outlets are thus in constant dialogue (or, preferably, ‘plurilogue’),3 not just generally with their peers, and patchily with their precursors in earlier generations but specifically with their specially recruited anonymised critics. Wrestling with obdurate drafts is often exasperating and lonely work, as Hogarth knew – as seen in a detail from his Distrest Poet (c.1736) below. Yet scholarly authors don’t work in isolation. A tribe of anonymous academic critics, friendly readers, and interventionist editors/publishers are looking over their shoulders. So it’s best to bite the bullet; to revise coolly; and then to publish and be damned/whatever.

2017-09 No3 Hogarth's distressed_poet

1 P.J. Corfield, ‘Historians and Language’, in P.J. Corfield (ed.), Language, History and Class (Oxford, 1991), pp. 1-29; slightly amended text also transl. into Greek for publication in Histor, 12 (May 2001), pp. 5-43.

2 A. Warner, ‘Finding the Aristocracy: A Case Study of Rural Sussex, 1780-1880’, Southern History, 35 (2013), pp. 98-126.

3 For this usage, see P.J. Corfield, ‘Does the Study of History ‘Progress’? And does Plurilogue Help?’ BLOG/61 (Jan. 2016), in www.penelopejcorfield.com/monthly-blogs.

For further discussion, see

To read other discussion-points, please click here

To download Monthly Blog 81 please click here

MONTHLY BLOG 80, WRITING ANONYMOUS ACADEMIC ASSESSMENTS

If citing, please kindly acknowledge copyright © Penelope J. Corfield (2017)

(*) This BLOG will be partnered
in September 2017 by a matching BLOG
on ‘Responding to Anonymous Academic Assessments’

2017-08 No1 black mask 
Writing anonymously encourages a certain acidity to emerge. Instead of the conventional politeness (‘Does my bum look big in this?’ No … not really’), it seems at first that the unvarnished truth will break through (‘Yes, it does’). In fact, however, there are multiple reservations to be made about that first rush of apparent candour. It’s very like the caveats that need to be made to that drinker’s favourite maxim: ‘in vino veritas’. Well, yes, sometimes. But there is also scope for exaggeration, melodrama, and error, as well as anger, bile, and crudity, within every alcohol-fuelled tirade.

The psychological mechanism of anonymous writing is ‘release’ – release from the conventions of politeness and, especially when writing in a hurry, release from the normal constraints of prudence. It’s like a rush of blood to the head. And it can easily become addictive. Probably a considerable proportion of people who unleash a tide of vitriol anonymously via the new social media surprise even themselves by their ferocity and lack of inhibition. Thus when confronted with the real person behind their on-line target, a number of Twitter trolls have apologised abashedly.1 These anonymous critics have been living in a little bubble of self-created alternative reality. The power of expressing anger-at-a-distance, from a position of apparent immunity, seems hard to resist. It’s as though thousands of previously unknown madcap Mr Hydes have been electronically released from within thousands of normally conventional Dr Jekylls. Yet, as in Stevenson’s fable, the split isn’t real. Jekyll and Hyde are one, each persona having responsibility for the other.2

2017-08 No2 jekyll and hyde

Happily, very few academics have divided personalities that would score very highly on the Jeykll/Hyde range. Or at least they restrain themselves from going ape in their capacity as examiners. That’s no doubt because they are thoroughly trained in a degree of self-control through their regular experience of anonymous assessment. These days, it’s usual for the names of examiners to be anonymised, as are the examination scripts which they mark. That is rightly done in order to avoid cronyism, favouritism, and unconscious biases.

And in cases where the examiners’ identities are known (for example when marking small specialist courses), it’s usual for scripts to be double-marked, before the two examiners meet to decide upon a joint mark – all subject to the controlling overview of a third external examiner (from another academic institution or at least another department), who is available to decide if the examiners can’t agree. Examinations are thus safeguarded against the handiwork of an impetuously unbalanced Mr Hyde.

It’s more tempting to let rip, however, when making individual anonymous assessments, for example when reviewing manuscripts for academic journals, or for publishers, or for the award of academic prizes/grants. There’s a whole behind-the-scenes world of what is known as ‘peer review’. Editors or publishers or prize-givers can make preliminary assessments of work submitted to them. There’s a lot of initial weeding. Yet they need specialist help to assess specialist research, especially in highly technical subjects. That’s where the anonymous assessors come in. Almost all academics spend a considerable amount of time on this sort of technical labour, often without any extra fee. It’s done pro bono, for the wider good of scholarship. Assessors are prodded with a series of questions: is this work original? is it properly substantiated? what changes are needed to make it publishable? But, at the same time, assessors are invited to write with freedom, hence risking a rush of blood to the head.

Interestingly, many early book reviews were written anonymously. The sting of a hostile notice was worsened by the author’s ignorance of the perpetrator of the barb. In the early nineteenth century, for example, when the astringent Edinburgh Review paid very high fees (up to 20 guineas a sheet) for strong opinions, one eminent literary victim characterised the journal’s anonymous reviewers as the ‘bloodhounds of Arthur’s Seat’.3

Since then, the fashion has swung decisively in favour of signed reviews when those appear in public. These days, academic authors who have laboured to draft an earnest encomium or a pointed critique need to get acknowledgment for their work, to show that they are not slacking. For many years, the major redoubt of anonymous reviews was the Times Literary Supplement (launched in 1902). An insider-academic game was trying to guess who had written which waspish put-down. I remember that, whenever anything particularly acerbic appeared, senior Oxford dons would murmur knowingly ‘Ah, Hugh Trevor-Roper again’,4 even if it wasn’t. Students were often impressed, while laughing secretly at all the fuss. In fact, the pages of the TLS were rarely dripping in authorial blood; and, when reviewer anonymity was dropped from 1974 onwards, the journal sailed onwards serenely without much change in tone.

That leaves anonymous assessment as the chief remaining terrain for academics to pontificate without acknowledging their handiwork. Supreme power at last? But no. Behind-the-scenes assessments are delivered within a range of unstated conventions requiring academic fairness and balanced judgment – especially when bearing in mind that all seeking to publish in a peer-reviewed outlet are equally liable themselves to be at the receiving end of one or more anonymous assessments. (See my next BLOG).

For me, writing such verdicts constitutes a specialist form of conversation-at-a-distance. Thus anonymous assessments are usually brisk and direct. There’s no need for the normal interpersonal courtesies of a face-to-face encounter. (Often indeed the original author’s name has also been anonymised). So there is no need for shared enquiries about mutual health and wellbeing. But the one-way conversation still entails the assumption that ideas have to be explained clearly to a willing listener. In the event of disagreement, it’s not enough to write: ‘Rubbish!’ Instead, it’s necessary to spell out why particular arguments and/or evidence fail to convince. Assessors are also invited to correct outright errors; and, if a piece of research is only marginally publishable, to provide suggestions for required revisions.

As those requirements imply, it’s much the easiest and quickest to express total praise. It then takes longer to reject a piece outright, because the reasons for rejection have to be fully elucidated. But the longest and trickiest task is to assess research that’s on the margins of being publishable. It’s helpful to strike an initially positive note, appreciating the choice of topic and the effort undertaken. Yet the negatives have to be explained frankly too, complete with constructive advice on transforming negatives into positives. That’s a challenging task to undertake at a distance, without being able to discuss the details with the recipients. (I knew one hyper-sensitive colleague who was so annoyed by one anonymous critique that she refused to revise and resubmit a potentially important essay, on the grounds that the editors were wasting her time by deferring to such an idiotic and ill-informed assessor.)

Overall, the initial attractions of anonymity quickly disappear. Whatever the medium, communications don’t take place in a vacuum. They have social/legal/cultural contexts and they have consequences. So whenever I tap my keyboard, the best short motto remains the one that I and a group of frank-speaking friends chose for ourselves, one merry evening years ago: truth, yes; but, fundamentally, Truth with Tact. Note: Not tact instead of truth; but both. Fusion rather than Jekyll/Hyde-type fission.

2017-08 No3 handshake diagram

1 For an example, see Daily Mail on-line, ‘Shamed Twitter Troll makes Humbling Apology Live on TV to Professional Boxer he Abused for Eight Months after the Fighter Tracked him Down’, 14 March 2013: http://www.dailymail.co.uk/news/article-2293235/Curtis-Woodhouse-Shamed-Twitter-troll-James-OBrien-makes-humbling-apology-live-TV-professional-boxer.html

2 R.L. Stevenson, The Strange Case of Dr Jekyll and My Hyde (1886).

3 R. Watson, The Literature of Scotland, Vol. 1: The Middle Ages to the Nineteenth Century (Basingstoke, 2006), p. 253.

4 For H. Trevor-Roper (1914-2003), historian, polemicist and sometime anonymous author, see A. Sisman, Hugh Trevor-Roper: The Biography (2010).

For further discussion, see

To read other discussion-points, please click here

To download Monthly Blog 80 please click here