Those 300-word statements on Practice-as-Research for the RAE/REF – origins and stipulations – ‘academic butt-covering’ or more problematic?

I wrote that my last blog post on the issue of composition and performance as research constituted final thoughts on the subject, at least for now, but one issue has been on my mind which I wanted to clarify for myself and others. This was to do with the ubiquitous 300-word statements included with many submissions to the RAE or REF. In the public debate on the subject, Camden Reeves (1h 38’14”) spoke of an ‘artificial privileging of certain types of composition over others’ and described (1h 40’55”) the 300-word statements as ‘ridiculous’, going on to say that ‘at a lot of universities, the 300 [word] statement was seen as the kind of research report’. Alexander Lingas (1h 43’12”) asked the panel whether ‘by having the 300-word statement, that it privileges certain type of things, so that by doing that, you end up encouraging particular types of things which actually, I have to say, are very congenial to the type of musicologically-informed performance, because that’s precisely the type of thing that 300-word statements is good for to say – why do you make those wacky decisions when you perform this music? – well, it’s because, in 300 words, and so it’s a type of academic butt-covering’.

Miguel Mera (1h 45’35”) claimed that ‘there was no requirement to complete 300 words with any submission . . . you didn’t have to’. But this was far from clear from the REF 2014 report, which contained the following comments:

In brief, the additional 300 words to make further evident the research imperatives and/or research process of an output (paragraph 71(b) of the ‘Panel criteria’) were used inconsistently and the question of the research imperative was not always well-articulated. (p. 16)

As in 2008 the best outputs in PaR were distinguished by clearly articulated research objectives. In a number of instances, the presentation of practice needed no more than a well-turned 300 word statement to point up the research inquiry and its findings, since the concerns outlined were then amply apparent within the practice itself (which was made available for assessment by a variety of means including DVD or CD recordings, photographic materials, scripts and scores, databases, etc.). (p. 99)

More generally, the 300 word statements too often displayed a misunderstanding of what was being asked for and provided evidence of impact from the research, or a descriptive account akin to a programme note, rather than making the case for practice as research. (p. 100)

Scott McLaughlin, in his report on the debate, noted Mera’s comments but added that ‘I get the sense that many Universities insisted on them’. In Paul Allain and Jen Harvie, The Routledge Companion to Theatre and Performance (London and New York: Routledge, 2014), we find the claim that

all submitted PaR has to be accompanied by a 300-word statement outlining the research imperatives and context, further supported by other forms of evidence. (p.234)

Similarly, in Michael Biggs and Henrik Karlsson, ‘Evaluating Quality in Artistic Research’, in Biggs and Karlsson (eds), The Routledge Companion to Research in the Arts (London and New York: Routledge, 2011), in the context of submissions in art and design for RAE 2008, the authors write of ‘the additional requirement of a 300-word statement arguing for the significance and impact of each submission’ (p. 414). Robin Nelson, in his Practice as Research in the Arts: Principles, Protocols, Pedagogies, Resistances (Basingstoke: Palgrave Macmillan, 2013), refers to ‘a simple verbal articulation of the research inquiry – such as might be achieved in as few as 300 words’, which ‘proves useful in almost all cases’ (p. 11), not specifically mentioning the RAE/REF or a requirement, but likely written with this in mind.

So where did the idea come from that 300 words was a requirement? It was first presented in Section 3 of the RAE 2001 report on Publications, entitled ‘Panels’ Criteria and Working Methods’, and said specifically that:

3.58.8 Those submitting practice as research may include (in the ‘other relevant details’ field of form RA2) a succinct statement of not more than 300 words for each item in this category listed under RA2; for this purpose practice is defined as all outputs listed in paragraph 3.58.12 b) to e) below. These statements should make clear how the practice embodies research as defined in the RAE. They may, where appropriate, include an indication of the aims, methods, procedures, innovation, significance, and context of the practice. It should be noted that the submission of such statements is not a requirement; the Panel will not expect them when the status of the practice as research is self-evident.

The RAE 2008 guidelines on submissions contained the following text:

94. Brief, additional information may also be given in RA2 ‘other relevant details’ to identify relevant, factual circumstances concerning any output. It need not be supplied in every case. It may be, for example:

• to identify a keynote address to a conference
• to identify an invited conference paper especially where the perceived status of the conference is high
• to indicate the significance or impact of an applied research outcome
• to identify the research content or author’s contribution in edited works, translations, or co-authored works.

In the case of a non-text output, it may be used to give further information on the whereabouts of a work or to note that a photographic, electronic or other record exists. It may not be used to volunteer opinions about the relative quality of an output. See panel criteria statements for further guidance, including the word limit for this text, which, unless otherwise stated, will be 300 words.

The panel criteria for music within Panel O for RAE 2008 included the following:

Researchers should accordingly submit such evidence as they deem necessary to enable subpanel members to assess it within the following guidelines:
a. Research output: this may be submitted alone where it is deemed to constitute
sufficient evidence of the research in itself.
b. Statement: it is recommended that a statement of up to 300 words is submitted in the ‘Other relevant details’ field of RA2, in cases where the research imperatives and the research significance of an output (such as: an artefact, curation, digital format, installation, performance or event, screening, tape, textbook, translation or video) might further be made evident by a descriptive complement. The statement might include: a brief description of the project and its stage of development; a rationale outlining questions addressed; a summary of approaches/strategies undertaken in the work; a digest of further evidence (if any) to be found in sub-paragraph 13c below. As previously indicated, the 300-word statement should also be used to clarify the relative contributions of researchers working on a collaborative research project. The sub-panel will ignore any evaluative commentary on the perceived quality of the research.

Then for REF 2014, the statement of panel criteria and working methods included the following statement:

49. For non-text or practice-based outputs (including patents, software and standards documents), all subpanels welcome the submission of a description in
REF2 of the research process and research content, where this is not evident within the output (maximum 300 words), as described in ‘guidance on submissions’ (paragraph 127a). (p. 25)

Then for Panel D, with included music submissions, the following:

b. Information about the research process and/or content: Submitting units may include a statement of up to 300 words in cases where the research imperatives and research process of an output (such as an artefact, curation, database, digital format, installation, composition, performance or event, screening, tape, creative writing, database, textbook, translation or video) might further be made evident by descriptive and contextualising information. Where the location or medium of the output is essential to a proper understanding of the research being presented this should be explained in the 300 words. The sub-panels will ignore any additional material that includes evaluative commentary on the perceived quality of a research output (p. 87)

The 300-word limit was further reinforced on p. 99.

It is clear then that whilst the 300 words was not strictly a requirement, there was a very strong incentive to include them with most submissions. How many, reading the words from 2001, are going to view their submissions, or those of their department’s faculty members as having a ‘self-evident’ status as practice-as-research? How many reading the 2008 guidelines are going to go against what is ‘recommended’?

Nicholas Cook, who was on the Music Panel which worked towards producing the 2001 definitions, wrote about the process in his article ‘Performing Research: Some Institutional Perspectives’, in Mine Doğantan-Dack (ed), Artistic Practice as Research in Music: Theory, Criticism, Practice (Farnham: Ashgate, 2015) pp. 11-32. In the RAE 1996, the first such accepting performance submission, according to Cook:

Detailed criteria for the assessment of performance as research were not in place, and when the outcomes were announced there was a widespread perception that the assessment of performance submissions had been surprisingly generous. Given some unease on the part of 1996 panel members, and the expectation that this outcome would prompt a significant increase in performance submissions for the following exercise, a major priority for the 2001 Music Panel (which was convened well in advance of the submission date) was to set the assessment of performance as research on a more principled basis: this was necessary to provide the panel with clear and transparent procedures for its evaluation, to guide institutions’ decisions about what work to submit to the RAE, and to inform their future planning. This panel, which I chaired, for the first time included professional performers, though they were based in the academic sector. (p. 22)

The panel realised that simply mapping criteria from composition-as-research over to performance would be insufficient, not least because the former rested on nebulously defined notions of quality and there were new difficulties entailed in the acceptance of submissions of film and commercial music (the research qualities of the latter of which, I have argued elsewhere, appear to me more tenuous). Whether market success, recognition by peers, or ideals of originality were to be primary criteria all proved difficult in discussion, as Cook points out (pp. 22-3). This now quite notorious definition of research was adopted:

2.12. ‘Research’ for the purpose of the RAE is to be understood as original investigation undertaken in order to gain knowledge and understanding. It includes work of direct relevance to the needs of commerce, industry, and to the public and voluntary sectors; scholarship; the invention and generation of ideas, images, performances, artefacts including design, where these lead to new or substantially improved insights; and the use of existing knowledge in experimental development to produce new or substantially improved materials, devices, products and processes, including design and construction. It excludes routine testing and routine analysis of materials, components and processes such as for the maintenance of national standards, as distinct from the development of new analytical techniques. It also excludes the development of teaching materials that do not embody original research.

As for performance, the following definition was adopted:

3.58.12 d. Performances: in accordance with the RAE definition of research, performance will be accepted as research where it applies or embodies new or substantially improved knowledge or insights, for instance in terms of interpretation, historical performance practice, or technical innovation. Performance is understood to include conducting and direction as well as instrumental or vocal execution; all forms of public output are eligible for submission, including publicly disseminated live or studio recordings, broadcasts, and public performances. In the case of broadcasts and public performances, institutions must be able to supply a recording (which need not be in the public domain). Reference may be made to such factors as the venue of the performance, the standing of broadcasting organizations or record companies involved in its dissemination, and prizes or other marks of recognition); relevant information should be provided in the ‘Other relevant details’ field of form RA2.

This is, I believe, a basically good definition, though it is difficult to define when interpretation does or does not embody ‘new or substantially improved knowledge or insights’ (I would say most good performances do), and I worry about ‘the standing of broadcasting organizations or record companies’ acting as a proxy for judging work in terms of its standing in economies of prestige or market utility. But this is not strictly relevant to the 2001 stipulation about 300-word statements given above, about which Cook says:

That last remark was aimed at composers working in established styles: we did not wish to inflict a burden of pointless documentation on institutions – or on ourselves, for that matter. (The perhaps intellectually shabby idea of ‘self evidence’ reflects the pragmatic approach to composition I described.) Nevertheless the invitation to submit succinct statements excited considerable opposition from the subject community, not all of it reasonable in our view. In the formal processes of consultation that preceded the finalisation of the criteria, certain respondents rejected our assumption that the content of performances or other forms of practice as research could be reduced to words, and accused us of intending to assess the succinct statements rather than the performances. Of course we never made any such assumptions or held any such intentions. The point is very simple, and I have already referred to it. Academic writings are self-documenting. That is the source of the conventions of good academic writing to which Candlin referred. But this is obviously not the case of performances: as Susan Hellauer (1997) says, ‘You can’t sing a footnote’. It follows that you can’t expect an assessor to be able to reconstruct from a performance the research process that has given rise to it. (Think of David Milson submitting a performance arising out of his AHRC Creative and Performing Arts Fellowship.) Once more, this is the point on which the other members of the UKCGE [UK Council for Graduate Education] study group were insisting, except that they were talking about a 40,000 word dissertation whereas we on the panel were talking about a 300 word statement. By inviting those submitting practice as research to provide such a statement, then, we were giving them the opportunity to ensure the assessor understood the research component. We were empowering them to set the terms on which they were to be assessed, and to present their work in the best possible light. (p. 25)

I disagree with Cook; a sensitive listener with some familiarity with the work in question and performance practice might very well be able at least to assess, if not necessarily reconstruct in every detail, the research process which has given rise to one of David Milsom’s performances. A 40 000 word dissertation would certainly elaborate the process to a high degree, if done well, but I am not really sure that a 300 word statement could – or rather, if some point can be elucidated in 300 words but not clearly heard from the performance without such guidance, I would question the extent to which it is embodied in that performance.

An article from 2003 by Peter Thompson (‘Practice as Research’, Studies in Theatre and Performance 22/3, pp. 159-180) gave further details on how this debate was conducted amongst those involved in drama and related disciplines, reproducing correspondence. Drama Professor Franc Chamberlain made the following observation:

To argue that the work itself is best disseminated by performing it seems reasonable to me (although I can imagine a number of counter-arguments), but I’m not convinced that this necessarily follows for research outcomes. Yet I’m not sure that anyone is really suggesting that we shouldn’t document the research outcomes in order to disseminate them – we’re only discussing which mode is most effective for evaluation. Perhaps the A4 (300 words) sheet, though no one would argue that it, in itself, is the dissemination of the research: that can happen in any way which the researchers consider appropriate for the community they wish to reach – if that’s a DVD or a book or a painting or a website, I have no problem.
[….] I don’t have to have been in Nigel Slater’s kitchen in order to follow a recipe – something which may well be an outcome of his PaR – I just use the recipe to make the food: and then I eat it! (p. 166)

I would be very concerned at the implications for musical or other performance if scores, or scripts were seen simply as ‘recipes’ which are to be followed, thus removing most creative input from the performer (or chef).

Performance Arts Professor Susan Melrose wrote the following:

On the broader question, there is significant history, elements of which can be chased up on the website of the UK Council for Graduate Education. The Performing Arts sector’s advice to the Quality Assurance Agency was quite specific, and ran along the lines of the appropriateness of a mixed-mode higher-degree submission, which might include the submission of an ‘artefact or performance’, together with an appropriately-weighted written component (the weighting to be determined by individual universities). The QAA itself concluded that mixed-mode submissions (in, for example, creative and performing arts) were appropriate, and published that view in the documents (available for inspection) which emerged after the consultation process.

From this perspective, then, ‘just the “doing” itself ’ has not been entertained in the sector with regard to higher-degree submissions, any more than it was entertained by the 2001 RAE Panel. That Panel brought judgement to bear upon practice (and its ‘succinct statement’ of 300 words) as research. (pp. 177-8)

It is clear from this that – at least to some involved in the process in 2001 – the written component was certainly seen as something necessary.

So the debate on the 300 words, which to all intents and purposes I think should be seen as a requirement, even if the letter of the guidelines does not strictly say this, continues. Whilst in agreement with McLaughlin’s response to Reeves’ point about how the requirement put at a disadvantage those musicians less verbally articulate amounts to ‘a particularly hollow form of special-pleading’ – as I myself have said, 300 words is not in itself that much of an imposition – otherwise I have a lot of sympathy with the positions outlined by Reeves and Lingas, and worry about two points in particular. One is whether reading 300-word statements becomes a substitute for listening carefully to work; I have been in a situation where I have been forced to ask which if any of those passing judgement on my 5-CD recording of Michael Finnissy’s five-and-a-half-hour The History of Photography in Sound – for which I also produced a near-300 page accompanying monograph – had actually spent the five-and-a-half hours listening to it just once. But also, as Reeves says, whether this essential stipulation ends up disproportionately favouring work which can be summed up in a snappy 300 words, perhaps peppered with plenty of vogueish buzzwords, and detailing aspects of obvious novelty, whereas work irreducible to such things (I would struggle to do such a thing with Arnold Schoenberg’s Fünf Orchesterstücke, op. 16, Pascal Dusapin’s Third String Quartet, or Reinhard Goebel’s recording of Ignaz Biber’s Rosenkranz-Sonaten, to give just a few examples) will end up being marginalised as a result. This outcome is worse than simply a few musicians having to do some ‘academic butt-covering’.

Advertisement

---