Those 300-word statements on Practice-as-Research for the RAE/REF – origins and stipulations – ‘academic butt-covering’ or more problematic?

I wrote that my last blog post on the issue of composition and performance as research constituted final thoughts on the subject, at least for now, but one issue has been on my mind which I wanted to clarify for myself and others. This was to do with the ubiquitous 300-word statements included with many submissions to the RAE or REF. In the public debate on the subject, Camden Reeves (1h 38’14”) spoke of an ‘artificial privileging of certain types of composition over others’ and described (1h 40’55”) the 300-word statements as ‘ridiculous’, going on to say that ‘at a lot of universities, the 300 [word] statement was seen as the kind of research report’. Alexander Lingas (1h 43’12”) asked the panel whether ‘by having the 300-word statement, that it privileges certain type of things, so that by doing that, you end up encouraging particular types of things which actually, I have to say, are very congenial to the type of musicologically-informed performance, because that’s precisely the type of thing that 300-word statements is good for to say – why do you make those wacky decisions when you perform this music? – well, it’s because, in 300 words, and so it’s a type of academic butt-covering’.

Miguel Mera (1h 45’35”) claimed that ‘there was no requirement to complete 300 words with any submission . . . you didn’t have to’. But this was far from clear from the REF 2014 report, which contained the following comments:

In brief, the additional 300 words to make further evident the research imperatives and/or research process of an output (paragraph 71(b) of the ‘Panel criteria’) were used inconsistently and the question of the research imperative was not always well-articulated. (p. 16)

As in 2008 the best outputs in PaR were distinguished by clearly articulated research objectives. In a number of instances, the presentation of practice needed no more than a well-turned 300 word statement to point up the research inquiry and its findings, since the concerns outlined were then amply apparent within the practice itself (which was made available for assessment by a variety of means including DVD or CD recordings, photographic materials, scripts and scores, databases, etc.). (p. 99)

More generally, the 300 word statements too often displayed a misunderstanding of what was being asked for and provided evidence of impact from the research, or a descriptive account akin to a programme note, rather than making the case for practice as research. (p. 100)

Scott McLaughlin, in his report on the debate, noted Mera’s comments but added that ‘I get the sense that many Universities insisted on them’. In Paul Allain and Jen Harvie, The Routledge Companion to Theatre and Performance (London and New York: Routledge, 2014), we find the claim that

all submitted PaR has to be accompanied by a 300-word statement outlining the research imperatives and context, further supported by other forms of evidence. (p.234)

Similarly, in Michael Biggs and Henrik Karlsson, ‘Evaluating Quality in Artistic Research’, in Biggs and Karlsson (eds), The Routledge Companion to Research in the Arts (London and New York: Routledge, 2011), in the context of submissions in art and design for RAE 2008, the authors write of ‘the additional requirement of a 300-word statement arguing for the significance and impact of each submission’ (p. 414). Robin Nelson, in his Practice as Research in the Arts: Principles, Protocols, Pedagogies, Resistances (Basingstoke: Palgrave Macmillan, 2013), refers to ‘a simple verbal articulation of the research inquiry – such as might be achieved in as few as 300 words’, which ‘proves useful in almost all cases’ (p. 11), not specifically mentioning the RAE/REF or a requirement, but likely written with this in mind.

So where did the idea come from that 300 words was a requirement? It was first presented in Section 3 of the RAE 2001 report on Publications, entitled ‘Panels’ Criteria and Working Methods’, and said specifically that:

3.58.8 Those submitting practice as research may include (in the ‘other relevant details’ field of form RA2) a succinct statement of not more than 300 words for each item in this category listed under RA2; for this purpose practice is defined as all outputs listed in paragraph 3.58.12 b) to e) below. These statements should make clear how the practice embodies research as defined in the RAE. They may, where appropriate, include an indication of the aims, methods, procedures, innovation, significance, and context of the practice. It should be noted that the submission of such statements is not a requirement; the Panel will not expect them when the status of the practice as research is self-evident.

The RAE 2008 guidelines on submissions contained the following text:

94. Brief, additional information may also be given in RA2 ‘other relevant details’ to identify relevant, factual circumstances concerning any output. It need not be supplied in every case. It may be, for example:

• to identify a keynote address to a conference
• to identify an invited conference paper especially where the perceived status of the conference is high
• to indicate the significance or impact of an applied research outcome
• to identify the research content or author’s contribution in edited works, translations, or co-authored works.

In the case of a non-text output, it may be used to give further information on the whereabouts of a work or to note that a photographic, electronic or other record exists. It may not be used to volunteer opinions about the relative quality of an output. See panel criteria statements for further guidance, including the word limit for this text, which, unless otherwise stated, will be 300 words.

The panel criteria for music within Panel O for RAE 2008 included the following:

Researchers should accordingly submit such evidence as they deem necessary to enable subpanel members to assess it within the following guidelines:
a. Research output: this may be submitted alone where it is deemed to constitute
sufficient evidence of the research in itself.
b. Statement: it is recommended that a statement of up to 300 words is submitted in the ‘Other relevant details’ field of RA2, in cases where the research imperatives and the research significance of an output (such as: an artefact, curation, digital format, installation, performance or event, screening, tape, textbook, translation or video) might further be made evident by a descriptive complement. The statement might include: a brief description of the project and its stage of development; a rationale outlining questions addressed; a summary of approaches/strategies undertaken in the work; a digest of further evidence (if any) to be found in sub-paragraph 13c below. As previously indicated, the 300-word statement should also be used to clarify the relative contributions of researchers working on a collaborative research project. The sub-panel will ignore any evaluative commentary on the perceived quality of the research.

Then for REF 2014, the statement of panel criteria and working methods included the following statement:

49. For non-text or practice-based outputs (including patents, software and standards documents), all subpanels welcome the submission of a description in
REF2 of the research process and research content, where this is not evident within the output (maximum 300 words), as described in ‘guidance on submissions’ (paragraph 127a).
(p. 25)

Then for Panel D, with included music submissions, the following:

b. Information about the research process and/or content: Submitting units may include a statement of up to 300 words in cases where the research imperatives and research process of an output (such as an artefact, curation, database, digital format, installation, composition, performance or event, screening, tape, creative writing, database, textbook, translation or video) might further be made evident by descriptive and contextualising information. Where the location or medium of the output is essential to a proper understanding of the research being presented this should be explained in the 300 words. The sub-panels will ignore any additional material that includes evaluative commentary on the perceived quality of a research output (p. 87)

The 300-word limit was further reinforced on p. 99.

It is clear then that whilst the 300 words was not strictly a requirement, there was a very strong incentive to include them with most submissions. How many, reading the words from 2001, are going to view their submissions, or those of their department’s faculty members as having a ‘self-evident’ status as practice-as-research? How many reading the 2008 guidelines are going to go against what is ‘recommended’?

Nicholas Cook, who was on the Music Panel which worked towards producing the 2001 definitions, wrote about the process in his article ‘Performing Research: Some Institutional Perspectives’, in Mine Doğantan-Dack (ed), Artistic Practice as Research in Music: Theory, Criticism, Practice (Farnham: Ashgate, 2015) pp. 11-32. In the RAE 1996, the first such accepting performance submission, according to Cook:

Detailed criteria for the assessment of performance as research were not in place, and when the outcomes were announced there was a widespread perception that the assessment of performance submissions had been surprisingly generous. Given some unease on the part of 1996 panel members, and the expectation that this outcome would prompt a significant increase in performance submissions for the following exercise, a major priority for the 2001 Music Panel (which was convened well in advance of the submission date) was to set the assessment of performance as research on a more principled basis: this was necessary to provide the panel with clear and transparent procedures for its evaluation, to guide institutions’ decisions about what work to submit to the RAE, and to inform their future planning. This panel, which I chaired, for the first time included professional performers, though they were based in the academic sector. (p. 22)

The panel realised that simply mapping criteria from composition-as-research over to performance would be insufficient, not least because the former rested on nebulously defined notions of quality and there were new difficulties entailed in the acceptance of submissions of film and commercial music (the research qualities of the latter of which, I have argued elsewhere, appear to me more tenuous). Whether market success, recognition by peers, or ideals of originality were to be primary criteria all proved difficult in discussion, as Cook points out (pp. 22-3). This now quite notorious definition of research was adopted:

2.12. ‘Research’ for the purpose of the RAE is to be understood as original investigation undertaken in order to gain knowledge and understanding. It includes work of direct relevance to the needs of commerce, industry, and to the public and voluntary sectors; scholarship; the invention and generation of ideas, images, performances, artefacts including design, where these lead to new or substantially improved insights; and the use of existing knowledge in experimental development to produce new or substantially improved materials, devices, products and processes, including design and construction. It excludes routine testing and routine analysis of materials, components and processes such as for the maintenance of national standards, as distinct from the development of new analytical techniques. It also excludes the development of teaching materials that do not embody original research.

As for performance, the following definition was adopted:

3.58.12 d. Performances: in accordance with the RAE definition of research, performance will be accepted as research where it applies or embodies new or substantially improved knowledge or insights, for instance in terms of interpretation, historical performance practice, or technical innovation. Performance is understood to include conducting and direction as well as instrumental or vocal execution; all forms of public output are eligible for submission, including publicly disseminated live or studio recordings, broadcasts, and public performances. In the case of broadcasts and public performances, institutions must be able to supply a recording (which need not be in the public domain). Reference may be made to such factors as the venue of the performance, the standing of broadcasting organizations or record companies involved in its dissemination, and prizes or other marks of recognition); relevant information should be provided in the ‘Other relevant details’ field of form RA2.

This is, I believe, a basically good definition, though it is difficult to define when interpretation does or does not embody ‘new or substantially improved knowledge or insights’ (I would say most good performances do), and I worry about ‘the standing of broadcasting organizations or record companies’ acting as a proxy for judging work in terms of its standing in economies of prestige or market utility. But this is not strictly relevant to the 2001 stipulation about 300-word statements given above, about which Cook says:

That last remark was aimed at composers working in established styles: we did not wish to inflict a burden of pointless documentation on institutions – or on ourselves, for that matter. (The perhaps intellectually shabby idea of ‘self evidence’ reflects the pragmatic approach to composition I described.) Nevertheless the invitation to submit succinct statements excited considerable opposition from the subject community, not all of it reasonable in our view. In the formal processes of consultation that preceded the finalisation of the criteria, certain respondents rejected our assumption that the content of performances or other forms of practice as research could be reduced to words, and accused us of intending to assess the succinct statements rather than the performances. Of course we never made any such assumptions or held any such intentions. The point is very simple, and I have already referred to it. Academic writings are self-documenting. That is the source of the conventions of good academic writing to which Candlin referred. But this is obviously not the case of performances: as Susan Hellauer (1997) says, ‘You can’t sing a footnote’. It follows that you can’t expect an assessor to be able to reconstruct from a performance the research process that has given rise to it. (Think of David Milson submitting a performance arising out of his AHRC Creative and Performing Arts Fellowship.) Once more, this is the point on which the other members of the UKCGE [UK Council for Graduate Education] study group were insisting, except that they were talking about a 40,000 word dissertation whereas we on the panel were talking about a 300 word statement. By inviting those submitting practice as research to provide such a statement, then, we were giving them the opportunity to ensure the assessor understood the research component. We were empowering them to set the terms on which they were to be assessed, and to present their work in the best possible light. (p. 25)

I disagree with Cook; a sensitive listener with some familiarity with the work in question and performance practice might very well be able at least to assess, if not necessarily reconstruct in every detail, the research process which has given rise to one of David Milsom’s performances. A 40 000 word dissertation would certainly elaborate the process to a high degree, if done well, but I am not really sure that a 300 word statement could – or rather, if some point can be elucidated in 300 words but not clearly heard from the performance without such guidance, I would question the extent to which it is embodied in that performance.

An article from 2003 by Peter Thompson (‘Practice as Research’, Studies in Theatre and Performance 22/3, pp. 159-180) gave further details on how this debate was conducted amongst those involved in drama and related disciplines, reproducing correspondence. Drama Professor Franc Chamberlain made the following observation:

To argue that the work itself is best disseminated by performing it seems reasonable to me (although I can imagine a number of counter-arguments), but I’m not convinced that this necessarily follows for research outcomes. Yet I’m not sure that anyone is really suggesting that we shouldn’t document the research outcomes in order to disseminate them – we’re only discussing which mode is most effective for evaluation. Perhaps the A4 (300 words) sheet, though no one would argue that it, in itself, is the dissemination of the research: that can happen in any way which the researchers consider appropriate for the community they wish to reach – if that’s a DVD or a book or a painting or a website, I have no problem.
[….] I don’t have to have been in Nigel Slater’s kitchen in order to follow a recipe – something which may well be an outcome of his PaR – I just use the recipe to make the food: and then I eat it!
(p. 166)

I would be very concerned at the implications for musical or other performance if scores, or scripts were seen simply as ‘recipes’ which are to be followed, thus removing most creative input from the performer (or chef).

Performance Arts Professor Susan Melrose wrote the following:

On the broader question, there is significant history, elements of which can be chased up on the website of the UK Council for Graduate Education. The Performing Arts sector’s advice to the Quality Assurance Agency was quite specific, and ran along the lines of the appropriateness of a mixed-mode higher-degree submission, which might include the submission of an ‘artefact or performance’, together with an appropriately-weighted written component (the weighting to be determined by individual universities). The QAA itself concluded that mixed-mode submissions (in, for example, creative and performing arts) were appropriate, and published that view in the documents (available for inspection) which emerged after the consultation process.

From this perspective, then, ‘just the “doing” itself ’ has not been entertained in the sector with regard to higher-degree submissions, any more than it was entertained by the 2001 RAE Panel. That Panel brought judgement to bear upon practice (and its ‘succinct statement’ of 300 words) as research. (pp. 177-8)

It is clear from this that – at least to some involved in the process in 2001 – the written component was certainly seen as something necessary.

So the debate on the 300 words, which to all intents and purposes I think should be seen as a requirement, even if the letter of the guidelines does not strictly say this, continues. Whilst in agreement with McLaughlin’s response to Reeves’ point about how the requirement put at a disadvantage those musicians less verbally articulate amounts to ‘a particularly hollow form of special-pleading’ – as I myself have said, 300 words is not in itself that much of an imposition – otherwise I have a lot of sympathy with the positions outlined by Reeves and Lingas, and worry about two points in particular. One is whether reading 300-word statements becomes a substitute for listening carefully to work; I have been in a situation where I have been forced to ask which if any of those passing judgement on my 5-CD recording of Michael Finnissy’s five-and-a-half-hour The History of Photography in Sound – for which I also produced a near-300 page accompanying monograph – had actually spent the five-and-a-half hours listening to it just once. But also, as Reeves says, whether this essential stipulation ends up disproportionately favouring work which can be summed up in a snappy 300 words, perhaps peppered with plenty of vogueish buzzwords, and detailing aspects of obvious novelty, whereas work irreducible to such things (I would struggle to do such a thing with Arnold Schoenberg’s Fünf Orchesterstücke, op. 16, Pascal Dusapin’s Third String Quartet, or Reinhard Goebel’s recording of Ignaz Biber’s Rosenkranz-Sonaten, to give just a few examples) will end up being marginalised as a result. This outcome is worse than simply a few musicians having to do some ‘academic butt-covering’.


Performance-as-Research – A Reply to Luk Vaes

The Belgian pianist and scholar Luk Vaes has published a new blog post following two previous ones (here and here) responding both to the announcement of the debate on composition, performance and research on November 25th, and also to my article on the subject which is one of the texts for discussion there. I would like to publish here a response I have also added in the comments section of his post.

I do believe that Vaes, coming from a context of ‘artistic research’ rather than ‘practice-as-research’, is inclined towards too-fixed and narrow (and sometimes counterproductive) conceptions of research, at least implicitly. But I would also like to ask him whether he thinks practice-based outputs alone can suffice as research (ever?) or only with substantial written documentation? This debate has recurred often in wider literature on practice-as-research. And should these standards be applied differently to composition and performance?

The major objection given to documentation of practice is that research councils, academic promotion panels and others simply read that and do not bother to listen to/watch/etc the actual artistic work involved. This is a very real danger, especially when (outside of the REF) non-artists may be involved with the decisions. If documentation is required as well as artistic output, only a mode of judging which looks at both in detail could ever be satisfactory.



Luk Vaes writes:

I don’t agree with the jump from “opening up research questions” to actually being “research as a result”, nor do I think performance-based research should be considered on the same level (much legitimate systematic musicology – e.g. performance science – is performance-based or -led). I more than agree with that “additional demand”, as I find the explication of the research to be essential to its identity. As long as it is impossible for me to assess how (and how exactly) Ian has learned from Gieseking, Cziffra, et all., how exactly this has opened up new questions, how exactly this worked in a certain way (and not in perhaps certain other ways), what the conclusions are, etc., it is not worth it to use a new term to describe the age-old process he described. Research is a collective effort, with peer-interaction as a fundamental, i.e. peer-based and peer-oriented. Contrary to matters of composition, I can consider myself to be a peer of Ian’s, but, from his performances, I cannot tell any of the above to a level that informs me about his research.

As far as the first ‘leap’ is concerned, let me put the ‘research as a result’ comment in context:

But my approach is far from uncommon, and in this sense the articulation of practice in research terms is a positive and productive activity. It may be less spectacular than some of the wilder fringes of theatre and visual performance – such as Lee Miller and Joanne “Bob” Whalley’s joint PhD project, collecting of urine-filled bottles on the M6, replacing them with other detritus, renewing their wedding vows in a service station, then grounding this in the thought of Deleuze and Guattari, Bakhtin, dialogism, heteroglossia and semiotic multi-accentuality, deliberately framed in such a way as to frustrate Popper’s criteria of falsifiability – but is no less ‘research’ as a result.

The only point here is that whilst critical engagement with aesthetic, technical and interpretive questions doesn’t look as spectacular as the above, that doesn’t mean such work should not equally warrant being considered research.

As far as practice-based research is concerned, this is a bit of a nebulous term, for sure; I had in mind in this context written work produced by practitioners relating to their own work, rather than just any musicology dealing with performance. But we need a more specific term for this, for which the term practice-as-research is often used in my view erroneously.

As far as needing to understand how the engagement with Gieseking, Cziffra, or whoever impacts upon the final output (which might be in the form of a marked negation of aspects of this playing, or adoption and mediation of aspects which are far from obvious), well a piece of written work might be able to explain this, and such research is useful, but one might say exactly the same about being able to know how complex row transformations impact upon a composition when these are not perceptible without guidance. Note that earlier in my article I say:

At a REF panel discussion in February 2015, it was argued that the REF can entail a large amount of financial support for innovative practice-based work. There remain various obstacles towards achieving this (not least from individual institutions inclined to downgrade practice-based work in general), but it is not an unrealistic goal. If this requires practitioners to articulate ways in which their work has value and consequences not just in and of itself but also to others as a contribution to knowledge, this seems a fair price to pay.

but also:

Nor does musical practice become research simply by virtue of being accompanied by a programme note, which funding and other committees can look at and ignore the practical work.

and also:

I have some doubts as to whether some composition- and performance-based PhDs, especially those not even requiring a written component, are really equivalent in terms of effort, depth and rigour with the more conventional types.

Others will argue that simply the final output should suffice to demonstrate the quality of the research; I am not going that far, though do see the danger of the documentation of the process being judged practically independently of the result. To convince you that engagement with various other musicians’ work, in a myriad different ways, has significantly informed my practice, is something which I do not think would be difficult given sufficient space (certainly more than the 300 words required by the REF). This is not a reflection on the quality of the performance, but whether the process involved in its creation can fairly be judged as research.

I bring this up primarily, though, because composers are frequently able simply to submit their compositions with a 300 word statements, and that suffices to justify their work as research, in a way which is much rarer for performers. Numerous composers working in UK university departments produce only compositions, no written work, whilst there are significant differences in terms of expectations made by departments upon performers in this respect. I think this is a major inequity, and also that these debates in a musical context are too heavily dominated by composers.

What we are sometimes left with is that only the most obvious (and often extremely basic) aspects of performance are considered ‘research’ – employing a few extended techniques, using a slightly new type of instrument, playing some unusual rhythms, and so on. The dutiful performer-scholar will play this music and write up a short amount of pragmatic ‘how to do it’ information, and leave the much more complex issues of interpretation, style, genre, and aesthetics to a handful of over-general and meaningless platitudes (‘it is important to phrase this music well’, ‘it should still be beautiful’, ‘one should make it sound like a real piece of music’, and so on). What I am trying to argue is that the whole business of fashioning and crafting strategies for these latter aspects more deserves to be considered research than simply writing something like ‘I tried playing this sonority by using this object to stop the string. I played it to the composer like that, and then with another object, and they preferred the first, so we went with that.’ This latter is really just a type of skills training rather than critical research.