I am writing this piece during what looks like the final phase of the USS strike involving academics from pre-1992 UK universities. A good deal of solidarity has been generated through the course of the dispute, with many academics manning picket lines together discoverying common purpose and shared issues, and often noting how the structures and even physical spaces of modern higher education discourage such interactions when working. Furthermore, many of us have interacted regularly using Twitter, enabling the sharing of experiences, perspectives, vital data (not least concerning the assumptions and calculations employed for the USS future pensions model), and much else about modern academic life. As noted by George Letsas in the Times Higher Education Supplement (THES), Becky Gardiner in The Guardian, Nicole Kobie in Wired, and various others, the strike and other associated industrial action have embodied a wider range of frustrations amongst UK-based academics over and above the issue of pensions: to do with casualisation and marketisation in academia, the growth of bloated layers of management and dehumanising treatment of academics, the precarious conditions facing early career researchers (ECRs), widespread bullying, and systemic discrimination against female academics, those from minority groups, and so on. Not least amongst the frustrations are those about various metrics employed to judge ‘performance’ relating to the government Research Excellence Framework (REF, formerly the Research Assessment Exercise (RAE)), and new Teaching Excellence Framework (TEF).
In this blog post, I will outline a short history of the RAE/REF with relevant links, and collect together recent comments about it and suggestions for alternatives. For most of this (except a few places), I will attempt to outline the arguments of others (including my own expressed online) on either side, rather than try to unpack and critique them – this blog is undoubtedly a ‘survey text’ in the sense often dismissed by REF assessors, though hopefully should serve some useful purpose nonetheless! In an academic spirit, I would welcome all comments, however critical (so long as focused on the issues and not personalised towards any people mentioned), and will happily correct anything found to be erroneous, add extra links, and so on. Anyone wishing to make suggestions in these respects should either post in the comments section below, or e-mail me at the addy given at the top of this page.
One of the most important pieces of sustained writing on the RAE and REF is Derek Sayer, Rank Hypocrisies: The Insult of the REF (London: Sage, 2014), a highly critical book which carefully presents a large amount of information on its history. I draw extensively upon this for this blog, as well as the articles by Bence and Oppenheim, and Jump on the Evolution of the REF, listed below. A range of primary documents can be found online, provided by the Higher Education Funding Council for England (HEFCE) and its counterparts in the rest of the UK, on RAE 1992, RAE 1996, RAE 2001, RAE 2008, and REF 2014. These are essential resources for all scholars investigating the subject, though obviously represent the perspectives of those administering the system. Equally important are Lord Nicholas Stern’s 2016 review of the REF, and the 2017 key policy decisions on REF 2021, made following consultation.
There are many other journalistic and scholarly articles on the REF and its predecessors. Amongst the most important of these would be the following:
Michael Shattock, UGC and the Management of British Universities (Buckingham: Society for Research into Higher Education & Open University Press, 1994).
Valerie Bence and Charles Oppenheim, ‘The Evolution of the UK’s Research Assessment Exercise: Publications, Performance and Perceptions‘, Journal of Educational Administration and History 37/2 (2005), pp. 137-55.
Donald Gillies, ‘How Should Research be Organised? An Alternative to the UK Research Assessment Exercise’, in Leemon McHenry, Science and the Pursuit of Wisdom: Studies in the Thought of Nicholas Maxwell (Heusenstamm: Ontos Verlag, 2009), pp. 147-68.
Zoë Corbyn, ‘It’s evolution, not revolution for REF’, THES, 24 September 2009.
John F. Allen, ‘Opinion: Research and how to promote it in a university’, Future Medicinal Chemistry 2/1 (2009).
Jonathan Adams and Karen Gurney, ‘Funding selectivity, concentration and excellence – how good is the UK’s research?’, Higher Education Policy Institute, 25 March 2010.
Ben R. Martin, ‘The Research Excellence Framework and the ‘impact agenda’: are we creating a Frankenstein monster?’, Research Evaluation 20/3 (1 September 2011), pp. 247-54.
Dorothy Bishop, ‘An Alternative to REF 2014?’, Bishopblog, 26 January 2013.
University and College Union, ‘The Research Excellence Framework (REF): UCU Survey Report’, October 2013.
Paul Jump, ‘Evolution of the REF’, Times Higher Education Supplement (THES), 17 October 2013.
Peter Scott, ‘Why research assessment is out of control‘, The Guardian, 4 November 2013.
John F. Allen, ‘Research Assessment and REF’ (2014).
Teresa Penfield, Matthew J. Baker, Rosa Scoble, Michael C. Wykes, ‘Assessment, evaluations, and definitions of research impact: A review’, Research Evaluation 23/1 (January 2014), pp. 21-32.
Derek Sayer, ‘Problems with Peer Review for the REF‘, Council for the Defence of British Universities, 21 November 2014.
‘Telling stories’, Nature 518/7538 (11 February 2015).
Paul Jump, ‘Can the research excellence framework run on metrics?’, THES, 18 June 2015.
HEFCE (chaired James Wilsdon), ‘The Metric Tide: Report of the Independent Review of the Role of Metrics in Research Assessment and Management’, 8 July 2015.
James Wilsdon, ‘The metric tide: an agenda for responsible indicators in research’, The Guardian, 9 July 2015.
Paul Jump, ‘Is the REF worth a quarter of a billion pounds?’, THES, 14 July 2015.
J.R. Shackleton and Philip Booth, ‘Abolishing the Research Excellence Framework’, Institute of Economic Affairs, 23 July 2015.
James Wilsdon, ‘In defence of the Research Excellence Framework’, The Guardian, 27 July 2015.
Alex Jones and Andrew Kemp, ‘Why is so much research dodgy? Blame the Research Excellence Framework’, The Guardian, 17 October 2016.
James C. Conroy and Richard Smith, ‘The Ethics of Research Excellence’, Journal of Philosophy of Education 51/4 (2017), pp. 693-708.
A Short History of the RAE and REF to 2014
There were six rounds of the RAE, in 1986, 1989, 1992, 1996, 2001 and 2008, with the gaps between each becoming progressively larger. The REF has run just once to date, in 2014, with the next round scheduled for 2021.
The first ‘research selectivity exercise’ in 1986 was administered by the University Grants Committee (UGC), an organisation created after the end of World War One. As noted by Bence and Oppenheim, there was a longer history of the development of Performance Indicators (PIs) in higher education through various metrics, but definitions were unclear, so this exercise was viewed as an attempt to convert other indicators into a clear PI, which it was thought would add efficiency and accountability to university funding through a competitive process, in line with other aspects of the Thatcher government’s policies.
The 1986 exercise involved just the traditional universities, and only influenced a small proportion of funding. It consisted of a four-part questionnaire on research income, expenditure, planning priorities and output. Assessment was divided between roughly 70 subject categories known as Units of Assessment (UoAs). There were wider criticisms of the 1986 exercise, to do with differing standards between subjects, unclear assessment criteria, and lack of transparency of assessors and an appeals mechanism. As such it was much criticised by academics, and reformed for 1989, in which ‘informed peer review’ was introduced for assessment, following wide consultation. This year, a grading system from 1 to 5 was also introduced based upon national and international criteria, 152 UoAs were used, sub-committees were expanded, and details of two publications per member of staff submitted were required, as well as information on research students, external income and plans. It was used to allocate a greater proportion of funding. There were still many criticism, to do with the system favouring large departments, a lack of clear verification of accuracy of submissions, and late planning causing difficulties for institutions preparing their submission strategies.
Other important changes affecting higher education took place during this early period of the RAE, including the abolition of tenure by the Thatcher government in 1988, then the 1992 Further and Higher Education Act , which abolished the university/polytechnic distinction, so that the latter institutions could apply for university status, and then be included in the RAE. The Act also established four funding councils for England, Scotland, Wales and Northern Ireland to replace the UGC, and made research funding allocated entirely on a selective basis, replacing previous systems of funding based upon student numbers. There had been no formula funding for research in polytechnics, so the new system radically altered the balance, allowing them to compete openly with the more traditional institutions for such funding.
RAE 1992 then brought major new changes, with institutions able to select which ‘research active’ staff to put forward, a longer timescale allowed for research in the arts and humanities, improved auditing processes, and reduced assessment down to 72 UoAs. 192 institutions participated, covering over 43,000 full-time equivalent researchers. Practically all university research funding from this point was determined by the exercise, based upon a quality rating, the number of research-active staff, amount of research income and some consideration of future planned activity. Departments which were given an assessment of 1 or 2 would not receive any funding. The result was that the older universities received 91% of the available funding, new (post-1992) universities 7% and colleges 2%. 67% of departments were ranked 1, 2 or 3. This led to objections that the system was biased in favour of the older and larger universities, which had supplied many of the panelists for certain UoAs. Some results were challenged in court, and a judge noted a need for greater transparency.
Changes for RAE 1996 involved the submission of four publications for selected research-active staff, and stiffer requirements on a cut-off date for outputs being placed in the public domain. Rating 3 was divided into 3a and 3b, and an extra 5* rating introduced, while each panel was required to make clear their criteria for assessment. 60 subject panels, with chairs appointed by the funding councils on the basis of recommendations of previous chairs, and other panel members selected on the basis of nominations from various learned societies or subject associations. These considered 69 UoAs on the basis of peer review. This was also the first RAE which allowed performance submissions for musicians (see below), which was encompassed in the following definition of ‘research’ provided by the funding councils:
‘Research’ for the purpose of the RAE is to be understood as original investigation undertaken in order to gain knowledge and understanding. It includes work of direct relevance to the needs of commerce and industry, as well as to the public and voluntary sectors; scholarship*; the invention and generation of ideas, images, performances and artefacts including design, where these lead to new or substantially improved insights; and the use of existing knowledge in experimental development to produce new or substantially improved materials, devices, products and processes, including design and construction. It excludes routine testing and analysis of materials, components and processes, eg for the maintenance of national standards, as distinct from the development of new analytical techniques.
* Scholarship embraces a spectrum of activities including the development of teaching material; the latter is excluded from the RAE.
One of the major problems encountered had to do with academics moving to other institutions just before the final date, so those institutions could submit their outputs, as well as early concerns about the power invested in managers to declare members of staff ‘research-inactive’ and not submit them. Furthermore, it was found that outcomes were biased towards departments with members on assessment panels. Once again, no funding was granted to departments graded 1 or 2. This time, however, 43% of departments were ranked 4, 5 or 5*, a rise of 10% since 1992.
The changes to RAE 2001 involved panels consulting a number of non-UK-based experts in their field to review work which had already been assigned top grades. Sub-panels were created, but there were also five large ‘Umbrella Groups’ created, in Medical and Biological Sciences; Physical Sciences and Engineering; Social Sciences; Area Studies and Languages; and Humanities and Arts. Some new measures also acknowledged early career researchers, some on career breaks, and other circumstances, and a new category was created for staff who had transferred, who could be submitted by both institutions, though only the later one would receive the resulting research funding. Expanded feedback was provided, and electronic publications permitted, though different UoAs employed different criteria in terms of the significance of place of publication and peer-review. 65% of departments were now ranked 4, 5 or 5*. 55% of staff in 5 and 5* departments were submitted, compared to 23% in 1992 and 31% in 1996.
The Roberts review of 2002 expressed concern about how the whole exercise could be undermined by ‘game-playing’, as institutions were learning to do. Furthermore, there were concerns about the administration costs of the system. A process was set in place, announced by Gordon Brown, to replace the existing RAE (after the 2008 exercise) with a simpler metrics-based system. As detailed at length in Sayer, despite major consultations involving many important parts of the UK academic establishment, an initial report and proposals of this type were quickly changed to a two-track model of metrics and peer review, then the whole plan was almost completely abandoned.
RAE 2008 itself had fewer major changes. Amongst these were a renewed set of assessment criteria, especially as affected applied, practice-based and interdisciplinary research, a two-tiered panel structure, with sub-panels undertaking the detailed assessment and making recommendations to main panels, who made broader decisions and produced a ‘quality profile’ for a department, in place of the older seven-point system. Individual outputs were now given one of five possible rankings:
4*: Quality that is world-leading in terms of originality, significance and rigour
3*: Quality that is internationally excellent in terms of originality, significance and rigour but which nonetheless falls short of the highest standards of excellence
2*: Quality that is recognised internationally in terms of originality, significance and rigour
1*: Quality that is recognised nationally in terms of originality, significance and rigour
Unclassified: Quality that falls below the standard of nationally recognised work. Or work which does not meet the published definition of research for the purposes of this assessment
By 2008-9 (before the results of RAE 2008 took effect) about 90% of funding went to just 38 universities, but from 2009 48 institutions shared this amount (after 15. As Adams and Gurney have noted, the weighting of the 2008 exercise meant that the difference between obtaining 2* and 3* was greater than that between 3* and 4*, or between the previously 4 to 5 or 5 to 5* rankings. 54% of 2008 submissions were ranked either 3* or 4*, 87% 2*, 3* or 4*.
The plans for post-2008 exercises were finally published in September 2009 by HEFCE, indicating a new name, the REF, but otherwise the system was much less different to those which preceded it than had been assumed. Now the ranking was to be based upon three components: ”output quality’ at 60%, ‘impact’ at 25%, and ‘environment’ at 15% (later revised to 65%, 20% and 15% respectively). Outputs were to be assessed as before, though for sciences, citation data would informed various panels. ‘Environment’ was assessed on the basis of research income, number of postgraduate research students, and completion rates. But the most significant new measure was ‘impact’, reflecting the desires of the then Business Secretary Lord Mandelson for universities to become more responsive to students, viewed as customers, and industry, defined as ‘an effect on, change or benefit to the economy, society, culture, public policy or services, health, the environment or quality of life, beyond academia’. Each department was to submit a general statement on ‘impact’ as a whole, and could submit between 2 and 7 impact ‘case studies’, depending upon the number of research-active staff submitted to the REF. This was a huge shift, and restricted to impact which could be observed during the cycle between exercises, and derived from research produced when the academic in question was already at the submitting institution.
Other changes including a major shift in the number of UoAs and sub-panels to 30, and just four main assessment panels. One single sub-panel would assess outputs, environment and impact. However, the same number of experts were involved as before.
Since REF 2014, the Stern Report has informed significant changes to the system, in part intended to avoid the potential for gaming. Following further consultations, it has been announced that a minimum of one output and a maximum of seven from each member of a department will be submitted. Further measures have been introduced to ensure that most short form text-based submissions must be ‘Open Access’, available freely to all, which generates its own set of issues. Further plans for REF 2028 indicate that this will also apply to long form submissions such as monographs; the situation for creative practice outputs currently appears not to have changed, but this situation may be modified. HEFCE was abolished at the end of March 2018, and replaced in England by the new Office for Students (OfS) and Research England, the actions of which remain to be seen.
The RAE and REF have caused huge amounts of resentment and anger amongst academics, and produced sweeping changes to the nature of academic work as a whole. Sir Peter Swinnerton-Dyer, architect of the first RAE (interviewed in Jump, ‘Evolution of the REF’) argued against many of the subsequent developments, and with every reform to the system, institutions would put greater pressure on individuals, especially those in junior positions, leading to some of the awful cases of chronic stress, mental illness and bullying which have been detailed recently on social media.. Many report that REF submissions constitute the only research valued by their institutions. A Head of Department (HoD) or other REF supervisor who achieved a high REF scoring could expect to win favour and further promotion from their management; in practice, this often meant cajoling and bullying of already-overworked staff with threats and intimidation about whether they would maintain their job, and and little favour or support shown to those who might not produce the right number of 3* or 4* outputs. Those dealing with mental health issues, trying to balance impossible teaching and administrative workloads (all fuelled by the Mandelsonian idea of the student-as-consumer) and research demands with major care commitments for children or the elderly, were often driven to breakdowns or to quit academia; some cases of this are documented below. Academics ceased, in the eyes of many managements, to be human beings towards whom they had a duty of care as their employers, but merely as potential cash cows, to be dispensed with if there was any pause in this function.
Gaming of the system continued in many forms from RAE 1992 onwards. Many institutions would award 0.2 FTE or short-term contracts in the run-up to the RAE/REF, so that institutions could profit from particular individuals’ outputs (not least ECRs who might have a monograph and were desperate for any employment record on their CVs). All of this could mean that rankings were unrepresentative of the research carried on by the majority of a department’s full-time, permanent staff. Research projects taking more than 6-7 years were greatly disadvantaged, or at least those embarked on them would still have to produce four other world-leading outputs in during a RAE/REF cycle, in many institutions, sometimes in order to retain a position at all. Callous HoDs or other REF managers could dismiss some work which had occupied academics for years (whilst maintaining hefty teaching and administration workloads) as merely 2*, on the grounds of its being ‘journalistic’ (often it was relatively readable), a ‘survey text’ (if it drew upon a wide range of existing scholarly literature), or the like, often with crushing impacts on the academics concerned.
The period of the RAE’s history saw other sweeping changes to Higher Education in the UK. Between 1963 and 1970, numbers of young people attending university had doubled following the Robbins Report, but then remained essentially static until the late 1980s, when over a decade numbers rose from 17% in 1987 to 33% in 1997 (see Ann-Marie Bathmaker, ‘The Expansion of Higher Education: Consideration of Control, Funding and Quality’, in Steve Bartlett and Diana Burton, Education Studies: Essential Issues (London: Sage, 2003), pp. 169-89). Since then numbers participating have continually risen, to a peak of 49% in 2011. This was an unrepresentative year, the last before the introduction of trebled tuition fees, which were a disincentive for students to take a gap year, followed by a concomitant dip of 6% (to 43%) in 2012, then a further rise to 49% in 2015, exceeding the pre-2011 peak of 46%, thus confounding (at least to date) those who predicted that increased fees would lead to decreased participation.
Sayer points out that there are few equivalents for the REF elsewhere in the world and none in North American or Europe. Furthermore, few have sought to emulate this system. Some of those cited below argue that most of the known alternatives (including those which preceded the introduction of the RAE) may be worse, others (including myself) cannot accept that this is the ‘best of all possible worlds’. I would further maintain that the human cost of the REF should not only be unacceptable, but illegal, and that only a zero tolerance policy, with criminal charges if necessary (even for the most senior members of management) could stop this. Dignity at work is as important in this context as any other, and little of that is currently on display in UK academia.
Creative Practice and Non-Text-Based Outputs
An issue of especial relevance to those engaged in performing-arts-based academic disciplines such as music, theatre, or dance (and in many cases also creative or other forms of writing, the visual arts, and so on) is that of outputs submitted to the REF in the form of creative practice. By this I mean specifically outputs in the form of practice (i.e. practice-as-research), as opposed to those simply documenting or critically analysing one’s own or others’ practice. I have previously blogged extensively on this subject, following the publication of a widely read article by John Croft (‘Composition is not Research’, TEMPO 69/272 (April 2015), pp. 6-11) and replies from me (‘Composition and Performance can be, and often have been, Research’, TEMPO 70/275 (January 2016), pp. 60-70 ) and from Camden Reeves (‘Composition, Research and Pseudo-Science: A Response to John Croft’, Tempo70/275 (January 2016), pp. 50-59), and a subsequent public debate on the subject. Amongst the issues raised, some of them familiar from wider debates on practice-as-research which are referenced in my own article, were whether creative practice on its own can stand as research without requiring additional written documentation (not least the now-familiar 300-word statements which can be regarded as deemed essential by the REF, as I argue in response to a claim made by Miguel Mera in that debate), whether creative work which most resembles ‘science’ is regarded as more ‘research-like’, an implicit claim unpacked by Reeves (as one colleague put it to me, ‘if it has wires going into it, it’s more like research’), with all this implies in terms of (gendered) views of STEM versus the humanities, or whether certain types of output are privileged for being more ‘text-like’ than others (scores versus recordings, for example) and thus some practitioners are at an advantage compared to others (here I give some figures on the relative proportions of composers and performers in different types of music departments). Attitudes to the latter vary hugely between institutions: at least one Russell Group department was happy to award a chair to a performer whose research output consists almost exclusively of performances and recordings, mostly as part of groups, while at others, especially those without strong representation of the performing arts amongst managements, such outputs are hardly valued at all and are unlikely to be submitted to the REF, nor win promotion for those who produce them.
Another issue is that of parity between creative practice outputs and other types. Many creative practitioners will never have had to submit their work to anything like peer review in the manner known for articles and monographs, and questions arise as to, for example, what number or type of compositions or recordings, visual art works or dance performances should be viewed as equivalent to the production of a monograph, when assessing promotion and the like? Music departments in which half or more of the faculty is made up of practitioners (usually composers) may have limited experience of peer review, or for that matter of wider academic debates and discourses, and some might argue that they are able to get ahead in their professions with considerably less time and effort than their equivalents who produce more traditional outputs. This is, I believe, a very real problem, which then maps onto questions of the significantly different requirements for producing different types of creative practice outputs, and needs serious consideration if there is to be any semblance of fairness within such academic departments.
Sayer also notes how many works in the humanities gain impact over an extended period of time, giving works of Walter Benjamin, Michel Foucault and Benedict Anderson as examples, and also notes how many can remain intensely relevant and widely cited long after publication, in distinction to a science-based model of cumulative and rapidly-advancing knowledge, whereby a certain passage of time leads to some outputs being viewed as outdated.
Over the last few days, various academics have been commenting on the REF, mostly on Twitter. I attempt to collect the most important of these here.
One of the first important threads came from geographer Julia Cupples (@juliecupples79). In this thread, she called the fundamental status of REF classifications ‘ludicrous’, argued how problematic it would be to direct research exclusively for REF and elite British academics, called the demands of ‘originality’ for a single publication ‘masculinist and colonial’, argued that female authors and those from ethnic minorities are at a disadvantage, not least because of less likelihood of citation. The ranking of junior colleagues by senior ones was labelled ‘one of the most toxic mechanisms in place in the neoliberal academy’, making a mockery of most other means of achieving equality, and so that the REF works against attempts to ‘dismantle discrimination, build collegiality, prevent academic bullying, and decolonize our campuses’. This thread was widely tweeted and praised, inducing others to share similar stories, with Cupples responding that the REF is ‘a means to discipline, humiliate and produce anxiety’. Not all agreed, with Germanist Michael Gratzke (@prof_gratzke) arguing that the peer review element for arts and humanities was a good thing, and that as the scheme would not disappear, one needed to deal with it reasonably. More respondents were sympathetic, however. Urban Studies Professor Hendrik Wagenaar (@spiritofwilson) cited the REF as a cause of ‘the demeaning command-and-control management style that has infected UK universities, and the creation of the soulless apparatchiks that rise up through the ranks to take every ounce of pleasure out of research and writing’, and how it prevents ‘a climate of psychological safety, trust, mutual respect, and togetherness; a place where it is safe to take risks’. Molly Dragiewicz (@MollyDragiewicz) asked whether metrification fetishises ‘engagement’, though a different view was taken by Spanish musicologist and novelist Eva Moreda Rodriguez (@TheDrRodriguez), in response to some queries of my own to Cupples. Cupples had said that it would be ‘deeply problematic if we started writing for REF and a panel of elite British academics rather than for our research communities’, to which I asked about the definition of a ‘research community’ and why they should be exempt from external scrutiny and issues of parity with other (sub-)disciplines, also pointing out that both the Chicago School of Economics or some groups of racial theorists would have fitted this category. Cupples maintained that such communities were not groups of academics, but Moreda asked in return how ‘we avoid academic work being judged on the basis of whether it reinforces& confirms the basic tenets & prejudices of said research community?’, as well as whether such community engagement was already covered through impact assessment?
Around the same time, drama lecturer Kate Beswick (@ElfinKate) blogged on ‘REF: We need to push back against a system that has lost its way’. Whilst accepting the need for assessment of academic research, she noted how layers of bureaucracy were created to game the system, the growth of internal practice REFs, the pressure to produce outputs simply to satisfy the REF rather than for any other value, and the new pressures which will follow implementation of open access policies. This, argued Beswick, would force scholars to find ‘REF compliant’ publishers, which would compromise academic objectivity, rigour, reach and international credibility. However, she did not suggest any alternative system.
However, the first major thread in defence of the REF came from historian David Andress (@ProfDaveAndress). Andress argued that the RAE/REF enabled quality research funding to go to post-1992 institutions, that every alternative had worse biases, and that the distributive mechanism was so wide that it could almost be called ‘a relic of socialism’, concluding with the confident claim that ‘If you get rid of it, you will definitely get something worse’. This was sure to produce many responses. Clinical psychologist Richard Bentall (@RichardBentall), who was a panelist in 2008 and 2014, argued that the process was ‘conducted with absolute fairness and integrity’, but the problem was with the interpretation of it by universities (a point which many others would also evoke in other threads). Bentall noted how his own former institution gave an edict telling no researcher to publish 2* papers, which constitute 80% of world science, so that the REF ‘has become an end in itself’. I myself responded that many places have concluded that research is of no value unless beneficial to the REF, also raising the question (about which I am most definitely in two minds) as to whether we need to accept that some institutions need to be focused on teaching rather than research, rather than all scrambling over a sum of government money which is unlikely to increase. Some subsequent interactions have however made me rethink this. I also noted how some assessors have little knowledge of anything beyond their own narrow and underdeveloped fields, but which nonetheless are felt necessary to be represented on panels, noted (as would many others) how a similar process is not used in many other countries, and was sceptical about any ‘better than any conceivable alternative’ argument. Andress responded that he was not saying that, but that better alternatives which can be conceived cannot be easily put into effect, and also that, in light of the expansion of the sector, ‘RAE/REF is on the positive side of the ledger’, and shouldn’t simply be dismissed. In a series of tweets, I also expressed some questions about whether all aspects of the expansion had been positive, without corresponding increases in the level of secondary education, which can have a net levelling effect when the Oxbridge/Russell Group model is applied to institutions with very different types of student bodies, from this arguing that REF was a part of a process which pretended there were not major differences between institutions, and causes huge pressures for academics at institutions where the teaching demands are higher for students with less inclination towards independent study. These are highly contentious arguments, I realise, which I want to throw out for consideration rather than defend to the last.
Moreda also responded to Andress, taking a medium view. In a thread, she acknowledged the potential of the REF for management to use to bully academics and the inordinate use of resources, but noted that it had enabled her to gain an academic position in the UK, which would otherwise have been very difficult without an Oxbridge pedigree, also on account of having a foreign accent, with little teaching experience at that point, and so on. However, she did also temper this by noting that the ability to produce REFable publications relied upon her being ‘able-bodied and without caring duties’, and that a continued discourse was required in order to consider how to accommodate others.
I asked REF defenders whether REF panellists ever read more than a few pages of a monograph, because of the time available, or listened carefully to audible outputs (rather than reading the 300-word statements which can act as spin)? Moreda responded by framing the issues as whether the REF or equivalent can ever be free of corruption, and whether such a system needs to exist at all. She was ambivalent about both questions, but also disliked the implied view of some REF-opponents that ‘research shouldn’t be subjected to scrutiny or accountability’. Whilst agreeing on this latter point, I argued that REF does not really account for parity between disciplines and sub-disciplines, some with vast differences of time and effort (especially where archival or fieldwork are involved) required for producing an equivalent output. I proposed that no output should receive 3* or 4* where authors ignore relevant literature in other languages, and that the standards of some journals should be scrutinised more. Moreda essentially agreed with the need for wider factors to be taken into account, whilst (in somewhat rantish tone!) I continued that examiners needed a wide range of expertise across multiple sub-disciplines, and asked how in historical work like hers and mine (I work on music in Nazi and post-war Germany, she works on music in Franco’s Spain and amongst Spanish exiles) how many would know if we were making up or distorting the content of the sources? Knowing of a time when there was a leading REF assessor who could not read music, I asked how they could judge many music-related outputs, and both Moreda and I agreed there could be merit in using non-UK examiners, while I also suggested that a department should be removed from the REF when one of their own faculty members is on a panel, because of the potential for corruption.
Theatre and Performance/Early Modern scholar Andy Kesson (@andykesson) posted a harrowing thread relating to his early career experiences at the 2014 REF, for which his outputs were a monograph and an edited collection. In the lead-up, he was informed that these were ‘”slim pickings” for an ECR submission’, and pushed to get them out early and develop other publications. This came at a time when Kesson’s father died and he was forced to witness his mother in the late stages of a long-term fatal illness. Whilst deeply upset by these experiences, Kesson tried to explain that he would struggle to fulfil these additional publication demands, and was told this work was non-negotiable. After the death of his mother, her own father also became extremely ill, and Kesson was forced to do his work sitting next to his hospital bed. When offered a new job, his previous institution threatened legal action over his ‘slim’ REF submission, leading to a dispute lasting two years. Many were upset to read about the callousness of Kesson’s former institution. Social identity scholar Heather Froehlich (@heatherfro) responded that ‘academics are the most resilient people on earth, who are willing to endure so much yet still believe in their absolute singular importance – only to be told “no, you are wrong” in every aspect of their professional lives’. However, one dissenting voice here and elsewhere was that of Exeter Dean and English Professor Andrew McRae (@McRaeAndrew), who cited Wilsdon’s defence of the REF mentioned earlier, and argued that no QR money would ever be given without state oversight, asking whether a better model than the REF existed? Engineering Professor Tanvir Hussain (@tanvir_h) argued that the problem was with Kesson’s institution’s interpretation of REF rules rather than the rules themselves, a theme which others have taken up, on how the ambiguities of the REF are used as a weapon for favouritism, bullying and the like.
Geographer Tom Slater (@tomslater42), having read many of the worst stories about people’s experiences with the REF, called out those who serve on panels, making the following claims:
A) you are not being collegial
B) you are appallingly arrogant if you think you can offer an evaluation of the work of an entire sub-discipline *that has already been through peer review*
C) you are not doing it because somebody has to
D) you are not showing “leadership”
E) you are contributing to a gargantuan exercise in bringing UK academia into international disrepute
F) you are making academia an even more crappy for women, minorities, critical thinkers, and great teachers
G) if you all stood down, HEFCE would have massive problem
Various people agreed, including in the context of internal pre-REF assessments. Another geographer, Emma Fraser (@Statiscape) suggested simply giving any REF submission a 4*, a suggestion Slater and sociologist Mel Bartley (@melb4886) endorsed, and was made elsewhere by novelist and creative writing lecturer Jenn Ashworth (@jennashworth). Linguistics scholar Liz Morrish (@lizmorrish) was another to focus on the behaviour of individual institutions, maintaining that ‘the
#REF was NEVER intended to be an individual ranking of research. It was intended to give a national picture and be granular only as far as UoA. What you are being asked to do is just HR horning in on another occasion for punishment’. Slater himself also added that ubiquitous terms such as ‘REFable’ or ‘REF returnable’ should be abandoned.
Paul Noordhof (@paulnoordhof) asked in this context ‘Suppose there were no REF, or equivalent, linked to research performance. What would stop the University sector achieving efficiency savings by allowing staff numbers to reduce over time and doubling teaching loads? Especially for some subjects’, but Slater responded that collective action from academics (as opposed to the more common action supporting and promoting the REF) would stop this. Slater also responded directly to McRae’s earlier post, including the statement ‘Careful what you wish for’, by arguing that ‘most would wish for a well funded sector where we don’t have to justify our existence via an imposed, reductive, compromised, artificial assessment system that destroys morale. Careful what you lie down for’.
Italian social scientist Giulia Piccolino (@Juliet_p83), responding to my retweeting of Slater’s original thread, called herself ‘the last defender of the REF’, which she felt to be ‘a bad system but the least bad system I can imagine’, a similar position to that of Andress. In response, I suggested that a better system might involve the submission of no more than two outputs from any department, allowing much more time to be spent on peer review. Piccolino noted that in other countries where she had worked, appointments depended simply on one’s PhD supervisor (a point she also made in response to Cupples), that scholars stop researching after receiving a permanent job (but still try and control junior figures) (something I have observed in some UK institutions), and so argued that while the REF could could be improved and humanised, it seemed a break on arbitrary power as encountered elsewhere. Piccolino’s returned elsewhere to her theme of how the transparency and accountability of the REF were an improvement on more corruptible systems, with which many UK academics were unfamiliar.
The debates with McRae continued, after his response to Cupples, in which he called the REF ‘an easy target’ and suggested that its demise would leave academics reliant on grants (a view endorsed wholeheartedly by Piccolino), claimed that many would prefer to replace peer-review with metrics, and that impact produced some important activity. Legal academic Catherine Jenkins (@CathyJenkins101) asked if things were so bad before the introduction of the RAE in 1986, to which McRae responded that he did not work in the UK then, but saw the problems of an Australian system in which publications in ‘a low-achievement environment’ in which many had not published for years, did not help a younger academic get a job. Modern Languages scholar Claire Launchbury (@launchburycla) argued that the modern Australian system (despite, not because of, its own ‘Excellence in Research for Australia’ (ERA) system for research evaluation) was practically unrecognisable in these terms. In response to a query from Marketing lecturer Alexander Gunz (@AlexanderGunz) relating to the lack of a REF equivalent in North America, McRae responded that that system was radically different, lacking much central funding, but where ‘state institutions are vulnerable to the whims of their respective govts, so in that respect greater visibility/measurability of performance might help’. Cupples herself responded to McRae that ‘The vast majority of universities in the world have no REF (and neither did British universities not so long ago) and yet research gets done and good work gets published’. Historical sociologist Eric R. Lybeck (@EricRoyalLybeck), a specialist in universities, echoed the view of Swinnerton-Dyer in hearkening back to the ‘light touch’ of the first RAE, which ‘would be an improvement’, and also argued against open access, saying this ‘distorts and changes academic practices’.
Film lecturer Becca Harrison (@BeccaEHarrison) posted her first REF thread, detailing her disillusion with UK academia as a result of the system, noting that she was told when interviewing for her first post-PhD job that her research ‘had to be world leading’ (4*) in order to get an entry-level job, and feeling that even this might amount to nothing because ‘there are 100 ECRs with 4* work who need my job’. This led her to support calls to boycott preparations for the REF as part of continuing industrial action. Another thread detailed common objections to the REF, then in a third thread, Harrison detailed her experiences with depression and anxiety attacks during her PhD, leading to hair loss and stress-induced finger blisters making it impossible to type, as well as early experiences with a poorly-paid teaching fellowship together with a non-HE job to pay bills, working 18 hour days in order to produce a monograph and endlessly apply for jobs. In her first full-time job, Harrison encountered bullying, misogyny from students, a massive workload and obsessiveness about production of 4* outputs. This did not lead to a permanent contract, but a new job offer came with huge requirements just for grade 6/7. She rightly said ‘please, people implementing REF, people on hiring committees, please know that this is what you’re doing to us – and that when we’ve done all this and the system calls us ‘junior’ and treats us like we don’t know what we’re doing we will get annoyed’.
Some further questions were raised by several on the new rules on open access, for example from Politics scholar Sherrill Stroschein (@sstroschein2), who argued that this would ‘just make book writers produce best work outside of REF’. But this important debate was somewhat separate from the wider question of the value of the REF, and what system might best replace it, which I decided to raise more directly in a new thread. There were a range of responses: musicologist Mark Berry (@boulezian) argued for a move away from a model based upon the natural sciences, and claimed that ‘Huge, collaborative grants encourage institutional corruption: “full economic costing”‘, while Moreda alluded to an article from 2017 about the possibility of a ‘basic research income’ model, whereby everyone had a certain amount allocated each year for research, so long as they could prove a reasonable plan for spending it (David Matthews, ‘Is “universal basic income” a better option than research grants?’, THES, 10 October 2017, though engineer David Birch responded that this would ultimately lead to another system similar to the REF). She saw how this would be insufficient for most STEM research and some in the humanities, but this could then be supplemented by competitive funding, as is already the case. Berry made a similar point to Moreda, also noting how much money would be saved on administration, whilst Cupples also agreed, as did sociologist Sarah Burton (@DrFloraPoste). Sums of up to around £10K per year were suggested; Burton also added that larger competitive grants should be assigned on a rotating basis, so that those who have had one should be prevented from holding another for some years, to create openings for post-graduate researchers (PGRs) and ECRs. I responded that this might exacerbate a problem already prevalent, whereby time-heavy species of research (involving archives, languages, old manuscripts, etc.) would be deterred because of the time and costs involved; Burton agreed that ‘slow scholarship’ is penalised, especially ethnographic work (this type of point was also made by archaeologist Rachel Pope (@preshitorian), comparing time-intensive archaeological work with ‘opinion pieces’ judged as of similar merit), while Moreda suggested that some ‘sliding scale’ might be applied depending on whether research involves archives and the like, though acknowledged this could result in ‘perverse incentives’.
I also noted that one consequence of Burton’s model would be a decline in the number of research-only academics, but that it would be no bad thing for all to have to do some UG core teaching (with which Cupples agreed). Burton’s response was ambivalent, as some are simply ‘not cut out for teaching in a classroom’, though I suggested similar problems can afflict those required to disseminate research through conferences and papers, to which Burton suggested we also need to value and codify teaching-only tracks for some. Moreda was unsure about the proposal to restrict consecutive grants, especially for collaborative projects, though also suggested that such a model might free up more money for competitive grants. Noting earlier allegations of careerism, etc., Berry argued that one should not second-guess motivations, but there should be space for those who are not careerists, and that it would be helpful for funds to assist with language or analytical skills or other important things.
I asked who might have figures for (i) no. of FTE positions in UK academia at present (to which question I have since found the figure of 138,405 on full-time academic contracts, and 68,465 on part-time academic ones, in 2016-17); (ii) current government spending on research distributed via REF (the figure for 2015-16 was £1.6 billion), and (iii) the administrative costs of REF (for which a HEFCE report gives a figure of £246 million for REF 2014). This latter figure is estimated to represent roughly 2.4% of a total £10.2 billion expenditure on research by UK funding bodies until REF 2021, and is almost four times that spent on RAE 2008. Nonetheless, its removal would not make a significant difference to available research funds. If one considers the ‘basic research income’ model (in the crudest possible form) relative to these figures, an annual expenditure of £1.6 billion would provide £10K per year for 160,000 full-time academics, which would be a very large percentage. if the part-time academics are assumed to average 0.5 contracts.
An arts and humanities scholar who goes by the name of ‘The Underground Academic’ (@Itisallacademic) (hereafter TUA) felt the basic income model would prevent a need to apply for unnecessary large grants, and also expressed personal dislike for collaborative projects, a view which runs contrary to orthodox wisdom, but was backed by Moreda and Berry. I agreed and also questioned the ‘fetishisation of interdisciplinary work’ as well. TUA responded with a pointer to Jerry A. Jacobs, In Defense of Disciplines: Interdisciplinarity and Specialization in the Research University (Chicago: University of Chicago Press, 2013), which is a sustained scholarly critique of interdisciplinarity, so often assumed to be an unquestionable virtue. Burton also asked that employers and funders value book-based research more, and expressed frustration that her own work on social theory is deemed ‘easy’, to which I added an allusion to a common situation by which reading-intensive work, often involving carefully critical investigation of hundreds of books, can be dismissed as entailing a ‘survey text’.
There were a range of other more diverse responses. Cupples also argued that the New Zealand system, the Performance-Based Research Fund (PBRF), whilst imperfect, was ‘a thousand times better than the REF’; Cupples and Eric Pawson authored ‘Giving an account of oneself: The PBRF and the neoliberal university‘, New Zealand Geographer 68/1 (April 2012), pp. 14-23. Amongst the key differences Cupples outlined were individual submissions, crafting of one’s own narrative, own choice of most suitable panel, own choice of nominated outputs, information on how one did oneself (not available to others), and greater support from departments.
Piccolino returned to her earlier questions about the potential for corruption in non-REF-based academic cultures, and asked ‘which system guarantees that people are hired for being committed, dedicated researchers vs being friends, friends of friends, products of elite institutions etc?’. Following Cupples mention of the PBRF, Piccolino also mentioned the Italian abilitazione nazionale, providing criteria for associate and full professors, but she suggested it was of little effect compared to patronage and the need for compliant researchers. This system was, according to Piccolino, closer to the REF than the German Habilitation. She also drew attention to a scathing article on corruption in Italian academia (Filippomaria Pontani, ‘Come funziona il reclutamento nelle università’, Il post, 11 October 2016).
Social scientist Gurminder K. Bhambra (@GKBhambra) pointed out the intensification of each iteration of the REF, with the current post-Stern version more individualised and pernicious than before. Medievalist James T. Palmer (@j_t_palmer) argued that REF is not the primary means of distributing research funding, because the majority is distributed through competition, though the REF may determine university funding in general (a profound observation whose implications need wider exploration).
Medieval and early modern historian Jo Edge (@DrJoEdge) asked why, in a REF context, peer-reviewed book chapters are seen as inferior to journal articles, to which Andress replied that (a) some believe book peer-review is less rigorous, as chapters are pre-selected and reviewed collectively; (b) the chapters will have less impact since less easy to find through the usual search engines (a point which Burton said she had also heard); (c) old-style elitist prejudice.
A sardonic exchange proceeded between three musicians or musicologists : composer Christopher Fox (@fantasticdrfox, himself a REF 2014 panelist), Berry, and me. Fox felt that ‘the current UK research model is counterproductive in the arts’ and that ‘Competition is a useless principle around which to organise our work’. I asked what it would mean to rank the work of leading late-twentieth-century composers such as Pierre Boulez and Jean Barraqué, Luciano Berio and Luigi Nono, Brian Ferneyhough and Robin Holloway, or the playing of pianists Aloys Kontarsky and David Tudor, or clarinettists Harry Sparnaay and Armand Angster, as 3* or 4*, especially if non-musicians were involved in the process? Fox also referenced US composers Terry Riley and Pauline Oliveros, and as how one can fix criteria which account for the disparities in their aesthetic intentions, while Berry pointed out that Anton von Webern (almost all of whose works are short in duration) would ‘never have been able to “sustain his invention over a longer time-span”‘, alluding to a common criteria for composition. Conversely, I asked if Erik Satie’s Vexations (which consists of two lines of music repeated 840 times), or the music of La Monte Young (much of it very extended in duration) should ‘have been regarded as streets ahead of most others, if submitted to REF?’, in response to which musicologist (French music expert) Caroline Potter (@carolinefrmus), author of several books on Satie, alluded to an upcoming ‘REF-related satire’ which ‘seems like the only sane way to deal with the business’. I asked about whether all of this contributed to a ‘a renewed, and far from necessarily positive, concept of the “university composer” (or “university performer”)’ (terms which have often been viewed negatively, especially in the United States), when academia is one of the few sources of income. Fox felt that this culture encouraged ‘the production of compositions that only have significance within academia’. I also raised the question of whether academics looked down on books which could be read by a wider audience, which Berry argued stemmed from envy on the part of those with poor writing skills.
Independently, cultural historian Catherine Oakley (@cat_oakley) echoed the views of Kesson and Harrison, as regards the impact of REF upon ECRs, who need ‘monograph + peer-reviewed articles’ to get a permanent job, yet start out after their PhDs in ‘precarious teaching posts with little or no paid research time’.
Elsewhere, industrial relations expert Jo Grady (@DrJoGrady) advocated boycott of preparations for the REF and TEF. In a series of responses, some asked how this could be done, especially when individuals are asked to submit their own outputs for internal evaluation. Further questions ensued as to whether this might lead to some of the worst (non-striking) academics undertaking the assessment.
Sayer himself (@coastsofbohemia) also contributed to these Twitter exchanges. In a first thread, he alluded to a passage from his book: ‘In a dim and distant past that is not entirely imaginary (and still survives for the shrinking minority of faculty members in N America) research was something that academics undertook as a regular part of their job, like teaching … Universities … expected their staff to publish … and academics expected universities to give them sufficient time to pursue their research … There was no *specific* funding for time for research but … the salary was meant to support and remunerate a staff member’s research as well as his or her teaching … [whereas today] Because the only govt support for universities’ “research infrastructure … and pathbreaking research …” comes through QR funding and QR funding is tied to RAE/REF rankings, any research that scores below a 3* necessarily appears as unfunded. The accomplishment of the RAE/REF … is to have made research *accountable* in the literal sense of turning it into a possible object of monetary calculation. This makes the REF a disciplinary technology in Foucault’s sense … which works above all through the self-policing that is produced by the knowledge that one’s activities are the subject of constant oversight. Both inputs (including, crucially, academics’ time) and outputs (as evaluated by REF panels and monetized by the QR funding formula) can now be *costed.* The corollary is that activities that do not generate revenues, whether in the form of research grants or QR income, may not count in the university’s eyes as research at all.’ In response to a question from me about his feelings on the argument that RAE/REF had helped post-1992 institutions, Sayer argued that there were other alternatives to no funding or REF-based funding, alluding to some of the suggestions in his article on peer review listed earlier. In a further thread, he summarised these arguments: the relative merits of peer review vs. metrics was ‘not the issue’. Sayer asserted that ‘Peer review measures conformity to disciplinary expectations and bibliometrics measure how much a given output has registered on other academics’ horizons’, and that neither of these are a reliable basis for 65% of REF ranking. Instead, he suggested that more weight should be allocated to research environment and resources, research income, conference participation, journal or series editing, professional associations, numbers of research students, public seminars and lectures, all of which are measurable.
Literature and aesthetics scholar Josh Robinson (@JshRbnsn) joined the discussions towards the end of this flurry of activity. Coming into one thread, he noted that internal mock-REF assessments meant ‘that the judgements of powerful colleagues with respect to the relative merits of their own & others scholarship can never be held to account’, since individual scores are not returned to departments, also arguing that this would be exacerbated in REF 2021. In response to McRae, Robinson added his name to those advocating a basic research income, which McRea said would technically be possible, but in practice ‘would redistribute tens of millions per year from RG to post-92 unis. Try that on your VC!’. Robinson’s response was to quote McRae’s tweet and say ‘the manager at a Russell Group insitution shows what he’s actually afraid of.’ But in response to a further statement in which Robinson thought that what his VC ‘would be afraid of would be a generally good thing’, McRae suggested that this might simply lead VCs to make redundancies. Robinson pointed out that an allocation by FTE researcher would provide an incentive to hire more people with time for research. Robinson has indicated that he might be able to make available a recent paper he gave on the REF, which I would gladly post on here.
But Morrish, responding that McRae’s claim that the REF is ‘the price we pay, as a mechanism of accountability’, retorted that ‘the price we pay’ is ‘a) Evidence of mounting stress, sickness and disenchantment among academics REF-audit related; b) Ridiculous and career-limiting expectations of ECRs’.
A few other relevant writings have appeared recently. Socio-Technical Innovation Professor Mark Reed (@profmarkreed) and social scientist Jenn Chubb (@JennChubb) blogged on 22 March calling on academics to ‘Interrogate your reasons for engaging in impact, and whatever they are, let them be YOUR reasons’, referencing a paper published the previous week, ‘The politics of research impact: academic perceptions of the implications for research funding, motivation and quality’, British Politics (2018), pp. 1-17. Key problems identified included choosing research questions in the belief they would generate impact, increased conflicts of interest with beneficiaries who co-fund or support research, the necessity of broadening focus, leading to ‘shallow research’, and more widely the phenomenon of ‘motivational crowding’, by which extrinsic motivations intimidate researchers from other forms, and a sense that impact constitutes further marketisation of HE. Chubb and Richard Watermeyer published an article around this time on ‘Evaluating ‘impact’, in the UK’s Research Excellence Framework (REF): liminality, looseness and new modalities of scholarly distinction’, Studies in Higher Education (2018), though I have not yet had chance to read this. Historian Tim Hitchcock (@TimHitchcock) also detailed his experiences of the RAE/REF from the late 1980s onwards, first at North London Polytechnic. Hitchcock argues that:
I have always believed that the RAE was introduced under Thatcher as a way of disciplining the ‘old’ universities, and that the 1992 inclusion of the ‘new’ universities, was a part of the same strategy. It worked. Everyone substantially raised their game in the 1990s – or at least became more focussed on research and publication.
Hitchcock goes on to detail his experiences following a move to the University of Hertfordshire after RAE 1996. He notes how hierarchies of position (between Lecturer, Senior Lecturer, Reader, Professor) became more important than ever, and recruitment was increasingly guided by potential RAE submissions. However, Hitchcock became more disillusioned when he took a position at the University of Sussex after REF 2014, and saw how the system felt ‘more a threat than a promise’ in such places, in which REF strategy was centrally planned. He notes how ‘The bureaucracy, the games playing and the constantly changing requirements of each new RAE/REF, served a series of British governments as a means of manipulating the university system’, the system was increasingly rigged in favour of ‘old’ universities, and made life increasingly difficult for ECRs, who had to navigate ever-bigger hurdles in order simply to secure a permanent position. Hitchcock concludes that:
Higher education feels ever more akin to a factory for the reproduction of class and ethnic privilege – the pathways from exclusion to success ever more narrowly policed. Ironically it is not the ‘neo-liberal’ university that is the problem; but the ‘neo-liberal’ university dedicated to reproducing an inherited hierarchy of privileged access that uses managerialism and rigged competition to reproduce inequality.
He does not write off the potential of the REF to change this, and appears to see the particular ways it is administered and used (and viewed by some in ‘old’ universities) as the problem.
There is more to say about the Thatcherite roots of the RAE, her disdain for the ‘old’ universities, especially after her alma mater, Oxford University, refused in 1985 to award her an honorary doctorate, and what the 1992 act meant in terms of a new vocational emphasis for higher education in general, to which I may return in a subsequent blog post.
It is very clear that the majority of Academic Twitter are deeply critical or bitterly resentful of the RAE/REF, and most believe reform to be necessary. Editorial director of the THES, Phil Baty (@Phil_Baty) offered up a poll asking whether people thought the REF and RAE had been positive or negative; the results were 22% and 78% respectively (and further comments, mostly making similar points to the above, followed). The arguments pro and contra, as have emerged over the weekend can be summarised as follows:
Pro: provides some transparent external scrutiny and accountability; enables funding for post-1992 institutions; enables some to find work who would find it impossible in other systems dominated by patronage; is a better model than any other which has been discovered; employs peer-review rather than metrics.
Contra: invests too much power in managers; creates bullying and intimidatory atmosphere at work through REF preparation mechanisms; makes job market even more forbidding for ECRs; highly bureaucratic; very costly; dominates all research; time-consuming; discriminatory; sexist; colonialist; makes few allowances for those with mental health, care, family, or other external commitments; uncollegiate; employs assessors working outside their area of expertise; uses too many UK academics as assessors; marginalises 2* work and book chapters; fetishises collaborative or interdisciplinary work; falsely erases distinctions between institutions; relies on subjective views of assessors; artificially bolsters certain types of creative practice; is not employed in almost any other developed country; employs mechanisms more appropriate that STEM subjects than arts, humanities and social sciences; has increased pressure on academics with every iteration; causes huge stress and sickness amongst academics.
Stern has not been enough, and there is no reason to believe that those making the final decisions have much interest in the welfare of lecturers, or for that matter the creation of the best type of research culture. Major reform, or perhaps a wholly new system, are needed, and both government and the OfS and Research England should listen to the views expressed above. And new employment laws are urgently needed to stop the destruction of academics’ lives which is happening, regularly as a result of the REF.
Video of Research Seminar on Composition and Performance as Research, and some wider responses to John Croft and othersPosted: December 9, 2015
Here is the video of the research seminar which took place on November 25th, 2015, on the subject of ‘Can Composition and Performance be Research?’, which featured a panel made up of Christopher Fox (Professor of Composition at Brunel University and editor of Tempo), myself (pianist and Head of Performance at City University), Miguel Mera (composer and Head of the Department of Music at City University), Annie Yim (pianist and DMA student at City University), and Camden Reeves (composer and Head of Music, University of Manchester). Christine Dysers (PhD student in Music at City University) was unable to be present due to illness, but a statement by here was read out by Sam MacKay (PhD student in Music at City University and organiser of the seminar). The session was chaired by Alexander Lingas (Undergraduate Programme Director and Reader in Music, City University). Greatest of thanks are also due to Bruno Mathez for making and editing the video.
A short article in response to the occasion has been posted at the City University Music Department has been posted by PhD student in music Roya Arab.
The panellists were responding to two key articles: John Croft’s ‘Composition is Not Research’, Tempo 69/272 (April 2015), pp. 6-11, and my own ‘Composition and Performance can be, and often have been, Research’, Tempo 70/275 (January 2016), pp. 60-70. As of this week, Camden Reeves’ article ‘Composition, Research and Pseudo-Science: A Response to John Croft’, Tempo 70/275 (January 2016), pp. 50-59, and Croft’s reply to Reeves and myself, ‘Composition, Research and Ways of Talking’, Tempo 70/275 (January 2016), pp. 71-77, have been published – these are not yet available via open access, but can be downloaded from Tempo for those with access to this.
Here I wanted to summarise the arguments I presented at the forum, and also respond to some of Croft’s response. Some of my thinking has moved on a little from the positions I outlined in my Tempo article (which I acknowledge may contain some inner contradictions or inconsistencies), but the majority of positions presented there are ones I continue to uphold.
The debate has been dominated by the issue of whether composition can be research, with much less attention given to performance; I would like to redress that balance. I believe that it is tacitly accepted that a musical composition is likely to qualify as some type of research much more than is the case for musical performances and recordings. This is reflected in the relative numbers of composers and performers employed in academic positions in universities. I have compiled some approximate figures for the situation as it exists in autumn 2015, in large measure using data derived from departments’ own websites. These figures are slightly modified and checked from those given at the seminar – if anyone notices any other omissions or major errors, do let me know and I will make the appropriate corrections.
There are 53 departments offering various types of music or music-related degree [Edit: Some other departments could also be included, which I will add when editing this post at some point in the near future], excluding the ten UK conservatoires, in which the status of composition and performance is of a different nature. These are as follows:
Russell Group (19): King’s College and Queen Mary, University of London; Birmingham; Bristol; Cambridge; Durham; Leeds; Liverpool; Manchester; Newcastle; Nottingham; Oxford; Sheffield; Southampton; York; Cardiff; Edinburgh; Glasgow; Queen’s University, Belfast.
Mid-ranking Institutions (‘Other’) (13): Royal Holloway and Goldsmiths Colleges, and School of Oriental and African Studies, University of London; City University; Brunel; Hull; Keele; Open University; Salford; Surrey; Sussex; Bangor; Aberdeen
Post-1992 Institutions (received university status after 1992) (21): West London; East London; London Metropolitan; Westminster; Middlesex; Kingston; Anglia Ruskin; Bath Spa; Brighton; Canterbury Christ Church; Chichester; De Montfort; Falmouth; Hertfordshire; Huddersfield; Liverpool Hope; Oxford Brookes; Winchester; Wolverhampton; Edinburgh Napier; Ulster
I have looked only at composers and performers employed in academic positions (i.e. integrated into the academic career structure from Lecturer to Professor) at these institutions. On the basis of research outputs, I have counted those composers and/or performers who have also produced a fair number of written outputs as being ‘0.5’s for the purposes of counting. I have counted only university (not college) appointments at Oxford and Cambridge. By this method, I arrive at the following figures:
Total Staff: 691
Composers: 198 (28.7%)
Performers: 76 (11%)
Practitioners: 274 (39.7%)
Total Staff: 318
Composers: 89.5 (28.1%)
Performers: 21 (6.6%)
Practitioners: 110.5 (34.7%)
Total Staff: 160
Composers: 45.5 (28.4%)
Performers: 13 (8.1%)
Practitioners: 58.5 (36.5%)
Total Staff: 213
Composers: 63 (29.6%)
Performers: 42 (19.7%)
Practitioners: 105 (49.3%)
Thus there is a ratio of around 4.3:1 of composers to performers at Russell Group institutions, 3.5:1 at mid-ranking institutions, but 3:2 for post-1992 institutions. Performance is clearly less regularly valued as an academic field of study in the more prestige institutions, compared to composition (where the representation is very similar across the sector).
There is a highly sophisticated debate (and concomitant outputs) on practice-as-research in fields such as theatre and dance (my own former institution, Dartington College of Arts, was at the forefront of this). The apparently clear distinction between ‘creative’ and ‘professional’ practice mentioned by Mera in the seminar is however far from clear-cut; it is widely debated and problematized in critical literature, rarely defined clearly, and some departments elide the distinction by using concepts such as ‘Creative Professional Practice’. In comparison to all of this, the debate in music has been rather elementary. Composition has been an accepted academic field for a long time, like fine art and drama; but changes in the RAE/REF in the mid-1990s, allowing the submission of practice-based outputs, forced a re-thinking of this. It is in this context that more fundamental questions about the status of composition and performance in academia have come to the fore, as they have had to consider the types of issues and paradigms developed in other practice-centered disciplines.
I believe that practically all composition and performance are research in some sense; in the case of musical performance the following would be some of the types of research questions that any performer has to answer in order to play a piece of music:
- Which tempi should be used for various large-scale sections of the score in question?
- How much flexibility should be employed within these broad tempi?
- On a smaller scale, what forms of stylisation and elasticity would be most appropriate for playing various types of rhythms?
- Through various combinations of accentuation, articulation and rhythm, to what extent, and where, should one tend towards continuity of line, or more angular approaches?
- In polyphonic or contrapuntal textures, to what extent should one be aiming to project a singular voice which is foregrounded above others, or a greater degree of dynamic equilibrium between parts
- Should one aim for a singular prominent climactic point within a movement, or can there be several of roughly equal prominence?
I could continue with many more; what is important is that by articulating them in this fashion I am not simply making explicit what might as well remain implicit in the acts of musical preparation and performance, but also underlining the fact of their being choices in various respects, not necessarily something which all performers acknowledge (inwardly or outwardly) or act upon. ‘Gigging’ performers, or those who value primarily ‘intuitive’ approaches, might be amongst those less likely to be concerned about the possibilities of rational choices in the process of preparing a performance or recording.
But even if most practice is a type of research, there remain different levels of which such research is conducted – though this is equally true of written work. The question of ‘is X research?’ is banal and inconsequential; what matters is how we determine equivalence of quality between different manifestations of research. We should be wary of over-rating either practice-based or written work which entails a fraction of the thought, prior skills, time and rigour of the most intensive types of research, and ensure a critical research culture exists amongst practitioners if musical institutions are to be more than dressed-up low level conservatoires.
The possibilities for peer review of work whose output is in the form of practice have not been sufficiently explored, and I propose we need a ‘space’, equivalent to a journal, for reviewing and then either publishing (where outputs can be placed online), or simply detailing and drawing attention to (where outputs are copyrighted elsewhere) creative work. I would welcome any communications from others who might be interested in trying to set such a thing up.
Various participants in the seminar appeared to assume that I did not believe that practice could be research unless accompanied by a written component. This is by no means my belief; rather I have questioned whether some relatively unreflective practice should be considered equivalent to more traditional forms of research, but would again emphasise that these questions also apply to some types of written output. Mera pointed out my comments on popular and cultural studies, in which fields I find great variety of quality, and suggested this is true of much work on contemporary music too: I would wholeheartedly agree, and have argued as much on this blog, as well as in various book reviews and review-articles which have appeared recently (as in my extended study of critical reception of Brian Ferneyhough, in which I have given a harsh view of hagiographical writing).
I wish to add a few comments on some points made by Croft in his response to my article. There are many problems with this response and ways in which I believe he misrepresents various of the figures he critiques, but I will limit myself here to his responses to my article. Croft writes the following:
The distinction at work here, loosely put, is between discovery and invention. Before my critics leap on this statement with accusations of essentialism or definition-mania, let me repeat that an attempt to characterise something is not an essentialising move – it is, however, an attempt to get at a fundamental difference between two types of activity: describing and presenting; making and finding out; or, in Aristotelian terms, poiēsis and epistēmē. It’s hardly a new idea, and deserves more than the breezy dismissal it receives, both from Reeves and from Ian Pace in his response. Einstein was not just ‘making something’. He was describing the world. A composer, on the other hand, is making an addition to the world that is not primarily descriptive. (And no, not like a smartphone or a blancmange.)
Smartphones and blancmanges aside (why are they so fundamentally different to musical composition in terms of their relationship to description?), I do not accept that either Reeves’ response or my own entail a ‘breezy dismissal’; in my own case I dispute how clear-cut is the dichotomy presented by Croft. He goes on to locate cases within literature on practice-as-research which themselves frame the concept of research so as to include creative practice, with which I would agree. The following is the definition of research supplied by the REF:
1. For the purposes of the REF, research is defined as a process of investigation leading to new insights, effectively shared.
2. It includes work of direct relevance to the needs of commerce, industry, and to the public and voluntary sectors; scholarship; the invention and generation of ideas, images, performances, artefacts including design, where these lead to new or substantially improved insights; and the use of existing knowledge in experimental development to produce new or substantially improved materials, devices, products and processes, including design and construction. It excludes routine testing and routine analysis of materials, components and processes such as for the maintenance of national standards, as distinct from the development of new analytical techniques. It also excludes the development of teaching materials that do not embody original research.
3. It includes research that is published, disseminated or made publicly available in the form of assessable research outputs, and confidential reports (as defined at paragraph 115 in Part 3, Section 2). (p. 48)
I do not know why Croft is resistant to this type of highly inclusive definition, though suspect (as indicated in my Tempo article) that this reflects an analytical/positivist philosophical bent rather than the more synthetic and idealistic attitude which I find more enlightening. Research does not merely describe the world, but can create new forms of perception and experience, such as are fundamental to artistic creation. One does not have to be a postmodern relativist (I am certainly not) to see that research can shape rather than merely identify reality. Composition does not come from nowhere, and all music is produced and heard in relation to other music and sonic phenomena; to treat musical creation independently of reference (whether or not willed by the composer) is in my view simplistic. Croft goes on to conclude:
This is not the place to launch a critique of STS [Science and Technology Studies], but I do think practice-as-research is in trouble if it depends on a view of science that confuses ideas and things so profoundly. However, Pace seems to espouse a version of this view in his suggestion that, if Einstein had not come up with relativity, someone else might have come up with an ‘entirely different paradigm’ instead. Most physicists would find this idea absurd. (p. 75)
The above relies on a flagrant misquotation; in my Tempo article I wrote the following:
It is by no means necessarily true that, as Croft says ‘if Einstein had not existed, someone else would have come up with Relativity’; someone might have come up with a quite different, but equally influential paradigm. (p. 68)
Nowhere here or elsewhere in the article do I use the phrase ‘entirely different paradigm’. The point is that ‘Relativity’ is not itself the phenomena being identified, but a scientific model use to give shape to external phenomena. I will leave it to others to debate whether this was the only possible model which could have been used, or for that matter whether this model will always remain undisputed in the future.
Croft also writes:
Pace, at one point, agrees that composition is ‘not intrinsically research’, but that it might entail various activities that are research. If this is his view, we do not disagree; this is exactly what I said in my original article. But at another point he states that ‘research’ is just a word for what composers have always been doing, except for the additional requirement of supporting text. One interpretation of this might be that composition is research, and the text simply points out how – but this would contradict the earlier statement that composition is not intrinsically research. Another would be that composition is not research until turned into research by the text. This certainly doesn’t square with our usual use of the word ‘research’. You could, in principle, do scientific, literary or historical research without writing anything down. Moreover, if documentation can turn non-research into research, this undermines the ‘material thinking’ justification for practice-as-research: if we take this line seriously, then compositional knowledge-how would not be amenable to translation into knowledge-that. This is a far cry from Pace’s insistence on ‘explicit articulation to facilitate integration into academic structures’. (p. 76)
Pace seems to think that without such an accompanying text, composing becomes merely a matter of composers composing ‘in the way they always have done’. This points, perhaps, to a tendency to dismiss any idea of a domain of irreducible non-conceptual thought as some kind of romantic fantasy of ineffability. I have no problem with ‘opening a window’ on the compositional process, but when this is anything but superficial, it is often poetic and rarely in the language of aims and objectives; nor is it a matter of ‘making explicit’ for the purposes of ‘integration’, as Pace puts it. Amenability to such language does not turn something into research, as we have seen; but in any case, much of what makes music meaningful is generally resistant to such ‘integration’. (p. 77)
Here is what I wrote:
Croft’s basic formulation that composition is not intrinsically research is one I accept in this naked form, and I would say the same about performance. But both are outputs, which can entail a good deal of research. A new type of blancmange or smartphone may not themselves be intrinsically research either (nor, as Lauren Redhead vitally points out, is writing), but few would have a problem seeing them as valid research-based outputs. (p. 64)
All I am arguing there is that an output is not itself research but the product of research. Croft could as easily read the above as saying that writing is not research, and dismiss all attempts to produce written articles and books, as he uses it to suggest that I am supporting his position. Another passage to which he refers is:
Unlike Croft, I believe that composition-as-research, and performance-as-research (and performance-based research) are real activities; the terms themselves are just new ways to describe what has gone on earlier, with the addition of a demand for explicit articulation to facilitate integration into academic structures. (p. 70)
This needs to be read in the context of these previous statements:
Ultimately his [Croft’s] model of research seems to require a particular type of conceptually based knowledge which can be communicated verbally, which I find too narrow. (p. 64)
What is being asked, not unfairly, of a composer employed in a research-intensive university is that at the least they verbally articulate the questions, issues, aims and objectives, and stages of compositional activity, to open a window onto the process and offer the potential of use to others. As a performer I am happy to do this (and wish more performers would do so) and I do not see why it should be a problem for composers too (the argument that this is unnecessary, as all of this can be communicated solely through the work itself, is one I find too utopian). (p. 67)
Nor does musical practice become research simply by virtue of being accompanied by a programme note, which funding and other committees can look at while ignoring the practical work. (p. 69)
I am a bit more reticent about the second of these statements now than when I wrote the article. The point here was a pragmatic one, which might be somewhat at odds with the sentiments elsewhere. Documenting process can surely do no harm, and indeed do a lot of good in terms of clarifying and facilitating the dissemination of research, but on the other hand one should not necessarily privilege written outputs in this respect, as I said in the talk. But this does not contradict my basic view that practice can be research independently of any written element, in strong distinction to the position Croft (and at first Mera) appear to attribute to me. Documentation does not make something research, it just helps a little with making research more accessible. 300-word statements hardly seem a huge price to pay, though I remain somewhat in two minds about this point. [Addendum: see my later post about the 300-word statements and their history]
I also wrote:
Composers may wish to be paid a salary to compose or perform in the way they always have done, but perhaps they would then be better employed on a teaching contract for composition with the recognition and remuneration for their composition or performance coming from elsewhere. (p. 67)
All I am saying here is that composers should not automatically assume they are high-level academics, any more than should those who write articles and book chapters. It hardly seems so unfair that they are held to research standards just like other types of academics.
Croft takes further exception to my arguments here:
Pace’s suggestion that composition is somehow a less demanding activity for an academic to undertake, and that it needs the words to make up the difference, hardly warrants a response and has no bearing on the question at hand. (pp. 76-7)
I have some doubts as to whether some composition- and performance- based PhDs, especially those not even requiring a written component, are really equivalent in terms of effort, depth and rigour with the more conventional types. (p. 69)
This is the same point as I made about composers expecting to have to put in no extra effort when working in universities. But Croft neglects my qualifier ‘some’. I have certainly seen some other PhDs which are absolutely on a par with more conventional types, just believe these are not always typical.
I end with my fundamental point: trying to provide very exclusive definitions of ‘research’ is fruitless; what is needed is to find equitable ways of assessing composition, performance, written and other types of outputs in ways which do not put any work at a disadvantage simply because of the form of the output.
Research Forum, ‘Can Composition and Performance be Research? Critical Perspectives’, City University, November 25th, 2015, 17:30Posted: November 4, 2015
On November 25th, 2015, at 17:30, a special Research Forum will take place at City University’s Department of Music, Performance Space, College Building. For further details and booking enquiries, please contact: Sam.MacKay.firstname.lastname@example.org . The City University event page for this is here.
In this special form a group of panellists will lead a discussion on current debates about the relationship between practice and research. The discussion will centre on two articles in particular: John Croft’s recent and significant article ‘Composition is not Research’ (Tempo, 69/272 (April 2015), pp. 6-11) and Ian Pace’s reply ‘Composition and Performance can be, and often have been, Research’ (forthcoming in Tempo, 70/275 (January 2016)). Both of these can be downloaded here.
Composers and performers in UK university music departments are often employed in full academic positions and are expected to produce research, participate in the Research Excellence Framework, apply for research funding, and demonstrate all these things in order to qualify for career advancement. This situation creates imperatives often distinct from, and sometimes conflicting with, those informing their practical work outside of an academic context. Different institutions can have hugely differing perspectives on the research credentials of practice-based work, and the experiences and fortunes of such practitioners working in academia have varied correspondingly.
John Croft’s article ‘Composition is not Research’ threw down a gauntlet in its rejection of the possibility that compositional outputs can be measured as research in the same manner as more conventional outputs. Croft called for an end to the integration of composers into existing research structures of universities, and a return to the idea of ‘research equivalence’ instead.
This article has generated a good deal of discussion on blogs and social media since its appearance, some of which has been markedly hostile. The January 2016 issue of Tempo will feature two articles in response, one by composer Camden Reeves, the other by City Head of Performance Ian Pace, entitled ‘Composition and Performance can be, and often have been, Research’.
In this article, Pace provides an extended critique of Croft’s arguments, drawing upon wider debates on practice-as-research from beyond the musical field, arguing that Croft’s definitions of research are too narrow, that composition and performance frequently constitute research as much as any other types of outputs, and that the real issue is deriving equitable criteria for judging very different types of research outputs, though this is equally a problem between divergent types of written work.
Christopher Fox (Professor of Composition at Brunel University and editor of Tempo)
Ian Pace (pianist and Head of Performance at City University)
Miguel Mera (composer and Head of the Department of Music at City University)
Annie Yim (pianist and DMA student at City University)
Christine Dysers (PhD student in Music at City University)
Camden Reeves (composer and Head of Music, University of Manchester)
Chair: Alexander Lingas (Reader in Music, City University)
Piers Hellawell, ‘Treating Composers as Researchers is Bonkers.’
Luk Vaes, ‘When Composition is not Research.’
Lawrence Dunn, ‘Squaring the damn composition-research circle.’
Martin Parker Dixon, ‘Composition can be research (some comments on John Croft’s recent article).’
David Pocknee, ‘Composition Is Not A Jaffa Cake, Research Is Not A Biscuit: A Riposte to John Croft.’
Lauren Redhead, ‘Is Composition Research?’
Nicholas Till, ‘Opus versus Output’
Huib Schippers, ‘The Marriage of Art and Academia: Challenges and Opportunities for Music Research in Practice-based Environments.’
Christopher Fox, ‘Music for a Dis-Uniting Kingdom?’ (Including some reflections on composition as research).
Ian Pace, ‘Musicological Observations 4: Can Commercial Music be Research?’ (distinct from the forthcoming Tempo article mentioned above)
And some earlier relevant articles more widely on practice and research:
Christopher Frayling, ‘Research in Art and Design.’
Linda Candy, ‘Practice Based Research: A Guide.’
Joanne ‘Bob’ Whalley and Lee Miller, ‘Partly Cloudy, Chance of Rain: A Case Study’, in John Freeman (ed) Blood, Sweat and Theory: Research through Practice in Performance. (Middlesex University Press, London, 2010), pp. 218-232.
[ADDENDUM: A link to a response to this by Luk Vaes, and then my own response, can be found here]
Across social media and through blogs and elsewhere, there have been numerous responses to the article ‘Composition is not Research’ by John Croft (Tempo, Vol. 69, Issue 272 (April 2015), pp. 6-11). Amongst the most notable of these are the excellent replies by Luk Vaes (‘When composition is not research’, 5/6/15) and Lawrence Dunn (‘Squaring the damn research-composition circle’, 8/6/15) and the detailed critiques by Martin Parker Dixon (‘Composition can be Research (some comments on John Croft’s recent article)) and David Pocknee (‘Composition Is Not A Jaffa Cake, Research Is Not A Biscuit: A Riposte to John Croft’). I have written an extended article in reply to Croft (entitled ‘Composition and Performance can be, and often have been, Research’), which will appear in the December 2015 issue (Vol. 69, Issue 274) of Tempo, alongside another response from Camden Reeves, and replies from Croft to these. Croft’s article entails some arguments earlier presented in a more extended but also informal manner by Piers Hellawell (‘Treating Composers as Researchers is Bonkers’, Standpoint, May 2014). Back in 2012, Lauren Redhead wrote an interesting if problematic short piece on the subject (‘Is Composition Research’, 17/1/12). Most notable of Redhead’s arguments, and one which has had insufficient impact on subsequent debates on musical practice-as-research (though this argument regularly appears in wider debates relating to other art forms) is her response to claims that the act of composition is not in itself research, by pointing out that neither is the act of writing.
I do not want to reiterate the arguments in my Tempo article here (suffice to say that I think it is important to make a clear distinction between the radical conception of practice-as-research, and the milder notions of practice-based research or research-based practice), but rather to move onto an area not covered there, on the relationship and compatibility of music (or any other artistic practice) subject to a commercial imperative with that music being a form of research. This is what Redhead has to say on this subject:
Is composition a commercial enterprise?
It does seem to be – which also undermines research contributions made by composers. The problem facing composers researching in universities is this: composition costs money. Performers, venues, people who record and document performances all have to be paid. And unlike in science disciplines where large budgets are available to provide necessary materials for research, music departments have no budget for this. However, all of these things and people are necessary since unless compositional research is performed, and preferably by internationally known performers who have little or no interest in research, in international venues in countries which don’t even recognise the contributions made by practice-led researchers, it is not valued highly. This research is valued on its commercial success.
It is interesting to note that while this seems not to be the case for traditional musicological written research, the recent debate around academic publishing has thrown this into question. All research is valued (publically) on its ability to make money for someone else. This commercial condition both devalues practice-led research and exemplifies how the process of valuing research devalues all kinds of research.
The above is a little loose in terms of definitions: performance and recording of compositions cost money, but that is not the same thing as the costs being directly related to the act of composition (just as production of hard-copy books and their dissemination cost money, but these actions are not synonymous with the research which informs the content of the books). And whilst there is almost no compositional or other artistic practice which is entirely autonomous of commercial demands (if it is required to generate and attract some external paying audience), there are clear differences in degree. Artistic work which will be considered to have failed if it has not achieved hundreds of thousands of sales (in whatever form) is obviously in a different league from that for which the primary objective is to find an audience of 50 or so people on a few occasions. To take an example from another discipline, the appointment of Martin Amis as Professor of Creative Writing at the University of Manchester in 2011, I suspect there are strong reasons to believe that reputation, as linked to sales of books, was a much greater priority than the extent to which those books themselves constitute research. Members of the Performance Writing faculty at Dartington College and later University College Falmouth may not be entirely independent of commercial concerns, but these are of a vastly smaller order of magnitude than for Amis – but I would say those Performance Writing scholars’ work has much greater claims on embodying research.
Music departments do sometimes have some budget for hosting concerts or producing recordings, and research funds can sometimes be used towards these ends. Such concerns are equally if not more important for performers, who do seem to be cast rather in the role of composers’ servants in the above rather than independent creative practitioners and sometimes researchers in their own right, and whose work genuinely does not exist with performance and recordings.
Academic institutions, especially those associated with the humanities, to my mind do provide arenas where it is possible to carry out intellectual and creative (and other) work, involving genuinely independent critical and self-critical thinking, in which few things are taken as read, everything is rigorously questioned on a regular basis, with a fair degree of autonomy from commercial or other external function. This type of research is valued for its integrity, rigour, pioneering nature, and so on, though short-term demands (in the UK) that the ‘impact’ of such research be demonstrated can complicate matters. Even in more obviously vocational disciplines (such as medicine or law) the institutions of academia provide (at least in theory) security for independence of thought such as are by no means necessarily present in external environments where other pressures arise which might compromise integrity. Such vocationally-linked academic work has application, but it is possible to reflect critically on the nature and manifestations of that application; academics in these fields do not simply make up a service industry for an external employer.
But the idea of practice-as-research (not simply research-based practice, or practice-based research) becomes difficult where the practice is highly subservient to external imperatives, and this is especially true for highly commercial music. A fundamental measure of practice-as-research is the extent to which it embodies responses to key research questions (and by no means is all practice of this nature, as I argue in my article), but when that practice even more fundamentally has to demonstrate a high degree of market utility, what are the chances of that research being able to be undertaken independently? Only if the research questions are directly linked to market utility, or are unlikely to affect it; both situations difficult to imagine unless those questions are very banal.
Certainly there are manifold possibilities for commercially-oriented research – these could include research into production of the deadliest new weapons, or into new strategies for tax avoidance for large companies – but this is a long way from a spirit of independent and humanistic research, and the research questions are then not usually formulated by the researcher. A commercial composition supposed to embody the question of how to write a music which fulfills certain external criteria in terms of style, duration, mood, and so on, all in order to amplify or enhance something else, is not really engaged with any sort of imaginative or searching research questions. This does not mean that music linked to other media, such as theatre, film, dance or even video games, cannot be research or for that matter genuinely creative practice. The film scores of Ennio Morricone or Michael Nyman amply demonstrate the possibilities in this respect, but both composers were able to compose with a degree of autonomy of their own. It was as much a case of Leone or Greenaway (or numerous others) filming scores as Morricone or Nyman scoring films.
A broad conception of research which I believe underlies a lot of the best work in the humanities – critical (and self-critical), humanist, open-ended, and without overly pre-empting its conclusions – cannot in my opinion easily be reconciled with fulfilling a narrow brief such as is provided by commercial imperatives, except perhaps on rare occasions where commercial and other motivations are found to coincide. To believe the latter is the rule rather than the exception is to demonstrate unwavering faith in late capitalism.
The humanities, and specifically the possibilities inherent therein in a research environment not dictated by narrow external interests, appeal to me as a space allowing some autonomy from commercial and functional imperatives. But this is deeply under threat as alternatives to neo-liberal ideology become ever more marginalised within academia. And the term ‘research’ assumes a fraction of its best meanings when commercially appropriated. For this reason, I believe we should be wary of considering commercially-focused musical production as research other than in very exceptional circumstances.
 Writing is just one medium amongst many, but which happens to be dominant in other non-artistic fields, and as such occupies a privileged status. But research can equally be made manifest in experimental contemporary dance, sound art, curation, pedagogical projects, software, or many other possibilities. What is then required, though, is for those who judge this research and award funding and promotion accordingly to have the level of expertise, sensitivity, and discernment to be able to gauge the extent to which that work does indeed manifest the research, not just read an associated statement which may be little more than spin.
 I am not hostile to the concept of ‘impact’ per se, nor the principle by which it becomes a criterion for allocation of research funding. My problems are with the ways in which it has been implemented in the UK, its short-term nature in a time when academics move between institutions but their ‘impact’ is not allowed to, and the simplistic division between academic and non-academic work which it requires at present.