Hi Everyone, Shanta Aphale-Coles here, University of Manchester.

Hi everyone Helen Lau, Coventry University

Hi there everyone - David Hartley, Oxford Brookes

Good Afternoon all. Phil Fiddaman from the University of Hertfordshire

Afternoon everyone - Allice Hocking from the University of Plymouth

Rob Singh, University of Essex - hello!

Good Afternoon! Alison Price, representing EEUK (Enterprise Educators UK)

Afternoon all, Mark Claridge from the Institute of Cancer Research

Ceri Jones, Swansea University

Hi Everyone, Julia Keyton here, from University of Liverpool

Good afternoon, Alexis Holden from University of Central Lancashire, Preston

Dan King from Research Consulting, hello!

Afternoon all - Heather Cobb, University of Kent

Hi everyone - Sue Graham Northumbria University here - NB make sure you set your chat to 'all panellists and attendee's as it defaults to panellists only! Sue

*attendees

Great tip, thank you Sue!

Hello, Mel Nakisa, Bucks New University

Hi everyone, Sara Priston from Bath Spa University

Hi All - Tracey Wond, HEIER (Higher Education, Impact and Evaluation Research) and UK Evaluation Society

Hello everyone. Paul Manners from National Coordinating Centre for Public Engagement here.

Hi All, Amanda Baxendale, University of Derby. Good tip Sue Graham!

Hello Everyone. Fatimah Awan, Guildhall School of Music & Drama

Thanks Sue, hadn't picked this up

Afternoon everyone, Susan Matos, University of Reading

HI All , Amie Longthorne from University of Liverpool

Hello - Jeremy Hague, Nottingham Trent University

Hi All. Emma Burke Innovate UK (UKRI)

Hi everyone. Liz Jenkinson from the University of Hull.

Afternoon all, Paul Roberts, University of Sussex

Hello all, Sophie Jones-Tinsley from Leeds Trinity University

Hi - Liam Sutton, University of Bradford

Hello, Andrew Jackson, Bishop Grosseteste University in Lincoln

Hi everyone - Ruth Rayner, Leeds University

Afternoon all. Sarah Roberts from Liverpool School of Tropical Medicine

Mike Capaldi, Newcastle University - greetings all

Feels like we're almost all in the same place. Good to virtually see everyone - Sarah from University of Portsmouth

Greetings all - Tatiana Schofield, RCA

Good afternoon all, Claire Walsh from Research Consulting

Hello everyone, Hannah Timpson, LJMU

Jonathan Bradley, University of Sheffield, Arts and Humanities KE, hi everyone, good to be here...

Afternoon everyone, Michael Millan, Anglia Ruskin University

Hello everyone. Lisa Williams van Dijk Royal Agricultural University

Hi - Emma Calverley, University of Hull

As a small provider, the overlap with REF does present capacity challenges to mobilise people around this

Phil - when are institutions going to formally sign up to the Concordat?

Some concerns with overlap with REF preparation.

I think a stretching commitment is good

i'd echo capacity issues alongside REF submissions in smaller institutions

Would like a significantly larger gap between this and REF

Would be good to delay for a few months. It will be a demanding year!

Concur with James P, while generally OK with timeline, it will be a challenge for HEIs with small RKE offices

Bottleneck in prep for both REF and KEC, even if deadline is later. Same staff, as you say. We also don't yet know when UKRI evidence submission for Impact will be, which will need to be wrapped up. Evaluation after REF would be better.

A specific date for KEC submission would help with internal planning.

It would be helpful to understand the expected detail / length of the action plan and assessment process (it would be manageable if the action plan is reasonable)

Presumably we can tick yes to both...

Important that this is owned at an institutional level, and not just something done by the KE office.

can't access the poll as am using web browser option

Just comment on here if you cannot access the poll

i cant access the poll either as using web browser

Head of Institution may nominate another of their Senior Leadership Team instead...

i cant access the poll either as using web browser

I can't access the poll as using web browser also

We have a various other concordats that we are developing actions plans for eg Career Development of Researchers, Research Integrity etc - please can we have some harmonisation in action plans and reporting requirements!

I would support the previous post - we are hoping to align the self-evaluation activities for both the KE and Researcher Development Concordats in order to avoid duplication of effort.

I think some guidance and an exemplar would be helpful.

Guidane

I think it would be useful to have guidance please

Not clear yet on how the self-evaluation summary would be used (other than how used by us) and the need for us to submit it

We have found the model we used for the HR Research Excellence Concordat a helpful example.

Guidance is good

Some guidance would be very welcome

Guidance is necessary so we have confidence in what we are submitting. Without it, it will be harder to get a clear and fair response to our evaluation.

I'm hoping that PraxisAuril, NCCPE and ARMA can work together to provide some guidance across our memberships in this area.

Need some degree of guidance to ensure consistency and evaluation against the same things.

No guidance would give freedom to institutions not normally permitted, but may make evaluation difficult..

Guidance would be helpful to achieve a degree of consistency across institutions. In addition, without guidance it's likely to take much longer to produce the self-evaluation due to the likely internal debate about how to do this.

Guidance would be welcome, as it would give some consistency and confidence in the process of self-evaluation

From a standards perspective, guidance is not in itself a mandate, it's support. Mandates are more akin to codes of conduct etc.

Would like some guidance please!

Guidance would be helpful

i agree with the comments on getting some guidance.

I agree guidance would help for consistency

Guidance is essential to ensure a useful and uniform process

Specific guidance would be useful

If there is no mandated template people are likely to duplicate other evaluations, so this carries a risk of duplication of reporting. Why bother?

Prefer guidance. Otherwise it is a guessing game that can be time consuming in itself!

I agree with Liz and Rachel!

Clarity around the scope of the self-evaluation would be helpful

I think guidance need to be provided by the assessment body - even if developed by PrA and others

Lack of guidance opens up risk of under/overdeveloping the action plan.

Some guidance would be helpful to avoid it being the interpretation of one person at a given institution. And there’s clearly a difference between the concept of guidance and detailed guidance!

Guidance would be welcomed- not too prescriptive but provides an outline of what's expected to some extent

EEUK members report lack of consistency in current metrics/national work and always attribute this to interpretations on guidance - so some 'boundaries' or guidance/expectations and samples might be needed

Guidance can be non-prescriptive but still help to ensure some level of consistency in approaches across the sector.

Also agree with comments above about some guidance being helpful

Is there enough experience and understanding on self-evaluation to allow a "no guidance" approach to be successful - identifying and supporting good practice in this area is needed I feel.

Outline guidance would be helpful

I think that member organisations can work together to provide guidance but we really do need to understand the purpose of the evaluation and what it will be used for. I agree with comments about consistency of approach here.

Guidance would be useful, otherwise we could go too far or too short.

Guidance would be useful, however that could change our perception on how workable are the timelines.

Definitely some guidance, doesn't have to be mega detailed, can reference the good practice that Praxis Auril, NCCPE promote

Non-prescriptive guidance helpful, I feel, to help with achieving consensus across HEI.

Template and least and a framework definitely needed if no guidance is being provided. You will get a wildly varied style of concordats of varying lengths if not.

A framework would be helpful for efficiency and consistency; I think the eight principles should map quite well/easily against some guidance

Need more clarity on the process and evaluation to ensure action plans are meeting intent. Guidance / best practice examples should be part of this as a no guidance approach could lead to additional burden

Happy not to have guidance but how will this lead to comparability across the sector? Is there an intention to benchmark or not?

Good practice guidance helps with Research Integrity Concordat (this is provided by ARMA/UKRIO)

Why would we be required to submit our self-evaluation summary? How will it be used?

The Consordat is across the sector as a tool for self-criticism. To ensure regualrity of interpretation of the Concordat, guidance and, to a lesser extent, assessment tools would be useful.

Definitely looks like a majority in favour of guidance being provided...

I agree guidance would be helpful - particularly as there would be a number of ways this could be approached

Guidance for consistency. Otherwise results will be impossible to compare between institutions

Depends on the consequences of "getting it wrong" - if our self-evaluation is to be /criticised/ then there should be some sort of framework.

Guidance would be helpful particularly against what will be Concordat evaluation at a later stage. Without guidance there will be more freedom to be unique as an institution but less ability to compare institutions with one another

The panellists will need some understanding of self-evaluation methodologies to ensure consistency

Disciplinary variances will also be easier to reconcile, with some non-prescriptive guidance.

With HR Excellence there is no template for a Gap Analysis, but a template for the Action Plan which is helpful to ensure consistency across sector and with evaluations

Useful if we can structure in a way to stimulate learning across institutions.

Needs to seen as of similar prestige as other concordats. Guidance to help with evaluation and transferability across institutions important.

A range would be very welcome

Sympathy with the diversity argument against getting too specific

Helen, The idea is not to compare institutions against each other but to evaluate institutions on their own merit in relation to their institutional vision

Could we judge scale using the cluster model research done for KEF? Page limits would be helpful.

Page limits are useful, but think there should be one limit for all. Smaller institutions can always write less.

Page limits definitely are needed. Also presumably pages will be text only.

Page limits based on FTE possibly; similar to environment statemetns in REF

Page limits useful, perhaps an upper limit, but no requirement for every HEI to reach the upper limit. Reflecting scale of HEI

I agree with Michael regarding the page limits

would a max page limit to be used at the discretion of individual institutions be relevant

Page limits would be helpful

How does the Concordat fit in with other external evaluations such as the Annual Monitoring Statement linked to the HEIF Strategy?

Page limits would be helpful as a guideline for institutions

How are you envisioning HEIF strategies and KEC action plans working together?

@Rachel - good question!

@ Rachel comment above - Not all HEIs currently receive HEIF so this needs to be borne in mind if are linking this KEF to HEIF strategies

Clarity around HEIF/KEF/KEC (indeed REF) alignment / interrelationships would be helpful.

@ Peter - agree!

Second @Peter

Agree @Peter

Agree@Peter

Can you clarify how the action plans will be used post submission? Will they be confidential or made available to the public?

Gender balance (Athena Swan) in KEC activities, promotions for staff leadership alongside ECR career development etc.

Great ambition to be non-competitive. If the goal is to maximise learning then we need to encourage honesty about what isn't working. Is this where the gap analysis comes in? Are there other processes people have been involved in that have worked really well to elicit useful learning?

Evaluator per university type - why don't you use the KEF Cluster Groups as universities have been put in Clusters based on their 'type'

Carole - you beat me to it.

@Carole - agree

yes agree

I agree with Carole

@carole - agree

Yes good idea

Agree

Why is a general call for nominations being sent to NCUB partners. You could use PraxisAuril as well

We will Carole, the NCUB point was about going beyond HE

also need to consider 3rd sector and social enterprise stakeholders as evaluators

Agree with Carole, although we are a four nations institution with a non-traditional/ non-campus set-up so perhaps have slightly different opportunities and challenges to implementation and evaluation of KEC principles.

What about involving the AUTM in the evaluation for an international perspective?

Also within HEIs could to hear voices beyond the KE Offices - e.g. would be good to get HR colleagues as an example as much of the concordat relies on HR policies being amended (reward & recognition, training and devt etc)...

Membership organisations could offer a route to non-academic evaluators but shouldn't be the only route

*good to

Sounds like 20-25 evaluators

That matches my maths!

Every 3 years sounds about right.

If this is a pilot, and is non-competitive, why are we killing ourselves to fit this into a timetable which will be impossible for many HEIs, bearing in mind REF clashes and potential loss of colleagues due to COVID-related cutbacks in HEIs??

I agree with Jo.

Every 2-3 years.

As a non-competitive process a rolling approach is an option - e.g. taking 1/3 of universities per year over three years. This also reduces evaluator fatigue and helps build learning from one year to the next.

Every year becomes over kill - every 3 years feels about right

Need to keep this on the internal agenda to keep up momentum for positive change. Maybe 3 year full refresh - with annual updates with lower burden.

every 3 years

I like Dan King’s idea of a rolling process.

Presumably every 5 years to coincide with the action plans?

@Jo … I agree, we need to perhaps not overburden the sector with too many unconnected reviews, though there is benefit in this to help meet the 2.4% agenda

Large exercise every 5-6 years sounds too close to REF and burden associated. Prefer the more frequent, ligher touch suggestions. Also rolloing process approach

@Kathryn - my thoughts exactly. Useful to keep it on the agenda, and to hopefully retain some internal knowledge of the previous submission.

Suggest 5-year plan / time horizon and 3-year interval

I think it would be good for evaluators to learn from other types of HEIs so cross-cluster seems appropriate

Correlating the timelines & purposes of these exercises is key…

Is there some learning from other countries that we can build on?

if it is anticipated that action plans would need to be iterated / refined then 5 years seems too long

The least competitive process I know is that of PRME - Principles of Responsible Management Education. Formative, self reflective, on a signatory basis with a two year progress report/action plan, report available publicly. The expansiveness of this compared would make 3yrs+ logical.

Yes agree @kathryn - this will help drive up change and improvement and keep momentum

3 - 5 years seems reasonable provided it remains light touch. It will keep momentum going and assist with other exercises such as various Concordats, Athena Swan, and eventually REF to monitor progress as we go along.

the 5 year / 3 year option works for my thinking

It's a report on how we're delivering our individual strategies, so some sort of rolling process which ends up synchronising with institutional 5-year cycles might be worth considering.

frequency may depend on how onerous/valuable the pilot is, so may need a short pause after the pilot, but 3 years sounds right

Every 3 years feels about right.

@Jo I understand your concerns but this effort could/should help Government feel comfortable with continuing to fund HEIs to do KE, as well as enabling improvement across sector and within HEIs.

@Dan King, I agree that a rolling process should be considered as an option

Agree with 3 years

@Jo I thought that was the purpose of the KEF

Agree with Carole. Concordat is more about self-improvement than demonstrating value to HMG

Without wanting to kill the panel members, is a single iteration enough for the first run through? HR Excellence in Research which has a similar process at least has the possibility of informal feedback before formal submission - could that be incorporated? Or alternatively a second go if submission needs more work.

I support @Michael's comment

I'm unclear on what this feedback will look like, so difficult to know how to respond to the question being asked

Where deficiencies are identified it would be helpful to have suggestions about how to improve - actual practical hints and tips.

How much of the evaluation material will make its way back to government and in what format do you think, if the motivation is to reassure government we're trucking along nicely, internationally

Yes would be helpful to see an exemplar of what we are talking about re feedback

A grading structure would not be helpful

Would a "disappointing" submission ever be asked to resubmit?

Will there be any public disclosure or summary of the feedback - anonymised or otherwise?

Will the next evaluation refer back to the earlier one, if the evaluation is HEI context based, when/where will the 'what next' be evaluated and how?

@Kathryn: to add, what would make a submission disappointing?

It would be good to have suggestions about how to improve and perhaps a KEC network or similar, where we can learn from each other & share best practice.

Internally this could be used for performance management for the named contact.

Some idea of how your evaluation compares with other 'sector' comparators would be useful - not ranked, but enough to get a view of how you're doing against the 'norm'

Given the challenging environment we are all working in at the moment, pandemic and severe financial constraints HE is facing, will there be an opportunity to explain why an institution may not be able to fulfil their commitments that they signed up to at the beginning?

I wonder whether in the pilot it would be appropriate to investigate the resources assigned by HEIs to write it

As well as ways to improve, also celebrating examples of excellence...showcase short film or similar?

@kathryn, we don't expect to have a pass/fail for this pilot. However there is a chance you might want to work on your action plan once reading your feedback if it suggests there is a lot of room for improvement

thank you phil

Where will the slides be?

I like Shanta's idea.

Agree that some comparison to best practice in evaluation would be helpful.

Many thanks

Many thanks for this webinar- provided some food for thought

Very helpful update and direction of travel, thank you

Really helpful webinar - thanks.

Thanks!

Thank you!

Thanks, very helpful and good to chat to everyone.

Thanks!

These webinars are great, thank you.

Thank you for the presentation, very helpful.

Really helpful - is it perverse to be looking forward to it!

Thank you all, and good bye.

Many thanks for a very useful webinar

Thanks all

Thanks

Thanks - really helpful presentation

@ Shanta - examples of good practice would be really helpful

Thanks, good discussion.

Very useful -thanks

Thank you, very helpful

Thanks All

Thanks for taking the time to update!

Thank you, very helpful!