Talking about evidence-based practice

Talking about the value of evidence-based practice is something we love to do, so it was a pleasure taking part in a podcast with Virtual Not Distant. They specialise in helping people in organisations become more effective at building and managing virtual teams. They also have an excellent blog and podcast series, focusing on how the 21st century workplace is changing and what that means for how we collaborate.

In this podcast, Dr. Richard A MacKinnon, Insight Director at Future Work Centre talks about the importance of using evidence to inform decisions and design interventions at work, and warns that we should all be wary of not being led by fashions and fads. In fact, if we thought more like scientists – asking more questions, being systematic, testing interventions and learning what works and what doesn’t – organisations would make better decisions and investments about their people.

Click here to listen to the podcast: http://virtualnotdistant.com/futureworkcentre/

(Interview with Richard MacKinnon begins around 19 minutes 40 seconds.)

Lies, damned lies and statistics

By Katherine Evans

Data and statistics are all around us, and it’s no different at work. Whether we’re making hiring decisions, analysing employee opinion surveys, or evaluating the success of training programmes, dealing with data is often a key step in making decisions.P1_scientist_logo

How well we do this can make the difference between making a good and a bad decision, so it’s vital that we’re armed with the best tools and skills for the job.

We’re launching a series called ‘Think like a scientist!’ and over the next few months, we’ll share a series of blog posts and downloadable resources, to help you to understand and apply scientific and statistical principles and to challenge research and statistics more effectively. We’ll be explaining how to spot good research and evidence when it’s presented to you and how our own biases and thinking errors can prevent us making rational, evidence-based decisions.

In short, we want to help everyone think like a scientist!

Our first paper ‘Lies, damned lies and statistics‘ explores some of the most commonly encountered statistical mistakes and misconceptions. We explain how data can be used to tell a certain story, and overhyped to make unreasonable claims. We’ll also give advice and tips to think about when you encounter statistics in your organisation, to help you make sure you’re getting the full picture.

Click here to download “Lies, damned lies and statistics”.

Thinking of ditching email? Read this first.

The “no email” trend seems to be taking off – but we’re not convinced. Is abandoning it the right response to our growing frustration with email? Or could we just learn to use it better? 

Why no email?magnifying-glass-email

Writers in the press have recently lauded a new ‘no email’ movement, where members eschew email in favour of other channels, including instant messaging. They report increased productivity and relief at the reduction in the amount of emails they have to respond to.

They’re starting from the premise that email is a problem that needs to be solved. They point to the volume of email we receive as something requiring this ‘nuclear option’. A recent article described it as “the scourge of intrusive, hard-to-follow, back-covering corporate exchanges.”

A little harsh, no?

What’s the problem?

Well, that’s a very good place to start. If we’re going to abandon an entire communications medium, we should be really clear on the problem we’re trying to solve. Otherwise, we run the risk of this being yet another expensive and disruptive workplace fad.

Are we trying to solve the problem of too much email being sent? Received? Too much time taken on reading email? Stress and anxiety arising from email? Inefficiency? Productivity? You see, once you scratch the surface, there are many potential problems associated with email, but if we’re going to address them, we need to be explicit about what we’re trying to do.

And while there are many problems associated with email use, we can’t ignore the ugly truth: people cause emails. Aside from automatically generated ‘spam’ messages we get, the vast majority of the emails in our inbox come from people just like us. Email is the tool, we are the cause of email problems.

Research over the last twenty years, including our own ‘You’ve got mail!‘ project from 2015, has illustrated that email is a double-edged sword. It can be a time-waster, a distraction, a source of anxiety and stress and the origin of interpersonal conflict at work. On the other hand, it facilitates remote and flexible working, allows us to communicate with many people at once, regardless of time zone or country and frees our brains up from having to remember everything that was said – there’s a built-in audit trail.

So what’s to be done?

If we can be specific about the problem we’re trying to solve, we can then move on to some potential solutions. For example, research demonstrates that the volume of email we receive doesn’t relate to email pressure as strongly as some of our own behaviours – like checking email early in the morning or late at night. We can become more aware of our email habits and question how much they’re helping us or hindering us.

Email incivility or conflict could be addressed by teaching people how to both write emails correctly and how to deal with perceived conflict in all kinds of interpersonal situations. Interruptions and productivity issues may also be helped by giving email users guidance in how to use their email application, how to silence alerts and how to focus on a single task at a time, rather than multi-task.

A core problem with email is that it’s a powerful tool, but one that’s simply handed over to users without training or even a user manual. And so we copy what others do, which is rarely a great idea. Maybe it’s time we learnt how to use it better.

So what can I do?

You are of course free to abandon email right now! But think about whether you and your colleagues would benefit more from learning how to use this flexible and powerful tool to better effect, or whether you want to replace it with an instant messaging app, which everyone has to use and will quickly replace email in terms of volume and content of messages. In other words, it will become the ‘new email’.

If you want to improve your experience of email (or know someone who could do with some tips!) have a look at our guides of email use for individuals and organisations. And if you’d like to learn more about our recent research, you can download our paper “You’ve got mail!

Employee engagement: emperor’s new clothes?

Introduction

Last week, we held the latest in our series of ‘Evidence Matters’ breakfast seminars, in London. The focus was ’employee engagement’, an extremely popular topic in the HR world. We wanted to get under the skin of this concept, examine the claims made for its impact on people and organisations and start a debate. And thanks to delegates who attended, we had a very enjoyable and interactive session!

Our focus at the Future Work Centre is evidence-based practice and, while it’s a popular topic, we were determined to get past the hype and examine the evidence for the impact of employee engagement. We launched our white paper “Employee Engagement: The emperor’s new clothes?” at the event and you can download a copy here.

Wait. What’s wrong with ‘engagement’?

We argue that there are four central problems with engagement:

  1. There is no agreed definition of engagement. Looking at the photo below, you can see the variety of responses our delegates gave when asked ‘what do you mean by engagement?’. And they’re not alone. A lack of a shared understanding obviously leads to some additional problems, outlined below.

    Defining engagement

    Defining engagement

  2. Leading on from defining engagement, there is the problem of measurement. Employee engagement is primarily ensured using surveys, yet these surveys vary significantly in their contents and focus. If there isn’t agreement on what’s being measured and what’s included in the measurement itself, how can we make any claims about the topic we’re looking at?
  3. Engagement itself overlaps considerably with other concepts, such as employee satisfaction and motivation. In fact, some measures of engagement correlate with satisfaction to such an extent that they’re essentially measuring the same thing! It looks like engagement is really a combination of several existing concepts, not something new and different.
  4. Finally, our review of the literature on engagement illustrates there is very little quality evidence supporting the claims made about its impact on employee and organisational performance. Yes, there are plenty of case studies and ‘thought leadership’, but these do not represent good quality evidence.

So what’s the alternative?

We’re not suggesting that practitioners throw away their surveys and stop asking employees how they feel about work. We suggest you think about the following:

  • Instead of accepting the claims made by engagement advocates, consider the problem you’re trying to solve with an employee survey. If you can’t define the problem, maybe there isn’t one and maybe you don’t need a survey!
  • If you have a clearly-defined problem, and gathering more feedback from employees would help you understand it better, that should guide the design of your employee survey, not beliefs about what you should be measuring.
  • Find out what works in your organisation. Examine the impact employee opinions have on your key metrics – after all, that’s why they’re key. If you don’t find a relationship between satisfaction and productivity, look elsewhere. If you’re looking to increase productivity, that should be your starting point – not an assumption that increasing engagement (whatever that is) will increase productivity (it probably won’t!)
  • There are alternative and better-validated concepts you can measure in the workplace, if they’re relevant. These include employee satisfaction, commitment and motivation. And there are plenty of measures out there you can use to assess them. Don’t assume that because your survey mentions satisfaction that it measures it scientifically. But be warned, the evidence points to these factors having a weak association with either individual or organisational success.
  • Consider the value you’re getting from your employee survey. Are you analysing your data beyond simple descriptive statistics (e.g. 8 out of 10 people think it’s a great place to work). Does the time and effort expended on regular surveys give you any return at all? Be brave and consider changing your approach.

Find out more

You can find out more about this topic and how we can help you address it, by downloading our white paper or signing up for one of our future free events. Check out our events page for more information.

 

Reconsidering the performance review

magnifying-glass-reviewThe annual performance review is something that remains memorable to managers and their teams – sometimes for all the wrong reasons! Managers often question the workload involved in rating the performance of their direct reports, while the recipients of the review sometimes question the fairness and accuracy of the process. And yet, we continue to do these reviews.

Over the last year, the HR press has reported that a number of high profile organisations are moving away from this kind of performance management, often replacing the annual review with coaching-style ‘check-ins’. At the Future Work Centre, we wanted to examine the narrative of ‘everyone is abandoning the performance review’ from an evidence-based perspective.

We shared our findings at a recent breakfast seminar and in our report, ‘The death of the performance review: fashion, fad or forward-thinking?’.

What did we find?

Organisations conduct annual performance reviews for many reasons – from clarifying goals and expectations, delivering feedback through to helping managers get to know their teams. A key problem with the review is its lack of clear and consistent purpose, even within organisations.

The case for abandoning the performance review is quite compelling. The performance review combines the worst of both worlds: it’s unpopular with those involved and can be deeply time-consuming, particularly for managers with numerous direct reports. From an evidence-based perspective, there’s little data supporting the hypothesis that performance reviews lead to improvements in employee performance.

review-clipboardResearch has also consistently highlighted the lack of fairness and accuracy in manager ratings of performance – simply because managers are human too, falling prey to the thinking errors and cognitive biases we all experience. They may simply reply on the most recent examples of performance, rather than the most representative – something referred to as the ‘recency effect’.

Further, a lack of rigorous training for managers means that feedback can be delivered poorly and the goals set may be inappropriate for the individual and/or the organisation.

That said, we don’t advocate wholesale abandonment of the performance review, without a suitable replacement. So what can we do to ensure the reviews are as useful as possible?

  • Establish clarity of purpose, ensuring that all stakeholders know what the performance review is for and what is expected of them.
  • Goal-setting is effective in many conditions, but must be done properly. Providing a combination of flexibility (should conditions change) and autonomy seem to be helpful here.
  • Training for managers in how to give effective and actionable feedback is key. This training could also highlight how our biases reduce the objectivity of reviews along with strategies for dealing with this.
  • Ensure that rating frameworks and scales are extremely clear and usable.
  • Ensure that your organisation is clear about how people can develop once they’ve received their feedback. Feedback without the support and resources to improve is deeply unhelpful.

We never advocate a one-size-fits-all approach, so the above recommendations, while anchored in evidence, should be viewed in the light of your own organisational context. It might also be useful to consider the following questions if you’re thinking about changing the performance review:

  • How would you rate the success of your performance review as it stands? Have you any evidence that it’s working well or not working?
  • Have you clearly communicated both the purpose of the reviews and the standards you’d like participants to adhere to?
  • Are you clear on how employees view the performance review? Is it helpful, or a time-consuming distraction?
  • Do you know how you would effectively evaluate how well your process is working?

How we can help

Understanding where you are now

We can help you evaluate the process and identify its strengths and areas for improvement.

Improving your current process

If you’d like to improve the quality of your performance review, we can design and run appropriate training for your managers.

Designing and evaluating a new performance review process

If you’re considering abandoning the performance review completely, we can evaluate your plans with you, and help you design and implement an alternative process which meets the needs and culture of your organisation.

So if your organisation is looking to optimise the performance review process, and you want to take an evidence-based route, get in touch: info@futureworkcentre.com

The journey to evidence-based practice

By Charlotte Abbott

At the Future Work Centre, we’re passionate about researching the world of work. Evidence underpins everything that we do, and we believe it’s vital for organisations to take an evidence-based approach in order to make the right decisions.

We aim to bridge the gap between high quality research about people and work, and the actions you take in your organisation. We want to make sure you’re making the right choices, for the right reasons, for the right people.

What are the challenges to an evidence-based approach?

We recognise that it can be difficult to use an evidence-based approach in the way you work. Often, the pressure to make decisions, the need to implement change quickly or the lack of resources mean that taking the time to look at the facts fully just seems unmanageable. However, being aware of these constraints is already a step in the right direction – and we can help you to overcome these barriers.

How can we help?

We’re here to support you in your journey to become an evidence-based practitioner – making decisions that are based on the facts, rather than popular opinion or ‘because everyone else is doing it’. So whether you want to hire the right person for the job, improve your performance management systems or train your leaders, we can help you make a knowledgeable decision, using nothing but the facts and our wide breadth of expertise.

Our Insight into Action workshops have been specifically designed to help you learn more about evidence-based practice. The open and friendly workshops begin with exploring the principles of an evidence-based approach and the influences on its success, such as the intervention itself, who is involved and what else is happening at the organisation. You’ll get the opportunity to explore what makes a skilled evidence-based practitioner, and what bias and ‘faulty thinking’ you may face during your journey. Following this, you’ll discuss your specific organisational challenge or initiative in smaller groups. This time with other professionals allows you to gain more insight into your challenge and see it from a different, more objective perspective, enabling you to leave with practical, actionable steps to address your challenge.

If you’re interested in attending a workshop, visit our Insight into Action page to find out more and book your place on one of the upcoming events.

Additionally, we’re happy to offer these workshops in-house for your team, so please contact us if you would like to run the workshop for your organisation.

At the Future Work Centre, we want to help you on your journey to evidence-based practice, so whether it’s supporting you in the first steps or guiding you all the way to your destination, we’re here to help.

Pathway to success

By Katherine Evans

As an early career psychologist, still new to the world of occupational psychology, I was lucky enough to take part in the Future Work Centre’s session about developing new psychologists at the Division of Occupational Psychology (DOP) annual conference last week in Nottingham. This was a great opportunity for me to share my past experiences as a student, worried about finding a job at the end of my master’s course, and what it’s like to be a trainee on the Future Work Centre’s development programme, Pathway.

It’s well known that today’s graduate job market is tough, and this trend is certainly no different in occupational psychology. Jobs are scarce, and even competition for unpaid internships is plentiful. But there’s another challenge too – while our MSc programmes equip us with the academic theory and knowledge we need, students are generally not taught the professional skills needed to be a successful practitioner or to meet the minimum requirements of potential employers. This means that there is often no clear path for talented but inexperienced new graduates of occupational psychology. As a result, many move into HR, work for test publishers, or end up doing something completely unrelated.

I feel extremely fortunate to have been offered a role at the Future Work Centre, and to be able to undertake their development programme, Pathway. This programme aims to create well-rounded occupational psychologists with all the necessary skills to be successful practitioners. I spend approximately 20% of my working week on developmental activities, such as attending sessions on topics like coaching skills and finance for psychologists, and working on my BPS Chartership entries. What the Pathway programme really means for me is that I can be confident about my future; after three years on the programme, I know I will be well equipped to continue my career in whatever direction I choose to take.

While unfortunately it’s not possible to take every occupational psychology graduate onto the Pathway programme, the Future Work Centre is dedicated to making development available to as many future practitioners as possible. That’s why we’re opening up the Pathway programme, meaning that graduates will be able to sign up for individual modules from the programme, and organisations will be able to incorporate it internally to develop their new talent. So, whether you want to improve your commercial awareness or go through the BPS Chartership process in a supportive, group environment, the Future Work Centre can help with your professional development.

If you’re interested in finding out more about the Pathway development programme, please contact info@futureworkcentre.com

Taking your first steps as an occupational psychologist

By Rhiannon McGuinness

It’s graduation day and you’re feeling a real sense of pride and relief at finishing your occupational psychology course. Whilst celebrating with your friends, one of them asks you ‘So what’s next for you?’

How would you answer?

Within the field of occupational psychology, there is a wealth of different career paths available to early career psychologists:

  • You could work in an HR department, perhaps within a learning and development team, or helping to select and assess the best people for the job.
  • You could work for consultancies with a specialised focus on occupational psychology, which could each have a unique niche, e.g. selection and assessment techniques.
  • You could work within an educational environment, such as a career adviser within an university.

Plus many more!

But what’s tricky is that there isn’t a clear-cut career path as there is with other professions, such as accountancy or nursing. It can feel overwhelming for early career psychologists – especially when it’s well-known within the profession that not all post-graduate courses provide the practical guidance and work opportunities to students that might help them build a clearer picture of their career direction.

So, how do you overcome this challenge?

Being a relatively new occupational psychologist and having experienced some of these feelings myself, I wanted to share my own perspective on what I found most useful in overcoming this challenge.

After gaining some valuable experience within an HR environment, I realised that for me, there wasn’t enough opportunity to share my passion of occupational psychology within this type of role. So, I went back to the drawing board and thought carefully about my ideal role and working environment, and how I was going to find it. I did this by working through the following steps:

The first thing I did was spend time reflecting on the type of psychologist I wanted to be:

  • What positive impact or contribution at work can I make?
  • What is the one thing I can change at work for the better of all?
  • Importantly, what sort of psychologist do I want to be? In which areas do I want to have a meaningful impact?

In answering these questions, I gained a better picture of what was important to me in my ideal role. This helped narrow down my search, and prevented me from repeating previous mistakes of applying for potentially unsuitable work environments simply because they were hiring!

Once I had an idea of what would make work meaningful to me, I was better able to take control of shaping my career, which I did by:

  • Attending networking sessions within the OP field. This not only enhanced my understanding of the various career paths within the profession, it also provided the opportunity to meet with likeminded professionals who gave me valuable advice and support, which really helped me through the next step of my career journey.
  • Completing company research. After attending a few networking events, I had several business cards and delegate lists at my fingertips, which helped kick off my company research. Using these sources, I began to read through company websites, blogs, social media updates, and learnt about different organisations within the OP field; their differing cultures, their ethos and whether this matched up with my own values.

By completing these steps, I was able to gain a greater understanding of the potential work opportunities available to me as a new OP.

I decided that I would ideally like to work with others that are as passionate about OP as I am, and who are driven by the intrinsic rewards of improving people’s working lives, rather than just earning money or awards. This narrowed down my search to a few potential organisations, one of which was the Future Work Centre. And although it’s early days within my role, I can honestly say I have never been happier at work!

So, although this process may initially feel quite daunting, I would encourage any early career psychologists to follow these steps as it could really pay off in the long-term!

Leadership development: some worrying disconnects

A recent post from HR Magazine – Leadership development ‘disconnected’ from business needs – caught our eye and got us thinking about how leadership development methods are viewed by organisations. The article references the 2015 Leadership Survey by Mercer, where 81% of organisations surveyed fail to calculate the return on investment made in leadership development. The survey also points to a disconnect between the learning and development methods that organisations think are most effective and the ones they actually use.

This news is worrying on a number of fronts. Firstly, if organisations believe there are superior leadership development methods available, but fail to use them, we have to wonder why. If, as the article suggests, face-to-face learning is ranked as one of the least effective leadership development methods and yet is used by 63% of firms, are L&D professionals ignoring their effectiveness or just heavily wedded to one method?

Secondly, we would question the ranking of the development methods outlined. What is the evidence for online learning being a superior method of leadership development to face-to-face learning? For all employees? In all kinds of organisations? It’s doubtful that’s the case.

This kind of message has the potential to influence thinking about L&D methodologies, regardless of the context in which they’re applied. And in taking an evidence-based approach, we have to remember the impact on, and perspectives of, our stakeholders. Online learning won’t suit all employees, any more than hot-desking or flexible working will suit all employees. It’s a top-down, one-size-fits-all mentality that doesn’t reflect the diversity of employees.

Thirdly, the low percentage of organisations whose leadership strategy has an associated business case points to another disconnect – that between development activities and organisational strategy. If you’re not developing your future leaders in line with your strategy, you’re simply developing them to move elsewhere, where they’ll find a better organisational fit. At best, you’re developing people into the kind of leaders you needed five years ago, not the leaders you’ll need in the next five to ten years.

Finally, the lack of ROI calculation is worrying – but perhaps not that surprising, as we’ve previously referred to the minimal engagement in evidence-based training and development evaluation. Leadership development represents a significant investment, one that should be evaluated just as any other investment. And while evaluation of these kinds of initiatives can be complex, they’re not impossible.

To connect leadership development with business needs we recommend:

  1. Linking leadership development methods to the evidence base and ensuring they are a good fit for the organisational population being developed. This has to go beyond what feels right, or what has always been done.
  2. Ensuring a robust evaluation process is in place before the leadership development activity even begins. Simply handing out ‘happy sheets’ is just not enough and can’t hope to contribute to an ROI calculation in any meaningful way.
  3. Working to ensure that leadership development strategy is firmly aligned with the organisational strategy, and reviewing this as frequently as strategy is reviewed. This ensures alignment and a periodic refresh of the leadership development ethos and focus.

We’ve also put together a set of resources and events to help you apply an evidence-based approach to evaluating training and development in your organisation.

Beyond the ‘happy sheet’

It’s the end of the financial year and your CEO corners you in the lift and asks: ‘How’s that £2 million training programme going? What impact has it had on our productivity?’ Would you be able to answer?

UK businesses invest significant sums of money in training and development every year – investment that should be objectively reviewed to calculate a return and evaluate its impact. However, the CIPD’s Learning & Development Report 2015 illustrated that the evaluation of training and development initiatives is far from widespread in the UK.

According to the CIPD, one in seven organisations surveyed undertake no form of training and development evaluation at all. Whilst over a third of organisations reported limiting their evaluation to measuring the satisfaction of employees taking part in the activity – what is often referred to as the ‘happy sheet’.

What’s wrong with the ‘happy sheet’?

When organisations rely on what delegates report immediately following training, they miss out on a rich source of data and evidence. Typically, delegates are asked to complete a short survey about their experience of the training, how useful they found it and what observations they have about factors like training venue, facilitator and materials.

This kind of information gives you a snapshot of what the training experience was like, shortly after the training is completed. But what does this tell you about how much delegates learnt, remembered or applied in practice? Very little, if anything.

It is useful information to gather, but represents just one piece of the evaluation puzzle. Let’s look at this another way: are you investing in training and development activities to raise employee satisfaction with training? Is that really your key metric? Probably not. You should be measuring outcomes at a more detailed level, focusing on the factors that are important to you, your delegates and your organisation.

Reliance on the ‘happy sheet’ is superficial at best. It doesn’t tell you how much the training is working, for which employees and in what way.

So, what should organisations do?

While not a new concept in itself, evidence-based practice in the workplace has grown in popularity in recent years. It involves focusing on the evidence behind interventions (e.g. training and development), evaluating this evidence and using the outputs of evaluation to inform decisions. Evidence-based practice means we don’t have to rely on blind faith or gut feel. Or ‘happy sheets’.

Adopting an evidence-based approach means we rely less on what has always been, instead questioning assumptions, seeking evidence, weighing up data and making decisions with the intention to review outcomes and learn from our experience. And learning from experience makes the evaluation an ongoing process, not something that’s only done once.

Evidence can take many forms, including scientific data, stakeholders’ perspectives and your own professional experience. Combining multiple sources of data and evidence will give you a more rounded picture of any organisational challenge and prevent you from making decisions based on a single data point.

Fundamentally, evidence-based practice gives organisations increased clarity on what works, for whom and in what way.

How we can help

We’ve developed an evaluation framework and a set of resources and events to help you adopt this approach in your organisation – as well as field those difficult questions from your CEO.