How to assess a leadership team

What should you measure in a leadership team?

Measure the team, not its members. The aim is a picture of how the team is set up and how it works, so that you can change both.

A useful frame has four layers, set out in leadership team effectiveness. One well-documented instrument, the Team Diagnostic Survey, was built on research about the conditions that foster team effectiveness (Wageman, Hackman and Lehman, 2005). It asks how well a team is structured, supported and led, and about members' work processes and reactions to the team.

Outcomes deserve a place even though they are hard to measure. Drath and colleagues describe direction, alignment and commitment as the outcomes of leadership in a group (Drath et al., 2008). Hackman's three tests add the team's ability to keep working together and its members' own development (Haas and Mortensen, 2016).

Which methods can you use to assess a leadership team?

Four methods are common, and each sees part of the picture. A survey is a practical starting point, and one other method adds depth.

Researchers use observation to study meetings closely. Bang and colleagues combined members' ratings with observers' data from top management group meetings (Bang et al., 2010). Kauffeld and Lehmann-Willenbrock coded 92 videotaped team meetings (Kauffeld and Lehmann-Willenbrock, 2012). Functional interaction, such as problem solving and action planning, went with more satisfying meetings, and better meetings went with higher team productivity.

Stakeholders matter because the first of Hackman's tests is judged by the people who receive the team's work (Haas and Mortensen, 2016). Ask them about outcomes, such as how clear the direction they receive is. Do not ask them to rate individual members.

Popular models can start a useful conversation too. If your team uses one, such as the five dysfunctions of a team, check the research on each part first.

What are good leadership team effectiveness survey questions?

Good questions are about the team. In a referent-shift model, each member answers about the team as a whole rather than about themselves (Chan, 1998). Edmondson's measure of psychological safety, for example, asks about “this team” (Edmondson, 1999).

The usual rules of item writing apply too (Hinkin, 1998). Keep each statement short and simple, in words members use. Make each one address a single issue, because a double-barrelled item mixes two things and confuses people.

Hinkin also cites evidence that reliability rises up to five scale points and then levels off, and he suggests five-point agreement scales. Keep the set short as well, since short measures reduce bias from boredom and fatigue (Hinkin, 1998). For a full measure he suggests four to six items per construct; a quick pulse can use fewer.

Example questions

The statements below are examples written for this article, not items from any published instrument. Each is answered on a five-point scale from strongly disagree to strongly agree.

• Conditions: “It is clear who is a member of this team.”
• Conditions: “This team spends its time on issues that need all of us.”
• Processes: “Each item in our meetings has a clear goal.”
• Processes: “Our discussions stay on the issue at hand.”
• Processes: “When we disagree, the disagreement is about the work.”
• States: “On this team, it is safe to raise a concern about a decision.”
• States: “We can rely on each other to do what we agreed.”
• Outcomes: “The people who report to us hear the same priorities from each of us.”

Notice what each example does. It names one thing, it speaks of “we” or “this team”, and it describes something a member could see happen. None of them asks a member to judge a colleague.

Why does agreement matter as much as the average?

A team average only means something if members broadly agree. Multilevel researchers treat a property such as a team climate as shared only when members' views converge (Kozlowski and Klein, 2000). Before averaging, they check agreement within the group (LeBreton and Senter, 2008).

A common index is rwg (James, Demaree and Wolf, 1984). It compares how much members' answers vary with how much they would vary if people answered at random. A value near 1 means close agreement, and a value near 0 means no more agreement than chance (O'Neill, 2017).

LeBreton and Senter (2008) offered rough bands, from a lack of agreement below about 0.3 to very strong agreement above 0.9. O'Neill (2017) cautions against applying any single cut-off mechanically, including the popular 0.70. He also notes that with few raters, statistical tests of agreement are hard to pass.

The spread can be a finding in its own right. Chan's (1998) typology includes a dispersion model, in which the spread of members' views is itself the property of interest. Cole and colleagues argue that variance among members can add to what the mean tells you (Cole et al., 2011).

In practice, read three things. Is the level high or low? Do members agree? Where they do not, what might explain the split? In a team of four to eight, treat agreement as a prompt for discussion, not a verdict.

How do you keep a leadership team survey anonymous?

Anonymity matters because people say more when an answer cannot be traced to them. One study looked at students known to have cheated (Ong and Weiss, 2000). Under anonymity, 74 per cent admitted it, against 25 per cent under confidentiality. The setting was not a workplace, but it shows how much traceability can matter.

Senior people have their own reasons for caution. Executives in Bang's interviews thought flagging an unclear goal or a drifting discussion would be seen as negative and draw unwelcome consequences (Bang, 2012).

Removing names is not enough in a small group. Sweeney examined how released data can be re-identified, and proposed a rule called k-anonymity (Sweeney, 2002). Each person's data must be indistinguishable from that of at least k-1 others. For a team survey, that means a minimum number of answers before any result is shown.

In a team of four to eight, several common displays can point to one person:

• The lowest or highest score. If one person answered 1, the minimum is their answer, and colleagues often know who is unhappy.
• A distribution. In a small team, a bar showing one answer at the bottom is close to naming a dissenter.
• Subgroups. Splitting six people by role or location can leave groups of one or two.
• Changes in membership. If one person joins or leaves between rounds, the change in the average can help reveal their answer.
• Early results. Scores shown while answers are coming in let anyone who checks before and after a colleague answers see that answer.

How often should a leadership team be assessed?

Start with a baseline, then use short pulses. Haas and Mortensen suggest a quick check every few months, with deeper reviews when problems arise (Haas and Mortensen, 2016).

Teams change, and team research has moved to models that take account of time (Mathieu et al., 2008). A pulse after you change something tells you whether the change worked. Ask the same questions each time, so the answers can be compared.

Space the pulses so that the team has time to act between them. Measuring again before anything has changed adds little.

What mistakes should you avoid when assessing a leadership team?

• Rating or ranking individuals. A team assessment is about the team. Scores per member turn it into an appraisal of people, which is a different exercise with different risks.
• Using results for performance evaluation. If scores feed bonuses or reviews, members have a reason to answer strategically. Keep the results for the team's own development.
• Showing results before everyone has had the chance to answer. Early results can sway those still to answer, and they can expose individuals.
• Leaving interpretation to the leader. Bang's interviews suggest that executives may keep concerns to themselves (Bang, 2012). A leader reading the results alone may miss what they mean.
• Measuring without acting. Agree one or two changes after each round, and check them at the next pulse.

Results on safety and trust are easier to discuss with some groundwork. See psychological safety in leadership teams for ways to make that conversation easier.

What can you do this month?

You can run a simple assessment in a month with a short survey and one meeting. The steps below keep it anonymous and useful.

1. Choose eight to twelve statements across the four layers, each written about “we” or “this team”.
2. Agree the minimum number of answers, and what will be shown, before anyone answers.
3. Collect answers anonymously, and show nothing until the round closes.
4. Share each statement's average and a plain note on agreement, with no individual scores, lowest scores or subgroups.
5. Discuss the results in a meeting, starting with the statements where members disagree most.
6. Pick one change, and check it with a short pulse in two or three months.