Assessing Communication Quality and Stakeholder Performance

A reader who’s a risk manager on large physical infrastructure projects (tunnels, bridges, etc.) asked me the following question. “How do you evaluate the quality of coordination and communication between different stakeholders? And how do you track the performance of each stakeholder? (In both cases, stakeholders are teams, not individuals.)”

First, a disclaimer. I have never worked on large physical infrastructure projects. So, I will answer with my experience with other types of business projects with multiple stakeholder teams. That said, let’s answer the two questions.

Assessing Communication Quality

The single most effective way to track the quality of communication between stakeholders is to make, now and then, direct observations: attend calls, read emails, etc., and check whether the communication is honest and direct, whether problems are surfaced and addressed or hidden until it's too late, whether people are proactive or reactive, etc.

Give particular attention to whether people focus on "ticking the box" or on "creating actual progress and derisking the project." Examples of the former include progress updates that are a recap of what happened last week instead of being about possible obstacles and how to address them, problems that are raised and noted down instead of addressed, decisions that are routinely postponed to gather more information, focus on not hurting feelings instead of being upfront, etc.

Also, note whether the attitude is collaborative (“We’re all working together against the common enemy of delays, overbudgets, and low quality”) or adversarial (“We each have our agenda and they are negative-sum”).

Of course, there are other ways to track the quality of communication. Still, they are all either inferior to direct observations or complementary, and I would never use them instead of direct observations. That said, here is a short overview of them:

• Survey participants on the quality of meetings and communication. I profoundly dislike surveys because they are extremely superficial, and when they do surface problems, they seldom surface them with enough context to make them easily actionable.

• Measure response time metrics and decision velocity. This is not a terrible idea, but it does not substitute direct observations, and if you do direct observations, you will probably not need this.

• Track the rework ratio and change order frequency. In theory, this is good to track. In practice, it might invite hiding problems, and in general, it only surfaces problems once it’s too late.

• Track the ratio of issues resolved vs. escalated. Same as above.

• Using AI to analyze communication. In theory, it is good, but in practice, it only analyzes a subset of communication (the written or recorded part) and misses the rest. A better use of AI is to make sure people do not miss important information.

Assessing Stakeholder Performance

First of all, if you have never heard of the Pyramid of Risk, read this. A fundamental principle is that you should track the performance of stakeholders against all levels of the Pyramid, not only the top one, and you should make sure that good performance on one level is not sufficient; a good performance at all levels is required, otherwise things will turn sour soon. What should the levels of the pyramid represent? It depends on the project and stakeholder at hand, but as an indicative example, the top (incidents) could be no defect and delay in the deliverables, the middle (near misses) could be just about discussing small problems and ensuring they are addressed in such ways that they won’t happen again, and the bottom (behaviors) could be good communication quality, etc. It is important that you track both outcomes (lagging indicators) and behaviors (leading indicators).

Secondly, when you assess stakeholder performance during a project (not at the end), the most important question you want to discuss is, “If things continue like this, what does the trajectory for this project lead to?” It is imperative you discuss this collaboratively and not adversarially.

Thirdly, you must dig deep (and ensure that your connections in the stakeholder groups do the same). By this, I mean ensuring that:

• Conversations about progress and possible obstacles do not stay generic and superficial but instead are concrete and specific.

• People routinely check the situation first-hand instead of relying on second or third-hand information, such as reports.

If you don’t dig deep, not only will you only discover problems until damage is done (and when discussing them will get everyone on the defensive), but you (and everyone else involved) will also miss so many opportunities for improvement.

Addressing Discomfort

I know well that everything proposed here requires going out of the comfort zone.

My experience tells me it’s worth it, but that doesn’t make it any easier, of course.

So, here are some considerations that have helped me in this regard in the past:

Direct observations can feel like intrusions – but only if you start pointing out mistakes publicly during the observations themselves. You can prevent them from feeling like intrusions by doing the following: observe not with an attitude of “cathing mistakes” but of “understanding what’s going on and why people do what they do;” genuinely look more for best practices to reproduce rather than mistakes to correct (though, of course, both will surface); always discuss problems collaboratively and not adversarially; do not make problems about the other person (“you’re doing this and it’s wrong”) and instead refer to your experience in a way that makes the problem common (“in other projects, we had this problem, and to prevent this from happening again, let’s do this…”).

If you fear “looking like the bad guy,” present all the initiatives discussed in this project not like “I, Luca, believe we should do this” (which sounds arbitrary, self-serving, and unnecessary) but rather as “Projects like this are frequently a pain for everyone because of this and that problem, and one way to avoid them is doing X and Y.” Everything you do shall be done not to serve you or the project owner but to serve the project.

Half of the discomfort stems from the idea that “there is an easier way” or “you can get away with direct observations and hard conversations.” Experience teaches you mostly can’t.

Discussing problems is easy when done early, before people sink their efforts and ego into something; when done late, it inherently breeds defensiveness and blame-throwing. The more you want to avoid the latter, the earlier problems should be surfaced.

When dealing with multi-stakeholder projects, discomfort is inevitable. You can just decide whether hard conversations will take place before or after problems happen.

I hope this helps.