360 Surveys

Positive Harm™

How Great Training Can Cause Long-term Harm to Organizations
(and the key to stopping it)

By Bruce Bennett

Trainers Are Driven To Help People Get Better.  That’s Why I Chose Training As A Profession.  We Are The Doctors And Nurses Of The HR World.  Great Training Motivates, Inspires And Is Designed To Drive Improvement.  Great Training Created One Of The Most Significant Crises Of My Career.

I was the on-site training manager for an organizational wide training effort at our largest client.  After a year of training, I was not seeing any of the improvement we promised.  I was getting worried and began a quest to understand why.  Over time I came to recognize Positive Harm™.

The Lesson Of The Frog

Alain was a Senior Executive we were lobbying for support.  We were discussing the employee reaction the training.  By all accounts, the training was a huge success.  I noticed a small, delicate, glass frog on Alain’s desk. The frog seemed out of place, odd.  As we talked with Alain, I had to ask about the frog.  Alain had been hoping for the question.  “That frog,” he said, “would always be on my desk as a reminder of one of the most important lessons I’ve ever learned.”

Alain was from a small town in France, famous for the European storks that nested around the outskirts of town. People came from all over Europe to see the huge nests, and majestic storks. Ever larger crowds spawned an annual stork festival which developed into an economic engine, supercharging the local economy. The town grew and sprawled beyond it’s traditional boarders. 

The town reveled in its growth.  It was years before town leaders realized the decrease in visitors was not an anomaly, it was a trend.  The decline persisted.  The town and fell on difficult financial times. 

Town leaders hired consultants to save the struggling economy.  The consultants linked the downward trend in tourist visits to a decline in the number of storks that nested in the areas surrounding the town.  In spite of the success of the all the festival activities, people really came to see the storks. 

The town leaders hired scientists to figure out how to attract more storks to the area.  It did not take long for the scientists to change their focus from the storks to a tiny green frog, a stork delicacy, that lived in the marshlands on the outskirts of town.  New construction damaged the habitat of the small green frog and the frog population was on the verge of collapse. The storks quit coming because there were fewer frogs to eat.  The visitors quit coming because there were fewer storks to watch.

Alain paused then said, "I keep the frog on my desk to remind me to always look out for unintended consequences.”

Where To Find The Harm

As Alain finished his story, I knew I needed to shift my attention from the success of the training in the classroom to discovering unintended consequences of the training outside the classroom.  Could it be that the overwhelming positive training experience was somehow hampering the long-term desired results? 

I interviewed trainers and trainees.  An interesting theme began to emerge. In the classroom,  trainers painted a compelling picture of hope, change and organizational improvement.  Expectations for change were very high. Outside the classroom, people hit a very different reality.  There was little enthusiasm or support for change. Coworker reactions to the training content ranged from the silent treatment to ridicule.  A few trainees tried to implement the training concepts.  Most trainees didn’t dare try.  Many trainees felt betrayed.  

Each training session was full of positive experiences.  Now, I had discovered the harm.

Positive Harm™ happens when great programs create high expectations for change but nothing changes.

How Is Positive Harm Manifest?

Positive Harm™ happens when great programs create high expectations for change but nothing changes. Often, it’s not until after the training that hope and reality clash.  Individuals have to process the aftermath alone. The excitement of the training can be so loud and demanding, it can be months or years before leaders realize that the difference between expectations and reality is harming the chances of real change.

Continued interviews and research identified three forms of Positive Harm™ directly tied to great training.

1) Individual Cynicism 

When high expectations are dashed against an unyielding reality, it’s a spawning ground for cynicism.  Many organizations have developed have code phrases to describe individual cynicism about change.  

One training class had just started when a long-time employee interjected, “Welcome everyone to the next flavor-of-the-month.”  The resistance was out in the open.  The trainer decided to address the issue head on.  The employee was given an opportunity to explain. The employee said he appreciated the time off the job, but based on his years of experience, it would take management 6 - 9 months to see that nothing was changing and they would bring in a new program and training would start all over again.

Another employee described his attempt to use one of the skills learned in training with a coworker. The coworker was familiar with the training content and when the employee used the new skill, the coworker said," don't pull that <<insert training concepts here>> on me“.  He was not willing to try to use another new skill.  He had decided the material was good, but it would not work here.

2) Organizational Resistance 

When you have enough cynics, resistance to change starts in infiltrate the culture.

At one company, organizational development professionals from all around the company were invited to a special meeting at a manufacturing plant. No one knew why they had been invited.  The meeting began with a brief introduction form the plant HR manager who then said," We've spent three years and $3 million implementing what is widely recognized as one of the best training programs in the country. After 3 years and 3 million dollars, we haven't moved the needle on any meaningful performance measure and we are trying to figure out what's next." 

As the meeting proceeded, it became clear that after spending three years and millions of dollars with no measurable results, the organization was frozen with indecision about what to do next.    We had been invited to the meeting not only to help them figure out what’s next, but to help them find the will to undertake a new change initiative. 

After the meeting I went out on the floor and interviewed employees about the training. In the minds of the employees the training was a huge success, very positive.  Most knew the skills taught in class and could recite the training concepts. Many described how the training concepts had helped their personal lives. 

When asked why the training had not produced change, the employees all provided unrelated to the training.  When asked what they thought needed to be done next to improve productivity, they expressed the same unwillingness to try something new again.  One employee said, “if this didn’t work, nothing will.”

The next productivity initiative will meet significant organizational resistance before its even launched.  

Often, it’s not until after the training that hope and reality clash.

3) Leadership Diversion

One training team was brought into a manufacturing plant to "fix the frontline supervisors." The executive committee provided a specific list of changes the front line supervisors needed to make. The team developed great training that was very well received. 

It didn't take very many training sessions to discover that the frontline supervisors learned how to do their jobs from their superintendents. The superintendents hand learned how to do their jobs from the area managers. And, the area managers had learned how to do their jobs from the executive committee.  Nothing taught in training would be able to overcome the influence of the executive team, area managers and superintendents.  

The executive committee had checked the ‘fix it’ box by hiring trainers to address the problem.  No matter how good the training, it was clear to the trainers that change is unlikely until the executive committee address the problems up the leadership chain. Trainers tried to motivate people to push change up the chain.  Those being pushed resisted. Those pushing because cynical and stopped pushing.  Yet, the executive committee heard such wonderful things about the training, they decided the long-term answer was more training.  

Unrecognized Positive Harm™ becomes a subtle destructive cycle.  Eventually the damage becomes so great it can’t be ignored.

The Key To Stopping Positive Harm™

The great news is that when you recognize Positive Harm™ and get the data to understand it, you can eliminate the harm and build on the positive.  As I’ve told these examples to others they often respond with descriptions of Positive Harm™ in their organizations.  When you can identify Positive Harm™ you have discovered a significant, unaddressed and often unspoken barrier to change.  And, you’ve found the key to eliminating the harm.

One simple question will help you identify Positive Harm™ in your organization.  "If not, why not?”:

  •     If trainees did not use the new skills on the job, why not?  
  •     If trainees did use the skills but are not using the skills any more, why not? 
  •     If trainees understand the new skills and are not using them, why not?

In the “fix the front-line supervisors” example above, trainers started to hear stories of Positive Harm™.  The trainers asked for permission to do “if not, why not?” interviews with people who had been through the training months earlier.   After many interviews the stories started to fall into clear categories.  With enough stories that they could describe the categories in a way the protected the anonymity of the story tellers, they took their finding to the executive committee. 

After careful consideration of the new information, the executive committee stopped the frontline supervisor training and decided the Executive Committee should go through the training first. Once the executive committee had been trained, the area managers were trained, then the superintendents. By the time the front-line supervisors were signed up to come to training, all of their leaders had been trained and were using the training concepts in daily conversation.  The front-line supervisors came to training with a whole new willingness to listen, learn and act. 

One simple question will help you identify Positive Harm™ in your organization.

Training Can Be All Positive With No Harm

Alain tells me that it took years of careful planning and constant monitoring, but the habitat for the frogs was restored.  The storks are back.  The visitors are back.  The town is thriving again.  Once the town leaders identified the Positive Harm™ of success, the knew how to sustain growth. 

Great training does not always cause Positive Harm™.  If it does, and the Positive Harm™ goes unnoticed, the Positive Harm™ becomes a downward spiral that can incorporate the harm into the culture.  The “If not, why not?” question identifies the harm.  Armed with that data, organizations can stop the harm and accelerate the positive impact of training - great training can produce great results when Positive Harm™ is eliminated.

The Flaw In The Fatal Flaw With 360 Surveys

By Bruce Bennett

A 360 Flaw?

In his HBR Blog post, “The Fatal Flaw with 360 Surveys”, Marcus Buckingham argues that 360 surveys are “at best, a waste of everyone’s time, and at worst actively damaging to both the individual and the organization.”   Yet, 77 % of organizations surveyed by T&D magazine have used and continue to use 360’s.  Something does not add up. It turns out that the ‘Fatal Flaw’ is faux flaw only found poorly constructed 360 surveys.

It’s clear that Marcus has had some bad experiences with 360 feedback and those experiences color his perspective. Condemn all cars because the engine in the lemon you bought is broken and you miss the value of every other car on the road. Marcus describes a ‘fatal flaw’ in 360 surveys that is equivalent to a broken engine in a lemon used car. 

The flaw Marcus has discovered occurs because raters score a leader subjectively.  Raters, he suggests, mentally compare the leader to themselves and score accordingly, making the data worthless.  This is a flaw faced by every survey, not just 360 surveys.  It is not a flaw, it is a reality of collecting survey data. It becomes a flaw when a 360 survey contains poorly written questions.

The Flaw Disappears

Write the questions well and the flaw does not exist.

A specific, observable behavior is among the first requirements for a good survey question. If the behavioral description is clear, anyone and everyone observing a leader can recognize when the behavior is performed.  In statistical parlance it’s called inter-rater reliability.

Such observations are not only the basis of good 360 questions, they are the basis of all scientific research.   Whether it’s a biologist counting bacteria in a petri dish or a naturalist studying tree frogs in the Amazon, the quality of their data depends on how well they have defined what they are looking for.

In a 360 survey question, the behavioral description is combined with a scale, both have to be good for the question to work.  Chose the wrong scale and you destroy the question.   Agreement scales are very risky because you are trying to measure what the person is thinking and you can’t see ‘thinking’.  A frequency scale, combined with a clear behavioral description, creates the condition for valid observations.  How many bacteria in the petri dish? How many times did this leader ____?  

Take the question, “Do you agree this person has a clear vision for the future?” The flaw Marcus describes is present.   No answer can be independently confirmed because no one rater can see the thinking that produced the answer.  Change the question, “How often does this person discuss her vision with you?” and you will get data that can be confirmed by anyone witnessing the discussions. The flaw disappears.

Sample Size

How many people have to agree before their observations are worthwhile?  Researchers pull a representative sample when it’s too difficult to poll the whole population.  Random samples are critical with large populations.  Respondent groups in 360 feedback are small populations and often 100% of possible respondents are invited.  100% is the entire population, its not a sample - so it can’t be a skewed sample.  If the question is, “how often does the leader do behavior X when working with his direct reports” and 100% of his direct reports respond, you can’t get a better sample.

If all research had that luxury of polling 100% of the research population, the Chicago Tribune would not have declared Dewey the President on election night in 1948 and no one would have to wait up for the election results today.

The Importance of the Gap

More important than sample size is the question of what the assessment is measuring?  

An excellent 360 does not try to determine if a leader POSSESSES a particular set of skills, instead it measures if the leader DEMONSTRATES the skills.  

An excellent 360 is not just about self-awareness, it’s about illuminating the environment in which the leader functions.  A difference or gap in respondent group scores does not mean one respondent group is right and the other wrong.  The gap may not mean the leader is lacking a particular skill. It does means there is an important difference in the perception of reality and that difference can inhibit the leaders ability to lead.   

Bridging the gap may not be a skill issue, it may be a motivation issue, a system issue or a lack of information.  360 feedback is a powerful diagnostic tool that highlights issues then tells the leader what group to talk with and what to talk about to uncover root of the issue. The gap data helps effective leaders understand the real issue, then make relevant changes to resolve the issue.

Not all gaps are bad.  Often others rate a leader higher than the leader rates herself.  In many high performing organizations, the opposite of ‘benevolent distortion’ occurs.  Leaders, driven by the sense that they can never be good enough, rate themselves lower than their actual performance. 

That type of gap identifies an underutilized asset in the leaders toolbox.  Understanding how valuable others consider a specific behavior informs the leader how to build on her strengths and use those underutilized, highly valued behaviors more often.

Gaps illuminate the environment in a way that makes it easy to understand. 360 surveys are the most effective tool for finding gaps

360 Surveys Work Well

360 feedback is hard to do well.  Observing human behavior is not as easy or exact as counting bacteria in a petri dish, but clear behavioral descriptions and the right scale produce valid data.  360's have been used for more than half a century and organizations keep using them because, when done well, 360's are valuable tools that help leaders get better.

Even Marcus admits he’s “…seen some extraordinary coaches use 360 results as the jumping off point for insightful and practical feedback sessions.”  Extraordinary coaches don’t use flawed data and to know what to do with good data.  360 surveys are a powerful leadership development tool.   

If you happen to buy a lemon, with a broken engine, don’t let it prevent you from experiencing the thrill and performance of a well-designed automobile.