The limitations of HR analytics in nonlinear systems
One of the many benefits of a summer vacation is the chance to catch up on your reading list. This summer I read the excellent book ‘fooled by randomness’ by the famous author Nassim Nicholas Taleb, who also wrote the book ‘The black swan’. As a professional with an interest in HR analytics there were two short chapters that particularly struck me as relevant for data driven HR.
The book is mainly about the impact of chance and luck on financial markets. He writes in a humorous way about human flaws in judging cause and effect and especially the common mistake of confusing success caused by personal skill with success caused by luck.
In HR analytics we use the logic of math, data and statistics to make better decisions about people and predict the behavior of people. But maybe the reality of work, organizations and behavior is not always driven by logic. Maybe chance plays a bigger role then we would like to think. If that is (partly) true then we are trying to predict a non-logic reality through logically constructed algorithms. It is like trying to understand Chinese with only a French dictionary. Chances are your algorithms will be about as accurate as flipping a coin.
First the author talks about nonlinear systems and effects. A nonlinear effect is a situation where a change in output is not proportional to a change in input. For instance one drop of water can make a bucket of water overflow while the millions of drops before did not have the same impact. We have a tendency to think that life progresses in a linear way. We think that if you study at university for 4 years you will have learned twice as much as when you studied for two years. In reality, your actual intellectual progress can stagnate for months when studying a tough new subject, while on the other hand making a giant leap during an inspiring class of just two hours.
Organizations often perform like nonlinear systems. For example, a manager with ten years of experience may perform worse than a manager with just one year of experience. You can have meetings that last for hours and not accomplish a thing but also make a decision in five minutes that will completely change the performance of an entire company for years to come. A startup with little resources can disrupt a billion-dollar industry. Small events can have a big impact.
Nonlinear systems often behave in a chaotic way and are hard to predict with math. Seemingly small events can lead to major change and vice versa. It is hard to predict which of all those small events will have impact as these systems are often dynamic and can change faster than you can do research.
Most startups will never be able to disrupt an industry. But you just cannot ignore the small chance that one will because if it does your company might be out of business.
Path dependent outcome
A second interesting topic is that of ‘path dependent outcome’. This means that economic outcomes of events depend on earlier economic outputs. In other words: history matters. A famous example is that of the ‘QWERTY’ keyboard. The QWERTY keyboard was designed for old school typewriters. The format is not optimal for our current computers, it is slow and not logical. Attempts to introduce more logical keyboards failed because people got used to the old way of typing and did not want to change. Anybody in a large organization will recognize these patterns. Large corporations often resist change. Things might not be organized in an optimal way but it is hard to change things when people are used to a certain way of working.
Organizations carry a huge amount of legacy with them. Decisions made in the past influence the current situation. Using data analysis to understand and improve processes that are influenced by these path dependent outcomes can lead to misleading conclusions if insufficient historical data was included to truly determine the influence of past decisions.
Add chance and luck to this mix and you get a system that is hard to predict using statistics. Someone might get selected for a promotion because he is friends with the right people, and not because of his skill. A sales rep can land a million-dollar deal because she happens to know someone from high school. We use data analysis to understand these differences in performance while in reality there is a lot of chance involved.
So how can we deal with this in HR analytics?
First off, finding predictable patterns in nonlinear systems is very complex but it is not impossible. There is a field in math called ‘chaos theory’ that deals with this. Chaos theory is complex and it will take a lot of time and resources to get the right results.
Although the effects explained by chaos theory play a role in organizations they do not influence each and every HR process. It is possible to find replicable patterns in a lot of common HR data and these will definitely help in improving decision making. But chaos theory provides us with a glimpse of the limitations of HR analytics. We might never be able to fully understand all the aspects of work floor behavior.
Besides that, we should always be hesitant in accepting large impact conclusions. Just like in science where a theory is only accepted if the same conclusion is reached through multiple experiments. Another best practice in science is running randomized controlled trials and experiments. This is hard in HR analytics but it is not impossible. Sometimes you can offer a similar training or other intervention at different times to different teams. You can compare the team that received ‘treatment’ with the one that did not yet receive training. If you carefully collect data you might get information on actual causality.
In the end HR analytics is not science. We are trying to run a business and make a profit. If your only goal is to make the smartest algorithm in existence you are better off pursuing a career in science. In business there is no time to conduct large scale research when faced with tough decisions under a deadline. But it is good to be aware of the way emotions, chance and chaos influence our judgment. Chance and history have a big impact on the current state of your organization. But in the end a poorly informed decision is probably still better than a decision based on no information at all.