TrendWatcher

Predicting the Future of Analytics

Printer
| More
Predicting the Future

On January 22nd of this year, corporate members of i4cp's evidence-based HR research working group met at the Hertz headquarters in Park Ridge, NJ. Jay Jamrog, i4cp's SVP of Research, asked the group this question about predictive analytics: "Is anybody here creating predictive analytics that aren't about turnover?"

The room was silent.

What HR Metrics Organizations Measure
Here's the thing I often ask when discussing predictive analytics: What is the endgame? The concept is solid in theory - with the right data, reasonable assumptions about the future can be made and thus you can position your investments to get you out ahead of your competitors. Predicting the future gives you the ultimate in flexibility and adaptability.

But what's predictable?

Of course, when dealing with predictive analytics, we're really talking about probability rather than the pure statistical modeling - more Nate Silver than SPSS. There's nothing inherently wrong with that, but it does mean there should be some discussion about what is realistic. Can certain statistics even be predicted? Looking at our latest research on analytics, the top five metrics identified by respondents when asked: "What do you measure?" are outlined in the chart above.

Move from efficiency metrics to effectiveness metrics

Referred to as efficiency metrics, the metrics listed above are often the easiest to collect and generally measure how well HR executes its tasks. Efficiency metrics fall into the "necessary but not sufficient” category. They are worth paying attention to in order to manage HR better, but measuring them is not a driver of business success. Do most business leaders really care about how many trainings were completed last month? How long it takes to hire a new employee? How many people are leaving the company?

The better reward for your investment of time and money comes from effectiveness metrics, which measure the intended impact of HR programs and practices upon the broader organization. Looking at the top five most popular metrics, there is a marked lack of effectiveness metrics.

Here are three effectiveness metrics that, if you aren't doing so already, you should consider tracking:

  • Quality of attrition
    Knowing that 100 employees left your company last month is mildly useful. But how many of those employees were fired? Retired? Were many of them low performers, or did you lose some of your best employees? Tracking the quality of attrition enables organizations to gain a sense of the broader stories underlying talent management, including effectiveness of frontline leaders. Developing a more comprehensive understanding of when attrition occurs, and within which groups of employees, can empower an organization to take action in areas that ultimately strengthen it and contribute to better performance
  • Quality of hire
    Your company filled 100 positions in record time last month. Great. But are those new hires any good? Quality of hire communicates the value an organization is getting for effort and money spent on recruiting. In addition, it also reflects how effective your business is at assimilating its new hires into the organization and works with them to ensure appropriate alignment and development. Indeed, quality of hire is as much a post-hire measure of organizational effectiveness as it is a pre-hire measure of success. It is important to note that quality of hire should be defined based on the business as well as the job role. However, i4cp's research has revealed that certain metrics should be considered, including: first year or project (if contracted) retention rate, first year (or project) performance ratings, and hiring manager satisfaction. Watch i4cp's video on How to Measure Quality of Hire for more details.
  • Quality of movement
    Most frequently, internal movement is associated with an employee receiving a promotion. Increasingly, firms recognize that lateral or even downward movement can be useful. When any type of internal movement takes place, an organization needs to know whether or not the action has addressed a need and delivered value toward achieving company objectives. To truly assess the value of a move, the company must not only track it, but also look at the results that occur because of the move. Did productivity for that position rise? Has the promoted worker remained in the job for a specified period of time?

But what about predictive metrics?

Again, none of those effectiveness metrics are tied to anything predictive. So although some of the larger companies are working on predictive analytics, the best use of time for most organizations is in developing effectiveness metrics.

This is not to say that predictive analytics aren't worth keeping an eye on. As the techniques and models become more refined, there will be greatly accessibility to predictive models. Still, proceed with caution, as predictive models don't always guarantee predictability.

For instance, you may have heard about Big Data. Big Data has a lot of virtues, but one foreseeable issue with huge data sets is that everything can be correlated to everything. The result: making predictions based on those correlations leads to what I call "The world is flat" syndrome, in which assumptions are made based on goofy correlations with no real relation to what's actually causing the change.

Recommendations

There is a future in predictive analytics, but that future is too complex to predict accurately. My prediction? Effectiveness metrics will continue to have the biggest financial impact, at least for the foreseeable future. HR organizations should therefore make sure they are measuring the effectiveness of human capital and its impact on business performance. To get started, or improve existing efforts, we recommend you start small (e.g., focus on critical job roles within a function or division) and use the data you already have or have relatively easy access to. Then, pursue these four actions:

  • Create company-wide standard definitions: Long before decisions can be made using data, the data itself must be reliable. Effectiveness measures need to be consistent across the organization and reflect accurate, up-to-date information in order to be relevant and usable. There are many techniques that can be used to ensure timeliness and reliability, including automation of processes and the formation of data councils.
  • Standardize reporting: Having a familiar look to published reports is more important than many people realize. In addition, consistency in reporting will allow you to set internal benchmarks and see changes over time.
  • Identify the components of the effectiveness metric: If you are measuring quality of attrition, for instance, you need to know your critical roles. Anything that is measured and tracked should be in alignment with broader, strategic talent management.
  • Beware of false trends and generating causation from correlation: The desire to predict based on perceived trends can be strong. Remember that there are hundreds of factors, if not more, than can affect any given outcome.

Comments

Very clear and helpful article. With all the hype about Big Data and predictive analytics people in HR can tend to be confused. Reading Freakonomics gave me a healthy and realistic respect for the power, but also the cost and sophistication of predictive analytics. Pointing HR to effectiveness measures is a good contribution.

Thanks Cliiff
I think there are stronger statistical test that get down to causation that can be use in HR. I think Structured Equation Modeling is one of those methods that can help you be predictive in areas that matter. Like identifying those high performers that are at RISk for leaving the organization. I still believe the future is getting to predictive in the HR arena just like we have in marketing.
@Jeannie - Thank you very much. I also enjoyed Freakonomics - it's a great example of both the usefulness and danger of relying on statistical inferences.

@Cathy - I agree wholeheartedly that HR analytics can learn a lot from the fine folks in marketing, and I also agree that SEM is one of the best statistical tools for identifying causality. I do wonder, though, about what the final result will look like. For those companies that are able to predict flight risk, how will they use that data to mitigate that risk? I still believe predictive analytics are the future, but that the current cost of pursuing predictive analytics might outstrip the potential rewards, except in extremely large companies (where there is a large enough population for large scale statistical modeling).

I'm very interested in those companies that have a predictive model that they trust, as I do think it can be very useful. I think that further insights into how that model was made and implemented would be of great interest to the community.
Cliff--great stuff. Two things I would add: 1) the predictive analytics and metrics are all fine to have, but the analytics need to be made actionable and practical for front-line leaders to actually move the needle on the predictive metrics, 2) the cost of predictive analytics is not out of reach for any organization--all of our tools incorporate structural equations modeling (e.g. employee surveys, 360s, perf mgmt, succession planning etc) and we do this work for organizations as small as 100 employees to over 300,000 employees.

The danger of this new focus on just analytics is that it will only make HR better at a bad game--running more reports that don't connect to real business outcomes.
Thanks Scott - couldn't agree more that analytics in and of themselves are not the answer, it's definitely the decision making and critical thinking that is really what missing. And yes, it may be overly simplistic to point to cost as the main detriment to producing predictive analytics. However, would you agree that forces outside of the organization can affect predictions (e.g. the general unemployment rate could be related to turnover)? It's the sheer number of factors (internal and external) that I think makes prediction, when it comes to humans, so difficult. All that said, I have seen your model, and I believe you are spot on with the SEM approach.
Thanks Scott - couldn't agree more that analytics in and of themselves are not the answer, it's definitely the decision making and critical thinking that is really what missing. And yes, it may be overly simplistic to point to cost as the main detriment to producing predictive analytics. However, would you agree that forces outside of the organization can affect predictions (e.g. the general unemployment rate could be related to turnover)? It's the sheer number of factors (internal and external) that I think makes prediction, when it comes to humans, so difficult. All that said, I have seen your model, and I believe you are spot on with the SEM approach.
Cliff--absolutely agree that forces outside of the organization can affect predictions. One of the key tenets of SEM is to include as many relevant factors/variables as possible (such as unemployment rates, demographics etc) in the analysis. This way you create a full model and can zero in as closely as possible on what is really 'causing' business outcomes to move. It's important to be conservative with predictions and really help front-line managers to prioritize what they should work on, whether it's employee attitudes, training, competencies etc. It's also important to remember that the Finance function faces the same issues--their algorithms are full of assumptions and unknowns too--and yet organizations make HUGE investments based on their predictions........
Add a comment

 Name (required)
 Email (will not be published) (required)
Commenting on this article requires human verification. Please enter the two words in the textbox below before clicking the post button
By posting, you agree to i4cp's Terms and Conditions.