Carrier Management: AI, Accuracy, and the Pursuit of Perfection

Artificial Intelligence - August 7 2019

This week, Carrier Management featured an exclusive article by Chisel AI’s CEO and founder Ron Glozman titled AI, Accuracy, and the Pursuit of Perfection. In the article, Ron discusses some of the obstacles that are slowing the progress of AI adoption within the commercial insurance industry, including: 

  • Hesitancy about where to start
  • Questions about how to measure AI outcomes
  • Unreasonable expectations about AI
  • User mistrust of Artificial Intelligence

Ron looks at the issue of accuracy for AI solutions and unpacks how the perils of perfection can hold back innovation. He offers a practical perspective on what insurance companies should really care about when evaluating AI solutions. Hint: It’s not the technical jargon commonly used by data scientists to describe how well a machine learning model works such as accuracy, precision, recall, and F1 score.

How Accurate is Accurate Enough?

In the Harvard Business Review, Larry Clark relates an anecdote that gets to the heart of the problem of AI accuracy:

I was working with a business intelligence executive who told a story to illustrate this problem. His internal client wanted to use a machine learning algorithm to improve his operations. His team was only about 25 percent accurate at predicting certain events using traditional analytics approaches. He wanted a machine learning algorithm that could improve their performance, with a target of 85 percent accuracy. When he was told that the machine learning algorithm could probably get him to 50 percent accuracy (twice as good as what his team could do), the client refused to implement it. Instead of seeing the massive improvement, he said, “Why would I roll out a solution that was wrong half the time?”

As Clark tells it, the unreasonable pursuit of perfection led this client to walk away from a 2X increase in accuracy. Significant incremental improvements in insurance business operations don’t come along every day and failing to seize them is tantamount to leaving a pile of money lying on the table.  

This shows just how deeply humans misunderstand or mistrust AI. An AI solution may be many times more accurate than a human knowledge worker at performing a given task, but the first time the AI solution makes a mistake, the tendency is for users to conclude that the solution isn’t very good and can’t possibly be trusted.

When evaluating AI solutions for commercial insurance, executives should bear in mind that even a small improvement in operational efficiency, underwriting margins, quoting capacity, or speed to bind can have an enormous impact on their bottom line.

Ultimately, the success or failure of an AI solution, like any other enabling technology, should be measured based on its impact on the business. Does the AI solution make it easier and faster for knowledge workers to do their work? Does the solution reduce operational costs, increase underwriting capacity and margins, help mitigate risk, or enhance the quality of the experience you provide to customers? These are the kind of tangible business outcomes that insurance brokers and carriers – and their customers – are seeking from artificial intelligence. And you don’t need to be a data scientist to understand them.

You can read the full article on the Carrier Management website here.



Browse different topics

Recent Posts