Prepare for the challenges of deploying machine learning for analytics


 

analytics1_090513.jpg

The pressure to exploit analytics
as a strategic advantage is increasing for an obvious reason: The economic
future is becoming more uncertain, and the entire point of analytics is to
manage uncertainty. But just as the uptake of business
intelligence (BI) over the past decade had its learning curve and practical
barriers, analytics also presents a set of formidable challenges to an
enterprise that hasn’t yet invested.

Conventional analytics — which standard
BI tools and many easy-to-use third-party packages enable — extend familiar
measures and indicators along extrapolative lines that allow decision makers to
look ahead or tighten the focus of well-known metrics to eliminate waste.
That’s all useful, and readily within the grasp of most organizations, without
much outside assistance.

Advanced analytics (i.e., data-driven predictive power based on models that don’t come from a human
expert’s head) is a capacity that goes in different directions altogether, to
the point of requiring a new way of thinking among those who want to master
it.

The high cost of uncertainty

Conventional analytics are often
linear and easily visualized. Those who have done a deep dive into Microsoft’s
SQL Server BI suite are acquainted with the ease-of-use the stack offers, its
endpoint more often than not being Excel, where casual BI users can crunch
carefully culled datasets to their hearts’ content, combining what-if with what
they already know very well.

Advanced analytics perform a different function: Bodies of data are scanned for hidden patterns,
often with no foreknowledge of what those patterns might be; this requires
specialized software (Hadoop and its many cousins) and often calls for dedicated
hardware. The critical distinction between conventional and advanced analytics is the former is guided by a human mind and intent on a particular result, whereas the latter is open ended with nobody at the wheel.

Staging data for driving

Advanced analytics occur when a
dedicated computer scans large bodies of data for hidden patterns,
relationships undetected that can yield critical business insights. This is far
beyond looking to historical data for verification of what someone has already
gleaned or intuited, or to trim an existing relationship for efficiency’s sake.

Diving into an ocean of data in search of undefined treasure is an exercise in
self-discipline and asks the user to surrender even the construction of the
algorithm that does the exploring. When machine learning is employed, the
algorithm doing the pattern detection refines itself as it progresses, free of
human oversight. This is a serious shift in mindset regarding the process at
work — the people digging for results are making a conscious decision to trust
the machine.

Matters of interpretation

Another huge shift in mindset when employing machine learning is understanding that you’re relying on the
machine to tell you what’s important. In conventional analytics, you know what
you’re going for when you start your search; when the machine learns, you don’t have an
outcome in mind (though you may be hoping for one), and you take whatever the
machine gives you.

That’s hard enough to do if
you’re new to this style of BI, but it gets really
challenging when you turn discovery over to the machine learning process, and
you’re presented with results that take you places you weren’t expecting to go.

The ultimate challenge of machine
learning as an analytics platform is the revelation of patterns within data
that bear no obvious connection to what was previously known. For instance, you’ve studied
client portfolios and discovered hard truths about why former clients
defected; or, you’re looking at your markets and finding that sales drop off
consistently at certain times for reasons that are your fault. The
machine produces patterns that demonstrate realities you never would have
thought to seek.

Where it gets painful

Studies have consistently shown that data-driven analytics yield decisions that
are more effective than human expert opinion can deliver. This is humbling,
even humiliating, but solidly established. Numbers don’t lie, but human beings — even executives with the best intentions and the most successful records — lie
to themselves.

Committing to machine learning, and driving business decisions
from the data upward, demands resolve, humility, and a
true sense of discovery.

 

This entry was posted in Big Data. Bookmark the permalink.