Expert Systems

Is there a fundamental flaw in AI implementation, as @jrossCISR suggests in her latest article for Sloan Management Review? She and her colleagues have been studying how companies insert value-adding AI algorithms into their processes. A critical success factor for the effective use of AI algorithms (or what we used to call expert systems) is the ability to partner smart machines with smart people, and this calls for changes in working practices and human skills.

As an example of helping people to use probabilistic output to guide business actions, Ross uses the example of smart recruitment.

But what’s the next step when a recruiter learns from an AI application that a job candidate has a 50% likelihood of being a good fit for a particular opening?

Let's unpack this. The AI application indicates that at this point in the process, given the information we currently have about the candidate, we have a low confidence in predicting the performance of this candidate on the job. Unless we just toss a coin and hope for the best, obviously the next step is to try and obtain more information and insight about the candidate.

But which information is most relevant? An AI application (guided by expert recruiters) should be able to identify the most efficient path to reaching the desired level of confidence. What are the main reasons for our uncertainty about this candidate, and what extra information would make the most difference?

Simplistic decision support assumes you only have one shot at making a decision. The expert system makes a prognostication, and then the human accepts or overrules its advice.

But in the real world, decision-making is often a more extended process. So the recruiter should be able to ask the AI application some follow-up questions. What if we bring the candidate in for another interview? What if we run some aptitude tests? How much difference would each of these options make to our confidence level?

When recruiting people for a given job, it is not just that the recruiters don't know enough about the candidate, they also may not have much detail about the requirements of the job. Exactly what challenges will the successful candidate face, and how will they interact with the rest of the team? So instead of shortlisting the candidates that score most highly on a given set of measures, it may be more helpful to shortlist candidates with a range of different strengths and weaknesses, as this will allow interviewers to creatively imagine how each will perform. So there are a lot more probabilistic calculations we could get the algorithms to perform, if we can feed enough historical data into the machine learning hopper.

Ross sees the true value of machine learning applications to be augmenting intelligence - helping people accomplish something. This means an effective collaboration between one or more people and one or more algorithms. Or what I call organizational intelligence.

---

Postscript (18 December 2017)

In his comment on Twitter, @AidanWard3 extends the analysis to multiple stakeholders.

Framed as risk reduction there are several different types of risk for different stakeholders in your scenario. An extended analysis still works. https://t.co/OvU03vH2eY

— Aidan Ward (@AidanWard3) December 14, 2017

This broader view brings some of the ethical issues into focus, including asymmetric information and algorithmic transparency. 

---

Jeanne Ross, The Fundamental Flaw in AI Implementation (Sloan Management Review, 14 July 2017)

Design Choice and Iteration

There are two misleading ideas (espoused theories) about how architects and designers make decisions.

According to classical decision theory (Herbert Simon), one or more designers develops a series of alternative options, working collaboratively or in competition, and then the design team or the client selects the one that best fits the requirements. This approach may not yield a perfect solution (optimizing) but hopefully produces a good enough solution (satisficing).

Although this approach is used in some design fields, it is unusual in the enterprise space for a designer or design team to produce more than one solution. In the event that the first option is not good enough, the design protocol allows for something called iteration, which seems to repeat certain design steps until a good enough solution is produced.

But in this context the word iteration is misleading. In computer science, the word simply means repeating an action until some condition is met. It is not always possible to determine in advance whether a given iteration will ever terminate - this is known as the Halting Problem.

In design, however, the word iteration seems to refer to something more like backtracking - undoing or discarding some work and trying something different. Designers are generally not taught to backtrack properly. Instead, they learn to avoid backtracking at all costs, and the little backtracking loop on the design protocol comes to be regarded as a safety feature one hopes never to use. (Unless you can blame someone else for changing the requirements or constraints.)

Furthermore, backtracking is popularly seen as a sign of design inexperience or incompetence, and designers who fail to produce a good enough design at the first attempt may try to conceal this fact. Consequently any backtracking that does occur tends to lack visibility and transparency, and hardly any useful learning takes place.

This suppression of backtracking may be one of the reasons why we are surrounded by designs and architectures that aren't very good.


See also Meaning of Iteration (Sept 2012)

Fast data - speed over precision?

@bmichelson posts a piece on an HP website called Fast Data: Speed over precision for better decision-making (21 June 2011), summarizing a recent interview in MIT Sloan with Ali Riaz and Sid Probstein of software company Attivio https://sloanreview.mit.edu/article/why-companies-have-to-trade-perfect-data-for-fast-info/.


Riaz and Probstein are not claiming that better decision-making is just about speed: it also requires a continuous refinement loop, including rethinking one's premises. In other words, we need multiple feedback loops at different speeds, to handle different levels of complexity. As I pointed out in my piece on TIBCO's slogan Two Second Advantage, although simple decisions can be taken quickly, complex decisions need what I call "time for understanding".

One of the architectural challenges of organizational intelligence is to coordinate a complex array of sense-making and decision-making processes operating at different characteristic tempi, and to maintain a proper balance between the very fast (reactive) and the comparatively slow (reflective). Probstein identifies Amazon.com as a successful exemplar.

It is common for technologists to make a fetish of speed, but business effectiveness and organizational improvement need "time for understanding" as well. Riaz and Probstein appear to understand this, and I look forward to seeing how this understanding is supported by their software.

Decision-Making Services

James Taylor (ebizQ) explains why we need a separation between process and decision.

• Interesting debate on business process and decisions (April 2009)
• Decision services (Jan 2007)

One of the reasons for this separation is that decisions are subject to business policies or rules, and such policies are often subject to change over time and/or variation between organizational units. If you code the decisions into process logic, perhaps using BPML to generate process services, then you will have to modify the code or BPLM script whenever the policy changes, and implement different versions for different parts of the organization.

But there are some more flexible ways of architecting decision services.

• General-purpose rule engine, driven by a standard policy language.
• Clusters of related business decisions (for example planning and scheduling) can be wrapped into a capability service.

The advantage of these approaches is that they help to separate the generic parts of the process from the specific parts, and thus increase reuse and sharing of the generic parts, while increasing accuracy and differentiation in the specific parts.

In his post on business processes and decision services, James discusses some of the technical issues that arise with this separation. It is certainly true that some technical optimization may be desirable in some situations, and this may mean implementing and deploying the process and decision services together as a fairly tightly coupled unit of software (which CBDI SAE calls an Automation Unit). But at the specification level, the logical services should still remain distinct.

For more examples of policy separation see my articles on Business Modelling for the CBDI Journal, which are archived on the Everware-CBDI website http://everware-cbdi.com/cbdi-journal-archive.