Sunday, November 11, 2012

Co-Production of Data and Knowledge

Here's an analogy for the so-called hierarchy of Data, Information, Knowledge and Wisdom (DIKW).
  • Data = Flour
  • Information = Bread
  • Knowledge = A Recipe for Bread-and-Butter Pudding
  • Wisdom = Only Eating A Small Portion

Note that Information isn't made solely from Data, Knowledge isn't made solely from Information, and Wisdom isn't made solely from Knowledge. See also my post on the Wisdom of the Tomato.

That's enough analogies. Let me now explain what I think is wrong with this so-called hierarchy.

Firstly, the term "hierarchy" seems to imply that there are three similar relationships.
  • between Data and Information
  • between Information and Knowledge
  • and between Knowledge and Wisdom
 as well as implying some logical or chronological sequence
  • Data before Information
  • Information before Knowledge
  • Knowledge before Wisdom
while the pyramid shape implies some quantitative relationships
  • Much more data than information
  • Much more information than knowledge
  • Tiny amounts of wisdom

But my objection to DIKW is not just that it isn't a valid hierarchy or pyramid, but it isn't even a valid schema. It encourages people to regard Data-Information-Knowledge-Wisdom as a fairly rigid classification scheme, and to enter into debates as to whether something counts as "information" or "knowledge". For example, people often argue that something only counts as "knowledge" if it is in someone's head. I regard these debates as unhelpful and unproductive.

A number of writers attack the hierarchical DIKW schema, and propose alternative ways of configuring the four elements. For example, Dave Snowden says that "knowledge is the means by which we create information out of data". Meanwhile Tom Graves suggests we regard DIKW not as ‘layers’, but as distinct dimensions in a concept-space.

But I don't see how any of these DIKW remixes escapes the most fundamental difficulty of DIKW, which is a naive epistemology that has been discredited since the Enlightenment. You don't simply build knowledge out of data. Knowledge develops through Judgement (Kant), Circular Epistemology and Dialectic (Hegel), Assimilation and Accommodation (Piaget), Conjecture and Refutation (Popper), Proof and Refutation (Lakatos), Languaging and Orientation (Maturana), and/or Mind (Bateson).

These thinkers share two things: firstly the rejection of the Aristotelian idea of "one-way traffic" from data to knowledge, and secondly an insistence that data must be framed by knowledge. Thus we may validate knowledge by appealing to empirical evidence (data), but we only pick up data in the first place in accordance with our preconceptions and observation practices (knowledge). Among other things, this explains why organizations struggle to accommodate (and respond effectively to) weak signals, and why they persistently fail to "connect the dots".

And if architects and engineers persist in trying to build information systems and knowledge management systems according to the DIKW schema, they will continue to fall short of supporting organizational intelligence properly.

For a longer and more thorough critique, see Ivo Velitchkov, Do We Still Worship The Knowledge Pyramid (May 2017)

Other references

Related post: Seeing is not observing (November 2012)

Updated 8 December 2012
Link added 26 November 2017

1 comment:

Unknown said...

Glad you dislike DIKW too. I liken the distinction to the distinction between different kinds of mechanical parts/systems:

data: "atomic" parts, eg nuts, bolts, pipes, rails

information: complex parts, eg, hinges, pulleys, motors, engines

knowledge: complete mechanisms, eg, bicycles, cars, ovens, refrigerators

wisdom: complete systems, eg roads+gas stations+cars+repair shops

The boundaries among levels of aggregation and composition of mechanical systems are just as vague as those between DIKW.

They also beg the question, "What useful work do such distinctions accomplish?" IMO, neither set of distinctions does any useful work.