"A person may take his umbrella, or leave it at home, without any ideas whatsoever concerning the weather, acting instead on general principles such as maximin or maximax reasoning, i.e. acting as if the worst or the best is certain to happen. He may also take or leave the umbrella because of some specific belief concerning the weather. … Someone may be totally ignorant and non-believing as regards the weather, and yet take his umbrella (acting as if he believes that it will rain) and also lower the sunshade (acting as if he believes that the sun will shine during his absence). There is no inconsistency in taking precautions against two mutually exclusive events, even if one cannot consistently believe that they will both occur." [Jon Elster, Logic and Society (Chichester, John Wiley, 1978) p 84]
Austrian physicist Erwin Schrödinger proposed a thought experiment known as Schrödinger's cat to explore the consequences of uncertainty in quantum physics. If the cat is alive, then Schrödinger needs to buy catfood. If the cat is dead, he needs to buy a spade. According to Elster's logic, he might decide to buy both.
At Schrödinger's local store, he is known as an infrequent purchaser of catfood. The storekeeper naturally infers that Schrödinger is a cat-owner, and this inference forms part of the storekeeper's model of the world. What the storekeeper doesn't know is that the cat is in mortal peril. Or perhaps Schrödinger is not buying the catfood for a real cat at all, but to procure a prop for one of his lectures.
Businesses often construct imaginary pictures of their customers, inferring their personal circumstances and preferences from their buying habits. Sometimes these pictures are useful in predicting future behaviour, and for designing products and services that the customers might like. But I think there is a problem when businesses treat these pictures as if they were faithful representations of some reality.
This is an ethical problem as well as an epistemological one. I have a recollection (I can't find the details right now) of a recent incident in which a British supermarket, having inferred that some of its female customers were pregnant, sent them a mailshot that assumed they were interested in babies. But this mailshot was experienced as intrusive and a breach of privacy, especially as some of the husbands and boyfriends hadn't even been told yet.
Instead of trying to get the most accurate picture of which customers are pregnant and which customers aren't, wouldn't it be better to construct mailshots that would be equally acceptable to both pregnant and non-pregnant customers? Instead of trying to accurately sort the citizens of an occupied country into "Friendly" and "Terrorist", wouldn't it be better to act in a way that reinforces the "Friendly" category?
Situation models are replete with indeterminate labels like these ("pregnant", "terrorist"), but I think it is a mistake to regard these labels as representing some underlying reality. Putting a probability factor onto these labels just makes things more complicated, without solving the underlying problem. These labels are part of our way of making sense of the world, they need to be coherent, but they don't necessarily need to correspond to anything.