More on well-posed questions and how to ask them

Yesterday we discussed what are called 'well-posed" questions.  Science is the enterprise by which we try to answer questions we have about Nature and the world.  Science is, if anything, a kind of method of investigation, a philosophy of what counts as knowledge.  But it's not trivial that you can't answer a question until you've asked it, and if you haven't done that properly, well....

The history of western approaches to asking and answering questions about Nature is long and varied.  In the earliest western writings, the 'classics' such as Ptolemy, Aristotle, the Pythagoreans, Hippocrates, Euclideans, and the Epicurians, the idea was that one could observe the world and then think-out theories of how the world is.  There were experimental works and formal theories, like geometry, but much was done without formal or systematic ways of codifying knowledge. Science was, to a great extent, opinion.  There were schools of thought populated by groups with different opinions, but little way to resolve them.  Some became dominant (such as the four humors and medicine), but their epistemology--their knowledge basis--was informal.

French salon in the age of Enlightenment; Wikipedia

In the so-called 'Enlightenment' period in European history (to our chagrin, we know very little about the history of science and philosophy elsewhere in the world), the idea took hold that only by careful, systematic observation of specific, constrained instances of Nature could answers to general questions be developed.  Induction based on repeated observation, reduction to basic causal elements, and formal types of reasoning (especially, mathematics) were approaches to the world.  Underlying this (as in the classical times before, to an extent), was the idea that Nature was a material phenomenon (perhaps started and occasionally interrupted by God, depending on one's religious views), that followed consistent processes, or 'laws'.  If Nature were not, then it would be un-knowable because what we thought we knew could always be excepted here, there, or anywhere.

Along with laws came notions that we now know as well-posed questions.  As we said yesterday, a well-posed question is one that is available for study by proper science.  Whether the Mona Lisa or pepperoni pizza are good are not well-posed questions.  Among many reasons, the answer differs for any person, and without any seriously known predictability, and they are personal judgments that, unlike laws of Nature, can change willy-nilly.

The fallibility of well-posed questions
But, as we noted yesterday, a well-posed question should have these sorts of properties:

1. It is structured so that a solution exists, at least in principle 
2. Such a solution should be unique 
3. The solution's behavior changes systematically with the initial conditions 
4. An appropriately sufficient set of variables should be adequately measured.

But there are ways perhaps to clarify some of the problems that we didn't mention.  For example, most statements in science rest upon definitions.  Definitions of terms for things in the world can be arbitrary--we choose them as humans, assuming they apply to reality.  But they need not be as applicable as they sound.  So a scientific question can be clear-sounding but in fact not well-posed:

Q1:  "What is the cause of type 2 diabetes?"

Sounds fine, and it is a simple sentence, but does it meet the criteria of a well-posed problem?  And is it answerable?

For starters, one must define what is meant by 'type 2 diabetes'. The implication is that it is a state of being (assuming we're discussing humans, or at least mammals).  But what 'state'?  One must specify more, or as the humanities-jargon would have it, 'unpack' the phrase.  Some measurement is implied, but what?  Or is it some outcome?  If we define 't2d' to be some level of blood glucose, measured in some specific way, then we might argue that Q1 is well-posed.  Indeed, we can answer it with a discussion of the biomechanism that causes high blood glucose.  But that's not helpful if what we really want to know is why that mechanism has gone off in some people.

And, often what happens is that the definition changes--diagnostic glucose cutoffs are changed, or the definition changes from glucose levels to some other trait, such as blood pressure, neuropathy, vision problems or some set of these purported glucose-dependent outcomes.  That is we can define the trait in terms of a risk state (glucose levels) or an outcome state (kidney failure).

Blood glucose meter; Wikipedia

Now the question, no matter how simple in structure, is not so clearly well-posed.  Even with a constant definition, the measure can change.  We measure not blood glucose but glycosylated hemoglobin, or we use a different measuring machine.  Then we face various data consistency problems, and certainly whether past data can be used any longer.

Or, we could say that 'diabetes' is the state when you have a particular genotype.  That may sound strange, but in a law-like universe isn't the genotype just a form of trait, and other things like glucose level or kidney failure just secondary, indirect measures of the  'true' trait?

Q2:  "Do factors f1, f2, f3, and f4 cause diabetes?"

This version sounds better because it's more specific.  Here, let's now forget the problems with defining diabetes, taking that for granted.  We just move the same issues towards f1--f4: defining them and how they are measured.  Now we have to consider whether the question means all these factors, or just 'some combination' of them.  If the latter, then the question is not well-posed until it is re-phrased to be clear about what 'combination' means.  Presence or absence?  Some quantitative measure on the factors?  If f1 alone never leads to diabetes, is it a 'cause'?

If we define terms clearly then one might assume that 'a' solution to the question does exist, at least in principle, at least for some specific data.  But if definitions can change then the solutions will change, having no bearing on Nature but only on human culture--the way we choose to make our definitions and measurements.  One thing that is clear is that most of the time questions like Q2 are not clearly posed.

What about uniqueness?  If the answer is not unique, then the law-like assumptions about Nature are somewhat strange.  Causes should have outcomes and if you measure carefully, shouldn't you get just one outcome?  Or one cause per outcome?  But we know that's not true (in the broadest sense, there are many ways to die, but there are also many paths to heart disease, or asthma and allergic response, and so forth.)  If many different causes have purely law-like properties, then Q1 is actually not a well-posed question, no matter how simple and clear it sounds.  Q2 might be better, for example, if each possible combination of the f's has a unique outcome; but if there is a factor f5 that is not being measured, the question will not have a unique answer.

For this reason, the escape from the idea of uniqueness is to say that many different causal situations each uniquely cause diabetes is correct in a sense, but then even Q2 is poorly posed.

Q3:  "Do factors f1, f2, f3, and f4 cause diabetes by altering its probability of occurring?"

This is a common escape-valve way to pose the question.  It's an escape valve because it introduces probabilities, which has to do with repetition of identical conditions, which never happens. But it allows almost anyone to collect almost any sort of data and 'answer' the question in a technical sense of getting an answer.  Probability is an ethereal concept but one that inevitably involves fractions of outcomes of a given type out of multiple outcomes, real or imagined.  Or it may mean something about the fundamental way that the factors act or interact.  One attractive attribute is that it can mean anything and you don't have to specify in clear terms.  Statistical analysis yields a test, but since the outcome may or may not occur, Q3 can always have an 'answer'.  If some statistical significance level is used, the answer is 'yes' or 'no', but such levels are subjective not objective.  And even if the true (but unknown) answer is 'no', we know from sampling theory that finding exactly zero effect is unlikely even if there is no actual causation involved.

So, criterion 3 for well-posed questions:  Does 'diabetes' change in some orderly way with changes in the test variables?  Often we can't really tell or we use some assumed patterning test to make statistical judgments (e,g., regression coefficients).  Again this requires many aspects of definitions and assumptions about the nature of replicability.  In particular, given what we know about genes and environmental factors, the mix of factors, the sampled contexts, is unique in each study or sample.  This means that we cannot actually replicate findings, and replication is one of the classical ways to check our answers to well-posed questions (or even to ill-posed questions).

Next?

In a law-like universe, we should be able to replicate exactly, at least in principle.  In the living world, we can't.  In the living world, replication is not part of evolution's deal--variation is.  In that sense, even non-replication doesn't invalidate a finding from a previous study.  So we're stuck to a great extent with approximate statistical answers to questions as we happen to pose them today, that we want to extend to the future but know that this depends on changes in the factors which we cannot in principle predict.

Statistical sampling methods and probabilistic thinking are the ways we address the ill-posed nature of many of our evolutionary and biomedical questions. Sometimes this works well enough to answer poorly-posed questions satisfactorily enough for our particular purposes.  With strong causal factors that are prevalent in the population of inference, things work and Enlightenment scientific criteria take us where we want to go.  In that case, we don't even need well-posed questions, or we don't care how they fail to measure up, because we get what we want.

But what we are facing along a broad front of evolutionary and biomedical (and agricultural) interests are poorly posed questions that are not giving us what we want.  What we tried to suggest yesterday, and we've discussed in many posts, is that what is needed are well-posed questions for which our methods and concepts of causality are apt, or questions that force us to develop appropriate methods and concepts of causality.

It's time for someone to fill in the blank:

Q4: