Playing on the real strings, or just your heart strings?

A recent study of whether a combination of professional and amateur violinists preferred old or new violins has gotten a lot of press, here first in the NYT, and then again yesterday.  The study was double-blinded, and 'scientifically run', and claims to be the first to properly test the hype about Stradivariuses.  Touted as 'gotcha' results, showing that people only think Strads are great because they are expensive, the study, published in the Proceedings of the National Academy of Sciences, reported that experts can't tell whether they are playing a priceless Stradivarius or a violin by a modern maker.

We found that (i) the most-preferred violin was new; (ii) the least-preferred was by Stradivari; (iii) there was scant correlation between an instrument's age and monetary value and its perceived quality; and (iv) most players seemed unable to tell whether their most-preferred instrument was new or old. These results present a striking challenge to conventional wisdom.

Here's the NPR treatment, complete with a sound test.

So, the most recent NYT story apparently accepts the comparative study, but it turns out that, according to our daughter Amie who is a professional violinist, by and large the world of professional musician does not.  In fact, one of the participants in the study says here that they were not asked to identify the old violins, they were asked to choose their preference.   She describes her experience:

Upon arriving, I was fitted with modified welders' goggles, and I entered a darkened room. I was then presented with 10 pairs of violins. For each pair, I had a minute to play whatever I wanted on the first violin, then a minute to play whatever I wanted on the second, without switching back and forth. After playing each for one minute, I was asked to choose which of the two I preferred. Then on to the next pair -- 10 times altogether. I thought I was testing 20 violins!

As it turns out, I was testing 6 violins, just paired up differently each time. One always was an old violin, the other was a modern, and they used different combinations against each other.

She points out that the old instruments weren't optimized (sound post adjusted, new strings, etc), while the new instruments were.  

The test was not over after the 20 violins, which were really six violins. After that part of it, the six violins were laid out on the bed, and I was given 20 minutes to play with them as I liked. My task was to choose which of these violins I would take home if I could, and also to decide which of the six was "best" and "worst" in each of four categories: range of tone colors, projection, playability and response.

She preferred one of the moderns, although she did say in her comments for the investigators that she thought it had potential, not that it was already great (good violins improve with time as they are played, although modern violins can either improve, or they can lose their sound, so the reputation of a modern maker is only enhanced if their violins stand the test of time).  That is, this violinist suspected it was a modern violin.  And, she points out that they were not asked whether they could tell the difference, just which they preferred -- despite the inferences all over the web that even professional violinists can't tell old from new, and only like Strads because they are expensive and have a mythic reputation.

Another one of the participants, "an accomplished amateur violinist and violin maker", believes the study was well-run and the results perfectly credible, as he says here.

So, what's going on?  Was this a valid study or wasn't it?  Does it bust the Stradivarius myth?  Some violinists have pointed out that testing instruments in a small hotel room, where their sound can't project, seriously hinders a player's ability to judge.  Others that the researchers'  conclusions weren't properly inferrable from what the players were asked to judge, and of course if that is so, no matter how 'scientifically valid' the study was, the conclusions are largely the researchers' interpretations rather than objective findings.   And, some of the participants were amateurs, and even if they were excellent musicians, their experience with playing in concert halls, where an instrument really makes a difference, is necessarily limited. Of course, musical quality is largely subjective and perhaps knowing the instrument is a Strad can make some people enjoy it more.  And, the impartiality of the researchers, some of them modern instrument makers, might be an issue.

The quality of the sound of a string instrument depends on many variables, not least of which is the listener's preferences.  But also, the bow with which it's played, the adjustment of the sound post, the quality of the strings, and even what kind of strings, what the player chooses to play, and whether it's the same piece on each instrument.  And so forth.  And of course, Strads have been fixed, tuned, adjusted, revarnished, and so on so that today's Strad is not the same as what Stradivarious himself built.

This is not like judging red wine, which can just be poured into a glass, allowed to breathe for a set number of minutes, and then tasted.  Preference for wine is still subjective, but at least the factors that can influence its taste are part of the essence of the wine, unlike the music that comes out of a violin. If fact, it's been shown repeatedly that blindfolded, even experts can't reliably tell if a wine is great or just good, or often, even if it's red or white!

Scientific methods can be applied to a multitude of questions, but the question has to be clear, the variables controlled, and the subjectivity of the answers at a minimum.

PNAS hitting the bell of deep science yet again!  Still, if not the most profound kind of science, mythbusting is important, even if it's not in the interest of vintners or, in the case of violins, of auction houses.  But, should this study be considered the final word?