Re: Re: Claim your $1M from the Great Randi
Psuedoscientist? Randi? Now I've heard it all.
It only takes one individual to return a statistically valid ABX positive result. ONE.
A million null results will still only suggest reasonable limits on the likliehood of a positive result being generated, not offer immutable proof that it can never occur. Therefore, it also suggest reasonable limits on the human acuity of the sense under test... hearing in this case.
Poplulation size should only be a concern if trying to find those limits. But for the individual who swears that effect X is audible, he/she is the only person who needs to participate. After all, I can tell an apple from an orange 100% of the time by looking at them, no matter how many consecutive trials I had to get correct, no matter how much money was on the line, no matter how many people were watching, etc.
I think perhaps this is a bit tangent to the tread topic, however. Hats off to Randi once again for not being afraid to question the ludicrous.
Kuei Yang Wang said:
Psuedoscientist? Randi? Now I've heard it all.
It only takes one individual to return a statistically valid ABX positive result. ONE.
A million null results will still only suggest reasonable limits on the likliehood of a positive result being generated, not offer immutable proof that it can never occur. Therefore, it also suggest reasonable limits on the human acuity of the sense under test... hearing in this case.
Poplulation size should only be a concern if trying to find those limits. But for the individual who swears that effect X is audible, he/she is the only person who needs to participate. After all, I can tell an apple from an orange 100% of the time by looking at them, no matter how many consecutive trials I had to get correct, no matter how much money was on the line, no matter how many people were watching, etc.
I think perhaps this is a bit tangent to the tread topic, however. Hats off to Randi once again for not being afraid to question the ludicrous.
I read that Randi's business card describes him as "Professional Charaltan". What he means by this is that he and others in his line of work take pride in what they do as **entertainers**. He says in an interview that they consider those who use a magicians techniques to fleece the gulable are prostituting their artform.
Just took a look. It seems Shakti stones will also add 2.7 horsepower to you car's engine output. I'm a pretty sedate driver so that doesn't really ring my chime. However, I used to follow F1 closeley, so I decided to check out www.formula1.com to see if there was any mention, since that is one place where +/- 2.7 HP can make a critical difference. I followed a few links but found no mention. Anyway, when Ferrari, McLaren, Honda, Jaguar etc start putting Shakti stones "under the hood" I'lltart being more impressed.
The analogy is limited. In wine tasting there is no notion of an ideal such as "straight wire with gain". Rather there is a celebration of variety and novelty. There may be a few audiophiles that would like the idea that a dozen different amplifies each sound significantly different and they could try out each one. I don't think folks into wine tend to have the idea of seeking the one perfect ideal wine, whereas the equivalent is fairly common in audio.
By the way, the most interesting and informative wine tasing sesions I've had were double blind tests. Rather than inhibiting perceptions it seems to draw them out and make the subtlties easier to percieve. As far as I know the only time DBT in the wine world was questioned was when California wines beat out all the French offering at a session ijn Paris a few years back. Some of the French press and wineries wanted the competion repeated with the labels showing so the results would be fair. The laughter in Napa & Somona counties could be heard for miles!
sam9 said:
I think similar analogy can be applied to audio, as there is no ultimate reference outside the designer's implementation. There is no such thing as straight wire with a gain, as some claim that even different wires sound differently. There is no ultimate reference, as whenever we reproduce something, we run it through the equipment. So we are always hearing the quality of the equipment that is reproducing it.
"Even at the recording session or at a concert, what you perceive differs according to where you are. When somebody says original acoustic sound, he is talking about his perception or image of the sound. In this sense, what decides about the sound of audio component is the designer's taste and sense which is the result of his musical experience. Same thing can be said on the reproducing system. It represents the musical experience of the audiophile who set it up. Reproducing the software through audio components is not just a mere passive act but is actually a very creative one."
Wine is not a good analogy. If you line up three well-made Santa Barbara Pinot Noirs and three Russian River Pinot Noirs, good tasters will have no trouble separating them under blind conditions.
If you make two wines side by side, one with indigenous yeast, the other with cultured yeast, they can readily be distinguished under blind conditions.
If you make two wines side by side, one in French oak, one in Hungarian oak, they can be readily distinguished under blind conditions.
Wine tasters somehow manage to distinguish (if not always correctly identify) wines under blind conditions, with no excuses about test pressure, that blind tasting is not like drinking wine with dinner, left-brain/right-brain, delays between sips and sniffs, whatever. Blind tasting is the gold standard for deciding which winemaking procedures make a difference and which are preferred.
I'm terribly amused that Larry Archibald tastes wines blind.
If you make two wines side by side, one with indigenous yeast, the other with cultured yeast, they can readily be distinguished under blind conditions.
If you make two wines side by side, one in French oak, one in Hungarian oak, they can be readily distinguished under blind conditions.
Wine tasters somehow manage to distinguish (if not always correctly identify) wines under blind conditions, with no excuses about test pressure, that blind tasting is not like drinking wine with dinner, left-brain/right-brain, delays between sips and sniffs, whatever. Blind tasting is the gold standard for deciding which winemaking procedures make a difference and which are preferred.
I'm terribly amused that Larry Archibald tastes wines blind.
I didn't say it would be a good anology, I only mentioned it could be more appropriate than comparing oranges and apples. After all, we are discusing here 3 different senses and in two examples, we are dealing with material objects. Perhaps sound waves are more tricky?
Still, I would not be able to distinguish three Santa Barbara Pinot Noirs and three Russian River Pinot Noirs, while I would have no problem finding apple from orange.
Still, I would not be able to distinguish three Santa Barbara Pinot Noirs and three Russian River Pinot Noirs, while I would have no problem finding apple from orange.
SY, almost anyone can do BLIND TESTING, it is ABX testing that is virtually impossible to pass, because it FORCES a decision of a certain kind.
For example, about 25 years ago while in Japan for HK, I was asked to listen to 3 separate audio circuits, not made by me, in a blind test. I could distinguish them and point out which was the best. It amazed the Japanese, but it did not surprise me. I do it all the time.
For example, about 25 years ago while in Japan for HK, I was asked to listen to 3 separate audio circuits, not made by me, in a blind test. I could distinguish them and point out which was the best. It amazed the Japanese, but it did not surprise me. I do it all the time.
ABX
In wine tasting, we call this a triangle forced choice test. They work beautifully, they are quite standard in the wine industry, and in fact, that's the type of testing I use with my own products. Our customers demand this sort of data.
I find it interesting that people doing ABX testing in audio distinguish some surprisingly subtle differences. It's an appropriate modality for testing certain kinds of variables, but there are other blind testing methods which are more suitable for other problems.
In wine tasting, we call this a triangle forced choice test. They work beautifully, they are quite standard in the wine industry, and in fact, that's the type of testing I use with my own products. Our customers demand this sort of data.
I find it interesting that people doing ABX testing in audio distinguish some surprisingly subtle differences. It's an appropriate modality for testing certain kinds of variables, but there are other blind testing methods which are more suitable for other problems.
SY, you are saying to me that you have 2 or 3 selections and that you taste each one, and then you are given an unmarked selection and you decide which of the original selections that it is. You do this 20 or more times in a row, and you have to be right 95% of the time to have any significance to your decisions. Wow! I once observed a bar bet like this, and the person couldn't tell the difference between cola, 7up or ginger ale after a few tries. Of course, to be fair, we must make the wines a similar as possible. I might suggest adding sugar or other components in order to 'even up' the wines, so that we are not considering taster preference. ;-) Can you see a parallel to this in audio testing?
Here's a test that we did today:
We compared wines bottled at the same time but with two corks having different levels of an additive that we're testing. Five panelists (prequalified to be able to detect wine flaws like TCA, volatility, brettanomyces, and the like). Five trials per panelist. Each trial is a set of three glasses, two of which have the same wine, the third of which is "different." The panelist tries to detect the odd wine out in each trial trio. All panelists scored 4/5 or 5/5 correct; by using a t-test, we determined that they can distinguish between wines bottled with the two corks to better than 95% confidence level. I'm not going to use that additive!
We compared wines bottled at the same time but with two corks having different levels of an additive that we're testing. Five panelists (prequalified to be able to detect wine flaws like TCA, volatility, brettanomyces, and the like). Five trials per panelist. Each trial is a set of three glasses, two of which have the same wine, the third of which is "different." The panelist tries to detect the odd wine out in each trial trio. All panelists scored 4/5 or 5/5 correct; by using a t-test, we determined that they can distinguish between wines bottled with the two corks to better than 95% confidence level. I'm not going to use that additive!
A possible advantage the wine industry has is a standardized nomiclature relating a specific adjective to a specific taste which is related to a specific chemical component. My brother-in-law how owned a winery for a while, educated me (atempted to would be more accurate) on a few examples. I remember best "vegatative" and how to relate it to the tasye of green beel peppers and why it results from greenery included with the crush. I was also shown how there is training available using sample scents in vials so that everyone means the same thing when they use a particular adjective.
Apart from reading some some definitions in TAS once, which added to the mystery rather than reduce it, I'm unaware that audio has anything similar. It would seem possible to create a standard set of .wav files that define "warm", "laid back", "analytical", etc etc. -- but maybe that would just let too many of us nulkultorni in to the drawing room.
Apart from reading some some definitions in TAS once, which added to the mystery rather than reduce it, I'm unaware that audio has anything similar. It would seem possible to create a standard set of .wav files that define "warm", "laid back", "analytical", etc etc. -- but maybe that would just let too many of us nulkultorni in to the drawing room.
We like to believe that we've got a standardized vocabulary, but we don't, really. For example, one famous critic uses the term "Asian spice." What does that mean? Five spice powder? Cumin? Cinnamon? Saffron? And everything's relative- one person's rich and velvety is another person's cloying and hot.
SY, you are saying to me that you have 2 or 3 selections and that you taste each one, and then you are given an unmarked selection and you decide which of the original selections that it is. You do this 20 or more times in a row, and you have to be right 95% of the time to have any significance to your decisions. Wow! I once observed a bar bet like this, and the person couldn't tell the difference between cola, 7up or ginger ale after a few tries. Of course, to be fair, we must make the wines a similar as possible. I might suggest adding sugar or other components in order to 'even up' the wines, so that we are not considering taster preference. ;-) Can you see a parallel to this in audio testing?
John, no, I don't follow the analogy. We're not trying to even things up, we're trying to see if doing thing "X" makes any discernable difference to the wine. First, one has to determine if there IS any difference before deciding which is preferred. Preference tests come second. I should make the side comment that "being right 95% of the time" is not the same thing as "being able to distinguish A from B at a 95% confidence level." The latter is what we do in sensory testing, not the former.
Bars tend to be lousy environments for sensory testing; that's one reason that these bar bets are so funny. When I was doing amplifier tests, I didn't set my speakers up in a boiler room. And to forestall the next question, we do NOT swallow the wine!
Fifteen glasses of wine per panelist would result in a lot of puke being generated, and the aroma of accumulated vomit might tend to obscure the aroma nuances we're looking for, especially if you're the fourth or fifth panelist.
One more fun test we did: my drinking/cooking buddy and I once visited the famous wine critic Robert Parker. Parker set up a group of 27 wines in a blind manner. We were asked to determine which wines were French and which were American. I've got a good technical palate, but my palate memory is quite ordinary. My buddy has a superb palate memory. In any case, my buddy got 25 right out of 27, and directly identified 5 of the wines (he missed zero in his direct identification). For some reason, the left brain/right brain thing and test pressure didn't seem to bother him. (for the record, I got 19/27, not particularly significant)
And, finally, successful blind testing/identification is part of the examination process for professional certifications like WSET, Master Sommellier, and Master of Wine.
Bars tend to be lousy environments for sensory testing; that's one reason that these bar bets are so funny. When I was doing amplifier tests, I didn't set my speakers up in a boiler room. And to forestall the next question, we do NOT swallow the wine!
Fifteen glasses of wine per panelist would result in a lot of puke being generated, and the aroma of accumulated vomit might tend to obscure the aroma nuances we're looking for, especially if you're the fourth or fifth panelist.One more fun test we did: my drinking/cooking buddy and I once visited the famous wine critic Robert Parker. Parker set up a group of 27 wines in a blind manner. We were asked to determine which wines were French and which were American. I've got a good technical palate, but my palate memory is quite ordinary. My buddy has a superb palate memory. In any case, my buddy got 25 right out of 27, and directly identified 5 of the wines (he missed zero in his direct identification). For some reason, the left brain/right brain thing and test pressure didn't seem to bother him. (for the record, I got 19/27, not particularly significant)
And, finally, successful blind testing/identification is part of the examination process for professional certifications like WSET, Master Sommellier, and Master of Wine.
My wife can do that. Two years ago we were with friends in France, and they served us several different wines from the area to taste and talk about it. I'm not a particular expert, but after tasting the third or fourth glas (no bottles visible) my wife said: That's not a French wine! Really, my friend said, amused. No, my wife said, it's Italian, Tuscany, I think from Montalcino, either a 97 or 98 from Vasco Sassetti. Dead silence. Friend off to the kitchen, came back, you are absolutely right, 98 Rosso di Montacino from Vasco Sassetti.... Joke from his wife....
Now, I find this absolutely fascinating (there more about her that's fascinating, but that's another story). There was no indication or expectation that an Italian wine was to be served. How do you guys do it???
Jan Didden
Now, I find this absolutely fascinating (there more about her that's fascinating, but that's another story). There was no indication or expectation that an Italian wine was to be served. How do you guys do it???
Jan Didden
- Status
- This old topic is closed. If you want to reopen this topic, contact a moderator using the "Report Post" button.