That certainly was not the case with the StereoReview 'test'.
Many of the subjects were self-described 'experts' who were sure that they could differentiate between the amplifiers that they had heard in the open listening sessions, before the blind test.
In other words, the subjects were already biased in favour of audible differences, before testing. Still, they failed to prove that 'ability' when tested.
What are 'artificial conditions'?
This is the 'listen to it for a month in your home and you will be able to tell it is less-fatiguing' idea?
Or 'burn in your cable for 2 months before deciding if it is good' ?
It is well, well known that our brains can 'adjust' to new sensory environments rather well.
Put on your sunglasses and the world looks green for a few seconds, then normal.
Move to a new house by the railway, and the trains wake you up for a few weeks. After that, sound sleep.
So, how to define 'artificial conditions' in light of our neurology?
"Those people" meant the group that conduct/conducted ABX or other controlled tests without paying attention to the basic scientific requirements for sensory tests .
That a test situation do influence the participants is well known and therefore the recommendation about training and usage of positive controls exist, but are widely ignored.
The only way to overcome this problem is to test people without their knowledge of being in a test. We did that once with preamplifiers but it is quite difficult and you can´t normally repeat it.
The only other way is to train the participants and to use positive controls to see which sensitivity level they reach under the specific test conditions.
As I read it, you have mis-stated Thorsten_L's position.
Not so easy now, is it? ;-)
I think you´ve given the answer in your post; it normally takes time to adopt to
new situations.
What did you think would the training do?
A "artificial" conditions is every condition departs from the normal listening situation of the person who did claim the audible difference.
If we could test them in _exactly_ the _same_ situation as before there wouldn´t be any need for discussion of conditions.
P.S. No, if you rethink it carefully it is part of the same reasoning.
(your post #126)
Last edited:
You mean by 'can't normally repeat it' that your results cannot be replicated by other researchers, or even by yourself?
That would neatly side-step one of the essential steps in validating results, no? Assuming this is still in the world of science...
Perhaps you mean something different by 'repeat' ?
I think you´ve given the answer in your post; it normally takes time to adopt to
new situations.
What did you think would the training do?
Using the examples in my post, the 'training' would convince the sunglasses wearer that the glasses were not green, and convince the sleeper that the trains no longer ran during the night.
Using hifi examples, the subject would certainly come to the realization that the new amplifier which initially sounded terrible, now sounds much better after a few weeks.
And what, exactly does this interesting brain behaviour have to do with the qualities of green glasses or bad amplifiers?
Training='power of suggestion' ?
How does this differ from the salesman's script in the audio store? "I'm sure you will hear that this McIntosh sounds much more refined.....listen....."
New assertion (i.e. "demonstrably inadequate methods" ) still no argument to back it up.
Benjamini/Hochberg developed a new method to overcome some of the limitations of the mentioned Bonferroni-correction (they did not address the gaussian field approach) and Craig Bennett might have done excellent work, but could you cite a paper in which the method choosen by Oohashi et al. was shown to be "demonstrably inadequate"?
No, training is a valid point for answering certain classes of questions. Same with positive controls. For example, suppose the question to be answered is, "What is the smallest noticeable level change that any skilled listener can detect?" One wants to use listeners who have spent time practicing hearing level differences within the test conditions. And in the tests, there should be a few selections with level changes above established thresholds to see if they are detected. And it must be understood that these requirement are valid for some questions and not others.
The novelty of Jakob's position is his assertion that positive controls are valid when they do not represent the phenomena under test, e.g., level and EQ changes in a test of the audibility of silver versus copper wire with no significant level or EQ changes.
yes, it would be interesting to know what a positive control for Cu vs Ag would actually be???
here's a coollink about silver... http://www.britishpathe.com/record.php?id=8724
here's a coollink about silver... http://www.britishpathe.com/record.php?id=8724
This seems reasonable. As does a prescreen hearing test. Perhaps it's useful to include both skilled and "layman" listeners in some tests. Not so much as a control, but for data comparison purposes. Or maybe it would be better to determine the screening & training after the listening tests. I'm not 100% comfortable with being so selective in choosing test subjects because it automatically shifts test results.
Thanks, SY. It makes sense to me, a bit.
This is really just a variant on my desire that the 'golden ear' should display a recent audiology report, I think. Or even, the 'ordinary untrained listener'....
But, I guess a lot of people in the audio world would say that audiology tests for hearing loss are invalid because the test conditions are 'artificial' ? "I don't care what your testing shows, Doc! My wife hasn't complained yet! And she sees me many hours of the day."
Established thresholds: Presumably these come with their own data and statistics? (eg ' n% of listeners age 25-35 could detect a difference of y dB, etc etc). No single number could be meaningful, I think.
And how does one 'practice' hearing level differences? Is the listener 'told' which of the practice samples is louder ("C'mon, can't you hear that?") or just presented with a series of less-and-less dramatic differences?
You can see my bias here- I think that the biological/neurological differences between people are so great that the term 'any skilled listener' doesn't have much meaning without a great deal of context.
Sorry to intrude on the more technical aspects of this discussion...it brings back memories (I won't say 'happy memories') of arcane 'bunfights' I witnessed in grad school.
Vic, that's not far from the truth. Typically, the listener will be provided with "open" variants of the stimulus. Then once he feels that he has a handle on the differences, single trial blind with immediate feedback until he's comfortable making the choices. "Established thresholds" mean exactly that- thresholds for which there is air-tight data in the literature (i.e., refereed journals with multiple replications at other labs).
Yeah, hearing tests and so forth are nice, but even old tin ears like me can be quite good at picking up audible phenomena like dynamic compression, phase shift, data compression, EQ, whatever. Floyd Toole had some nice stuff in his book on qualifying and training listeners. Interestingly, he found that trained and untrained listeners came to similar results (assuming normal hearing), however trained listeners got there faster and could make more accurate assessments.
Check out Sean Olive's website for links to training materials and a lot of detailed info on this very interesting question.
Yeah, hearing tests and so forth are nice, but even old tin ears like me can be quite good at picking up audible phenomena like dynamic compression, phase shift, data compression, EQ, whatever. Floyd Toole had some nice stuff in his book on qualifying and training listeners. Interestingly, he found that trained and untrained listeners came to similar results (assuming normal hearing), however trained listeners got there faster and could make more accurate assessments.
Check out Sean Olive's website for links to training materials and a lot of detailed info on this very interesting question.
AUDIO EDUCATED GOLDEN EARS RULE! AUDIO EDUCATED PROFESSIONAL MUSICIANS are required to decide the absolute truth of audio equipment. ABx testing can help a designer find relative truth, but only professional musicians can help a designer find absolute truth.
My employer built an anechoic room to test computer equipment. Naturally, several audio enthusiasts were able to gain access to this room for after hours testing including ABx experiments. What I personally found was:
1) It was easy for me to hear differences between broad general classes of products - examples: monopole vs. dipole; best solid state vs. best tube; normal cables vs. funky cables, etc…
2) It was easy for me to determine what I liked best between broad general classes of products, especially at the technology extremes – examples: I favor dipoles over horns; AND a $10 AWG 12 speaker cable sounded the same as a $500 AWG 12 cable, BUT.. sounded better than a $500 AWG 32 silver wire cable; speakers with 94-96 db/watt SPL have some special magic over 88 db/watt commercial boxes; I can hear crossovers in the vocal range(80-1,500Hz); etc… This experience drives my DIY investments. PERSONAL TRUTH.
3) AUDIO EDUCATED PROFESSIONAL MUSICIANS are required to decide the ABSOLUTE TRUTH of audio equipment. YES, GOLDEN EARS RULE! ABx testing can help a designer find relative truth, but only “audio educated” professional musicians can help a designer find absolute truth. If your local Soprano does not like your audio product, then it needs a re-design! You may love it, but few customers will buy it.
This is what I have learned from ABx testing audio equipment.
NEVER EVER run your audio equipment over 85db/watt when children are present!
My employer built an anechoic room to test computer equipment. Naturally, several audio enthusiasts were able to gain access to this room for after hours testing including ABx experiments. What I personally found was:
1) It was easy for me to hear differences between broad general classes of products - examples: monopole vs. dipole; best solid state vs. best tube; normal cables vs. funky cables, etc…
2) It was easy for me to determine what I liked best between broad general classes of products, especially at the technology extremes – examples: I favor dipoles over horns; AND a $10 AWG 12 speaker cable sounded the same as a $500 AWG 12 cable, BUT.. sounded better than a $500 AWG 32 silver wire cable; speakers with 94-96 db/watt SPL have some special magic over 88 db/watt commercial boxes; I can hear crossovers in the vocal range(80-1,500Hz); etc… This experience drives my DIY investments. PERSONAL TRUTH.
3) AUDIO EDUCATED PROFESSIONAL MUSICIANS are required to decide the ABSOLUTE TRUTH of audio equipment. YES, GOLDEN EARS RULE! ABx testing can help a designer find relative truth, but only “audio educated” professional musicians can help a designer find absolute truth. If your local Soprano does not like your audio product, then it needs a re-design! You may love it, but few customers will buy it.
This is what I have learned from ABx testing audio equipment.
NEVER EVER run your audio equipment over 85db/watt when children are present!
How would she know, she spends most of her time listening to vocals thru her skull.
Seriously, when most musicians are listeneing to music there listenig to the MUSIC not the sound stage or bass extension. Listening to a band from the drummers (or whoever) perspective is usually not the right mix. I have found most pro musicians are not that discerning when it comes to Hi Fi. Recording engineers on the other hand usually are.
Last edited:
- Status
- This old topic is closed. If you want to reopen this topic, contact a moderator using the "Report Post" button.