Is that really ensured?
Having heard literally numerous times before that under test condition noone will be able to hear any difference will have no impact on participants?
Isn´t it interesting? What about "ears only" ? No problem to do "ears only" before and now, just in a second, it is "ears and hawthorne effect only" ?
And now we have some additional "should" 😉
I´ve wrote that habituation error is linked to the internal criterion problem and - as said quite often before -for good reason some laboratories don´t tell their participants which effect will be tested.
Wait a minute, we started with "ears only" and now we have "ears and Hawthorne and (maybe) Rosenthal and (maybe) some other biases only" 😉
And again there is suddenly a "should" incorporated.
But is it correct? Usually you don´t want just to gather data, you want correct data instead.
Whatever you think about "strong influence" and that removing of a certain biase will improve test results, the only important thing is, that your results are _correct_ (most of the time).
You have to check if your assumptions are true; hoping, assuming and "shoulds" don´t help.
Thats a pretty good example of a mechanistic view. You are using a detector that does not work in the same way as usual test gear does.
Your listeners don´t work that way and if you analyze the results of published controlled listening test you inevitably notice that these are incorrect, because obviously additional bias effects are still at work.
Did you really start to believe in the straw man that you´ve created?
I think, the authors appear to say that insistence on DBT _alone_ as the only valid scientific approach arises from a belief in Mechanism, which is a quite different position.
Again, i think the authors only attack the claim that the "DB or nothing" is true.
As they later explain, the scientific tool set is also a matter of social intervention and accordance, so we have to admit that "DB wherever possible" is part of the modern tool set, but we have also to admit that a huge percentage of study results is nevertheless simply incorrect (most probably) and that other tests belong to the scientific tool set as we now know that the "DB only approach" has some weak points too.
You should care, of course to care isn´t the same as to blindely 🙂 believe. If you don´t care it simply means that you have to follow your own philosophic arguments and those might be much more erroneous than those from the philosophers.
It´s philosophy of science, if you like it or not and quite a lot of famous physicists had their share in developing it. Btw imho you´ll have quite a hard time to match Feynman´s criterions for good scientific practice to the usual "DB or nothing" or "ears only" claims and the usual DOE in the audio field as well.
No, he didn´t but you erected an Aunt Sally (thanks for that, i didn´t know about the british version before)
The authors just attacked some quite common misbeliefs about DBs in general and in the audio field.
As said before, you should not take your own straw man to serious. 😉
Sometimes people who expect to have no difficulty hearing a 'night and day' difference take part in a blind test and find that they can hear no difference. Sometimes someone who expects to hear no difference finds that he can hear a small difference. This means that expectation bias does not affect a blind test in the gross way it obviously affects a sighted test.Jakob2 said:
Don't be silly. "Ears only" means ears are the input, not the only functional part of the body. For the avoidance of doubt, when I and others say 'ears only' we assume that the liver, kidneys, spleen, stomach, bladder etc. (oh, and the brain) are all working in addition to the ears.
"Suddenly"? I guess nitpicking at words is a sign you have no substantive argument?
If by mechanistic you mean that an amplifier uses physical effects to produce a mental perception of sound then yes I am mechanistic. The alternative is to believe that non-physical processes are taking place in the amplifier or in the interface between the amplifier and the listener. It appears that some audiophiles must believe this, because their claims can only make sense under this assumption. Curiously, they almost certainly spend the rest of their lives carefully avoiding anything non-physical (e.g. religion, crystals) and regard themselves as modern rational people. Actually, I believe in non-physical things (off limits for this forum) but I don't believe they have any relevance to testing audio equipment.
We need to be clear on this point, because those who want to follow JC's 'professors' need to understand what they are signing up to: non-physical processes taking place in the interface between an audio system and a listener. The claim is not just that current science is wrong, but that physical science is necessarily inadequate. Note that it is the interface which is being considered; the mental processes inside the listener are a different issue which are not the point.
There are two ways to answer this:
1. The authors may have erected their own straw man - does anyone insist on DBT alone i.e they believe that nothing can be gleaned whatsoever from any sighted tests? Gross differences may be audible in sighted tests.
2. Even if this were the case and such people exist I don't follow the logic that says that "alone" forces a belief in Mechanism which would be absent without the 'alone'.
So the authors are wrong either way. A belief in physical processes alone does not require DBT alone. A belief in DBT alone does not necessarily arise only from a belief in Mechanism.
They don't "explain" that, but they may assert it. As a scientist I disagree. The "scientific tool set" is not a social construct but a reliable means for determining scientific truth. I firmly believe that audio systems fall within the realm of science.
No, they said (perhaps not explicitly) that rational science is not adequate for the testing of audio equipment. Or they said nothing at all. I don't intend wading through a thicket of false assumptions to determine whether what they said was wrong or merely vacuous.
As I said, it is nice to know how weak is the case for the opposition!
A good series of tests will include, tests with audible differences and tests with identical signals.
Jakob2,
Good question!
Your post got me reading more about A/B and ABX testing. Usually my test was a single sitting listening test where I compared A to B randomly 10 times. There in the X factor was in play. From reading a few articles some tests can be up to 25 sets of randomized 10 ABX trials. Of course the way I read it I could do a 10 ABX comparisons today and run another randomized 10 ABX trial test, of the same equipment, the next day or a few days later or a week or a month later.
Doing 25 sets of 10 ABX trials in a one sitting day might be good for statistics but it would suck for the listener. Sounds like the test could be biased in favor of statistics at the expense of breaking down the confidence of the listener by turning his brain to mush. If subjected to the above test I would think for the test to be credible a break of at least 4 to 5 minutes after each set of 10 ABX comparisons and at least 15 to 20 minutes should be given between some predetermined number of trials of listening tests of 10 ABX comparisons.
Example: 10 comparisons of ABX = 20 listening samples of the source material plus comparisons asked for by the listener to refresh his memory of A or B for X identification if so desired.
Just picking a number out of the air 4 trials of 10 sets could easily subject the listener to listen to preselected music segments of source material 85 to 90 times.
I would think a break of at least 15 to 20 minutes would be in order before the next battery of trials would commence. After all the object of the listening test is to find out if the listener can hear differences 9 out of 10 times. I would hope the test is not meant to just wear down the listener making him tired and thus become confused in what he is hearing. It is not a police interrogation.
For the test to be credible I would think the listener should feel confrontable, not pressured by a timeline.
See page 168 of the PDF article:
https://linearaudio.net/sites/linearaudio.net/files/LA Vol 2 Yaniger(1).pdf
" There is no inherit reason why the listening and judgement need to be rapid- with proper controls in place, they can extend over days or weeks, if so desired".
Jakob2, I need to thank you for your post. It is obvious I did not ask SY for a detailed outline, protocol, of the testing procedure. As the listener, the listener should know well in advance, in detail, the protocol procedure.
Also from the start I stated nothing in my audio system would be changed. The switching of A and B will be done through the digital switch in the Cambridge DacMagic DAC. No outboard switcher was ever mentioned. I will also need to conform with SY we are in agreement on that.
I feel I have the home advantage in the ABX testing. The listening test will be done in my home, my listening room, my audio equipment. Time will tell......
Last edited:
I'm unaware of any test where the listener has 25 sets of 10 trials. You may be misreading or looking at tests that are trying to answer a different question than this one.
If one uses an ABX format, 12 trials is generally sufficient to get a reasonable confidence level for the question at hand : "Can Jim reliably hear the difference between Transport A and Transport B?"
If one uses an ABX format, 12 trials is generally sufficient to get a reasonable confidence level for the question at hand : "Can Jim reliably hear the difference between Transport A and Transport B?"
I must have misunderstood.
https://en.wikipedia.org/wiki/ABX_test
SY,
Thanks for the response. So are You saying there will be only 12 trials of me listening to a randomized A and B. 12 comparisons total. 24 total combined plays of both A and B transports. Plus if I ask to hear a transport to refresh my memory to identify X.
I was going to send you an email to get a detailed outline/protocol of the test procedure.
Being you have responded here on DIY, are we in agreement the digital switch in the DacMagic DAC will be used to switch from one transport to the other transport.
Jim
Last edited:
Yes, assuming it works without giving any cues (e.g., a distinctive click when changed).
And yes, 12 trials, the repetitions within the trial (comparing A, B, and X) are up to you, completely in your control.
And yes, 12 trials, the repetitions within the trial (comparing A, B, and X) are up to you, completely in your control.
That misses the point in Rosenthal results and in our discussion; you´ve asserted that the Rosenthal effect "should" be removed in un-sighted tests.(The "should" must already provoke a check if it´s really true).
Rosenthal effect means that any strong proposition about the awaited performace of participants might have an impact.
I´ll do my very best; if you could try to do the same it certainly would help...
Nice try, but being funny does not justify to use an incorrect term. Especially as there exists already a well known term i.e. "blind" and "double blind" .
Even if you fear that "blind" could be misleading as people might think that they have to wear a blindfold, you could use for example "no peeking" instead.
Usage of "ears only" implies the impossible (just ears) while in reality it is just removing the "sighted bias".
I beg your pardon, but the claim was that "ears only" is impossible and the arguments are provided by the existence of the long list of bias effects that are not affected by "no peeking" .
By mechanistic i mean the assertion that after removing the "sighted bias" (which is assumed to be the most important bias) test results have to be more correct per se.
As said before, analysises of published controlled listening tests has shown that this "mechanistic" approach failed, means the results aren´t more correct per se.
Maybe the misunderstanding is due to different meanings of the term "Mechanism" .
Otherwise could you cite some paragraphs where the claim of "non-physical" interactions is established?
Btw, if physical sciences follows the Mechanism approach (wrt human listeners) it is necessarily inadequate.
You´ll find this assertion even now in nearly every discussion on these "strange effects" , but of course, as stated before, it is even more depressing as it depends on the personal belief- if the results is "no difference" sighted listening seems to be fine...
You might be right in your second remark although i´m not sure that the authors really meant "only" , as they wrote "many scientits believe ...because their traning lead to something like..." (not an exact quote)
Nevertheless is the scientific tool set a social construct, as it otherwise couldn´t evolve. "Determining scientific truth" is sort of selfreferrencing.
Btw any literature (that i know) about history of science provides a lot descriptions about this social construct process.
As said before, don´t take your own straw man too serious.
Enhancing it by stating "or they said nothing at all" does not help, it´s still a straw man.
As you just read the first page you can´t know.... 😉
These are socalled positive and negative controls and proposed from yours truy nearly right from the beginning. Unfortunately a lot of people merely interested in listening tests with negative results were negating the importance of these controls.
The number of really sound experiments in this fied is surprisingly small.
- Status
- Not open for further replies.