No. What he is saying is that execution of a statistically unbiased test is not as easy as some think.
The very act of being in a test, and that the person under test may well be concentrating on hearing a difference (not the state of mind one is in when "just listening") is a bias in itself.
Do also keep in mind that in particular an ABX test is unable to tell you that 2 DUT are not different. If the results are that a difference is detected it is a strong result. ie in an ABX test it is likely that the test will in many cases have a null result when there indeed are differences. The results of a null test only apply to the test itself.
dave
I cannot agree more and too often forgotten.
yes ab test helps, but its also over time that I can really figure out wheter I like the new components or not.
once the brain stop trying to find a difference in the SQ...
What he said ("a listening test is an experiment like any other and therefore prone to exactly the same error possibilities.") fits the term "straw man argument". He is saying, because "any other experiment" is prone to error (blanket statement), it is prone to "exactly the same error possibilities" without citing a single real life example of audio DBT.
In such heightened concentration mode, it could may well be more helpful for detecting difference, no? Wouldn't that be an enhancement and not a hindrance that Jakob2 is implying?
My question was, has there been a real life case of audio DBT which the audible difference was missed even though it was really there? If so, how did they know the audible difference was really there?
BTW, have you seen published results of audio DBT? They show percentage of correct / incorrect answers gathered from xxx number of tests, larger the xxx the better.
I see no issue with people choosing components based on their personal preference. I do question those who post claims like this "I hear much bigger difference in DAC then I ever did in amps."
doubt as much as you want, but this is my experience.
going from a buffalo 2 dac to a sd player sd-1 sounds like you added a subwoofer so much the details and resolution in the bass and the sound went from digital sound to analog naturalness
but its probably because of a 0.2 db difference in my blind test 😉🙄
Last edited:
Assuming that's what you had. If I were to guess, your test was probably not actually blind and your levels were much different than that.
Once you get the hang of how to set up a good test, you'll find the experience liberating. It's enlightening to actually be able to use your ears only for sound evaluation, and you find out what's real and what's your brain lying to you (as brains do to all humans).
No, because you are using different parts of the brain and are in a different state of mind … a really good test might involve subjects unaware that they were being tested and direct readout of their brain activity. The brain works hard to fill in the missing bits of an audio reproduction presentation and a read-out that shows the brain working less hard to do this would be an indicator of a better DUT.
Yes. And quite a spectacular case. One of the standards committee was about to approve a lossy compression standard where 100s participated in a double blind study before approving it and about to approve it when a single outsider came in and pointed out that the algorithm created a noticable whistle in the output. Then everyone heard it where they completely missed it before.
Perhaps someone can help with the name of the person (well known) and the incident.
Yes. And althou i have forgotten most of the statistics i took at uni (i have an honours degree in statistical mathematics) i well know that there are lots of pitfalls to avoid in such testing.
dave
I will get a good spl meter but yes all blind test. you still havent showedthe data that 0.2db can be heard.
its like when you change of cart or phonostage, do you really need to do a blind listening test to know what you hearing is a major update or not?
how can 0.5db of difference can effect soundstage, bass extension, adding details, more relaxed sound.
SPL meter won't get you what you need- I thought I was clear about that? These are electrical devices, you have to level match electrically.
There's a lot of literature on level matching- a few minutes of searching will turn it up for you. More importantly, you have to truly do things double blind. If after that you can still reliably hear a difference, then the difference is real. if you don't, then you learned something interesting.
At the risk of excessive pimping of my own writing, please read my Linear Audio article, Testing One... Two... Three, available for free download at their website.
Perfect for Clever Hans errors. Double blind is essential.
Of course it is. That's the question that the experiment must answer- do I (youknowyou) actually hear the differences between DAC A and DAC B? Trying to redefine the question that the experiment is answering is not useful.
Far, far more onerous is the generalization of uncontrolled anecdote. A well-controlled n=1 at least tells us *something*.
I can't seem to find anyone on this thread doing that.
I spent a great deal of time in my article explaining the need to precisely define the question to be answered before designing the experiment; unfortunately, there are folks who want to base any criticisms of a protocol on their idea of the question THEY want answered. 😀
Much sophistication in the arguments here (esp. from the psychologists). But I'd like to take the "worm's eye view". Other than those posting here who are terminally arrogant about their absolute ability to judge gear by ear, how is a person to proceed in order to be sure (assuming they are not too insecure to handle the truth)?
I want to discern quality as judiciously as I can. Doesn't everybody? So, here's my "Ladder of Scrutiny" concept. If after an audition, you think Gear A is better than B, you need to go up the Ladder of Scrutiny by adding additional niceties to your testing and see if you can still tell the difference. I take for granted that without an A and a B at hand for comparison, you are just begging to fool yourself.
Can I hear a difference? If so, can I now confirm the loudness AND the colouration are identical? The long history of ABX testing is based on an assumption you've matched both and that colouration isn't present. Quite simple to check both by playing pink noise. Anybody think their expensive DAC should be louder or have any different colour on pink noise? If there's no match, you're stuck until you can bring about a match. (Tube-sound lovers can love solid-state sound indiscriminably if you EQ the SS device to the tube colour.)
If you still hear differences, have you set up your rig to allow instantaneous switching? And instantaneous replay of the test music?
Next, can you get a buddy to do the switching in a way you can't guess at (or Sy's favourite artifact, the Clever Hans trick where the tester even inadvertently is communicating choices by the sound of switch clicks, timing, or whatever).
Next, ABX.
I'm sure others can add smart thoughts and corrections. But the goal is to help YOU satisfy your doubting mind about your gear.
Ben
I want to discern quality as judiciously as I can. Doesn't everybody? So, here's my "Ladder of Scrutiny" concept. If after an audition, you think Gear A is better than B, you need to go up the Ladder of Scrutiny by adding additional niceties to your testing and see if you can still tell the difference. I take for granted that without an A and a B at hand for comparison, you are just begging to fool yourself.
Can I hear a difference? If so, can I now confirm the loudness AND the colouration are identical? The long history of ABX testing is based on an assumption you've matched both and that colouration isn't present. Quite simple to check both by playing pink noise. Anybody think their expensive DAC should be louder or have any different colour on pink noise? If there's no match, you're stuck until you can bring about a match. (Tube-sound lovers can love solid-state sound indiscriminably if you EQ the SS device to the tube colour.)
If you still hear differences, have you set up your rig to allow instantaneous switching? And instantaneous replay of the test music?
Next, can you get a buddy to do the switching in a way you can't guess at (or Sy's favourite artifact, the Clever Hans trick where the tester even inadvertently is communicating choices by the sound of switch clicks, timing, or whatever).
Next, ABX.
I'm sure others can add smart thoughts and corrections. But the goal is to help YOU satisfy your doubting mind about your gear.
Ben
Last edited:
The best red-book dac I have heard is the Chord Chordette in all its incarnations.
I have only 5% hi-rez material and this dac sounds better than any Saber, Wolfson, or Burr Brown dac, diy or otherwise I have made or encountered, even in hi-rez mode. Seriously this competes with cost no object designs.
Chord really went the right direction with using a programmable chip.
I have only 5% hi-rez material and this dac sounds better than any Saber, Wolfson, or Burr Brown dac, diy or otherwise I have made or encountered, even in hi-rez mode. Seriously this competes with cost no object designs.
Chord really went the right direction with using a programmable chip.