DAC blind test: NO audible difference whatsoever

Status
This old topic is closed. If you want to reopen this topic, contact a moderator using the "Report Post" button.
You have a strange definition of psychoacoustics which I think we addressed before but what part of my statement do you disagree with "be preferred because it presents sound in a more realistic illusion, not because it's colored"

I would like to hear how the only way a more realistic illusion is created is "by shifting the frequency response from flat"
 
Markw4 said:
The problem is when four people are tested and fail to hear something, and then a claim is made the test has proven that no person in the entire world could have heard anything. DF96, you know that can't be right don't you? The claim doesn't logically follow from the experiment, completely aside from whether the experiment was a good one or not.
I have already made my position clear on the issue of extrapolation.

EDIT: Suppose for a moment the original experiment in the thread was a good one. What could be logically claimed from the result? It seems to me it would logically follow that a minimum of four people in the world could not hear something under the conditions of the test. That's it.
That might be the 'logical' conclusion. However, if we can assume that the four people were in some sense 'normal' then the test gives us some information about what others might experience in a similar test. As I keep saying, it is a data point.

If instead of just applying logic, we wanted to go into the world of biostatistics, then if the test were a good one we could produce some statistics. But, with only 4 test subjects, a sample size of 4 subjects out of a population of 7.6 billion people in the world the statistics would be meaningless. If we had a 100 or 1,000 test subject chosen randomly from the population (and they must be selected randomly, not volunteers), then we could maybe start talking about trends for many or most people in the population. Even then we couldn't say what would be impossible for every single human on earth based on the statistics.
Such tests are unlikely ever to happen for audio. This provides great scope for audio marketing, which sometimes seems to be based on the idea that if X has not been proven to be false then X must be true (even when any reasonable person might expect X to be false, and actually proving X false beyond reasonable doubt is exprimentally difficult).

If I stopped four cars on the street and asked what fuel they use I might find two petrol, one diesel and one hybrid. Four is a hopelessly small sample of the millions of cars in the UK, although it will be a better sample of the many thousands of cars in Birmingham and maybe quite a representative sample of the cars on that street in that minute. From my data I cannot conclude that 50% of cars use petrol, but I can tentatively conclude that perhaps a significant number of cars use petrol. Of course, someone wanting to promote hybrid cars (by telling people that everyone else uses a hybrid, so why don't you?) would scream that my statistics were hopelessly unreliable. In a strict mathematical sense he would be right. The interesting question to ask is: if I happened to find three hybrids and a petrol, would he still scream 'bad statistics' or would he congratulate me on an excellent sample?

mmerrill99 said:
That's the point - you are trying to consciously discern a difference & you can't - this is one of the pitfalls with ABX, it relies on actually spotting a specific difference
No it doesn't. ABX does not ask people to identify the difference, but merely say whether they believe X is A or B.

In ABX difference testing, don't try to spot a specific difference - just pick what you genuinely think sounds like A or B. Don't be biased by others telling you that there should be 'night and day' differences between DACs - do give it a real shot & ask yourself on each trial which do I think/feel it is even if you can't say why.

There's a lot of talk about blind testing eliminating biasing but a bias towards 'not hearing any difference' is one bias that blind testing (single or double) does not eliminate. If one believes that there is no difference to be heard between cables for instance & the participant knows that cables differences are being tested for, then the almost 100% outcome is that no difference will be heard - the motivation just isn't there
Curiously, blind testing also seems often to eliminate the 'hearing a difference' bias too. People who firmly believe that cables make a difference are often unable to demonstrate this when blind.
 
I also would like a clarification on this.

If it were guaranteed that the two DACs are indiscernible then of course no difference can be detected.
But given the context of this discussion and the tests done, it is just the question _if_ they are indiscernible.

The listener originally thought during sighted listening he could detect a difference, during the "blind" test he only got a result so that the nullhypothesis could not be rejected, hence he concluded that no audible difference (at lest to him) exists.

But how does he know?
It could have been that his impression from the sighted listening was actually correct while his result from the "blind" test does not reflect the reality.

The answer is, he can´t know and we as outside observers can´t know either, therefore the idea to try another test protocol. Of course now the problem exists that the listener is biased to strongly due to his knowledge so any new test approach must take that into consideration.
 
What can a DAC do to the sound that affects the illusion? By far the major contributors to the illusion are the speakers and the way they interact with the room and listener's brain

OK, I see your understanding of psychoacoustics is limited to FR changes & therefore it's all about the speakers & room. Glad we teased this out of you as I suspected from your previous posts revealed your understanding of psychoacoustics was incorrect.

If you want to inform yourself about how we perceive distinct auditory objects in an auditory scene, you may be able to answer this for yourself i.e. read Bregman & auditory scene analysis.
 
@DF96,

i am still interested in your answers to my questions about the analysis of these data points and possible conclusions and about the inclusion of test results as data points when we don´t know enough about the tests.

Wrt to your fuel example, i think the difference is obvious as the fuel type is easily measured while in our discussion the difference is measureable but any result depends on listeners and side effects.
 
..

No it doesn't. ABX does not ask people to identify the difference, but merely say whether they believe X is A or B.

In ABX difference testing, don't try to spot a specific difference - just pick what you genuinely think sounds like A or B. Don't be biased by others telling you that there should be 'night and day' differences between DACs - do give it a real shot & ask yourself on each trial which do I think/feel it is even if you can't say why.
You probably have no experience of ABX testing - look at what the one participant in the test says here - what does he "focus" on?
Why would 'trained listeners' be an crucial element in ABX testing - what are they being trained in - how to guess one thing sounds the same as another? No, they are training in the sound of specific distortions


Curiously, blind testing also seems often to eliminate the 'hearing a difference' bias too. People who firmly believe that cables make a difference are often unable to demonstrate this when blind.


When people who have a preconceived notion that there are no differences between whatever devices & they think they are testing two such devices, will NOT perceive a difference, no matter what is ACTUALLY being tested in the blind (unless there is major gross differences). In other words a pre-bias of a null result will result in a null result :)
 
...Of course now the problem exists that the listener is biased to strongly due to his knowledge so any new test approach must take that into consideration.

Yes & his admission that he would fail an AB preference test disqualifies him from any properly administered perceptual test unless one includes hidden anchors & controls which will likely expose if his bias is affecting his perception. Pre-screening & post-screening are usually part of well administered perceptual testing
 
Status
This old topic is closed. If you want to reopen this topic, contact a moderator using the "Report Post" button.