DAC blind test: NO audible difference whatsoever

Status
This old topic is closed. If you want to reopen this topic, contact a moderator using the "Report Post" button.
If I stopped four cars on the street and asked what fuel they use I might find two petrol, one diesel and one hybrid.

There are only few types of fuel in common usage, but there is a huge number of different ways people can hear, with it perhaps being unique to each human on earth. I would be happy to recommend a good book on biostatistics. At least you know control theory and don't make silly comparisons in that area.
 
Because that's the most likely thing a DAC would do that would affect the psychoacoustics, raising the upper midrange for example can increase the "presence"

All DACs I have ever seen are pretty flat. The linear distortion they have that is objectionable usually has to do with phase shifts in the antialias filter. The filters also can have some small passband ripple, but not a wide peak like the whole upper midrange. The ripple might make a difference in some cases.

In addition to those things they also have nonlinear distortions, such as making cymbals sound like bursts of noise. They also affect quality of the stereo image illusion. They have other distortions as well that are not like what amplifiers do, so trying to think of DAC distortions as being much like amplifier distortion (or speaker distortion) would be incorrect. For example, another nonlinear DAC distortion is for intersample overs, which is very different from clipping.
 
Jakob2 said:
It could have been that his impression from the sighted listening was actually correct while his result from the "blind" test does not reflect the reality.
Yes. The converse could be true too.

The answer is, he can´t know and we as outside observers can´t know either, therefore the idea to try another test protocol. Of course now the problem exists that the listener is biased to strongly due to his knowledge so any new test approach must take that into consideration.
There already was the problem that the listener was originally biased to expect to hear a difference and was surprised to find that under blind conditions he could not. We need to avoid one of the following two scenarios:
1. the listener originally expected to hear no difference - so all his tests are biased and can be ignored (unless he hears a difference, which shows he has now overcome his bias)
2. the listener originally expected to hear a clear difference, but under blind conditions he did not - so now all his subsequent tests are biased and can be ignored (unless he hears a difference); the original test can also be ignored because it was clearly inadequate.

mmerrill99 said:
When people who have a preconceived notion that there are no differences between whatever devices & they think they are testing two such devices, will NOT perceive a difference, no matter what is ACTUALLY being tested in the blind (unless there is major gross differences). In other words a pre-bias of a null result will result in a null result
You appear to be arguing that expecting a null result guarantees a null result, so the result can be ignored. Expecting a non-null result but finding a null result also can be ignored (according to this thread). So the only result with any validity is a non-null result. Have I understood you correctly? If so, anything which someone somewhere declares to be audible must necessarily be regarded as audible because no test can demonstrate that it is non-audible. A corollary of this is that all claims made in advertising are true, and all money spent on expensive audio equipment is money well spent (however 'creative' is the good story attached).
 
[...]the original test can also be ignored because it was clearly inadequate.[..]
As the ABX test and the claim based on it too.
I really can't take ANY ABX test seriously without a positive control. Especially since there is abundant evidence, shown by Jakob2 with very thorough and substantiated posts, that ABX is by no means the most adequate, let alone the only existing test, and especially with not well trained people there are other formats to prefer.
I really wonder why no one listens to him. He knows much more about this topic than most here.


Must be preconception...
 
Yes. The converse could be true too.

Of course, but
-) i really think, that "could have been" (this way) already includes the possibility that "it could have been the other way round". Am i mistaken?

-) meaning depends on context too. At this point the listener expressed it as a fact that no audible difference between the two DUTs exists (at least to him) because he concludes it from his tests and subsequently asked what another test could show, because of this "fact".
So in this context - to answer the questions - there was no need to explicitely state that he might be correct with his conclusion.


There already was the problem that the listener was originally biased to expect to hear a difference and was surprised to find that under blind conditions he could not. We need to avoid one of the following two scenarios:
1. the listener originally expected to hear no difference - so all his tests are biased and can be ignored (unless he hears a difference, which shows he has now overcome his bias)
2. the listener originally expected to hear a clear difference, but under blind conditions he did not - so now all his subsequent tests are biased and can be ignored (unless he hears a difference); the original test can also be ignored because it was clearly inadequate.

It seems that you have missed what i wrote repeatedly, i.e. that any controlled test only results in "the nullhypothesis can be rejected" or "the nullhypothesis can not be rejected" and that any further conclusion (expecially about the reasons for these results) are only warranted if the test was objective, valid and reliable , which means fulfilled the main quality criteria.

If you didn´t miss it, but don´t really know what it means, just ask. As said before in a forum we should try to help each other out and nobody is expected to be an expert for everything.

May i politley request again answers to my questions? For your convenience, let me repeat the questions (explicitely and implecitely ones):

"Didn´t that mean that you had to accept any test result as long as you didn´t get enough information to consider it as most likely "sloppy"?
For example, if i don´t tell you if a test was "blind" or "sighted" and furthermore as little as in this thread, wouldn´t you have simply to accept the results as data points?

Gathering of data is nothing to condemn, but that seems to be a strange approach where you get a lot of data points but don´t know what the independent and what the dependent variables were.

I´d interested to learn in which way any analysis of this data could be executed and what conclusions could be based on it."
 
Last edited:
Thanks Mark, I suspected phase shifts may be a contributing factor, the audibility of those though is another issue. How do the nonlinear distortions affect the quality of the stereo image illusion?

For a monophonic signal source panned somewhere between the two speakers, one can close one's eyes and point to the perceived location in space somewhere between the speakers. The stability and perceived width of the location vary with DAC quality. With a very good DAC, the perceived virtual source location is solidly fixed in space and very narrow. Maybe 1/2" or 1" wide, possibly less. With lower quality DACs it may seem to be spread out in space, maybe 6" wide or of uncertain width too indistinct to tell exactly.

Location of a virtual source position is dependent on volume levels and on timing between the two channels. It turns out human hearing is very sensitive to small timing differences for estimating location. Something like that corresponding to a fraction of a wavelength of 20 kHz or up around there somewhere, but I would have look up exact numbers that have been measured. One thing to perhaps keep in mind is that groups of neurons in brains learn to phase-lock to various sound patterns including volume, frequency, and others, anything that is a pattern to a brain. Phase-locking can be very sensitive to small changes.

EDIT: Regarding phase shifts, some DACs have various antialiasing filters that can be selected. For DACs that do have that option, many people hear differences between the filters. That can be disconcerting since one can't know which if any of the sounds is the correct one for perfect reproduction. Maybe none of them.
 
Last edited:
Jakob2 said:
It seems that you have missed what i wrote repeatedly, i.e. that any controlled test only results in "the nullhypothesis can be rejected" or "the nullhypothesis can not be rejected" and that any further conclusion (expecially about the reasons for these results) are only warranted if the test was objective, valid and reliable , which means fulfilled the main quality criteria.
A test which only has a statistical outcome cannot reject (or fail to reject) anything, in spite of forms of words which may suggest otherwise. It can only give some indication about the likelihood of something being true, and hence the likelihood of it being false. This assumes, of course, that it cannot be both true and false and that it is a meaningful statement. The apparent glee with which some people make the point which you are making (but you make it quite soberly) suggests that they quite like the idea that difference can 'proved' but indistinguishability cannot.
 
A test which only has a statistical outcome cannot reject (or fail to reject) anything, in spite of forms of words which may suggest otherwise.

Yes, it can. It can to a predefined confidence level. In medicine that's usually 95% chance the result is correct. Sometimes 95% sure isn't sure enough, it depends.

Also, if something cannot be proven true, that doesn't prove it is false. It just means we don't know if it's true.

Amazon.com: Statistics For Dummies (For Dummies (Math & Science)) (9781119293521): Deborah J. Rumsey: Books
 
From post 696. I'll repeat it because the we're on the test, confidence, and hypothesis.

The OP has enough samples now to extract statistics.

The AB test should have a binomial distribution, as you only have 2 outcomes, just like a coin toss. So the 95% confidence interval, assuming its a unbiased random pick, is 0.5+/-1.96*( 0.5 * 0.5 / 263)^0.5 or p= 0.5 +/-0.0604 or [0.4396 to 0.5604] which is what the OP got (actual 0.475).

Now for some semantics. If your null hypothesis was "there is a difference", then you are within 95% confidence of a random result, so you cannot statistically say "there is a difference". For most people the converse is also true that "there was no difference" detected.

I don't understand the controversy it generated. :scratch1:
 
Last edited:
Status
This old topic is closed. If you want to reopen this topic, contact a moderator using the "Report Post" button.