A particular ABX test might not tell us what level of difference is audible, but a set of ABX tests could tell us if it is possible to vary the difference and see when it becomes distinguishable.mmerrill99 said:
Whatever the outcome of a test of any sort, there will always be grounds for dismissing the result if it is too inconvenient. However, most ears-only tests give results which are not too surprising given our existing knowledge.
You have avoided all posts about ABX testing which deal with it's possible flaws, not just from me so your agenda is very obvious.
Last edited:
first: you failed to include this vital information into your first post
second: as what i expect thats where the problem lies, basically class A amps (such as Nelson Pass) are the de-facto standard for listening tests, never a class D
See? "The test was no good because it used a different type of amplifier from the ones I prefer". Use a Class A amp and he will then complain about your cables. Use silly expensive cables and he will complain about your room. Fix the room and people will complain about the listeners. Then the music. Then the mains conditioning. Then the weather. If someone believes that a $3000 DAC must sound better than a $30 DAC then no test will satisfy him.
No. So what is your point? Please don't trot out the old one about not commenting unless you have experienced it. I have never experienced Mount Everest, but I believe that climbing it might be difficult.
Failure of logic, yet again.
You think that ABX tests are somehow consistent i.e that if you do one ABX test it has the same conditions as the next ABX test. Wrong, each ABX test has a particular set of varying conditions & we have no way of determining how these conditions have effected the results. As I said ABX tests are severely skewed towards null results & we have no way of knowing if these were false negatives due to lack of controls within the test.
Comparing many ABX tests that masks real differences tell us nothing. One needs controls within each ABX test to evaluate the sensitivity of that particular test & it's conditions.
When a test is based on lack of knowledge & lack of concern for how to properly run a test, it's obvious what the result will be & your attempts at excusing such cavalier approach to this test & your general lack of knowledge r care about such tests, reveal an agenda which you ascribe to others.
I suggested a blind preference test which you reject so it seems that you don't actually believe in "most ears-only tests" despite your words.
I would suggest that you shouldn't comment on ABX testing when you demonstrate no knowledge or interest in informing yourself about them.
If I have a Wavebourn 'niceness' knob which I can vary then ABX will tell me at what point on the dial it becomes distinguishable. Your problem is that you have decided that all amateur ABX tests must necessarily produce false results, so there is no pleasing you.mmerrill99 said:
Silly question. As there is only a weak correlation between price and performance, and DAC weaknesses are likely to be very different from speaker weaknesses it is certainly not necessary to test a $3000 DAC with $3000 speakers. In any case, there must always be some wriggle room for the doubters - if only $30k speakers had been used then the DAC difference would have been obvious? Maybe $30k cables too?kinsei said:
Sadly there will never be a meeting of the minds on this. Those to whom it is a religion that they can hear differences in everything, will never actually set up a blind test (using as long a time period as long or short as they wish, location of their choice and more to make it acceptable) as they don't want to get any results that doesn't agree with their vision.
Not much point in going any further as it is getting pretty deep already.
Not much point in going any further as it is getting pretty deep already.
why bother to answer a silly question? this is not a question indeed
when testing a dac, you simply want ALL other components to be as ideal as possible
otherwise you loose the ability to reveal the difference
Last edited:
Distinguishable wrt what? In general, casual listening and/or under test conditions?
You´re now nearly approaching the concept (mandatory) of positive/negative controls.
It starts with correct denotion; the term "ears-only" is a misnomer because a listening test is most likely never of the "ears-only" variant, neither a "sighted" nor a "blind" one. A bias-free human is extremely rare (that afair is what science tells us).
Of course dismissing because of inconvenience does occur, but it seems that liking of test results gathered in methodologically flawed tests isn´t uncomm either.....
Usually it is the sole responsibility of an experimenter to show that a test was valid, objective and reliable.
If he failed to show it, it doesn´t help to complain about the justified critic.
Last edited:
Your problem is that you don't know the first thing about ABX tests & refuse to learn so there's no hope of educating you and yet you are a vocal poster on this thread. All of what I say is verifiable in the perceptual testing research & testing standards as has been verified by Jakob2 & others (& available to read if you were interested) - all that you are saying is your own wishes, hopes, desires, beliefs based on nothing other than an agenda
Cup of agenda, anyone?? JB is brewing & DF96 is serving!!
Sorry, i don´t mean it offensive but you´ré sort of citing a "otherbelievers tale".
There is a plethora of research existing about proper design of experiments in general and more specific about sound controlled listening tests. Most of it is routinely ignored in "ABX tests" of the kind we are discussing now.
To cite the ITU-R BS.1116-3:
"Some of these considerations will be discussed later in this document. It should be understood that the topics of experimental design, experimental execution, and statistical analysis are complex, and that only the most general guidelines can be given in a Recommendation such as this. It is recommended that professionals with expertise in experimental design and statistics should be consulted or brought in at the beginning of the planning for the listening test. "
and furthermore:
"It must be empirically and statistically shown that any failure to find differences among systems is not due to experimental insensitivity because of poor choices of audio material, or any other weak aspects of the experiment, before a “null” finding can be accepted as valid. In the extreme case where several or all systems are found to be fully transparent, then it may be necessary to program special trials with low or medium anchors for the explicit purpose of examining subject expertise"
(Recommendation ITU-R BS.1116-3; Methods for the subjective assessment of small impairments in audio systems)
Last edited:
I don't think DF96 is a bad guy. Also, I don't think he necessarily has an agenda to dismiss every claim of hearing something outside of what existing research has shown for about 95% of people.
He may have an agenda to get people to admit that there have to be some limits to human hearing, whatever they may be. It would seem odd to have people who never admit to failing to hear a difference for no reason other than that they have probably reached the limits of what they can detect.
If we want to be like scientists and we are interested in what we can hear, then we should be interested in trying to accurately pin down our own limits too. To do that, at some point it becomes necessary to do some kind of blind testing with ourselves, but no saying it has to be ABX. It just needs to be honestly blind.
If we think we can do better non-blinded, fine. But, that doesn't mean we shouldn't want to find out and know our own blind limits under some particular conditions. No reason for embarrassment whatever the outcome. Doesn't bother me if somebody else can hear things I can't, good for them.
He may have an agenda to get people to admit that there have to be some limits to human hearing, whatever they may be. It would seem odd to have people who never admit to failing to hear a difference for no reason other than that they have probably reached the limits of what they can detect.
If we want to be like scientists and we are interested in what we can hear, then we should be interested in trying to accurately pin down our own limits too. To do that, at some point it becomes necessary to do some kind of blind testing with ourselves, but no saying it has to be ABX. It just needs to be honestly blind.
If we think we can do better non-blinded, fine. But, that doesn't mean we shouldn't want to find out and know our own blind limits under some particular conditions. No reason for embarrassment whatever the outcome. Doesn't bother me if somebody else can hear things I can't, good for them.
Controls are something that has been studiously & continuously ignored by JB & DF96 & as you correctly identify, this is mandatory for any such tests.
I didn't even want to correct this false term "ears-only" as all I said about auditory perception shows that it is far more than the ears being used in such tests which is why perceptual testing needs careful attention & knowledge.
You are correct to point out how the phrase "ears-only" is a misleading term meant to trick the unwary that it is only the ears that are being tested.
Earlier in the thread you said this, which caught my eye:
Do you have evidence of that specific scenario? I.e. (presumably blind) long-term listening tests with (presumably) repeatable preference expressed, but where there is no consciously detectable difference? It'd be a very interesting result.mmerrill99 said: