And this is the most blatant diversionary tactic!
Of course ABX testing "is incapable of deciding that 2 DUT are the same." It was never designed to do that!
Its purpose is to show that they are different.
This is the key question.
If the test was implemented over and over, and no differences are heard, we might conclude that no audible differences existed.
We might reasonably question, however, whether the tests were flawed, either in conception or in implementation, and that this is what caused the null results.
ABX testing isn't per se "scientific", whatever that means, as seems to have been suggested.
And that's why the argument never gets anywhere. Its based on a flawed methodology.
Failing to hear a difference is an inconclusive result. It doesn't "prove" there isn't a difference.
A positive result only occurs when someone can reliably identify the difference under controlled conditions, confirming that the difference is real and lies within the limits of human perception.Then the null hypothesis (that there is no difference) can be rejected.
Last edited:
No.
The test provides useful information, just not the absolutes that people are looking for. I find it particularly useful in that I now know, in my situation, there is no point in buying an expensive DAC.
Last edited:
Always had this conslusion - the "audio memory" of the people is very limited unless trained. People hear difference, but forger it the moment the music stops playing. It's how our brain works and it takes a lot of forcing him into developing that memory and ability to pick such details. Some people can't even hear a major difference in tonality change - something which professional musicians pick instantly.
But overall - similar and well executed modern DAC's do sound close, but not indentical. The improvements are about getting those last 5% of details and clarity out of that limited 44.1 recording. And at 192khz all DAC's sound surprisingly well...
But overall - similar and well executed modern DAC's do sound close, but not indentical. The improvements are about getting those last 5% of details and clarity out of that limited 44.1 recording. And at 192khz all DAC's sound surprisingly well...
There are some complaints because evidence from controlled listening tests for this difference in sound is still missing....
Jacob as you well know, there's always a mix of type 1 and type 2 errors. The false positive rate of word of mouth anecdotes is inordinately high and some will complain that the false negative rate of controlled listening tests are too high.
Edit: I don't remember the original test methodology to tell whether it says anything with any weight or not. My assumption is it's a scattershot.
Edit: I don't remember the original test methodology to tell whether it says anything with any weight or not. My assumption is it's a scattershot.
Last edited:
Do you mean the ABX protocol in general?
Hard to say; if an experimentor is interested if a difference between two DUTs exists, the ABX is one of the options, but the experimenter should be aware of the fact that participants (most likely) need more accomodation time to get used to the special conditions before reaching sufficient sensitivity.
Usually at this end level of decision from costumers it is the least interesting question, preference matters more, therefore an ABX does not help.
As said before, it is crucial to express the objective as clear as possible and to choose then the most appropriate test method for the task.
I´d say, the test described in this thread doesn´t provide useful information, as (due to the lack of all the important details) it only allows the general conclusion "the null hypothesis could not be rejected", but no forther conclusions wrt reasons for this result. (beside the most obvious one
)How could you conclude that?
We know next to nothing about listener`s sensitivity under these test conditions (leaving aside the quite confuse description of the test actually done) and next to nothing about the degree of difference that could have been detected.
Furthermore, why should their result be important for your needs?
Just for clarification, if their results would have been positive, would you then be forced to buy an "expensive" DAC without listening for yourself?
But even if you find you get a null result in such a test does not mean you can’t hear a difference under normal conditions. If is very often the case that the way you listen during the test is different than when you listen for pleasure.
So the test, with null result, still serves no purpose.
dave
Do you mean the ABX protocol in general?
Hard to say; if an experimentor is interested if a difference between two DUTs exists, the ABX is one of the options, but the experimenter should be aware of the fact that participants (most likely) need more accomodation time to get used to the special conditions before reaching sufficient sensitivity.
Usually at this end level of decision from costumers it is the least interesting question, preference matters more, therefore an ABX does not help.
As said before, it is crucial to express the objective as clear as possible and to choose then the most appropriate test method for the task.
I´d say, the test described in this thread doesn´t provide useful information, as (due to the lack of all the important details) it only allows the general conclusion "the null hypothesis could not be rejected", but no forther conclusions wrt reasons for this result. (beside the most obvious one
How could you conclude that?
We know next to nothing about listener`s sensitivity under these test conditions (leaving aside the quite confuse description of the test actually done) and next to nothing about the degree of difference that could have been detected.
Furthermore, why should their result be important for your needs?
Just for clarification, if their results would have been positive, would you then be forced to buy an "expensive" DAC without listening for yourself?
Are you trying to tell me what is important to me, in my situation??
As for the rest, were off in circles again.
You never know. You need to get out there and listen for your own taste.
I had 2 of the most hifi Sony cd players which I sold now. I am as happy listening to another version of the imperfection of digital music, the I/V is shown AK4396: best solution for output stage 10 years later I post it there
I think its a question of perspective.
If someone is asking 100 times the price, I don't expect to have to listen carefully to hear a difference!
Difference in price relativity:
MSB Select II: 90k
Fiio Dac: $100
Cost difference: 900x
Difference instantly obvious? Open to debate.
Worth it? Only you can decide.
2018 Ferrari 488 GTB: 250k
Honda Civic: 19k
Cost difference: 13x
Difference instantly obvious? Yes.
Worth it? Only you can decide.
Personally, with 90k to burn only on audio I’d put 95% of it toward loudspeakers and handling room response, where things are much more clearly influential and measurably different.
But with 90k to burn generally, I’d rather build a hospital in a third world country with a high infant mortality rate.
However, to each their own.
Regardless of your interpretation of the results, it does suggest depreciating returns which I think is the important takeaway.
Can one hear the difference after intimate involvement in their own listening space? I’m not sure, but even if the answer is yes it doesn’t strike me as the area best to place my personal energy.
MSB Select II: 90k
Fiio Dac: $100
Cost difference: 900x
Difference instantly obvious? Open to debate.
Worth it? Only you can decide.
2018 Ferrari 488 GTB: 250k
Honda Civic: 19k
Cost difference: 13x
Difference instantly obvious? Yes.
Worth it? Only you can decide.
Personally, with 90k to burn only on audio I’d put 95% of it toward loudspeakers and handling room response, where things are much more clearly influential and measurably different.
But with 90k to burn generally, I’d rather build a hospital in a third world country with a high infant mortality rate.
However, to each their own.
Regardless of your interpretation of the results, it does suggest depreciating returns which I think is the important takeaway.
Can one hear the difference after intimate involvement in their own listening space? I’m not sure, but even if the answer is yes it doesn’t strike me as the area best to place my personal energy.
Last edited:
That´s why i mentioned it.
Wrt "inordinately", it is what i´d assume too, but have to admit that due to my own bias i think the false positive rate is much higher in some effect classes than in others.
Basically we don´t know mostly because we don´t try to find out what is really going on. Due to our own experiments that i´ve done with around 130 different people during the years, i know that some underestimated, some overestimated their hearing abilities, that i´ve seen (no pun intended) some very kean listeners who are totally uninterested in sound quality.
Overall it is imo the same as with every other human ability/expertise - there exists a wide range and you have to find out who is supposed to be more often right in his evaluation compared to others.
And i´ve learned that using only quantitative methods can be quite misleading; qualitative methods do often give a lot more of useful insight.
And it was very interesting/revealing to see that people are able to give very consistent, precise and specific descriptions of sound field properties while nevertheless struggling to get positive results in controlled "blind" listening tests using the same musical excerpt.
That´s my recollection too....
Last edited:
- Status
- This old topic is closed. If you want to reopen this topic, contact a moderator using the "Report Post" button.