In the case of the FiiO D03K which is using the CS4344 DAC chip the answer is 'no'. The DS for this chip clearly shows a 0.2dB droop in the FR by the filter cutoff frequency.
ABX testing seeks to prove a hypothesis and disprove a null hypothesis.
An example of a hypothesis is that two speakers sound different.
The null hypothesis is that they are indistinguishable from each other.
Statistically valid differences (i.e they sound different) can be demonstrated over a sufficiently large sample size to be a statistically significant difference between the two speakers.
The null hypothesis has no such clear cut conclusion as random guessing over a sufficiently large sample size will return 50% for either choice, which is the same result you get if the two devices are indistinguishable
An example of a hypothesis is that two speakers sound different.
The null hypothesis is that they are indistinguishable from each other.
Statistically valid differences (i.e they sound different) can be demonstrated over a sufficiently large sample size to be a statistically significant difference between the two speakers.
The null hypothesis has no such clear cut conclusion as random guessing over a sufficiently large sample size will return 50% for either choice, which is the same result you get if the two devices are indistinguishable
Exactly - if you sat a deaf monkey down to do an ABX test you would get a null result signifying the close to 50% split was pure guess work - it says nothing about differences not being heard. That is why hidden controls are needed or use a test with hidden controls - this would expose the monkey as being an unsuitable participant. It's what are called false negatives i.e the participant doesn't perceive a difference that really exists - it's total focus is to eliminate false positives i.e perceiving differences which don't actually exist
As soon as you approach the 50.00% point, over a large number of rounds & participants, it ''proves'' it's indistinguishable.
Still with large number of rounds and participants, if it's either south or north of 50%, then it means something is happening (potential identification). A 35% result would raise questions same as a 65% result. In fact, i'd be curious with 45 or 55%...
But 50.00% ?
When participants ALL said the perceived differences are subtle or very thin ?
-----
NOW, in the absolute... Does that mean no human on the planet could successfully spot a difference? No, not at all.
But the larger the sample, the tighter the test, the most cross-referenced info we can get (comments), the more we get close to a very helpful answer.
Which, in that case, would translate into a very simple yet blunt:
''Don't put too much money on Digital-to-Analog converters''
Joke aside, it IS possible to cheat an ABX test.
If you don't have the abilities to spot the difference, you cannot cheat, but if you have the abilities to spot the difference, you COULD tell otherwise (you would know the right and wrong answers, therefore you can manipulate them).
That would be twisted, but it's possible. So, yeah, if we are to be paranoid, then yes an ABX test can be cheated.
''The Golden Ears who Knows'' that would make a nice James Bond title, isn't it ?
125 success
138 error
total 263
I'm not sure i'll have the patience to reach 1,000... My mind is already onto the organisation of the 4th blind test and, frankly, i'm slowly but surely starting to lose interest in that DAC duel...
I'd like to point out also the fact that i'm not using the switchbox anymore. So i have to be very careful (or the person who controls the equipement while i'm doing it) when i plug/unplug the two sets of cables. Each time the tune stops, i remove the cables regardless if it would keep the same next time (sequence is pre-written), i smart*** could find a pattern just there and make a false positive.
But like i said, so far no positive. So no worries about that kind of problem or the SPL-matching not being perfect...
But like i said, so far no positive. So no worries about that kind of problem or the SPL-matching not being perfect...
to be honest i don't care much about the global results, i'm more interested in the ONE individual or the ONE tune or the ONE sound system set-up that would provoke a positive.
Only ONE positive identification event is needed. Wouldn't make the exotic DACs anymore worthy for the vast majority of us, but at least it would be made possible to spot the difference, in a particular context. That would be something.
Then, i could move from a 19,99$ Fiio to something else, and go on with another DAC duel... That was the first idea, you know? The Fiio was supposed to be the starter, the little warm-up.
Only ONE positive identification event is needed. Wouldn't make the exotic DACs anymore worthy for the vast majority of us, but at least it would be made possible to spot the difference, in a particular context. That would be something.
Then, i could move from a 19,99$ Fiio to something else, and go on with another DAC duel... That was the first idea, you know? The Fiio was supposed to be the starter, the little warm-up.
It isn’t a real test unless you calculate the statistical error. And if you learn enuff to do that you will realize the limits of the test. Your ignorance of the science is leading you down the garden path.
dave
dave
Of course, the person at the controls is not passing the test at the same time... And it's blinded, as really blinded: blindfold goggle.
For the sequences, it was generated as randomly as possible: lines of ABABABBBBABAAABABAAAABBBABABABABBBB then cut anywhere in-between and put on the participant's sheet. I'm not tempted to use a computer to generate random sequences because a computer is not really able to generate true random stuff, last time i checked.
That's the closest to serious-science i've seen of a blind test test on DIYaudio, Dave. Correct me if i'm wrong by pointing the thread to me. I'll be more than happy to read it.
As for the ABX testing per se as being limited, of course it is. Everything is limited... There is no absolute proof of anything. At the risk of repeating myself for, like, the 10th time... I'm not seeking for absolute answers, i just want the big picture. IS IT OR IS IT NOT a day & night difference. Obviously, it's not, on the DAC question. That ship has sailed already. ALL the audiophiles who participated to that test so far won't put big money on a DAC, probably for the rest of their lives. It's over. No coming back. I'm sure selling mine, without any second thought.
The big picture, in case you all objectors, have forgotten, is not only we should've witnessed a positive identification between a 19,99$ and a 3,000$ devices... but that (yet to come) difference still would've to be proven WORTHY, if any would come...
Identification is only the very first, basic, step.
Then, the appreciation testing would be on the to-do list. ''is the more expensive device sounds better, now that you can differentiate them?'' Then the evaluation of price v.s. quality. ''Ok now that you prefer the more expensive one, HOW MUCH would you pay extra for it''...
See where i'm getting at? Is it clearer now?
We're LIGHTYEARS from that.
LIGHTYEARS.
Identification is only the very first, basic, step.
Then, the appreciation testing would be on the to-do list. ''is the more expensive device sounds better, now that you can differentiate them?'' Then the evaluation of price v.s. quality. ''Ok now that you prefer the more expensive one, HOW MUCH would you pay extra for it''...
See where i'm getting at? Is it clearer now?
We're LIGHTYEARS from that.
LIGHTYEARS.
To continue on that...
If you cannot differentiate the two, the least expensive wins.
If you can differentiate the two but prefer the least expensive.. guess which wins?
If you can differentiate, prefer the more expensive, but don't think it's worthy of the 150x price tag over the cheapest one, the least expensive wins.
That's 3 scenarios that says ''DO. NOT. PUT. BIG. MONEY. ON. DAC.''
Like i said, we're lightyears even from the 3rd scenario.
If you cannot differentiate the two, the least expensive wins.
If you can differentiate the two but prefer the least expensive.. guess which wins?
If you can differentiate, prefer the more expensive, but don't think it's worthy of the 150x price tag over the cheapest one, the least expensive wins.
That's 3 scenarios that says ''DO. NOT. PUT. BIG. MONEY. ON. DAC.''
Like i said, we're lightyears even from the 3rd scenario.
Hello all,
I haven't been on here in years after my planned DIY DAC build stalled due to life events but I am starting to look back into it as a winter evening project (new thread on its way).
I must say that this thread (I haven't read the whole 68 pages but I feel I have picked up the general gist) has made rather interesting reading.
I am now a bit more comfortable that with my HiFi setup in a domestic family living room I can't go wrong with whatever DAC I build.
I haven't been on here in years after my planned DIY DAC build stalled due to life events but I am starting to look back into it as a winter evening project (new thread on its way).
I must say that this thread (I haven't read the whole 68 pages but I feel I have picked up the general gist) has made rather interesting reading.
I am now a bit more comfortable that with my HiFi setup in a domestic family living room I can't go wrong with whatever DAC I build.