DAC blind test: NO audible difference whatsoever

[JonBocani]

(UPDATES BELOW)


That blind test was long due, it's now done.

Started with a 30$ unit (Fiio) against a 3000$ one (Forssell) and once SPL-matched (massive gain difference), no one could tell the difference in a ABX test.

Then, we switched to a different set-up, using a pair of B&W CM9 speakers and the Forssell against a Eximus DP1 (3500$ or so). Same result: impossible to spot them in a ABX.

We were only 4 participants, but regardless it didn't feel like day & night difference to start with... ''Eyes opened'' we FELT differences, but couldn't prove it in the ABX.

Cables, amplifiers, Lossy v.s. Lossless/HD, EQ'd mid drivers, DAC... Nope. Nothing is passing a ABX blind test.

I'm pretty sure, now, that the human auditory capacities are very, very, overestimated.

The good news is: we can probably save a LOT of money.


-------------------------UPDATE 14th Nov 2017------------------------

Here is some pictures, including the whole room while we change set-ups (the 3rd one is on his way...)

An externally hosted image should be here but it is no longer accessible. Please upload images instead of linking to them to prevent this.

An externally hosted image should be here but it is no longer accessible. Please upload images instead of linking to them to prevent this.

An externally hosted image should be here but it is no longer accessible. Please upload images instead of linking to them to prevent this.

An externally hosted image should be here but it is no longer accessible. Please upload images instead of linking to them to prevent this.

-------------------UPDATE 16th Nov 2017-------------------------------



System set-ups #1 & 2 were very useful to spot the possible weaknesses of the whole process and equipement. So today was the Day 1 of the system set-up #3, which includes the following changes, based on observations and also comments and thoughts on this very thread:

.: New DIY speakers including Faital Pro 18FH500 and ribbon tweeters RAAL 140-15D.
.: High quality Solen passive crossover components, including dual Film & Foil capacitors for the tweeters.
.: Instead of iTunes and a computer, we used a Astell & Kern AK300 as the source (Toslink direct line out).
.: No switchbox anymore, a manual switch is made with each pair of RCA cables, in a way the participant cannot know.
.: Larger selection of music to choose from.
.: Shorter AND/OR longer music excerpts duration, whichever makes the participant more comfortable.
.: Uncompressed and also some 24/96 music files.
.: Closer distance to the speakers, from a previous 3,12m to 2,25m.
.: Better SPL-matching, using pure tones and double-checked with 2 differents mic.

Basically, to our actual knowledge, everything was carefully made to optimize the chances of a positive identification in that 3rd and final attempt. That test will continue to run for few days, we welcome participants from DIYaudio. Here are the first results:

3rd set-up - DAY 1

First participant was there last sunday (set-ups #1 & 2) and, sighted, was optimistic about his chances. Felt confident that changes made was for the best. FAILED to identify on every music excerpts.

Second participant, non-audiophile, male 34yo, and did not participate previous rounds. FAILED to identify on every music excerpts.

Comments collected afterwards was unanimous: even sighted, the potential audible differences were extremely thin. They both felt they could grasp some hints, but they didn't made it, through the test. They were the most confident with the 24/96 music file Angel of Harlem from The Persuasions and the 16/44 Ungear Moi from CoH, but it didn't change the results whatsoever compared to other music files.

Conclusions of Day 1: From a very promising 3rd set-up, the hopes are now getting extremely low to find a participant who will be able to spot the 19,99$ DAC from the 3000$ one. We already know that 100% of the population is now an impossible target, we do hope that in the next days we'll find new participants that will get better results.


3rd set-up - DAY 2

3rd participant, non-audiophile, male 35yo, and did not participate previous rounds. FAILED to identify on every music excerpts.

4th participant, audiophile background, female 42yo, and did not participate previous rounds. FAILED to identify on every music excerpts.

5th participant, non-audiophile, male 45yo, and did not participate previous rounds. FAILED to identify on every music excerpts.

6th participant, non-audiophile, female 40yo, and did not participate previous rounds. FAILED to identify on every music excerpts.

7th participant, audiophile +DIY background, male 40yo, just passed audiogram (8dBHL, symmetric). FAILED to identify on every music excerpts.

[wwenze]

The only DAC that I could really dare to say I heard a difference was a (fake?) TDA1543 DAC.
And forget hearing, it was measurable. (HF rolloff)

But after modding it and fixing the design error (*not really but long story) it sounded the same as the not-NOS DACs.

[abraxalito]

Originally Posted by JonBocani

The good news is: we can probably save a LOT of money.

Not only that but a LOT of time too

So based on your tests, an ES9023 DAC (say this one SA9023+ES9023?????DAC??/???-??? @ 69RMB) feeding a TPA3116 amp (say this one ??????? TPA3116D2 Hifi Audio Amplifier 200W?????-??? @ 75RMB) plus a laptop power supply is all you'll ever need?

[JonBocani]

Unless my test was flawed somehow, as far as i'm concerned the best solution would be to use the built-in converters on the DSP of your choosing.

[Praudio]

What model Fiio is ?

[planet10]

Originally Posted by JonBocani

I'm pretty sure, now, that the human auditory capacities are very, very, overestimated.

Or it speaks to how good an ABX test is. BTW the ABX is incapable of statisically deciding that 2 DUT are the same. Your results are only applicable to your text on that day.

dave

[abraxalito]

Originally Posted by JonBocani

Unless my test was flawed somehow, as far as i'm concerned the best solution would be to use the built-in converters on the DSP of your choosing.

From my pov your test isn't so much 'flawed' rather the wrong test. I listen to music for how it makes me feel. So I'll continue with equipment which delivers the emotional goods. I'm happy that you've found a super-cheap solution which meets your needs

[Bigun]

given that modern DACs are now highly evolved with extremely high performance I see no reason why they should sound differently unless implementation issues have not been addressed, or as is common in audio, we simply prefer a little flavour and sometimes an old wine is best

I plan to save a lot of time and money myself

[vdi_nenna]

Don't forget output stages.

[JonBocani]

Originally Posted by planet10

Or it speaks to how good an ABX test is. BTW the ABX is incapable of statisically deciding that 2 DUT are the same. Your results are only applicable to your text on that day.

dave

ABX testing method is widely used by multi-billions pharmaceutical companies for decades. In fact, it's a sina que non condition for FDA approval.

My methodology for this particular test may be flawed, but i don't think the ABX method is.

Or prove me wrong.