(UPDATES BELOW)
That blind test was long due, it's now done.
Started with a 30$ unit (Fiio) against a 3000$ one (Forssell) and once SPL-matched (massive gain difference), no one could tell the difference in a ABX test.
Then, we switched to a different set-up, using a pair of B&W CM9 speakers and the Forssell against a Eximus DP1 (3500$ or so). Same result: impossible to spot them in a ABX.
We were only 4 participants, but regardless it didn't feel like day & night difference to start with... ''Eyes opened'' we FELT differences, but couldn't prove it in the ABX.
Cables, amplifiers, Lossy v.s. Lossless/HD, EQ'd mid drivers, DAC... Nope. Nothing is passing a ABX blind test.
I'm pretty sure, now, that the human auditory capacities are very, very, overestimated.
🙁
The good news is: we can probably save a LOT of money.
-------------------------UPDATE 14th Nov 2017------------------------
Here is some pictures, including the whole room while we change set-ups (the 3rd one is on his way...)
-------------------UPDATE 16th Nov 2017-------------------------------
System set-ups #1 & 2 were very useful to spot the possible weaknesses of the whole process and equipement. So today was the Day 1 of the system set-up #3, which includes the following changes, based on observations and also comments and thoughts on this very thread:
.: New DIY speakers including Faital Pro 18FH500 and ribbon tweeters RAAL 140-15D.
.: High quality Solen passive crossover components, including dual Film & Foil capacitors for the tweeters.
.: Instead of iTunes and a computer, we used a Astell & Kern AK300 as the source (Toslink direct line out).
.: No switchbox anymore, a manual switch is made with each pair of RCA cables, in a way the participant cannot know.
.: Larger selection of music to choose from.
.: Shorter AND/OR longer music excerpts duration, whichever makes the participant more comfortable.
.: Uncompressed and also some 24/96 music files.
.: Closer distance to the speakers, from a previous 3,12m to 2,25m.
.: Better SPL-matching, using pure tones and double-checked with 2 differents mic.
Basically, to our actual knowledge, everything was carefully made to optimize the chances of a positive identification in that 3rd and final attempt. That test will continue to run for few days, we welcome participants from DIYaudio. Here are the first results:
3rd set-up - DAY 1
First participant was there last sunday (set-ups #1 & 2) and, sighted, was optimistic about his chances. Felt confident that changes made was for the best. FAILED to identify on every music excerpts.
Second participant, non-audiophile, male 34yo, and did not participate previous rounds. FAILED to identify on every music excerpts.
Comments collected afterwards was unanimous: even sighted, the potential audible differences were extremely thin. They both felt they could grasp some hints, but they didn't made it, through the test. They were the most confident with the 24/96 music file
Angel of Harlem from The Persuasions and the 16/44
Ungear Moi from CoH, but it didn't change the results whatsoever compared to other music files.
Conclusions of Day 1: From a very promising 3rd set-up, the hopes are now getting extremely low to find a participant who will be able to spot the 19,99$ DAC from the 3000$ one. We already know that 100% of the population is now an impossible target, we do hope that in the next days we'll find new participants that will get better results.
3rd set-up - DAY 2
3rd participant, non-audiophile, male 35yo, and did not participate previous rounds. FAILED to identify on every music excerpts.
4th participant, audiophile background, female 42yo, and did not participate previous rounds. FAILED to identify on every music excerpts.
5th participant, non-audiophile, male 45yo, and did not participate previous rounds. FAILED to identify on every music excerpts.
6th participant, non-audiophile, female 40yo, and did not participate previous rounds. FAILED to identify on every music excerpts.
7th participant, audiophile +DIY background, male 40yo, just passed audiogram (8dBHL, symmetric). FAILED to identify on every music excerpts.