It is difficult/impossible to say which is the most important "variable" because a clearly expressed topic/hypothesis that you want to examine is still missing.
A clearly expressed research question is the beginning, then you are able to develop the experimental conditions to examine it.
As long as any of such tests do not include at least one positive control (a known audibility, eg taken from another test) nobody can take any conclusion.
Such tests without positive control could even easily be sabotaged, by just writing down the results at random. That would "prove" (according to the belief af many here) that there is no audible difference.
Or just chose literally deaf people. Or whatnot.
With that I'm by no means saying that it was the case, but just to point out that if no positive controls are used (where the participants "must" be able to discriminate) , the whole test might be valid for the participants, but not for the whole audience here.
Such tests without positive control could even easily be sabotaged, by just writing down the results at random. That would "prove" (according to the belief af many here) that there is no audible difference.
Or just chose literally deaf people. Or whatnot.
With that I'm by no means saying that it was the case, but just to point out that if no positive controls are used (where the participants "must" be able to discriminate) , the whole test might be valid for the participants, but not for the whole audience here.
That's why I wanted Jo to give details of what he seemed to suggest in an earlier post - that positive results were being seen but they turned out to be a 0.2dB amplitude difference - if the details check out, this would indicate the sensitivity of the setup & participants to such differences - but if these results weren't statistically significant, then it's just coincidence/luck
Threshold can be obtained only by:
1. Enough audible differencial between DUTs
2. ''Golden ears'' participant
Point 1. would mean to find the poorest quality DAC available anywhere, which would be silly and totally useless. The cheapest one (19,99$) is enough. And the Forssell is considered a very high end studio converter. Also we use an Audiophile DAC which gave us same (non-audible) results.
Point 2. Passed a certain point, the ''golden ears'' quest is irrelevant. Only 10-20% of the population that could hear a difference, OK. But not only 0.01%... That test is not some unicorn chase, it's meant to have a sample of the population and typical users of such converters.
And, no. At least one participant is proven ''not deaf'' by 2 recents audiograms. And there is absolutely no indications that other participants were ''deaf'' or with any problem making them unqualified to go through that blind test.
Last edited:
Update: we will have new participants soon, including younger people and musicians/sound engineers.
I will try to reach 15-20 participants and over 500 trials, after i'll close the test and will update with the conclusions.
I will try to reach 15-20 participants and over 500 trials, after i'll close the test and will update with the conclusions.
I think i will have to repeat few things, here in that thread, every 10 pages or so...
Occam's razor: If nobody can hear a difference, it's probably because there is no audible difference.
Occam's razor: If nobody can hear a difference, it's probably because there is no audible difference.
Good Luck 
This test is a convincing eye-opener only when you go through by yourself. By witnessing the whole thing, in person.
Last call invitation on diyaudio: final day of that DAC blind test will be the 10th december. PM me if interested.
Last call invitation on diyaudio: final day of that DAC blind test will be the 10th december. PM me if interested.
The lack of participant is a problem in any test that is looking for reliable answers reflecting a large comparable (population and/or typical users).
The type of participants (profile) is also very important. 8-10 years old girls wouldn't reflect the typical users of said digital to analog converters...
All that is simply common sense, really.
Now, the lack of participant is to be taken with a grain of salt. Only 2-3 participants is a real problem, but chances that 15-20 participants are not qualified (deaf) for such test, are next to non-existent.
In any case, no absolute answers can be given, only more or less % of certainty.
Put simply: a (hypothetical) 95% of chance that converter's sonic differential are inaudible would provide enough information to make decisions about that type of component.
There is no need to dig any deeper or to make something simple awfully complex: absolute answers... are not the answer. Some audiophiles thought DACs were giving day & night kind of results, they're not, end of story for most of the population. Now, is some golden ear on the planet COULD spot one single difference in a particular context? Maybe. Maybe not. But does it really matter?
For most of the population: NO.
You, diyaudio members, could take the following perspective:
What would be your ODDS to get one successful identification result ?
At this moment, i think everybody would agree that it's a very long shot. Nobody here would bet real money on their chances.
Now, some wish to pursue a crusade against the very concept of the ABX test... and that is another topic entirely.
What would be your ODDS to get one successful identification result ?
At this moment, i think everybody would agree that it's a very long shot. Nobody here would bet real money on their chances.
Now, some wish to pursue a crusade against the very concept of the ABX test... and that is another topic entirely.
What s being told to you is that you don't know the revealing quality of your ABX test. Only by knowing that can the test be evaluated.
I asked about the 0.2dB differences you seem to say were being picked up in the ABX - were they - any details? This would go a long way towards showing your test was sensitive enough to reveal small differences.
You are doing more harm to the concept of ABX testing than anybody else by ignoring all the good advice given in this thread - you are deaf to this advice!!
LOL - too bad it isn't a requirement at diyAudio for participants to identify their commercial interests as DAC designers or manufacturers...
Here is a simple, repeatable test, with samples, on the audibility of hearing digital audio artefacts: Fun With Digital Audio – Bit Perfect Audibility Testing - Bits and Bytes - Computer Audiophile
Why is it important? It establishes a baseline on what level of audibility one can hear a difference whether it is a frequency or bit-depth change. For a broad frequency change, the best I could do is about 1 dB difference. For bit-depth, roughly -72 dBFS, which corresponds to 12 bit resolution. If one spends time at Hydrogenaudio Forums the results I achieved are typical of what others have concluded.
Given that virtually every DAC measures well within +- 1 dB frequency response and better than 12 bit resolution, it comes as no surprise to me that any properly designed and implemented DAC sounds the same. In fact, that is the entire raison d'être behind digital audio. Don't take my word for it, try the samples yourself.
Also, try Archimago's Musings in which Archimago has measured and listened to dozens of DAC's, including some of the cheapest DAC's available and one can see the results exceed our hearing capabilities.
Another objective measurement, in which one can also participate in is:
Gearslutz Pro Audio Community - View Single Post - Evaluating AD/DA loops by means of Audio Diffmaker As a studio guy, I use the top AD DA converter in the list, but I doubt I can hear the difference relative to the other DAC's.
Finally, if you worried about the jitter boogeyman getting you, try listening to these jitter samples: jitter_1
The reason I am posting this is to provide some additional links to listening and measurement tests to put into perspective what is being said here on diyAudio. I hope folks try some of these listening tests and determine for oneself what is audible and what is not.
Here is a simple, repeatable test, with samples, on the audibility of hearing digital audio artefacts: Fun With Digital Audio – Bit Perfect Audibility Testing - Bits and Bytes - Computer Audiophile
Why is it important? It establishes a baseline on what level of audibility one can hear a difference whether it is a frequency or bit-depth change. For a broad frequency change, the best I could do is about 1 dB difference. For bit-depth, roughly -72 dBFS, which corresponds to 12 bit resolution. If one spends time at Hydrogenaudio Forums the results I achieved are typical of what others have concluded.
Given that virtually every DAC measures well within +- 1 dB frequency response and better than 12 bit resolution, it comes as no surprise to me that any properly designed and implemented DAC sounds the same. In fact, that is the entire raison d'être behind digital audio. Don't take my word for it, try the samples yourself.
Also, try Archimago's Musings in which Archimago has measured and listened to dozens of DAC's, including some of the cheapest DAC's available and one can see the results exceed our hearing capabilities.
Another objective measurement, in which one can also participate in is:
Gearslutz Pro Audio Community - View Single Post - Evaluating AD/DA loops by means of Audio Diffmaker As a studio guy, I use the top AD DA converter in the list, but I doubt I can hear the difference relative to the other DAC's.
Finally, if you worried about the jitter boogeyman getting you, try listening to these jitter samples: jitter_1
The reason I am posting this is to provide some additional links to listening and measurement tests to put into perspective what is being said here on diyAudio. I hope folks try some of these listening tests and determine for oneself what is audible and what is not.
I might be deaf for misleading advices, but i'm not physiologically deaf. As shown by my 2nd audiogram passed in one week...
http://www.diyaudio.com/forums/the-lounge/315242-time-visit-audiologist.html#post5259116
The audiologist from the Musician's Clinic was kind enough to give me a copy of one of the very few studies on extended high frequency. Rodriguez Extended Frequency study (2014) - International Journal of Audiology. Very interesting as you can see on the link above.
645 healthy subjects, all groups of age, 8khz, 9khz, 10khz, 11.2khz, 12.5khz, 14khz and 16khz.
I'm not 100% convinced the dBHL could have a direct impact on such ABX test's outcome, but it could be interesting to have some younger people results. Differences between age groups are huge, especially 14khz+... and as soon as you reach 50yo, pretty much all very high frequencies (8khz+) are affected in some dramatic way (average subject).
We're talking 86db differencial between 5-19yo average and 50yo 95th percentile+ at 14khz. That's ultra-sensitive V.S. almost deaf.
http://www.diyaudio.com/forums/the-lounge/315242-time-visit-audiologist.html#post5259116
The audiologist from the Musician's Clinic was kind enough to give me a copy of one of the very few studies on extended high frequency. Rodriguez Extended Frequency study (2014) - International Journal of Audiology. Very interesting as you can see on the link above.
645 healthy subjects, all groups of age, 8khz, 9khz, 10khz, 11.2khz, 12.5khz, 14khz and 16khz.
I'm not 100% convinced the dBHL could have a direct impact on such ABX test's outcome, but it could be interesting to have some younger people results. Differences between age groups are huge, especially 14khz+... and as soon as you reach 50yo, pretty much all very high frequencies (8khz+) are affected in some dramatic way (average subject).
We're talking 86db differencial between 5-19yo average and 50yo 95th percentile+ at 14khz. That's ultra-sensitive V.S. almost deaf.
Last edited:
But is the ABX test deaf? - that's what you have refused to test or to answer if 0.2db was differentiated in your ABX test to any significant level ?
... i'd like to add that i'm having difficulties to see any relevance of seeking for high-end audio equipment if one is having really poor hearing capacities... Unless your audiogram shows otherwise, if you're 50yo+, chances that your capacities are much handicaped are high.
5-19yo people not only have much more sensitive (dBHL) hearing, but their frequency response is flatter, even the worst among them (95th percentile)... Worst is 9.5db differential between 11.2khz and 16khz (average people tested in that group of age).
...while we talking almost 30db within the group of age 40-49yo (8khz V.S. 14-16khz)...
and within the group of 50-59yo, it goes to almost 40db and reaches 56.5db in the 95th percentile...
And all that is just for the 8khz-16khz bandwith, there is probably even more differencial if lower frequencies are also compared.
Older people needs DSP in their systems more than anybody, is that it ? Or maybe it's just a total waste to spend money and energy on high-end gears 😕
Well, that Rodriguez study is quite surprising.
5-19yo people not only have much more sensitive (dBHL) hearing, but their frequency response is flatter, even the worst among them (95th percentile)... Worst is 9.5db differential between 11.2khz and 16khz (average people tested in that group of age).
...while we talking almost 30db within the group of age 40-49yo (8khz V.S. 14-16khz)...
and within the group of 50-59yo, it goes to almost 40db and reaches 56.5db in the 95th percentile...
And all that is just for the 8khz-16khz bandwith, there is probably even more differencial if lower frequencies are also compared.
Older people needs DSP in their systems more than anybody, is that it ? Or maybe it's just a total waste to spend money and energy on high-end gears 😕
Well, that Rodriguez study is quite surprising.
Last edited:
Well Jon thankfully you are wrong, high frequency hearing loss is actually no big deal for the older audiophile, and the loss is usually so gradual that one adapts without even being aware of it......ah.....happy times ahead for all
It's not a SPL's ABX test, no. Studies in that regard are probably available on internet, but as far as i know, it's somewhere between 0.1db and 1.0db for the vast majority of the population.
We unoffically spotted differencial in the 0.5-1.0db range, with system set-ups 1 & 2, without much effort, but after with the 3rd set-up, a 0.2db diff was more or less identified, just before it was recalibrated to a tighter match.
Now it's somewhere between 0.00db and 0.10db diff, difficult to know with our equipment. Anyway, DUTs themselves are most probably not 0.00db on all bandwith, as a member pointed out... So we're probably thinking beyond the SPL-matching required for such test, but rather in the limits of DUT compensation.
gradual, i understand... but why it wouldnt be a big deal for the high frequency AND the unflatten response ?
What's so great about flat, and when is it flat, at what level, and where do I have to sit, and...........and..........and.......?