DAC blind test: NO audible difference whatsoever

Status
This old topic is closed. If you want to reopen this topic, contact a moderator using the "Report Post" button.
Guitar distortion is often quite extreme, but guitarists are often very particular about exactly what the distortion sounds like.

Yes, I have dozens of guitars, amps, etc, I get that.


In principle, yes. However, some people can hear much lower than -40dB distortion depending on the exact distortion. THD as a measurement correlates very poorly with human perception, which as been known for many decades. Primarily, it is still used because it is easy to measure, and manufacturers feel consumers expect to see it. Its not clear what distortion metric you are thinking of.

There are a range of distortion types. Most all are measured by the an artifact's level below the fundamental. An ITU IMD test at -90 I can readily hear, crossover distortion I can maybe reach -60.

But still, with amps delivering way beyond that...

What would you estimate?

That was my point for posting originally: Let everyone measure using a tool that can incrementally introduce reduced levels of distortions of various types, and then build a repository of info. And then we know. What is your threshold for slew rate distortion? Crossover? Quantization? Amplitude imbalance?

Can someone really hear what others claim cannot be heard?

Science and testing can answer all of these.
 
@QAMATT - I would love to see where we can find all possible distortions reported for any device?

I don't know of a place either. But I'd come at this from another angle: Rather than comparing two amps that might have wildly different characteristics that interact in ways we could hardly guess, why not use the same amp, speaker, etc, and then contaminate a source material with a single type of distortion and establish a threshold for that?
 
Rather than comparing two amps that might have wildly different characteristics that interact in ways we could hardly guess, why not use the same amp, speaker, etc, and then contaminate a source material with a single type of distortion and establish a threshold for that?

Rather a long shot to assume that all distortions are within the passband of the source. In my experience many SQ differences are due to out-of-band signals getting into the audio so the 'distortion' couldn't be included in a digital stream it would need application at the analog output of the DAC.
 
I don't know of a place either. But I'd come at this from another angle: Rather than comparing two amps that might have wildly different characteristics that interact in ways we could hardly guess, why not use the same amp, speaker, etc, and then contaminate a source material with a single type of distortion and establish a threshold for that?

But I thought your claim was that an amplifier with ALL distortions below your threshold of hearing is what you choose? I just wondered how you determined this "ALL distortions" profile for an amplifier?
 

TNT

Member
Joined 2003
Paid Member
I went back and reread the test methodology and to me it was clear. The reference file is compared to an AD/DAC loop the frequency response of the loop is part of the diff. There is no indication at all of an explicit attempt to remove all the "linear" differences and show only the THD/IMD and other added distortions. I saw mention of timing and overall gain, in fact the gain correction for the whole file is given in the results. As for timing it would make sense to align the beginning and end and do a SRC to make them meet.

I would say beginning only.

//
 
Rather a long shot to assume that all distortions are within the passband of the source. In my experience many SQ differences are due to out-of-band signals getting into the audio so the 'distortion' couldn't be included in a digital stream it would need application at the analog output of the DAC.

And from mmerrill99

But I thought your claim was that an amplifier with ALL distortions below your threshold of hearing is what you choose? I just wondered how you determined this "ALL distortions" profile for an amplifier?

Yes, as a person that believes in measurements and values transparency, it's pretty easy for me to pick an amp. I'm trying to understand the person that relies more on listening than specs and provide them a framework for demonstrating their unique skill. That framework is that you start with a system they know and trust, introduce gross but specific distortions (eg amplitude, zero-crossing, etc) in the source material, gradually lessening those distortions until they can no longer detect them via ABX.

It is the folks with golden ears that can hear things I cannot that intrigue me. Which is why I offered to paypal anyone $50 to anyone that could demonstrate the ability to detect a 0.8 dB amplitude imbalance on their favorite system. I really am curious how their ears operate relative to mine.
 
I really am curious how their ears operate relative to mine.

It probably has little to nothing to do with how their ear operate. It probably mostly has to do with some brain wetware DSP or whatever you want to call it. Apparently, groups of neurons are able to phase lock with amplitude or frequency variations, or with other patterns. How it works still currently being studied, but its pretty clear that it doesn't work like our engineered systems we are accustomed to thinking about.
 
And from mmerrill99

Yes, as a person that believes in measurements and values transparency, it's pretty easy for me to pick an amp.
But you previously admitted that there is no full set of measurement that characterises an amp so do you just believe in this delimited set of measurements despite knowing that they don't characterise the performance fo the device?
 
And from mmerrill99

It is the folks with golden ears that can hear things I cannot that intrigue me. Which is why I offered to paypal anyone $50 to anyone that could demonstrate the ability to detect a 0.8 dB amplitude imbalance on their favorite system. I really am curious how their ears operate relative to mine.

According to Jonboc, he has something like 3 listeners who have differentiated 0.5dB or was it 0.2dB - can't remember the claim now but it seems you owe him $150 if he can present the evidence?
 
It probably has little to nothing to do with how their ear operate. It probably mostly has to do with some brain wetware DSP or whatever you want to call it. Apparently, groups of neurons are able to phase lock with amplitude or frequency variations, or with other patterns. How it works still currently being studied, but its pretty clear that it doesn't work like our engineered systems we are accustomed to thinking about.

Indeed, lots of people seem to assume that the ears are the most important aspect of this but as you say, it's the brain that is doing the heavy lifting & mostly determines what we hear. Unfortunately, engineering measurements are not cognisant of this aspect of audibility.
 
That's what it looked like the GS tests did, I wonder how badly mismatched two clocks could be on the better converters? Even then the SRC process could have plenty of artifacts of it's own.

Way down on page 17 of this presentation is diagram of how some of the clocking works in some of the newer DACs.
http://www.esstech.com/files/4614/4095/4305/about-jitter.pdf

Looks like even if synced to a master clock there is no guarantee sample points will be exactly matched in time, or what the processing latency is.

Also, Benchmark Media remarks that when their DAC clocks are switched to a new source, there is about a 250ms settling time in which frequency shifting of the analog output occurs.
 
Last edited:
What established threshold? According to Earl Geddes who published some of the research on what people can hear, published research used fairly small sample sizes and probably is about right for 95% of the population. Having done the research, maybe he is more optimistic about how accurate it is than I am. Even if he is spot on, that still leaves 5% of the population that would be expected to fall outside the "limit." That's not such a small number either. Out of all the people here, what percentage do you suppose are in the top 5% of IQ tests? Likely many more than 5% of DIYaudio users, I would say based on the complexities of engineering. Maybe some of them are also well up into the top 5% of listeners in terms of distortion perception. How would you know one way or the other? One can't know exactly because it hasn't been measured. What would you estimate?
Let's assume that the top 5% are indeed over represented on this forum. It's probably over optimistic considering the age distribution shown by the recent poll in the lounge but ok.

How do you expect those 5% to be distributed in performance ? It is not an unreasonable guess that most of those 5% are in fact doing only a bit better than the 95% (as for the IQ bell curve). One can hardly fault QAMAtt for thinking that 20dB is a reasonable buffer to include most of the outliers and that the few completely of the charts individuals are only of "academic" interest.

But you previously admitted that there is no full set of measurement that characterises an amp so do you just believe in this delimited set of measurements despite knowing that they don't characterise the performance fo the device?

:yikes:

First off, QAMAtt never admitted that "there is no full set of measurement that characterises an amp". He only admitted that he didn't know of a place "where we can find all possible distortions reported for any device".

Secondly, even if we accept that the currently available measurements might eventually be completed in the future, it wouldn't mean that "they don't characterise the performance of the device." They do it quite well on the contrary. Otherwise the GedLee metric couldn't have a good correlation to audibility. We should not confuse measurements and metrics. The failings of a single THD number kind of approach is not a failure of measurements but of the way we interpret them.

And lastly: Your logical fallacy is loaded question
 
Let's assume that the top 5% are indeed over represented on this forum. It's probably over optimistic considering the age distribution shown by the recent poll in the lounge but ok.

It could be that age actually works in the opposite direction, at least with ability to learn listening skills. Hearing loss tends to occur with increasing age, but as Earl Geddes pointed out much of that can be compensated for by turning up the volume some.

Even in my case, where I can't hear over 10kHz at any practical volume level and some tinnitus is also present, some frequencies still get through. Of those frequencies, my brain DSP still seems to work pretty well. No doubt if I live long enough that will go too, but for now it still works better than my ears.

The other thing regarding age, I ran across some research recently showing there are certain age ranges where brains are more adept at developing musical recognition skills, such as pitch perception. There appears to be a late peak that occurs somewhere in the range from about 60 - 70 years old. Interestingly, I am finding I am getting better at some of those things than I ever was in earlier in life. Didn't have any idea of why until I can across that.

The other big factor besides demographics is training. Some people may have a natural affinity for distortion perception, such as produced by DACs, but training can improve perception perhaps similarly to how learning a new spoken language can turn something like meaningless tiny pitch inflections, or whatever (tongue clicking, grunts, etc. in some languages), such as in some Chinese languages and dialects, into expressions with very different meanings. At first it all sounds the same, nothing to hear but some noises. With practice the brain adapts to recognize subtleties.

Of course, there is a question as to whether it may be advisable to practice hearing distortion and other such things aside from music. JBL certainly uses trained listeners to evaluate their products. I know some of the designers here in the forum rely a lot on listening tests in addition to measurements although they mostly tend not to say much about it, likely so as to avoid pointless arguments.

Also, when we talk about what is or is not "audible" it depends on what we mean by that. If audible means what the most talented, most trained listener in the world can hear on their best day, that's one thing. If we mean what 95% of the population can usually hear under normal listening conditions, that's a very different thing.

Obviously, one can't take four untrained listeners, have each each one of them represent 25% of the world's population, and by having all four of them fail to hear something, conclude that that something is inaudible using either of the above definitions.
 
Last edited:
First off, QAMAtt never admitted that "there is no full set of measurement that characterises an amp". He only admitted that he didn't know of a place "where we can find all possible distortions reported for any device".
ok then he should be able to put together the complete set of measurements that characterise an amplifier -doesn't have to be all from the same place although the measurement conditions do need to be equivalent to have something that can be combined in such a way. Anyway, let's see these full set of measurements.
Secondly, even if we accept that the currently available measurements might eventually be completed in the future, it wouldn't mean that "they don't characterise the performance of the device." They do it quite well on the contrary. Otherwise the GedLee metric couldn't have a good correlation to audibility. We should not confuse measurements and metrics. The failings of a single THD number kind of approach is not a failure of measurements but of the way we interpret them.
huh?
 
It is the folks with golden ears that can hear things I cannot that intrigue me. Which is why I offered to paypal anyone $50 to anyone that could demonstrate the ability to detect a 0.8 dB amplitude imbalance on their favorite system. I really am curious how their ears operate relative to mine.

I'm lost here - how would this 'golden eared' person's passing your $50 test reveal how their ears operate? Assuming they pass, it'll tell you merely that they do operate - what am I missing?
 
It probably has little to nothing to do with how their ear operate.

Yes, when I mention the car is sluggish when stepping on the accelerator, the mechanic and myself both understand there is a complicated system at work. "The accelerator" is a proxy for a complex system, involving computers, linkages,sensors, transducers, etc. I'm not suggesting to the mechanic that the pedal itself is the culprit.

Let's use "your ears" as a proxy for a very complicated system, OK?

mmerrill99 wrote:

According to Jonboc, he has something like 3 listeners who have differentiated 0.5dB or was it 0.2dB - can't remember the claim now but it seems you owe him $150 if he can present the evidence?

Close. In a previous post on this thread I linked to an app I wrote (with a link to youtube video) that that will let you degrade a WAV file with a range of distortions (zero crossing, amplitude, quantization, etc). And it will cryptographically sign your results so they can be verified. In the post I think I offered $50 to the first to detect 0.8 dB with the app, but I'd happily pay $50 for the first person to hit the 0.8 dB threshold, and $100 to the first person to cross the 0.5 dB threshold, and $200 to the first person to cross the 0.2 dB threshold (payment via paypal only). We just need to agree on source music, as certain types of source music would make it easy to use test equipment to cheat. But yeah, if you can sit in your listening room with your favorite equipment and get it right, say, 9 out of 10 times (95% confidence interval) that can be verified, I'd happily pay the first folks to prove it. And I think a lot of us would have their world view re-aligned.

And we could repeat the process (including $) for quantization, zero-crossing, slew rate, TANH, clipping distortion, etc
 
It might take some practice for somebody to get good at testing the way you want and with the reliability you want. Perhaps more than a few hours involving very demanding concentration. You want somebody to do that for only $50? Seems to me there is no incentive at all to do it for money, and no incentive to do it for any other reason. If it were effortless it might be another matter, but it probably wouldn't be. If you want some takers, you might try offering $5,000. No doubt some people would seriously try.

Another thing to consider might be if you get people motivated enough to try to provide results you specify, it would probably increase motivation for cheating. Don't know how your program works, but Foobar ABX can be beaten.
 
Last edited:
We have discussed over the years (in this thread too) the differences in testing for general results for populations and audibility to (at least) one listeners.

it is good tradition to look up what was already found in the past (and mabye try to replicate) and it was already reported by Clark (~198x) that in their ABX tests the limit for broadband level differences between music samples
was a bit below 0.3 dB.

Frindle wrote in his (linked earlier) publication:
"The single most important factor required to get a valid test is to set the relative levels between direct and the system under test very accurately. A difference of 0.1dB will be readily heard, but it will not necessarily manifest itself as level change. We have shown that with differences in level below 0.2dB, we subconsciously search for the cause. Any listener will tell you there is a difference but each will interpret it as a different artefact. These seem to change with attention and come up as anything ranging from frequency response difference to stereo image shifting."

and furthermore:

"As a point of interest, no listener ever believes that it is only a level change. Therefore it takes much self discipline for even the most experienced listener to make himself check levels if effects occur."

Frindle listed some of the differences they were able to detect (using ABX tests) in their system:
Paul Frindle, Are We Measuring the Right Things? Artefact Audibility Versus Measurement, UK 12th Conference: The Measure of Audio (MOA) (April 1997), paper no. MOA-05, 45 .
 

Attachments

  • Frindle1997_detected.jpg
    Frindle1997_detected.jpg
    96.6 KB · Views: 147
Last edited:
QAMAtt: I think you should consider awarding the $50 to Jakob2 who has provided the proof you want and much more. It is in the form of published research using ABX protocol rather than your program, but its may be just as reliable or better. Again, Foobar ABX can be cheated, don't know about your program. On the other hand, Frindle is serious engineer and researcher at Sony Oxford who has been doing this stuff for a long time.
 
Last edited:
Status
This old topic is closed. If you want to reopen this topic, contact a moderator using the "Report Post" button.