Originally, there were four people who couldn't hear any difference. It can happen. Four people is a very small sample to represent all the billions of people in the world. There are other people who can hear some difference between DACs. It turns out whether or not someone hears a difference or not has little to do with ears and much more to do with how some of the aural processing in their brains happens to be wired. For people who do hear some difference, it can be nice to have somebody explain that a particular sound effect is caused by jitter. That's all.
First off it looks like a great deal of effort went into designing, conducting and reporting the tests and I applaud the effort. That results showed no difference is an important indication of how careful one must be in designing a test so that results indicating the sensitivity of test are also reported. In the future such tests could, if administered with greater thoroughness might help build a more comprehensive understanding of the various necessary test controls, sensitivity, and documentation for reproduce-ability and improvement.
From the comfort of an armchair here are my suggestions:
1. determine the background noise and other ambient factors of the test environment (noise level, power spectrum, ambient temp, humidity, altitude, light level, light temp, chair type, (who knows what factors influence perception?)
2. determine the acoustic path, impulse response, for each speaker to the listening position.
3. As part of the listening test have prepared the same audio with various impairments to various degree (Noise, harmonic distortion, inter mod distortion) so that the ABX test is applied between the various impairments and DUTs. Something similar is done for double blind vocoder testing where the Harvard sentences are encoded, and the identical set is distorted with additive noise at various levels and listeners indicate perceived quality between the original source, original source + calibrated noise level, and encoded source.
Conducting such tests is a huge effort! However the results would allow you to state exactly what level of SNR, THD, TIM your listeners could detect in the test environment. For example you found nobody could discern >1%THD from any DAC, but could discern >2% THD with 90% confidence with DAC 1 but only >4%THD from DAC 2 you just might be able to say DAC 1 is better than DAC 2. However if nobody could discern < 5% it might be that the test conditions were insufficiently sensitive or your human sample was.
From the comfort of an armchair here are my suggestions:
1. determine the background noise and other ambient factors of the test environment (noise level, power spectrum, ambient temp, humidity, altitude, light level, light temp, chair type, (who knows what factors influence perception?)
2. determine the acoustic path, impulse response, for each speaker to the listening position.
3. As part of the listening test have prepared the same audio with various impairments to various degree (Noise, harmonic distortion, inter mod distortion) so that the ABX test is applied between the various impairments and DUTs. Something similar is done for double blind vocoder testing where the Harvard sentences are encoded, and the identical set is distorted with additive noise at various levels and listeners indicate perceived quality between the original source, original source + calibrated noise level, and encoded source.
Conducting such tests is a huge effort! However the results would allow you to state exactly what level of SNR, THD, TIM your listeners could detect in the test environment. For example you found nobody could discern >1%THD from any DAC, but could discern >2% THD with 90% confidence with DAC 1 but only >4%THD from DAC 2 you just might be able to say DAC 1 is better than DAC 2. However if nobody could discern < 5% it might be that the test conditions were insufficiently sensitive or your human sample was.
Sorry, I got very busy over the holidays.
My issue isn't that Clark and Frindle were lacking enormous details. My issue was that you portrayed them, IMO, as a bit more definitive. And then upon reading the actual papers, there was no meat there.
Which goes back to my original point: The science here is poor (something you didn't really refute).
But peer review wouldn't accept a statement such as Clark's without supporting evidence. Clark's statement, when he made it, wasn't widely known and didn't cover the methodology. And it's still suspect today, nor has it be replicated. Thus, how did it get through peer review? Peer review, by design, would reject new and novel statements of fact without supporting data.
Much old auditory research is suspect, at least in some ways. Almost none of it provides sufficient information for accurate replication. Most of it was also done with small numbers of subjects and apparently no, or close to no, efforts were made to identify particularly talented/skilled listeners.
For example, if you test 200 hundred people, perhaps more than some aural research, suppose you have one person in the group that is 1 in 1000 or 1 in 10,000. Are you going to discard that person's results as an outlier or investigate? If you are interested in average people, the decision is easy.
Also, if one were to say that people that are in the top .1% or greater are so rare as to be negligible, that't not necessarily so in a audio forum. One reason some people may be attracted to an interest in audio is that they are very acute listeners. We just don't know. There is a lot that could be and should be studied, but no funding for it so it doesn't happen.
For example, if you test 200 hundred people, perhaps more than some aural research, suppose you have one person in the group that is 1 in 1000 or 1 in 10,000. Are you going to discard that person's results as an outlier or investigate? If you are interested in average people, the decision is easy.
Also, if one were to say that people that are in the top .1% or greater are so rare as to be negligible, that't not necessarily so in a audio forum. One reason some people may be attracted to an interest in audio is that they are very acute listeners. We just don't know. There is a lot that could be and should be studied, but no funding for it so it doesn't happen.
Last edited:
The part I like the most is how a great number of audiophiles will sit around and bloviate about a whole host of apparently obvious sonic attributes, but when you cover their eyes and the magic abilities vanish, suddenly, the evidence requirement shoots up by 10 orders of magnitude, and we end up 150 pages deep, chasing white papers.
The ABX isn't meant to determine if 2 DUTs are the same, and it's not supposed to, it's only meant to determine if the guy who claims his thing has silky smooth treble can even point to it when he can't see that it's in use.
It doesn't need to be "the same", it only needs to show if an audible impression remains once bias is subtracted, and if the original claim doesn't need to stand up to scrutiny, neither does the test.
If we put as much effort into doing listening tests as some people seem to want to spend trying to tear them apart, or gaslight other people with language technicalities, we'd all be a lot smarter.
The ABX isn't meant to determine if 2 DUTs are the same, and it's not supposed to, it's only meant to determine if the guy who claims his thing has silky smooth treble can even point to it when he can't see that it's in use.
It doesn't need to be "the same", it only needs to show if an audible impression remains once bias is subtracted, and if the original claim doesn't need to stand up to scrutiny, neither does the test.
If we put as much effort into doing listening tests as some people seem to want to spend trying to tear them apart, or gaslight other people with language technicalities, we'd all be a lot smarter.
Perhaps helpful to avoid trouble with the moderators if one reviews the rules from time to time.
THE RULES
NOT ALLOWED:
Disruptive behavior of any sort, including offensive language, trolling, threadjacking, insults, intimidation, harassment or other disrespectful or antisocial behavior. (Notes 1 & 3)
I had to Google "bloviate". I think it is a reasonable description of what happens part of the time on most forums.
To suggest that golden ears often lose their powers when decoupled from golden eyes is hardly insulting, but merely a statement of experimental fact. People are still arguing about what is the mechanism for this loss of power, so the loss of power is not really disputed. The favoured explanation among golden ears themselves is 'test stress' or just 'bad tests'.
Those of us lacking golden ears do not wave the forum rules in the air when we are accused of being deaf, stupid or poor.
To suggest that golden ears often lose their powers when decoupled from golden eyes is hardly insulting, but merely a statement of experimental fact. People are still arguing about what is the mechanism for this loss of power, so the loss of power is not really disputed. The favoured explanation among golden ears themselves is 'test stress' or just 'bad tests'.
Those of us lacking golden ears do not wave the forum rules in the air when we are accused of being deaf, stupid or poor.
lol, sorry, I only learned that word myself a couple of years ago, and the only reason I remembered it is because of how appropriate it is to describe some of the things that go on in audio circles.
DF96, There are many ways to state facts. Leaving out adjectives that rudely express negative opinions of others persons motivations, thinking abilities, writing skills, etc., is perhaps the easiest to implement.
Also, there is a possibility the some of what the "golden ears" claim is factually true. To summarily dismiss all of it equally as inflated or empty (bloviated) is inaccurate. To describe some people's reports of listening experiences as "magic" is a way of saying they are fools, idiots, and hallucinating self-delusional morons. That's a way of referring to people believed to be stupid and below one's own self.
In addition, if you or anyone else is accused of being deaf, stupid, or poor, I am on your side that such verbiage is inappropriate, and not conducive to civil conversation.
What tends to happen when people slip into insulting each other, is that escalation tends to occur and pretty soon people are insulting each other's mothers. Then posts get deleted and threads closed.
Rather than respond to DrDyna in that way I am choosing to try to keep it civil.
Also, there is a possibility the some of what the "golden ears" claim is factually true. To summarily dismiss all of it equally as inflated or empty (bloviated) is inaccurate. To describe some people's reports of listening experiences as "magic" is a way of saying they are fools, idiots, and hallucinating self-delusional morons. That's a way of referring to people believed to be stupid and below one's own self.
In addition, if you or anyone else is accused of being deaf, stupid, or poor, I am on your side that such verbiage is inappropriate, and not conducive to civil conversation.
What tends to happen when people slip into insulting each other, is that escalation tends to occur and pretty soon people are insulting each other's mothers. Then posts get deleted and threads closed.
Rather than respond to DrDyna in that way I am choosing to try to keep it civil.
Would it help if I said that I consider myself an audiophile?
Some of this stuff I've been conditioned to find strange, like, nobody would have a problem with judging televisions while using ear plugs, but when audio guys are around, you mention a blindfold, and it's like a 300 ton freight train of people trying to chip and whittle away at it because they don't like the results.
Some of this stuff I've been conditioned to find strange, like, nobody would have a problem with judging televisions while using ear plugs, but when audio guys are around, you mention a blindfold, and it's like a 300 ton freight train of people trying to chip and whittle away at it because they don't like the results.
Regarding who DrDyna was referring to in his post, he described some people as "chasing after white papers." In this thread it was Jackob2 and myself that posted some of the references to studies and other materials for the purpose of thoughtful and scientifically oriented discussion. Therefore, since DrDyna hasn't said he wasn't referring to me and people like me specifically, I can only infer that he intended us to be included in his description.
Last edited:
You have to make it inaccurate with evidence, and the fact that this idea gets so much resistance is testimony to pretty much every other negative connotation that you've tried to loosely infer from my post.
I didn't try to insult anyone, you've just strawmanned it into that, because it's easier to dismiss it than it is to address it.
I disagree. It's is nothing like a 300 ton freight train. I would agree with you that some people who consider themselves audiophiles are probably mistaken in some of their listening impressions. I would further agree that blind testing of some type is needed. However, I don't believe the currently available ABX test systems are designed to accurately measure perception of very small differences in sound quality. I have stated at other times a few fairly simple changes to the systems that I suspect would remedy the problems, but no makers of ABX test systems seem interested in trying it.
You are making an accusation as to my motivation that is untrue and that you have no knowledge of.
- Status
- This old topic is closed. If you want to reopen this topic, contact a moderator using the "Report Post" button.