Blind DAC Public Listening Test Results

Status
This old topic is closed. If you want to reopen this topic, contact a moderator using the "Report Post" button.
I know this seems crazy, but... there are people who have beliefs but are actually curious about reality and want to find out what's real and what's illusion.

I can't figure out why that's crazy at all. It was my own experience when I was actually working in the industry. I believed DACs sounded different, but I couldn't for the life of me hear the difference. I really wanted to, but I couldn't. So I can't believe you're meaning this 'crazy' thing seriously at all - is it just another of your trolls?
 
Actually that is the definitin of SCIENCE - the curiosity to test new theories.

Actually you must be meaning the curiosity to try to falsify one's hypotheses. Which is what I do when working on a DAC design - formulate something like 'this sibilance effect I hear is due to this decoupling capacitor'. So I change the cap, and listen out for if the sibilance changes with that one change.

There are faith-based pseudo-scientists, but they just hijacked the name of what they belive in.

Sure, been there, seen that. In fact I see it all the time on this forum - primarily amongst those who are 'objectivist' in approach. RS is one such and I've called him out on it before.
 
For me I already hear differences - that does demonstrate that there are differences for me. I'm not particularly interested in demonstrating those differences to others - where's the benefit?

I believed DACs sounded different, but I couldn't for the life of me hear the difference. I really wanted to, but I couldn't. So I can't believe you're meaning this 'crazy' thing seriously at all - is it just another of your trolls?

Res ipsa loquitur.
 
Well, he probably wants you to decide for yourself: could or could not hear differences :D
Which is what I do when working on a DAC design - formulate something like 'this sibilance effect I hear is due to this decoupling capacitor'. So I change the cap, and listen out for if the sibilance changes with that one change.
See now you are into "belive" mode because you formulated the hypothesis and you are testing it. You need either peer review (which can be biased too) or MEASUREMENTS to back up your finding. You can devise a tool/procedure to find the difference that you think you hear and then use the tool.
 
Last edited:
For me I already hear differences - that does demonstrate that there are differences for me.

Plenty of people "hear" differences between different pieces of equipment. Sometimes this is because the equipment actually has audible differences. Other times, the equipment has identical audio qualities, but other influences cause people to "hear" differences that do not actually exist - for example, comparing at imperfectly matched volumes, the placebo effect (i.e. bias and altered perceptions based on positive/negative expectations), etc. The only way to reliably determine whether you're "hearing" actual differences vs. differences that do not actually exist, is to perform a blinded, controlled listening test.

I'm not particularly interested in demonstrating those differences to others - where's the benefit? They can (and sometimes do) think up other reasons for people getting statistically improbable results.

Well, let's think for a minute. If you are absolutely positive that 2 different DAC's sound completely different to you, and you are absolutely sure that these differences you hear are not being unduly influenced by knowing the identity of the DAC you're listening to, then you should have absolutely no problem identifying which is which when you're blinded, right?

Correct - RS is one individual who holds such a belief - as I've already pointed out,

I don't think you can say that necessarily. In one of RS's earlier blog "experiments," he tried to show that two DAC/headphone-amp's were actually different. He used both measurements and blind listening tests to try to demonstrate this difference (although I do not agree that his methodology and analysis were rigorous enough to draw any conclusions).

there's no incentive for him to design a well controlled experiment when its more likely to produce a result contrary to his existing beliefs.

I'm under the impression that RS would be interested in designing an experiment that will would test his hypothesis, but he could use some assistance in this department. Scientists conduct experiments all the time where the results are different from what they expect - that's the process of science, called the "scientific method."

No, that's a misunderstanding. What use are beliefs when you have experience? If I have a belief I'm 1.88m tall how will that help me if I'm in fact 1.6m ? Or if my belief about my height does accurately reflect my measured height is it a useful belief?

You're confusing three different terms: belief, experience, and empirical data (or results derived from an experiment). To borrow your example, you might "believe" you are 1.88m tall. Your belief may come from "experience" (for instance, you have used a non-calibrated measuring tape and looked in the mirror in order to come up with the figure of 1.88m. You may secretly wish that you are taller, so you just estimated that the top of your head was at the 1.88 meter mark). However, the FACT that you are 1.6m would be determined by an unbiased, controlled measurement. That is, using a calibrated measuring device, having an unbiased person do the measuring (so you can't "round up" "accidentally"), repeating the measurement several times - in which case the measurement may come up as 1.6m. In this case, someone's belief, which came from experience (and not a controlled, unbiased experiment), was actually an incorrect belief. And this can easily happen to people in the audio world as well when they "believe" there are differences between piece of audio equipment based on their "experience" rather than on controlled, unbiased listening tests.

I can tell that process of science and the scientific method is somewhat foreign to you, whereas you seem like a very knowledgeable engineer - but you might want to know that this is how much of our understanding of the natural world, medicine, and "science" is derived."
 
Plenty of people "hear" differences between different pieces of equipment. Sometimes this is because the equipment actually has audible differences. Other times, the equipment has identical audio qualities, but other influences cause people to "hear" differences that do not actually exist - for example, comparing at imperfectly matched volumes, the placebo effect (i.e. bias and altered perceptions based on positive/negative expectations), etc.

Whilst I'm not clear what you mean by 'identical audio qualities' I do acknowledge the placebo effect, yes. In many cases what people are hearing is placebo - this is my supposition.

The only way to reliably determine whether you're "hearing" actual differences vs. differences that do not actually exist, is to perform a blinded, controlled listening test.

People do actually hear things when placebo effects are occurring. So words like 'actual' or 'actually exist' are moot unless you're going to give us a philosophical treatise on their precise definitions?

Well, let's think for a minute. If you are absolutely positive that 2 different DAC's sound completely different to you, and you are absolutely sure that these differences you hear are not being unduly influenced by knowing the identity of the DAC you're listening to, then you should have absolutely no problem identifying which is which when you're blinded, right?

That would be my hypothesis, yes. However I'm not really interested so much in 'differences' as 'preferences'. I'm seeking the most enjoyable DAC, not the one that sounds the most different from others. Incidentally I'm not absolutely sure that my listening isn't being influenced by being sighted. How could anyone be absolutely sure of that?

I don't think you can say that necessarily.

Cite the particular quote of mine that I can't say, necessarily, then. When we have specific quotes we can discuss them, I'm not interested in abstract discussions.

I'm under the impression that RS would be interested in designing an experiment that will would test his hypothesis, but he could use some assistance in this department.

Let's wait for him to ask then. Presumably since he's not asking he himself does not perceive the need for assistance.

Scientists conduct experiments all the time where the results are different from what they expect - that's the process of science, called the "scientific method."

But RS is only a scientist in the realm of audible differences between DACs in name, not in action. In terms of behaviour (which to me is way more important than how people present or self-identify) he's religious, not scientific. By religious I mean he bases some of his behaviour on non-falsifiable assertions. So his name 'RS' is rather a sweet irony :D

You're confusing three different terms: belief, experience, and empirical data (or results derived from an experiment). To borrow your example, you might "believe" you are 1.88m tall. Your belief may come from "experience" (for instance, you have used a non-calibrated measuring tape and looked in the mirror in order to come up with the figure of 1.88m.

I can't figure why the word 'experience' would turn up in scare quotes here. If I had the experience of measuring myself with a dodgy tape measure and didn't care about parallax I could still observe my measurement (even though its erroneous).

You may secretly wish that you are taller, so you just estimated that the top of your head was at the 1.88 meter mark). However, the FACT that you are 1.6m would be determined by an unbiased, controlled measurement.

I don't see how 'FACT's or wishes are relevant here?

That is, using a calibrated measuring device, having an unbiased person do the measuring (so you can't "round up" "accidentally"), repeating the measurement several times - in which case the measurement may come up as 1.6m. In this case, someone's belief, which came from experience (and not a controlled, unbiased experiment), was actually an incorrect belief. And this can easily happen to people in the audio world as well when they "believe" there are differences between piece of audio equipment based on their "experience" rather than on controlled, unbiased listening tests.

You said I was confusing things before. I wasn't but I most certainly am confused now :)

I can tell that process of science and the scientific method is somewhat foreign to you

Since we're discussing science, I'm curious - let's be having some in action. Which of my words on this thread have led you to this? Please show them and your reasoning from them. In other words - evidence please for your assertion.
 
People do actually hear things when placebo effects are occurring. So words like 'actual' or 'actually exist' are moot unless you're going to give us a philosophical treatise on their precise definitions?

Hearing involves the transduction of sound to neural impulses, which is then perceived and communicated by the brain. But yes, you can "hear" sounds that do not "actually exist." For example, a person with psychiatric disease and auditory hallucinations can perceive and report sounds/voices/etc. that are not actually present. I'm really not sure where this request for a "philosophical treatise" comes into play.

That would be my hypothesis, yes. However I'm not really interested so much in 'differences' as 'preferences'. I'm seeking the most enjoyable DAC, not the one that sounds the most different from others.

Having DAC "preferences" implies that there are "differences." For someone who believes that DAC's sound different, it is actually much easier to experimentally demonstrate "differences" than it is to show "preferences" (the latter requiring an individual to consistently pick the better sounding DAC, rather than simply identifying when they sound different." So a study method testing for same/different actually favors your hypothesis that some DAC's sound better than others.

Incidentally I'm not absolutely sure that my listening isn't being influenced by being sighted. How could anyone be absolutely sure of that?

Simple! You just design a listening test where the listener cannot be influenced by "sight" (i.e. you blind the listener to which DAC he is listening to). When you ask why I'm under the impression that you are not familiar with the scientific method, this is a perfect example of why I think that. Nobody with basic knowledge of experimental design would even ask the question you did because the answer is obvious.

You also seemed to have lots of trouble understanding why listeners who strongly believe that DAC's sound different (as opposed to ones who don't think there's a difference) would be the type of motivated listener group that would be better suited to correctly identifying differences (should they exist) in a blinded same/different test. In fact, the more adamant they are about "big differences," the better!

Smellygas: "I can tell that process of science and the scientific method is somewhat foreign to you"
Since we're discussing science, I'm curious - let's be having some in action. Which of my words on this thread have led you to this? Please show them and your reasoning from them. In other words - evidence please for your assertion.

See above. This is kind of the reason I really hate getting into discussions about blind/controlled listening tests on audio forums. It's because there's virtually no overlap between the disciplines of research science and engineering (which is an *applied* science). Almost nobody here understands the scientific method, the concept of designing an experiment to test a hypothesis, and doing a proper analysis. If any of this got copied/pasted onto a research science forum, I bet everyone would be slapping their foreheads and banging their heads on the desk, it's so ridiculous. This is why there's so much audio mythology out there.

You know what I think? I think some DAC's have audible differences. For instance, an NOS DAC with -4dB at 20kHz rolloff, yeah, that's going to sound different from one that is flat to 20kHz. However, in cases where 2 DAC's have almost ruler-flat responses, vanishingly low distortion, excellent linearity, and noise/DR that is sufficiently low, I honestly do not think the differences are as great as people claim they are. I also do not think you'll be able to hear such differences without very high end equipment, particular really good speakers. When I had $300 loudspeakers, I bet I could not differentiate between a Benchmark and a made-in-china-clone-ebay DAC. But with B&W 800-series speakers in my living room and Stax Omega II electrostatic headphones upstairs, I THINK I can hear differences between my Benchmark and other DAC's after doing level-matched non-blinded comparisons.

But sadly, I don't think anyone will ever be able to quantify the degree of differences between DAC's. Engineers are good at engineering, but not at conducting research science. And even when there are audio enthusiasts that
are capable of properly conducting, analyzing, and publishing a study that clearly demonstrates that listeners can/can't identify differences between equipment reliably and to what degree of consistency, the problem is the audio community at large is wholly incapable of understanding the scientific process, a controlled experiment, a simple statistical analysis, or why the results actually demonstrate what they do! So it's useless!


So, I give up. Good luck, happy listening.
 
This is kind of the reason I really hate getting into discussions about blind/controlled listening tests on audio forums. It's because there's virtually no overlap between the disciplines of research science and engineering (which is an *applied* science). Almost nobody here understands the scientific method, the concept of designing an experiment to test a hypothesis, and doing a proper analysis.

Strongly disagree. Yes, there are the usual nuts, trolls, egomaniacs, and salesmen, but there are some first-rate scientists here as well. And many who aren't, but get the ideas intuitively.
 
jesus that was a rant and a half

scientific method is a great way to examine and describe our reality repeatedly, but it still comes down to belief that the science is meaningful to the individual; particularly some of the methods that involve abstraction

I agree with the above, there are some very serious and brilliant scientists on this forum, just as there are people of the more fruity kind and many in between.
 
Last edited:
Hearing involves the transduction of sound to neural impulses, which is then perceived and communicated by the brain.

In going into such detail it really helps to get the terminology accurate. Firstly there is no sound except as a result of the auditory perceptual process. So vibrations are transduced into neural impulses, not sounds.

But yes, you can "hear" sounds that do not "actually exist."

Again, confused. No sounds actually exist - what exist are vibrations. Sounds are all in the mind.

Simple! You just design a listening test where the listener cannot be influenced by "sight" (i.e. you blind the listener to which DAC he is listening to). When you ask why I'm under the impression that you are not familiar with the scientific method, this is a perfect example of why I think that. Nobody with basic knowledge of experimental design would even ask the question you did because the answer is obvious.

Ah I see - its merely your opinion that I'm not familiar with the basis knowledge of science. There I was, for a moment thinking you were doing science! Here we were talking about 'absolute certainty' - but you're describing a blind listening test where the results are statistical. The result could be overwhelmingly probable with enough trials, but that's not absolute certainty.

You also seemed to have lots of trouble understanding why listeners who strongly believe that DAC's sound different (as opposed to ones who don't think there's a difference) would be the type of motivated listener group that would be better suited to correctly identifying differences (should they exist) in a blinded same/different test. In fact, the more adamant they are about "big differences," the better!

No the trouble in understanding is your own - I've said that such people may well want to take part in tests like that.

You know what I think? I think some DAC's have audible differences. For instance, an NOS DAC with -4dB at 20kHz rolloff, yeah, that's going to sound different from one that is flat to 20kHz.

Have you done an experiment to test your hypothesis?

However, in cases where 2 DAC's have almost ruler-flat responses, vanishingly low distortion, excellent linearity, and noise/DR that is sufficiently low, I honestly do not think the differences are as great as people claim they are.

Plenty of weasel words in there - by which I mean non-falsifiable ones. Like 'almost', 'vanishingly', 'excellent', 'sufficiently' and 'as great as'. If you're going to do science you'll have to learn to make falsifiable hypotheses.

I also do not think you'll be able to hear such differences without very high end equipment, particular really good speakers. When I had $300 loudspeakers, I bet I could not differentiate between a Benchmark and a made-in-china-clone-ebay DAC. But with B&W 800-series speakers in my living room and Stax Omega II electrostatic headphones upstairs, I THINK I can hear differences between my Benchmark and other DAC's after doing level-matched non-blinded comparisons.

Again a weasel word - 'good'. So given that it seems that B&W speakers are 'good' what criteria do they satisfy to earn this accolade?

So, I give up. Good luck, happy listening.

I hope you had as much fun in this discussion as I did :)
 
Ah I see - its merely your opinion that I'm not familiar with the basis knowledge of science. There I was, for a moment thinking you were doing science! Here we were talking about 'absolute certainty' - but you're describing a blind listening test where the results are statistical. The result could be overwhelmingly probable with enough trials, but that's not absolute certainty.

This is another example of how you are clearly unfamiliar with the science and the scientific method, and so much time is wasted explaining things that you really should have learned in a high school biology class. When scientific studies are performed, it's almost never possible to have "absolute certainty." But just like jurors are held to the "beyond a reasonable doubt" (not "absolute certainty") test, we can use statistics to determine how likely it is that an experiment found a difference that didn't truly exist or that it failed to show a difference that did indeed exist. And if the likelihood of both is very small (we usually arbitrarily pick 1% or 5%), then we tend to accept the findings as likely to be true. And when others repeat the experiments independently and find similar conclusions, then we become even more confident. The fact that you start implying that an experiment cannot prove anything with "absolute certainty" and is therefore useless really does indicate your disconnection from science.

I'm sure you're a great engineer, but it's very clear that you do not have the background or capacity to understand the implications of a simple scientific experiment, its analysis, and conclusions that can be drawn.
 
This is another example of how you are clearly unfamiliar with the science and the scientific method, and so much time is wasted explaining things that you really should have learned in a high school biology class. When scientific studies are performed, it's almost never possible to have "absolute certainty."

Yeah I'm aware of that. So it looks like you're tilting at windmills, and indeed wasting your time. The words you used were 'absolutely sure' - so given that we're talking about science why did you choose those words?

The fact that you start implying that an experiment cannot prove anything with "absolute certainty" and is therefore useless really does indicate your disconnection from science.

How does it do so please? If you'd explain the steps you're taking in reasoning then I will be happy to point out your faux pas.

I'm sure you're a great engineer, but it's very clear that you do not have the background or capacity to understand the implications of a simple scientific experiment, its analysis, and conclusions that can be drawn.

Once again, evidence please. We are attempting to do science here no?
 
In going into such detail it really helps to get the terminology accurate. Firstly there is no sound except as a result of the auditory perceptual process. So vibrations are transduced into neural impulses, not sounds....Again, confused. No sounds actually exist - what exist are vibrations. Sounds are all in the mind.

Are you joking? "Sound" is the transmitted mechanical vibration of a medium (like air). "Hearing" is what requires transduction of the vibration to neural impulses and subsequent perception by the brain. For example, if I play a note on a piano, but I have earplugs on, and I can't hear anything, did the piano still make a sound? Answer that one, my friend.
 
Yeah I'm aware of that. So it looks like you're tilting at windmills, and indeed wasting your time. The words you used were 'absolutely sure' - so given that we're talking about science why did you choose those words?

How does it do so please? If you'd explain the steps you're taking in reasoning then I will be happy to point out your faux pas.

Once again, evidence please. We are attempting to do science here no?

Listen, I have absolutely no idea what you're talking about now.

But let me be clear. It is definitely possible to design a controlled, blinded, listening experiment to test the hypothesis that EquipmentA has audible differences from EquipmentB. It will involve motivated listeners and lots of trials to achieve statistically significant results. Statistical analysis will be required. And in order to interpret the experiment in a meaningful way, at least a basic background in the scientific method and experimental design/analysis will be required of the reader.

At this point, you have demonstrated that you do not belong to this group. In fact, most people so-called "audiophiles" do not. Therefore, even if someone does conduct a rigorous scientific experiment with very low alpha and beta error, and obtains highly significant results one way or the other, it's a waste of time because the general audiophile community will never accept the results like a scientific community would.
 
Listen, I have absolutely no idea what you're talking about now.

In actuality that happened some time ago. But if you're interested in learning, try asking specific questions about the particular bits that made you get lost. I'm keen to improve my communications skills so will be happy to elucidate further.

But let me be clear. It is definitely possible to design a controlled, blinded, listening experiment to test the hypothesis that EquipmentA has audible differences from EquipmentB. It will involve motivated listeners and lots of trials to achieve statistically significant results. Statistical analysis will be required. And in order to interpret the experiment in a meaningful way, at least a basic background in the scientific method and experimental design/analysis will be required of the reader.

Yep, fairly uncontroversial. Did you think I'd object to that? The level of motivation is where I have my doubts - and motivation in and of itself isn't sufficient. If its to be truly meaningful it will require motivation AND ability to listen. That's much harder because those who have the ability will (very likely) lack the inclination.

At this point, you have demonstrated that you do not belong to this group. In fact, most people so-called "audiophiles" do not. Therefore, even if someone does conduct a rigorous scientific experiment with very low alpha and beta error, and obtains highly significant results one way or the other, it's a waste of time because the general audiophile community will never accept the results like a scientific community would.

I can't think why you'd do it to try to satisfy the audiophile community. Why not do it to satisfy yourself? In general human beings are not persuaded by evidence (those who claim that they are being amongst the least influenced by it) so if your intention is to persuade, then use oratory, or marketing not scientific experiments.
 
No.

Unless your girlfriend was also in the room having kittens about why you'd do such a crazy thing. Then she'd hear a sound.

Next question?

Okay, so you're saying that playing a note on a piano doesn't make a sound unless someone hears it!? Let's say I play a note on the piano, and I'm wearing earplugs and can't hear anything, but I recorded it on a tape recorder. Did the tape recorder record a SOUND?
 
Status
This old topic is closed. If you want to reopen this topic, contact a moderator using the "Report Post" button.