Blind DAC Public Listening Test Results

Status
This old topic is closed. If you want to reopen this topic, contact a moderator using the "Report Post" button.
In going into such detail it really helps to get the terminology accurate. Firstly there is no sound except as a result of the auditory perceptual process. So vibrations are transduced into neural impulses, not sounds.

Normally a vibration of air qualifies as sound, if its spectral content falls into the audio band.

So a sound is a sound if a human (or sometimes another animal) _could_ hear it, even if nobody is around. :)
 
Normally a vibration of air qualifies as sound, if its spectral content falls into the audio band.

So a sound is a sound if a human (or sometimes another animal) _could_ hear it, even if nobody is around. :)

I'm content that we operate differing (even non-overlapping) definitions of the word 'sound'. In this context was doing rigorous analysis, not normal day-to-day chit-chat. :D
 
That sums it up.

Well, isn´t telling apart a crown from a threshold under blind test conditions basically the same as telling apart a cheap "whatever" amp from an expensive one?

By eliminating the biggest variables, yes, it does.

Sensory testing isn´t like mathematics- even if you think one bias mechanism is more important than any other, it isn´t guaranteed that test results are more correct (or simply correct) after elimination of the "big one" .

See for example the data from same/different tests on audio topics; in these cases we know for the "same presentation" what the correct answer has to be and the error rates are normally extremely high.
So, evidence exists, that after elimination of the "big one" the next bias mechanism might have an impact of the same magnitude.

But, while the "big one" favours false positives, the next under blind conditions favours obviously false negatives.
 
Last edited:
Most people hate people that disagree with them.
Same people hate blind tests because they might prove their oppinions wrong.
Therfore they need to dismiss the blind tests and people conducting them.

I think this is often the case. But in my many years of experience discussing blind listening tests with "audiophiles," the #1 reason for people dismissing them is that they do not have a basic high-school level grasp of the scientific method and the basic principles of study design (like randomization, bias, control, the placebo effect, etc.).

I'm content that we operate differing (even non-overlapping) definitions of the word 'sound'. In this context was doing rigorous analysis, not normal day-to-day chit-chat. :D

No, in this context, you're using your own personal definition of the word "sound," which is problematic. For instance, according to your usage:
Let's say a 1kHz sine wave at 70dB relative to a human being is playing from a loudspeaker. According to you, there is no sound coming from that loudspeakers unless the person hears it. If he inserts earplugs and doesn't hear the 1kHz sine wave, then according to you, there IS NO SOUND. And in such a case, if I capture that sine wave on a tape recorder, and the person still has the earplugs in, then according to you, the tape recorder DID NOT record a sound.

It goes on and on to the point of further ridiculousness. If you want to make up your own definitions for commonly used words, that's your business, I guess.
 
I'm content that we operate differing (even non-overlapping) definitions of the word 'sound'. In this context was doing rigorous analysis, not normal day-to-day chit-chat. :D

Oh, i´m sure that i do miss a lot, especially as german and english treat acoustics sometimes very different in wording and meaning of phrases or making distinction or not making distinctions.

In this case it seems to be the more rigorous definition; in german it is called "Schall" if it falls in the human range but the springer handbook of acoustics for examples calls it "sounds".

The next paragraph would be "sounds that we hear" and "sounds that we don´t hear" as infrasound, ultrasound, hypersound and so on.

Harry F.Olson said: "1.2. Sound Waves. — Sound is an alteration in pressure, particle displacement or particle velocity propagated in an elastic material or the superposition of such propagated alterations.
Sound is also the sensation produced through the ear by the alterations
described above." :p
 
No, in this context, you're using your own personal definition of the word "sound," which is problematic.

Yep, guilty as charged. When 'sound' is confined to percepts then there's no difficulty in describing what goes on in, say, the McGurk effect. In that case, two people presented with the same vibrational stimulus report differing sounds.

For instance, .... the tape recorder DID NOT record a sound.

It goes on and on to the point of further ridiculousness.

Can't see the ridiculousness you're claiming is there.

If you want to make up your own definitions for commonly used words, that's your business, I guess.

Like your idiosyncratic use of the word 'clearly' for things that aren't in fact clear (i.e. observable) at all?
 
I think this is often the case. But in my many years of experience discussing blind listening tests with "audiophiles," the #1 reason for people dismissing them is that they do not have a basic high-school level grasp of the scientific method and the basic principles of study design (like randomization, bias, control, the placebo effect, etc.).

Well, as I said earlier, I don't believe the scientific method really applies to listening tests or to wine tasting because of the uncontrollable variables of human hearing and human taste/smell. What is the standard for human hearing? You can't perform a test without a standard. If it's "flat from 20-20kHz", then you need babies and young children for the test because those are the only ones who can hear that well. The rest of us, it's a crap shoot. There is no standard for human hearing because everyone hears differently so you can't use the scientific method to evaluate audio gear. You can however find people with exceptional hearing and let them tell you what they hear, just as you can find people with exceptional taste buds and let them tell you what they taste in the wine. Did you know there are people in Africa who can clearly see the rings of Saturn and the moons around Jupiter? (I think that's right) Can you? I sure as hell cannot.

People apply the scientific method to wine tasting and audio listening in order to make themselves feel more validated and secure in their assumptions about which wine or amplifier is better, or more precisely, they use it to validate the belief in the scientific method. People who need this kind of validation are insecure in my opinion. I don't need this kind of nonsense. If I can't hear it, it's not real to me. Likewise, if I can't taste it, it is not real to me either. Everyone's sensory reality is defined by their ability to discern, and thus everyone's is unique, and it changes with age.
 
Last edited:
Likewise with audio. If you can't hear it, then there's no point in buying or making super duper gear. Most older people know this. Likewise, it is basically a waste of time doing listening tests (unless you are a gifted listener) with ordinary people because, as RS found out, half of them can't tell a difference.

First of all, RS's sample size, methodology, and analysis do NOT allow anyone to conclude that ordinary listeners could or could not hear a difference. Secondly, it has already been demonstrated in a paper by the Harman group that untrained ordinary listeners, when blinded, are still capable of hearing differences among loudspeakers and have similar preferences to trained listeners. You can use ordinary people, but it will be harder to show small differences.

Applying double blind listening tests is as absurd to me as doing double blind wine tasting.

Based on WHAT? Peforming an UNBLINDED listening test is patently absurd to me because it doesn't account for the placebo effect (bias), which has a huge influence on listener preferences, and of a far greater magnitude than actual equipment differences. This has been demonstrated over and over, the most common example being listeners being told that a new expensive cable was swapped in (when it had not) and listeners describing all the fantastic audible improvements that they heard.

And honestly, wine tasting SHOULD be blind (and often it is). When WA/WS/WE/W&S/etc publish a score of 90 or better, that vintage is going to see tremendous sales and profit. When a score of 89 or lower is rendered, then that's basically a kiss of death profit-wise in comparison. In order to separate the influence of financial gain/loss (i.e. advertising) and the hurt feelings from the winemaking community from influencing the wine score, wine tasting should definitely be performed blind (and I believe many of them are). But wine scoring isn't itself a research science, and indeed, when wine judges.

Even further, the fact that women in general can hear and taste better than men has not been a factor in the results that I've seen anyway. Did you know that women can see colors better than men? They have an extra rod in their retinas that men don't have so they can discern green-blue better.

That's amazing, since rods are actually the photoreceptors that perceive black/white and CONES are the photoreceptors for color.
 
First of all, RS's sample size, methodology, and analysis do NOT allow anyone to conclude that ordinary listeners could or could not hear a difference. Secondly, it has already been demonstrated in a paper by the Harman group that untrained ordinary listeners, when blinded, are still capable of hearing differences among loudspeakers and have similar preferences to trained listeners. You can use ordinary people, but it will be harder to show small differences.

I wasn't talking about RS's experiment. I don't care about some paper by the Harmon Group. You seem to be grasping at straws and avoiding the core issue I presented.
 
Based on WHAT? Peforming an UNBLINDED listening test is patently absurd to me because it doesn't account for the placebo effect (bias), which has a huge influence on listener preferences, and of a far greater magnitude than actual equipment differences. This has been demonstrated over and over, the most common example being listeners being told that a new expensive cable was swapped in (when it had not) and listeners describing all the fantastic audible improvements that they heard.

And honestly, wine tasting SHOULD be blind (and often it is). When WA/WS/WE/W&S/etc publish a score of 90 or better, that vintage is going to see tremendous sales and profit. When a score of 89 or lower is rendered, then that's basically a kiss of death profit-wise in comparison. In order to separate the influence of financial gain/loss (i.e. advertising) and the hurt feelings from the winemaking community from influencing the wine score, wine tasting should definitely be performed blind (and I believe many of them are). But wine scoring isn't itself a research science, and indeed, when wine judges.

And yet you're trying to turn audio testing into a research science?

If you had honest and unbiased listeners with highly gifted hearing, then non-blind testing would be just fine.

You sure are making a lot blather about my own personal opinion. Why so much effort on your part? Are you judging me or something?
 
That's amazing, since rods are actually the photoreceptors that perceive black/white and CONES are the photoreceptors for color.

So, you are now trying to prove that you are smarter than me because you remembered this and I didn't, and that's supposed to prove your point that you know you're right and I'm wrong? Nah, sorry, don't buy it.

It is cones after all. I made a simple mistake and you pounce on it like some kind of hungry lion and then try to rub it in my face. Nah, you're not really open minded enough to have this discussion in my opinion. I've said my piece, you will never change my mind, and now I'm done. Goodbye.
 
It is cones after all. I made a simple mistake and you pounce on it like some kind of hungry lion and then try to rub it in my face.

OTOH you could have taken it on the chin and out-pedanted him by replying with the fact that black and white are still colours and what rods are sensitive to is not in practice black and white like he claimed but rather intensity (or luminance).:cool:
 
A
Well, as I said earlier, I don't believe the scientific method really applies to listening tests or to wine tasting because of the uncontrollable variables of human hearing and human taste/smell. What is the standard for human hearing? You can't perform a test without a standard. If it's "flat from 20-20kHz", then you need babies and young children for the test because those are the only ones who can hear that well. The rest of us, it's a crap shoot. There is no standard for human hearing because everyone hears differently so you can't use the scientific method to evaluate audio gear. You can however find people with exceptional hearing and let them tell you what they hear, just as you can find people with exceptional taste buds and let them tell you what they taste in the wine.

The education of a typical scientist who designs/performs scientific experiments includes: 4 yrs B.S., 2 yrs M.S. 4 yrs Ph.D., and ~4yrs postdoc, and that's just to become a junior investigator. Now why on earth do people on an internet forum frequently think that their "common sense" and "intuition" somehow give them the same knowledge and ability in this area?!

Yes, OF COURSE you can design and conduct a listening test to describe differences (or lack of) between piece of audio gear. You're bringing up the very basic issue of "generalizability," which is whether the result of the study actually apply to you. In your case, if you have high frequency hearing loss, then a study that involved listeners with normal age-appropriate hearing (and yes, this can be tested, and yes there are objective clinical ranges that are considered "normal") may not apply to you. But just because your ability to hear small differences may not be as good as 50 listeners recruited for a study, it doesn't mean that we can't study the equipment with a listening test! It just means that you may not be able to apply the results of this study to your own personal situation. That's why the scientific method is so important. A scientist can look at a study, its methodology, and its analysis to see if it's applicable to him/her.

Now, I'm sorry to say this, but your suggestion to eliminate blind listening tests involving multiple listeners, trials, and analysis, and replace it with a single unblinded listener with especially trained hearing is just ridiculous. Not only do you make the study even less generalizable (because the average audiophile has average audiophile hearing, not expert-trained hearing), but you also introduce the likelihood of bias (by not blinding) and you lose the benefit of statistical analysis (n=1).

People apply the scientific method to wine tasting and audio listening in order to make themselves feel more validated and secure in their assumptions about which wine or amplifier is better.

First of all, the scientific method is NOT applied to wine tasting, I don't know where you're getting that from, that's ridiculous. Wine scoring is about as unscientific as it gets (single test subject, no repetition, single wine sample, no objective scale). Secondly, people apply the scientific method to audio listening in order to eliminate bias and the placebo effect from listener observations, not for the reasons you state.

People who need this kind of validation are insecure in my opinion. I don't need this kind of nonsense. If I can't hear it, it's not real to me.

Likewise, when you do hear differences, how do you know if they're real? How do you know if it isn't just the influence of positive expecations (the placebo effect)? That's why you need BLINDING when performing listening tests, in order to eliminate this factor.
 
OTOH you could have taken it on the chin and out-pedanted him by replying with the fact that black and white are still colours and what rods are sensitive to is not in practice black and white like he claimed but rather intensity (or luminance).:cool:

I was actually expecting you to argue that light doesn't exist unless someone actually sees it, just as you argue that there is no sound unless someone hears it.
 
First of all, the scientific method is NOT applied to wine tasting, I don't know where you're getting that from, that's ridiculous. Wine scoring is about as unscientific as it gets (single test subject, no repetition, single wine sample, no objective scale).

It is, and it's done routinely. Don't confuse real analytical wine tasting with Robert Parker giving a sniff and declaring a wine to score 92.

As with anything else in sensory analysis, the choice of test format, subjects, and sample size depends on the specific question the researcher is trying to answer.
 
@ dirkwright,

telling apart a crown from a threshold does not necessarily mean that the crown does sound more worse. (slowly moving out of that trap....) :)


Using scientific methods should ensure, that your study of an effect gives results due to this effect but not due to anything else.

So, using expert listeners can already be part of the scientific approach, but which listeners to use depends on the question beeing studied.
And if you´re only interested in differences evoked by audible differences you would normally let the expert listeners work under blind conditions in a controlled experiment.

But as any controlled experiment is an artificial environment, you have to ensure that your participants work in a proper way under these test conditions and at that point the positive controls come into play.

If somebody has to use controlled listening experiments for personal evaluation is an interesting question. I´d say, no it is not needed, but doing some controlled listening (at least single blinded) helps to learn a lot about the way your perception works.
 
Status
This old topic is closed. If you want to reopen this topic, contact a moderator using the "Report Post" button.