Blind Listening Tests & Amplifiers

[Ilianh]

This is impossible to proove since it is a false statement and idea, it´s really that simple.

My post carried a message about how different sound is a personal thing and highly dependant on complicated external factors.
But using my testing methods that I think are the best, I can prove to myself that many expensive amps/parts do not sound different.

What makes you think good parts need to be expensive???

Think? nothing, the statemet was based on what I've seen that other qualify as good parts, most of them were preety expensive.
In other words, most of the time wherever you see the words "audiophile quality" you see a long multiple digits number next to it.

This is my experience as well, making blind tests on very small differences a bad day is a bad idea.

It makes ANY test invalid.

It's just that one side compares sound on feelings based on what they heard, the other side on dissimilarities in the sound.

[jcarr]

Peter:

>Why anybody has to prove anything?<

In my experience, I must agree that there is a strong psychological bias in many of the things that we hear. But equally importantly, there also appears to be a strong psychological bias in many of the things that we _don't_ hear. In many cases, it appears that prior knowledge and prior attitudes both have a significant effect on the perceived outcome, but that outcome also may or may not be verifiable.

Be as that may, I do consider blind testing to be a useful tool for audio development, and I certainly intend to keep it as a permanent part of my own test arsenal. OTOH, I don't think that there is any need to rely solely on it, or to categorically reject any change simply because it fails a blind test.

For example, let us say that we have an amplifier builder who specializes in cool-looking aluminum cabinets and nice worksmanship. If those cabinets enable the end-user to obtain a more enjoyable total experience (of which listening is but one part), that is good enough, I think. At least for the normal world - outside of the lab.

Now if said amplifier builder can figure out how to take those same cabinets and replace the contents with something that is easier (and perhaps cheaper) for him to build, and yet maintain the same level of total end-user enjoyment, he will be ahead of where he is now.

The starting point would be to build multiple sets of amplifier cabinets that are visually identical. Next, change only the contents, but make sure that the weight and mass distribution of all of the test units remain the same. If something is changed and it isn't visibly obvious, somewhere inside the cabinet there should be a description for later validation. Give the amplifier sets to someone else and ask them to mark the amplifiers as arbitrary numbers or letters (so you don't know what is what). Ask a third person to set up a 10-cycle (or more) sequence (of their own liking - the only conditions are that each amplifier set must appear in the sequence at least once, and the sequence should not be in strict alternation). Ask a fourth person to put the amplifiers into your system according to the sequence, while you take listening notes. Use as much time as you like.

If you have access to other audio systems and listeners, ask them to do this, too.

At the end of the testing, collect all of the listening notes, open up the amplifiers and see how the contents compare with the listening notes.

I also recommend that your test audio system observes proper power polarities throughout, has an independent true ground, the speakers have been carefully set up so that acoustic reflections, standing waves and response regularities are well-suppressed, and critical listening is conducted at times when the ambient noise level is as low as possible.

In any case, blind testing can be interesting to apply, and to many more things than just audio. But based on what I have observed, if you subject many of the things that you normally enjoy or indulge in to blind testing, your cost of living will go way, way down.

regards, jonathan carr

[janneman]

Jonathan, re: your post above: I only wish I had written it....

Jan Didden

[mrfeedback]

Hi Jonathon,
You detail here a very vigorous blind testing method, and one that one that I would give credence to also.

In particular your test conditions of - ' I also recommend that your test audio system observes proper power polarities throughout, has an independent true ground, the speakers have been carefully set up so that acoustic reflections, standing waves and response regularities are well-suppressed, and critical listening is conducted at times when the ambient noise level is as low as possible.' are entirely required, indeed mission critical to meaningful testing, blind or otherwise.

Also noteworthy is that typical systems can be properly optimised for one listening position only.

At the end of the testing, collect all of the listening notes, open up the amplifiers and see how the contents compare with the listening notes.

I have been to your website and see the equipment that you are creating, and have read many of your comments on theory and circuit techniques, so I conclude that you are both Objective and Subjective - say so if I am wrong.

Question is - Have you found any surprises using your mode of blind testing, or indeed correlations with your own subjective findings for your products ?.

Eric.

[Bratislav]

Originally posted by jcarr

In my experience, I must agree that there is a strong psychological bias in many of the things that we hear. But equally importantly, there also appears to be a strong psychological bias in many of the things that we _don't_ hear. In many cases, it appears that prior knowledge and prior attitudes both have a significant effect on the perceived outcome, but that outcome also may or may not be verifiable.

Be as that may, I do consider blind testing to be a useful tool for audio development, and I certainly intend to keep it as a permanent part of my own test arsenal. OTOH, I don't think that there is any need to rely solely on it, or to categorically reject any change simply because it fails a blind test.

In this case we are not (at least *I* am not) talking about preconceived perceptions and biases. I'm talking about the true, undisputeable ability to detect/measure/whatever.
The process of detecting something involves known, reliable set of premises and a method and set of tools that is _capable_ of detecting the item we're looking into. If we are to measure half a dozen of Fabry-Perrot etalons and pick the Calcium line one over bunch of H-alpha (for unitiated, it will differ in thickness by couple of nanometers), we simply can't do that with vernier calliper. A measuring device MUST have higher resolution than a thing we're trying to measure. Anything else is a waste of time.

In this sad case we've wasted time so far, we are using set of ears (golden for some lucky people), set of speakers and a room to evaluate differences among a set of well designed amplifiers. Those differences are SEVERAL ORDERS OF MAGNITUDE SMALLER than detection threshold of our measuring set. This is a fact; it is not disputable, despite the efforts of subjectivists to invent unknown to science distortion mechanisms (who invariably FAIL to actually put the number at the end of their pseudoscientific tirade of how much such and such distortion mechanism adds to the distortion spectrum).
Note that there is no dispute that everyting (well, ALMOST everything) they discuss may have SOME effect. Back EMF feedback/crostalk - definitely plausible. Hearable ? Definitely NOT.
Skin effect in cables. Transmission reflections. Sure. All known and well understood. But UTTERLY irrelevant to audio cables, frequencies and signal magnitudes. Why ? Because our detecting setup (ears/speakers/room) simply does not have the resolution required. (and to Pan - before another tirade about how wonderful and brilliant your speakers are, please check distortion figures on Accuton drivers. What are distortion residuals at say 20W input ? I bet they will be way on the wrong side of 1%. And that is even before your room comes in and plays its own bit. It will be same for ATC, Dynaudio, Excel, Audiom, you name it).
You are trying to detect differences in amps' distortion figures using speakers and room that will distort perhaps a thousand times more. Or couple of million times (in case of a 75$ resistor vs 3c one). Or perhaps BILLION times more, in case of square vs round feet or directionality of one resitor.
Yeah, sure.

But as I said before, one thing is for sure. People believe in whatever they want and that is certainly their right. If you believe that amp with square feet sounds better than one having round feet, be my guest.

How does one remove himself from this forum again ?

[phase_accurate]

You are trying to detect differences in amps' distortion figures using speakers and room that will distort perhaps a thousand times more.

Which OTOH doesn't mean that the speaker's distortion is more audible than that generated by the amp.

Regards

Charles

[mrfeedback]

Hi Peter, good to see you back here.

Maybe the golden ears were not that golden after all?

Yes, and it is very likely that the test conditions were not optimal - either way, the same invalid results.
See my post about comments earlier about pitch perception, for it applies here too.

If your approach to music is only dependant on testing, collecting facts and the need to prove something, than I feel sorry for you.

Yes, fine points in circuit techniques and measurements are one factor, but the final sonic result is the ultimate arbiter.
Experience in comparing a large number of audio systems and individual equipments is required to form reliable opinions.

I think that the 'engineer and measurements only' types here should go out and do some live sound mixing for an education in appeciation of very fine differences in distortions, noises and harmonic structures, and the sonic results of micro changes in levels and eq and dynamics of all of the above.
Live mixing requires discrimination in hearing good enough to pick out a particular instrument in the whole mix, and is a learned and practiced skill.
Once this degree of discrimination is learned and achieved, then it is a simple matter to distinguish different audio system components - see Pan's posts, and I expect that the 'all same measuring amplifiers sound the same' members here have nil or little experience or skill at this level and mode of listening.

If I feel a need to change a resistor or different feet, it's only because I can't enjoy the music the way I usually do, and I don't give a crap about your requirement to prove that it actually makes a difference.

I agree that changing to a 'same measuring resistor' etc will not change the typically simply measured results, but it can sound the sonics from a system that is not sitting 'right', to one that really is pleasing.

And mrfeedback used to joke about those things as well one time

)........"I know you can hear the difference, all I'm saying is that there is no difference!"

I was not joking there - that was a comment on the 'lunatic fringe', as I consider some of the Peter Belt's claims.
I consider his assertions to be very strongly in the 'psychological bias' zone, and some of his claims to be rather extraordinary - no I have not tried putting photos in the freezer and I probably never will, but I do understand the process he speaks of.
The quote above also is directed at the 'objectivist only' members here, who are consistently asserting that they cannot measure differences, and state therefore they do not exist.

The situation so far, is that various members are saying that they can hear fine differences in amplifiers, and others are saying that they cannot measure them.
So the task at hand, is between us to devise a revealing objective testing methodology that correlates to the listening observations.
Null testing as presented is flawed, standard FR and THD+N methods are inadequate, so it remains to devise a reliable testing specification.

Let us put our heads together on this.

Eric / - Always been keen to see test results that correlate to listening testing.


It was also a

[carlosfm]

nw, let's summarize your points of view:

1 - There's no audible difference between an Onkyo AV amp and a Bryston stereo amp.

2 - You don't believe in burning-in (including speakers), because there's no difference.

3 - Suddenly people began to talk about burning-in, and you don't believe in it.

Well...
Answers:

1 - I told you, you're deaf, and I recommended you cotton buds.

2 - Again, incredibe, I'm astonished.

3 - Ok, so every amp sounds the same, and cables simply don't make a difference (bell wire is ok).
Wasn't this that people believed until the late 70's?
Someone has to revolutionate something and have a different oppinion for the first time.
That's evolution.
Sometimes you notice improvements after burning-in, sometimes you don't.
But speakers? Oh, please...

Note: nw, I've seen very elaborate reports and studies that say caples cables don't make a difference.
But I can show it to you. They make difference.
You have to learn how to "filter" information, and don't believe in everything just because it seams highly scientific, or because it's Mr. A or B that says it.

Let me tell you that today you notice improvements after burning-in of equipment that you may not notice some 30 years ago.
Today everything is getting more and more "transparent".
I mean, component quality has evoluted, you know?
30 years ago your preamp may have TL071 op-amps inside, today
it may have OPA627s (if you're a lucky guy).

[SY]

Proving that every person cannot hear the difference between most amps is not possible IMO

Not your opinion, just plain fact. What you say is exactly correct.

It's much like ESP. No-one can prove that there isn't SOMEONE out there with ESP. But when conditions are set up to prevent cheating, test after test after test comes up null. At a certain point, one can provisionally say, "There doesn't appear to be any ESP going on." That provisional conclusion must change when and if a valid, replicable test comes along that does show an effect. Not that I'm holding my breath!

Likewise, it's provisionally safe for someone designing a box of gain to be guided by the principle that if it has sufficiently low THD/IM distortion, not overloaded, low enough source impedance to keep FR variation under 0.1 dB, etc, etc, it will have an output indistinguishable from the input. All it will take to shatter this belief is one, just one valid test where a subject or subjects can indeed make this distinction. Hasn't happened yet, but any intelligent person will allow for the possibility that it could.

[carlosfm]

SY,

How can you guarantee an amplified signal can be indistinguishable from the input?
Can you hear line-level sounds on your speakers?
Every amp has a sonic signature, and nobody knows what's neutral.
Even the recording on the disc isn't neutral.
Forget the specs, sometimes an amp/preamp sounds better to you, even with worst specs than others.
I just get amazed that some people say an NE5532/4 is more neutral than an OPA627.
What's neutral??!!
The NEs have a cold and undynamic sound (typical cd sound, in fact).
That's neutral?
Just LISTEN!
Forget those THD specs, because they tell you nothing about neutrality.
Neutral is hearing the band live without any form of amplification.