Blind Listening Tests & Amplifiers

Status
Not open for further replies.
This is impossible to proove since it is a false statement and idea, it´s really that simple.

My post carried a message about how different sound is a personal thing and highly dependant on complicated external factors.
But using my testing methods that I think are the best, I can prove to myself that many expensive amps/parts do not sound different.




What makes you think good parts need to be expensive???

Think? nothing, the statemet was based on what I've seen that other qualify as good parts, most of them were preety expensive.
In other words, most of the time wherever you see the words "audiophile quality" you see a long multiple digits number next to it.

This is my experience as well, making blind tests on very small differences a bad day is a bad idea.

It makes ANY test invalid.

It's just that one side compares sound on feelings based on what they heard, the other side on dissimilarities in the sound.
 
Peter:

>Why anybody has to prove anything?<

In my experience, I must agree that there is a strong psychological bias in many of the things that we hear. But equally importantly, there also appears to be a strong psychological bias in many of the things that we _don't_ hear. In many cases, it appears that prior knowledge and prior attitudes both have a significant effect on the perceived outcome, but that outcome also may or may not be verifiable.

Be as that may, I do consider blind testing to be a useful tool for audio development, and I certainly intend to keep it as a permanent part of my own test arsenal. OTOH, I don't think that there is any need to rely solely on it, or to categorically reject any change simply because it fails a blind test.

For example, let us say that we have an amplifier builder who specializes in cool-looking aluminum cabinets and nice worksmanship. :) If those cabinets enable the end-user to obtain a more enjoyable total experience (of which listening is but one part), that is good enough, I think. At least for the normal world - outside of the lab. :D

Now if said amplifier builder can figure out how to take those same cabinets and replace the contents with something that is easier (and perhaps cheaper) for him to build, and yet maintain the same level of total end-user enjoyment, he will be ahead of where he is now.

The starting point would be to build multiple sets of amplifier cabinets that are visually identical. Next, change only the contents, but make sure that the weight and mass distribution of all of the test units remain the same. If something is changed and it isn't visibly obvious, somewhere inside the cabinet there should be a description for later validation. Give the amplifier sets to someone else and ask them to mark the amplifiers as arbitrary numbers or letters (so you don't know what is what). Ask a third person to set up a 10-cycle (or more) sequence (of their own liking - the only conditions are that each amplifier set must appear in the sequence at least once, and the sequence should not be in strict alternation). Ask a fourth person to put the amplifiers into your system according to the sequence, while you take listening notes. Use as much time as you like.

If you have access to other audio systems and listeners, ask them to do this, too.

At the end of the testing, collect all of the listening notes, open up the amplifiers and see how the contents compare with the listening notes.

I also recommend that your test audio system observes proper power polarities throughout, has an independent true ground, the speakers have been carefully set up so that acoustic reflections, standing waves and response regularities are well-suppressed, and critical listening is conducted at times when the ambient noise level is as low as possible.

In any case, blind testing can be interesting to apply, and to many more things than just audio. But based on what I have observed, if you subject many of the things that you normally enjoy or indulge in to blind testing, your cost of living will go way, way down. :D

regards, jonathan carr
 
Proper Blind Testing

Hi Jonathon,
You detail here a very vigorous blind testing method, and one that one that I would give credence to also.

In particular your test conditions of - ' I also recommend that your test audio system observes proper power polarities throughout, has an independent true ground, the speakers have been carefully set up so that acoustic reflections, standing waves and response regularities are well-suppressed, and critical listening is conducted at times when the ambient noise level is as low as possible.' are entirely required, indeed mission critical to meaningful testing, blind or otherwise.

Also noteworthy is that typical systems can be properly optimised for one listening position only.

At the end of the testing, collect all of the listening notes, open up the amplifiers and see how the contents compare with the listening notes.

I have been to your website and see the equipment that you are creating, and have read many of your comments on theory and circuit techniques, so I conclude that you are both Objective and Subjective - say so if I am wrong.

Question is - Have you found any surprises using your mode of blind testing, or indeed correlations with your own subjective findings for your products ?.

Eric.
 
jcarr said:

In my experience, I must agree that there is a strong psychological bias in many of the things that we hear. But equally importantly, there also appears to be a strong psychological bias in many of the things that we _don't_ hear. In many cases, it appears that prior knowledge and prior attitudes both have a significant effect on the perceived outcome, but that outcome also may or may not be verifiable.

Be as that may, I do consider blind testing to be a useful tool for audio development, and I certainly intend to keep it as a permanent part of my own test arsenal. OTOH, I don't think that there is any need to rely solely on it, or to categorically reject any change simply because it fails a blind test.


In this case we are not (at least *I* am not) talking about preconceived perceptions and biases. I'm talking about the true, undisputeable ability to detect/measure/whatever.
The process of detecting something involves known, reliable set of premises and a method and set of tools that is _capable_ of detecting the item we're looking into. If we are to measure half a dozen of Fabry-Perrot etalons and pick the Calcium line one over bunch of H-alpha (for unitiated, it will differ in thickness by couple of nanometers), we simply can't do that with vernier calliper. A measuring device MUST have higher resolution than a thing we're trying to measure. Anything else is a waste of time.

In this sad case we've wasted time so far, we are using set of ears (golden for some lucky people), set of speakers and a room to evaluate differences among a set of well designed amplifiers. Those differences are SEVERAL ORDERS OF MAGNITUDE SMALLER than detection threshold of our measuring set. This is a fact; it is not disputable, despite the efforts of subjectivists to invent unknown to science distortion mechanisms (who invariably FAIL to actually put the number at the end of their pseudoscientific tirade of how much such and such distortion mechanism adds to the distortion spectrum).
Note that there is no dispute that everyting (well, ALMOST everything) they discuss may have SOME effect. Back EMF feedback/crostalk - definitely plausible. Hearable ? Definitely NOT.
Skin effect in cables. Transmission reflections. Sure. All known and well understood. But UTTERLY irrelevant to audio cables, frequencies and signal magnitudes. Why ? Because our detecting setup (ears/speakers/room) simply does not have the resolution required. (and to Pan - before another tirade about how wonderful and brilliant your speakers are, please check distortion figures on Accuton drivers. What are distortion residuals at say 20W input ? I bet they will be way on the wrong side of 1%. And that is even before your room comes in and plays its own bit. It will be same for ATC, Dynaudio, Excel, Audiom, you name it).
You are trying to detect differences in amps' distortion figures using speakers and room that will distort perhaps a thousand times more. Or couple of million times (in case of a 75$ resistor vs 3c one). Or perhaps BILLION times more, in case of square vs round feet or directionality of one resitor.
Yeah, sure.

But as I said before, one thing is for sure. People believe in whatever they want and that is certainly their right. If you believe that amp with square feet sounds better than one having round feet, be my guest.

How does one remove himself from this forum again ?
 
Re: Re: Re: Re: Re: I Know What I Hear.

Hi Peter, good to see you back here. ;)
Maybe the golden ears were not that golden after all?
Yes, and it is very likely that the test conditions were not optimal - either way, the same invalid results.
See my post about comments earlier about pitch perception, for it applies here too.
If your approach to music is only dependant on testing, collecting facts and the need to prove something, than I feel sorry for you.
Yes, fine points in circuit techniques and measurements are one factor, but the final sonic result is the ultimate arbiter.
Experience in comparing a large number of audio systems and individual equipments is required to form reliable opinions.

I think that the 'engineer and measurements only' types here should go out and do some live sound mixing for an education in appeciation of very fine differences in distortions, noises and harmonic structures, and the sonic results of micro changes in levels and eq and dynamics of all of the above.
Live mixing requires discrimination in hearing good enough to pick out a particular instrument in the whole mix, and is a learned and practiced skill.
Once this degree of discrimination is learned and achieved, then it is a simple matter to distinguish different audio system components - see Pan's posts, and I expect that the 'all same measuring amplifiers sound the same' members here have nil or little experience or skill at this level and mode of listening.
If I feel a need to change a resistor or different feet, it's only because I can't enjoy the music the way I usually do, and I don't give a crap about your requirement to prove that it actually makes a difference.
I agree that changing to a 'same measuring resistor' etc will not change the typically simply measured results, but it can sound the sonics from a system that is not sitting 'right', to one that really is pleasing.
And mrfeedback used to joke about those things as well one time
)........"I know you can hear the difference, all I'm saying is that there is no difference!"
I was not joking there - that was a comment on the 'lunatic fringe', as I consider some of the Peter Belt's claims.
I consider his assertions to be very strongly in the 'psychological bias' zone, and some of his claims to be rather extraordinary - no I have not tried putting photos in the freezer and I probably never will, but I do understand the process he speaks of.
The quote above also is directed at the 'objectivist only' members here, who are consistently asserting that they cannot measure differences, and state therefore they do not exist.

The situation so far, is that various members are saying that they can hear fine differences in amplifiers, and others are saying that they cannot measure them.
So the task at hand, is between us to devise a revealing objective testing methodology that correlates to the listening observations.
Null testing as presented is flawed, standard FR and THD+N methods are inadequate, so it remains to devise a reliable testing specification.

Let us put our heads together on this.

Eric / - Always been keen to see test results that correlate to listening testing.


It was also a
 
This one is to nw_avpjile

nw, let's summarize your points of view:

1 - There's no audible difference between an Onkyo AV amp and a Bryston stereo amp.:bigeyes:

2 - You don't believe in burning-in (including speakers), because there's no difference.

3 - Suddenly people began to talk about burning-in, and you don't believe in it.

Well...
Answers:

1 - I told you, you're deaf, and I recommended you cotton buds.

2 - Again, incredibe, I'm astonished.:clown:

3 - Ok, so every amp sounds the same, and cables simply don't make a difference (bell wire is ok).
Wasn't this that people believed until the late 70's?
Someone has to revolutionate something and have a different oppinion for the first time.
That's evolution.
Sometimes you notice improvements after burning-in, sometimes you don't.
But speakers? Oh, please...:devily:

Note: nw, I've seen very elaborate reports and studies that say caples cables don't make a difference.
But I can show it to you. They make difference.
You have to learn how to "filter" information, and don't believe in everything just because it seams highly scientific, or because it's Mr. A or B that says it.

Let me tell you that today you notice improvements after burning-in of equipment that you may not notice some 30 years ago.
Today everything is getting more and more "transparent".
I mean, component quality has evoluted, you know?
30 years ago your preamp may have TL071 op-amps inside, today
it may have OPA627s (if you're a lucky guy).:nod:
 
Proving that every person cannot hear the difference between most amps is not possible IMO

Not your opinion, just plain fact. What you say is exactly correct.

It's much like ESP. No-one can prove that there isn't SOMEONE out there with ESP. But when conditions are set up to prevent cheating, test after test after test comes up null. At a certain point, one can provisionally say, "There doesn't appear to be any ESP going on." That provisional conclusion must change when and if a valid, replicable test comes along that does show an effect. Not that I'm holding my breath!

Likewise, it's provisionally safe for someone designing a box of gain to be guided by the principle that if it has sufficiently low THD/IM distortion, not overloaded, low enough source impedance to keep FR variation under 0.1 dB, etc, etc, it will have an output indistinguishable from the input. All it will take to shatter this belief is one, just one valid test where a subject or subjects can indeed make this distinction. Hasn't happened yet, but any intelligent person will allow for the possibility that it could.
 
SY,

How can you guarantee an amplified signal can be indistinguishable from the input?
Can you hear line-level sounds on your speakers?
Every amp has a sonic signature, and nobody knows what's neutral.
Even the recording on the disc isn't neutral.
Forget the specs, sometimes an amp/preamp sounds better to you, even with worst specs than others.
I just get amazed that some people say an NE5532/4 is more neutral than an OPA627.
What's neutral??!!
The NEs have a cold and undynamic sound (typical cd sound, in fact).
That's neutral?
Just LISTEN!:idea:
Forget those THD specs, because they tell you nothing about neutrality.
Neutral is hearing the band live without any form of amplification.
 
Disabled Account
Joined 2002
Peter Daniel said:
>>.....on the contrary, that is precisely what blind tests acheive....no doubts.....niente...nada....period<<

I guess you still didn't get it. I mean all people. How can you practically carry it out? I am sure that there are exceptional cases that can hear differences we can't even dream about and only testing all the people on Earth would prove beyond doubt that you are right in your delusion.

yes all people ...no one can hear to 1MHz....basic biology...old chap......basically...20Hz-20KHz..exceptionaly, 15Hz-22KHz......ALL people EVERYONE of them on the planet cannot tell the difference in a blind test, between two or more low distortion amps. driven to same volt. swing across the same load in turn. Elementary:nod:
 
Disabled Account
Joined 2002
jcarr said:
Peter:

>Why anybody has to prove anything?<

In my experience, I must agree that there is a strong psychological bias in many of the things that we hear. But equally importantly, there also appears to be a strong psychological bias in many of the things that we _don't_ hear. In many cases, it appears that prior knowledge and prior attitudes both have a significant effect on the perceived outcome, but that outcome also may or may not be verifiable.

Be as that may, I do consider blind testing to be a useful tool for audio development, and I certainly intend to keep it as a permanent part of my own test arsenal. OTOH, I don't think that there is any need to rely solely on it, or to categorically reject any change simply because it fails a blind test.

For example, let us say that we have an amplifier builder who specializes in cool-looking aluminum cabinets and nice worksmanship. :) If those cabinets enable the end-user to obtain a more enjoyable total experience (of which listening is but one part), that is good enough, I think. At least for the normal world - outside of the lab. :D

Now if said amplifier builder can figure out how to take those same cabinets and replace the contents with something that is easier (and perhaps cheaper) for him to build, and yet maintain the same level of total end-user enjoyment, he will be ahead of where he is now.

The starting point would be to build multiple sets of amplifier cabinets that are visually identical. Next, change only the contents, but make sure that the weight and mass distribution of all of the test units remain the same. If something is changed and it isn't visibly obvious, somewhere inside the cabinet there should be a description for later validation. Give the amplifier sets to someone else and ask them to mark the amplifiers as arbitrary numbers or letters (so you don't know what is what). Ask a third person to set up a 10-cycle (or more) sequence (of their own liking - the only conditions are that each amplifier set must appear in the sequence at least once, and the sequence should not be in strict alternation). Ask a fourth person to put the amplifiers into your system according to the sequence, while you take listening notes. Use as much time as you like.

If you have access to other audio systems and listeners, ask them to do this, too.

At the end of the testing, collect all of the listening notes, open up the amplifiers and see how the contents compare with the listening notes.

I also recommend that your test audio system observes proper power polarities throughout, has an independent true ground, the speakers have been carefully set up so that acoustic reflections, standing waves and response regularities are well-suppressed, and critical listening is conducted at times when the ambient noise level is as low as possible.

In any case, blind testing can be interesting to apply, and to many more things than just audio. But based on what I have observed, if you subject many of the things that you normally enjoy or indulge in to blind testing, your cost of living will go way, way down. :D

regards, jonathan carr

...Peter Daniel take note:nod:
 
diyAudio Senior Member
Joined 2002
Bandwidth.

Hi,


yes all people ...no one can hear to 1MHz....basic biology...old chap......basically...20Hz-20KHz

So, you conclude that since people can't hear, measurably, as you say, beyond that 15Hz to 22KHz bandwidth they automatically can't discern what's going on beyond and above those frequencies?

Ears are really good diff amps and no two amps will ever sound alike to me, even if they measure the same, well, measured with our conventional tools that is.

As said before, measure all you like...I know it doesn't tell the whole story....not by a long shot.

BTW and FWIW, I absolutely agree with JCs' testing approach.
Moreover, I think Eric would agree also, as would Peter Daniel IMO as would other members.

Cheers,;)
 
Very Limited Listening Responses.....

The established limit of direct hearing response is 15kHz-20 kHz or so.
Above this bone conduction comes into play, and these 'ultrasonic' frequencies are sensed, rather than directly heard.

To you 'objectivist only' guys, go read some of the stuff according to Rupert Neve, who talks at length about this and the need for recording and replay systems to be responsive out to 40+kHz.
The null testing showing lack of matching at high frequencies would tend to reinforce this.
Ears are really good diff amps and no two amps will ever sound alike to me, even if they measure the same, well, measured with our conventional tools that is.
Yeah, I repair amplifiers everyday, in fact I have repaired thousands of amplifiers, and they all have their own sonic characters.
BTW and FWIW, I absolutely agree with JCs' testing approach.
Moreover, I think Eric would agree also, as would Peter Daniel IMO as would other members.
Yup, I already said so. ;)
I just do not agree with the assertions made by these 'objective' adherents.
Frank, do you notice how they are getting dismissive and abusive, and consistently ignore relevant questions ?.

Eric.

"But what if I don't WANT it to sound like the orchestra is in the room?!?"
Errr, just get her a K-mart shelf system, $2 shop headphones, and a bottle of red. ;)
 
Timed machinery

Pan said:
I think the closest we come to jitter in analog gear is a turntable or tape machine with "wow and flutter", or simply unstable time factor of the moving part.

/Peter

I understand jitter to be a time displacement of when a waveform is played back vs. when it is supposed to played. I don't know how it is measured, but I would guess it is a max variance of X microsec/bit - bit may be delivered to the DAC early or late by X microsec.

So when you say slew rates are not similar to jitter, is this because the slew rates in opamp, transistors and tubes are so high as to be assumed to be vertical. I don't see that as true...

Since one of the reasons why the digital sample rate is 44.1Khz+ is so that we can get a factor of 2 vs. top end freq., it follows that having a slew rate of 1V/microsec would give a "resolution" of about 500kV/sec. If I extrapolate this to something like a 70V PA system, then, the 500kV/sec gives us maximum voltage at about .000145sec, which is very slow or about 7Khz.

Put another way, if I put a perfect sine wave of 50Hz at the amp input with a +/-2V peak and a gain of 10, the amp must output +/- 20V at 50Hz. With a slew rate of 1V/microsec we get 20microsecs or 1/50,000 sec to deliver +20V. The wave at 50Hz need to get to peak or 20V in 1/50,000 of sec or .001 cycle. Now if we assume that the amp is linear <.1% then input=output. But I don't assume that slew rate is perfect so there must be some difference between input and output.

It follows that a slew rate of 4V/microsec for the above amp is the equivalent of a 4X oversample at 50Hz.

Here is the bad news. At 500Hz, you need slew of 5V/microsec to get equivalent and at 5000Hz you need 50V/microsec, which is very, very high.

If you lower your gain to 5, you only need 25V/microsec at 5000Hz and so forth.

:)ensen.

Note: Math originally off by 1000. Corrected <5 minutes.
 
When A Square Is Not A Square.

Hi Purplepeople ,
You are correct in your maths, however, from bandlimited (20 kHz) vinyl or digital recorded audio, you are never going to get a 20 kHz square wave, or a rectangular pulse corresponding to 20 kHz rise-time.
If you assume (and this is an approximation only) that the fundamental and odd harmonics up to the fifth will reasonably reconstruct a square wave, then 4 kHz is the upper limit for 44.1 kHz digital systems.
IOW, a cdp can never properly reproduce say, a 10 kHz rise-time pulse, and neither can vinyl for that matter.

In the March 2003 EW/WW magazine is an article worth reading - "Fourier Analysis - Defeating The Gibbs Phenomenon" that shows that the coventional regard that a fundamental plus odd order harmonics will approximate a square wave is wrong, even if including harmonics up to the 111th, and causes and will always cause 8.9% overshoot.
Addition of harmonics speeds the rise-time but does not change the overshoot condition.

Hope this helps.

Eric.
 
Mikek:

>no one can hear to 1MHz....basic biology...old chap......basically...20Hz-20KHz..exceptionaly, 15Hz-22KHz<

Most people can "hear" far lower than 15Hz, albeit not as an independent signal. If you take a 1kHz tone and amplitude-modulate it with a 0.5Hz sine wave, the result will be a repetitive crescendo-decrescendo pattern that cycles at the rate of 30 times a minute.

regards, jonathan carr
 
Eric:

>I conclude that you are both Objective and Subjective<

Probably true. I am objective because I enjoy being so, and because this is the ideal way - the way that I wish the world would operate all the time. I need to be subjective because staying wholely objective leaves too many messy details to ignore. Also, most audiophiles' reactions to audio products are primarily subjective, and if I were to totally ignore this fact, I likely wouldn't be able to sell any product.

OTOH, in many ways, I do enjoy being subjective. It certainly feels more relaxed than when I am being objective. Also, when my modus operandi is predominantly objective, I find it rather harder to blend into daily society, and I end up offending other people much more easily. Whatever.

Theory and objectivity drive my designs to a vastly greater extent than subjectivity, although I will not deny that the choice of which theory or which area to focus my energies on may in fact be a subjective call. I do listen, but I always insure that other, visually identical units that contain different guts are present in the listening test. I believe that a phrase that I frequently use is "just to keep ourselves honest."

>Have you found any surprises using your mode of blind testing<

"Surprise" is contigent on "expectation", is it not?

>or indeed correlations with your own subjective findings for your products?<

Sometimes yes, sometimes no. If I listen to something, but don't hear a consistent, repeatable difference, at the end of the day I will go with the configuration that theory, calculation and measurements suggest is the correct choice.

regards, jonathan carr
 
In the March 2003 EW/WW magazine is an article worth reading - "Fourier Analysis - Defeating The Gibbs Phenomenon" that shows that the coventional regard that a fundamental plus odd order harmonics will approximate a square wave is wrong, even if including harmonics up to the 111th, and causes and will always cause 8.9% overshoot.

This holds for some but not all cases, specifically the case where upper harmonics are totally cut off with no phase correction/precorrection or interpolation.

If you want overshoot and ringing, high-end moving coil phono cartridges will give you plenty.
 
Status
Not open for further replies.