Blind Listening Tests & Amplifiers

Status
Not open for further replies.
Disabled Account
Joined 2002
baxandal..

nw_avphile said:

Wasn't Mr Otala under contract (a consultant) with Harmon Kardon when he published that paper? Considering HK waved it around as The Next Big Thing in amplifier design in all their marketing materials one has to wonder about his motives for writing it when his paycheck was coming from HK--an amplifier manufacture?

Baxandal also pioneered the null difference test just before that period--a test that's very revealing of TIM, etc. under real-world conditions.


The baxandal series running from late '78 to early '79 in Wireless World, (now Electronics World), should be compulsory reading for anyone discussing feedback 'TIM' etc..in linear audio amps.

More Dr. cabot stuff from the audio/radio section...below...

download chapter 13.3 from:


http://www.tvhandbook.com/support/TVHB_updates.htm
 
Mark

I think we're actually arguing from the same side;)

The mere fact that the listener has an expectation that he is listening to one of the amps has a distorting effect on his perceptions of the test, let alone any bias set up by the question it's self.

Maybe the way to approach this is to have an extra, unknown amp C that is also randomly selected among with the A and B amps, and ask the question "what amp do you think you are listening to, A, B, or can't tell"

This will act as a lie scale, because if a test subject can hear differences then the "can't tell" answer will have an equal score to A and B, yet any bias or preconceptions on the subjects behalf will show as they will not deliberately select the "can't tell" answer.

BTW folks, this is only one methodology- there are others that are different, yet work the same way, out there, so don't think you know all the answers! :devily:
 
Blind realities...

One more very important bit of info on blind testing that's not been really discussed:

Let's for a moment, completely ignore the statistics, math, etc, involved with blind testing. A very revealing thing can come out of doing the MOST SIMPLE blind test: A new personal paradigm.

Most people I've encountered have never taken part in an audio blind test before. From comments here, that would seem to be consistent with this group as well (for you statisticians, "most" means more than 50%! ;) ). It can be a very eye opening experience even before you learn your "score".

In my experience, many audiophiles who are CERTAIN they hear obvious differences between two things are AMAZED they can't hear any difference once they don't know what they're listening to. With some tests, no level matching is required. A good example would be changing brands of capacitors, or even easier, changing interconnects.

It goes like this: Someone else connects either the $3 Radio Shack RCA interconnects or your $250 Kimber Silver ones at random, you play some music, the someone then does a known swap (not a random or fake one), you play some more music, and this amazing look spreads across the listener's face as they realize they sound the same! They throw up their arms and ask if you're really swapping as they can't be sure they hear ANY difference. The paradigm has shifted...

Now, if you're trying to publish a paper, write a book or convince a skeptic, THEN you need to drag out the statistics, do it by the numbers, and be able to point confidently at your test data. But if the listener is the honest open minded sort, and they verify that yup, you did indeed switch from their $250 interconnects to the $3 ones, and yup, they did sound pretty much the same, it's an eye (ear?) opening moment!

So... don't let the numbers and math scare you. Have your significant other, audio geek buddy, son, daughter, or whoever do some swaps from a random starting point and see if you can hear any difference. You can use the results of that simple exercise to see if more testing/research/whatever is justified.

In the end, I don't know what SY would say (as it sounds like he's gone through a similar "paradigm shift"), but for me, it's been relaxing to no longer stress over nearly as many things in my system compared to before my newfound knowledge. It honestly has improved my listening pleasure a bunch.

Prior to my paradigm shift, I was somewhat envious of not being able to afford The Really Good Stuff. But once you realize that, in a blind test, some of things you already have sound exactly the same as The Really Good Stuff, it makes you smile with newfound joy! You take great pleasure in the fact that say your amplifier sounds as good as that $8000 one you brought home from the dealer but returned when it flunked a blind test. Or, in a more DIY sense, your $2 capacitors sound the same as the ones GoldenEar spent $100 on.

But most of all, it allows me to simply focus on the MUSIC more without worrying nearly as much about the hardware. THAT, as American Express might say, is priceless.
 
Proof versus Significance

My apology for the boring stats bit. (especially since I have no formal training in stats) :headbash:

We are deviating slightly in this from what I believe NW was suggesting at the beginning.

The statement, that simple blinded listening by an individual may allow them to more objectively (by reducing psychological overlay) assess the merits of two devices (or one device + a mod) is still reasonable.

There is no burden of "scientific proof" here. The individual can simply make up their own mind as to if they believe the cost differential is justified, given their ability to tell things apart solely on sonic performance.

An individual is equally entitled to say, but I live in a subjective world, I accept psychological factors will influence what I hear, I am happy to accept this and moreover believe it is a part of my overall experience. A good example would be me, I like listening to my Aleph5 more than the 4, because I did a better job, it is neater, etc. I accept this, however I have no illusion that I could tell them apart blindly.

The important bit here, is again there has been no proof, it is simply the subjective impressions of the individual concerned. So long as these recommendations are seen within their scope of evidence, fine.

I strongly think NW's suggestion and intent were well placed.

Where I do get slightly "pissed-off" is where I see commercial manufacturers claiming wildly unbelievable benefits from things where there is (1) no commonsense reason why, and (2) without a shred of robust scientific analysis.

The later is where I see "proper studies" and real stats having their place.

The suggestion that a few people actually try it for themselves would, I suspect, be educational.

mark
 
diyAudio Senior Member
Joined 2002
Hold It...

Hi,

It goes like this: Someone else connects either the $3 Radio Shack RCA interconnects or your $250 Kimber Silver ones at random, you play some music, the someone then does a known swap (not a random or fake one), you play some more music, and this amazing look spreads across the listener's face as they realize they sound the same! They throw up their arms and ask if you're really swapping as they can't be sure they hear ANY difference. The paradigm has shifted...

Get yourself a cheap transisssstor radio and be happy... or you could get your ears checked...while you're at it, have your brainfunctions checked too, I start to worry.

I mean how difficult can the listening to and swapping of cables be?

Are the Ratshack cables really the worst you could imagine? How about a mile of nichrome wire?

Sorry but this is more and more starting to look like a serious waste of time to me.

Nothing personal, just an observation.:(
 
diyAudio Senior Member
Joined 2002
PARADIGMS AND OXYMORONS.

Hi,

But most of all, it allows me to simply focus on the MUSIC more without worrying nearly as much about the hardware. THAT, as American Express might say, is priceless.

Yes, the moment you give up searching you'll end up enjoying the music more, less stress listening etc.

OTOH, I would get bored living with the imperfections of the system I listen to...guess it's just me.:)

Cheers,;)
 
Re: HOLD IT...

fdegrove said:
Get yourself a cheap transisssstor radio and be happy... or you could get your ears checked...while you're at it, have your brainfunctions checked too, I start to worry.
Here come the personal attacks again! Was there ANYTHING in my post aimed at you Fred to justify calling me deaf and stupid?


fdegrove said:
I mean how difficult can the listening to and swapping of cables be?
Exactly, so what's the harm in trying it blind?


fdegrove said:
Sorry but this is more and starting to look like a seious waste of time to me.

Nothing personal, just an observation.:(
Well the first part of your reply was very personal, and if you consider it a waste of time, you're welcome to just ignore this entire thread. It doesn't seem to apply to you anyway.
 
diyAudio Senior Member
Joined 2002
Uh?

Hi,

Was there ANYTHING in my post aimed at you Fred to justify calling me deaf and stupid?

Dear man, it wasn't personal at all. And yes, I do start to worry when you make it personal...you confuse Fred and myself (I feel kinda flattered here) which is why I suggest to give it a rest.

All this thread has done so far is to conveniently disregard any counterarguments and from what I observe you're only trying to convince yourself that all the effort spent at making things better is just not worth it.

Fine by me, but why do you need to convince other people of that lazy philosophy?

That, is a decision you have to make for yourself and as such is personal.

Cheers and do some soulsearching please,;)
 
Is Anybody Actually Listening Properly ?

"Now, if you're trying to publish a paper, write a book or convince a skeptic, THEN you need to drag out the statistics, do it by the numbers, and be able to point confidently at your test data. But if the listener is the honest open minded sort, and they verify that yup, you did indeed switch from their $250 interconnects to the $3 ones, and yup, they did sound pretty much the same, it's an eye (ear?) opening moment!"

Sure, due to electrical values being similar, expect interconnects to sound pretty much the same overall.

However the sonic differences percieved by experienced listeners are not due to Frequency Response changes per se, but more so due to fine changes in spectral/impulse response due to differing conductors and dielectrics mostly.

These changes require familiarity with the system and the music, and these sonics changes can require extendes listening time in order to correlate/diffrentiate them.
With practice this period can become short, but for some listeners this is never going to happen mainly I suspect because of listening in a wrong mode, or lack of hearing accuity.

Picking sonics differences in interconnects is a bit like recognising voices coming down a telephone line.
Once you have heard and imprinted the characters of a particular set of voices, you are then able to reliably discern which voice is coming down the line.

I have no trouble doing this over the telephone, and I have no trouble differentiating interconnects and other kit and modifications.

I will admit to cheating somewhat in this area, because since first noticing fine sonic changes I have spent much time investigating and understanding the causes and natures, I have found an interesting way of completely cleaning system sound, and understanding this technique has indeed sharpened my hearing accuity and I now easily hear spectral/impulse changes in an instant.

When you understand this concept, you can then at will 'effect' system sound and predictably get the sonic results desired.
I can now get 'vinyl' sound from a cdp at will, but I would say that vinyl sound is flawed and coloured, but not neccsarily objectionably so, but when it comes down to it I much prefer dead clean sound.

Studio condenser microphone sound can be cleaned or 'effected' too, and this is a perfectly revealing test that most 'audiophiles' never get the opportunity to try, and it is this sort of learning that is required to become an expert listener.

As Bernard stated yonks ago, we have a very powerful DSP between our ears that requires training, and once trained becomes a very discriminating and reliable over time.

It seems to me that some listeners have bugs and loops in their dsp programming.

Eric.
 
Not again ......

And Frank says .... Get yourself a cheap transisssstor radio and be happy... or you could get your ears checked...while you're at it, have your brainfunctions checked too, I start to worry.

We're back to the sandpit again. :bawling:

Putting forward obtuse arguments, bordering upon insulting is counterproductive to the discussion and does nothing but diminish people's respect for the respective author.

This comment applies equally to both sides of the camp.

Lets keep sensible, please :(

mark
 
Re: Giving it a rest

nw_avphile said:

I hope some of you got something out of the blind/null discussions and will perhaps be encouraged to try some blind and/or null testing of your own. I'm going to take fdegrove's advice and give it a rest for now.

Have fun everyone.

Here's something I tried and you could try also...
Take an amplifier that works well without NFB (it's usually tube amp) and have a listen. Install switchable NFB loop in it (let's say -just 6 dB). Have somebody switch NFB in and out (and adjust volume) while you are listening to some piano music. Listen if there is any difference ;)
 
You call that blind test?!

Hi, am I missing something here?
If you want someone to do a blind test on two amps, he shouldn't know what is playing.
That man had a Bryston amp, and he knew you were comparing that to an Onkyo AV receiver.
Of course his mind was biased into saying he didn't like what he was hearing.
I really think that the Onkyo is crap, and if you really think that it really compares to any Bryston amp, my god!:devily:
Another thing:
Are you comparing power amps, integrated amps or what?
I can show you my Nad Receiver 7000 (not AV!) playing with it's original crappy JRC op-amps in the pre seccion, and then with OPA2604 from Burr Brown.
If you can't hear the difference, you can try some of these:

http://store2.yimg.com/I/safetycentral_1724_5943551

If it doen't work, then ask for medical help.
Or do you think that crappy Onkyo has OPA627s?
Ha!:clown:
It's people like you that muck up the Hi-Fi business.
You're the type of salesman that today is selling hi-fi, and tomorrow is selling socks.:nod:
 
Re: You call that blind test?!

First of all, thanks pinkmouse for doing some house cleaning here to keep this thread a bit more on-topic.

Second, it looks like we have a new contributor... carlosfm. Thanks for joining this controversial debate. It would appear you haven't read much of the thread so I'll try to bring you up to speed a bit :)

carlosfm said:
Hi, am I missing something here?
If you want someone to do a blind test on two amps, he shouldn't know what is playing. That man had a Bryston amp, and he knew you were comparing that to an Onkyo AV receiver.
Of course his mind was biased into saying he didn't like what he was hearing.
The vast majority of blind tests are done with full knowledge of the components involved. Most of the fancy ones are done with an "ABX" comparator that allows the listener to select A (the Bryston), B (The Onkyo) or X (A or B chosen at random). They know what A and B are. They're welcome to listen to each as much as they want. When they choose X, they only have to decide if it sounds more like A or B. Sounds easy, right? Well Tom Nousaine has documented ABX tests for a number of things from cables to CD players to amplifiers and the results are really interesting! He shattered all sorts of audiophile "beliefs" with his published articles on the subject.

My Onkyo/Bryston test didn't use an ABX comparator, because some audiophiles argue that putting a few relay contacts in the signal path somehow masks all the other differences you're trying to hear. So I swapped cables.

The main purpose behind the sort of blind test we're discussing here is to determine if any difference can be heard between two components . If there IS a difference, then the listener can decide which one they like better if they want. But the main point of the test is to establish if they can even reliably hear a difference.


carlosfm said:
I really think that the Onkyo is crap, and if you really think that it really compares to any Bryston amp, my god!:devily: .
First of all, just to be clear, I'm NOT suggesting the Onkyo is the equal of the Bryston in every way. The Bryston can certainly play louder, drive lower impedance loads better, etc. But keeping within the limits of both amps, in this particular high-end system, with this particular golden eared musician listening, he couldn't reliably hear any difference and admitted as such.

Lots of audiophiles, including the guy who owned the Bryston, thought the Onkyo should sound like crap. They thought the same thing about the cheap Carver amp that went up against two $10,000+ monoblock tube amps in the Carver/Stereophile challenge. They thought the same thing about the cheap Yamaha integrated that went up against two $15,000 monoblock amps in the Sunshine Audio challenge. In all those cases, "golden eared" audiophiles, listening on very high-end, high resolution systems, could NOT hear a statistically significant difference (that is their guesses were close to being random) between the cheap mainstream component and the much more expensive high-end component in blind tests.

How do you explain the above? Don't you think the editors of Stereophile (they participated in the Carver blind test) would have done just about anything to not lose the challenge? The same is true for the owner of a very high end dealer--Sunshine Audio in Florida. None of these people have come up with any good excuses as to why they couldn't hear a difference.

carlosfm said:
Another thing:
Are you comparing power amps, integrated amps or what?
Amplifiers of any kind. The Onkyo/Bryston test should have favored the Bryston even more as we were not just comparing the power amps, but the whole signal chain from the output of the CD player to the input to the speakers. So it was also the Bryston preamp up against the internal preamp in the receiver.

carlosfm said:
I can show you my Nad Receiver 7000 (not AV!) playing with it's original crappy JRC op-amps in the pre seccion, and then with OPA2604 from Burr Brown.
I'm not saying your op-amps don't make a difference, but it doesn't sound like you have done a blind test either (which in this case would be hard to do)?

Basically you spend a lot of time, effort, perhaps money, etc. to swap out op-amps. So right there your mind is already biased towards wanting to hear something in exchange for what you've put into the project. Further, you're biased by what others have said about the op-amps (which is likely what made you buy OPA2604's in the first place). So when you solder them in and listen to them, you're STRONGLY BIASED towards hearing some kind of difference. This psychological bias has been very well documented. It's just human nature. It's why they conduct serious wine tastings with the labels hidden. See this post for further details on the psychological effects I'm talking about:

http://www.diyaudio.com/forums/showthread.php?postid=150164#post150164

Here are some references to much of what's talked about above:

Nousaine, Thomas, "Wired Wisdom: The Great Chicago Cable Caper", Sound and Vision, Vol. 11 No. 3 (1995)

Nousaine, Thomas, "Flying Blind: The Case Against Long Term Testing", Audio, pp. 26-30, Vol. 81 No. 3 (March 1997)

Nousaine, Thomas, "Can You Trust Your Ears?", Stereo Review, pp. 53-55, Vol. 62 No. 8 (August 1997)

Toole, Floyd E., "Listening Tests - Turning Opinion Into Fact", Journal of the Audio Engineering Society, Vol. 30, No. 6, June 1982, pp. 431-445.

Toole, Floyd E., and Olive, Sean E., "Hearing is Believing vs. Believing is Hearing: Blind vs. Sighted Tests, and Other Interesting Things", 97th AES Convention (San Francisco, Nov. 10-13, 1994), [3893 (H-5], 20 pages.


The other half of this thread involves subtracting the input of any amplifier from the output while it's playing real music driving real speakers. This is known as null difference testing and what you're left with is ANY AND ALL distortion, errors, etc. produced by the amplifier. This includes phase error, TIM, frequency response problems, THD, IM, errors caused by the NFB, etc, etc. Anything that causes the signal driving the speakers to deviate from the input signal shows up in this test.

So I ask you this question: If you perform the null difference test on your stock NAD and get an "error signature" playing your favorite music driving your favorite speakers. Then you swap out the op-amps and repeat the test. If there's no difference, doesn't that mean both op-amps should sound the same?

The whole point of this thread, which even a few of the devoted golden eared folks have admitted, is there's an unavoidable psychological bias when you do regular non-blind listening.

carlosfm said:
It's people like you that muck up the Hi-Fi business.
You're the type of salesman that today is selling hi-fi, and tomorrow is selling socks.:nod:

I'm not selling anything... I used to sell high-end audio and would have been on your side, but some blind listening tests, and further research, changed all that. Now I'm only encouraging folks to listen in a way that removes psychological bias, and pointing out a kind of testing that's very useful when working on amplifiers.

If you read through this rather lengthy thread, you'll find there's a lot of objective evidence supporting what I've said above and very very little objective evidence against it. Mostly the golden ear types have resorted to insults, changing the subject or very weak technical arguments that have easily been refuted. What's being discussed here undermines some of what many audiophiles believe in so it's hardly surprising some folks get really defensive--just like you did in your post (i.e. suggesting I need medical help, etc.).

But you're welcome to read through the thread and make up your mind. I'm not twisting anyone's arm, just trying to keep things factual and objective and bring an alternative perspective to high-end audio.
 
The null in op amps world...

If people try a null test betwen the Ne5534 and the new fashionable op amps...they will be surprised...

In my CD Sony XA50 ES i have changed the opa 2132 for the NE...with superlatif resuslts...

Just the oposite of the other people do...:bigeyes:

I´m not realy a folower of fashion!!:goodbad:
 
Ex-Moderator
Joined 2002
Re: Another Way ?.

mrfeedback said:
Hi Al,
How about copying those posts to a new thread, and insert links into posts in the the original thread stating that this sub- discussion has been moved (copied), and thereby allowing the original thread to continue unedited, and continuous.
Any errant posters would be soon redirected to the relevant thread discussion.

Eric.

Hi Eric

It's on my mission list, but sorting the thread initially took 45min, and I ran out of time as I had to go to work!

I will endevour to do it in the next couple of days...;)
 
Status
Not open for further replies.