Another Objective vs Subjective debate thread - diyAudio
Go Back   Home > Forums > General Interest > Everything Else

Everything Else Anything related to audio / video / electronics etc) BUT remember- we have many new forums where your thread may now fit! .... Parts, Equipment & Tools, Construction Tips, Software Tools......

Please consider donating to help us continue to serve you.

Ads on/off / Custom Title / More PMs / More album space / Advanced printing & mass image saving
Reply
 
Thread Tools Search this Thread
Old 23rd June 2011, 10:15 PM   #1
just another
diyAudio Moderator
 
wintermute's Avatar
 
Join Date: Aug 2003
Location: Sydney
Blog Entries: 22
Exclamation Another Objective vs Subjective debate thread

I've decided to start this thread, with the hope that if there is a thread which the entire reason for existence is the debate of topics relating to what can be measured and what can be heard, that hopefully people will use this place to get their viewpoints across rather than doing so (and derailing in the process) in other threads where the debate may not be welcome.

By all means still question posts in other threads, but please feel free to link (from here) to other threads or link from other threads to here (and try and steer any off topic debate from other threads to here).

I personally sit somewhere in the middle, with a lean towards the objective camp. I do feel that there are quite probably some things that we can hear that we don't yet have measurements that show this. But I also strongly believe that there are things that we think we hear, that aren't actually there as well.

Anyway hopefully I won't regret having started this thread. Please remember that it is ideas you are debating. No matter how much you disagree with someone it is there idea/opinion that you are disagreeing with. Don't get personal

Some will think that starting a thread like this is a waste of peoples time and forum bandwidth. This may be true, but anyone who chooses to participate does so of their own volition. You can ignore this thread if you want to


Tony.
__________________
Any intelligence I may appear to have is purely artificial!
Some of my photos
  Reply With Quote
Old 23rd June 2011, 10:48 PM   #2
diyAudio Member
 
Radugazon's Avatar
 
Join Date: Aug 2010
Location: Sulawesi
First customer...

I agree that probably measurements dont say all, and that the subjectivists claims can't be always systematically wrong.

My trouble is once one of these tiny subtle things have been detected by hearing only, how can the subjectivist know that this is a decisive and repeatable improvement ? I have a good experience of A/B testing with a digital filter, but honestly I wonder how to choose if I dont see some chart by my eyes. Bias is a trap.

Our hearing sense is very elaborated (and not well known yet, for sure), but all the primates are mainly visually determinated. The concerned cortical areas are a proof of this. Measurements with visual representation are probably imperfect but I guess they do 99% of the job.
  Reply With Quote
Old 23rd June 2011, 10:55 PM   #3
diyAudio Member
 
Steve Eddy's Avatar
 
Join Date: Sep 2002
Location: Sacramento, CA
There is no "Objective vs Subjective" debate.

There is nothing to debate between true objectivists and true subjectivists.

The debate has always been between pseudo-objectivists and objectivists. And that's how it should be framed, instead of lumping true subjectivists in with the pseudo-objectivists.

se
__________________
The Audio Guild
  Reply With Quote
Old 24th June 2011, 05:00 AM   #4
Previously: Kuei Yang Wang
 
Join Date: Nov 2002
Hi,

The problems are many.

For example, an engineer likes to have a goal that is easily verifiable.

If I tell him: "Design an Amplifier with less than 0.001% THD at 20KHz and DC-1MHz bandwidth and 500W output power." the engineer will be happy and plug ahead.

If I tell him: "Make me an amplifier that sounds good." his first question will be: "How do I measure when it sounds good?"

From there we move on to Double Blind Tests in attempt to measure, except of course they do the tests without verification and calibration and are then not surprised (neither am I incidentally) when the tests return "everything sounds the same" because bad statistics where applied, yet this not realised and any well meaning criticism of the method is taken badly...

Meanwhile the GEB (Golden Ear Brigade) find when listening to the Amplifiers that just because an Amp has lower THD it does not reliably sound better than one with higher THD. So they actually quite rightly feel that THD measurements are meaningless as guide of sound quality or fidelity.

In fact since Olson and D.E.L. Shorter of the BBC in the 1950's and so farcumulating in the Geddes/Lee distortion metric serious points, backed by serious blind listening tests that THD as single dimensional number is useless.

Tests showed that for some conditions (mainly the spectral distribution of harmonics) several percent of THD are not distinguishable in blind tests from undistorted signals with music, while small amounts (<< 0.1%) of the right kind of distortion are not only extremely audible, but considered by all listeners extremely obnoxious.

In an ideal world of course Engineers and Scientists alike would rush to take advantage of such improved methodes to measure, instead we see severe resistance to their adoption and not just because it is a little more difficult than pressing the "run" button an AP2.

From there (THD is meaningless) it is but a small step to completely reject measurements (as non seem to correlate well with "good sound"), which of course upsets the engineers, who cling to their measurements like a baby to it's mother and hence leads to challenges from them "prove to me what you are hearing", which is also understandable.

Worst, in many cases both sides mean quite well. Not for nothing do they say that the way to hell is paved with good intentions.

The problem is very much that of the blind men and the elephant.

Click the image to open in full size.

All sides in this debate have parts of the whole picture, so to speak a partly idea of the whole elephant in the room, however, instead of comparing notes and learning from each other so they can fully percieve the Elephant in the room, theyr prefer to ignore the Elephant entierly and prefer to shout at each other and accuse the other of lack of reason, murder or want of sense and the like, as they clearly do not agree with the very reasonable (or not I feign) view held by whoever is speaking.

If and when I have made an amplifier that measures better than a previous one (I did this in the 80's) but I am being told by experienced listeners that it sounds much worse than the older one, I would be ill advised to tell my listeners that they are only imagining things, should not tell me such a thing until they have completed a double blind test and generally to depreciate their viewpoint.

Instead it would probably behove me well to listen and to try to understand what is happening with an open mind, instead of an a priori "I'm right, this Amp is better and they are old fools".

"Moral:

So oft in theologic wars,
The disputants, I ween,
Rail on in utter ignorance
Of what each other mean,
And prate about an Elephant
Not one of them has seen!"

Ciao T

PS, for those who would like to enjoy the whole poem, it is here:

Blind Men and the Elephant - Word Information
  Reply With Quote
Old 24th June 2011, 05:35 AM   #5
Previously: Kuei Yang Wang
 
Join Date: Nov 2002
Hi,

Quote:
Originally Posted by Radugazon View Post
I have a good experience of A/B testing with a digital filter, but honestly I wonder how to choose if I dont see some chart by my eyes. Bias is a trap.
It sure is, however it is CRITICALLY IMPORTANT to remember that Bias works both ways.

Subjectivists may be prejudiced to hear small differences where non exist (such a bias BTW will throw of a small scale blind test under the ABX protocol where the whole sample is made up of subjectivists) and objectivists may be prejudiced to not hear small differences that in fact actually exist (such a bias BTW will throw of a small scale blind test under the ABX protocol where the whole sample is made up of objectivists).

I would suggest that for any blind test to beat expectation bias it requires either very large numbers of participants (it also needs these to aquire statistical power, that is to able to be held relevant in the broader world), or it needs to performed in a way that eliminates expectation bias.

There are a number of ways of doing this "elimination", among them would be a more sophisticated way to judge the difference, that is instead of making just the AB choice in ABX testing, use a wide range of factors which are rated and compared to the actual sequence in which the items where presented, then evaluating these more complex returns using a confidence interval, or by obscuring the nature of the difference being evaluated to name two.

Quote:
Originally Posted by Radugazon View Post
Our hearing sense is very elaborated (and not well known yet, for sure), but all the primates are mainly visually determinated. The concerned cortical areas are a proof of this. Measurements with visual representation are probably imperfect but I guess they do 99% of the job.
The question is, 99% of what job?

And are you suggesting that in principle it should be possible to devise tests and protocols that will allow 99% of "good sound" to be measured, or are you suggesting that current extant and widely practised measurements provide 99% of what we need to know about "good sound", plus if so, would care to detail what these are?

Ciao T
  Reply With Quote
Old 24th June 2011, 05:39 AM   #6
ra7 is offline ra7  United States
diyAudio Member
 
Join Date: Feb 2009
Blog Entries: 1
At least this thread was graced by one beautiful post!

That was thoroughly enjoyable, Thorsten!

Edit: I meant the one before.
  Reply With Quote
Old 24th June 2011, 05:50 AM   #7
diyAudio Member
 
abraxalito's Avatar
 
Join Date: Sep 2007
Location: Hangzhou - Marco Polo's 'most beautiful city'. 700yrs is a long time though...
Blog Entries: 100
Send a message via MSN to abraxalito Send a message via Yahoo to abraxalito Send a message via Skype™ to abraxalito
Quote:
Originally Posted by Steve Eddy View Post
There is nothing to debate between true objectivists and true subjectivists.
Indeed not. There would merely be perfectly generative dialogue.

Quote:
The debate has always been between pseudo-objectivists and objectivists. And that's how it should be framed, instead of lumping true subjectivists in with the pseudo-objectivists.
No, the pseudo-objectivists object to being lumped in with the true subjectivists. That's why they're called 'objectivists' - they object. As a true subjectivist myself I don't object to this, I merely find it amusing.
__________________
When you design something for other people you don't have as much motivation to make it beyond excellent - Woz
  Reply With Quote
Old 24th June 2011, 06:16 AM   #8
diyAudio Member
 
Join Date: Apr 2004
Location: 12km off the alaska highway in northern BC
How can there even be a discussion between what one claims to hear subjectively and measured data used to assess the parametres of an electronic or electromechanical product?
I can only assess what I can measure, I simply cannot assess what you can hear.

The only way to assess what the subjectivist claims to hear is again by measuring it through the statistical evaluation of a blind test. A mechanism typically unacceptable to most subjectivist, or when accepted the results, when not according to subjectivist expectations, are heavily criticized as to validity of statistical evaluation or methodology despite prior agreeing to the test protocol and the statistical methods used.

The usual excuse however is the mythical "test stress" deteriorating the performance of the testee.

In light of this I can only see the discussion to be utterly fruitless and wasting all the participants time.

As an example - I was taken to task when I claimed that I was unable to hear a difference between amplifiers of vastly different designs bryston, technics switchmode amp, hypex, quad, sansui etc.) when driving speakers within the amps parameters and having specs below what it is said to be inaudible - or even borderline audible.
As a subjective statement nobody really has a right to criticize me - that is simply my experience, and an unwelcome one because I had uselessly spent 2500 on a bryston amp when spending 50$ on a used quad 405 had achieved the same result.
Just to demonstrate that any subjective discussion is nothing more than trading anecdotes - like telling the typical fishermen lies about the one that got away.

therefore - good by.

Last edited by audio-kraut; 24th June 2011 at 06:28 AM.
  Reply With Quote
Old 24th June 2011, 06:26 AM   #9
diyAudio Member
 
abraxalito's Avatar
 
Join Date: Sep 2007
Location: Hangzhou - Marco Polo's 'most beautiful city'. 700yrs is a long time though...
Blog Entries: 100
Send a message via MSN to abraxalito Send a message via Yahoo to abraxalito Send a message via Skype™ to abraxalito
Quote:
Originally Posted by audio-kraut View Post
I can only assess what I can measure, I simply cannot assess what you can hear.
There is the rub - the 'objectivist' does indeed ***** it - as worthless.

<edit> - did I spell the word 'assess' wrongly so I invoked the wrath of the spelling censor?
__________________
When you design something for other people you don't have as much motivation to make it beyond excellent - Woz
  Reply With Quote
Old 24th June 2011, 06:35 AM   #10
Previously: Kuei Yang Wang
 
Join Date: Nov 2002
Hi,

Quote:
Originally Posted by audio-kraut View Post
The only way to assess what the subjectivist claims to hear is again by measuring it through the statistical evaluation of a blind test.
This we agree.

Quote:
Originally Posted by audio-kraut View Post
A mechanism typically unacceptable to most subjectivist, or when accepted the results, when not according to subjectivist expectations, are heavily criticized as to validity of statistical evaluation or methodology despite prior agreeing to the test protocol and the statistical methods used.
Well, first, the criticism of the Statics came from Statisticians, though it tends to be quoted by subjectivists. The ABX Mafia decided not adjust their tests despite this, meaning their tests lack statistical power to claim any reasonable applicability.

Second, the criticism including Placebo/Nocebo comes from blind tests in other areas of science. Given the nature of the tests and the general strong emotional involvement of all participants these effects will be elevated. This means that much larger tests sets will be needed to overcome these. The ABX Mafia decided not adjust their tests despite this, meaning their tests lack statistical power to claim any reasonable applicability.

Quote:
Originally Posted by audio-kraut View Post
The usual excuse however is the mythical "test stress" deteriorating the performance of the testee.
Test stress is very real. I experienced it personally.

Our local audio club once set up a test between a stock Marantz CD Player and one I had modified (actually the very one I wrote about in TNT-Audio).

The test was blind, I sat with the rest of the audience. In a previous test at my home (also blind) that I just did for myself the differences where clear.

In the public test I failed to hear ANY difference. I mean ANY.

Funnily enough, about a dozend+ or so guys in the room, many in seats with a much poorer listening position than mine found the differences plain, obvious and more than 9 out of 10 preferred the modified player consistently.

I should add that the modified player was not appreciably different in THD and IMD, SNR or Frequency response, however the IMD and THD spectrum showed quite drastic differences in terms of "grass blades" above the noisefloor, when looked at, but where swamped in the single number IMD/THD tests.

Quote:
Originally Posted by audio-kraut View Post
As an example - I was taken to task when I claimed that I was unable to hear a difference between amplifiers of vastly different designs bryston, technics switchmode amp, hypex, quad, sansui etc.) when driving speakers within the amps parameters and having specs below what it is said to be inaudible - or even borderline audible.
Well, I would not take you to task for observing you did not hear any difference. I would take you to task if you therefore claimed that no audible differences can exist though, as this does not from your failure to observe such.

Quote:
Originally Posted by audio-kraut View Post
In light of this I can only see the discussion to be utterly fruitless and wasting all the participants time.

therefore - good by.
Good bye then.

Ciao T

Last edited by ThorstenL; 24th June 2011 at 06:38 AM.
  Reply With Quote

Reply


Hide this!Advertise here!
Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are Off
Pingbacks are Off
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
The OFFICIAL "Snail Shell" Thread!!! SkinnyBoy Subwoofers 158 4th May 2013 09:06 AM
more objective vrs subjective (from snubberized GC PS thread) Upupa Epops Everything Else 30 30th January 2005 08:55 PM


New To Site? Need Help?

All times are GMT. The time now is 03:23 AM.


vBulletin Optimisation provided by vB Optimise (Pro) - vBulletin Mods & Addons Copyright © 2014 DragonByte Technologies Ltd.
Copyright 1999-2014 diyAudio

Content Relevant URLs by vBSEO 3.3.2