Another Objective vs Subjective debate thread

Status
This old topic is closed. If you want to reopen this topic, contact a moderator using the "Report Post" button.
Just another Moderator
Joined 2003
Paid Member
I've decided to start this thread, with the hope that if there is a thread which the entire reason for existence is the debate of topics relating to what can be measured and what can be heard, that hopefully people will use this place to get their viewpoints across rather than doing so (and derailing in the process) in other threads where the debate may not be welcome.

By all means still question posts in other threads, but please feel free to link (from here) to other threads or link from other threads to here (and try and steer any off topic debate from other threads to here).

I personally sit somewhere in the middle, with a lean towards the objective camp. I do feel that there are quite probably some things that we can hear that we don't yet have measurements that show this. But I also strongly believe that there are things that we think we hear, that aren't actually there as well.

Anyway hopefully I won't regret having started this thread. Please remember that it is ideas you are debating. No matter how much you disagree with someone it is there idea/opinion that you are disagreeing with. Don't get personal :)

Some will think that starting a thread like this is a waste of peoples time and forum bandwidth. This may be true, but anyone who chooses to participate does so of their own volition. You can ignore this thread if you want to :)


Tony.
 
First customer...

I agree that probably measurements dont say all, and that the subjectivists claims can't be always systematically wrong.

My trouble is once one of these tiny subtle things have been detected by hearing only, how can the subjectivist know that this is a decisive and repeatable improvement ? I have a good experience of A/B testing with a digital filter, but honestly I wonder how to choose if I dont see some chart by my eyes. Bias is a trap.

Our hearing sense is very elaborated (and not well known yet, for sure), but all the primates are mainly visually determinated. The concerned cortical areas are a proof of this. Measurements with visual representation are probably imperfect but I guess they do 99% of the job.
 
Hi,

The problems are many.

For example, an engineer likes to have a goal that is easily verifiable.

If I tell him: "Design an Amplifier with less than 0.001% THD at 20KHz and DC-1MHz bandwidth and 500W output power." the engineer will be happy and plug ahead.

If I tell him: "Make me an amplifier that sounds good." his first question will be: "How do I measure when it sounds good?"

From there we move on to Double Blind Tests in attempt to measure, except of course they do the tests without verification and calibration and are then not surprised (neither am I incidentally) when the tests return "everything sounds the same" because bad statistics where applied, yet this not realised and any well meaning criticism of the method is taken badly...

Meanwhile the GEB (Golden Ear Brigade) find when listening to the Amplifiers that just because an Amp has lower THD it does not reliably sound better than one with higher THD. So they actually quite rightly feel that THD measurements are meaningless as guide of sound quality or fidelity.

In fact since Olson and D.E.L. Shorter of the BBC in the 1950's and so farcumulating in the Geddes/Lee distortion metric serious points, backed by serious blind listening tests that THD as single dimensional number is useless.

Tests showed that for some conditions (mainly the spectral distribution of harmonics) several percent of THD are not distinguishable in blind tests from undistorted signals with music, while small amounts (<< 0.1%) of the right kind of distortion are not only extremely audible, but considered by all listeners extremely obnoxious.

In an ideal world of course Engineers and Scientists alike would rush to take advantage of such improved methodes to measure, instead we see severe resistance to their adoption and not just because it is a little more difficult than pressing the "run" button an AP2.

From there (THD is meaningless) it is but a small step to completely reject measurements (as non seem to correlate well with "good sound"), which of course upsets the engineers, who cling to their measurements like a baby to it's mother and hence leads to challenges from them "prove to me what you are hearing", which is also understandable.

Worst, in many cases both sides mean quite well. Not for nothing do they say that the way to hell is paved with good intentions.

The problem is very much that of the blind men and the elephant.

An externally hosted image should be here but it was not working when we last tested it.


All sides in this debate have parts of the whole picture, so to speak a partly idea of the whole elephant in the room, however, instead of comparing notes and learning from each other so they can fully percieve the Elephant in the room, theyr prefer to ignore the Elephant entierly and prefer to shout at each other and accuse the other of lack of reason, murder or want of sense and the like, as they clearly do not agree with the very reasonable (or not I feign) view held by whoever is speaking.

If and when I have made an amplifier that measures better than a previous one (I did this in the 80's) but I am being told by experienced listeners that it sounds much worse than the older one, I would be ill advised to tell my listeners that they are only imagining things, should not tell me such a thing until they have completed a double blind test and generally to depreciate their viewpoint.

Instead it would probably behove me well to listen and to try to understand what is happening with an open mind, instead of an a priori "I'm right, this Amp is better and they are old fools".

"Moral:

So oft in theologic wars,
The disputants, I ween,
Rail on in utter ignorance
Of what each other mean,
And prate about an Elephant
Not one of them has seen!"

Ciao T

PS, for those who would like to enjoy the whole poem, it is here:

Blind Men and the Elephant - Word Information
 
Hi,

I have a good experience of A/B testing with a digital filter, but honestly I wonder how to choose if I dont see some chart by my eyes. Bias is a trap.

It sure is, however it is CRITICALLY IMPORTANT to remember that Bias works both ways.

Subjectivists may be prejudiced to hear small differences where non exist (such a bias BTW will throw of a small scale blind test under the ABX protocol where the whole sample is made up of subjectivists) and objectivists may be prejudiced to not hear small differences that in fact actually exist (such a bias BTW will throw of a small scale blind test under the ABX protocol where the whole sample is made up of objectivists).

I would suggest that for any blind test to beat expectation bias it requires either very large numbers of participants (it also needs these to aquire statistical power, that is to able to be held relevant in the broader world), or it needs to performed in a way that eliminates expectation bias.

There are a number of ways of doing this "elimination", among them would be a more sophisticated way to judge the difference, that is instead of making just the AB choice in ABX testing, use a wide range of factors which are rated and compared to the actual sequence in which the items where presented, then evaluating these more complex returns using a confidence interval, or by obscuring the nature of the difference being evaluated to name two.

Our hearing sense is very elaborated (and not well known yet, for sure), but all the primates are mainly visually determinated. The concerned cortical areas are a proof of this. Measurements with visual representation are probably imperfect but I guess they do 99% of the job.

The question is, 99% of what job?

And are you suggesting that in principle it should be possible to devise tests and protocols that will allow 99% of "good sound" to be measured, or are you suggesting that current extant and widely practised measurements provide 99% of what we need to know about "good sound", plus if so, would care to detail what these are?

Ciao T
 
There is nothing to debate between true objectivists and true subjectivists.

Indeed not. There would merely be perfectly generative dialogue.

The debate has always been between pseudo-objectivists and objectivists. And that's how it should be framed, instead of lumping true subjectivists in with the pseudo-objectivists.

No, the pseudo-objectivists object to being lumped in with the true subjectivists. That's why they're called 'objectivists' - they object. As a true subjectivist myself I don't object to this, I merely find it amusing.
 
How can there even be a discussion between what one claims to hear subjectively and measured data used to assess the parametres of an electronic or electromechanical product?
I can only assess what I can measure, I simply cannot assess what you can hear.

The only way to assess what the subjectivist claims to hear is again by measuring it through the statistical evaluation of a blind test. A mechanism typically unacceptable to most subjectivist, or when accepted the results, when not according to subjectivist expectations, are heavily criticized as to validity of statistical evaluation or methodology despite prior agreeing to the test protocol and the statistical methods used.

The usual excuse however is the mythical "test stress" deteriorating the performance of the testee.

In light of this I can only see the discussion to be utterly fruitless and wasting all the participants time.

As an example - I was taken to task when I claimed that I was unable to hear a difference between amplifiers of vastly different designs bryston, technics switchmode amp, hypex, quad, sansui etc.) when driving speakers within the amps parameters and having specs below what it is said to be inaudible - or even borderline audible.
As a subjective statement nobody really has a right to criticize me - that is simply my experience, and an unwelcome one because I had uselessly spent 2500 on a bryston amp when spending 50$ on a used quad 405 had achieved the same result.
Just to demonstrate that any subjective discussion is nothing more than trading anecdotes - like telling the typical fishermen lies about the one that got away.

therefore - good by.
 
Last edited:
Hi,

The only way to assess what the subjectivist claims to hear is again by measuring it through the statistical evaluation of a blind test.

This we agree.

A mechanism typically unacceptable to most subjectivist, or when accepted the results, when not according to subjectivist expectations, are heavily criticized as to validity of statistical evaluation or methodology despite prior agreeing to the test protocol and the statistical methods used.

Well, first, the criticism of the Statics came from Statisticians, though it tends to be quoted by subjectivists. The ABX Mafia decided not adjust their tests despite this, meaning their tests lack statistical power to claim any reasonable applicability.

Second, the criticism including Placebo/Nocebo comes from blind tests in other areas of science. Given the nature of the tests and the general strong emotional involvement of all participants these effects will be elevated. This means that much larger tests sets will be needed to overcome these. The ABX Mafia decided not adjust their tests despite this, meaning their tests lack statistical power to claim any reasonable applicability.

The usual excuse however is the mythical "test stress" deteriorating the performance of the testee.

Test stress is very real. I experienced it personally.

Our local audio club once set up a test between a stock Marantz CD Player and one I had modified (actually the very one I wrote about in TNT-Audio).

The test was blind, I sat with the rest of the audience. In a previous test at my home (also blind) that I just did for myself the differences where clear.

In the public test I failed to hear ANY difference. I mean ANY.

Funnily enough, about a dozend+ or so guys in the room, many in seats with a much poorer listening position than mine found the differences plain, obvious and more than 9 out of 10 preferred the modified player consistently.

I should add that the modified player was not appreciably different in THD and IMD, SNR or Frequency response, however the IMD and THD spectrum showed quite drastic differences in terms of "grass blades" above the noisefloor, when looked at, but where swamped in the single number IMD/THD tests.

As an example - I was taken to task when I claimed that I was unable to hear a difference between amplifiers of vastly different designs bryston, technics switchmode amp, hypex, quad, sansui etc.) when driving speakers within the amps parameters and having specs below what it is said to be inaudible - or even borderline audible.

Well, I would not take you to task for observing you did not hear any difference. I would take you to task if you therefore claimed that no audible differences can exist though, as this does not from your failure to observe such.

In light of this I can only see the discussion to be utterly fruitless and wasting all the participants time.

therefore - good by.

Good bye then.

Ciao T
 
Last edited:
Originally Posted by audio-kraut
I can only assess what I can measure, I simply cannot assess what you can hear.
Originally Posted by abraxalito
There is the rub - the 'objectivist' does indeed ***** it - as worthless.
If it cannot be assessed, it cannot be assessed a value. That does not have to mean "worthless" as a euphemistic perjorative. Measurements are used to assess value, whether they be grams, minutes (we all spend time), barrels, carats, or THD percentages. If a person assigns some subjective value, irrespective of measurements, to audio reproduction components, they still must sell that value to others. Wrapping it real pretty doesn't always work. This is a marketplace of ideas, and the how&why insistence of the objectivist equates to "where's the value?"
 
The elephant story is a very good picture of the situation, it's nice to think of this kind of illustration. With 99% of the job, I mean that without the base of the imperfect measurements we have, the blind men would have to describe the elephant without even touching it.

A funny experiment would be to make a contest : one hour in front of an unknown 3 ways system, active and with a digital EQ+Filter controlled from the listening seat. No measurement allowed, just music and pink noise. Could the elite of the GEB be able to make it sounding uncolored ? For me no doubt, I can't, I need some help from the measure.
 
Hi,

The elephant story is a very good picture of the situation, it's nice to think of this kind of illustration. With 99% of the job, I mean that without the base of the imperfect measurements we have, the blind men would have to describe the elephant without even touching it.

Not quite, to map the measurements common these days for HiFi onto the elephant it would be like if the wise man finding the tusk would measure the smoothness of it and conclude from that something about the smoothness of the surface of the elephant.

The measurements currently done are not at all like the Elephant story, but rather like that of the drunk man looking for his lost car keys.

Finding him searching under a street light for something, he is approached by a policeman who ask what he is doing. When told "looking for the car keys I dropped while trying to get into my car" the policeman asks where the car is, as non is near the street light. The drunk man points somewhere into the darkness quite a bit distant. The policeman is bemused and ask the drunk man why he is looking for the keys here, under streetlight, if they got lost over there, only to be told "but the light is better here".

Sure, we can measure many things, but they generally show so poor correlation with reality that they are next to useless.

Speakers typically have several percent THD at rated power in the midband (never mind in the bass) yet they are almost never measured for distortion, instead we argue about small percent fraction of THD in amplifiers, indeed a case of "the light is better here". Equally, A Speakers DI shows quite good corellation with percieved sound, given minimum standards of on axis response
flatness, yet it is almost never measured.

There are many more such areas. I would indeed suggest that contrariwise to your assertion current common measurements do perhaps at best a 10% job in telling us about sound quality and the rather disturbing tendency to measure individual items makes sure any issues caused by interactions between multiple pieces are safely ignored.

Quite frankly, given the state of affairs in electronics fields outside Audio, one cannot fail to see the current state in audio as mostly at the level of the dark ages, so no wonder we get so much "Eye of Newt", "Blood of Bat" and so many Witchhunters, because in fact that which is important is simply not measured.

A funny experiment would be to make a contest : one hour in front of an unknown 3 ways system, active and with a digital EQ+Filter controlled from the listening seat. No measurement allowed, just music and pink noise. Could the elite of the GEB be able to make it sounding uncolored ? For me no doubt, I can't, I need some help from the measure.

Well, it may be funny. However sound engineers in my generation routinely had do something like this using non-digital means. I would not necessarily claim that routinely and reliably achieved uncoloured sound (this does need some form of available reference), but those that where certainly could achieve quite consistent sound, with different PA systems and in different Venues, using no measurements at all (RTA's and Crown TEF's being too expensive for routine on site use).

Ciao T
 
Tony,

The title to the thread has been changed slightly as it was misleading. I apologise for taking liberties.

Tony.

I think having it as the official thread and being able to point people who pollute other threads with the same to:

1) Contribute to the debate if they have anything to add
2) Read all the arguments posited and see that no repetition is needed

Then we can debate more useful things else where and perchance (one may hope, for eternally springs this weakest of all emotions, hope) move for the current state of "war" between the Hegelian Deamons Thesis and Antithesis into the new realm of synthesis, where the warring opposites are united into new understanding (actually, to be precise thesis, antithesis and synthesis are more Kant and Fichte than Hegel).

Ciao T
 
If I may pose a slightly off-topic question:

Years ago I worked at a high-end audio store, and my boss mentioned that he thought that one of the items he sold had no effect whatsoever on sound. When I questioned the ethics of this, he responded.

If the customer spends $100 on something that he thinks improves his stereo system, and he goes home and enjoys his stereo at least $100-worth more, then he is happy. The salesman is happy, the manufacturer is happy. Everyone is happy. Whether or not there is an actual improvement in the sound of his stereo is irrelevant. The efficacy of the product may be fake, but the customer's enjoyment is real. Thus, YES, it is ethical.

To this day I don't know whether or not I agree with him.

So my question is: Let's assume, for the sake of argument, that somewhere someone is selling something that is bogus, and they know it. But people buy it and like it and are happier because of their purchase. Is the salesperson ethical?
 
AX tech editor
Joined 2002
Paid Member
So your question boils down to: is it ethical to make somone happy by misleading him? (Yes its black-and-white). Tough one.

Can I bring in another on-topic view? The notion that measurements have no relation to how something actually sounds.
I think it very well does. For example, if an amp has a largish Zout, that will modify the speaker freq response which, if it is large enough, will be audible.

The confusion comes from the fact that people's preference don't seem to correlate with "objective" performance. Listeners may well prefer the amp with the largish Zout because the end result pleases them more, and then say: see, the meausrement (largish Zout) doesn't correspont with the sound quality: I like it.
But I think we can all see the logical fallacy here.

My point is that measurements say something about how transparent and faithful an amp reproduces the source material. It does not say how or if people like the result, because that depends on personal preference and a lot of other things, which have been mentioned before in this thread.
Thoughts?

jan didden
 
Hi,

Can I bring in another on-topic view? The notion that measurements have no relation to how something actually sounds.
I think it very well does.

My contention is not if the measurements have ANY relation to sound, but rather that for almost all common measurements "better" measurements correlate very poorly with "better sound".

For example, if an amp has a largish Zout, that will modify the speaker freq response which, if it is large enough, will be audible.

Sure, but the questions are "What is largish" and what if a speakers frequency response is more flat with a high source impedance than a low one? Or has lower distortion? Or both?

The confusion comes from the fact that people's preference don't seem to correlate with "objective" performance. Listeners may well prefer the amp with the largish Zout because the end result pleases them more, and then say: see, the meausrement (largish Zout) doesn't correspont with the sound quality: I like it.
But I think we can all see the logical fallacy here.

The fallacy here is at side of the "meter reader" brigade who promote a single dimensional measurement on the principle that the number has merit in itself.

Low output impedance/High Damping Factor
Low THD
Flat on axis response in speakers

are among my pet peeves.

My point is that measurements say something about how transparent and faithful an amp reproduces the source material.

Only in an electrical sense. There is no reference to system context or audibility.

Unless we can instead use a distortion metric that includes both system interactions (e.g. distortion cancellation between speaker and amplifier, a factor that is most easily observed with SE Tube Amp's) and audibility (e.g. using GedLee metrics instead of THD) the measurement is useless as judge of quality.

Note, I do frequently and extensively measure audio gear. In the case of my own designs (which generally do not consider the typical measured "quality" as relevant or desirable) I simply use measurements as a quick and reasonably objective way to confirm it performs as designed in these parameters.

However, following objective measurement tests each and every product is auditioned and we do occasionally find products in these tests that do not subjectively perform on par, yet passed the measurement stage.

Ciao T
 
So my question is: Let's assume, for the sake of argument, that somewhere someone is selling something that is bogus, and they know it. But people buy it and like it and are happier because of their purchase. Is the salesperson ethical?
Perhaps the more pertinent question is: Are people who buy something that is bogus and like it and are happier because of their purchase ethical?
 
AX tech editor
Joined 2002
Paid Member
[snip]However, following objective measurement tests each and every product is auditioned and we do occasionally find products in these tests that do not subjectively perform on par, yet passed the measurement stage.

Ciao T

Thorsten,

In such cases, do you find what (in possibly technical, measurable terms) causes that discrepancy? Can you say what the shortcomings are versus an amp that DOES perform on par? That would be hugely interesting.

jan didden
 
Status
This old topic is closed. If you want to reopen this topic, contact a moderator using the "Report Post" button.