Hi,
Most Audio DBT's having achieved notoriety follow one particular standard and test design^, which is based around an approach that maximises test situation stress and uses questionable statistics.
^ ABX as promoted by "ABX Corporation" who's commercial and financial failure no doubt influenced the outlook of several of the main players in the ABX Mafia, who ended up bitter and vengeful over the fact that audiophiles elected to buy "magic wire" instead of their gizmo, which would have demonstrated to them that 16 Gauge zipcord was indistinguishable from the expensive "magic wire".
I often employ blind tests, precisely to remove bias, when evaluating new and in my view promising ideas (like Class D Amplifiers). However, I find no use for ABX specifically, in part because ABX alone does not allow a preference to be established and in part because to conduct an ABX tests that balanced the risk of statistical errors between both types of errors# is beyond the means and funding I have available.
# the statistics in ABX testing focus and are strongly weighted towards an avoidance of incorrectly accepting a difference exists, but in their normal form create a near certainty that small differences will not be picked up.
My issue with certain specific types of double blind tests is not that they are blind, nor even that they return null results (which proves nothing), but the fact that they use bad statistics, bad experimental design and completely lack any controls, together with the routinely sensationalist promotion of the results in the service of a particular ideology/concept, putting them on the same level as stage magic and side-shows as far any scientific merit is concerned and in line with the worst "News Corp" style gutter journalism.
For an illustration of the opposite I'd point to this:
God is in the Nuances Page 3 | Stereophile.com
from the heading "Expert Testimony" onwards, which contrasts VERY nicely with the previously referenced piece by Nousaine...
Ciao T
Most Audio DBT's having achieved notoriety follow one particular standard and test design^, which is based around an approach that maximises test situation stress and uses questionable statistics.
^ ABX as promoted by "ABX Corporation" who's commercial and financial failure no doubt influenced the outlook of several of the main players in the ABX Mafia, who ended up bitter and vengeful over the fact that audiophiles elected to buy "magic wire" instead of their gizmo, which would have demonstrated to them that 16 Gauge zipcord was indistinguishable from the expensive "magic wire".
I often employ blind tests, precisely to remove bias, when evaluating new and in my view promising ideas (like Class D Amplifiers). However, I find no use for ABX specifically, in part because ABX alone does not allow a preference to be established and in part because to conduct an ABX tests that balanced the risk of statistical errors between both types of errors# is beyond the means and funding I have available.
# the statistics in ABX testing focus and are strongly weighted towards an avoidance of incorrectly accepting a difference exists, but in their normal form create a near certainty that small differences will not be picked up.
My issue with certain specific types of double blind tests is not that they are blind, nor even that they return null results (which proves nothing), but the fact that they use bad statistics, bad experimental design and completely lack any controls, together with the routinely sensationalist promotion of the results in the service of a particular ideology/concept, putting them on the same level as stage magic and side-shows as far any scientific merit is concerned and in line with the worst "News Corp" style gutter journalism.
For an illustration of the opposite I'd point to this:
God is in the Nuances Page 3 | Stereophile.com
from the heading "Expert Testimony" onwards, which contrasts VERY nicely with the previously referenced piece by Nousaine...
Ciao T
Try putting one foot in boiling water the other in a salted ice bath. Mathematically on average you are comfortable!
I do not oppose neither blind tests nor double blind tests, yet I find them unnecessary, I note the same differences on both blind and sighted listening tests.
I need no blind tests in order to note hum coming from my speakers. Neither do I need blind tests in order to note when the stage imaging becomes wider, or narrower. I need no blind tests in order to note when the middle-high frequencies are harsher, or more refined. I need no blind tests in order to note that the sound of my highly acclaimed Universal Payer is unbearable to my ears.
When I move my speakers around the room to find out their best spot, I do it sighted and seeing doesn't interfere in any way in me locating the ideal position (in my listening room).
Those who say that one cannot trust ones' sighted listening make some major adjustments to sound setup impossible.
Possibly some people are interested more in sticking to their ideas, or in arguments, than in making the most, or the best possible sound, of their existing sound setup.
I need no blind tests in order to note hum coming from my speakers. Neither do I need blind tests in order to note when the stage imaging becomes wider, or narrower. I need no blind tests in order to note when the middle-high frequencies are harsher, or more refined. I need no blind tests in order to note that the sound of my highly acclaimed Universal Payer is unbearable to my ears.
When I move my speakers around the room to find out their best spot, I do it sighted and seeing doesn't interfere in any way in me locating the ideal position (in my listening room).
Those who say that one cannot trust ones' sighted listening make some major adjustments to sound setup impossible.
Possibly some people are interested more in sticking to their ideas, or in arguments, than in making the most, or the best possible sound, of their existing sound setup.
How can everything sound the same if we're all different? I also have had my religious experiences with amps and wines, well to be correct I should have used the singular with wine because it only happened once - Vega Sicilia 1968.
I don't believe in double blind testing for several reasons, one of them is that I'm not the same person that I was five minutes ago. This may sound like a joke but it's not. This is philosophy. I'm sorry to say but there are a LOT of people who will never know what I'm talking about. No use trying to explain it.
This IS the nub of it. Further, 3 minutes ago you experienced 'a', now you experience 'B'. One experience interacts with the other, and in 'strict test' conditions resultant confusion can only be expected. To me this 'strict testing' field is the ideal realm for control freaks.
I would far rather have my own experience of the relevant sound of a system/item, preferably playing a recording from my own collection, than all of the statistical evidence produced by conductors of such tests.
Last edited:
Well, personally, I'm interested in sound quality more than in comparative testing.
Comparative testing wouldn't assist me in finding the best position of my speakers.
It seems to me that we are asked to believe that X sounds better than Y, reliably and repeatably (maybe even obviously), yet when the comparison is restricted (by whatever means) to just the sound of X and Y the difference can disappear. We are then asked to believe that this is because of 'stress', apparently induced by the requirement to actually do what has been claimed - namely distinguish X and Y by their sound alone. We are also asked to believe that those proposing such tests do so with the aim of demonstrating that X and Y cannot be distinguished by sound alone, yet the outcome could equally well prove them wrong.
The curious thing is that there may be some evidence that some people can tell X from Y on sound alone, but we can't investigate this and find out how if nobody will do the tests. Maybe we will just have to rely on anecdotal evidence from readers' wives?
Have I missed something?
The curious thing is that there may be some evidence that some people can tell X from Y on sound alone, but we can't investigate this and find out how if nobody will do the tests. Maybe we will just have to rely on anecdotal evidence from readers' wives?
Have I missed something?
As for myself, I'm dealing with what I hear, not with any beliefs whatsoever. There is no belief on my part in sighted locating the best position of my speakers, nor in any other aspect affecting the sound quality of my setup.
Yes, you missed one minor thing- for some mysterious reason, the "stress" doesn't prevent people from reliably distinguishing data compression, level, phase, polarity, overload recovery, frequency response, polar pattern... Very mysterious indeed.
Don't worry Joshua, most hi end people do open testing, with some success, I believe. If it were not so, then we would not get anything done. You, Joshua, will always be criticized for your opinions, much as I am. So what, you are happy with what you have, and the decisions you have made, I have done the same with buying classical guitars, and even hi fi equipment. Others might select a wine that they like better than some others, or whatever. So be it.
The trouble I have with listening "tests" in general is that the test isn't a 100 MegOhm scope probe, it's more like someone talking to me while I'm trying to do something else. Listening to music and listening for a "test" are very different gigs for me, and I doubt their applicability to each other.
Thanks,
Chris
Thanks,
Chris
So we should give up trying to find any correlation between engineering/science and sound quality, and amplifier 'design' and construction should be purely a matter of art or pronouncement from a guru or journalist?Chris Hornbeck said:
I suspect part of the problem is the word 'test'. It brings back painful (for some) memories of maths test in school; the time when the nerdy kids could smile and all the rest were embarrassed. The reason why sound tests are painful is the claims which have preceded them: X and Y can be reliably distinguished by sound alone, when in reality it may be that the claimant has never actually tried to distinguish them by sound alone (although he sincerely believes that he has, being immune to sight/cost/styling/design/designer etc.). If I claimed to be able to multiply 6-digit numbers by mental arithmetic I would feel stressed in a test of this.
Even for a 'normal' hi-fi enthusiast it would be embarrassing to discover that his fancy setup actually sounds no better than a mass-market shiny box, so there is self-imposed pressure to hear a difference. Given this, it is astonishing that so few tests actually do show a difference. When a tiny difference is heard, it is then subject to the usual male bragging so something which one has to strain to hear becomes "obvious/huge".
From time to time, something happens to bring us down to earth. It might be one channel blowing an output transistor (so you get just half-wave rectified signals), or speaker phase being inverted on one channel, or RIAA (or FM deemphasis) networks failing - any of these should be easily noticeable yet we often find that 'something seems wrong but I can't quite put my finger on it'. When we discover and fix the fault we are surprised we didn't immediately hear it and identify it. Gnats and camels?
Sy,
You missed a minor thing. Many of the tests you listed that where double blind and had positive results (including that of Juergen Ackerman that illustrated many interesting things apart from the observation that Amplifiers do indeed sound different, those of Mr. Toole regarding speaker performance and JJ's regarding what level and type of data compression causes audible degradation) differ radically from the ABX Tests Noisaine/Krueger/Clark/Lipschitz et al use and promote and continue to use and promote unchanged in spite of drastic criticism in many areas over the decades.
The ABX Crowd (like some other supposed "Skeptics") have a clear agenda and have concocted a tests that allows them to produce supporting evidence with great reliability while appropriating the mantle of scientific testing unjustly and unjustifiably (as ye olde Bard once famously said: "And thus I clothe my naked villainy. With old odd ends, stol'n forth of holy writ; And seem a saint, when most I play the devil."). Yet many continue to credit their mockery of scientific auditory testing with the same relevance or credibility as extended to the real scientific tests.
Ciao T
You missed a minor thing. Many of the tests you listed that where double blind and had positive results (including that of Juergen Ackerman that illustrated many interesting things apart from the observation that Amplifiers do indeed sound different, those of Mr. Toole regarding speaker performance and JJ's regarding what level and type of data compression causes audible degradation) differ radically from the ABX Tests Noisaine/Krueger/Clark/Lipschitz et al use and promote and continue to use and promote unchanged in spite of drastic criticism in many areas over the decades.
The ABX Crowd (like some other supposed "Skeptics") have a clear agenda and have concocted a tests that allows them to produce supporting evidence with great reliability while appropriating the mantle of scientific testing unjustly and unjustifiably (as ye olde Bard once famously said: "And thus I clothe my naked villainy. With old odd ends, stol'n forth of holy writ; And seem a saint, when most I play the devil."). Yet many continue to credit their mockery of scientific auditory testing with the same relevance or credibility as extended to the real scientific tests.
Ciao T
Thank you, John.
I'm not worried and I don't mind the criticism. I know what I'm doing and I know how to get the best possible sound quality (to my own taste) from my stereo setup, with the money I have. Should I win the lottery, I'd have a better sounding setup.
It seems to me that people who are saying that I cannot find the best spot for my loudspeakers without blind tests are clueless in regards of voicing a setup. Also, people who say I don't really hear hum in my setup without double blind tests are clueless as for what I hear and what I don't hear, or imagine that I hear. It only amuses me to encounter others telling me what I actually hear and what I imagine to be hearing.
Since my enjoyment of listening to reproduced music isn't dependent upon what others say, I don't mind the criticism at all. Moreover, possibly, those who are stuck to some ideas prevent themselves the possibility of learning and of getting better sound quality with the sound equipment they have. However it is up to them, I don't mind it much.
Hi Chris,
Like you, I listen to the music reproduced by my stereo setup, not to the "audiophile parameters" of my setup. It seems to me that in this I may defer from the majority of the audiophiles.
Now, when I make any change to the setup, I do listen to the sound parameters, in order to note what the change made to the sound. However the ultimate answer to whether the change improved or degraded the sound quality is the degree the music itself is more, or less, convincing to me.
Hi,
For one, I'm a consumer, not a designer of sound equipment.
As a consumer, I found out that published technical specification fail to inform me of the sound quality of any piece of gear.
Also I haven't found even one reviewer that all his reviews are in accord with my own preferences.
Moreover, the sound quality of a power amplifier is highly correlated with the loudspeakers it is attached to and the sound quality of loudspeakers is highly correlated to the amplifier driving it.
Due to all the above, the one and only thing I can rely upon in choosing components to my sound setup is what I hear with my own ears. Nothing can possibly substitute it.
As for audio designers, at least some of the best of them systematically use listening tests, in addition to engineering considerations and measurements.
It is my firm view that designing audio gear of excellent quality is the combination of engineering and art. It seems to me that those who stick to engineering alone limit themselves as to what may be attained in regards of excellent sound quality.
Also, setting-up a sound system, or voicing a sound setup, involves a kind of art, on top all everything else.
- Status
- Not open for further replies.