I think those that want to believe such "tests" have their own reasons. I, for one, am extremely doubtful when people say anything so subjective about any of their perceptions including sight and even when they tell you a story about something that happened to them one time forgetting that you were actually there and it's not the way you remembered it at all. There is a lot of repetition in this thread and it makes me wonder if memory is playing tricks on people..........unless, of course, they have their own reasons for keeping a long dead thread alive.....just like me.....
True also Scott, Im the curious kind, read all the hype on audiophile fuses & it makes me wonder. The reason that I ask is because my mono amps has 2 fuses per channel at the input supply rails to the modules so Im curious to test out if there’s any benefit to be had as compared to generic glass fuses which Im using now. Anyway have order some schurter fuses. Will report when I’ve tested them.
Cheers
Cheers
Jakob--in an avoidance of making a disaster out of partial quotes, I will say that I was responding to both you and mmerrill99 in one post, and jumped topics.
I think I threw you off on a few colloquial terms: we're in agreement on powering and hashing the language in a few different ways.
I find Leventhal's argument technically correct, but the spirit of the argument (as I understood) was around "clear as day" audible effects that disappeared into the noise when tried in a DBT (and yes I believe ABX in specific). In any case, yes, no party is terribly innocent, nor capable of saying, "we don't know".
Well conducted tests of any size though do add to a (hazy, admittedly) Bayesian bounds.
I think I threw you off on a few colloquial terms: we're in agreement on powering and hashing the language in a few different ways.
I find Leventhal's argument technically correct, but the spirit of the argument (as I understood) was around "clear as day" audible effects that disappeared into the noise when tried in a DBT (and yes I believe ABX in specific). In any case, yes, no party is terribly innocent, nor capable of saying, "we don't know".
Well conducted tests of any size though do add to a (hazy, admittedly) Bayesian bounds.
I don't believe you represent the 'spirit' of Leventhal's argument correctly which is in his first paragraph
"subtle differences actually heard by the listener will likely go unidentified by the experimenter when the data is analyzed. The problem with these listening studies is that the experimenters conducted too few trials (for example, 16), and used the .05 level of significance when subjecting the data to a statistical test of significance. Only in a large-trial listening study can the results be tested at a significance level as small as .05 without the risk of overlooking small differences becoming unacceptably high."
Read more at The Highs & Lows of Double-Blind Testing Page 2 | Stereophile.com
Seems a perfectly reasonable argument to me which he then backs up with statistical elaboration
It's that he completely sidesteps in his "subtle differences" the entire firestorm was borne out of the "clear as day" part. Not out of subtle differences. So taken out of context (and the rebuttals I've read of this article in stereophile are of the order of "let's talk about large effects before we worry about this or that subtle detail").
It's a game of semantics and guess what people have latched on to? You got it, that DBT is useless because it can't resolve this or that thing they're hearing. Completely missing the fact that it bounds effect size. As I alluded to, no one wants their pet idea broken into a thousand pieces, so they find ways to discount a test that might point to that (or at least bound it to a small effect/subtle differences).
And while I've said this "ABX thread" is generally poor methodology, it points toward a suggestion that differences between various electronics are small, which I figured that was already obvious. Certainly can't prove anything. That's something I take exception with your crusade against DBT -- you can't completely throw the baby out with the bathwater.
It's a game of semantics and guess what people have latched on to? You got it, that DBT is useless because it can't resolve this or that thing they're hearing. Completely missing the fact that it bounds effect size. As I alluded to, no one wants their pet idea broken into a thousand pieces, so they find ways to discount a test that might point to that (or at least bound it to a small effect/subtle differences).
And while I've said this "ABX thread" is generally poor methodology, it points toward a suggestion that differences between various electronics are small, which I figured that was already obvious. Certainly can't prove anything. That's something I take exception with your crusade against DBT -- you can't completely throw the baby out with the bathwater.
So you are assigning to his arguments something which he hadn't stated? The arguments put forth by ABX apologists are not relevant to his treatment of the statistical flaws in the commonly run ABX audio tests seen on forums.
Ah but this "clear as day" phrase is just something which picked up & run with by ABX apologiziers. It's obvious to everyone that's involved in this hobby that newly discovered improvements are often described in glowing, possibly hyberbolic terms - happens in every hobby. This is one factor. Another possible factor is something I stated & was quoted by hubsand when he re-ignited this thread
"Maybe some aspects of our auditory perception are not instant A/B differences but rather more subtle differences which are teased out over time. Being able to A/B these subtle differences may well require training/experience to be able to differentiate"
Again, one has to ask - are there audible differences which are audible outside of such tests but very difficult to differentiate within such tests?
Again you are harking back to "clear as day" differences. Let's ask this question - lots of people report "more realism" "more soundstage depth" & consider these major differences in what they are hearing - these are not characteristics which lend themselves to be easily differentiated by instant A/B comparisons. It requires an understanding the nature of auditory perception to understand this.
If you are saying that a shoddy & flawed test which often is purported to be on the side of science & which many people are fooled into believing has some value, then I disagree completely with you. Give me, any day, anecdotal impressions that don't pretend to be somewhat more definitive than anecdotal impressions.
BTW, my crusade is against the dubious misuse of ABX testing - an already dubious test methodology made worse by inappropriate tests
Last edited:
@ DPH,
surely the "night and day" thing is a matter of, maybe not semantics but typical exaggeration born in a more and more heated discussion at that time.
And while it might have been that Leventhal missed this "night and day" line he was just pointing to the fact that the ABX proponents totally forgot about the whole statistical power issue. I would not blame them - one of the founders once told me that they even had some help/recommendations by statisticans - as the importance of power was (and is still) neglected by a lot of people in several fields.
But, and i think it is a big but, if you got some professional justified critic, you might of course not be happy, but you should appreciate the chance to get better in your search for the truth.
The quite obvious hostility in the various responses by Clark and others imo didn´t really fit to that fiction (i.e. of pure truth seekers).
The ad hoc arguments (can´t be of relevance or only looking for big differences) weren´t that convincing either. And as a scientist one should know that ad hoc arguments might be plausible/reasonable but defintely need experimental corrobation but they weren´t obviously interested in that.
If it were proclaimed right from the beginning that it was only about the "big difference/no relevance" thing i´m sure someone had pointed out that a measure was needed to quantify when a perceived difference has to be considered as "important/relevant" or as "big difference" or "night and day" but that did not happen - or i completely missed it.
And speaking of exaggeration, if it´s "night and day difference" vs "you can´t have heard it (because it is inaudible)" and a small/subtle but perceptable difference exists, whose exaggeration was larger?
surely the "night and day" thing is a matter of, maybe not semantics but typical exaggeration born in a more and more heated discussion at that time.
And while it might have been that Leventhal missed this "night and day" line he was just pointing to the fact that the ABX proponents totally forgot about the whole statistical power issue. I would not blame them - one of the founders once told me that they even had some help/recommendations by statisticans - as the importance of power was (and is still) neglected by a lot of people in several fields.
But, and i think it is a big but, if you got some professional justified critic, you might of course not be happy, but you should appreciate the chance to get better in your search for the truth.
The quite obvious hostility in the various responses by Clark and others imo didn´t really fit to that fiction (i.e. of pure truth seekers).
The ad hoc arguments (can´t be of relevance or only looking for big differences) weren´t that convincing either. And as a scientist one should know that ad hoc arguments might be plausible/reasonable but defintely need experimental corrobation but they weren´t obviously interested in that.
If it were proclaimed right from the beginning that it was only about the "big difference/no relevance" thing i´m sure someone had pointed out that a measure was needed to quantify when a perceived difference has to be considered as "important/relevant" or as "big difference" or "night and day" but that did not happen - or i completely missed it.
And speaking of exaggeration, if it´s "night and day difference" vs "you can´t have heard it (because it is inaudible)" and a small/subtle but perceptable difference exists, whose exaggeration was larger?
Last edited:
@mmerrill, you've completely gone blind then on your crusade, as it's losing interest in truth if you only argue from one side. You're not seeing a bigger picture and looking at the whole issue from a very narrow mindset. I get eyeroll-y about pseudoscience all the time, but where were you when XRK was running his "taste test" comparisons of speakers and the broad swaths of claims people were making there? Let's not talk about how much those could mislead people... (only as an example). It's certainly not like only one group is at fault, even if/when well-meaning.
@Jakob, I think they're pretty much equal exaggerations. Stats is still a huge huge problem, and even if more and more journals are demanding that one's stats protocols be made public, there's so much p-hacking that goes on and other forms of creative statistics that is a sad state of affairs.
@Jakob, I think they're pretty much equal exaggerations.
Stats is still a huge huge problem, and even if more and more journals are demanding that one's stats protocols be made public, there's so much p-hacking that goes on and other forms of creative statistics that is a sad state of affairs.
Having read audiophile and diy forums for years, the "harm" done by calls to dbt or abx has always appeared small to me compared to the herd effect that can be generated by a webpage such as this one: Humble Homemade Hifi - Cap Test
To think that everyone takes what is posted with a grain of salt or frames it correctly is wishful thinking. Newbies discovering the world of quality audio (and I've been among these) can easily be awed and dazzled by the amount of pseudo-expertise made available to them. And they will believe about anything. I keep a "cd mat" as a warning to myself.
It's not a bad thing to remind people sometimes that extraordinary claims require extraordinary proof. Especially when we know how easily people can fool themselves on the topic.
To think that everyone takes what is posted with a grain of salt or frames it correctly is wishful thinking. Newbies discovering the world of quality audio (and I've been among these) can easily be awed and dazzled by the amount of pseudo-expertise made available to them. And they will believe about anything. I keep a "cd mat" as a warning to myself.
It's not a bad thing to remind people sometimes that extraordinary claims require extraordinary proof. Especially when we know how easily people can fool themselves on the topic.
^ All good points, it's a shame that when it is explained that a fool and their money are easily parted, it doesn't go down too well I'm sorry to say I've largely given up trying to protect people from themselves, I'm more interested in attempting to understand the basics which it's many peoples agenda to hide from me.
I'm sorry to say I've largely given up trying to protect people from themselves, I'm more interested in attempting to understand the basics which it's many peoples agenda to hide from me.I don't know the XRK "taste test" but you keep coming back to this what-about-ism & as I already said, anybody with any sense treats anecdotal impressions as "possibly interesting & maybe worth investigating personally but let's see".
If people are misled by this then they really are naive & not capable of judging for themselves - no matter what the claims. Hey, we have all discovered this for ourselves - made mistakes along the way believing in some hyped up claim - I wouldn't call it misleading people - it's up to people to evaluate the claims for themselves - let's hope we have all learned from this?
As I also said, all hobbies have this element but it's the claim by ABXers that this test (no matter how shoddily administered) has a higher veracity than anecdotal reports, that I object to.
Yea, people don't usually take too kindly to being called fools.^ All good points, it's a shame that when it is explained that a fool and their money are easily parted, it doesn't go down too well
It's similarly a shame when the shoddiness of ABX tests & claims deriving from such tests are pointed out, it's not accepted as a true scientific search for truth - rather a defense is mounted which ignores looking at the test itself, or getting a deeper understanding of auditory perception itself.
<snip>
@Jakob, I think they're pretty much equal exaggerations.
Could be, although in my book the assertion of impossibility marks the boarder because you can´t exaggeration more.
But anyway, from a scientific point of view, if my model predicts something is inaudible then i should be very much interested if it _is_ _in_ _reality_audible.
Absolutely. It´s really fascinating to read about what´s going on, sometimes intentionally sometimes just due to a lack of knowledge or severe misunderstandings.
Last edited:
I understand your point, but wouldn´t it be better to help people by explaining the possible traps and telling them what to do to get better in evaluation by listening?
The current state is to demand a "blind test" whenever an idea is discussed that is contrary to someones belief or the known thresholds of audiblity.
And the result is always accepted provided it is a negative one, otherwise it faces the arguments i´ve listed a couple of posts before.
That point of view does not help because we know that even very audible differences can be overruled by bias effects. Which means that you might have rescued a newbie from buying an overpriced "......." but do not help him evaluating the new loudspeaker he is going to buy.
As jakob says, if you are trying to 'protect' newbies then educating them about the pitfalls of anecdotal impressions AND the pitfalls of ABX testing would be a more appropriate & balanced approach, I believe.
Anyway, I don't agree with you about the 'harm' done by anecdotal impressions - I have seldom seen a thread of such impressions where it isn't pointed out the possible flaws in such impressions but seldom seen such flaws pointed out about ABX testing. Why? Because people get tired/are not equipped to argue against the ABX brigade claiming science is on their side
It's also not a bad thing to point out that ABX testing on audio forums is generally flawed & also learning something about auditory perception, its workings & how best to attempt validating it.
- Status
- This old topic is closed. If you want to reopen this topic, contact a moderator using the "Report Post" button.