| john curl |
For the record, this is the situation about my wire 'measurement'. I still measure DIFFERENCES in shielded cables with my test setup. I doubt, at this time, that it is due to distortion in the center wire, itself. This was my original hypothesis, due to the fact that Dr. Vandenhul had measured wire distortion with a different test set-up. However, on further investigation of what Vandenhul had measured, he was operating at a much lower operating level than I can get my equipment to operate. I can also see differences between clean and dirty contacts, and the presence of mumetal near the wire.
At this time, however, I don't know where the distortion is coming from, and Steve Eddy doesn't either.
I do not promote this test any further, because I have run into a 'dead end' where I can measure differences, but they do not reflect similar measurements of similar cables on other equipment. Are there diodes in wires, etc? Of course there are. Virtually any impurity or oxide should create a barrier of some kind and amount. I don't think, however, that this is the main component of what I am measuring, which is unique and repeatable for a given wire configuration. |
|
|
| janneman |
| quote: | Originally posted by john curl
[snip]This was my original hypothesis, due to the fact that Dr. Vandenhul had measured wire distortion with a different test set-up. [snip] |
John, typical you to call him 'Dr Vandenhul'. We always call him Aalt-Jouke or AJ when we talk about or to him. That doesn't seem to diminish HIS accomplishments. A very pleasant, accessible and likeable man, he is.
(BTW his last name is written as 'van den Hul', but I know that the american way is to write it as you do).
Jan Didden |
|
|
| john curl |
| Thanks, Jan, for your input. I just wrote AJ's name quickly, without checking first. He did tell me that he has measured problems of distortion in wires at very low levels, about 20 years ago. This of course, is very important to me, because I still make and use moving coil preamps that work approximately 1000 times lower level than a typical digital or tuner input. Interestingly enough, one of his cables (the only one that I have) has always measured very well in my test, along with a few others. This led me to believe that I was measuring what he was talking about, but my working level is typically 30-70mV, and his measurement was MUCH lower, perhaps 60dB lower, and so I am not emulating his measurement. |
|
|
| fdegrove |
Hi,
| quote: | | We always call him Aalt-Jouke or AJ when we talk about or to him. |
If we must dot the diodes and cross the cables:
The name should be spelled:
Aalt Jouk van den Hul and yes, Mr. VDH does hold a doctor title...
Flicker noise anyone?:clown:
Cheers,;) |
|
|
| janneman |
Indeed Frank, I stand corrected. Must break that habit...
Jan |
|
|
| janneman |
| quote: | Originally posted by john curl
Thanks, Jan, for your input. I just wrote AJ's name quickly, without checking first. He did tell me that he has measured problems of distortion in wires at very low levels, about 20 years ago. This of course, is very important to me, because I still make and use moving coil preamps that work approximately 1000 times lower level than a typical digital or tuner input. Interestingly enough, one of his cables (the only one that I have) has always measured very well in my test, along with a few others. This led me to believe that I was measuring what he was talking about, but my working level is typically 30-70mV, and his measurement was MUCH lower, perhaps 60dB lower, and so I am not emulating his measurement. |
John,
I had the chance to obtain a Fairchild/Electro-Metrics EMC-10, which can measure audio band signals from 0.006uV upwards (yes that's correct!). I am all exited to apply it to opwer supply noise and also cable stuff. Hope to get it next week.
Jan Didden |
|
|
| RHosch |
| quote: | Originally posted by Kuei Yang Wang
This means that very obvious changes can be reliably detetced with the small sample sets used by the Charlatans in the ABX crowd (right in line with the basic tenets of statistiscs as science) and that subtle changes cannot be detetected by these methodes with the usual sample sizes. |
Oh for crying out loud, how many times does someone have to spell out to you why the above is absolutely, undeniably, categorically FALSE before you quit regurgitating the same incorrect, old, tired excuses? If a single person can detect a change reliably only 0.5% more often than by chance alone, controlled testing (including possible ABX) can absolutely be used to confirm that ability to a high degree of certainty.
I'll say it again... so many concepts have been so badly butchered in this thread (here statistics, and in Thorsten's post just above the one I pulled the quote from the scientific method once again (clue - observation need not proceed theory)) that it is truly frightening.
You make the case for knowing instead of believing, but the more I read the more I am convinced it is indeed your knowledge that is faulty. |
|
|
| Kuei Yang Wang |
Konnnichiwa,
| quote: | Originally posted by RHosch
Oh for crying out loud, how many times does someone have to spell out to you why the above is absolutely, undeniably, categorically FALSE before you quit regurgitating the same incorrect, old, tired excuses? If a single person can detect a change reliably only 0.5% more often than by chance alone, controlled testing (including possible ABX) can absolutely be used to confirm that ability to a high degree of certainty. |
Absolutely. HOWEVER, if I wish to demonstrate THE PROBABLE ABSENCE a small difference (say 0.22/0.17) to a level of significance of .05 I require a very large sample.
Moreover, if 32 People all find that they can identify a given change to a .2 certainty level the whole group requires to be aggregated and requires to now allow the audibility of the suggested effect. However, if I insist that ALL 32 People show a .05 certainty level I could claim (and thius what is happening quite frequently) that no-one could hear the effect with any significant certainty, while I have in fact data that supports the position that the effect is audible.
For those who have not studied Statistics and wish to get a reasonable idea of the position and why I will continue to insist on it are invited to consult the discussion previously published in Stereophile:
The Highs & Lows of Double-Blind Testing
Do not be alarmed by the source in a subjectivist audio magazine, I feel the discussion presented there is completely fair and allows both sides to represent their positions well. I leave it to the genteele reader to draw their own conclusions. Here just one quote I absolutely love:
"When data are nonsignificant, one scientist may conclude that differences are inaudible, another may conclude that it is wiser to withhold judgment (because, for example, it is always possible that ancillary equipment used in the listening test masked otherwise audible differences), another may decide to issue challenges, and a fourth scientist may decide to have spare ribs for dinner. These four scientists, having decided what interpretation to make when listening data are nonsignificant, may be interested in the probability that their significance test will label data as nonsignificant when differences are audible, forcing them to make that interpretation rather than correctly conclude that differences are audible.
For example:
•Scientist 1 wants to know the risk of concluding that differences are inaudible when differences are, in fact, audible.
•Scientist 2 wants to know the risk that he will withhold judgment when differences are, in fact, audible.
•Scientist 3 wants to know the risk that he will issue challenges when differences are, in fact, audible.
•Scientist 4 is beneath contempt because he is eating spare ribs while I am hungry and writing this damn letter!"
Have fun reading and allow me to rest my case. I'm ahmm.... Hungry, from writing all these darn posts.
Sayonara |
|
|
| fdegrove |
Hi,
| quote: | | Have fun reading and allow me to rest my case. I'm ahmm.... Hungry, from writing all these darn posts. |
To recap: Scientist #4 has, with 0.05% certainty, left you some spare ribs.
Cheers...errrrrrrr Guten Appetit, ;) |
|
|
| john curl |
| I do wish that some of you would contribute something that we can discuss on this subject. |
|
|
| TNT |
I dont see how any constructive discussion can continue (or has been ongoing the last 10 pages) .
- No one here has tried the stones
What would You like to discuss, James Randi ?
/ |
|
|
| geoffkait |
| New poster. I've had the Shakti Stone for about 8 years as well as a pair of the smaller Shakti On-lines. I find the large "stone" to have only one location where it improves the sound noticably; in other locations (in my system) it does nothing or slightly degrades the sound. Not intending to stir up any additional controversy, the Shakti stone is "directional" -- it should be rotated to find the best orientation. I can imagine the sound in the room could become discombobulated (relatively speaking) if several of the stones were strew about indiscriminately. The smaller On-lines, intended for cables and interconnects, did not work for me where suggested, but they do improve the sound on top of the output transformers of my humble Fisher 500C receiver. (The Fisher is not stock, having undergone an upgrade by BWS Consulting some years back, including power supply mods and aircraft-grade Bendix output tubes.) YRMV. :) |
|
|
| diy_audio_fo |
| quote: | Originally posted by john curl
For the record, this is the situation about my wire 'measurement'. I still measure DIFFERENCES in shielded cables with my test setup.
...
I can also see differences between clean and dirty contacts, and the presence of mumetal near the wire.
|
John,
do you say that you shield the cable under test with mumetal?
Mumetal is a magnetic shield and we use a lot of it to shield photomultipliers from Earth's magnetic field. |
|
|
| john curl |
| Mumetal is pretty nonlinear, compared to copper or aluminum. |
|
|
| Arthur-itis |
New poster. I've had the Shakti Stone for about 8 years as well as a pair of the smaller Shakti On-lines. I find the large "stone" to have only one location where it improves the sound noticably; in other locations (in my system) it does nothing or slightly degrades the sound.
I'm sure you believe that. If you can prove it in a double blind comparison, you might be one million dollars richer.
If you beleive in magic rocks, perhaps you should try some of those wooden frisbees, the Mpingo disks. I guarantee they are at least as effective as the rocks. |
|
|
| Arthur-itis |
Citing a refernce to a Sterophile article regarding ABX testing is on a par with popping your radiator cap to check your gasoline level.
The article in question should then be compared to the following:
Comments on "Type 1 and Type 2 Errors in the Statistical Analysis of
Listening Tests" and Author's Replies 674942 bytes (CD aes4)
Author(s): Shanefield, Daniel; Clark, David; Nousaine, Tom; Leventhal, Les
Publication: Volume 35 Number 7/8 pp. 567·572; July 1987
Transformed Binomial Confidence Limits for Listening Tests 468821 bytes (CD
aes5)
Author(s): Burstein, Herman
Publication: Volume 37 Number 5 pp. 363·367; May 1989
Abstract: A simple transformation of classical binomial confidence limits
provides exact confidence limits for the results of a listening test, such
as the popular ABX test. These limits are for the proportion of known
correct responses, as distinguished from guessed correct responses.
Similarly, a point estimate is obtained for the proportion of known correct
responses. The transformed binomial limits differ, often markedly, from
those obtained by the Bayesian method.
Approximation Formulas for Error Risk and Sample Size in ABX Testing 442116
bytes (CD aes4)
Author(s): Burstein, Herman
Publication: Volume 36 Number 11 pp. 879·883; November 1988
Abstract: When sampling from a dichotomous population with an assumed
proportion p of events having a defined characteristic, the binomial
distribution is the appropriate statistical model for accurately
determining: type 1 error risk (symbol); type 2 error risk (symbol); sample
size n based on specified (symbol) and (symbol) and assumptions about p; and
critical c (minimum number of events to satisfy a specified [symbol]). Table
3 in [1] pre;sents such data for a limited number of sample sizes and p
values. To extend the scope of Table 3 to most n and p, we present
approximation formulas of substantial accuracy, based on the normal
distribution as an approximation of the binomial.
If you haven't figured it by now the subjective audio press doen't like ABX because it's bad for sales. |
|
|
| serengetiplains |
| quote: | Originally posted by Arthur-itis
If you haven't figured it by now the subjective audio press doen't like ABX because it's bad for sales. [/B] |
Wow, thanks for the tip. Say, have you a peer-reviewed article using appropriate statistical methods showing in fact that the subjective audio press doesn't like blah blah? |
|
|
| Steve Eddy |
| quote: | Originally posted by john curl
Mumetal is pretty nonlinear, compared to copper or aluminum. |
Well yeah, given that MuMetal is a high permeability ferromagnetic material and that copper and aluminum are both paramagnetic (actually I think they're both very slightly diamagnetic) and have permeabilities about the same as air.
However unless you're approaching saturation, MuMetal is far more linear than iron and steel.
se |
|
|
| Steve Eddy |
| quote: | Originally posted by john curl
At this time, however, I don't know where the distortion is coming from, and Steve Eddy doesn't either. |
But I know where it's NOT coming from. It's NOT coming from the cables. And if it's not coming from the cables, that leaves your test equipment. Which I couldn't care less about.
se |
|
|
| SY |
| Based on what I saw, it is not likely to be an intrinsic property of the cables but rather some sort of interaction between them and the test gear. My guess is that John concurs. The difference is metaphysical, admittedly, if you believe that artifacts 120 dB or more down could be audibly significant. That's where John and I part company :D |
|
|
| fdegrove |
Hi,
| quote: | | The difference is metaphysical, admittedly, if you believe that artifacts 120 dB or more down could be audibly significant. That's where John and I part company |
This kind of reminds of people saying, what the heck: 5% distortion isn't audible, so what if it has 5% distortion?
They all hear it when that same 5% isn't there however.
Same with the -120dB distortion artefacts, you don't hear them.
You quite probably don't hear it when one of those is removed either...
Remove a couple more and all of a sudden it becomes obvious that -120B isn't that inaudible as we first thought it to be.
Do two amplifiers that measure the same also sound the same?
Cheers,;) |
|
|
| serengetiplains |
| quote: | Originally posted by SY
The difference is metaphysical, admittedly, if you believe that artifacts 120 dB or more down could be audibly significant. That's where John and I part company. |
"Audibly significant" contains two notions: "audible" and "significant." Perhaps you and John part company on the "significant" part which IMHO concerns a value preference (ie, the audibility concerned is not worth the trouble).
As to "audible," Gregory Soo of Emm Labs recently told me Meitner's new DSD gear works as it does, in part, because he pushed the circuit's noise level to -145dB. Seems, according to Meitner, that "120dB or more down" is audible. Judging from reviews his professional and consumer equipment has garnered, the "significance" is worth troubling about, at least if one is concerned about pushing the state of the art? |
|
|
| SY |
| quote: | | Do two amplifiers that measure the same also sound the same? |
Depends on what you're measuring, n'est-ce pas? |
|
|
| Steve Eddy |
| quote: | Originally posted by SY
Based on what I saw, it is not likely to be an intrinsic property of the cables but rather some sort of interaction between them and the test gear. My guess is that John concurs. |
Apparently he doesn't as even well after Bruno's measurements were made public (the ones in which he measured the same cables that John did) John said he stood behind his measurements.
So not only does he apparently not concur, he also apparently dismisses Bruno's measurements.
se |
|
|
| fdegrove |
Hi,
| quote: | | Depends on what you're measuring, n'est-ce pas? |
Assuming amp X and amp Y both undergo a battery of measurements and both show the same measured results, would they necessarily sound the same?
Je pense que non...
Conversely, could I make two amps X and Y to measure and sound the same given that same battery of test?
Probablement bien...
Cheers,;) |
|
|
| SY |
| quote: | Originally posted by Steve Eddy
Apparently he doesn't as even well after Bruno's measurements were made public (the ones in which he measured the same cables that John did) John said he stood behind his measurements.
So not only does he apparently not concur, he also apparently dismisses Bruno's measurements.
se |
I played around with his test setup and confirmed that he is indeed measuring what he claims to be measuring. The only dispute is the source of the distortion. And I don't even think there's a dispute there, if I understand John correctly.
| quote: | Originally posted by fdegrove
Hi,
Assuming amp X and amp Y both undergo a battery of measurements and both show the same measured results, would they necessarily sound the same?
Cheers,;) |
Depends on what's in that battery, nicht wahr? |
|
|
| Steve Eddy |
| quote: | Originally posted by SY
I played around with his test setup and confirmed that he is indeed measuring what he claims to be measuring. |
I've never had any doubts that he's measuring the distortion he claims to be measuring.
| quote: | | The only dispute is the source of the distortion. And I don't even think there's a dispute there, if I understand John correctly. |
Well, all I know is that even after Bruno's measurements, which measured well below where John's able to measure yet didn't turn up any signs of the distortion John is measuring, clearly indicating that the distortion John was measuring wasn't being produced by the cables themselves, John said that he stood behind is measurements and would continue to stand behind them until someone could show him exactly what was causing his test gear to produce the distortion he's been measuring.
In other words, he dismisses Bruno's masurements and continues to assume that the distortion is being produced by the cables NOT until it can be shown that the cables aren't producing such distortion, but until someone can explain why his test gear is producing the distortion he's measuring.
I dunno about you, but it looks to me that there's still a dispute here.
se |
|
|
| SY |
| quote: | Originally posted by john curl
For the record, this is the situation about my wire 'measurement'. I still measure DIFFERENCES in shielded cables with my test setup. I doubt, at this time, that it is due to distortion in the center wire, itself. This was my original hypothesis, due to the fact that Dr. Vandenhul had measured wire distortion with a different test set-up. However, on further investigation of what Vandenhul had measured, he was operating at a much lower operating level than I can get my equipment to operate. I can also see differences between clean and dirty contacts, and the presence of mumetal near the wire.
At this time, however, I don't know where the distortion is coming from, and Steve Eddy doesn't either.
I do not promote this test any further, because I have run into a 'dead end' where I can measure differences, but they do not reflect similar measurements of similar cables on other equipment. Are there diodes in wires, etc? Of course there are. Virtually any impurity or oxide should create a barrier of some kind and amount. I don't think, however, that this is the main component of what I am measuring, which is unique and repeatable for a given wire configuration. |
This looks pretty clear to me, Steve. |
|
|
| john curl |
It's a sorry thing that Steve Eddy can't let this go. If he can 'trash' me he will. He has called me just about everything over the years. Most of us think this is 'insecurity' on his part, but it does get old sometimes.
For the record, I have a lot of test equipment gathered over the last 30 years. It is NOT brand new, but averages about 15 years behind what is available today. I have modified some older test equipment like my Sound Technology with better, lower noise, IC's for lower distortion and it was calibrated by the factory about 10 years ago, where they entirely replaced my circuit boards while they were at it.
I developed this test setup which would cost about $30,000 if it was purchased new when it was last available in order to measure my power amp designs, especially at the transition between class A and class B with a 4-8 ohm load. As my designs use negative feedback, and run fairly heavy idle current, this distortion is difficult to distinguish with a standard Sound Technology, or the HP339 which were standards in the industry for many decades, and are still useful for lab testing. As they are NOT computer controlled, they don't work as well on an assembly line.
Well, in my latest JC-1 design, this transition shows added distortion at -115dB at about 10W when I switch between high bias and low bias, which is available on the amplifier. Now many of you don't design amplifiers, so it might surprise you that I think that this distortion is important, but if I can measure it, then I can optimize the amplifier components for lowest distortion. It is like making a better running engine in an automoble. Those of you with cheap American cars probably don't know what I am talking about ;-) but European and Japanese car owners know what I am referring to.
Now, do I believe that 110-115dB down is directly hearable? No, not as a single tone, BUT as multitone IM generated with a higher order nonlinearity (kink in the transfer function), perhaps. In fact I am counting on it.
My wire tests come from developing this measurement, and unfortunately an individual connecting wire can measure -115dB down in some cases, so I am stuck with having to test my wires for testing equipment. It may not be the wires themselves, but a system interaction. I don't know yet. |
|
|
| Steve Eddy |
| quote: | Originally posted by SY
This looks pretty clear to me, Steve. |
My apologies. I didn't read that last part. Indeed he seems to have changed his tune from a couple months ago.
se |
|
|
| janneman |
Nice last post, Steve.
Jan Didden |
|
|
| Arthur-itis |
quote:
Originally posted by Arthur-itis
If you haven't figured it by now the subjective audio press doen't like ABX because it's bad for sales. [/B]
Wow, thanks for the tip. Say, have you a peer-reviewed article using appropriate statistical methods showing in fact that the subjective audio press doesn't like blah blah?
Just a history of rags like Stereophile printing whatever they can to try and discredit ABX, but nothing on the followups that debunk them when they do.
I don't understand why it's so difficult for people to grasp, ABX is the standard for detecting small differences in audio.
The claims that it is designed to show no difference or that it is somehow flawed are not supported by any credible data.
www.pcabx has an ABX comparator you can download and learn more about the process. |
|
|
| Kuei Yang Wang |
Konnichiwa,
| quote: | Originally posted by Arthur-itis
Citing a refernce to a Sterophile article regarding ABX testing is on a par with popping your radiator cap to check your gasoline level. |
Did you actually bother to read the article? If so, you might have noticed that all the criticism of ABX Tests and their statistical evaluation where based upon the following AES Paper:
Les Leventhal - How Conventional Statistical Analyses Can Prevent Finding Audible Differences In Listening Tests, Preprint 2275 (C-9)
And the JAES (and thus fully peer-reviewed) article:
Les Leventhal - Type 1 and Type 2 Errors in the Statistical Analysis of Listening Tests (JAES, Vol.34 No.6)
| quote: | Originally posted by Arthur-itis
The article in question should then be compared to the following:
Comments on "Type 1 and Type 2 Errors in the Statistical Analysis of Listening Tests" and Author's Replies 674942 bytes (CD aes4)
Author(s): Shanefield, Daniel; Clark, David; Nousaine, Tom; Leventhal, Les Publication: Volume 35 Number 7/8 pp. 567·572; July 1987 |
Yes, absolutely. I assume you have actually READ the above and noted the clobering the ABX Mob received (again)?
| quote: | Originally posted by Arthur-itis
Transformed Binomial Confidence Limits for Listening Tests 468821 bytes (CD aes5) Author(s): Burstein, Herman
Publication: Volume 37 Number 5 pp. 363·367; May 1989
Abstract: A simple transformation of classical binomial confidence limits provides exact confidence limits for the results of a listening test, such as the popular ABX test. These limits are for the proportion of known correct responses, as distinguished from guessed correct responses. Similarly, a point estimate is obtained for the proportion of known correct responses. The transformed binomial limits differ, often markedly, from those obtained by the Bayesian method. |
This one I am not familiar with, but as it is still based on the binominal distribution it does not address the fundamental issues of small sample size and all the issues of Type 1 and Type 2 errors remain in force.
| quote: | Originally posted by Arthur-itis
Approximation Formulas for Error Risk and Sample Size in ABX Testing 442116 bytes (CD aes4)
Author(s): Burstein, Herman
Publication: Volume 36 Number 11 pp. 879·883; November 1988
Abstract: When sampling from a dichotomous population with an assumed proportion p of events having a defined characteristic, the binomial distribution is the appropriate statistical model for accurately determining: type 1 error risk (symbol); type 2 error risk (symbol); sample size n based on specified (symbol) and (symbol) and assumptions about p; and critical c (minimum number of events to satisfy a specified [symbol]). Table
3 in [1] pre;sents such data for a limited number of sample sizes and p values. To extend the scope of Table 3 to most n and p, we present approximation formulas of substantial accuracy, based on the normal distribution as an approximation of the binomial.
|
I doubt that there are any findings materially different from these presented by Levental. After all statistics have not changed.
| quote: | Originally posted by Arthur-itis
If you haven't figured it by now the subjective audio press doen't like ABX because it's bad for sales. |
No, the "subjective audio press" do not the ABX Mob because their behaviour is clearly and repeatedly proven unscientific and clearly (on the evidence of the ongoing presentation of their results) repeatedly and with an obvious agenda (as evidenced by various complaints fielded by the ABX crowd to Trade regulators over time).
I personally do not care for people who present conclusion of "There is no audibile difference between A and B!" as authoritive pronouncements when their data is not capable of supporting this thesis with ANY resonable level of confidence. In fact, if I based testimony in a trial on a similar set of Data and analysis I'd be guilty of perjury.
Sayonara |
|
|
| pinkmouse |
| quote: | Originally posted by Arthur-itis
I don't understand why it's so difficult for people to grasp, ABX is the standard for detecting small differences in audio |
Some people have an aversion to being classified as just a data point. The old quote from the 60's surreal TV series "The Prisoner" comes to mind- "I am not a number, I am a free man!" |
|
|
| Kuei Yang Wang |
Konnichiwa,
| quote: | Originally posted by Arthur-itis
I don't understand why it's so difficult for people to grasp, ABX is the standard for detecting small differences in audio. |
You are ALMOST right. I completely UNDERSTAND ABX Testing.
ABX is the SELFDECLARED standard for testing differences in audibility.
Good thing that SERIOUS RESEARCHERS do NOT employ it, like for example when researching perceptual coding. If they did Dolby Digital and MP3 would be a lot worse than they are now....
So yes, it is a standard, but NOT a standard that is widely aknowledged oustide the ABX Mob.
| quote: | Originally posted by Arthur-itis
The claims that it is designed to show no difference or that it is somehow flawed are not supported by any credible data. |
Levental has proven by teh use of most ELEMENTARY statistics that the ABX test is highly likely to find "NO DIFFERENCE" in situations where the difference is audible but small. The ABX Mob failed to re-adjust their testing system and criterias following this criticism for nearly 20 Years. That is for nearly 20 Years iut had been PROVEN to them that their test is unlikely to detect small audible differences and they failed to adjust their test. Surely you do not wish to suggest that the failur to corect their methode is down to gross incompetence and negligence?
Sayonara |
|
|
| Arthur-itis |
Konnichiwa,
quote:
Originally posted by Arthur-itis
Citing a refernce to a Sterophile article regarding ABX testing is on a par with popping your radiator cap to check your gasoline level.
Did you actually bother to read the article? If so, you might have noticed that all the criticism of ABX Tests and their statistical evaluation where based upon the following AES Paper:
Les Leventhal - How Conventional Statistical Analyses Can Prevent Finding Audible Differences In Listening Tests, Preprint 2275 (C-9)
And the JAES (and thus fully peer-reviewed) article:
Les Leventhal - Type 1 and Type 2 Errors in the Statistical Analysis of Listening Tests (JAES, Vol.34 No.6)
quote:
Originally posted by Arthur-itis
The article in question should then be compared to the following:
Comments on "Type 1 and Type 2 Errors in the Statistical Analysis of Listening Tests" and Author's Replies 674942 bytes (CD aes4)
Author(s): Shanefield, Daniel; Clark, David; Nousaine, Tom; Leventhal, Les Publication: Volume 35 Number 7/8 pp. 567·572; July 1987
Yes, absolutely. I assume you have actually READ the above and noted the clobering the ABX Mob received (again)?
quote:
Originally posted by Arthur-itis
Transformed Binomial Confidence Limits for Listening Tests 468821 bytes (CD aes5) Author(s): Burstein, Herman
Publication: Volume 37 Number 5 pp. 363·367; May 1989
Abstract: A simple transformation of classical binomial confidence limits provides exact confidence limits for the results of a listening test, such as the popular ABX test. These limits are for the proportion of known correct responses, as distinguished from guessed correct responses. Similarly, a point estimate is obtained for the proportion of known correct responses. The transformed binomial limits differ, often markedly, from those obtained by the Bayesian method.
This one I am not familiar with, but as it is still based on the binominal distribution it does not address the fundamental issues of small sample size and all the issues of Type 1 and Type 2 errors remain in force.
quote:
Originally posted by Arthur-itis
Approximation Formulas for Error Risk and Sample Size in ABX Testing 442116 bytes (CD aes4)
Author(s): Burstein, Herman
Publication: Volume 36 Number 11 pp. 879·883; November 1988
Abstract: When sampling from a dichotomous population with an assumed proportion p of events having a defined characteristic, the binomial distribution is the appropriate statistical model for accurately determining: type 1 error risk (symbol); type 2 error risk (symbol); sample size n based on specified (symbol) and (symbol) and assumptions about p; and critical c (minimum number of events to satisfy a specified [symbol]). Table
3 in [1] pre;sents such data for a limited number of sample sizes and p values. To extend the scope of Table 3 to most n and p, we present approximation formulas of substantial accuracy, based on the normal distribution as an approximation of the binomial.
I doubt that there are any findings materially different from these presented by Levental. After all statistics have not changed.
The above referenced papers debunk Leventhal.
quote:
Originally posted by Arthur-itis
If you haven't figured it by now the subjective audio press doen't like ABX because it's bad for sales.
No, the "subjective audio press" do not the ABX Mob because their behaviour is clearly and repeatedly proven unscientific and clearly (on the evidence of the ongoing presentation of their results) repeatedly and with an obvious agenda (as evidenced by various complaints fielded by the ABX crowd to Trade regulators over time).
That's your opinion and theirs, it is not the opinon of everybody else doing audio research aside form the high end.
I personally do not care for people who present conclusion of "There is no audibile difference between A and B!" as authoritive pronouncements when their data is not capable of supporting this thesis with ANY resonable level of confidence.
Good, because no has done that, what they've done is said that this person or persons couldn't hear any difference, unless of course they did hear a difference, which does happen.
I guess that Sean Olive, Floyd Toole and Jim Johnston, and the BBC and hearing aid companies, are all charlatans being bought off by the pro ABX lobby, since they all use and endorse ABX for it's intended purpose. |
|
|
| Arthur-itis |
quote:
Originally posted by Arthur-itis
I don't understand why it's so difficult for people to grasp, ABX is the standard for detecting small differences in audio.
You are ALMOST right. I completely UNDERSTAND ABX Testing.
ABX is the SELFDECLARED standard for testing differences in audibility.
Good thing that SERIOUS RESEARCHERS do NOT employ it, like for example when researching perceptual coding.
Really Jim Johnston at Bell Labs used it and endorses it and I think he knows a fair amount about percetual coding.
If they did Dolby Digital and MP3 would be a lot worse than they are now....
Opinion stated as fact.
So yes, it is a standard, but NOT a standard that is widely aknowledged oustide the ABX Mob.
You are completely and utterly incorrect.
quote:
Originally posted by Arthur-itis
The claims that it is designed to show no difference or that it is somehow flawed are not supported by any credible data.
Levental has proven by teh use of most ELEMENTARY statistics that the ABX test is highly likely to find "NO DIFFERENCE" in situations where the difference is audible but small. The ABX Mob failed to re-adjust their testing system and criterias following this criticism for nearly 20 Years. That is for nearly 20 Years iut had been PROVEN to them that their test is unlikely to detect small audible differences and they failed to adjust their test. Surely you do not wish to suggest that the failur to corect their methode is down to gross incompetence and negligence?
No I wish to suggest that Leventhal was debunked in the papers I cited, and that's part of the reason I have no use for Stereophile. They'll cite him but not the people who debunk him. |
|
|
| SY |
| Sorry, Al, you ARE a number. |
|
|
| pinkmouse |
| quote: | Originally posted by SY
Sorry, Al, you ARE a number. |
Ooh, ooh, could I be an equation?....Please...Pleeeeease...:) |
|
|
| janneman |
...I want to be a prime!
Jan Didden |
|
|
| janneman |
| quote: | Originally posted by Arthur-itis
[snip] |
Arthur, is it possible to show who posted what, as the posters do? It's very difficult to keep track of what you state and what the other guy said.
Jan Didden |
|
|
| cbrodersen |
| quote: | | Arthur, is it possible to show who posted what, as the posters do? It's very difficult to keep track of what you state and what the other guy said. |
Yes, I agree. I gave up trying to follow the thread because Arthur doesn't understand how to use the quote function.
To return to the original dispute between John Curl and Steve Eddy, I would add that the issue is not who's right about the measurability/audibilty of -120dB distortion, or whether it comes form the cables or the test equipment. It's whether the average Joe Hi-Fi (and that includes me) can hear this, and whether this kind of extreme low-level information has, or will have, any meaningful impact on our stereo systems.
I made the decision long ago not to go chasing after these kinds of phantoms. For me and my system, standard interconnects do just fine--and I happen to think that I have a rather high-resolution system. Plus, as a trained classical musician, I think I can hear all the nuances that any "golded-eared" audiophile can.
Of course, if I were independently wealthy, I suppose I might get interested in expensive interconnects and all the "theory" connected with them, out of sheer boredom. But on a restricted budget, it makes sense for me to concentrate on the weakest links in the chain--the loudspeakers, and possibly the DAC and turntable/tonearm/cartridge.
Of course, YYMV, <i>chacun à son gout</> and <i>dem Jeden das Seine</i>. |
|
|
| john curl |
ABX testing is NOT the standard for serious evaluation of small audio differences.
JJ is a bright guy, BUT he never told us how he did his BLIND TESTS, as it was a company secret, AND he made derisive comments about the ABX tests as they are typically done, over the years.
Our own efforts in evaluating ABX testing showed that it was worthless, like the test of New Coke and Old Coke on the guy who invested 10's of thousands of dollars in a campaign to get 'Old Coke' back to the public. He failed it, even though they brought 'Old Coke' back as 'Coke Classic'. This was just a different disguise of the fact that many people preferred the older formulation, over the newer formulation, even if 'double blind' testing showed that they could NOT tell the difference. Go figure! |
|
|
| janneman |
| quote: | Originally posted by john curl
[snip]Our own efforts in evaluating ABX testing showed that it was worthless, like the test of New Coke and Old Coke on the guy who invested 10's of thousands of dollars in a campaign to get 'Old Coke' back to the public. He failed it, even though they brought 'Old Coke' back as 'Coke Classic'. This was just a different disguise of the fact that many people preferred the older formulation, over the newer formulation, even if 'double blind' testing showed that they could NOT tell the difference. Go figure! |
Easy. Just fill both types of bottles from the same tap, thereby realising increased sales PLUS economies of scale. ABX worthless??
Like selling the same cable with a different jacket and double (or triple, you have to be bold) the price. Go figure, indeed.
Jan Didden |
|
|
| john curl |
| Jan, I am not supporting phoney cable manufacturers or phoney cable claims by people who want to confuse the issue still further, or as a practical joke. I have some opinions of what some cables sound like. Some are 'bright', some are 'dirty' and still others are dull, or too smooth. I have enough cables in my lab stock to last a lifetime, IF I could believe that my opinions are imaginary. Convince me! ;-) |
|
|
| serengetiplains |
| quote: | Originally posted by Arthur-itis
I don't understand why it's so difficult for people to grasp, ABX is the standard for detecting small differences in audio.
|
Peter Aczel feels your pain. |
|
|
| janneman |
| quote: | Originally posted by john curl
Jan, I am not supporting phoney cable manufacturers or phoney cable claims by people who want to confuse the issue still further, or as a practical joke. I have some opinions of what some cables sound like. Some are 'bright', some are 'dirty' and still others are dull, or too smooth. I have enough cables in my lab stock to last a lifetime, IF I could believe that my opinions are imaginary. Convince me! ;-) |
Well, I was looking to it from a commercial pov. Think it through. If people readily report large audible differences (check this forum!) while DBT fails to back it up, then it must be clear that the perceived differences are not related to sound reproduction but other factors. That means that if you concentrate on those other factors, you can make a killing! And the 'victims' back you up, they report audible differences!
Actually, it is a surprise that there are not many, many more snake oil salesmen in audio..
The problem for the serious audiophile then becomes to figure out who is the bona fide innovator and who has done his commercial homework and is out for a quick buck.
In the last analysis, that was what the whole shakti discussion was all about, wasn't it?
Jan Didden
PS For Thorsten: yes I know that DBT is not accepted by everybody, but not all users of ABX are frauds and many have done an honest trial and came up empty handed. Over to you.
PPS For John C: What are you doing behind a PC on a sunny CA Sunday morning?? |
|
|
| fdegrove |
Hi,
| quote: | | That means that if you concentrate on those other factors, you can make a killing! And the 'victims' back you up, they report audible differences! |
Of course...That would work...For a while, not for 20 years though.
With all due respect to the Dutch National Audio Pride but wasn't the first VDH cable a rejacketed industrial cable??
I can still tell the exact reference and know where it was bought etc...
Nothing against it per se, it was a good cable and sure made a killing while the best was getting better....Go figure.
What if I told you that 83% of the high-end specialty audio wire is sourced from one and the same company?
Would that mean that all those cables inevitably sound the same?
Now if anyone would tell me that the bulk of audio wires are ridiculously overpriced, I sure wouldn't argue with that but so are most specialty products anyway.
Cheers,;) |
|
|
| TNT |
| quote: | Originally posted by janneman
snip.... but not all users of ABX are frauds and many have done an honest trial and came up empty handed. Over to you. [/B] |
Why "empty handed" ? Maybe there not any differences between a lot of gear. I think that result is as interesting as if a difference was proven. I don't have the luxury of having soemone re-convisning me that my 10000€ speaker wire ARE the best every week. (not that I own any).
There have been a lot of statistical theory in this tread. You who doubt the ABX testing, how can You be so shure that it is not You who are actually wrong. Maybe You are hearing things that does not actually exists because You are so involved in what You are doing, and want's so much to have made yet an improvment that You have entered a state of self-suggestion?
Things that can't be detected in ABX should carry a warning stick like cigarette packages (in sweden they say: Smoking kills You.) "The impact of sound by the contained equipment can not be detected in a blind test. It may however give You satisfaction and percived improvement for other resons".
/ |
|
|
| janneman |
| quote: | Originally posted by fdegrove
[snip]With all due respect to the Dutch National Audio Pride but wasn't the first VDH cable a rejacketed industrial cable??
I can still tell the exact reference and know where it was bought etc...
Nothing against it per se, it was a good cable and sure made a killing while the best was getting better....Go figure.
[snip] |
Frank,
What made you think that I thought that the Dutch were a specific sub-species of the general run of humanity??:o
Jan Didden |
|
|
| fdegrove |
Hi,
| quote: | Frank,
What made you think that I thought that the Dutch were a specific sub-species of the general run of humanity??
|
Errrrrrrr.....Leonid Tolstoi? :cool:
Cheers,;) |
|
|
| janneman |
| quote: | Originally posted by TNT
Why "empty handed" ? Maybe there not any differences between a lot of gear. I think that result is as interesting as if a difference was proven. I don't have the luxury of having soemone re-convisning me that my 10000€ speaker wire ARE the best every week. (not that I own any).[snip] |
I was using empty-handed in the sense that no audible differences were found in DBT although they were claimed. I thought that was clear from the context, but is was a bit short, sorry.
| quote: | Originally posted by TNT
[snip]You who doubt the ABX testing, how can You be so shure that it is not You who are actually wrong. Maybe You are hearing things that does not actually exists because You are so involved in what You are doing, and want's so much to have made yet an improvment that You have entered a state of self-suggestion?[snip]/ |
Are you addressing me? I don't doubt ABX testing, or generally DBT if correctly executed. And I fully agree with you that many, many of the claimed 'audible differences' are the result of other perception factors then the sound in itself. I thought we were past that station?
Jan Didden |
|
|
| janneman |
| quote: | Originally posted by fdegrove
Hi,
Errrrrrrr.....Leonid Tolstoi? :cool:
Cheers,;) |
Touche! Although I doubt that many will get the subtlety in that response of yours.
Jan Didden |
|
|
| TNT |
No - sorry, I understand I could be interpreted like that. "You" was used in an broad, non spceific manner. Sorry for the confusion.
/
| quote: | Originally posted by janneman
I was using empty-handed in the sense that no audible differences were found in DBT although they were claimed. I thought that was clear from the context, but is was a bit short, sorry.
Are you addressing me? I don't doubt ABX testing, or generally DBT if correctly executed. And I fully agree with you that many, many of the claimed 'audible differences' are the result of other perception factors then the sound in itself. I thought we were past that station?
Jan Didden |
|
|
|
| Arthur-itis |
quote:
Originally posted by Arthur-itis
I don't understand why it's so difficult for people to grasp, ABX is the standard for detecting small differences in audio
Some people have an aversion to being classified as just a data point. The old quote from the 60's surreal TV series "The Prisoner" comes to mind- "I am not a number, I am a free man!"
I don't think of ABX comparisons as being that impersonal, in fact I find the opposite. There's nothing more personal than using one's own personal ears to see if they can detect differences that are alleged to be present.
That these comparisons have validity is without question in my mind, otherwise I doubt hearing aid technology would be where it is today.
As a general question to the group: How many are aware of the facilities at the Harman campus for doing DBT's? They are used in teh development of the products made by the various companies under the Harman umbrella. |
|
|
| john curl |
| Arthur, you really DON'T KNOW what the ABX test is, what its problems are, or why we chose to ignore it. We have been dealing this for more than 25 years, where is your contribution? |
|
|
| Arthur-itis |
A note of apology for my ignorance of the proper way to show quoted text.
I promise to try and learn it before I respond to anything else. |
|
|
| diy_audio_fo |
| quote: | Originally posted by john curl
Mumetal is pretty nonlinear, compared to copper or aluminum. |
John,
Mumetal is good as DC magnetic field shield while copper and aluminum are good as EMI shield and have none or very little shielding capability against DC magnetic field.
Why did you use Mumetal? |
|
|
| fdegrove |
Hi,
| quote: | | Why did you use Mumetal? |
Where did you read that anyway?
| quote: | | They are used in teh development of the products made by the various companies under the Harman umbrella. |
You sometimes do get nice umbrellas when they're running a publicity campaign, don't you?....:xeye:
Cheers,;) |
|
|
| Arthur-itis |
Mr. Curl:
I do know what ABX is. Firstly it's not a TEST. It's a comparison.
My contribution? What do I need to have contributed to understand that there a lot of hype and ******** being passed off as science in the High End?
You are entitled to your opinion, I'm entitled to mine.
Until and unless there is a new more reliable, more discerning way to evaluate audio gear that removes bias and allows one to use one's ears, I'll stick with ABX.
In my expierience the people screaming the loudest about ABX and DBT's being flawed are the ones who stand to lose. It is not just an opinion however that the ABX form of comparison is the standard for evaluating differences in audio, everywhere but the high end community, and I've pointed out some of users.
Maybe you believe that Floyd Toole is wasting his employers time and money by using ABX, but I don't, and I don't I think Harman does either.
I'm a firm believer in maximizing my listening enjoyment by getting the most from my hi-fi dollar.
I know that, that really means, getting the best speakers one can afford, combined with fixing the listening environment and maybe some active EQ. |
|
|
| diy_audio_fo |
| quote: | Originally posted by fdegrove
Where did you read that anyway?
|
In post #401 John Curl wrote
| quote: |
I can also see differences between clean and dirty contacts, and the presence of mumetal near the wire.
|
|
|
|
| SY |
| quote: | | As a general question to the group: How many are aware of the facilities at the Harman campus for doing DBT's? They are used in teh development of the products made by the various companies under the Harman umbrella. |
Not to mention NRC. Of course, that's where Harman recruited Toole from.
There are two misconceptions that keep being said over and over:
1. Blind testing = ABX. This just is not the case. There are a variety of types of blind test, ABX is just one of them. If you don't like ABX or it's unsuitable for a particular comparison, you can use a different method. There's a million bucks out there- isn't it worth setting up some sort of blind test to snag it?
2. ABX doesn't show differences. Total BS; there are LOTS of perceivable differences that ABX tests have demonstrated. Like very fine sensitivity to level mismatch, to EQ, to polar pattern, channel balance, amplifier overload characteristics...
ABX is often scorned because the results of the testing have not proved the "clearly audible" phenomena that many people wish could be shown. That's the downside to doing things scientifically- your cherished hypotheses may not withstand the search for evidence. So the poor messenger gets shot. |
|
|
| fdegrove |
Hi,
| quote: | | In post #401 John Curl wrote |
Yes, read that too....
| quote: | | and the presence of mumetal near the wire. |
....only I don't think that necessarily means shielding of an I/C...Or does it?
The reason I asked is that I had the impression from your previous post that you automatically assumed mu-metal shielding.
Somehow I don't think that was implied but I could be wrong...
Cheers,;) |
|
|
| serengetiplains |
| quote: | Originally posted by SY
There's a million bucks out there- isn't it worth setting up some sort of blind test to snag it? |
Sy, get with the program, there ain't no million bucks out there from our friend Houdini-Randi for ABXing (or for whatevering) Shakti stones. Geez! |
|
|
| SY |
| Well, there's not if you can't detect these things audibly in a blind test. Otherwise, the prize money is still there, metaphysical word games notwithstanding. Randi has confirmed this in writing at his web site and via email. If he doesn't cough up, there's no shortage of lawyers who are willing to extract it from him- I'm sure you'd take that one on contingency. |
|
|
| john curl |
Arthur, do you know Floyd Toole? Do you work with him?
Most of you have little practical understanding of ABX testing. If you did, you would know that between properly equalized electronics, it is virtually impossible to hear any differences, BECAUSE of the test requirements (limitations). Since I design amps and preamps, I ignore ABX testing, but wish anyone else, who might be my competitor, to embrace it with all enthusiasm! ;-) |
|
|
| Arthur-itis |
Kuei Yang Wang thinks that Leventhal had something damaging to say on ABX, but obviously missed this in the Stereophile article:
"Les Leventhal's critique of the statistical analysis commonly used in blind
subjective testing is misleading, erroneous, and borders on the incompetent.
His letter is written in a style that prompts the casual reader to think
"Someone has finally figured out what's wrong with all those blind tests
where they don't hear anything." Not only has Leventhal failed to prove his
case; he has demonstrated his own lack of understanding of how the
audiophile benefits from double-blind testing. " |
|
|
| john curl |
Folks, just for completeness, I might mention that I have conversed with Floyd Tool, Drs. Lipshitz and Vanderkooy, David Clark (builder of the ABX box) and Les Leventhal on this subject over the years.
It has been shown that the worst caps (tantalum) that we could find, could not be detected in an ABX double-blind test, by one or two of the persons mentioned above. If the worst caps that we could find can't be detected, what is the point? Settle for an IC 150 and listen to music. |
|
|
| Bratislav |
| quote: | Originally posted by john curl
It has been shown that the worst caps (tantalum) that we could find, could not be detected in an ABX double-blind test, by one or two of the persons mentioned above. If the worst caps that we could find can't be detected, what is the point? Settle for an IC 150 and listen to music. |
Funny you mention tantalums. Some of the most raved about amps of all time (by same golden ears who claim to have detected single resistor directivity), Krell KMA's, use AMPLE amounts of tantalums (and no, NOT in the power supply, but in sensitive input differential double pair).
Did we hear any of the golden eared reviewers say anything but the usual high end nonsencial parabolas ("airy", "liquid", "silky", "holographic", etc.) ?
And if a person (*) cannot hear a tantalum in the circuit if he/she doesn't know if it is there, yes exactly - "what is the point" ?
(*) not ordinary person - but worlds golden eared high-end Gods |
|
|
| john curl |
Not ALL tantalum caps are bad. Just some cheap, offshore devices. In the '60's and '70's we all used tantalum coupling caps. Later, in 1978, I presented a paper at an IEEE conference on Audio, which showed significant nonlinear distortion in both tantalum and ceramic caps. Later, Walt Jung and Dick Marsh pointed out the effects of Dielectric Absorption in audio caps. So, tantalum caps can have both linear and non-linear distortion and the leads are usually magnetic as well, even with the best examples. What about a semi-defective component, made for the lowest possible price? Yes folks, you can actually measure differences between many cheap and expensive components.
Today, I tend to avoid all coupling caps, and use only the best bypass caps that can be used in the price range of the product. Don't tell HK! ;-) Let them find out for themselves. |
|
|
| arniel |
| quote: | Originally posted by john curl
It has been shown that the worst caps (tantalum) that we could find, could not be detected in an ABX double-blind test, by one or two of the persons mentioned above. If the worst caps that we could find can't be detected, what is the point? Settle for an IC 150 and listen to music. |
Just to claify, even if just one person in a group can detect a difference with a greater-than-random chance correctly, doesn't this prove the effectiveness of the test? Not everyone is going to have the same sensitivity to the same artefacts, as in video where I know people who seem to be unable to detect motion-blur in MPEG files, until I hit the freeze-frame button and point out the blocks to them... if only audio was that easy to analyze ;) |
|
|
| Kuei Yang Wang |
Konnichiwa,
| quote: | Originally posted by Arthur-itis
Kuei Yang Wang thinks that Leventhal had something damaging to say on ABX, but obviously missed this in the Stereophile article:
"Les Leventhal's critique of the statistical analysis commonly used in blind subjective testing is misleading, erroneous, and borders on the incompetent. His letter is written in a style that prompts the casual reader to think "Someone has finally figured out what's wrong with all those blind tests where they don't hear anything." Not only has Leventhal failed to prove his case; he has demonstrated his own lack of understanding of how the audiophile benefits from double-blind testing. " |
Hmmm. This was answer from Dave Clark, who had a clear financial interest in the ABX Gizmo and great emotional investment into the ABX Methode. His letter answering Les Leventals criticism of the statistics employed was answered by Clark in incoeherent emotional rambeling as above, with not a SINGLE point Levental made about statistics being addressed in a manner that would actually dsiproove it.
With the letter from which you quoted above Clark demonstrated an appaling lack of understanding for the most basic statististics and the usuall sour grapes attitude shown by his kind.
So, no I did not miss it at all.
Incoherent and statistically plainly wrong rambelings to answer a simple, straightforward and mathematically proovable criticism of the statistics employed by Clark. I fail to be impressed. Your quoting this suggest that your perception of reality is similarly removed from reality.
Sayonara |
|
|
| Kuei Yang Wang |
Konnichiwa
| quote: | Originally posted by arniel
Just to claify, even if just one person in a group can detect a difference with a greater-than-random chance correctly, doesn't this prove the effectiveness of the test? |
Well, it prooves not just the effectiveness of the test but also that in fact the observed effect is "true".
THAT IS, UNLESS you choose to exclude the most outlying datapoints (a common use in statistics), but doing so has implications for the analysis. So, if, as they have on occasions where one or two persons achieved a significant score, the "lucky coins" are excluded you also must excluded the "unlucky coins" (eg the most extreme outliers in the other direction) AND you must consider the rest of the data as a block analysed apropriatly.
So, if you have (say) 10 Test subjects all of which participate in a 16 Trial ABX test and we have one person scoring 8/8 and another 14/16 and we exclude them from the dataset as "lucky/unlucky coins"with the remaining 8 People in the set on average scoring on average 10/16 with no higher than 11/16 you would be in a position to conclude with a better than .05 significance level that the difference evaluated IS AUDIBLE.
Notabene the ABX Mob usually prefers to focus on the individual sets of DATA and would conclude and publish that after excluding the one "lucky coin" all other test data suggests that no difference is audible, as no subject could identify the difference with statistical significance, ignoring in effect the fact thatir accumulated data actually suggests audibility. It is not confidence inspiring that the ABX Mob almost never publishes the full experiemental data to allow a more complete analysis (and never mind the other fundamental problems with many of the documented test setups).
In fact, the ABX Mos and their test FAILED to find differences already aknowledged as audible in other Blind Tests (which where monitored and peer reviewed within the AES), a substantial pointer that their test methode is insufficiently sensitive for the reliable exclusion of small differences in their tests, a fact however not aknowleged by the ABX Mob and carefully disguised in their publications.
To repeat it in the moist simple terms. The limited datasets usually employed mean that it is NOT possible to reject with any certainty the hypothesis of a small audible difference EVEN if the test fails to show that, yet the ABX Mob regulary published pronouncements on the audibility (or rather the lack thereof) based on such data.
| quote: | Originally posted by arniel
Not everyone is going to have the same sensitivity to the same artefacts, as in video where I know people who seem to be unable to detect motion-blur in MPEG files, until I hit the freeze-frame button and point out the blocks to them... if only audio was that easy to analyze ;) |
This is the next problem.
Double Blind testing the ABX Style suffers from a number of problems that ensure that in effects all findings preseted by this group must be considered with extreme caution.
1) General testsetups tend to vary between seriously sloppy and/or deliberatly crippled to obscure any small (or even moderate differences, for example as presented here:
"The Audio Alchemy DDE Version 3.0 DA converter was driven from the digital outputs of the Marantz CD-63. The same/different comparison was between the DDE's outputs and the Marantz analog outputs, however the analog signal was put through another AD-DA conversion in a Marantz CDR-610 in record mode."
http://www.pcavtech.com/abx/abx_cd.htm
For the record, the CDR-610 is a "Pro" CD Recorder with pretty poor AD and DA conversion and far from "wire with gain)
2) The propensity of people who "believe" in audible differences to hear a difference where non exists (meaning where the test pair is "the same" and equal propensity of people who "disbelieve" NOT to hear a difference even where one exists (in other word the ability of people to hear exactly what they expect to hear, which should disbar anyone with a fixed believe or disbelieve to participate in test - something that especially applies to the Randi Style challenges regulary issued by the ABX Mob).
3) The lack of interest/relevance in certain potentially audible differences to a given person as well as the lack of analytical hearing training in most Audio Engineers, Audiophiles and the general public, which prevents many people from reliably identifying HUGE differences (I "ABX" Tested polarity reversal in one channel once, most participants failed their "ABX" Test already on this).
4) The statistics applied to the Data generated from the is subjected to inaproriate statistical analysis if you wish to ascertain if something is audible or not with a reasonable certainty, it is only aproriate to to be reasonable that small differences are not considered audible when in fact their are.
5) The often deliberate and fundamental misrepresentation of the resusts of the flawed statistical analysis as proof for the inaudibility of any differences, regardless of facts and reality, especially in areas where the ABX Mob feels differences SHOULD be inaudible.
The net result is that the data itself is highly suspect, the stastical evaluation of the data fundamentally flawed, a fact repeatedly pointed out to the Experimenters but studiously ignored and finally a misinterpretation of the statistical in pulication. To me at least this constitutetes a comprehensive inditement and AS ALL THE FACTS are known to the ABX mobs makes me at least question their motives in the strongest possible terms.
Sayonara |
|
|
| SY |
| quote: | | If you did, you would know that between properly equalized electronics, it is virtually impossible to hear any differences, BECAUSE of the test requirements (limitations). |
That's actually not correct. See, for example, "Some Amplifiers Do Sound Different," Carlstrom, Krueger, and Greenhill, Audio Amateur 13:3, 30 (1982) comparing an ARC D-120 to a CM Labs CM914a using ABX. Group score was 43/45. |
|
|
| arniel |
| quote: | Originally posted by Kuei Yang Wang
1) General testsetups tend to vary between seriously sloppy and/or deliberatly crippled to obscure any small (or even moderate differences, for example as presented here:
|
Um, as opposed to the rigourous analysis performed by the proponents of those lumps of rock we were gassing about at the start of this thread ;)
Thorsten,
It appears you're catching up with DestroyerX in the posting-coherency stakes! :clown:
One of the most beautiful effects of the internet is how it can bring an East German in North London so close to a Brasilian native in such an entertaining way :D ... |
|
|
| Kuei Yang Wang |
Konnichiwa,
| quote: | Originally posted by arniel
Um, as opposed to the rigourous analysis performed by the proponents of those lumps of rock we were gassing about at the start of this thread ;) |
Nope, not at all. As remarked before, I have zip opinion on Shakti Stones themselves, never having had any exposure to them. However, I am familiar with the effects of crystals (yup, new age/feng shui stuff) in general and in the context of my system and I have blind tested them.
What I am taking umbrage with are two main issues:
1) The pseudo scientific mindset prevalent in that poor excuse that calls itself modern "science" which will deny even the proof of their own eyes if it it disagrees with established dogma and doctrine (in audio as much as outside).
2) The making up or misinterpreting of data to support the dogmatically and doctrinally "correct" position.
Both of thee effectively eliminate any possibility of a serious and levelheaded investigation of phenomenae that appear to contradict exiting Dogma and Doctrine, regardless of the factual situation, it is basically to that group that "What cannot be (explained in their terms) must not (allowed to) be".
Sayonara |
|
|
| john curl |
SY, I don't know where you get your info about ABX testing, but it is virtually worthless for amps and preamps, UNLESS the devices under test significantly change the amplitude response, or have gross distortion. I generally don't design amps and preamps with gross distortion, or low damping factor, so it is worthless for me to use ABX to try to find any differences.
EVEN IF I could find a difference with an ABX test, it is one of the most insensitive ways to find anything. This is partly because of Les Leventhal's comparison of type 1 and type 2 sensitivity to differences with ABX. |
|
|
| john curl |
SY, I don't know where you get your info about ABX testing, but it is virtually worthless for amps and preamps, UNLESS the devices under test significantly change the amplitude response, or have gross distortion. I generally don't design amps and preamps with gross distortion, or low damping factor, so it is worthless for me to use ABX to try to find any differences.
EVEN IF I could find a difference with an ABX test, it is one of the most insensitive ways to find anything. This is partly because of Les Leventhal's comparison of type 1 and type 2 sensitivity to differences with ABX. |
|
|
| Prune |
| quote: | | ABX testing, but it is virtually worthless for amps and preamps, UNLESS the devices under test significantly change the amplitude response, or have gross distortion. |
Why? :scratch:
What method of blind testing is then appropriate? Would you say ABC/HR is also insensitive? That would eliminate the two most widely used methodologies. But maybe that's why... |
|
|
| john curl |
| AB or ABC testing works OK. But I hope that everyone understands that we designers were very interested in ABX testing when it first came out. We just found that it did not give us much real input. Just that similar electronics sounded the same, yet they did not sound the same when listened to without ABX testing. Heck, if I believed my own ABX test, I would still be listening to a Dyna PAS-3X preamp, at least for line livel inputs It sounded the same as the Levinson JC-2 to me. In fact, many designers dropped out of competing in the marketplace in the late '70's because they BELIEVED their own ABX test! What was the point of trying to 'improve' anything? |
|
|
| Steve Eddy |
| quote: | Originally posted by john curl
We just found that it did not give us much real input. Just that similar electronics sounded the same, yet they did not sound the same when listened to without ABX testing. |
But it's precisely because there can be perceived differences even when there are no actual audible differences that blind tests such as ABX and other double blind procedures exist in the first place.
So how do you know that it's the test that's at fault? Where has that actually been shown other than the various theories that have been put forth?
Seems to me that the reason most oppose ABX testing pretty much boils down to the fact that the test doesn't confirm their preconceived beliefs.
"Gee, I heard differences under sighted conditions but didn't under blind conditions. The test must be flawed!"
se |
|
|
| john curl |
| Steve, I dealt with this problem 25 years ago. Apparently, you haven't listened or read up on our efforts to resolve this dilemma. Professors of 'Philosophy of Science' have brought forth multiple pages of input. We have resolved it for ourselves, but you are welcome to your own personal opinion. Just don't impose it on me. I actually have to make better audio products and do audio research, so it is important to not restrict myself to ABX testing. If it were to believed, then I would have retired from audio design, like several of my contempories did 25 years ago. To me, ABX testing is like blindfolding someone, putting them in the backseat of two cars, and telling them to identify which car they are riding in. You know, Mercury or Mercedes! ;-) |
|
|
| Steve Eddy |
| quote: | Originally posted by john curl
Steve, I dealt with this problem 25 years ago. Apparently, you haven't listened or read up on our efforts to resolve this dilemma. Professors of 'Philosophy of Science' have brought forth multiple pages of input. |
Like I said, just theories.
| quote: | | To me, ABX testing is like blindfolding someone, putting them in the backseat of two cars, and telling them to identify which car they are riding in. You know, Mercury or Mercedes! ;-) |
Identification is self-evident in ABX. The listener doesn't have to listen to identify, they only have to listen to discern a simple difference. That's it. Absolutely no different than when you listen to discern a simple difference under sighted conditions.
se |
|
|
| geewhizbang |
I could probably tell which was which (Mercedes / Mercury) wearing gloves; just give the wheel a twitch.
The feel of the Mercedes would be distinctly different from the feel of the Mercury, especially if it was one of the really big boats.
*****
As for science, I find it rather troubling that so many here pile on various religiously expensive capacitors and resistors onto their projects, as if spending 8x as much a capacitor can make it so much noticeably better than an already adequate one.
Or solid state amps with expensive metal pointy feet, when - if anything - cheap rubber ones would scratch up your counter less and do a better job of damping any microphonic issues.
Not that any one of them can actually show that microphonics is an problem even worth worrying about with solid state amps. It's alleged that it CAN happen so we are encouraged to waste our hard-earned cash on something like this just in case.
None of these people seem to have much truck for proper AB blind or ABX testing. |
|
|
|
