AB or ABC testing works OK. But I hope that everyone understands that we designers were very interested in ABX testing when it first came out. We just found that it did not give us much real input. Just that similar electronics sounded the same, yet they did not sound the same when listened to without ABX testing. Heck, if I believed my own ABX test, I would still be listening to a Dyna PAS-3X preamp, at least for line livel inputs It sounded the same as the Levinson JC-2 to me. In fact, many designers dropped out of competing in the marketplace in the late '70's because they BELIEVED their own ABX test! What was the point of trying to 'improve' anything?
john curl said:
But it's precisely because there can be perceived differences even when there are no actual audible differences that blind tests such as ABX and other double blind procedures exist in the first place.
So how do you know that it's the test that's at fault? Where has that actually been shown other than the various theories that have been put forth?
Seems to me that the reason most oppose ABX testing pretty much boils down to the fact that the test doesn't confirm their preconceived beliefs.
"Gee, I heard differences under sighted conditions but didn't under blind conditions. The test must be flawed!"
se
Steve, I dealt with this problem 25 years ago. Apparently, you haven't listened or read up on our efforts to resolve this dilemma. Professors of 'Philosophy of Science' have brought forth multiple pages of input. We have resolved it for ourselves, but you are welcome to your own personal opinion. Just don't impose it on me. I actually have to make better audio products and do audio research, so it is important to not restrict myself to ABX testing. If it were to believed, then I would have retired from audio design, like several of my contempories did 25 years ago. To me, ABX testing is like blindfolding someone, putting them in the backseat of two cars, and telling them to identify which car they are riding in. You know, Mercury or Mercedes! ;-)
john curl said:
Like I said, just theories.
Identification is self-evident in ABX. The listener doesn't have to listen to identify, they only have to listen to discern a simple difference. That's it. Absolutely no different than when you listen to discern a simple difference under sighted conditions.
se
I could probably tell which was which (Mercedes / Mercury) wearing gloves; just give the wheel a twitch.
The feel of the Mercedes would be distinctly different from the feel of the Mercury, especially if it was one of the really big boats.
*****
As for science, I find it rather troubling that so many here pile on various religiously expensive capacitors and resistors onto their projects, as if spending 8x as much a capacitor can make it so much noticeably better than an already adequate one.
Or solid state amps with expensive metal pointy feet, when - if anything - cheap rubber ones would scratch up your counter less and do a better job of damping any microphonic issues.
Not that any one of them can actually show that microphonics is an problem even worth worrying about with solid state amps. It's alleged that it CAN happen so we are encouraged to waste our hard-earned cash on something like this just in case.
None of these people seem to have much truck for proper AB blind or ABX testing.
The feel of the Mercedes would be distinctly different from the feel of the Mercury, especially if it was one of the really big boats.
*****
As for science, I find it rather troubling that so many here pile on various religiously expensive capacitors and resistors onto their projects, as if spending 8x as much a capacitor can make it so much noticeably better than an already adequate one.
Or solid state amps with expensive metal pointy feet, when - if anything - cheap rubber ones would scratch up your counter less and do a better job of damping any microphonic issues.
Not that any one of them can actually show that microphonics is an problem even worth worrying about with solid state amps. It's alleged that it CAN happen so we are encouraged to waste our hard-earned cash on something like this just in case.
None of these people seem to have much truck for proper AB blind or ABX testing.
geewhizbang said:
Personally, at the end of the day I couldn't care less about blind testing. I listen for my own personal pleasure, not to satisfy some blind test or some piece of test gear, and I ultimately don't care how that pleasure comes about.
If something might sound better to me simply because it LOOKS good and not for any actual audible reasons, then give me looks.
se
geewhizbang said:
geewizbang,
Since you can't seem to be able giving up on me using cones (in my amps), here's the older thread dealing with vibrational matters and that's how I got stuck on cones (metal pointy feet):
http://www.diyaudio.com/forums/showthread.php?s=&threadid=11742&perpage=20&highlight=&pagenumber=1
Only if the CD player was also inside the same casing as the DAC could a logical reason be assigned to your (ahem) finding. In this case, perhaps you got less skippage if the CD was tilted slightly, but even that would probably be rather unlikely. If CD mechanisms were so much affected by tilt, it would probably tick off more than a few people.
If you didn't ABX this, it is just psychoacoustics. Pure and simple. There is absolutely no cause / effect mechanism that could change the sound with these feet.
It doesn't make any physical sense. I have a degree in Mechanical Engineering so I have some fairly good understanding of physical laws. There is absolutely no law of physics or electromagnetic theory that can explain this finding. It is so mysterious and utterly nonsensical that Randi would probably be willing to put his $1M at risk for this too.
I doubt that metal legs would have much of an effect on vibration. If vibration were a problem, then nice rubber legs would do even better, as they actually DAMP them.
Because you find it so easy to detect something that cannot possibly have an effect with these pointy legs, I also suspect the same thing is true of the expensive caps and (and especially the resistors) that you favor. In this case there could be a physical mechanism, but since the differences between the golden ear components and merely good ones are so negligible, it is rather unlikely.
So how about a full ABX? If your findings cannot be detected by such a test, then they aren't there.
If you didn't ABX this, it is just psychoacoustics. Pure and simple. There is absolutely no cause / effect mechanism that could change the sound with these feet.
It doesn't make any physical sense. I have a degree in Mechanical Engineering so I have some fairly good understanding of physical laws. There is absolutely no law of physics or electromagnetic theory that can explain this finding. It is so mysterious and utterly nonsensical that Randi would probably be willing to put his $1M at risk for this too.
I doubt that metal legs would have much of an effect on vibration. If vibration were a problem, then nice rubber legs would do even better, as they actually DAMP them.
Because you find it so easy to detect something that cannot possibly have an effect with these pointy legs, I also suspect the same thing is true of the expensive caps and (and especially the resistors) that you favor. In this case there could be a physical mechanism, but since the differences between the golden ear components and merely good ones are so negligible, it is rather unlikely.
So how about a full ABX? If your findings cannot be detected by such a test, then they aren't there.
Gee whiz bang,
I am also am sceptical of many tweaks and esoteric componants.
As you say, some people here "encourage" other people to buy fancy parts. I believe that they are sincere, generally, and I have certainly heard amps that seem to sound better than others.
These people are "encouraging" you to buy fancy parts, but not "forcing" you to, so if you don't believe that this exotic stuff works, why not spend your money on stuff that you think is a better value? I agree that it is fine that you express your opinion that this stuff isn't cost effective, but reasonable cost of some things is often relative to how much money people have to experiment with. Even a $65 gainclone kit is pretty cheap in the workld of hi fi amps!
For the gainclones in particular, I don't see the problem. Two kits are offered, one for people like yourself who don't feel that the premium parts are cost effective, one with premium components for those that want to try them.
It is generally accepted around here that the gainclones being sold here are a "group buy" - not a big profit making venture.
The higher price of the premium kit directly reflects the greatly higher cost of the resistors and capacitors included.
It's not even clear that that Brian or Peter make more on the premium kits.
I am also am sceptical of many tweaks and esoteric componants.
As you say, some people here "encourage" other people to buy fancy parts. I believe that they are sincere, generally, and I have certainly heard amps that seem to sound better than others.
These people are "encouraging" you to buy fancy parts, but not "forcing" you to, so if you don't believe that this exotic stuff works, why not spend your money on stuff that you think is a better value? I agree that it is fine that you express your opinion that this stuff isn't cost effective, but reasonable cost of some things is often relative to how much money people have to experiment with. Even a $65 gainclone kit is pretty cheap in the workld of hi fi amps!
For the gainclones in particular, I don't see the problem. Two kits are offered, one for people like yourself who don't feel that the premium parts are cost effective, one with premium components for those that want to try them.
It is generally accepted around here that the gainclones being sold here are a "group buy" - not a big profit making venture.
The higher price of the premium kit directly reflects the greatly higher cost of the resistors and capacitors included.
It's not even clear that that Brian or Peter make more on the premium kits.
It wouldn't be that hard to do a good ABX test.
I also strongly disagree with these sorts of assertions about something sounding "better" when the method of determining what is better has been done in a way that allows for pyschoacoustics to influence the result.
Not only that, if there really is a difference, I won't mind one bit if a good reproducible test shows this. But we have nothing but his opinion to go by, and judging from what he thinks about pointy feet, is already rather suspect.
We will make faster progress at improving audio reproduction if we spend less money and time barking up the wrong trees.
So let's do some reproducible testing before we start making claims.
I also strongly disagree with these sorts of assertions about something sounding "better" when the method of determining what is better has been done in a way that allows for pyschoacoustics to influence the result.
Not only that, if there really is a difference, I won't mind one bit if a good reproducible test shows this. But we have nothing but his opinion to go by, and judging from what he thinks about pointy feet, is already rather suspect.
We will make faster progress at improving audio reproduction if we spend less money and time barking up the wrong trees.
So let's do some reproducible testing before we start making claims.
The thing is, it is easy to misunderstand the title and think they are scientists, when in fact they are still philosophers. I've already said this elsewhere, but Penrose was right that one should wear his scientist's hat more often than his philosopher's hat.john curl said:
I think it would help your case here, and the understanding of those that mostly agree with SE but still have some doubts, if you would cite the materials you are referring to. This seems to be a common problem with audio forums, as I've asked for citations many times and just been ignored.
That's pretty much how I justify spending $7 for a freakin' resistor 🙂 Well, there is the small possibility I could hear a difference if the rest of the signal chain was improved. Perhas individual improvements too small to be noticed by themselves would add up overall to make an audible and testable difference. It would be interesting to take blind tests where no differences were found between given components, and repeat them but changing all the components simultaneously.
Folks, there are those of you who have never read about the problems with ABX testing or participated in an ABX test yourself. Unfortunately, you just can't make an 'informed opinion', only a wishful one. Study up, and we might have a more enlightened discussion on this subject. I recommend the back issues of 'The Audio Amateur' from 1978-1982+ for the most balanced discussions on the subject. The more fortunate might have a discussion with Les Leventhal on the statistics and how they are stacked against hearing any differences. Those with an engineering bent, might look for serious problems in many ABX tests, such as distortion overlap from sources being paralleled and little attention to 'accuracy' only 'sameness' in the audio signal. For example, if the music used was 6dB down at 15KHz compared to 1KHz, this was considered ok for ABX testing, so long as both components had the same defect in accuracy.
Paradoxically, that's not how it works IME. The small changes are not additive or at least the effect is not proportional to the number of incidences. Moreover, the change of a single resistor to a different brand may be perceived as an improvement while the change of all resistors may be detrimental.
Logic has little to do with all this and psychoacoustics, even less, as the results are often puzzling and not in line with expectations. There are so many times i have been disappointed with the performance of commonly 'approved' audiophile parts that to bring up any psychological explanations is utterly ridiculous. The disappointment is not that the 'boutique' parts sound the same as their industial equavalents, but rather that their sonic character is not perceived as improvement. Many of the 'boutique' parts, in particularly wiring, have much too pronounced a sonic flavour rather than none.
Provided i could hear none of these sonic flavours i'd be as frustrated and disgusted as any of the Pelicanists in this thread. Their arguments make a lot more logical sense than the confused ramblings of the subjective crowd. Sadly, my ears insist on keeping me firmly in the loony camp.
Konnichiwa,
How comes you are so certain? Have you ever considered the CD Error Correction as source of sonical differences? Have you ever had the case where playing music loud the vocalist hit a certain note with a good deal of power and the CD actually jumped (I am not talking about a big bass impulse)? Have you ever had a case where ripped CD which sounded previously bad (usually a result of many micro scratches overtaxing the CD error correction) sounded pretty good after EAC ripping and burning onto a good quality blank?
So, I could think of quite a few logical resonsn even besides the easily proven microphonics of the Master clock X-Tal.
You statement is clearly and objectively proovable in the same category as "Heavier than air flying machines are impossible".
A simple and clear case. More than a few blind tests (including one large scale one supervised by and under the auspices of the AES IIRC) clearly illustrated that absolute polarity (aka phase) is AUDIBLE with Music. The ABX test for the same failed to show this AKNOWLEDGED difference with music and was only able to show it using a test signal.
So, if an ABX test is not done or because it returns a Null result you BY FAR do NOT have any scientific justification to state: "it is just psychoacoustics.". And that in fact and in a nutshell is my umbrage with the ABX Mob. Misanalysed and misinterpreted results of shoddily implemented testing generalised.
You make another statement of the "Heavier than air flying machines are impossible" type. You may say "I cannot imagine or understand any cause / effect mechanism that could change the sound with these feet.". However, as you are far from omnicient (and clearly show a disturbing lack of understanding many basics of the subject) your statement, despite being voiced as statement of fact is actually merely a statement of religious doctrine and highly unscientific.
How can you be so sure?
But do you understand how a CD Player ACTUALLY works? Do you understand what is needed to correctly read the CD and what the error rates even with totally normal CD's are (never mind any extensively scratched or badly pressed)?
Hmmm. The simple fact is that if you vibrate the CD (or player) the error rate will go up increasing the number of samples that are either repeated or interpolated instead of the real samples that should have been read. Surely a mechanical engineer can appreciate how different ways of coupling surfaces subjected to vibration can change the actual amplitude of vibration observed at a given point? Surely you can appreciate that changing the actual points where surfaces are coupled to each other can change resonance frequencies in these surfaces and the amplitude of these resonances? I mean I DO NOT need a degree in mechanical engineering to know that, in fact 6th grade physics suffice.
Yes, your position may very well be right insofar that damping (decoupling) may be preferrable and you are allowed to doubt anything you like.
Again, you state a doctrinal and competely irrational believe as fact. Please discontine these attemps at religous conversiopn (conversion is a process by which you feed one position in words into a mans ears untill they begin to come out of his mouth).
Hmmm. Are you saying that the resistors and capacitors in question (which have real and demonstrable electrical effects which MAY very well be above the audibility threshold - BG Capacitors distortion certainly is) cannot make an audible difference because they are expensive? Now I find that sort of logic hard to stomach. Could you enlarge, while sticking to observed fact and not irrational believe?
Why ABX? Are you that desperate to obtain another "Null" result?
I have previously suggested the following tests as allowing a "calibration" of testers and testsetups for ABX Testing, by using phenomenae KNOWN to be audible:
1) Polarity reversal in one channel only.
2) Polarity reversal in both channels (Absolute Phase/Polarity).
3) 20KHz - 1st order lowpass filter.
4) Radioshack Goldpatch Interconnect vs. RG-213-MIL with WBT locking RCA's
(I agree that item 4 is arguable, the other three are not).
Would it now? Have you EVER tested the actual ABX test in regards to it's ability to show discrimination for small but definitly audible differences?
That is your perogative. You may disagree with anything you like. However, when you state obvious untruth as fact your credibility with rationally thinking people plumets rapidly even though your co-relligionists will no doubt applaud your defense and continued statement of dogma and doctrine.
Are you sure? How about the various items that have been shown in Blind testing OTHER THAN THAT BY THE ABX Mob to be audible (eg. certain capacitor differences, differences between speaker cables, differences in absolute polarity to name a few).
Excellent application of the "Heretic" principle. Anyone holding views and opinions not in line with the dogma and doctrine to which you adhere to is "rather suspect".
I completely agree. Where we part company however is the point as to which tree is the wrong one... ;-)
I am all for it. How about you start arranging some tests and then actually first tets them for sensitivity for KNOWN differences and then continue on suspected differences. And please ensure your tests are arranged such that they allow a significance of .05 to be used for both type 1 and type 2 statistcial errors. Secondly make sure that you operate scientifically by using not only a group of established "goldenears" (eg. people who can actually releibaly pass teh ABX test on at least items 1-3 preferablly items 1-4 on my list above) and a control group of "ordinary" people.
Then we MIGHT actually get somewhere.
I am aware of one published test that was carried out using at least a test methodology that allowed the results to be generalised with a reasonably. It was described in outline (and with a basic set of conclusions) in stereophile (see link) but I suggest you obtain the full details from Mr. Ackerman (and learn german if you need to)....
God is in the Nuances
BTW, I do not completely agree with Marcus Sauer as to the other article content (despite being TRiode/Fullrange Speaker etc. oriented), but I found the experiment by Mr Ackerman VERY interesting and revealing, both with regards to actual experimental practice if one desires to find answers to questions instead of producing predetermined and desired results and with regards to the results.
Sayonara
geewhizbang said:
How comes you are so certain? Have you ever considered the CD Error Correction as source of sonical differences? Have you ever had the case where playing music loud the vocalist hit a certain note with a good deal of power and the CD actually jumped (I am not talking about a big bass impulse)? Have you ever had a case where ripped CD which sounded previously bad (usually a result of many micro scratches overtaxing the CD error correction) sounded pretty good after EAC ripping and burning onto a good quality blank?
So, I could think of quite a few logical resonsn even besides the easily proven microphonics of the Master clock X-Tal.
geewhizbang said:
You statement is clearly and objectively proovable in the same category as "Heavier than air flying machines are impossible".
A simple and clear case. More than a few blind tests (including one large scale one supervised by and under the auspices of the AES IIRC) clearly illustrated that absolute polarity (aka phase) is AUDIBLE with Music. The ABX test for the same failed to show this AKNOWLEDGED difference with music and was only able to show it using a test signal.
So, if an ABX test is not done or because it returns a Null result you BY FAR do NOT have any scientific justification to state: "it is just psychoacoustics.". And that in fact and in a nutshell is my umbrage with the ABX Mob. Misanalysed and misinterpreted results of shoddily implemented testing generalised.
geewhizbang said:
You make another statement of the "Heavier than air flying machines are impossible" type. You may say "I cannot imagine or understand any cause / effect mechanism that could change the sound with these feet.". However, as you are far from omnicient (and clearly show a disturbing lack of understanding many basics of the subject) your statement, despite being voiced as statement of fact is actually merely a statement of religious doctrine and highly unscientific.
geewhizbang said:
How can you be so sure?
geewhizbang said:
But do you understand how a CD Player ACTUALLY works? Do you understand what is needed to correctly read the CD and what the error rates even with totally normal CD's are (never mind any extensively scratched or badly pressed)?
geewhizbang said:
Hmmm. The simple fact is that if you vibrate the CD (or player) the error rate will go up increasing the number of samples that are either repeated or interpolated instead of the real samples that should have been read. Surely a mechanical engineer can appreciate how different ways of coupling surfaces subjected to vibration can change the actual amplitude of vibration observed at a given point? Surely you can appreciate that changing the actual points where surfaces are coupled to each other can change resonance frequencies in these surfaces and the amplitude of these resonances? I mean I DO NOT need a degree in mechanical engineering to know that, in fact 6th grade physics suffice.
geewhizbang said:
Yes, your position may very well be right insofar that damping (decoupling) may be preferrable and you are allowed to doubt anything you like.
geewhizbang said:
Again, you state a doctrinal and competely irrational believe as fact. Please discontine these attemps at religous conversiopn (conversion is a process by which you feed one position in words into a mans ears untill they begin to come out of his mouth).
geewhizbang said:
Hmmm. Are you saying that the resistors and capacitors in question (which have real and demonstrable electrical effects which MAY very well be above the audibility threshold - BG Capacitors distortion certainly is) cannot make an audible difference because they are expensive? Now I find that sort of logic hard to stomach. Could you enlarge, while sticking to observed fact and not irrational believe?
geewhizbang said:
Why ABX? Are you that desperate to obtain another "Null" result?
I have previously suggested the following tests as allowing a "calibration" of testers and testsetups for ABX Testing, by using phenomenae KNOWN to be audible:
1) Polarity reversal in one channel only.
2) Polarity reversal in both channels (Absolute Phase/Polarity).
3) 20KHz - 1st order lowpass filter.
4) Radioshack Goldpatch Interconnect vs. RG-213-MIL with WBT locking RCA's
(I agree that item 4 is arguable, the other three are not).
geewhizbang said:
Would it now? Have you EVER tested the actual ABX test in regards to it's ability to show discrimination for small but definitly audible differences?
geewhizbang said:
That is your perogative. You may disagree with anything you like. However, when you state obvious untruth as fact your credibility with rationally thinking people plumets rapidly even though your co-relligionists will no doubt applaud your defense and continued statement of dogma and doctrine.
geewhizbang said:
Are you sure? How about the various items that have been shown in Blind testing OTHER THAN THAT BY THE ABX Mob to be audible (eg. certain capacitor differences, differences between speaker cables, differences in absolute polarity to name a few).
geewhizbang said:
Excellent application of the "Heretic" principle. Anyone holding views and opinions not in line with the dogma and doctrine to which you adhere to is "rather suspect".
geewhizbang said:
I completely agree. Where we part company however is the point as to which tree is the wrong one... ;-)
geewhizbang said:
I am all for it. How about you start arranging some tests and then actually first tets them for sensitivity for KNOWN differences and then continue on suspected differences. And please ensure your tests are arranged such that they allow a significance of .05 to be used for both type 1 and type 2 statistcial errors. Secondly make sure that you operate scientifically by using not only a group of established "goldenears" (eg. people who can actually releibaly pass teh ABX test on at least items 1-3 preferablly items 1-4 on my list above) and a control group of "ordinary" people.
Then we MIGHT actually get somewhere.
I am aware of one published test that was carried out using at least a test methodology that allowed the results to be generalised with a reasonably. It was described in outline (and with a basic set of conclusions) in stereophile (see link) but I suggest you obtain the full details from Mr. Ackerman (and learn german if you need to)....
God is in the Nuances
BTW, I do not completely agree with Marcus Sauer as to the other article content (despite being TRiode/Fullrange Speaker etc. oriented), but I found the experiment by Mr Ackerman VERY interesting and revealing, both with regards to actual experimental practice if one desires to find answers to questions instead of producing predetermined and desired results and with regards to the results.
Sayonara
analog_sa said:
Respect for what you're saying, analog_sa, and this isn't focussed on you, but I'd like to plead with everybody to drop these stupid nicknames that keep cropping up ("Pelicanists" etc.) as alienating others does no service to anyones arguments.
I have noticed this way of mocking people used by political ends on both sides in the UK, and all it does is cause more polarisation and less consensus.
So for the purposes of this debate, could we stick to subjectivist/objectivist worlds? (of course there will be some who inhabit their own patch of the space-time continuum, but lets just humour them).
Cheers,
Arnie.
john curl said:
John,
Think it through! "If the worst caps that we could find can't be detected" why does that AUTOMATICCALLY mean that the test is flawed? Why not conclude: 'It means that there is no audible difference'! And don't tell me that in non-blind testing you DID perceice a difference. That's the whole point! You perceived a difference that, after blind testing was shown NOT to be related to SOUND differences, but to other factors!
There are billions of dollars made by companies based on these simple, well understood psychological issues. I continue to be amazed that the audio crowd is so insecure that they cannot even accept the existence of these phenomena.
Jan Didden
john curl said:
With all respect John, this post is COMPLETELY IRRELEVANT to the discussion at hand. BUT it makes it clear why you would THINK they sound bad - if you know they are in the circuit, that is. Your measurements have set you up so strongly believing they sound bad, you are barred from reaching any other conclusion, to the point that you are ready to denounce any tests that shows non-audibility.
Jan Didden
Steve Eddy said:
Steve,
That is a step that precious few can make. If you see the slew of half-yearly new-look audio products that capitalize on exactly this sentiment, it is amazing that there are still reasonably intelligent people who *claim* to be concerned on the sound only, who believe their own subjective tests more than blind tests.
Jan Didden
john curl said:
John,
I think it is clear that it is not easy to set up a sensible and fair blind test, and often errors are made, unwilling or deliberate. But let me ask you a question: do you believe in blind testing at all? That is, suppose a blind test is set up that satisfies all your and your expert friends' rigourous requirements, and it shows (statistically acceptable to you and said friends) no audible differences, would you accept those results?
Jan Didden
- Status
- Not open for further replies.