What is wrong with op-amps?

Status
Not open for further replies.
"add that to the bias of the ABX test itself towards a null result"

Where does this tendency towards a null result come into play just because the test is an ABX or AB test protocol? That seems to presuppose that this type of testing gives faulty or misleading results. The only time I have seen complaints about ABX testing is here in the audiophile world where some just don't or won't seem to believe that the testing is practical and accurate in its results. Those who insist on peeking seem to be those who are preconditioned to the end results they want to achieve and personal bias is overwhelming the end result.

An ABX test is designed to minimise false positives & as a result is biased towards false negatives
 
The logic is to provide a simple way to make adjustments of distortion levels in test files in the context of a search protocol to find minimal distortion levels people here in the forum can hear on their systems at home.

It might be good to identify the most sensitive listeners and further study their limits.

https://www.quantasylum.com/content/Home/tabid/40/Post/342/ABX-Testing-and-Distortion

It's a good start towards that objective. 🙂

(1audio linked it a little while ago in the BT thread)
 
How about starting with one amp circuit where opamps could be swapped in and out. The circuit should be measured with precision test equipment with each opamp to show that it produces very low distortion in every case.

Then playback recorded cymbals through each opamp and see if anyone can tell a difference.

If so, fine. We have something to investigate further right there.

If not, the circuit could be designed to allow the opamps to amplify their own distortion somewhat, or to provide high gain of an attenuated signal. The idea is to increase measured distortion, until test recordings show audibly detectable differences in the recorded cymbals.

At that point we can infer that most circuits that have some kind of "opamp sound" probably are associated with some minimal distortion level attributable at least in part to the particular circuit topology.

....That's just off the top of my head. Feel free to refine the idea.

Further research could follow, but at least that would be some starting point.

Not the best plan.

you're working from the wrong direction.
work from the easiest to detect back toward least...

...many have said that they are not hearing anything audible from one opamp now.

the hardware I envision would permit blind ABX or other "swapping" by automatic means (in the best incarnation).

IF ur interested, I ought to start a thread on this...
 
Last edited:
Yes and no, respectively.

From Les Leventhal:
"The risk of Type 2 error increases, not only as you reduce Type 1 error risk, but also with reductions in the number of trials (N), and the listener's true ability to hear the differences under test (p). (All of this is explained in great detail in Leventhal, 1985.)
Read more at http://www.stereophile.com/content/highs-lows-double-blind-testing-page-3#5PAkIpjoI2pgRRbH.99"

As you know Type 1 errors are false positives & Type 2 errors are false negatives
 
Bear,
It seems you can look at this op-amp comparison in two ways with multiple devices. One way is to look at the cumulative affect of a series connection of the opamps and the other would be a parallel design with multiple opamps in a parallel configuration which I would think would lower the distortion numbers. Now how this tells you there are any problems with opamps I don't really follow, it just seems to be more towards identifying any particular sound quality an opamp may have compared to another. Of course to work the opamps must be close enough in parameters from one device to another that you can interchange the opamps without creating an unstable circuit. Of course this means that you aren't optimizing the test circuit for any particular opamp, you are just creating a generic circuit that can operate with multiple devices. I don't think you would want to compare a slow 5532 to some high speed devices as the circuits for one would be very substandard for the other.

This whole argument seems about as relevant as saying a car with a carburetor has better pedal feel than a car with fuel injection if both engines produced the same torque and horsepower with the same curves. Nothing wrong with opamps unless you think there is, not much more of an argument than that.
 
Bear,
It seems you can look at this op-amp comparison in two ways with multiple devices. One way is to look at the cumulative affect of a series connection of the opamps and the other would be a parallel design with multiple opamps in a parallel configuration which I would think would lower the distortion numbers. Now how this tells you there are any problems with opamps I don't really follow, it just seems to be more towards identifying any particular sound quality an opamp may have compared to another. Of course to work the opamps must be close enough in parameters from one device to another that you can interchange the opamps without creating an unstable circuit. Of course this means that you aren't optimizing the test circuit for any particular opamp, you are just creating a generic circuit that can operate with multiple devices. I don't think you would want to compare a slow 5532 to some high speed devices as the circuits for one would be very substandard for the other.

This whole argument seems about as relevant as saying a car with a carburetor has better pedal feel than a car with fuel injection if both engines produced the same torque and horsepower with the same curves. Nothing wrong with opamps unless you think there is, not much more of an argument than that.

Truly, I worry about some of you. Heck I worry about myself too, but for different reasons.

First off you come to a conclusion in your final paragraph there that is entirely unwarranted. No basis.

This is stupid simple.
Why no one has done this, I dunno.
If there's a reason, then I'll be the stupid one.
fine.

The case envisioned is 10 (to make a round number and to keep the math easy)in series. How that is accomplished is an implementation issue that needs discussion, but this is the conceptual phase.

So since distortion multiplies, and noise adds... you get a different result than using just one.

One could have "N" channels of 10 opamps available, but for now we can consider two for simplicity. One loaded with your "control" opamp, eg 5534 or 4558, etc. (can't hear any difference between them anyhow) and some other opamp to compare with eg AD797 or something like that... assume that each opamp string has been duly engineered per string, checked in the real world for "proper operation". (and documented)

That's it for starters in basic form. Simple enough.

The results in measurement (for AUDIO applications) will be null, aka no differences measured or heard, according to some. This is what remains be seen.

There are all manner of permutations that can be tried, but the simple idea is enough in concept to see what it is and where it leads?
 
Last edited:
What we need is a definitive test for GOOD / NO-GOOD.

Maybe animal testing , "this is your brain on op-amps" something like that?
 

Attachments

  • OP-amp-brain.png
    OP-amp-brain.png
    44.2 KB · Views: 217
From Les Leventhal:
"The risk of Type 2 error increases, not only as you reduce Type 1 error risk, but also with reductions in the number of trials (N), and the listener's true ability to hear the differences under test (p). (All of this is explained in great detail in Leventhal, 1985.)
Read more at http://www.stereophile.com/content/highs-lows-double-blind-testing-page-3#5PAkIpjoI2pgRRbH.99"

As you know Type 1 errors are false positives & Type 2 errors are false negatives

That is a detail of the specific trial design, not inherent to an ABX test. The author is juggling around the number of repetitions, study size, and pre-trial standards. More N and more repetitions are in a "pure sense" better at reducing errors (within reason), but that's regardless an ABX, triangle, forced pairing, etc. With real human beings fatigue is a real concern, which limits repetitions.

Small N and p=0.05 are only going to confidently catch very large effects.

Getting trickier by the moment!

What's the hypothesis being presented?
 
Last edited:
From Les Leventhal:
"The risk of Type 2 error increases, not only as you reduce Type 1 error risk, but also with reductions in the number of trials (N), and the listener's true ability to hear the differences under test (p). (All of this is explained in great detail in Leventhal, 1985.)
Read more at http://www.stereophile.com/content/highs-lows-double-blind-testing-page-3#5PAkIpjoI2pgRRbH.99"

As you know Type 1 errors are false positives & Type 2 errors are false negatives

Yes, I'm aware of that. And I read that special pleading when it came out. Unfortunately, his assertions are contradicted by experiment, but they sure help those folks for whom mystery and hucksterism are their means of obtaining cornpone.
 
That is a detail of the specific trial design, not inherent to an ABX test. ...

What's the hypothesis being presented?

See Tom Nousaine's disassembly of Leventhal's argument.

In any event, if you can hear a difference between two boxes of gain, not knowing which is which isn't going to change the sound. There's no special insight from peeking. So folks who are claiming to hear differences between opamps absent an obvious measurable problem have the burden of proof. Until that point, the claims are as plausible as those of alien abductions with anal probes.
 
That is a detail of the specific trial design, not inherent to an ABX test.
Yes, it is inherent to an ABX designed test. Please detail what is the 'specific trial design' which isn't inherent to ABX testing
The author is juggling around the number of repetitions, study size, and pre-trial standards. More N and more repetitions are in a "pure sense" better at reducing errors (within reason), but that's regardless an ABX, triangle, forced pairing, etc. With real human beings fatigue is a real concern, which limits repetitions.
He's purely specifying the statistics behind ABX testing for sample sizes, level of confidence.

Small N and p=0.05 are only going to confidently catch very large effects.

Getting trickier by the moment!

What's the hypothesis being presented?
Only a few posts ago you stated that DIY audio users typically can only do "non-academic, non-commercial" listening tests & now that fatigue is a real concern which leads to what I stated "ABX testing is biased towards null results" for what we typically see as ABX tests on audio forums.
 
Daniel, some excerpts:
Dr. Leventhal offers good basic statistical advice, but falls into a trap I've often found myself in. It's easy to forget that an experiment is valid based on its design, and that statistics only report on the reliability of the results. I also believe that the binomial parameter p, which the author uses as an analog for listener sensitivity, is quite high relative to the position established by the audiophile camp and in actual practice. This renders the author's conclusions about fairness moot.

Experiments are made valid (ie, measure what they claim to measure) by good design, not by statistical analysis. ...validity can only be determined by examining the test and its inherent characteristics. Leventhal is right by concluding that aggregation of "unfair" results is unfair, but he fails to examine the test itself for fairness. Statistics are just numbers. They are neither fair nor unfair. Numbers just don't care.

Fairness and high sensitivity are just what makes the ABX method so appealing: it contains the validity elements that constitute a fair test. Listener and administrator bias are controlled by concealing the identity of the device under test. The listener gets direct, level-controlled access to the device under test, the control device and X, with multidirectional switching and user-controlled duration....

How about sensitivity? Les Leventhal makes his entire fairness case around the idea that subtle differences may only be present 60-80% of the time during the tests. When p approaches 0.9 (differences present 90% of the time), the fairness coefficient evens up and even a 16-trial test meets all criteria for both Type 1 and 2 error. Notice that probability of error is not the same as actual error. Even a perfect one-trial experiment would have an unacceptably high risk of Type 1 and 2 error. So what makes for a sensitive listening test? What actual values can we expect for p?
A casual survey of any of the underground magazines shows that audiophiles typically find it fairly easy to perceive differences. Leventhal implies that p may be a low value when there is nothing in the audiophile position to support such a notion. Read any decent "audiophile" review and draw your own conclusion as to the value of p inherent in their position.

An examination of the 16-N tests referenced by Dr. Leventhal reveals conditions indicative of high sensitivity. Clark and Greenhill auditioned the devices under test prior to the test to identify sonic characteristics. The ABX blind tests were performed using their personal reference systems, with familiar program material and at their leisure. I find it difficult to believe that this procedure might have a sensitivity of under 0.9.

...If the differences are only present on 60% of all the program material available, and if your material is chosen from a random sample, then the sensitivity issue might apply. However, the identification of material where differences are present is imperative for sensitive testing. It also enables us to test for differences that may only be present 10%, or even 1%, of the time. We can make these tests by selecting programs in which differences are present 100% of the time during the test. It seems to me that this is what audiophiles do, and precisely what Clark and Greenhill, Shanefield, Lipshitz and Vanderkooy, et al, do also...

But all of this evades the central point- people making claims about IC opamps have the burden of proof. Speculation about mechanisms outside of obvious measurements can wait until someone, anyone, demonstrates the audibility of these unmeasured differences. All the evasion, foot-stamping, special pleading, sneering, and transparently commercial huckstering does not provide even a scintilla of evidentiary value.
 
Yes, it is inherent to an ABX designed test.

Does ABX mean something different to you than:

1.) being presented A,
2.) being presented B and
3.) then asked if X is A or B?

Uh, samples size and repetitions and pre-trial defined standards (and endpoints) are useful specifications to any trial. Whether that's a clinical trial, discovery criteria for the American Physical Society, or your ABX trial on the couch. Pretty sure I was asked to define these things, to whatever degree of success, for my HS biology class project.
 
See Tom Nousaine's disassembly of Leventhal's argument.
Link?

In any event, if you can hear a difference between two boxes of gain, not knowing which is which isn't going to change the sound. There's no special insight from peeking. So folks who are claiming to hear differences between opamps absent an obvious measurable problem have the burden of proof. Until that point, the claims are as plausible as those of alien abductions with anal probes.
Blind listening tests are statistical tools & when the reality of the statistics underlying these tests are not to the liking of those pushing blind listening, deflection is in order
 
Status
Not open for further replies.