AFOM: An attempt at an objective assessment of overall amplifier quality

One example from vintage tube amp world , here`s the graph which represent standard ~35W Ultralinear push-pull A-class OPS formed from two standard 6CA7 (EL34) pentodes , where we can see that dynamic THD consinst only from H2,H3 &H5 and nothing else , that OPS-charachteristic is measured alone, fully loaded ,and open loop of course .
 

Attachments

  • EL34-UL.gif
    EL34-UL.gif
    259.9 KB · Views: 74
I think the practical takeaway from my last few posts is that individual harmonics are a bit of a red herring, a group of harmonics is audible as a spectral peak, and if this peak is frequency modulated then it becomes far, far more audible.
And isn't this, as well, an explanation why we should not consider distortion below 0.00x levels as an unnecessary overengineering, because „nobody can hear that“? Recorded music has its own distortion to start with, and reproduction chain adds interaction opportunities.

Though, I would never accept one amplifier being superior on a single distortion number. At ASR, they are simply silly.
 
  • Like
Reactions: Bonsai and adason
To change a metric along the road just makes it impossible to compare products without a retest... I'd say go for an end game test.

About euphonic etc. sound... could be what is experienced when a truly neutral amp is introduced in a "compensated" system... show me an amp that measures like ABH1 but sound "euphonic"... I really strongly doubt it. But this is where we end up every time "SQ" is brought into the mix.

But OK, if there exists a proper DBT that: in the same system, matched levels to 0.1dB, only changed item is amp, one measures like ABH1 and the other equally good (using the other sites suite); could be identified by listeners - I'll accept it.

//
 
So where is this going to be published and who keeps the records? I'm a bit puzzled by the many references to commercial amps. Why would measuring those be of any interest in a DIY site (other than for reference)?
Another issue is that even with well documented protocols the measurements are bound to be dependent on measuring setups. In ASR all measurements are done by the same person using same measurement setup so those measurements are quite consistent even if they are lacking some important characterics. All in all the focus should be somewhere else than trying to compete with ASR. This is a DIY site, ASR is far from it.
 
The problem is it is genuinely hard to prove anything under 0.01% is audible. If I think I hear a difference, then I do try to reduce it (even to 0.00001%). But I'm fully aware this is silly and I couldn't do it if it wasn't also fun (and what is life without fun). Lately I've been focusing more on speakers, because the improvements I get that way are more rewarding, it seems. And room treatment is often just as important.

As for crossover distortion, many amps show the sign of decreasing distortion as power increases, without anyone complaining about the sound quality. So in many cases it doesn't seem to be important, as much as I dislike it on principle.
 
The tests being proposed should ideally be able to be done with a simple set of tools...
When you test one of your own amps, do you treat it as a black box and only look at its terminal behavior? If so, how do you inspect inner loops for stability?
Also, you appear to know from experience that small, intermittent bursts of oscillation can affect SQ. If so, how do you propose that it should be tested for in a way that will help guide the designer if you are limited to black box tests?

And can you prove it affects SQ by DBT? That was rhetorical, BTW. I find it interesting that people demand DBT when they feel like it and not consistently in every case where there is not formal "proof."

What about testing a big issue in the real world having to do with ground conducted noise, AC line noise, etc.? IMHO its not a serious effort if you don't have tests for real problems. That includes testing both channels at once, just like it will be used in the real world. IMHO it is a joke to measure channels one at a time, the do a quick 20kHz crosstalk measurement and call it done. Regarding crosstalk its not too hard to lay out an amp so that the crosstalk in one direction is different from the other direction. That's bad news when it happens. And what about power supply coupled misbehavior between channels?

Also, as a side note, when people speak of what is audible and what is not, it must be they are assuming the use of speakers like they have and or are familiar with. I have different speakers than most people, and they do make more of the upstream electronics audible. So far only one guy has bothered to visit and see for himself. He wrote about what he heard in another thread. My only point is that what is audible on a system may vary considerably from system to system, yet people sometimes speak of it as a universal absolute.
https://www.diyaudio.com/community/threads/dac-recommendation.376015/post-7560022
 
Last edited:
  • Like
Reactions: bimo
"As for crossover distortion, many amps show the sign of decreasing distortion as power increases, without anyone complaining about the sound quality."

I found it is true when I simulate many amplifiers with single-ended VAS. But symmetries VAS can be different, like this simulations.
 

Attachments

  • 2.png
    2.png
    69 KB · Views: 61
  • 1.png
    1.png
    73.8 KB · Views: 61
So where is this going to be published and who keeps the records? I'm a bit puzzled by the many references to commercial amps. Why would measuring those be of any interest in a DIY site (other than for reference)?
Another issue is that even with well documented protocols the measurements are bound to be dependent on measuring setups. In ASR all measurements are done by the same person using same measurement setup so those measurements are quite consistent even if they are lacking some important characterics. All in all the focus should be somewhere else than trying to compete with ASR. This is a DIY site, ASR is far from it.
This document is not trying to 'compete' with anyone. It's an initiative to foster a more holistic approach to amplifier design. As already stated, it is as much a design guide during the design phase of an amplifier as it is an end-product assessment tool.
 
I do not post much, so FWIW .... Time to focus on getting some thing done, make trade-offs and compromise. Suggestion, pick 6 or so major categories (stability, linearity, load sensitivity, dynamic testing, perceived sound quality, etc). Then establish a list of objective and measureable tests for each category. Maybe a category has no tests, so what? You then have the state of the art from which you can build later. Note that the categories can be rolled up into AFOM or rolled into a spider chart. The individual tests can be entered into a spreadsheet as dimensioned numbers. Just start categorizing and collecting what can be agreed to. First things first though. As PMA points out there needs to be agreement on whether this is a designing or reviewing/evaluating document. Both are interesting and useful.

edit: Looks like I am a bit late. Thank you Bonsai!
Bill
 
  • Like
Reactions: Bonsai
When you test one of your own amps, do you treat it as a black box and only look at its terminal behavior? If so, how do you inspect inner loops for stability?
Also, you appear to know from experience that small, intermittent bursts of oscillation can affect SQ. If so, how do you propose that it should be tested for in a way that will help guide the designer if you are limited to black box tests?

And can you prove it affects SQ by DBT? That was rhetorical, BTW. I find it interesting that people demand DBT when they feel like it and not consistently in every case where there is not formal "proof."

What about testing a big issue in the real world having to do with ground conducted noise, AC line noise, etc.? IMHO its not a serious effort if you don't have tests for real problems. That includes testing both channels at once, just like it will be used in the real world. IMHO it is a joke to measure channels one at a time, the do a quick 20kHz crosstalk measurement and call it done. Regarding crosstalk its not too hard to lay out an amp so that the crosstalk in one direction is different from the other direction. That's bad news when it happens. And what about power supply coupled misbehavior between channels?

Also, as a side note, when people speak of what is audible and what is not, it must be they are assuming the use of speakers like they have and or are familiar with. I have different speakers than most people, and they do make more of the upstream electronics audible. So far only one guy has bothered to visit and see for himself. He wrote about what he heard in another thread. My only point is that what is audible on a system may vary considerably from system to system, yet people sometimes speak of it as a universal absolute.
https://www.diyaudio.com/community/threads/dac-recommendation.376015/post-7560022
Mark,

I can't speak for other designers, but when I design an amp, I look at a whole range of factors. I typically spend two to three months in the paper design and simulation stage. Once that's done, I move to physical implementation and that is around 4 - 6 months to the point where I have something on the test bench and housed that I can call a finished product. These times include mechanical design/layout.

Part of the testing process is to look for some of the problems you list. I've never just designed and built an amp without testing it to the best of my ability.

If you look at some of the tests being proposed, they are designed as black box tests to root out the oscillation issues you raised (e.g. capacitive load testing, recovery from clipping, reactive load drive, etc). For ground noise, I assume you just mean hum and noise. That is pretty easy to measure and there's a lot of good documentation around that discusses amplifier wiring, layout, and PCB design practices to minimize noise. Testing to see if an amplifier is adequate is easy to do and you can make sure the amplifier is all but blameless in this regard.

Most audio test sets, and sound card PC software, simultaneously test both channels.

As mentioned in previous posts, you cannot measure everything. It would be quite easy to come up with a thirty page document and 40 or 50 tests that would put most practitioners off.
 
Last edited:
For ground noise, I assume you just mean hum and noise.
Not only that. There is plenty of evidence, such as that produced by Bill Whitlock, that in today's world there is a lot of conducted EMI/RFI on the AC line and particularly its ground. By various means including mode conversion (common mode to differential mode, or vice versa) it can work its way into an amp and cause problems. Such problems may be measurable with a sound card if one knows what to look for. As far as test generator, it may be that some household appliance motors on the same circuit as the amp are pretty accessible to anyone.

Regarding radiated EMI/RFI susceptibility around 2GHz, a cheap DECT phone bass station in proximity to an amp may produce FFT spurs at 100Hz intervals. That sort of test is low-cost and accessible for most people.

Some reading material attached. Try searching for the terms 'veiled' or 'grainy.' Who knows how to listen for those things? Its not so obvious as FR. And it doesn't necessarily show up as spurs. Again, you have to know what to look for.
 

Attachments

Last edited:
Might be something to look at in a future iteration of AAAP. The problem is you can take precautions in the amplifier design (as Whitlock and others have proposed), but ultimately you will need a systems approach if you are to guarantee no RFI or extraneous noise is getting into the signal chain because system set-up and location is so variable.
 
That's true. A good start is to clean up your test system. How many people really know if even their test system is squeaky clean? Once again, FFT spurs aren't the only thing to look for.

Moreover and IME, most people have probably never heard a really clean system. They don't know how bad their problems are. So, they think this or that effect is inaudible. Sure, its inaudible if buried under "veiled" or "grainy" sound (to use Whitlock's terminology).

In addition, someone who is used to impaired sound (to use ITU terminology) when exposed to really clean sound won't notice how different it is. However, if someone is used to really clean sound and then they are exposed to impaired sound, its hard not to notice the difference (at least that seems to be the common tendency, i.e. for "the average ear").
 
Last edited:
ASR has made an attempt for a complete measurement suite that covers all reasonable aspects of an amp. They highlight one parameter. No summed score.

But I'm not quite sure their suite is complete. And I think a summed score would be interesting and valuable as opposed to concentrate on one. I have high hope that DIYAUDIO can show how its done ;-)

No one would blindly buy on a one figure score. But it would surely help weeding out the nasties and get a well balanced product. I assume all parameters is presented that led up to a score.

//
The problem with one-dimension ratings, as say for example distortion or noise, is that it very quickly gets 'gamed'. I'm not the only one that's noticed a flurry of amps and HP amps, submitted by manufacturers on the other site, with stunningly low distortion. And of course, these products are getting lauded for their transparency, sonic finesse, etc. As I mentioned in an earlier post above, the Benchmark was downgraded because on remeasuring the latest version, distortion was a few ppm higher than the original. Maybe Benchmark went for a few ppm higher distortion because they chose to improve another performance metric for a more balanced overall product. Whatever, they have been punished for it, because people now look at the ratings and say 'The XYZ amp is better because it has 5 ppm lower distortion'.

As I was told time and again by my bosses over the years 'You get what you measure' 🙂
 
  • Like
Reactions: Markw4