Agreed by whom? Clear to whom?
Perhaps it's a difference in a translation? I would say that statement is categorically false. I also consider you one of the smartest folks around. So, perhaps we're using different definitions.
What I mean by objective is:
- "(of a person or their judgment) not influenced by personal feelings or opinions in considering and representing facts."
What I mean by subjective is:
- "based on personal opinions and feelings rather than on facts."
Your entire scoring and weighting system is / will be based upon opinion. Whether that opinion is shared by many and then thrust upon (told) to those whose products might be scored, does not make it any less subjective. Taking an objective measurement result and assigning a weighting or score to it based on an opinion makes it subjective. That has been clearly pointed out to you repeatedly.
With how many discussions there are around objective vs. subjective perhaps it would lower the noise if you told us what specifically you mean by those terms. Your definition can stand throughout the rest of the thread, but we'll at least understand your POV re: subjective and objective more clearly. Perhaps you'll convince me to use a different definition or to understand how your entire system is not a matter of opinion.
I would like to see the one that is not. I am not aware of any. Even the 5W/1kHz/4ohm SINAD chart at ASR is based on Amir's opinion that it is the most important parameter. We all have our opinions based on our experience. And I am saying that the experience is not transferable.
They are okay for PSS distortion, providing the harmonics are weighted appropriately. Its already a choice on the menu, with no new recipe needed.
I would still like to see some some measurement(s) of modulated signals. I suppose a selected multitone signal could suffice, but would any multitone test be reducible to a single meaningful figure of merit number?
Also, would it be possible to measure distortion with both channels driven? Or is it another problem that won't reduce to a simple number?
As far as that goes, how does one reduce oscilloscope displays to a single number. If that can be done for other tests, how about for distortion residuals?
Why is it subjective if the best vacuum tube amplifiers achieve 0.1% to 0.5% distortion? If we measure the amp and assess it within what the technology is capable of, and it performs well to those standards, then surely the amp is a good one? That is all my post said. Although distortion on a vacuum tube amp may be high by solid-state standards, you don't get rail sticking or slew rate limiting, etc so they score well in those areas. Where they can improve is in hum and noise (see for example Stereophile - many tube amps have high mains related hum and noise). Very low hum and noise on tube amps is possible - Sy Yanniger who used to frequent this site has shown some designs with hum levels down at -90 or -100 dB.
If you don't consider the broad brush of what is out there, then all we end up with is giving low-distortion solid-state amps high ratings, and everything else gets a bad rating.
Most audio test software today measures both channels simultaneously and you can set the generator part up independently as well - so for example, if you wanted to do cross-talk. So I am in agreement with your suggestion.
It does not appear we can extract meaningful single-digit data from tests that use some form of modulation. You get a plot, but then that's open to all sorts of visual interpretation because it is complex to interpret.
A workaround here (not a modulation signal as proposed by PMA though) might be to do a series 10 or so 2 tone IMD tests (eg 60Hz + 4 kHz) at multiple frequencies and then extract the IMD harmonics and have an agreed way of presenting this. As we discussed earlier in the thread, the challenge with existing multi-tone tests is that the total signal power limits the peak tone levels to just a few volts, so you are not exercising the amp in the large signal domain where non-linearity is more apparent.
It was explained zillion times and even in this thread that nonlinear distortion (call it harmonic, intermodulation, multitone - still the same nonlinearity description) is the worst measure of all to define amplifier “sound quality”.
How could two amplifiers have a same low distorsion if one is has insufficent slew rate since slew rate limitation will forcibly translate in high IMD ?
Chasing high slew rates is a misunderstanding of the real physics of an amplifier, it surely find its origins in the 70s when they noticed that a limited slew rate would produce lots of IM and as such was a limiting factor for linearity, so increasing the slew rate above the level at wich it has no more an influence on linearity is like using the same cure for all other possible diseases....
A musical instrument sound will appear as consistent if there s lots of even harmonics, but the more the odd harmonics the more the brillance of the sound, so odd harmonics are not that unmusical as one would think, any instrument that sound brillant has lots of them, back in the 70s analog synthesizers were often using square waves as base signal and then proceding to substractive synthesis with parametrable low passes filters, aka VCFs.
So here is where the suite needs to improve so that these things show on the outside... black box! Or perhaps they already do but we cant interpret the measurements properly?
//
As a scientist and audiophile I am interested in the science behind music reproduction.
I am not interested in some type of "static score rating" imposed based on a set of some purely engineering criteria. These criteria are themselves based on a subjective decision because we really DON'T KNOW what makes a good audio system.
I believe such criteria will work to "Bose" the market with a race to the bottom. If consumers are going to use such 'standard" then adventurous designs will never see the light of day... and it is in the adventure, going against orthodoxy, where real progress is made.
Just ask around... ask about the upgraded diodes in the XA252 SIT. By your measure of "quality" that amp would be zero... yet, it sounds fantastic!
I do not know what measurements are required, I just want to see a heuristic approach with an end point of enlarging the science of psychoacoustics... I provided you with a sample laboratory process but you seem to ignore it. My "lab" includes an iterative process with "reference" controls and both electronic measurements and objective listening by a sufficiently large and diverse panel of listeners. Please no double blind.... we are not trying to hear differences between devices, we are trying to hear differences between the original performance and the reproduction.
How large is this panel?
What we are going to measure?
What will the reference system look like?
I don't know, if I knew, we'd be 2/3rds done. This is what scientific research is like.
Let me note Nelson Pass' approach. He has stated that he measures to ensure some basic "quality" of his designs... but then he listens to them and adjusts accordingly. Now, his formal background was in Physics, which makes a lot of sense. But being involved in a commercial product, he stops after finding "what" makes it sound better.
What I'm interested in is "why" it sounds better.
BTW, I'm s till waiting to see WHAT IS YOUR DEFINITION OF QUALITY?
My definition of "audio quality" is the faithful reproduction of a music event.
My definition of "commercial audio quality" is something that has "audio quality" and will not burn down the house.
There.
Last edited:
So when the frequency response is totally messed up with peaks and troughs all over the place, you call it "faithful to its input"?
Maybe thats how it is? 🙂
I mean - how many amps have measured super and sounded sheit? I know I know, thats the tale. But are we really sure?
My suggestion:
- Accept the current typical measurement suite (objective)
- Add what might be missing (objective)
- define a summed, total score. (objective, if... read on...)
I actually think that if a suite contain e.g. 9 measurements that are all technical relevant to describe an amp performance, I don't see any need to weigh them individually. Just do a mean? But in order to be able to do that, one have to make up a scale for each measurement so that one could tell how well an amp did in say % achievement. The one net result would be a % figure. a 77% good amp.
Lets make each measurement scale upper limit really high -we dont want to get into figure skating conundrum where skaters started to get all 10s.
But by all means - set it so that it can be theoretically achieved. And most, if not all, would agree that a 100% would be transparent in all aspects. This of course disqualifies e.g. power as I think it would be perfectly OK to have one 10 watt amp go 100% - one just takes the by the producer advertised power and use that as a reference - if all is good at that power - 100% can be reached.
This would add something to the state of the nation I think.
//
Well, in fairness... it sounds fantastic for over an hour because it doesn't self-immolate after 30 minutes once the "upgraded" diodes were installed. Call them what they are... They're just a more appropriate part for the job. I assume we're talking about the same thing.
Tracking with this thread... To me, that's really a reliability factor vs. a "measurement" or "sonics" factor. If it were shipped commercially in that fashion, in my opinion, it deserves a 0 for quality. It can rate a 5,000 for sound or however it gets scaled. As Mark put it so well earlier with more technical terms... the results should not interfere or "beat" one another. It was highly unreliable, and it would deserve to be rated as such regardless of whether it allowed the angels to sing sweet nothings directly into your ears.
If the testing suite being proposed included hours (and/or something that pushes the limits of an amplifier) on the bench under loads, then something like that might reveal itself. I look forward to seeing the "stress tests". That's one of the areas I am very curious about as a DIYer.
re: how something sounds... off the table, I hope. Why do I want measurements around someone else's idea of what sounds good? Others may, but it's OT for this measurement suite, I hope.
^
emphasis added mine. +10000Or just scale each measurement ... who cares if the "score" goes beyond 100?
One of the things that's been unclear to me from the beginning is, "who's the audience?".
Do we really think circuit designers are not intelligent enough to digest the actual results? OR is what Bonsai really wants to do is INFLUENCE the designers to move more toward some agreed weighting of "goodness" through a weighted scale?
Both are fine... just some level of transparency around who this system is actually intended for and its overall goals (if successful) would be brilliant.
Me, I just want facts / data. Others can interpret it in their own way... I'll interpret it in mine. Heck, I'd read a design guide written by bonsai in a heartbeat. I'd alter my designs based on his input. What I don't need is some "scale" to influence my choices.
However, if the goal is to influence a "market" or "consumers" or "designers" fantastic. I can still support that. I'll just still beg for the data.
Any type of static "objective" rating based upon a subjective guess at to what constitutes a valid set of criteria is fundamentally fallacious because it is based on an initial "subjective guess".
The XZ252STI... aka Macho SIT, is a prototype. Pretty much everything made as DIY is a prototype and it tends to advance the science... DIY Audio is a proactive forum where new things are being proposed and built. It's as pure R&D as you'll ever see (THANK YOU NELSON, MARK, ALEK, etc, etc...).
But 99.99% of the audio world is purely based on commercial products, purchased by consumers and supported by a reactive media. The R&D is hidden and filtered by layers of profit margins, liability lawyers, bank statements, etc...
So, does this quest to come up with a measure of quality make any sense in the DIYAudio world? I very much doubt it. Would you ever see an amp with heatsink running 55F -or more?- No Way Jose. Would you see a design where it tends to immolate itself when you play some good old Can Can? Nyet!
If I wanted simplicity and believed marketing, I'd overpay for a Bose and be done with it.
The XZ252STI... aka Macho SIT, is a prototype. Pretty much everything made as DIY is a prototype and it tends to advance the science... DIY Audio is a proactive forum where new things are being proposed and built. It's as pure R&D as you'll ever see (THANK YOU NELSON, MARK, ALEK, etc, etc...).
But 99.99% of the audio world is purely based on commercial products, purchased by consumers and supported by a reactive media. The R&D is hidden and filtered by layers of profit margins, liability lawyers, bank statements, etc...
So, does this quest to come up with a measure of quality make any sense in the DIYAudio world? I very much doubt it. Would you ever see an amp with heatsink running 55F -or more?- No Way Jose. Would you see a design where it tends to immolate itself when you play some good old Can Can? Nyet!
If I wanted simplicity and believed marketing, I'd overpay for a Bose and be done with it.
Phono preamps aside, no, but most 0.001% amps are razor flat in response for the same reason they have low harmonic distortion, the feedback network is basically two decent metal-film resistors, defining both the linearity and the flat frequency response. As I said its the baseline. You probably want a quiet amp too, and a robust one. It should all be in the datasheet. You wouldn't buy a car without a datasheet, why buy an amp without one? And clearly different specs are important to different people and uses, but for a vehicle mpg is often very important, and an amp THD is often very important. For in-car audio noise performance is clearly less important than for a headphone amp - there cannot be a single figure of merit for a complex product.
Right on, brother!
Where do,you get the idea that there is a ‘panel’? There is no panel. There is no ‘reference system‘. What is going to get measured is what is in discussion.
No one is forcing you to submit your amplifier to anyone for review and no one is imposing this assessment framework on you. If you do not want to participate then don’t - if you have a system for assessing your amplifier in place with an experienced listening panel then please use it but don’t raise objections about this one that do not exist.
If your interest is in why it sounds better then feel free to start your own discussion group.