I woke up a bit cranky this morning and just wanted to rant about my biggest gripe with the DIY audio community: the lack of objective measurements. I was reading reviews of a kit I’m bidding for on e-bay, and couldn’t find a single review with any measurement data. They all described their own subjective opinion of the sound. And the evaluations were all over the place; everywhere from “the best 2-way monitor I’ve ever heard!” to “pretty good for the price”
Even when I read reviews in major magazines written by professionals, they give a brief overview of their measurements, if they have any, then say something like “Now on to the important stuff: What do they sound like?”
I’m a scientist, and I know that no publication would dream of publishing subjective opinion as data. It is far too influenced by the person’s expectations, context in which the judgment is made, and desires (what the person wants to see/hear). With respect to audio: How a component sound to one person playing their music in their room connected to their series of components tells me very little about how it will sound to me playing my music in my room connected to my stereo.
Of course, proponents of subjective evaluations will point out that no objective measurements can tell one how a component will sound. Human hearing is too complex, and too poorly understood to be reduced to a series of numbers and graphs. I agree with this completely, but that doesn’t mean that objective measurements have no value.
My point is this: It is impossible to accurately communicate what a piece of equipment will sound like, neither through measurements nor subjective evaluation. However, given the choice, objective measurements, done correctly, are far superior. If they are used ubiquitously, readers will learn to correlate measurements with their own perception of sound.
Also: I realize that in the past, DIY builders had no choice but to describe their creations in subjective terms, because testing equipment was too expensive. But now, with all the free or cheap software, like SpeakerWorkshop, the FRD Consortium or True RTA, instructions for a Wallin Jig, and with the low price of measurement microphones like the Behringer ECM8000 or Dayton EMM-6, quality testing equipment is within reach of any DIYer. If you can afford a few drivers and crossover parts, you can afford equipment to measure what you made with them.
So, sorry for the long rant, but I really think if more DIYers posted their measurements with their designs, the whole community would benefit.
-Byron Taylor
Even when I read reviews in major magazines written by professionals, they give a brief overview of their measurements, if they have any, then say something like “Now on to the important stuff: What do they sound like?”
I’m a scientist, and I know that no publication would dream of publishing subjective opinion as data. It is far too influenced by the person’s expectations, context in which the judgment is made, and desires (what the person wants to see/hear). With respect to audio: How a component sound to one person playing their music in their room connected to their series of components tells me very little about how it will sound to me playing my music in my room connected to my stereo.
Of course, proponents of subjective evaluations will point out that no objective measurements can tell one how a component will sound. Human hearing is too complex, and too poorly understood to be reduced to a series of numbers and graphs. I agree with this completely, but that doesn’t mean that objective measurements have no value.
My point is this: It is impossible to accurately communicate what a piece of equipment will sound like, neither through measurements nor subjective evaluation. However, given the choice, objective measurements, done correctly, are far superior. If they are used ubiquitously, readers will learn to correlate measurements with their own perception of sound.
Also: I realize that in the past, DIY builders had no choice but to describe their creations in subjective terms, because testing equipment was too expensive. But now, with all the free or cheap software, like SpeakerWorkshop, the FRD Consortium or True RTA, instructions for a Wallin Jig, and with the low price of measurement microphones like the Behringer ECM8000 or Dayton EMM-6, quality testing equipment is within reach of any DIYer. If you can afford a few drivers and crossover parts, you can afford equipment to measure what you made with them.
So, sorry for the long rant, but I really think if more DIYers posted their measurements with their designs, the whole community would benefit.
-Byron Taylor
I would challenge you to set a standard for DIY Audio. Publish a Wiki on measurement, stating what to measure, and how to interpret the results. Then run through it on each of your builds.
If this looks worthwhile, I will follow, and I assume others will as well.
I did a 5 year stint as a test engineer (Network) and realize how daunting truly meaningful measurement is.
OTOH, My brother rejects almost all measurements as irrelevant.
Note there are only a handful of manufactures that publish anything approaching meaningful data on their products.
Just my 2 cents.
Doug
If this looks worthwhile, I will follow, and I assume others will as well.
I did a 5 year stint as a test engineer (Network) and realize how daunting truly meaningful measurement is.
OTOH, My brother rejects almost all measurements as irrelevant.
Note there are only a handful of manufactures that publish anything approaching meaningful data on their products.
Just my 2 cents.
Doug
I don't know, I'd say the opposite. What I like about DIY is that everyone tries to measure their projects (if they are able) and posts data about their performance. Some people just take the plans or circuits of others and build them with no test equipment- it takes a deep commitment to diy to start acquiring test equipment.
What I don't like about professional reviews and high end products is that they supposedly have made the commitment but professional reviewers don't (for the most part) publish, take or care about measurements.
Oh, and every single pro review of a bookshelf or small monitor will eventually say "well its bass is not deep but you will be surprised how much bass this little speaker provides...".
What I don't like about professional reviews and high end products is that they supposedly have made the commitment but professional reviewers don't (for the most part) publish, take or care about measurements.
Oh, and every single pro review of a bookshelf or small monitor will eventually say "well its bass is not deep but you will be surprised how much bass this little speaker provides...".
Hi Byron
I've complained about the issue in different ways, in different places on many occasions.
Professional reviewers have promoted this approach. In fact it seems to be impossible the be a professional reviewer who depends largely on measurements these days. This creates a big problem for designers, because it is pot luck in performance terms as to whether the item you have designed will be panned or lauded.
One way of improving your chances of success is to include components which are 'audio grade', or simply expensive, or are drawn from the list of those which are 'flavour of the month' this month.
It's an appalling situation which arose round about the time when the standard of audio electronics improved to the point where the majority of amplifiers were of acceptable quality. The reviewers realised that they were being made redundant by improvements in technology, and when someone (known, but who will remain nameless here) invented 'subjective' reviews, the rest were quick to jump on the bandwagon.
I agree with lgreen that the standards in DIY are better than in professional circles, but it's difficult for even the best of us to keep our heads above water, because the environment in which we swim has become so polluted with disinformation.
w
I've complained about the issue in different ways, in different places on many occasions.
Professional reviewers have promoted this approach. In fact it seems to be impossible the be a professional reviewer who depends largely on measurements these days. This creates a big problem for designers, because it is pot luck in performance terms as to whether the item you have designed will be panned or lauded.
One way of improving your chances of success is to include components which are 'audio grade', or simply expensive, or are drawn from the list of those which are 'flavour of the month' this month.
It's an appalling situation which arose round about the time when the standard of audio electronics improved to the point where the majority of amplifiers were of acceptable quality. The reviewers realised that they were being made redundant by improvements in technology, and when someone (known, but who will remain nameless here) invented 'subjective' reviews, the rest were quick to jump on the bandwagon.
I agree with lgreen that the standards in DIY are better than in professional circles, but it's difficult for even the best of us to keep our heads above water, because the environment in which we swim has become so polluted with disinformation.
w
You're forgetting one important fact. Many professional reviewers (be it web or in magazines) are paid by the manufacturers to review their equipment. It may not be in the form of monetary payments, but a friend of mine has received a complete high-end stereo in return for favorable reviews in the magazine he used to write for. I remember a few magazines that were highly dependent on the advertising revenue from a speaker manufacturer. Somehow, all speakers made by that manufacturer always got rave reviews in that magazine. Odd how that works. How do you spell "conflict of interest" again? 
Besides, reviewers rarely are scientists. They don't know how to take the measurements or interpret them. Many times they seem to simply relay the marketing drivel that the manufacturers put out.
~Tom
Besides, reviewers rarely are scientists. They don't know how to take the measurements or interpret them. Many times they seem to simply relay the marketing drivel that the manufacturers put out.
~Tom
You're forgetting one important fact. Many professional reviewers (be it web or in magazines) are paid by the manufacturers to review their equipment. It may not be in the form of monetary payments, but a friend of mine has received a complete high-end stereo in return for favorable reviews in the magazine he used to write for. I remember a few magazines that were highly dependent on the advertising revenue from a speaker manufacturer. Somehow, all speakers made by that manufacturer always got rave reviews in that magazine. Odd how that works. How do you spell "conflict of interest" again?
Besides, reviewers rarely are scientists. They don't know how to take the measurements or interpret them. Many times they seem to simply relay the marketing drivel that the manufacturers put out.
~Tom
The instant we let professional reviewers into the argument things spiral down fast. People on eBay remarking "slightly sibilant highs mellowed by a marmaladey midriff" are just apeing the nonsense they read in magazines and on the internet. The entire hi-fi industry is so jam-stuffed full of rubbish it makes fox news political discourse look like a lecture on newtonian physics.
Hi fi is a *perfect* scam. I have known people with real money problems who were, nevertheless, addicted to buying all the esoteric gear and 'tweaks'.
As soon as the objective measurements were deemed worthless, these people were on the hook forever; they could never shake off the doubt that their system wasn't 'musical'. The measurements that said they were, basically, perfect were apparently meaningless. People could churn out any old tat that looked 'minimal' or finished in exotic materials, or even return to 1920s technology, and ignore the measurements that showed they were basically rubbish. And an entire industry devoted to selling oxygen-free cables, anti-resonance beads and clarity creams was born, because none of these things could be measured.
The story of Peter Walker of Quad seems to be quite relevant to this subject.
As soon as the objective measurements were deemed worthless, these people were on the hook forever; they could never shake off the doubt that their system wasn't 'musical'. The measurements that said they were, basically, perfect were apparently meaningless. People could churn out any old tat that looked 'minimal' or finished in exotic materials, or even return to 1920s technology, and ignore the measurements that showed they were basically rubbish. And an entire industry devoted to selling oxygen-free cables, anti-resonance beads and clarity creams was born, because none of these things could be measured.
The story of Peter Walker of Quad seems to be quite relevant to this subject.
This is a wonderful place because there are all types here- those who believe in measurement and those who eschew it. I'm a measurement junkie so am in perfect agreement with your rant. OTOH...
System lacking pace and rhythm? Are your woofers too slow? Is your frequency domain performance out of sync with your time domain performance? Is that 4pS jitter wrecking your soundstage? No problem. Just send me $499.95 and I'll use my secret process to specially treat 250 cc of rare dihydrogen oxide that, placed next to your amplifier, will compensate for all those problems and yield perfect sound.
CH
System lacking pace and rhythm? Are your woofers too slow? Is your frequency domain performance out of sync with your time domain performance? Is that 4pS jitter wrecking your soundstage? No problem. Just send me $499.95 and I'll use my secret process to specially treat 250 cc of rare dihydrogen oxide that, placed next to your amplifier, will compensate for all those problems and yield perfect sound.
CH
Bingo!
When it is easy (and cheap) to find amps which have (measured) flat response and very, very low distortion, what's left to talk about (in the amp area)?
Hello All,
I’d like to respond to some of the comments on my initial rant. I agree with Pano and Raypalmer when they say that this particular forum does have a lot of data relative to other forums and to manufacturer’s web sites. I appreciate the effort these people have made publishing their data and don’t mean to dismiss their efforts. I wrote my rant after searching 70 consumer reviews of an old Madisound kit at DIY AudioREVIEW and finding no useful information in the first 30 or so (then I stopped looking).
In response to DougL’s comments: Many rudimentary measurements are already pretty standard, such as frequency-response plots done with 2.83 volts measured at 1 meter. Though even then, some people sweep a sine wave, and some generate white noise and analyze the spectrum. An impedance curve is something most hobbyists can generate and anyone can interpret. I think that creating a wiki to standardize audio measurements is a great idea. Unfortunately, I’m a biologist, and an amateur at the audio thing, so I’m not the person to do it. At the same time, I suspect that creating a lot of exacting standards for the DIY community would be like herding cats, but it would be better than nothing.
Anyway, I’m much less grumpy this morning. My new soldering iron arrived and I love it and I’m going to finish my project today. The Stahl Tools TCSS is on sale at Parts Express. If you don’t like your current iron, rush out and buy this before the sale ends.
-Byron
. . . who does not work for PE and did not receive payment for telling everyone to buy PE’s product.
I’d like to respond to some of the comments on my initial rant. I agree with Pano and Raypalmer when they say that this particular forum does have a lot of data relative to other forums and to manufacturer’s web sites. I appreciate the effort these people have made publishing their data and don’t mean to dismiss their efforts. I wrote my rant after searching 70 consumer reviews of an old Madisound kit at DIY AudioREVIEW and finding no useful information in the first 30 or so (then I stopped looking).
In response to DougL’s comments: Many rudimentary measurements are already pretty standard, such as frequency-response plots done with 2.83 volts measured at 1 meter. Though even then, some people sweep a sine wave, and some generate white noise and analyze the spectrum. An impedance curve is something most hobbyists can generate and anyone can interpret. I think that creating a wiki to standardize audio measurements is a great idea. Unfortunately, I’m a biologist, and an amateur at the audio thing, so I’m not the person to do it. At the same time, I suspect that creating a lot of exacting standards for the DIY community would be like herding cats, but it would be better than nothing.
Anyway, I’m much less grumpy this morning. My new soldering iron arrived and I love it and I’m going to finish my project today. The Stahl Tools TCSS is on sale at Parts Express. If you don’t like your current iron, rush out and buy this before the sale ends.
-Byron
. . . who does not work for PE and did not receive payment for telling everyone to buy PE’s product.
I won't allow dihydrogen oxide in my house. In large quantities it is known to be toxic - it kills hundreds of people every year around the world, although some claim that this is due to the natrium monochloride contamination often found in it.
DO is also a high permittivity material so it is bound to interfere with the electric waves which audio equipment relies on.
DO is a powerful solvent - in fact it will dissolve almost anything so once you let any DO in your house you never know what might happen. Your whole house might disappear. DO is as dangerous as the LHC!
DO is also a high permittivity material so it is bound to interfere with the electric waves which audio equipment relies on.
DO is a powerful solvent - in fact it will dissolve almost anything so once you let any DO in your house you never know what might happen. Your whole house might disappear. DO is as dangerous as the LHC!
- Status
- This old topic is closed. If you want to reopen this topic, contact a moderator using the "Report Post" button.