Rant about lack of objective measurements in DIY audio - diyAudio
Go Back   Home > Forums > Member Areas > The Lounge

The Lounge A place to talk about almost anything but politics and religion.

Please consider donating to help us continue to serve you.

Ads on/off / Custom Title / More PMs / More album space / Advanced printing & mass image saving
Reply
 
Thread Tools Search this Thread
Old 5th March 2011, 04:43 PM   #1
diyAudio Member
 
ByronInPortland's Avatar
 
Join Date: Nov 2010
Location: Portland, Oregon
Default Rant about lack of objective measurements in DIY audio

I woke up a bit cranky this morning and just wanted to rant about my biggest gripe with the DIY audio community: the lack of objective measurements. I was reading reviews of a kit I’m bidding for on e-bay, and couldn’t find a single review with any measurement data. They all described their own subjective opinion of the sound. And the evaluations were all over the place; everywhere from “the best 2-way monitor I’ve ever heard!” to “pretty good for the price”

Even when I read reviews in major magazines written by professionals, they give a brief overview of their measurements, if they have any, then say something like “Now on to the important stuff: What do they sound like?”

I’m a scientist, and I know that no publication would dream of publishing subjective opinion as data. It is far too influenced by the person’s expectations, context in which the judgment is made, and desires (what the person wants to see/hear). With respect to audio: How a component sound to one person playing their music in their room connected to their series of components tells me very little about how it will sound to me playing my music in my room connected to my stereo.

Of course, proponents of subjective evaluations will point out that no objective measurements can tell one how a component will sound. Human hearing is too complex, and too poorly understood to be reduced to a series of numbers and graphs. I agree with this completely, but that doesn’t mean that objective measurements have no value.

My point is this: It is impossible to accurately communicate what a piece of equipment will sound like, neither through measurements nor subjective evaluation. However, given the choice, objective measurements, done correctly, are far superior. If they are used ubiquitously, readers will learn to correlate measurements with their own perception of sound.

Also: I realize that in the past, DIY builders had no choice but to describe their creations in subjective terms, because testing equipment was too expensive. But now, with all the free or cheap software, like SpeakerWorkshop, the FRD Consortium or True RTA, instructions for a Wallin Jig, and with the low price of measurement microphones like the Behringer ECM8000 or Dayton EMM-6, quality testing equipment is within reach of any DIYer. If you can afford a few drivers and crossover parts, you can afford equipment to measure what you made with them.

So, sorry for the long rant, but I really think if more DIYers posted their measurements with their designs, the whole community would benefit.

-Byron Taylor
  Reply With Quote
Old 5th March 2011, 05:31 PM   #2
DougL is offline DougL  United States
diyAudio Member
 
Join Date: Feb 2004
Location: Wheaton IL.
Blog Entries: 30
I would challenge you to set a standard for DIY Audio. Publish a Wiki on measurement, stating what to measure, and how to interpret the results. Then run through it on each of your builds.
If this looks worthwhile, I will follow, and I assume others will as well.
I did a 5 year stint as a test engineer (Network) and realize how daunting truly meaningful measurement is.
OTOH, My brother rejects almost all measurements as irrelevant.
Note there are only a handful of manufactures that publish anything approaching meaningful data on their products.

Just my 2 cents.

Doug
__________________
Scienta sine ars nihil est - Science without Art is nothing. (Implies the converse as well)
Mater tua criceta fuit, et pater tuo redoluit bacarum sambucus
  Reply With Quote
Old 5th March 2011, 06:01 PM   #3
lgreen is offline lgreen  United States
diyAudio Member
 
lgreen's Avatar
 
Join Date: May 2003
Location: San Diego, USA
I don't know, I'd say the opposite. What I like about DIY is that everyone tries to measure their projects (if they are able) and posts data about their performance. Some people just take the plans or circuits of others and build them with no test equipment- it takes a deep commitment to diy to start acquiring test equipment.

What I don't like about professional reviews and high end products is that they supposedly have made the commitment but professional reviewers don't (for the most part) publish, take or care about measurements.

Oh, and every single pro review of a bookshelf or small monitor will eventually say "well its bass is not deep but you will be surprised how much bass this little speaker provides...".
__________________
Retired from DIY (2010) but still lurking now and again. My DIY audio projects- PartTimeProjects.com.
  Reply With Quote
Old 5th March 2011, 06:21 PM   #4
Banned
 
Join Date: Jan 2008
Blog Entries: 2
Hi Byron

I've complained about the issue in different ways, in different places on many occasions.

Professional reviewers have promoted this approach. In fact it seems to be impossible the be a professional reviewer who depends largely on measurements these days. This creates a big problem for designers, because it is pot luck in performance terms as to whether the item you have designed will be panned or lauded.

One way of improving your chances of success is to include components which are 'audio grade', or simply expensive, or are drawn from the list of those which are 'flavour of the month' this month.

It's an appalling situation which arose round about the time when the standard of audio electronics improved to the point where the majority of amplifiers were of acceptable quality. The reviewers realised that they were being made redundant by improvements in technology, and when someone (known, but who will remain nameless here) invented 'subjective' reviews, the rest were quick to jump on the bandwagon.

I agree with lgreen that the standards in DIY are better than in professional circles, but it's difficult for even the best of us to keep our heads above water, because the environment in which we swim has become so polluted with disinformation.

w
  Reply With Quote
Old 5th March 2011, 07:32 PM   #5
Pano is offline Pano  United States
diyAudio Moderator
 
Pano's Avatar
 
Join Date: Oct 2004
Location: Milliways
Blog Entries: 4
Jeez Byron - have you really looked around this forum? Measurements abound. In fact this is one of the most measurement heavy audio forums I know.
__________________
Take the Speaker Voltage Test!
  Reply With Quote
Old 5th March 2011, 07:38 PM   #6
diyAudio Member
 
raypalmer's Avatar
 
Join Date: Dec 2008
Location: Burlington
Quote:
Originally Posted by Pano View Post
Jeez Byron - have you really looked around this forum? Measurements abound. In fact this is one of the most measurement heavy audio forums I know.
Yeah it's a scientific wasteland out there, quite the lush oasis in here.
__________________
Wherever you go there you are.
  Reply With Quote
Old 5th March 2011, 07:55 PM   #7
tomchr is offline tomchr  United States
diyAudio Member
 
tomchr's Avatar
 
Join Date: Feb 2009
Location: Greater Seattle Area
You're forgetting one important fact. Many professional reviewers (be it web or in magazines) are paid by the manufacturers to review their equipment. It may not be in the form of monetary payments, but a friend of mine has received a complete high-end stereo in return for favorable reviews in the magazine he used to write for. I remember a few magazines that were highly dependent on the advertising revenue from a speaker manufacturer. Somehow, all speakers made by that manufacturer always got rave reviews in that magazine. Odd how that works. How do you spell "conflict of interest" again?

Besides, reviewers rarely are scientists. They don't know how to take the measurements or interpret them. Many times they seem to simply relay the marketing drivel that the manufacturers put out.

~Tom
  Reply With Quote
Old 5th March 2011, 08:00 PM   #8
diyAudio Member
 
raypalmer's Avatar
 
Join Date: Dec 2008
Location: Burlington
Quote:
Originally Posted by tomchr View Post
You're forgetting one important fact. Many professional reviewers (be it web or in magazines) are paid by the manufacturers to review their equipment. It may not be in the form of monetary payments, but a friend of mine has received a complete high-end stereo in return for favorable reviews in the magazine he used to write for. I remember a few magazines that were highly dependent on the advertising revenue from a speaker manufacturer. Somehow, all speakers made by that manufacturer always got rave reviews in that magazine. Odd how that works. How do you spell "conflict of interest" again?

Besides, reviewers rarely are scientists. They don't know how to take the measurements or interpret them. Many times they seem to simply relay the marketing drivel that the manufacturers put out.

~Tom
The instant we let professional reviewers into the argument things spiral down fast. People on eBay remarking "slightly sibilant highs mellowed by a marmaladey midriff" are just apeing the nonsense they read in magazines and on the internet. The entire hi-fi industry is so jam-stuffed full of rubbish it makes fox news political discourse look like a lecture on newtonian physics.
__________________
Wherever you go there you are.
  Reply With Quote
Old 5th March 2011, 08:03 PM   #9
diyAudio Member
 
Join Date: Feb 2009
Location: UK
Hi fi is a *perfect* scam. I have known people with real money problems who were, nevertheless, addicted to buying all the esoteric gear and 'tweaks'.

As soon as the objective measurements were deemed worthless, these people were on the hook forever; they could never shake off the doubt that their system wasn't 'musical'. The measurements that said they were, basically, perfect were apparently meaningless. People could churn out any old tat that looked 'minimal' or finished in exotic materials, or even return to 1920s technology, and ignore the measurements that showed they were basically rubbish. And an entire industry devoted to selling oxygen-free cables, anti-resonance beads and clarity creams was born, because none of these things could be measured.

The story of Peter Walker of Quad seems to be quite relevant to this subject.
  Reply With Quote
Old 5th March 2011, 09:02 PM   #10
diyAudio Member
 
Join Date: Mar 2007
Location: Canandaigua, NY USA
This is a wonderful place because there are all types here- those who believe in measurement and those who eschew it. I'm a measurement junkie so am in perfect agreement with your rant. OTOH...

System lacking pace and rhythm? Are your woofers too slow? Is your frequency domain performance out of sync with your time domain performance? Is that 4pS jitter wrecking your soundstage? No problem. Just send me $499.95 and I'll use my secret process to specially treat 250 cc of rare dihydrogen oxide that, placed next to your amplifier, will compensate for all those problems and yield perfect sound.

CH
__________________
I may be barking up the wrong tree, but at least I'm barking!
  Reply With Quote

Reply


Hide this!Advertise here!
Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are Off
Pingbacks are Off
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
DIY Analog-to-Digital Converter project.Audio measurements tool Frex Equipment & Tools 274 9th May 2014 05:32 PM
How To Use My O-Scope for Audio Measurements DreadPirate Parts 1 19th September 2008 11:10 PM
My rant about audio D_o_S Everything Else 17 21st January 2008 07:28 AM
Audio Measurements hotvalve Tubes / Valves 13 13th January 2008 09:44 PM


New To Site? Need Help?

All times are GMT. The time now is 06:27 AM.


vBulletin Optimisation provided by vB Optimise (Pro) - vBulletin Mods & Addons Copyright © 2014 DragonByte Technologies Ltd.
Copyright ©1999-2014 diyAudio

Content Relevant URLs by vBSEO 3.3.2