Just as a quick followup, the Olive paper did attempt to correlate 2nd and 3rd order harmonic distortion at 90dB SPL from 20Hz-2kHz (I'm assuming that was the fundamental freq) with listener preferences (again, this was blinded). They did NOT find a correlation between mean and max % of 2nd or 3rd HD and listener preferences. My two criticisms are:
1) this was only done on a sample of 13 loudspeakers, and they were all relatively inexpensive
2) the studied may not have been powered to detect small correlations in distortion in this small sample. (i.e. there may have been a correlation, but the study was too small to find one).
On the other hand, the low to high range of 2nd and 3rd HD was very wide, which makes #1 less relevant. Also, the study easily found huge correlations with frequency response measurement-derived metrics in this small sample of 13. This suggests that if there WERE a correlation between distortion and listener preference, it was relatively small. Not surprisingly, this finding is 100% consistent with Dr. Geddes' work, described above, which found no correlation between THD and total IMD that was computer-generated and blinded listener preferences.
Which leads me to ask, why on earth do we even BOTHER to measure and publish tweeter distortion on websites like Zaph's, when there is basically zero evidence that the measurements they make (THD, IMD) actually predict sound quality!!?? In fact, there is substantial evidence that these measurements do NOT predict sound quality AT ALL! In fact, I searched Zaph's site high and low, and I was unable to find ANY evidence that traditional distortion measurements predict sound quality. This is ironic, because it looks like so much work was put into performing the measurements. Anybody want to comment?
SG
1) this was only done on a sample of 13 loudspeakers, and they were all relatively inexpensive
2) the studied may not have been powered to detect small correlations in distortion in this small sample. (i.e. there may have been a correlation, but the study was too small to find one).
On the other hand, the low to high range of 2nd and 3rd HD was very wide, which makes #1 less relevant. Also, the study easily found huge correlations with frequency response measurement-derived metrics in this small sample of 13. This suggests that if there WERE a correlation between distortion and listener preference, it was relatively small. Not surprisingly, this finding is 100% consistent with Dr. Geddes' work, described above, which found no correlation between THD and total IMD that was computer-generated and blinded listener preferences.
Which leads me to ask, why on earth do we even BOTHER to measure and publish tweeter distortion on websites like Zaph's, when there is basically zero evidence that the measurements they make (THD, IMD) actually predict sound quality!!?? In fact, there is substantial evidence that these measurements do NOT predict sound quality AT ALL! In fact, I searched Zaph's site high and low, and I was unable to find ANY evidence that traditional distortion measurements predict sound quality. This is ironic, because it looks like so much work was put into performing the measurements. Anybody want to comment?
SG
SG,
I appreciate the background searching that you have done to look at all the works that have been done on distortion and audible perception.
This whole thing about distortion amazes me. Everyone looks at all the measured data such as those being done by SL, Zaph and every audio site under the moon. None have taken the time to perform an actual scientific study to relate this back to perception. If we can't hear it, for whatever reason, then why should we care about it from a design standpoint. They all claim that there is clearly an audible difference. Difference, yes, but does it correlate to a positive change or is it just a change?
People such as Geddes, Olive and Toole have all done scientific research and papers which were peer reviewed there are others which still don't believe the data. They have not disproven the data they just don't want to hear the facts.
I understand where Geddes comes from as he just seems fed up with people disputing findings but offering up no data of there own. I hate to call anyone out by Pan here on diyaudio seems to take the cake here. Is some of this reality hard to swallow, sure. This is all I have heard since getting into audio as a hobby. It seems to make sense that this raw driver data should correlate directly to what we hear. These are the devices that are producing the sound. At the end of the day, it's just not true. Does this mean that distortion isn't important? Of coures not! It just means that we are chasing the wrong things.
Audio is a sort of religion. People want to believe what they want to believe and changing peoples minds with data can seemingly be pointless.
I appreciate the background searching that you have done to look at all the works that have been done on distortion and audible perception.
This whole thing about distortion amazes me. Everyone looks at all the measured data such as those being done by SL, Zaph and every audio site under the moon. None have taken the time to perform an actual scientific study to relate this back to perception. If we can't hear it, for whatever reason, then why should we care about it from a design standpoint. They all claim that there is clearly an audible difference. Difference, yes, but does it correlate to a positive change or is it just a change?
People such as Geddes, Olive and Toole have all done scientific research and papers which were peer reviewed there are others which still don't believe the data. They have not disproven the data they just don't want to hear the facts.
I understand where Geddes comes from as he just seems fed up with people disputing findings but offering up no data of there own. I hate to call anyone out by Pan here on diyaudio seems to take the cake here. Is some of this reality hard to swallow, sure. This is all I have heard since getting into audio as a hobby. It seems to make sense that this raw driver data should correlate directly to what we hear. These are the devices that are producing the sound. At the end of the day, it's just not true. Does this mean that distortion isn't important? Of coures not! It just means that we are chasing the wrong things.
Audio is a sort of religion. People want to believe what they want to believe and changing peoples minds with data can seemingly be pointless.
I'd like to see more hard data as well. When the major manufacturers of some of the best drivers such as Seas and Scan-Speak among others devote as much as they do to producing low non-linear distortion drivers that adds to their cost, I have to think that it's not all imagination. I just don't see them doing this for advertising alone.
Dave
Dave
Dave,
A lot of the markets purchasing decisions are being made almost specifically on driver distortion tests. Why would it not make sense from a manufacturing point of view to focus on developing the best driver for the tests that people are looking at?
Since numbers such as THD and IMD are the industry norm it would be a horrible marketing move to change direction without there being a paradigm shift to do so.
I would also love to see more studies done as the more that are completed will just further the knowledge and exposure.
A lot of the markets purchasing decisions are being made almost specifically on driver distortion tests. Why would it not make sense from a manufacturing point of view to focus on developing the best driver for the tests that people are looking at?
Since numbers such as THD and IMD are the industry norm it would be a horrible marketing move to change direction without there being a paradigm shift to do so.
I would also love to see more studies done as the more that are completed will just further the knowledge and exposure.
goskers said:
Where is the data that supports that? My current favorite midrange is the SS 12m/4630, a neodymium motor. No copper in the gap, considered by measurements to be somewhat average in the group for distortion. Yet this is also their most expensive pure midrange unit and is more expensive than their somewhat larger, newer midwoofer line of 15W drivers and counter to their older 13m/8636 and 13m/8640 (both of which I still have) that had significant copper caps.
SS is said to have made the decision to leave out the copper because their listening tests indicated that it was not needed. It was said to based on auditioning.
This is in total contradiction to the assumptions being made about the marketing aspect. I'm hesitant to just accept the currently evolving wisdom that it's all imagination and marketing.
Dave
Dave,
I was not pointing directly at you. Specifically the guys that I listed before; SL, John K., Mark K, have done tremendous work in the area of driver testing. There is a whole flock of DIY guys that follow these results when selecting drivers for a project. I am not here to come down on anyone as I have based past decisions directly on this data as well.
I was not pointing directly at you. Specifically the guys that I listed before; SL, John K., Mark K, have done tremendous work in the area of driver testing. There is a whole flock of DIY guys that follow these results when selecting drivers for a project. I am not here to come down on anyone as I have based past decisions directly on this data as well.
smellygas said:
This is the singular question. It is logically the responsibility of the people who CLAIM THD is important to PROVE it, not the other way around. In more than five years since our paper no evidence what-so-ever has come forth to support THD.
THD is measured "because we can" - there is no other reason. And its used by marketing "because it works" and there is also no other reason.
dlr said:
A very fair stance to take. I was very skeptical about certain claims until I made a trip to Detroit. What I heard backed up the data and statements that were the basis of the design. It only makes me wish that I would have made the trip earlier as I would have saved myself a good deal of money after all these years.
I don't wish to throw the baby out with the bath water, but the distortion data is hard to swallow, or rather its hard to tie to my own experiences. Most of us believe that the subtle differences we hear when A/B'ing (blind or not) are real.
I admittedly probably could not pass a blind test for some of it. However, I still have issues with inferences drawn from blind tests. I am sure this has been infinitely debated and I don't wish to bring this up again.
I just wonder if work has been done on differences in audibility based on short term ABX and longer term listening? It may just be too impractical or hard to test, I don't know. I am also not up on the literature (this is only a hobby for me).
I know for me personally, I discount a lot of what I perceive to hear at others' places. I can't disaggregate the sound of the whole into the part associated with the room, the speakers, the equipment, the cables, etc, etc that some profess to be able to do.
I probably couldn't pass a blind test on easy things with a system I was unfamiliar with and music I didn't choose myself. Could I be consistent on my own system with my own selection of music. Maybe. That is my best answer.
I have dones some single blind tests for stuff I was testing, component stuff and often could consistently determine in small sample size. I think we as hobbyists hope that the small subtle "improvements" are additive and that enough of them will add up to a larger improvement. That is a leap of faith, but one we accept readily.
Basically I am agnostic when it comes to a lot of this distortion audibility stuff as I can't seem to 100% jive this data with my own experience and intuition. This very well may be my own ignorance. ce la vie.
I admittedly probably could not pass a blind test for some of it. However, I still have issues with inferences drawn from blind tests. I am sure this has been infinitely debated and I don't wish to bring this up again.
I just wonder if work has been done on differences in audibility based on short term ABX and longer term listening? It may just be too impractical or hard to test, I don't know. I am also not up on the literature (this is only a hobby for me).
I know for me personally, I discount a lot of what I perceive to hear at others' places. I can't disaggregate the sound of the whole into the part associated with the room, the speakers, the equipment, the cables, etc, etc that some profess to be able to do.
I probably couldn't pass a blind test on easy things with a system I was unfamiliar with and music I didn't choose myself. Could I be consistent on my own system with my own selection of music. Maybe. That is my best answer.
I have dones some single blind tests for stuff I was testing, component stuff and often could consistently determine in small sample size. I think we as hobbyists hope that the small subtle "improvements" are additive and that enough of them will add up to a larger improvement. That is a leap of faith, but one we accept readily.
Basically I am agnostic when it comes to a lot of this distortion audibility stuff as I can't seem to 100% jive this data with my own experience and intuition. This very well may be my own ignorance. ce la vie.
gedlee said:
None of the data at two of the sites mentioned are presented in terms of THD. Multi-tone tests are the norm and both John and Mark point out that their data has to be taken with reservations and that their tests are not the last word. Neither claim anything definitive, but trends do show up with a fair amount of correlation between the two insofar as testing is concerned. And I don't think that either one would shrink from stating their beliefs from their experience. Neither one is selling a product.
It may be that some speaker system manufacturers are making broad claims using THD, that would not surprise me. I'm in agreement with you on that aspect, there should be some proof if THD or any method is used as a metric. I see no difference between that and using nothing more than the on-axis response for ad copy.
Dave
JoshK said:
Josh
It is very hard for anyone to accept that they can't hear what they think that they can. And yet, I have seen this so many times that I know that its true. I think that the most important thing that I can personally bring to audiophilia is the understanding of the fact that the vast majority of the claims being made in audio are probably not valid. I have not seen the case where a real proven difference could not be shown quantitatively and hence I demand a quantitative basis for all claims. This virtually always stops the discussion at "well it sounds good to me." As long as that claim is used as a justification no progress will ever be made.
And this is not a pointless distinction as vast sums of money are spent where it makes no difference while places where the differences are profound - like the loudspeakers and rooms - are left as totally subjective. If the consumer learns one thing from this discussion it needs to be that loudspeakers matter, that they are not "whatever you like" and that it is absolutely possible to quantify a good loudspaeker from a bad one. Further, as a direct result, the budget for an audio system must be heavily weighted towards the loudspeaker.
While it may be true that "small subtle "improvements" are additive", I highly doubt that "enough of them will add up to a larger improvement" on par with using great loudspeakers in well setup rooms. Everything will pail compared to these later two, to the point that obsession with the "subtle" stuff is a waste of time and money. The loudspeakers and room will be 95% or the experience, the rest maybe 5%. It is only logical that the expense should be doled out in this same ratio - I do. I have found this to be extremely rare and as such I have found virtually all other systems to be lacking. Certainly I have never seen a system at the cost of mine come anywhere close.
In the end this has to be the discussion because we all have limited budgets. If you don't have a limited budget then please CALL ME!
Dr. Geddes,
I don't disagree with the majority of your above statement. I guess it boils down to most of us audiophiles are stuck with the room we got, which sucks objectively speaking. In that context, with those constraints, there are only so many things that can be done so idle minds turn to things that can be done even if they don't matter.
Humbly,
Josh
I don't disagree with the majority of your above statement. I guess it boils down to most of us audiophiles are stuck with the room we got, which sucks objectively speaking. In that context, with those constraints, there are only so many things that can be done so idle minds turn to things that can be done even if they don't matter.
Humbly,
Josh
P.S. I think one of the smartest things you have done for the diy community is offer up the smaller form factor kits. Your Summas kicked *** in your room but lots of folks here don't have the luxury of a large dedicated room (that's a tough reality). Many have small listening room and listen near field.
P.S.S. my project with your 12" WGs is coming along nicely.
P.S.S. my project with your 12" WGs is coming along nicely.
- Status
- This old topic is closed. If you want to reopen this topic, contact a moderator using the "Report Post" button.