JonBocani eliminating potential errors is the way to go I agree. I think you are building an interesting body of work which as far as I am aware has not been done before. Having said that providing the equipment you used in the test remained consistent I doubt it would have changed the result. Are you planning to test off axis in the next run? I think that would be most illuminating.
My Listening Test Hypothesis
This has been discussed for years, but I haven't seen people touched more than the surface. So here is if you are interested:
Consult (or refer to) the Statistics. Understand the distribution/spread/variance/range of a sample space or dataset. Look at the attached data (from klippel distortion test) as an example. What can be learned/concluded from this graph?
(1) People have different hearing/listening skill.
There are "outliers", those who perform so badly and those who perform so well (we can call them the 10th percentile and the 90th percentile). Observe the number of the "top outliers". Actually, this test is limited to a maximum of |-54dB| hearing skill, which is too low to show the actual skill of the top outliers.
In term of IQ score, I'm above the 99th percentile. I believe that my hearing skill is also in this percentile (this is outside of the Klippel test data range or scope). There might be a few of you who are interested in this: listening skill has more to do with brain than with ear.
Question to BurntCoil (rhetorical): What is the 99th percentile of 12 people?
The above is a simple fact, not so important imho. The more important question is: What is the relationship between a human ABX listening score with how far the exact same human can be affected by the distortion/sound? My hypothesis has been: Even if a human cannot pass an ABX test, it doesn't mean that they are not affected. This is because human ABX skill is TOO low compared to what their brain can actually do.
For me, the above hypothesis is easy to prove by logic (not by experiment), taking my own experience as an analogy: Let's say I can hear and is thus affected by -54dB distortion. Assume last year I can't past -12dB distortion test (the main point here is that this is a learned process which requires time). Can I assume that last year I'm not affected by -54dB??
Here is my opinion (or better call it my hypothesis): Researchers were wrong thinking that human is not affected by THD below 0.1% just because they couldn't pass the test. The real distortion that can affect human is far below that number!
This has been discussed for years, but I haven't seen people touched more than the surface. So here is if you are interested:
Consult (or refer to) the Statistics. Understand the distribution/spread/variance/range of a sample space or dataset. Look at the attached data (from klippel distortion test) as an example. What can be learned/concluded from this graph?
(1) People have different hearing/listening skill.
There are "outliers", those who perform so badly and those who perform so well (we can call them the 10th percentile and the 90th percentile). Observe the number of the "top outliers". Actually, this test is limited to a maximum of |-54dB| hearing skill, which is too low to show the actual skill of the top outliers.
In term of IQ score, I'm above the 99th percentile. I believe that my hearing skill is also in this percentile (this is outside of the Klippel test data range or scope). There might be a few of you who are interested in this: listening skill has more to do with brain than with ear.
Question to BurntCoil (rhetorical): What is the 99th percentile of 12 people?
The above is a simple fact, not so important imho. The more important question is: What is the relationship between a human ABX listening score with how far the exact same human can be affected by the distortion/sound? My hypothesis has been: Even if a human cannot pass an ABX test, it doesn't mean that they are not affected. This is because human ABX skill is TOO low compared to what their brain can actually do.
For me, the above hypothesis is easy to prove by logic (not by experiment), taking my own experience as an analogy: Let's say I can hear and is thus affected by -54dB distortion. Assume last year I can't past -12dB distortion test (the main point here is that this is a learned process which requires time). Can I assume that last year I'm not affected by -54dB??
Here is my opinion (or better call it my hypothesis): Researchers were wrong thinking that human is not affected by THD below 0.1% just because they couldn't pass the test. The real distortion that can affect human is far below that number!
Attachments
I agree it would be helpful to increase the sample size but then we are both spending someone else's time and money which is always easy to do. As the testing has thrown up a result that was counter intuitive, no one predicted the result achieved, I think further experimentation is the only way to go. This happens in all fields where experimentation delivers a result outside of the accepted dogma. The principles and assumptions that delivered the current state of the art have to be questioned if they did not predict the result. Concerns over probabilities and degree of acuity apply when the results are fuzzy. In this case from the report it would appear they were not.
If your argument is we need more data to falsify the result I agree but the data to date suggests our current assumptions should be revised. Interesting times.
If your argument is we need more data to falsify the result I agree but the data to date suggests our current assumptions should be revised. Interesting times.
Last edited:
I cannot stress this enough: Frequency Response is very important but must/can be obtained within driver's mechanical /electrical limits.
I got comments from people who now thinks (based on my conclusions!) that you can take any cheap monitor from Best Buy or Amazon and transform it into some concert-level HiFi stuff...
No, guys, you cannot.
You cannot get 110db SPL @ 20hz from a cheapo monitor. In fact you cannot get 110db at any frequency with a cheapo monitor. And you probably cannot get 20hz either, at any listenable level, with a cheapo monitor.
Also, mechanical limits also includes directivity/power response and point source/diffused.
a 4'' driver would never mimic a line array of a dozen 8'' drivers... No matter how good is your EQ.
That would be easy to get done, but now it will raise a question:
Will i EQ off-axis or keep the on-axis EQ ?
JonBocani its your time and your dollar but ideally both in two runs.
However, as that may be understandably beyond your reserves and resources and as you have completed the test for on-axis EQ we may learn more by testing some drivers with off-axis EQ?
Thank you for the work you have done and sharing it for review-its an excellent effort.
However, as that may be understandably beyond your reserves and resources and as you have completed the test for on-axis EQ we may learn more by testing some drivers with off-axis EQ?
Thank you for the work you have done and sharing it for review-its an excellent effort.
No, that's none of my points. Look again at the Klippel distortion test. -54dB is the "limit" there. What do you think is the distortion of the drivers in Jon's test? You can safely use the Klippel test result (tho it is mistakenly conducted imo) to predict the result of Jon's test.
Sorry Jay but you will have to unpack that further. The test demonstrated that irrespective of the driver distortion when EQ'd the drivers were difficult to impossible to differentiate.
No doubt I am being stupid but I don't understand your point.
No doubt I am being stupid but I don't understand your point.
Difference in SPL level is the main determinant in DUT differentiation in an ABX. If FR is "flattened" then we can assume that there is no difference in SPL level (actually it is not perfect, there is always variance)...
So in Jon's test we must rely on other measures to differentiate sound, other than the SPL level. It is either the micro SPL level difference, or other measures such as the distortion...
The Klippel test shows -54dB as the limit (not a threshold). -54dB is equivalent to around 0.2% distortion... Please note that this is distortion at amplifier level, not at speaker level...
Good drivers have distortion below 0.4%. This is -48dB. If we assume that this has direct relationship with amplifier distortion, then we can predict that there are people who can differentiate it, but the number is so low (I don't want to bother using the klippel test data to calculate how many fraction of human it will be).
YES, it is difficult to differentiate, but not impossible. Do you want me to calculate statistically (based on the Klippel test result) how many respondents Jon needs to gather to get the positive result? (I hope not 😀)
My ears are good. Trust me, everything is in the measurement (People just don't agree on the threshold numbers). Like I said before, for me, measurements of these drivers tell much more than the test result.
Last edited:
Will the same on-axis EQ... BUT the whole set-up/chair moved closer to a wall (left or right) would be interesting as well ?
Thank you Jay, I will give that some thought in the morning as its getting late here.
( Oh go on then, if you know how to do the calculation it would be useful to know the minimum test population....... : )
( Oh go on then, if you know how to do the calculation it would be useful to know the minimum test population....... : )
Everything that can be counted does not necessarily count; everything that counts cannot necessarily be counted". Albert Einstein.
actually the plateamp's DSP limits to 1db increments as of now.
Will have 0.1db increment with the nanoDigi.
that's far from ideal, then again: people are mostly not able to identify, whether because of 0.3-0.5db difference or slightly difference FR or any other reason...
also, i have my doubts on the audible distortion levels.
Is there any ABX blind test regarding distortion that was made, and for each groups of frequencies or octaves ?
Educated-guess here: lower-octave distortion might be difficult to spot, even at high %
Is there any ABX blind test regarding distortion that was made, and for each groups of frequencies or octaves ?
Educated-guess here: lower-octave distortion might be difficult to spot, even at high %
I have read something like that. The author also related this to the Fletcher-Munson curve. But I'm not sure that's correct or not.
In my experience, single frequency (pure tone) is hard to detect. Easy is with music signal that covers a wide range of frequency. I think it is not the frequency that matters, but the instruments that can produce musical rhythm and pitch (at any frequency range)...
One of my processes is, I will listen the rhythm/pitch again and again until the "correct" rhythm/pitch is well registered into my memory. Then I compare one of the DUTs (whose rhythm/pitch is modified by the distortion) to that benchmark (the correct rhythm/pitch) in my memory.
Jon
Earl geddes has several papers on his website dealing with distortion. Some are double blind tests.
GedLee LLC
Earl geddes has several papers on his website dealing with distortion. Some are double blind tests.
GedLee LLC
Morning Jay, my thoughts for what they are worth
'Difference in SPL level is the main determinant in DUT differentiation in an ABX. If FR is "flattened" then we can assume that there is no difference in SPL level (actually it is not perfect, there is always variance)...
So in Jon's test we must rely on other measures to differentiate sound, other than the SPL level. It is either the micro SPL level difference, or other measures such as the distortion...'
This is an argument about measurement. Jon's test was originally about discovering the quality of available drive units but discovered that to experienced listeners the differences between units of wildly different designs is not detectable. In essence he has discovered that for the majority of listeners where drive units are EQ'd and SPL matched there is no differentiation. It may well be possible to measure differences but these are not relevant as they cannot be heard.
'Good drivers have distortion below 0.4%. This is -48dB. If we assume that this has direct relationship with amplifier distortion, then we can predict that there are people who can differentiate it, but the number is so low (I don't want to bother using the klippel test data to calculate how many fraction of human it will be).'
You note yourself that the number of people who may be able to differentiate low levels of distortion are outliers. Jon's test appeared to demonstrate that even when distortion levels were relatively high it was still difficult to impossible to differentiate between drivers for the majority of listeners.
This all applies to the test as conducted. As Jon notes there are a large number of factors that would differentiate e.g. Max SPL, directivity, total bandwidth. So the test results are not saying all drivers are equal and EQ is not a panacea enabling cheap drivers to equal expensive ones. What it does reveal is that from a listening perspective the difference between drivers on-axis vanish, and further than that it strongly suggests the differentiators lay in these other factors. SPL, bandwidth etc
'Trust me, everything is in the measurement (People just don't agree on the threshold numbers). Like I said before, for me, measurements of these drivers tell much more than the test result.'
You want to see the measurements in order to differentiate the performance between the drive units. But that was one of the discoveries in this interesting set of results. The majority of listeners could not hear the differences that were there so these measurements are irrelevant.
We will learn more by addressing the other factors as on-axis testing has been revealed to be less definitive than previously assumed.
Just in case you get the impression I am a bizarre subjectivist let me reassure you I am not. Physics rules, always has, always will. But that means data trumps theory and the data from the listening test suggests me may have been focussing on the wrong things.
Happy weekend all
Burnt
'Difference in SPL level is the main determinant in DUT differentiation in an ABX. If FR is "flattened" then we can assume that there is no difference in SPL level (actually it is not perfect, there is always variance)...
So in Jon's test we must rely on other measures to differentiate sound, other than the SPL level. It is either the micro SPL level difference, or other measures such as the distortion...'
This is an argument about measurement. Jon's test was originally about discovering the quality of available drive units but discovered that to experienced listeners the differences between units of wildly different designs is not detectable. In essence he has discovered that for the majority of listeners where drive units are EQ'd and SPL matched there is no differentiation. It may well be possible to measure differences but these are not relevant as they cannot be heard.
'Good drivers have distortion below 0.4%. This is -48dB. If we assume that this has direct relationship with amplifier distortion, then we can predict that there are people who can differentiate it, but the number is so low (I don't want to bother using the klippel test data to calculate how many fraction of human it will be).'
You note yourself that the number of people who may be able to differentiate low levels of distortion are outliers. Jon's test appeared to demonstrate that even when distortion levels were relatively high it was still difficult to impossible to differentiate between drivers for the majority of listeners.
This all applies to the test as conducted. As Jon notes there are a large number of factors that would differentiate e.g. Max SPL, directivity, total bandwidth. So the test results are not saying all drivers are equal and EQ is not a panacea enabling cheap drivers to equal expensive ones. What it does reveal is that from a listening perspective the difference between drivers on-axis vanish, and further than that it strongly suggests the differentiators lay in these other factors. SPL, bandwidth etc
'Trust me, everything is in the measurement (People just don't agree on the threshold numbers). Like I said before, for me, measurements of these drivers tell much more than the test result.'
You want to see the measurements in order to differentiate the performance between the drive units. But that was one of the discoveries in this interesting set of results. The majority of listeners could not hear the differences that were there so these measurements are irrelevant.
We will learn more by addressing the other factors as on-axis testing has been revealed to be less definitive than previously assumed.
Just in case you get the impression I am a bizarre subjectivist let me reassure you I am not. Physics rules, always has, always will. But that means data trumps theory and the data from the listening test suggests me may have been focussing on the wrong things.
Happy weekend all
Burnt
This is an excellent site, thank you JayReed.
Jay with regard to measurement and audibility this section is fascinating
The Perception of Distortion