FFTs as a measurement tool in Audio

Status
Not open for further replies.
Ex-Moderator
Joined 2002
Gentlemen, you seem to be missing the point. The current social paradigm of knowledge puts the burden of proof on those that submit claims that expand that knowledge base. If you have an extraordinary claim, well and good, that's what keeps our understanding of the universe moving forward, but you need to have results that are defined and repeatable.

Let's take your premise that audio measurements do not reflect your perception of sound quality. That's such a vague argument that it's no wonder that SY can't give you a test to prove it one way or another. Do you test for phase, phase change, frequency, air pressure, humidity, cochlear sensitivity, neuron transmission speed, cognitive ability, placebo effect, psycho-social influences, or even, being a little silly, local variations in the value of W, (the wavefunction of the universe)? Or maybe a combination of the above. You need a hypothesis that can be proven or disproven, say, "Rate of phase change in differential stereo signals inhibits soundfield localisation". Then you can work out a test.
 
Pinkmouse, how about I like the sound of my modifications & like the smell of it also :) People buy it too! I don't know what their motivations are?

But here's something to ponder - SY tested it & declared it no better than a cheap Chinese USB-SPDIF converter (never sure if he listened to it?). Pano listened to & tested the same unit (sent to him) Vs a cheap Chinese unit & found both measured differences & sonic differences (that he has stated he's confident he could identify in a blind test).

So is any of this testing of any value?
 
Yes, exactly & so SY's tests & conclusions are simply a statement of his conclusion based on his replay equipment, test equipment, technique & skill. Of absolutely no more value than anecdotal evidence from each one of the hundreds of users who report their delight with the sound.
 
Ex-Moderator
Joined 2002
Ah, but no. SY can give a list of exactly what equipment he used and the process he followed that can be repeated by any other person, those results if not repeatable will fall by the wayside and be forgotten. I have no doubt about the delight of your users, but you have no evidence of exactly what caused that delight, that can be experimentally tested. See above, it could be for instance, that those that hear a difference do so because they listened to the new device first in a period of higher than normal relative humidity, or they had just received good news about a promotion at work. I don't know, and I can't repeat those results to try and disprove them.
 
Ah, but no. SY can give a list of exactly what equipment he used and the process he followed that can be repeated by any other person, those results if not repeatable will fall by the wayside and be forgotten.
He did & came under criticism which he rejected as valid so I'm afariad his tests are discounted beacuse there is a fair amount of doubt over their validity. The only other testing was done by Pano & his results, for his playback system, test equipment & devices tested are contrary to SY's From thi swe can tell absolutely nothing.
I have no doubt about the delight of your users, but you have no evidence of exactly what caused that delight, that can be experimentally tested. See above, it could be for instance, that those that hear a difference do so because they listened to the new device first in a period of higher than normal relative humidity, or they had just received good news about a promotion at work. I don't know, and I can't repeat those results to try and disprove them.
So on the one hand we have dsiputed results from one person, SY & another set of measurements from Pano that are different - both of which have no general applicability. So how can any of these tests be of any value as there are too many variables at play.

As I said thay have no more validity than hundreds of anecdotal evidence & yet they seem to be more valued by some here.
 
Ex-Moderator
Joined 2002
He did & came under criticism which he rejected as valid so I'm afariad his tests are discounted beacuse there is a fair amount of doubt over their validity.

Did anyone actually repeat his tests under the same conditions with the same DUTs and prove his method or results wrong? If not, then the results stand, and the critique has no validity. You can't just dismiss experimental claims, you have to disprove them by your own experiment, and that then also has to stand up to repetition by others.

Pano's results are the same. He measured and found a difference between your converter and the Chinese one he had. That proves, until a better experiment is carried out, or his is repeated and found wanting, that he can hear/measure a difference between those two items. You can't conflate the two experiments.

Edit: Just to clarify, what I mean is that until proven otherwise, there are some Chinese converters that measure as well as yours. That's the only valid conclusion we can come to given the evidence.
 
Last edited:
I'm simply saying, & you seem to be missing the point, that these tests are so specific as to be of no more worth than the anecdotal evidence. You seem to say that they are os worth becasue they can be repaeted by anybody! How? By going to SY's or Pano's house & re-doing the tests?
 
Ex-Moderator
Joined 2002
I'm simply saying, & you seem to be missing the point, that these tests are so specific as to be of no more worth than the anecdotal evidence.

Yes, these tests are specific, but by doing specific tests, you nibble away at the larger problem piece by piece until you have an answer. So for instance, if I were the experimenter, I'd next want to compare the two Chinese converters against each other, and see if any differences between them could lead to clues as to what this nebulous thing we call " Audiophile Sound" is. One might have slightly greater high order distortion, or different jitter rates, and if you can then design an experiment to test if that factor produces an audible effect, then the world of audio has taken a step forward for the benefit of us all.

You seem to say that they are os worth becasue they can be repaeted by anybody! How? By going to SY's or Pano's house & re-doing the tests?

No, but if they publish controls, methods and apparatus, then anyone can duplicate them for themselves.That's how science works. ;)
 
Yes, these tests are specific, but by doing specific tests, you nibble away at the larger problem piece by piece until you have an answer.
So they are so specific & of no mopre value than anecdotal evidence as I have been saying. In fact I would think that on balance the anecdotal evidence is of more value for those wishing to judge the worth of a particular product/tweak. Yes, there is the unscientific nature inherent in these reports & the undeniable bias. Yes, I have found, to my dissappointement products, that didn't live up to the claims made of them by their users!
So for instance, if I were the experimenter, I'd next want to compare the two Chinese converters against each other, and see if any differences between them could lead to clues as to what this nebulous thing we call " Audiophile Sound" is.

It would only apply to these two specific units tested in this specific system under these specific conditions & has no more widespread applicability than the above tests
One might have slightly greater high order distortion, or different jitter rates, and if you can then design an experiment to test if that factor produces an audible effect, then the world of audio has taken a step forward for the benefit of us all.
Again, for a specifc set of conditions which can't be generalised - I hardly think this would advance the world of audio one iota.

No, but if they publish controls, methods and apparatus, then anyone can duplicate them for themselves.That's how science works. ;)
Including all the tweaks they have done to their system & the specific conditions that the have used, the mains supply noise, the RF intrusion from nearby transmitters, etc. I contend that the variables are too numerous to say that there is anything scientific about this.

I contend that your being optimistic & somewhat unrealistic in your expectations :)
 
Last edited:
Ex-Moderator
Joined 2002
I contend that the variables are too numerous to say that there is anything scientific about this.

Fair enough, but it's up to you to repeat the experiment and prove that these variables produce significant bias in the results. Then if proven, we have another step towards audio nirvana.

I contend that your being optimistic & somewhat unrealistic in your expectations :)

Well, the scientific method has worked quite well so far in human history. ;)
 
Jkeny, please read up on how science and experimentation is conducted. There is TONS of literature and commentary!

It has evolved as such over several hundred years and serves all avenues of science very well.

Your remarks are wincingly emabarrasing coming from someine who is otherwise obviously an educated person.
 
Jkeny, please read up on how science and experimentation is conducted. There is TONS of literature and commentary!

It has evolved as such over several hundred years and serves all avenues of science very well.

Your remarks are wincingly emabarrasing coming from someine who is otherwise obviously an educated person.

Try to argue the points rather than the person, please - it is much more scientific don't you think :)

BTW, I have been educated & trained in the scientific method at tertiary level so no need to be condescending
 
Last edited:
Fair enough, but it's up to you to repeat the experiment and prove that these variables produce significant bias in the results. Then if proven, we have another step towards audio nirvana.
No. it''s not up to me to do any such thing. I'm contending that the above experiments have so many variables that they are worthless. The scientific method insists that the variables are under control & manageable. You show me how they are?



Well, the scientific method has worked quite well so far in human history. ;)
Yes, if this followed the scientific method but see my comment above & then tell me it is following rigorous, repeatable scientific experiment!
 
You know it strikes me that you guys are trying very hard to defend the indefensible, why? Trying to quote the scientific method when the two experiemnst I cited are so far from repeatable is a joke & makes me wonder to what extent you actually understand what you are talking about? Was the reference to the science supposed to scare me? Really guys it's time this pseudo-science stopped & you honestly & genuinely adrresed real, repeatable experiements instead of this pretense at science!
 
Administrator
Joined 2004
Paid Member
Only if you can prove they both tested the exact same Chinese converter.

And they are not the same at all. Not even the same chip. Burr-Brown vs Cmedia. Between the Hiface and an Azuntech PCI card the measured differences were slight. I could not hear them.

However, Mr. Keny is simply trying to continue a thread that has already been closed. That is not allowed here. This thread my no need to be closed as well.
 
Really, pano, just because I make the same points as I made before, doesn't mean I'm continuing a thread that was already closed.

Yes your results both in measurements & listening showed little to no differences between the devices you mentioned. Again, the results stand in isolation as specific to your system.

All I'm trying to tease out is that there is no more validity to these results then there is to anecdotal evidence! Sorry if this message requires that the thread must be closed but that is up to the mods
 
Last edited:
Sorry to be late to the party- I was busy being interviewed by NPR today about my scientific research and the significant support by several government agencies. Apparently they haven't gotten the news yet that I don't understand the scientific method.:D

If you take the same set of numbers and perform an FFT on them, and you use the same apodization, the results will be the same, i.e., repeatable.

This IS about FFTs, right?
 
Status
Not open for further replies.