Well why not tell Sofaspud that - he seems to be the one needing your advice
Of course, but why the prevarication & qualifications that Aardvark raises - is it so difficult to admit this? It would seem to be a logical conclusion to any sane minded person? Is this where the root of the problem arises - the inability to admit that yes there is some value to the subjective view as a measure of a device (not just for that person)?
Last edited:
prescreening means identifying existing bias beforehand and then including that knowledge in your interpretation.
In simple terms around the hypothetical "ugly device" bias, show the participants ugly and fabulous devices and ask which they think will perform better. Then do the sighted listening tests.
By separating the two tests it is possible (all other factors accounted for) to interpret the result with the ugly bias discounted.
In simple terms around the hypothetical "ugly device" bias, show the participants ugly and fabulous devices and ask which they think will perform better. Then do the sighted listening tests.
By separating the two tests it is possible (all other factors accounted for) to interpret the result with the ugly bias discounted.
Its not prevarication or qualification - its standard practise for any test regimen whether the subjects are sane or not.
Yes, Aardvark, I was assuming that the participants had already made their opinion clear in this regard i.e their visual impression would lean them towards the opposite of their sonic impression. Now my question was when we factor this in does it make the sonic conclusion even more decisive?
That's one of the easiest questions so far on this thread. Simple - investment. The emotional kind, not the financial in case there are any doubters still remaining who haven't got the message yet.
Dunno if it's relevant, but there is an other dark side in the subjective appreciation : when reading the speaker forum, something is evident, most of the DIYers are absolutely pleased with their production. Is such a rate of bright success plausible when we know the difficulties of the task ?
Maybe it's the father effect, maybe it's because we have to be great optimists to perseverate in the hobby.
And it's not all, some people are professionnals, other are in a club or visit friends systems, go in fests, demos or concerts. They have at least a reference.
Some others are lonely, no audiophile life around, just the earphones. That's my case.
In these conditions, how to trust my own subjective opinion ? It's for sure biasland. I just can "measure" and pray that neither the mike or my method are going wrong.
Maybe it's the father effect, maybe it's because we have to be great optimists to perseverate in the hobby.
And it's not all, some people are professionnals, other are in a club or visit friends systems, go in fests, demos or concerts. They have at least a reference.
Some others are lonely, no audiophile life around, just the earphones. That's my case.
In these conditions, how to trust my own subjective opinion ? It's for sure biasland. I just can "measure" and pray that neither the mike or my method are going wrong.
I am always biased when hear reproduced music. Does not matter, which equipment is used, mine or not, I always search for reasons why it differs from real sounds. And it helps me a lot. If some prescreened people who are biased toward searching for errors are involved in tests of my equipment, I would be more than happy.
Or, knowing that the test is about sound quality and being cynical in nature, they deliberately choose the ugly one because they expect a twist...
I have a hypothesis that listeners listen in different ways. I'll illustrate with a visual analogy - an audio system is rather like a window. Some keep paying attention to what's on the other side of the window. Others notice there's dirt on the window. Telling the difference between the two is not trivial and does take a degree of training.
I mean BAF, Burning Amp Festival, where members of this exactly forum were presenting own DIY equipment. Do you mean members of our forum who come to festival were cynical in nature? At least, speakers did not look ugly. They were looking humble, like I said.
That's as good a reason as I've ever heard for excluding objectivists from listening tests.
Yes, it does take a degree of training, to tell what's wrong with the window: dirty glass, curved glass, broken glass, etc... But what is good, when people (does not matter how well they are trained) keep reacting like there is no glass at all. It is the biggest reward I am getting sometimes: subconscious mind reacts before conscious recognition tells that the sound is reproduced.
No. How can it?
Get it clear - you can only test for one thing at a time. If you are testing for subjective appeal of the sound, then you have to set the test up to do JUST that.
If there is any likelihood that the subject is making judgements on criteria OTHER than the sound, you have to control for those criteria and then analyse the results wit hthe knowledge of the controls you applied. The ugly pre-screen is a method for applying that control.
I get the feeling you are trolling here - this isn't rocket science - its barely science. Its 12th grade statistics.
If "the subject" jumps out of running water it is too late to apply any bias, no matter did it present or not. The test had been passed.
If "the subject" says, "I would like the house on the creek, frogs on your creek sing so nice!", I would not ask him to tell how well my system reproduces sounds. Or, when people look to the sky searching for helicopter, it is odd to ask them, are they qualified or not to test my PA system.
If "the subject" says, "I would like the house on the creek, frogs on your creek sing so nice!", I would not ask him to tell how well my system reproduces sounds. Or, when people look to the sky searching for helicopter, it is odd to ask them, are they qualified or not to test my PA system.
I mean BAF, Burning Amp Festival, where members of this exactly forum were presenting own DIY equipment. Do you mean members of our forum who come to festival were cynical in nature? At least, speakers did not look ugly. They were looking humble, like I said.
If that was the set-up., its soooo not a test capable of any statistcal significance its best described as "putting a finger in the air".
Thats not to say the opinions are NECESSARILY incorrect, just that they are STATISTICALLY unsupportable.
Aard, you're still stuck in the pretense or delusion of the possibility of applying any statistical significant test in audio without a huge expense/huge numbers of participants & all the pre-screening & variable controls that are needed. You are then criticising the test as if it was possible to do a statistically valid one!
So are you rejecting all possible listening tests & sticking strictly to instrument measurements? If you could state what you position is it might help to short circuit this circuitous dialogue.
So are you rejecting all possible listening tests & sticking strictly to instrument measurements? If you could state what you position is it might help to short circuit this circuitous dialogue.
I don't recall mentioning numbers anywhere, least of all number of subjects.
My pre-screening would apply as a pre-requisite whether the sample was 1, 100 or 10,000.
I also have not stated any preference for "instrument measurements". In fact even a cursory read of my posts would indicate that I'm talking about subjective listening tests - pretty sure I even say so...
Any circuitousness is arising from your prejudgement of my position. Kinda like judging a book by its cover, or an amp by its case.
You are really going around in circles now - what is it you want to say? You mention stats, then have some aversion to talking about numbers of participants, but have no problem telling Wavebourn his set-up is statistically unsupportable.
Much and all as I would love to chew the cud with you, I have better things to do - I'm off to bed.
- Status
- This old topic is closed. If you want to reopen this topic, contact a moderator using the "Report Post" button.