I´m not sure therefore my question. 🙂
As i know from experience (my own and from conducting controlled tests with other listeners) humans are able to train for test condition and to reach quite surprising sensitivity. But humans are different and not everybody does equally good within each test setup.
To have balls is fine but does apparently not help in doing tests if one doesn´t have experience in doing this sort of tests.
Neglecting the training issues would mean to act carelessly.
I'd just argue that it says "at the time I heard/didn't hear ______, but I wasn't well trained and therefore don't know if I missed something".
It's honest, it leaves definite possibility for something more carefully defined and gives you a good snapshot of where you are today.
Of course. Doing sighted, uncontrolled test are much easier, nobody bothers you about training and stuff, every moron can do it and therefor such results are much more reliable and convincing.
Or you can just run away and hope the issue goes away quietly.
James has repeatedly stated that he has heard differences in what came across as serious listening tests. He must have some competence in comparative listening.
Jan
Last edited:
That´s pure eristics, you can do better.
Or should i just copy a post from jan.didden where he complained about others posting carelessely things that already were explained numerous times?
I did explain numerous times, why training under the specific conditions (including positive and negative controls) is mandatory, you could study it in the ITU-R.BS1116-x, could read it in presentation from jj, could find it in most literature explaining the fundamentals of sound experiments, but obviously (and beyond my understanding) you constantly refused to do so.
Btw, sighted controlled listening tests don´t have to be more uncontrolled than "blind" controlled listening tests, because you could proceed (despite the missing "blinding") in exact the same way.
But if you missed to control the impact of the "sighted" condition another observer can´t decide if the results were correct or just biased (because the experimenter is not able to show internal validity).
Otoh, if you conduct a "blind" test and forget to control the impact of the test conditions - including the "blind" property - (if it is new to the participant(s)) another observer can´t decide if the results were correct or just biased (because the experimenter is not able to show internal validity).
Maybe i missed where he described his test protocol which included the "blind" property and was done with multiple trials.
If he didn´t had a strict test protocol (including the "blind" property and multiple trials with correct answers) than you should assume that James has some competence in comparative listening but not under the test conditions which he will be confronted with.
I don't understand what you are talking about Jakob. If James can hear it and it's obvious to him, how could he fail to hear it blind? He already knows what to listen for.
Of course it might be good for him to train up a little, take notes, get some tracks where he can quickly tell the difference. He can study for the test. But as he already knows the answers, how is the testing protocol going to affect that?
Of course it might be good for him to train up a little, take notes, get some tracks where he can quickly tell the difference. He can study for the test. But as he already knows the answers, how is the testing protocol going to affect that?
Dr Rod Rees told me 25 years ago, privately, to not be induced into double-blind tests as approved by Lipshitz, et al. So I don't do it, and personally I am not as good with this test as are some others of my acquaintance, so why bother? My job is to make the best audio electronics possible, and not hearing any differences does not help me discriminate between my design decisions. Here is one of Dr. Rees's articles on the subject. He did this sort of thing for a living!
Attachments
Last edited:
Things have moved on in the past 30 years, fyi. Also: our understanding about conscious and subconscious bias has not been kind to human's ostensible propensity towards logic and non-bias.
That paper runs a nice gamut of all the "throw the baby out with the bathwater" gambit about DBT. Yes, it should make limited claims as well, just like any other test.
Really, DPH? Dr. Rees who taught philosophy and history of science at a university is behind the times today? I think not.
There's a great saying by Planck: "Science advances one funeral at a time".
And: Does science advance one funeral at a time, researchers ask | PBS NewsHour
Yes, most professors (and students and everyone else too) are behind the times. I should know, I work with one! Extremely bright guy with glaring weaknesses in his understanding (not that he isn't wont to learn). I'm 100% confident I am too.
It's especially bad when you're citing something from 30 years ago that is written as an op-ed as your rationale. Doubly so when it comes to human perception/decision making.
Rees was a gasbag even then; his articles were hilariously awful if you could plow through the turgid prose. I assume he hasn't gotten any better with age.
I am a bit surprised because the effects are well studied and documented.
If an experimenter would try to find out if James did really percept what he thinks he did, under the exact conditions as before than you were right. (Leaving aside that even to know being part of an experiment could have an impact)
But as the experimenter will change the conditions there is more to it as just to know what to listen for.
That the listener has to evaluate a multidimensional perception makes things even more complex.
The experimenter believes, that it couldn´t make a difference? Well, he simply should test his belief, that´s what it is all about.....
If he knows the answers, why should someone test him at all?
Easy to answer, because you don´t trust in his ability to seperate real listening events from biased driven perceptions.
But as soon as you put a subject under a test regime you are willing to assume that this listener (whom you didn´t trust a second before) will be transformed into the most reliable perfect observer, unaffected by any unfamiliar circumstances. It´s a contradictio in ratio.
If you analyze published results of controlled listening tests, you will clearly notice that a lot of bias effects are still at work although "blinded" conditions were provided. The reason is simple, "blinding" removes just one bias mechanism, all the others are still at work. And, as the experiments of SDT (signal detection theory) have shown that is even given if the particpant were tested for quite simple signals and/or properties.
A multidimensional perception is even more prone to these bias effects.
Btw, the article by Ress/Shaffer that John Curl posted before contains very valuable information.
Enough straw men there amidst the woffle to create a fire hazard.
James has confidence and curiosity- there's always someone out there ready to talk him out of those virtues. 😀
James has confidence and curiosity- there's always someone out there ready to talk him out of those virtues. 😀
I read the first page. He seems typical of people who talk about science; I assume he didn't actually do much physical science? Not worth reading the rest, as it seems designed to massage the egos of true believers and affirm them in their beliefs, rather than seriously challenge the alternative view that listening tests should involve ears alone. Does he seriously not know the difference between 'blind listening' and 'blind meter reading'?john curl said:
It is always nice to know how weak is the argument of the other side. Thanks, John, for exposing this.
Alas, I read the whole thing. I drew me in, like a bad horror novel.
I was thinking the same thing, but could not believe it could be so easy.
There was one thing I did like, blind meter reading. I have been guilty of interpreting measurements with bias. You do have to be careful about that.
Do you have any reason to believe they aren't? This is not un-knowable rocket science, it's easy to test.
- Status
- Not open for further replies.