What kind of evidence do you consider as sufficient?

Status
Not open for further replies.
Please explain what you fiind so wrong with positive controls as suggested?

I already said use them if you want, but someone has to do the work. You also have to accept a number like 99% of the subjects could fail the controls if you stick with ABX. If Bob Katz is the only person out of 100 in the room that can reliably hear the difference in the DAC's (or whatever) that would be good to know.
 
I already said I have no problem with that discussion in the quoted paper.
I have to say I'm stunned. You have engaged myself & Jakob in a dogged defence of ABX testing over hundreds of posts across a number of threads & yet you now say the above?

Is this a change of your position upto now or just that it's the messenger that is of importance to you, not the message (despite what your signature says)? Actually don't bother answering - I'm not interested

Didn't Irwin Pollack prove ABX does not work well for certain sound recognition tasks in the 50's? I keep seeing the same thing here ABX = blind so blind = ABX. Not very good logic.
Nobody is saying this but you strive to try assigning this equivalence to me or Jakob whenever you can - straight from the bible of SY

Being a paid subject recognizing the sounds of certain consonants does not come with a lot of personal involvement (ego/financial interest). If you can find some papers that show having a strong vested interest in the outcome of a sighted test does not affect the results I would gladly read them.
Have no idea what you are talking about & frankly based on the last hundreds of posts, I don't care to know - further waste of time & energy with you is of no interest to me

How many times have I said I don't care what protocol you use, invent a new one, use any positive controls you want. If you're claiming that a blind test for audio impairment can't be invented, I guess we will just have to go on with the anecdotes.

I have always said I wish folks realized how difficult it is to do these tests for small differences.
Wow, just wow is all I have to say - goodbye!!
 
Last edited:
Still no proposals. Maybe we should revive the "what is wrong with op-amps", thread.

Yeah, it is getting to look more and more like nothing is going to happen.

I think there was a point Earl Geddes would have been willing to do some more research into the 5%, but he said it would take money for even a preliminary study. And it wasn't for dacs.

Even for comparing files, I have always thought it would take the existence of suitable software first to build up momentum.

There are people around, academics, who do cognitive listening research, maybe most of them musicologists. Like all academics, they probably need funding sources. Come up with some money to support dac comparison research they can publish and you will probably find some takers.

Here at diyaudio finding someone to take it all all on for free, for the mere glory, and then face years of threads arguing and defending, why wouldn't anyone want to do that?
 
@scott wurcer,

Again you don't read what I say "I don't see how sensitivity to level change has anything to do with two DAC's that can be adjusted to +-.01dB, but use it anyway if you want." is a summary dismissal.

It´s not sensitivity to a level change, but sensitivity to a perceived change in sound quality evoked by a slight level difference.

Let me make one thing clear blind and preferably double blind is an absolute necessity for evidence to be sufficient.

It´s necessary but not sufficient as a general guideline for scientific evidence, what you personally think is indeed a different matter.

This is my opinion, whether blind is sufficient is a different matter.

Sometimes the surrealism of these threads is stunning. The thread topic is about what kind of evidence _you_ consider as sufficient..... :cool:

So, what do _you_ consider as sufficient? What generally is considered as sufficient is something i already know. ;)

<snip> If you can find anything in their discussion where they suggest sighted testing might be equally valid please point it out.

I know you don´t like me mentioning it, but you used the "tell me where they suggest sighted testing ...." line three times in your last posts.
It is sort of implicite strawman, as it suggests to a reader that did not follow all the posts, that your opponent only considers sighted listening
as valid and that isn´t obviously not true.

That seems obvious, how do you know a priory the sonic characteristics of what is being tested. So how do you pick the level of the positive control and why eliminate a subject that fails say a 0.2dB level change test when high frequency IM is being tested for.

That´s the key point, as it is a chicken-and-egg problem. Noone of us knows if really an audible difference exists anyway, and therefore noone of us neither knows what kind of perceived difference exists nor what the physical reason might be.
That makes you "when high frequency IM is being tested for" assertion a bit strange, because we simply don´t know that.

We know, that according to our measurements and the model of hearing that we use, we conclude/predict that no difference will be audible.
That´s the rationale for using positive controls that require already quite good sensitivity of the listeners _under_ the _specific_ _test_ _conditions_ .
 
I have a blind listening procedure I use for myself when I want to do that. It is a good thing to do if differences are less than obvious, especially depending on the nature of perceived differences. But like anyone else, I have some tendency to think it not necessary when differences are plainly obvious. There is of course a potential trap there, and it is part of the risks we have to deal with, being human.

The "being human" part we have emphasized for good reasons in the past,as infallability will not be in reach.
What you describe is in line with usual practice in the sensory evaluation field; first you ask an expert and if he thinks the difference is small then controlled (blind) tests come into play. Professional sensory evaluation in a lab is costly and nobody wants to waste money.

In other words, any perceived differences always tend to seem plainly obvious when sighted, that tends to be the case for humans in general. Those here who think they never misjudge when listening sighted might be in for a big and potentially embarrassing surprise someday.

The risk is considerably lower if experiments follow best practice principles including the choice of test protocols, but nevertheless failure is always possible for mere humans. :)
 
@Jakob, as far as I can see this thread (& others along similar lines) are populated by a group of people who have no interest in the matter (some have declared this) & whose sole intent is to disrupt, denigrate & deny, deny, deny.

It's a waste of your time & expertise to waste energy on these people - at least that's my decision.
 
So how do you pick the level of the positive control and why eliminate a subject that fails say a 0.2dB level change test when high frequency IM is being tested for.

That´s the key point, as it is a chicken-and-egg problem. Noone of us knows if really an audible difference exists anyway, and therefore noone of us neither knows what kind of perceived difference exists nor what the physical reason might be.



That makes you "when high frequency IM is being tested for" assertion a bit strange, because we simply don´t know that.

We know, that according to our measurements and the model of hearing that we use, we conclude/predict that no difference will be audible.


That´s the rationale for using positive controls that require already quite good sensitivity of the listeners _under_ the _specific_ _test_ _conditions_ .


Here we go; word salad, circular deflection back to the hypothesis, and no answer to a perfectly legitimate set of questions (which I asked myself few pages ago, and more). QED.
 
@Pano,

Could be, but is not. I've attempted to follow what you are tying to say for years but it is mostly just mud. Yes, I've seem some so-called concrete information, but it just gets lost in the mix. Let me cite a music example. If you want the guitar solo to stand out you had better carve a space for it in the mix. Not only does it need to be a little louder than everything else, it needs to sit in its own tonal and spacial range. If not, the message of the guitar solo gets lost.

I see. But what about the agenda, is it settled by my (not so thick ;) ) explanation in my last answer?
To help people to recognise the pitfalls of controlled listening tests and to help them to get better tests/results when doing their own experiments?

Your message is getting lost in your overly thick posts. You can accuse me and others of not paying attention, but that is be wrong. We are trying to understand - but you are failing to get your point across. This isn't our failing, it's yours. We are open to understanding you, but you don't make it easy.

So basically, if i understand you correctly, just writing more plain (and shorter) would be better; maybe:

"Wrt audio tests (blind tests) there is a simple rule, setting up a really scientific test isn´t as easy at it seems. (A test is a scientific one if it is objective, valid and reliable)

The first step would be to learn from other fields (like psychology) and to read some of the standard literature to get feeling for the important elements of good experimental design.

We have to keep in mind that a listening test relies on the listening abilities of the listeners and so is as much of a test of the listeners as it is for a, maybe existing, difference.

An experimenter has to consider a lot of parameters and an important one is the choice of the test protocol.

The participants/listeners should have trained listening in general and should have trained to listen under the specific test conditions to get best/correct results."

What do you think,would it help?
 
Jeez, you make it sound like life and death. Perhaps you should read that great spiritual work "F**k It"

WHAT? It is not about life and death? You´re guilty of heretics......, where is the spanish inquisition if one needs it.... :D

Jakob2, what about your ties with audio electronics selling business?

My guess is... none.

or...

//

As scottjoplin already pointed out, there are ties, as audio development was and is part of my work.
 
Status
Not open for further replies.