Pitfalls? Lets stay with level matched DBT of DACs, in the comfort of listener's own setting and their own pace and duration of their own desire. There are pitfalls in such test?
You mention that i focused "on blind again", of course i do because that´s what is demanded just "a blind test" and quite frequently an "ABX" .
Yeah, the resonsible experimenters do a lot of research around sound contolled sensory experiments, that´s the reason why i quite often cite the published work and post excerpts or summaries or conclusions from it - the word salad you know......
The not so responsible posters in audio forums do other things like urging people to follow recipes without mentioning the conditions to be met to get correct results.
just from skimming, seems to be an interesting approach, the mentioned issues with AB and ABX tests aren´t really backed by data, which would be necessary for A/B; for ABX the data is already available.
But anyway even if the issues with A/B aren´t for real, another fresh approach is always worth a try, especially when trying sort of indirect measures (qualifies imo as a mix from quantitative and qualitative methods).
Maybe i´ll find time to digg in deeper for evaluation of the statistical implications.........
@mmerrill99,
as you´ve noticed before, that´s my optimistic side......
Last edited:
True
I think I asked this before, if listener screening per the ITU spec eliminates 999 out of 1000 subjects for a particular test why should we really care?
Last edited:
Could be even worse, what to do with the subjects that reliably identify a difference between DUTs, but fail miserably the +/-0.2dB level positive control test.
But have you never noticed Wurcer's multiple posts on many threads about ABX testing, warning people about how difficult these tests are as he claims "I have always said I wish folks realized how difficult it is to do these tests for small differences." ?
No? That's weird, as neither have I. Is it due to some inattentional blindness we have & everybody has who have never seen these posts? Perhaps or maybe his claim doesn't stand up to scrutiny?
Ironically enough, Wurcer's comment/claim is what we have been saying all along - go figure!!
Yea, you need lots of optimism in the face of the religious mindset that we see on this (& other threads). As you well know logic, evidence & scientific research means nothing to this mindset as evidenced here. One possible reason for continued posting is for the edification of lurking readers as the people posting here are not interested in scientific truth..........
@mmerrill99,
as you´ve noticed before, that´s my optimistic side......
You've put in a lot of time & effort as have I, in this & other threads to bring some scientific knowledge to the threads - maybe some readers find it of value but I don't see any hope for the core we see here.
What seems to be missing here is the desire to understand the complexities & nuances of auditory perception. Despite many efforts on our part we still see the same ol' mantras "Pitfalls? Lets stay with level matched DBT of DACs, in the comfort of listener's own setting and their own pace and duration of their own desire. There are pitfalls in such test?"
I know you've mentioned it before but the concept of change deafness seems to be an alien concept to all those who post mantras as above.
Let's see if they read this research "Musical Change Deafness:
The Inability to Detect Change in a Non-speech Auditory Domain" or if it is just categorised as word salad (other readers may appreciate it, anyway)?
This paper shows the inability of listeners to detect obvious differences between two pieces of music - musical change deafness.
As you know it is related to the well-known issue of change blindness in visual perception where people easily miss gross changes in visual scenes.
To allow open-minded readers achieve a balanced view of ABX testing, rather than the simplistic mantra above, consider how long it takes to do a spot the difference test for small differences between two images. These are two static images, not dynamic like a movie (or like music). So, if you were asked to spot the difference between two movies that had some differences in some of the frames - what would you tell the person making this request?
You really ask this? What would you do if you were running measurements in your labs & when you tried to calibrate the measurement equipment you found it was way off by orders of magnitude?
Hmm, let's see - maybe we would look at the test itself & examine if it is testing what it claims to test? Perhaps we would look at all the results of previous tests & re-examine the results, if we could?
I've often seen it stated - "just do a blind test & your eyes will be opened" (apart from the irony of this) I would suggest that if positive controls were included in such tests, yes, their eyes would be opened to just how much auditory perception is blind to differences for lots & lots of reasons that fall into the definition of change deafness
Their test results are discarded from inclusion in statistical evaluation - don't know what your dilemma is - looks like you are interested in manufacturing only one result otherwise you would not have to ask this question?
I'm flattered that you follow me, some personal agenda perhaps? AFAIK I never cost you money or fired you (I never fired anyone though one of my techs ended up in jail and saved me the trouble).
I have commented many times that what we are looking at is subtle and not so obvious that any test is a waste of time. I have spent most of the time on this criticizing the "you have to be deaf or a liar not to hear this immediately" comments. I consider subtle as being difficult to test for by any means (by implication) but go ahead put any spin on it you want.
Maybe you could find where I "doggedly" defend ABX in explicit terms, you will then ignore all the times I offered to accept any comfortable protocol as long as it was blind. Or maybe you simply ignore all the forum participants that reject all blind testing and accept their results with equal weight.
I have to say doing due diligence to follow the ITU spec seems a costly and time consuming venture, so I can see why someone with money on the line would have more to lose and the average consumer little to gain. Then there is the issue that even if there are significant results for difference of what use is it if it is not tied to a preference to what is being sold.
The point was what if only 1 in 1000 people can tell the difference reliably between A and B. Is that not a possible outcome? For visual acuity there is a small percentage of the population that can see into the UV, I can easily imagine a TV comparison only they could score highly on. Same might be true for smell and taste.
We don't poll meters BTW and take a vote.
Last edited:
Sorry to disappoint but I'm not following your posts but I do take note when your many replies to me & others indicate the opposite of what you claim about yourself ""I have always said I wish folks realized how difficult it is to do these tests for small differences."
For instance this post in the thread "DAC Blind test: no audible difference whatsoever" where I first came across your bias here
Of course, if they exist, you can point to all the posts you have made which state that blind testing for subtle differences is difficult & I'll accept I'm wrong - I'll wait - shouldn't be difficult t find as you "have always said.......how difficult...."
Last edited:
That's not what you said - you asked about people's results for the positive control - not about A & B
Is what not a possible outcome - you are all over the place with your logic and/or communication?
You really don't grasp the concept of positive controls, do you? I'm afraid it's a waste of much time & effort stating it again to you - I would suggest you spend some time & read into it but I know you are not interested in listening tests, as you have stated
You do make some convoluted analogies in your posts - if you have some point to make state it clearly, please.
Last edited:
Could be. A lot of it seems to be something that can be learned. Harder to learn though if no dac around with sufficient performance. In other words, there is to some extent a chicken-or-egg aspect to it, but that is unlikely to change anything. If technology happens to get to the point where better dacs are available at lower prices I think we will see more than 1 in 1000 that find them audibly superior and worth having.
The point actually is that if the test by which you judge the difference between DACs is flawed (forum suggested ABX tests) then you are possibly coming to a false conclusion - just about the same level of surety as sighted listening.
So, bottom line, you are discarding the results from every subject that do not pass the positive control test, disregarding their results in hearing DUT differences.
What was the hypothesis you were testing?
Last edited:
Please tell us your understanding of the purpose of positive controls.
The hypothesis of the blind test has nothing to do with positive controls - why do you ask?
Please don't deflect and answer the question: would you discard the results from every subject that do not pass the positive control test, disregarding their results in hearing DUT differences.
By doing so, in my opinion, you are also discarding the hypothesis "are there any audible differences between A and B?" and replacing it with "is there any human that can identify an audible difference between A and B?"
There is nothing wrong with this new hypothesis, but we would want to know exactly what is under test. Humans with extraordinary hearing abilities do exist, as much as some can see the Mars satellites or Saturn rings naked eyes. Problem is, what relevance has this new hypothesis?
Dear oh dear - I already answered & you already repeated my answer - I suggest you go back a post or two & read your own answer. Your level of duplicity & argumentation is amazing or maybe you just don't understand the answer?
Again you prove that you have no clue on what positive controls mean & what they are used for & furthermore refuse to read any research papers posted. Anti-science to the end. Not worth the effort of replying unless you can show that you have actually put in some effort to understand & not simply arguing from your belief system. Educate yourself on the matter & maybe you will be able to ask some sensible questions
Based on wrong premise, this is meaningless.
Last edited:
- Status
- Not open for further replies.