the one they prefer is the best. while the objective performance can be measured, personal enjoyment cannot be objectively measured. it does not really matter where the personal preference comes from, that is the goal.
Or we have to skip the statistical presentation tool view and go for signal waveform trace comparison for this kind of test with most resolving recording gear we can muster. Problem is the poor involved human who has to endure numerous repetitions and perform listening wise in a constant way.
When one think about it some more it seems not so lightly one could set up a resonable conditions for such an investigation - the humans being the most problematic part of the equation. And what would we talk about here? 🙂 No, back to the bantering ;-D
//
When one think about it some more it seems not so lightly one could set up a resonable conditions for such an investigation - the humans being the most problematic part of the equation. And what would we talk about here? 🙂 No, back to the bantering ;-D
//
And yet the outcome will only be that test subject A may think it's the best thing since sliced bread and test subject B may think it sounds like crap. Or should we aim for majority decisions: if over 50% of test subjects think it's the best thing since sliced bread then it is the best thing since sliced bread.
Might it not make sense to blind A/B test one dac with another brand or model that measures similarly before jumping to a conclusion that they can't possibly sound any different?
Why do so many people keep assuming the only kind of listening test is a preference test? Hasn't anyone learned anything about discrimination testing and or descriptive analysis methods?
I'm not sure who are you targeting with this comment. Every listening test referred to in this thread has been a sighted subjective preference test. On top of that sighted subjective preference has also been used as basis for totally speculative claims of technical superiority.
Really? What about the one (#103) at: https://www.diyaudio.com/community/...-ess-es9038pro-wow.388314/page-6#post-7153467
The member describes differences as obvious, and he appears to have used discrimination/descriptive analysis methods rather than preference. To declare it was nothing more than a matter of preference would not be a credible claim.
The member describes differences as obvious, and he appears to have used discrimination/descriptive analysis methods rather than preference. To declare it was nothing more than a matter of preference would not be a credible claim.
It also contains very specific difference that we at the level of being 'obvious.' In the past when listening skeptics have described audible difference in sighted listening and I asked if they could trust the result since it wasn't blind, they justified their claim of an audible difference by saying that it was so obvious that blind testing was not needed. In other words, there is hypocrisy among the skeptics. If you personally hear an obvious difference in sighted listening, then its real. If someone else does then is can be dismissed as sighted subjective. Suppose you listened to two dacs fully sighted and one played the sound of a buzzer and other played the sound of sine wav, would you need to be blinded to be sure? Of course not!
In post #103 the listening test is as biased as can get. They use equipment which is owned by the person arranging the listening test and which is clearly his preference. Also the music used in listening was selected by the same person so he had formed a preconception of how it should sound.
This is empiricism at its finest. "Trust me alone" is what I am getting out of that type of claim. Throwing in the "it is obvious" qualifier just makes it much less believable to me, especially in this case given other ways in which these DACs have been qualified.
The problem with that is it has been shown that the brain can warp perception to extremes when there are multiple stimuli present that can introduce bias. This is why sighted testing is in general completely unreliable. Add in the fact that the owner is using his own gear and just doing some random A/B comparison and you cannot really take anything meaningful away from the claims. Was there a "difference"? Maybe, very possibly. But how you can assign that to performance of one unit or the other being flawless vs faulty or whatever is just not possible. There is only a difference. So what?
I think you have here a too simple model of reality. I would not deny the phenomenon of perception bias. But I think this also depends on the level of change. You do not need a blind test to hear the difference between an amplifier on and off, right? On the other end of the scale, small changes, a blind test may not help much as well. Some changes may only expose itself after long duration listening with many listening tracks or by observing listening tiredness. Furthermore, we could
separate the ability to hear changes versus the ability to hear what is better or worse. The second is inherently more difficult and more bound to user preference, mood, surrounding factors and audio components and may even change over time. In my experience, almost all changes you make to equipment or even audio software can be audible. Finally, the perception bias may also work the other way around. If you believe firmly that all DACs sound the same, you may not be open to perceptions that show differently...
How many of you listening deniers think you know more about listening tests than Earl Geddes (who is a forum member, so he may find out if you try to trash him)? He has Ph.D. He has published multiple AES papers on what people can and can't hear. So who wants to raise their hand and say you know more than he does about listening tests? Anyone?
Next question, who has seriously studied the modern literature on perceptual testing? Not just for preference testing, but also for discrimination and descriptive analysis? Anyone?
Some of the recent complaints here appear to fly in the face of science. Sean Olive commonly uses certain musical tracks for testing. The same test subject may hear the same piece of music over multiple test sessions. Why? Because it will bias the listeners and therefore Sean Olive doesn't know as much as you do about how to do it right?
Next question, who has seriously studied the modern literature on perceptual testing? Not just for preference testing, but also for discrimination and descriptive analysis? Anyone?
Some of the recent complaints here appear to fly in the face of science. Sean Olive commonly uses certain musical tracks for testing. The same test subject may hear the same piece of music over multiple test sessions. Why? Because it will bias the listeners and therefore Sean Olive doesn't know as much as you do about how to do it right?
Last edited:
https://en.wikipedia.org/wiki/List_of_cognitive_biases
You really, really need to study this and do some self-assessment.
Many of your points have merit. But perhaps instead of making blanket statements using industrry specialists as a crutch (authority bias) , cite specific pieces of their work to support your claims. Have YOU read all of their work? If so, you'll be able to provide specifics.
Would I say I know more about listening tests than Geddes? No. Will I say I know more than him about cognitive science and qualia? Probably.
Edit: many of the aspersions you've cast upon engineers (for whom you seem to have little regard) could equally be applied to any hyper-specialised researcher, regardless of field, including the luminaries you've mentioned.
Last edited:
Name dropping is a sure sign that you have lost the argument. And supposing Earl Geddes and Sean Olive know lot about listening tests mentioning their names does not add any credence to your claims of listening tests.
An hour later and my post hit 1000 words. I reckon no-one in their right mind would want to wade through that much waffle. I'm happy to post it if folks are actually interested.
Mark, shall I PM it to you?
First I am not a "denier" but name dropping of famous, gifted, highly educated persons doesn't say much IMHO. My statement is that many know what they like best in a listening test and no serious reasons/scientific blah blah to prove them being wrong exists as far as I know. Even if things can be proven scientifically one can not argue about taste. Have been in many listening tests where a minority really liked quite mediocre sounding gear best. The people with titles can then say what they want but it won't matter. Jamie Oliver can cook you a five star meal but chances are likely that you like your mums food better.
Name dropping is more a weakness than a strength to prove anything. We had scientists here that had theories used for education that were only discovered many years later to be made up BS. It is all human stuff often with ego bias. Things generally only work OK when a group effort was done so persons weaknesses and strengths are eliminated/combined. The more serious the field is the higher the desired qualities of persons. This is audio where science and pseudoscience are equally valued and blue LEDs are cherished.
Have a look here at some single person designs that have clear design weaknesses. Being famous/known can be a burden instead of a blessing.
Last edited:
- Status
- Not open for further replies.