Go Back   Home > Forums > >
Home Forums Rules Articles diyAudio Store Blogs Gallery Wiki Register Donations FAQ Calendar Search Today's Posts Mark Forums Read

Digital Line Level DACs, Digital Crossovers, Equalizers, etc.

DAC blind test: NO audible difference whatsoever
DAC blind test: NO audible difference whatsoever
Please consider donating to help us continue to serve you.

Ads on/off / Custom Title / More PMs / More album space / Advanced printing & mass image saving
Reply
 
Thread Tools Search this Thread
Old 19th February 2018, 11:10 AM   #1591
Jakob2 is offline Jakob2  Germany
diyAudio Member
 
Join Date: Jun 2004
Location: germany
Quote:
Originally Posted by DF96 View Post
It is interesting to note all the reasons put forward as to why we should ignore any result which shows that two devices are not distinguishable:
- its the wrong question
- its the wrong people
- its the wrong day
- its the wrong auxiliary equipment
- the people were untrained/stressed/biased/prejudiced
- the music samples were too long/short/simple/complex
- the statistics don't prove anything
Posts like this make me always wonder as we had already discussed some of these points - sometimes even quite "excessive" with cited scientific evidence theory of statistical significance testing.

My concerns start with the usuage of "not distinguishable" as we simply donīt know about that (in the standard tests usually done).
As explained before, we are analysing the data under the premise that the null hypothesis is true (null hypthesis usually means in our cases "data could have been produced by random guessing") and decide about the compatibility of the _observed_ data with this assumption about the null hypothesis.

We do this decision by applying a statistical test (in our case it is quite often the exact binomial test) and using a predefined decision criterion (i.e. the significance level) and as a result we either refect the null hypothesis or we do not reject the null hypotheses.

In the latter case we do not know about the reasons for the observed data because we havenīt researched it. Therefore we canīt conclude that they were "not distinguishable" , may be the DUTs were distinguishable but something odd happened.
All we do know that we canīt reject the null hypothesis as the probability to get the observed data (and more extreme) was above our predefined decision threshold.

What about the ironically listed other factors (i assume that i understand the posters disbelief in the importance of all that correctly)?

Letīs compare it to a list of posted influecing variables in sighted listening:

- knowledge about brands
- different colors
- other peoples opinion about the DUTs
- expectation about a difference in general
- fear about expressing no difference
- not enough listening experience

and surely there are others as well.
In addition one has to maintain the argument that listeners are unable to compensate the impact of these bias factors in sighted listening tests but are totally shielded against any bias factor if only they take part in a "blind" listening test.
And of course you have to stand by this argument even if scientific knowlegde implies that it is wrong.

Quote:
Curious then that a test finding of 'distinguishable' is accepted almost without comment, even when the electrical difference combined with psychoacoustics means that the DUT ought (probably) to be indistinguishable.
Who decides if something not commented is accepted?
Mind reading abilities somebody?

Maybe people take anecdotical results as exactly that in hte sense of "might be correct or might be not" even if correct it might be relevant for someone else or might be irrelevant.

Quote:
I have dipped into this thread again. As things have not actually moved on I will dip out again.
Donīt want to be (too) offensive, but moving on starts with acceptance of long established scientific evidence of the bias effects in controlled listening tests (including the double blind feature).
  Reply With Quote
Old 24th March 2018, 05:09 PM   #1592
mikewxyz is offline mikewxyz  United States
diyAudio Member
 
Join Date: May 2017
Location: Portland, OR
I'm surprised this thread is still going.

Hi Jakob2: You clearly know a lot about stats and test protocol. You've talked about the gorilla experiment where he goes unnoticed a couple times. Your point being real things can escape our attention (Beta error). What is your view of the Alpha error risk (false positive) in audio tests - especially the less structured ones in the home or at the audio dealer? People claim all the time to see or hear things that aren't there? I just watched David Copperfield appear out of no where then shrink himself to 2'. I saw this with my own eyes! ;-)

I would say the preponderance of data shows the Alpha risk to be at least as high as the Beta risk.
  Reply With Quote
Old 26th March 2018, 11:13 AM   #1593
Jakob2 is offline Jakob2  Germany
diyAudio Member
 
Join Date: Jun 2004
Location: germany
Quote:
Originally Posted by mikewxyz View Post
<snip> You've talked about the gorilla experiment where he goes unnoticed a couple times. Your point being real things can escape our attention (Beta error).
Not only "real things" but even quite big real differences can remain undetected.

Quote:
What is your view of the Alpha error risk (false positive) in audio tests - especially the less structured ones in the home or at the audio dealer? People claim all the time to see or hear things that aren't there?
Itīs hard to say as it still depends on so many variables. Of course chances are high for false positives if some basic requirements (for examples no apparent level differences) are missing, and of course evaluation of audio equipment (and propper description of differences if there exist some) needs often practice.

Otoh i was up to now always able to get correct results in controlled (even blind ) listening tests when trying to find corrobation for something that iīve previously noticed in sighted listening conditions.

Quote:
I would say the preponderance of data shows the Alpha risk to be at least as high as the Beta risk.
Might be so, but it misses imo the point in a discussion like in this thread, as we are discussing the results of controlled listening tests and the possibility of further concluding to reasons for these results.

People donīt trust sighted listening tests and use some "blind" tests instead and it doesnīt make any sense in doing so if incorrect results are in the "new" tests as likely as they were in the "old" ones.
  Reply With Quote

Reply


DAC blind test: NO audible difference whatsoeverHide this!Advertise here!
Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
Will a small cap value difference be audible? jb0194 Multi-Way 2 8th February 2012 05:39 PM
Audible difference between DTS and Dolby? arjscott Everything Else 3 1st February 2007 12:33 PM
Which caps make the most audible difference? TDWesty Solid State 5 31st January 2007 05:49 PM
BrianGT Basic v/s Premium - any audible difference ? percy Chip Amps 1 4th September 2004 03:18 PM
RC Tolerance -- audible difference r_s_dhar Analogue Source 6 11th August 2003 07:47 PM


New To Site? Need Help?

All times are GMT. The time now is 05:32 PM.


Search Engine Optimisation provided by DragonByte SEO (Pro) - vBulletin Mods & Addons Copyright © 2018 DragonByte Technologies Ltd.
Resources saved on this page: MySQL 15.00%
vBulletin Optimisation provided by vB Optimise (Pro) - vBulletin Mods & Addons Copyright © 2018 DragonByte Technologies Ltd.
Copyright ©1999-2018 diyAudio
Wiki