Go Back   Home > Forums > >
Home Forums Rules Articles diyAudio Store Blogs Gallery Wiki Register Donations FAQ Calendar Search Today's Posts Mark Forums Read

The Lounge A place to talk about almost anything but politics and religion.

John Curl's Blowtorch preamplifier part III
John Curl's Blowtorch preamplifier part III
Please consider donating to help us continue to serve you.

Ads on/off / Custom Title / More PMs / More album space / Advanced printing & mass image saving
Reply
 
Thread Tools Search this Thread
Old Today, 08:41 AM   #9981
mmerrill99 is online now mmerrill99
diyAudio Member
 
Join Date: Aug 2014
Quote:
Originally Posted by PMA View Post
I agree on both. Too high volume can pick up noise-only related issues. 16 trials at least, agreed.
For the reason of possible "too high volume" issue, I constituted both of my latest test to be immune to this manipulation. This is made by sample music choice, cutting no-signal beginning and ending of files and further tests and file comparing.
the participants also have no freedom in what music they play or how/on what it was recorded.
Quote:
Regarding test to hidden controls sensitivity, IME I did what I could.
That does seem like a dodge - unless you have some controls you will never be able to answer the question I posed & the people doing your tests should also be aware that they are using a test procedure with unknown sensitivity.
  Reply With Quote
Old Today, 08:50 AM   #9982
PMA is offline PMA  Europe
diyAudio Member
 
PMA's Avatar
 
Join Date: Apr 2002
Location: Prague
Quote:
Originally Posted by mmerrill99 View Post
the participants also have no freedom in what music they play or how/on what it was recorded.
That is correct. However, in the latest test they are comparing an original file vs. the recorded file, that means an original with a chain: D/A - hybrid tube/SS power amp - A/D. The recorded file has so many variables and influences that the participants might have been able to get a positive ABX result?

I would understand if you complained against validity of a positive result, but why against negative result? Just a suspicion?
  Reply With Quote
Old Today, 09:01 AM   #9983
PMA is offline PMA  Europe
diyAudio Member
 
PMA's Avatar
 
Join Date: Apr 2002
Location: Prague
Quote:
Originally Posted by mmerrill99 View Post
That does seem like a dodge - unless you have some controls you will never be able to answer the question I posed & the people doing your tests should also be aware that they are using a test procedure with unknown sensitivity.
What kind of "some controls" are you requesting, and please describe it exactly.

I guess that any kind of measurements or file analysis is not enough for you.
  Reply With Quote
Old Today, 09:05 AM   #9984
mmerrill99 is online now mmerrill99
diyAudio Member
 
Join Date: Aug 2014
@PMA, what I'm pointing out is that your statement "absolutely free choice of almost anything," really doesn't stand up to scrutiny & the very fact that participants know they are being 'tested' is an overriding stressor. So I don't about you are free argument, either.

As I & & others have said, a relaxed listening to many types of music over some extended period is far different to what you are doing.

I also just have to point out the underlying bias that is seen in this form of testing - you just spent to some lengths in your latest test files to eliminate 'cheating' by too high a volume, yet you dismiss any effort to test the sensitivity of the test itself.

This bias is also seen in the writers of Foobar ABX who updated it a while back adding on anti-cheating measures (log file checksum) but not paying the slightest effort in implementing hidden controls - it would be easy for the ABX utility implement the hidden control I suggested.

All the efforts are to minimise positive results - no effort to validate the test itself

Last edited by mmerrill99; Today at 09:10 AM.
  Reply With Quote
Old Today, 09:15 AM   #9985
mmerrill99 is online now mmerrill99
diyAudio Member
 
Join Date: Aug 2014
Quote:
Originally Posted by PMA View Post
What kind of "some controls" are you requesting, and please describe it exactly.
I already described it in sufficient detail for anyone interested to implement - please reread a couple of posts back

Quote:
I guess that any kind of measurements or file analysis is not enough for you.
Is the reason for DBTs not to ascertain audibility? If not then why not just stick to measurements if you're sure they define all that can be heard?
  Reply With Quote
Old Today, 09:21 AM   #9986
Jakob2 is offline Jakob2  Germany
diyAudio Member
 
Join Date: Jun 2004
Location: germany
Quote:
Originally Posted by PMA View Post
If someone says he clearly hears a difference in a sighted test and he was listening to two bit identical files, then it is a false positive result produced by the sighted test.

How do I know that the ABX has produced a false negative result?
The usual approach to get more information would be an analysis following the hit/miss scheme used in the SDT experiments.
Unfortunately you must have a log of the trials to know what was presented as "A, B and X" and to look for some patterns in the results.

Quote:
Originally Posted by PMA View Post
<snip>

So, IMO, all your suggestions are fulfilled, the participant has absolutely free choice of almost anything, no stress, he does not have to go somewhere and fulfill the protocol of the test in unknown place with unknown people.

So, taking into account this almost absolute freedom of choice and no pressure, why are there almost no positive ABX results in tests like wire x tube pre with 0.5 - 1% distortion, or even when testing the original file vs. recording through hybrid power amp with same distortion, loaded with speakers? Is it because of "stress", or for the reason that the sound differences are at the threshold of hearing resolution, hearing cleaned from sighted biases like look, brand, good highend story etc.?
Which means that listeners that arenīt used to do such tests should figure out themselves which method provides correct results. But they donīt get help by any advice or positive controls that can be used for training purposes.

Good questions what the reason for having nonpositive ABX results is; usually an experiment should be designed to help answer those questions, but as you might remember, if it canīt be shown that a test is objective, reliable and valid you canīt draw further conclusions from the results.

Thatīs the reason why the test is done and still the questions remain.

Last edited by Jakob2; Today at 09:24 AM.
  Reply With Quote
Old Today, 09:43 AM   #9987
gpapag is offline gpapag  Greece
diyAudio Moderator
 
gpapag's Avatar
 
Join Date: Nov 2002
Location: Athens-Greece
John Curl's Blowtorch preamplifier part III
Quote:
Originally Posted by PMA View Post
That is correct. However, in the latest test they are comparing an original file vs. the recorded file, that means an original with a chain: D/A - hybrid tube/SS power amp - A/D. The recorded file has so many variables and influences that the participants might have been able to get a positive ABX result?
In terms of level balancing, you have done a great job. Plus the amp does not change the dynamic range of the signal (<0.2dB)

Quote:
Originally Posted by mmerrill99 View Post
I already described it in sufficient detail for anyone interested to implement - please reread a couple of posts back
Is this one?
Quote:
Originally Posted by mmerrill99 View Post
The use of some form of hidden controls, as recommended in ITU guidelines, is one possibility to ascertain is the participant sensitive to a certain level of known audible difference - for instance in one or more of the 16 trials of an ABX test, B could be an exact copy of A except that it has been adjusted by 1dB (or whatever is deemed appropriate) & if X is not identified correctly as A or B then we have an indication of false negative for this particular type of difference - does it generalize to lack of sensitivity to other small impairments? More than one trial & more than one run of ABX would be needed to evaluate the sensitivity of the participant/test to small impairments. Volume level is just one suggestion, easy to implement, to act as a hidden control - other differences are possible & other approaches have been suggested in the past
George
Attached Images
File Type: png avocado-apricot statistics.png (42.9 KB, 0 views)
__________________
["Second Law is a bitch." - SY]
["I insist on respecting the means of the average person working in their garage/basement." -Scott Wurcer]
  Reply With Quote
Old Today, 09:47 AM   #9988
Jakob2 is offline Jakob2  Germany
diyAudio Member
 
Join Date: Jun 2004
Location: germany
@ PMA,

we addressed the question for a positive control in the other threads already quite extensive.
The level difference is a good one, as listeners usually (means in a multidimensional evaluation) , if the level difference gets small between the samples, donīt recognize it as a level difference but as a sound difference.

It depends on the experience, at first listeners may have difficulties to recognize a level difference of 1 dB as such, while experienced listeners will still be able to do it.
Below 0.3 dB it gets more and more difficult and thatīs usually the region where imE all listeners only notice the sound difference but not the level difference as such.

So using level difference at different niveaus even allows to monitor the effect of training.

Wrt the sample size needed; what Leventhal did was to reinvent the power analysis concept that Jacob Cohen revided already in the mid 1960s.

To reach the usual SL = 0.05 you donīt 16 samples (thatīs why our preamplifier could be used although there were only two tests of 5 samples done) and wrt statistical power 16 samples do not really help either.

Iīve posted some power calculations before, so only in short now; if you want a power of at leas 0.8 and want still be able to get positive results for low detecting abilities under the test conditions, then it the variables are:

power = 0.8
SL = 0.05
p2 = 0.6
p = 0.5

and therefore the sample size needed would be 158.

Otoh, we could calculate the power that results for the same variables but a given sample size of 16:

SL= 0.05
p2 = 0.6
p = 0.5
sample size = 16

and then the actual power will be 0.167. Which means the chance to miss this difference (which _is_ perceptable, so the null hypothesis must be rejected) is a whopping 83.3 % .

If you would like to balance SL and statistical power to have the same error risk in all cases, things are getting even more worse.

Small effects, as DPH already mentioned, to detect is difficult.
  Reply With Quote
Old Today, 09:57 AM   #9989
mmerrill99 is online now mmerrill99
diyAudio Member
 
Join Date: Aug 2014
Thanks, Jakob2 & you are correct but let's keep it simple - I would be interested in the results of hidden controls from the typical ABX tests run here.
  Reply With Quote

Reply


John Curl's Blowtorch preamplifier part IIIHide this!Advertise here!
Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


New To Site? Need Help?

All times are GMT. The time now is 09:58 AM.


Search Engine Optimisation provided by DragonByte SEO (Pro) - vBulletin Mods & Addons Copyright © 2018 DragonByte Technologies Ltd.
Resources saved on this page: MySQL 15.00%
vBulletin Optimisation provided by vB Optimise (Pro) - vBulletin Mods & Addons Copyright © 2018 DragonByte Technologies Ltd.
Copyright ©1999-2018 diyAudio
Wiki