John Curl's Blowtorch preamplifier

Status
Not open for further replies.
Jump to Latest
#8,801
Hi Jan,

I guess there is a relationship between the number of subjects that do the test and the number of test runs?
Click to expand...

As a first estimate I would assume that the total sample size is what matters, so if you do 10 times 20 people or 20 times 10 wouldn't matter. That is, one assumes the people to exist independently of eachother. However that's just an approximation as there are always influences among people (peer pressure 😀 ).

What would the numbers be for comparing two amps, with an error estimate, say 5% (if that is a sensible number)?
Click to expand...

You mean 5% uncertainty?

Lucky me, wikipedia saves me from calculating that 😀

From http://en.wikipedia.org/wiki/ABX_test

If one ABX trial is performed, a confidence level of only 50% is achieved, the same as flipping a coin. With increasing trial iterations, increasingly accurate results are obtained. The company QSC, in the ABX Comparator user manual, recommended a minimum of ten listening trials in each round of tests, as this allows a 95% level of confidence in results. A 95% confidence level means the results are statistically significant.[1] QSC also recommended that no more than 25 trials be performed, as listener fatigue can set in and skew test results.
Click to expand...

Unfortunately they don't state a number of listeners, but I guess it's rather small, otherwise other arguments could take place, like sitting out of the sweet spot, being distracted by the beautifull legs of the lady sitting elsewhere...

However the AES-reference should contain that information.

Have fun, Hannes
 
#8,802
Jakob2 said:


The idea is appealing, but enlights a general difficulty in developing a control that is sufficient for a difference with unknwon audibility.

So, you as an experimentator has to find out which way to get the most sensitive listeners. That means to observe the learning curve and test on different sensitivity levels.
And of course include a negative control as well.

Unfortunately there is no easy answer for all the possible questions; and, not to forget, after all there is the golden rule of testing: ´the more controlled a test is, the less the practicability of results´ (maybe the translation does not catch all the meaning).

So it might be, that the best test method is to hide from the listeners that they are participating in a somewhat ´official´ test.
We talked about that earlier, if you present just two different units, (same look, but different circuits inside), and ask just for some help in the decision which one to prefer, then listeners are doing what they are doing normally to compare two units.

In the end you´ll see, if there is significant difference.
I tried that method once and was quite happy with the results (of course you have to randomly change the presentation order, marking and you have make sure that the units aren´t different for a look inside and so on).
Apart from the mentioned difficulties, it has several advantages, because it let the listeners do what they normally would do; they choose the music, they choose how long to listen to each sample, they choose the level, and so on. No need for any level adjustment as random errors will be averaged out.
Click to expand...


Ok, I get that. I'm not sure though if we want to select the sensitive listeners. Would we not prefer 'representative' listeners, the middle of the bell curve population? Only then can you make statements about what 'most people' will or will not hear

Jan Didden
 
#8,803
We'll be back after the commercial break for our sponsor.
 

Attachments

  • bt-in-your-eye-two.jpg
    bt-in-your-eye-two.jpg
    50.9 KB · Views: 705
#8,804
janneman said:



Ok, I get that. I'm not sure though if we want to select the sensitive listeners. Would we not prefer 'representative' listeners, the middle of the bell curve population? Only then can you make statements about what 'most people' will or will not hear

Jan Didden
Click to expand...

That´s one of the important questions indeed.
Normally before any action there should be something called the ´operationalisation´ , that means the experimentator has to think about what hypothesis should be tested and what test procedure (including all factors) will be the best fit for this purpose.

So, if you would like to know if the average listener will be able to hear something, then you´ll choose your listening panel randomly.
But in our discussions the question would be normally more like `is there at least one person that could hear a difference in this case? `
 
#8,805
h_a said:


I'm sorry to say so, but a lot of people do not know statistics and show utter ignorance.

To every correct statistic an error estimate must be given to be valid. The smaller the sample the larger the (possible) error. On the contrary, if the sample is sufficiently large, the error becomes - especially in non-border cases - insignificantly small.

The same for double blind testing: you need a rather large number of tests to be able to make a statistically significant statement - that means that one can separate the result well from plain guessing.

Of course that's a lot work and if there's no money behing it, nobody does it.

Without checking the error, I would think that one would need to repeat double blind testing at least 10 times to be on the safe side (same circumstances, same equipment).

Have fun, Hannes
Click to expand...

Hey!
someone gets it 🙂
 
#8,806
janneman said:



Ok, I get that. I'm not sure though if we want to select the sensitive listeners. Would we not prefer 'representative' listeners, the middle of the bell curve population? Only then can you make statements about what 'most people' will or will not hear

Jan Didden
Click to expand...
The whole point of the thread is a discussion of the edge of the bell curve, not the middle.
Were it about the middle, then you'd be right about who hears what.
But the thread is about the extreme edge.

btw, about 400 listeners for 5%
 
#8,813
myhrrhleine said:

The whole point of the thread is a discussion of the edge of the bell curve, not the middle.
Were it about the middle, then you'd be right about who hears what.
But the thread is about the extreme edge.

btw, about 400 listeners for 5%
Click to expand...


OK, let's be clear then. We are talking about amps that only a very few, with proper training and experience, on the edge of the bell, can hear a difference with with an otherwise good amp. Right?

Jan Didden
 
Status
Not open for further replies.
Top Bottom