No, that wasn't the test. Fremer just happened to be one person in the audience. Post Facto recasting of a test is cheating.
I can't make this link work.
Edit: Never mind. But was this the AES 1995 convention test?
So far, this is all I have to go by.
John
SY said:
It is not what I remember, it is what I make of the quote from jlsem about what Fremer said:
"Now, what amazed me about the result was that one of the amps was a vacuum tube amp that sounded way different from the otherS yet most test takers couldn't hear what was obvious to me."
So there should have been at least three amps.
Lets look at SY's example of 100 people doing 5 coin tosses each.
After the first toss, there's about 50 people with heads, 50 with tails. Those 50 with heads, after the next toss, will be roughly divided between 25 that again have heads. So now you have 25 people with 2 heads. By symmetry you also have 25 people with 2 tails.
Next toss, the 25 with 2 heads are now divided in say 12 that again have heads, so now you have 12 people with 3 heads. Two more tosses and you end up with 3 people with 5 tails and 3 with 5 heads.
So if you yourself want to get 5 tails, how often do you need to toss 5 times to be almost(!) sure to have 5 tails? Well, of course you could get lucky and get 5 tails in the first set of 5 tosses. But if you do 30 sets, you'r pretty sure (but not guaranteed) that you have a set of 5 tails.
Edit: I'm not 100% sure of this last bit. Anyone?
jd
After the first toss, there's about 50 people with heads, 50 with tails. Those 50 with heads, after the next toss, will be roughly divided between 25 that again have heads. So now you have 25 people with 2 heads. By symmetry you also have 25 people with 2 tails.
Next toss, the 25 with 2 heads are now divided in say 12 that again have heads, so now you have 12 people with 3 heads. Two more tosses and you end up with 3 people with 5 tails and 3 with 5 heads.
So if you yourself want to get 5 tails, how often do you need to toss 5 times to be almost(!) sure to have 5 tails? Well, of course you could get lucky and get 5 tails in the first set of 5 tosses. But if you do 30 sets, you'r pretty sure (but not guaranteed) that you have a set of 5 tails.
Edit: I'm not 100% sure of this last bit. Anyone?
jd
janneman said:
Janneman, flipping a coin give you statistically 😀 only two options to predict, changing that options to three or five change the odds of predicting it correctly five times in a row very, very, very slim.
Let's clear up some confusion here. The J. Gordon Holt article that Scott referenced has nothing to do with Michael Fremer and the 1995 AES Convention test (I can find no reference to Fremer at all). So let's just forget that. The archives on the AES website stop just short of the convention in question. Does anyone have a link to any information on the test in question without having to pay out the wazoo for on of the AES publications?
John
John
Andre Visser said:
Yes, but you never do a test only once. Because of your own statement, any one-off test is untrustworthe either way.
You do 20 or 50 tests with 20 or more people, you collect the results, and with 50 tests and 80% correct you can say that with 85% confidence that the two amps (or whatever) are audibly different. (I'm making up the numbers but they are in the ballpark).
But, and this is very important, by the same token there is a 15% chance that the amps were really NOT audibly different.
That's the thing with these tests: you try to run as many tests with as many people to up your confidence, but you never have certainty.
jd
Hi Andre
Maybe I have missed something, but I'm still waiting for a conclusion on this post.
Cheers
http://www.diyaudio.com/forums/showthread.php?postid=1834726#post1834726
Maybe I have missed something, but I'm still waiting for a conclusion on this post.
Cheers
http://www.diyaudio.com/forums/showthread.php?postid=1834726#post1834726
Andre Visser said:
I know but there seems to be a lot of disagreement about the conditions surrounding the other test. This one is fully documented and Fremmer freely admits "defeat". If a clever guy can take a "dime bag' of parts and make it equal some expensive tube amp why can't anyone admit that maybe enough off the shelf Belden wire could be made to sound indistinguishable from some $15,000 speaker cable.
stinius said:
That IS an interesting post Andre:
/quote
I've done a blind test once on a set of interconnects, the cable that I've used was a coaxial design that I did not use for quite a while because I prefer twisted pair screened cable that is directional. It took many sighted direction changes for me to hear a subtle difference. The outcome was that I've named the direction the wrong way round in 8 out of 9 tests, whatever that means.
/unquote
I can understand that you could hear differences, but how do you decide 'this is east-to-west', 'this is west-to-east'. How do you decide direction from what you hear?
jd
janneman said:
The thing with these conclusions is that it may be applicable to the average listener, not to all. Why don't they rather retest the people with 5/5 results to make sure it were not by chance but conveniently these results get rejected.
It's been 40 years since I had probability and statistics courses in college, but I don't think that's how it works. There is no guarantee of symmetry and besides such an argument doesn't really apply. In a group of 100 people, you would have to determine the motives of the subjects. Obviously such a group would include a large number of objectivists who whole-heartedly agree with the assumption in question and make no effort at discerning a difference (i.e, they don't care). Then there are people who think a difference exists but are bereft of the skills to determine correctly the difference. These two groups will produce the null results. Then there will be skeptics who posses the appropriate skills and come to be surprised to find that they can hear a difference and will score better than a random result. John Atkinson (who guessed 4/5) may fall into this group. And finally there are the persons who believe there exists said difference and have the ability to distinguish the difference and they score better than random probability would predict (maybe even 5/5 in the case of MF).
But there certainly exists no information as to the nature of the subjects under test, so all we have so far is Michael Fremer's word.
John
Sure, but many who claim to have better hearing or 'trained ears' either refuse to put that to the test under controlled conditions, or they blame the test when (it is typically stated) the differences were large before the test when they knew what they were listening to.Andre Visser said:
An average at least sets a baseline.
Andre Visser said:
That would be fudging the results, anything after that would be laughed away. That way I can prove anything. I'm sure you understand that 😉
Jan Didden
stinius said:
Hi Stinius, that was a very tiresome test that I haven't recovered from yet, I will try some day when I am cross with myself. 😀
jlsem said:
Indeed it is a different case than the listening tests. I just used SY's example to work it out further. You are also correct that there is no guarantee of symmetry, but with large numbers you get very close to certainty like 90% or 98%. So the 50-50 division will be pretty close, but when you go down to the last split of 6 people the split will probably (ahum) not be 3 heads-3 tails, but very likely 2-4 or 4-2. But the idea is that you would use as large numbers as possible to get your confidence up there toward 100%.
I don't agree to your 2nd argumentation. You assume that there is a difference and then you proceed to explain why some can and some cannot or resist to hear it. Maybe there isn't any, that is what the test should establish in the first place. If you already know that there is a difference, you can skip the whole test.
Edit: There is no 'Michael Fremer's word'. We know he scored 5/5. That doesn't tell us ANYTHING as it is a single sample. Just as it doesn't tell us anything, in isolation, that John Doe in the same test scored 0/5. MF says that he is sure of himself, but aren't we all 😉
jd
janneman said:
Jan, I've heard a small difference in the focus of the instruments in the soundstage, the 'direction' of the cable were taken from the direction the name were written.
My feeling about this test is that direction on that cable did not matter. I don't know whether the fact that it wasn't used for a long time have an influence though. 😀
Brett said:
Sure Brett but I'm interested in the 'ultimate' hi-fi system, not the average, spreading these beliefs as a rule make it sound like anything more than an average system is a waste of money and time.
- Status
- Not open for further replies.