SY said:
Why so strict? Here are the probabilities if someone is guessing (with a 50% chance of success):
5 or more 0.62
6 or more 0.38
7 or more 0.17
8 or more 0.06
9 or more 0.01
10 0.001
Also, the chance of guessing 8 or 9 right on one trial and then seven or better on a second trial is only 0.01.
Rob: That's why I gave myself a one or two guess margin for adjustment- I didn't have my 95% confidence interval table with me. The problem also comes in from the nature of a test that will circulate among multiple people- even at a 95% confidence interval, there's a 50:50 chance that someone will score perfectly by chance if there are twenty test subjects.
SY said:
Ah yes, I forgot about old Bonferroni.
So, if there are five subjects, there's about a 22% chance of at least one of them getting eight or more right just by guessing. There's only a 5% chance of at least one guessing nine or more, and a 2% chance of at least two of them guessing eight or more.
Hmm, I love it
SY, good idea.
I think using a cheap bipolar electrolytic shouldn't be a problem since you can buy 20 of them for a couple bucks or so, and find an exact match of "raw capacitance, warts and all ignored" to the premium part. Hey one could even ask what sounded *better* (electrolytics are bad at passing RF and who knows, may even be beneficial in a blocking cap application)
...
Buffer, violates the KISS principle
, besides the property under test is the difference between parts, not the difference between "LP box out" and "LP box in" - so no problem.
Why not make this an ongoing merry go round "hear the crap" game? Procedure: each person takes the test only once and sends the device on to the next of a list of candidates. And so forth. Mailing costs only once per candidate. Given the simplicity of the test each listener could even do 25 "shots". The results would always get sent incognito to the administrator / key holder. The testers would keep their score sheets secret of course. The administrator would get a growing database.
Advantage - with time we would get a largish number of people who can or cannot hear the difference. This tells us what percentage of people in the audiophile population can hear it (if at least one candidate is successful) and reduces possible type II error (false negative: small but audible differences missed due to insufficient resolution or number of test runs). It also averages out the various system resolution differences, cables, listener age and condition etc
SY, good idea.
I think using a cheap bipolar electrolytic shouldn't be a problem since you can buy 20 of them for a couple bucks or so, and find an exact match of "raw capacitance, warts and all ignored" to the premium part. Hey one could even ask what sounded *better* (electrolytics are bad at passing RF and who knows, may even be beneficial in a blocking cap application)
... Buffer, violates the KISS principle
, besides the property under test is the difference between parts, not the difference between "LP box out" and "LP box in" - so no problem.Why not make this an ongoing merry go round "hear the crap" game? Procedure: each person takes the test only once and sends the device on to the next of a list of candidates. And so forth. Mailing costs only once per candidate. Given the simplicity of the test each listener could even do 25 "shots". The results would always get sent incognito to the administrator / key holder. The testers would keep their score sheets secret of course. The administrator would get a growing database.
Advantage - with time we would get a largish number of people who can or cannot hear the difference. This tells us what percentage of people in the audiophile population can hear it (if at least one candidate is successful) and reduces possible type II error (false negative: small but audible differences missed due to insufficient resolution or number of test runs). It also averages out the various system resolution differences, cables, listener age and condition
etc
Round robin
Eric: That's sort of the way I want to do it, but I'll have to rerandomize it each time- UNLESS everyone wants to wait until everyone is done before scores are announced. You sort of implied this, too, so we're probably on the same wavelength. My only restriction will be that people who have openly announced their skepticism shouldn't participate, for the reasons that jcarr so clearly stated.
Eric: That's sort of the way I want to do it, but I'll have to rerandomize it each time- UNLESS everyone wants to wait until everyone is done before scores are announced. You sort of implied this, too, so we're probably on the same wavelength. My only restriction will be that people who have openly announced their skepticism shouldn't participate, for the reasons that jcarr so clearly stated.
I would agree, but unless I'm being dim, I fail to see how you can differentiate between cause and effect in each case.
The simple fact is that in an experiment to test for something, that something has to be the primary variable, with everything else being held as constant as possible.
It may be the case that the switch is not significant, but I fail to see how one can guarantee that and it has the potential to change characteristics with time and environment. They are also potentially microphonic.
I'm not trying to be pedantic, but if you leave doubts about variables, we will forever have the post-test discussions about the switch
Andy.
Re: Round robin
SY,
sounds like you refer to my post (MBK, #30).
You don't have to post detailed scores and the key when you publish partial results as you go along - no one will be able to reverse engineer the positions without having the key or at least the score sheets. The aggregate score (which is what we're interested in) won't reveal the positions.
Andy, Re: 'the switch as a variable': I have an open mind to differences between switches, cables, resistors et al., and yes in theory different positions of one single switch could sound different (arrrgh!!!) . However. I don't think any real world scientist in any real world experiment would go that far in his suspicions .... I mean ... maybe a single contact of any kind will have different oxidation conditions depending on time of day. Quite possible.
Arguments that the proposed setup is good enough:
Exhibit A: Using a switch makes contact irregularities as a significant factor far less likely than manually plugging and unplugging connectors, and unlike those a switch is not exposed to dirt, has less wear and tear etc. AND everyone is free to rotate the switch a dozen times pror to use to clean the contacts.
Exhibit B: If the switch and its wiring will wreck the cap and R comparison because their effects exceed those of C and R quality, quod erat demonstrandum, lez worry more about switches and layout than about caps and R's.
Exhibit C: on a 22 pole switch there is enough space for random distribution of the networks, which will randomize cabling and contact quality influences.
Just my opinion of course.
SY said:
SY,
sounds like you refer to my post (MBK, #30).
You don't have to post detailed scores and the key when you publish partial results as you go along - no one will be able to reverse engineer the positions without having the key or at least the score sheets. The aggregate score (which is what we're interested in) won't reveal the positions.
Andy, Re: 'the switch as a variable': I have an open mind to differences between switches, cables, resistors et al., and yes in theory different positions of one single switch could sound different (arrrgh!!!) . However. I don't think any real world scientist in any real world experiment would go that far in his suspicions .... I mean ... maybe a single contact of any kind will have different oxidation conditions depending on time of day. Quite possible.
Arguments that the proposed setup is good enough:
Exhibit A: Using a switch makes contact irregularities as a significant factor far less likely than manually plugging and unplugging connectors, and unlike those a switch is not exposed to dirt, has less wear and tear etc. AND everyone is free to rotate the switch a dozen times pror to use to clean the contacts.
Exhibit B: If the switch and its wiring will wreck the cap and R comparison because their effects exceed those of C and R quality, quod erat demonstrandum, lez worry more about switches and layout than about caps and R's.
Exhibit C: on a 22 pole switch there is enough space for random distribution of the networks, which will randomize cabling and contact quality influences.
Just my opinion of course.
That will happen even if I control for that. "Well, the moon's effect on the power grid wasn't accounted for." "No, no, it was random phase fluctuations in the local gravitational field." "You moron, you didn't account for the EMF from nearby power lines."
You can't convince a True Believer and I'm not intending to.
Switch contacts
If there are factors which are impossible to control for in designing the test box (like differences between switch contacts and phono jacks), we could control for them statistically.
We'd need to do a bunch of trials for each of several different test box configurations. Then we could check for a statistical relationship between the answers people are giving and both the resistor/capacitor combination and each particular switch contact/wire/phono jack combination.
If there are factors which are impossible to control for in designing the test box (like differences between switch contacts and phono jacks), we could control for them statistically.
We'd need to do a bunch of trials for each of several different test box configurations. Then we could check for a statistical relationship between the answers people are giving and both the resistor/capacitor combination and each particular switch contact/wire/phono jack combination.
Peter, I'm going to refrain from listing ways I know that one could cheat; why tempt fate? I'm just going to ask everyone involved to cross their hearts and promise to not measure, just listen.
Clearly I'm going to try to trim things tight enough to make a simple f3 measurement a tough thing to cheat with. And I'll take some elementary precautions against unsealing the box.
Clearly I'm going to try to trim things tight enough to make a simple f3 measurement a tough thing to cheat with. And I'll take some elementary precautions against unsealing the box.
- Status
- This old topic is closed. If you want to reopen this topic, contact a moderator using the "Report Post" button.