How much weight does ABX carry?

[Wellfed]

What does it take to have the results recognized as conclusive?

Has this site ever organized or sanctioned a specific ABX test?

Do these ever satisfy the detractors?

Where does an ABX have to take place to be recognized as valid?

I am proposing to test Machina Dynamica Brilliant Pebbles room treatment devices in such a way that no one could reasonably dispute the results. I need input on how to setup such a test and have it recognized as definitive. TIA for any assistance and/or counsel.

[Mr Evil]

I can't answer your question, but I can tell you that no one experiment can prove anything beyond doubt... It needs to be repeated by multiple independent groups to be considered good evidence by scientific standards.

[janneman]

There is another issue you need to deal with, the test participants. Record for every participant IN ADVANE whether they:

- believe there will be no effect;
- believe there could be an effect, but or not sure there is one;
- believe there will be an effect.

You don't want only participants that think there is no effect, because if none is found, that will kill you. Probably you don't want ANY of those. If you have only participants that think there is an effect and none is found, that will give you a strong case.

Note: if no effect is found, it does not prove that the effect does not exist. If the effect is found, it proofs that the effect exists.

Jan Didden

[SY]

Record every detail about the test protocol.

Make sure that the test subjects believe that they hear a difference before blind protocols are "turned on."

Have an intelligent and qualified skeptic review the test setup and be present during the test.

Make sure that the test is DOUBLE blind to prevent the Clever Hans effect.

Make sure that your test is statistically valid to establish a 0.05 significance (i.e., sufficient trials and/or sufficient listeners). Don't make the mistake of mixing statistics, that is, setting things up for a group, then trying to interpret statistics for one participant.

Make sure you have controls. For example, if you're placing a jar of rocks on a speaker as "A", the place a similar weight of nonmagic rocks on the speaker as "B."

With that stuff in place and a positive result, you'll have something solid and publishable.

[Wellfed]

The thought occurred to me that perhaps one of the local colleges would have a contingent that would be interested in administering such a test. Is this likely? What department(s) should I contact?

[richie00boy]

I wouldn't have thought that a bunch of college kids or even any joe public crowd would be experienced/aware enough to perceive any possible differences.

[Wellfed]

It is quite possible that I am naive, but I believe they could.

[SY]

This would be a superb project for someone in the Psych department. They know how to do blind tests and interpret statistics. And you've got a large pool of sharp-eared music listeners at any college for the experimenter to draw from.

[Wellfed]

Thanks SY, the Psychology department is where I'll head.

Any suggestions from anyone on what to test for?

Good/Better
Base/Different
Musical/not

or specific effect

larger soundstage
greater resolution
greater dynamics

My main interest is "does it sound better", adding a subjective element to the test.

[SY]

Start simple. Difference/no difference. If you can't get past that (and that's where I'll lay my bet), better/worse is sort of meaningless.