The rest of the world calls it a "triangle test." See my writeup about it in Linear Audio Volume 1.
Hi, yes, all good.
I find a very useful test regime is to have 3 files as you suggest above, two are the same (A1, A2) and the third (B1) is the different one, and for training purposes consecutively named A,B,C.
Now, for training purposes, this naming scheme is known to the listener/test subject, and the subject can flip between A,B or C, in in any sequence...ie ABC, ACB, BAC, BCA, CAB, CBA,.... ABA, ACA, BAB, BCB, CAC, CBC..... etc.
So, the purpose of the training is for the subject to know and learn/confirm the sounds of A1, A2 & B (A,B,C).
Once the subject is comfortable with differentiating the two sounds reliably,
the test is then to randomize the naming of the same three files as you suggest, and then for the subject sort them into A1, A2, B1 order.
This will give your statisticians resultant, but better/more useful because of the training, particularly so when small differences are being resolved.
The next level is to run four files (A1, A2, B1, B2), and to differentiate them correctly.
So, an online/download test requires two sets (known order/random order) of three files, and/or two sets (known order/random order) of four files.
That'll sort the wheat from the chaff.
Dan.
I apologize for this. The files are not broken, but were created by older SW and this probably results in old-fashioned header info. The files work on PC and PC sound cards but probably do not work on players that load data from USB flash.
When I loaded the original files into another SW and then saved, they started to work on any device. The original files have one advantage - you cannot put them into audio diffmaker and play with them 😉.
Please feel free to use mohammed li's files, I have checked them and their data content is OK.
Here is a simple logic: If it is sooo basic, the chance that PMA doesn't understand is sooo small.
I have mentioned to you that before deciding on the process, you have to set the objective first. Many here has (I believe wrongly) assumed that they know what PMA is trying to achieve. Sorry 😀
Jay, I do not understand your comments. Please wait until the test conditions are disclosed, you will see complete technical disclosure.
All I am interested in is to know more about human hearing abilities. I am not interested in comments like "huge difference on 1" speakers" etc., for the reason that it is a complete nonsense or it indicates some fatal error during data playback.
All I am interested in is to know more about human hearing abilities. I am not interested in comments like "huge difference on 1" speakers" etc., for the reason that it is a complete nonsense or it indicates some fatal error during data playback.
Last edited:
And how can you prove that it is a complete nonsense, or there was fatal error during data playback?? Will you follow kgrlee suggestion?
On the other hand, I might be able to explain later why these two "words" (which are data/information) are critically important: (1) HUGE difference (2) 1" SPEAKER.
I cannot follow krglee's suggestion immediately, for the reason that the test is running since yesterday and the votes are collected. I have already wrote this and I have already wrote I would start similar test with 3 files after the current one is completed.
I suggest for you to use foobar + ABX component and later you can tell us how successful you were in an ABX double blind test that is easily performed with foobar.
This current test is "as is", take it or leave.
I suggest for you to use foobar + ABX component and later you can tell us how successful you were in an ABX double blind test that is easily performed with foobar.
This current test is "as is", take it or leave.
Jan was unfortunately simply wrong in his statement.
A lot of procedures in sensory testing are based on statistical assumptions/methods and are quite often counterintuitive (at least at the beginning).
@ PMA,
what kgrlee proposes is a variation of the ITU-R BS.1116 method (which describes a ABC/HR protocol).
We had numerous discussion in various thread about listening tests and what should be the first step is to establish a hypothesis that will be tested.
Difference? Preference? Ranking?
Only descriptive statistics for the tested group or possibility to interfere to a broader population?
After that choose a test protocol.
The abc requires a lower number of trials for the same significance level, but if you are doing a online poll, you neither know about the number of participants nor are you restricted by limited resources.
And maybe you are more interested in keeping not only the alpha error low but the beta error as well.
The more complicated or unusual the test protocol is, the more time will the participants need to get used to it.
If the task is ranking or description you have to deliver the scale / hedonic scale.
For a starter i´d recommend reading the ITU-R BS.1116:
http://www.itu.int/dms_pubrec/itu-r/rec/bs/R-REC-BS.1116-1-199710-I!!PDF-E.pdf
Last edited:
Sorry, i have to make several corrections to this paragraph:
We had numerous discussions in various threads about listening tests; to establish a hypothesis that will be tested should be the first step.
Difference? Preference? Ranking? Directionality?
Only descriptive statistics for the tested group or possibility to infere to a broader population?
We had numerous discussions in various threads about listening tests; to establish a hypothesis that will be tested should be the first step.
Difference? Preference? Ranking? Directionality?
Only descriptive statistics for the tested group or possibility to infere to a broader population?
Last edited:
I have received some partial results by now, and they are quite interesting. So I do not think this test is useless.
One of the MOST important principles of a properly conducted DBLT is that the listeners are NEVER given info on what the test is about.
They should NOT be able to "figure it out". They are only required and should rely SOLELY on their auditory discrimination.
If you don't like the test, that's OK.
But please don't sabotage the test for other people by posting your rankings and comments identifying particular presentations. This is a HUGE psychoacoustic influence.
And if its not obvious, properly conducted DBLTs are ALL about psycho-acoustics and test EXACTLY what should be important to us.
_______________
Jakob2, thanks for the link to ITU-R BS.1116. IMHO, this is a variation of my ABC test cos I was using this in the 80's for serious work ... not the other way round 🙂
Jakob2, that first principle, that the listener is NEVER told what he is listening to, is so important that though ..
"numerous discussion in various thread about listening tests and what should be the first step is to establish a hypothesis ... that choose a test protocol"
.. are vital, the listeners are NEVER part of this discussion.
From the listener's viewpoint, my protocol is the simplest possible.
They are asked to rank their preference for each of the 3 'presentations' on a scale of 10 and add any comments they wish.
That's ALL they are told.
This very simple protocol evolved over nearly 2 decades of DBLT and has proved to be the most effective and least stressful both for true golden pinnae and the man/woman in the street.
There are other complications depending on what is being assessed but this is sufficient for assessing OPAs.
____________
The main remaining objection, but I don't know how to get around this in a internet poll of WAV files, is that I have the listeners choose their own music/CDs. This gets around those who find the music badly recorded bla bla. Having the listener comfortable with the test and finding at least one 'presentation' sounds good is important.
The stress of assessing sound quality on stuff they are unfamiliar with and don't, like is a major fault of many (most?) public blind tests.
____________________
Max, a much better use of your 'training' sessions is to just repeat the ABC test several times using the same protocols ie still tell them NOTHING. When doing serious work of this sort, you are always trying to get useful data out of limited trials that statisticians will be throwing their hands up in horror.
Don't forget one of the most important results you are looking for in your test is that most people (or even true golden pinnae) may NOT be able to tell the difference reliably on music.
_____________________
Anyone know if Foobar ABX can be used to carry out ABC tests to BS.1116 ?
Last edited:
I am pretty sure he has not "figured it out". It is impossible for him/anyone to figure out what was tested.
Strongly disagree. If you're trying to determine whether a particular listener (or, in this case, a particular subset of the listener universe) can hear the difference between two stimuli, it absolutely doesn't matter if the listener knows that (for example) the comparison is between gold wire and silver wire as long as there is double blind control to ensure no non-auditory cues. Same thing for A/B preference testing- if there's just two stimuli, A and B, it doesn't matter if people know that one is gold and one is silver, it only matters that they not know that A is gold and B is silver. Or in a triangle test, that there's gold and silver, but no knowledge of which is which.
The main (but not only) issue with the single pair A/B stimulus is that factors like presentation order are not controlled. To get a valid result, there has to be multiple repetitions, with gold and silver being randomized between A and B. And of course, there's the group dynamics issue, even when the "group" is communicating via email, PM, or phone rather than in person or on the forum. That's close to impossible to control in a gang bang like this.
OK guys, I suggest we shall be collecting votes until Sunday night. I will put a question on Sunday if there is anyone remaining who would intend to vote. Then, after we close the poll, I would ask you to describe your listening impressions and after that you will get full technical disclosure.
- Status
- Not open for further replies.