I haven´t read all post, so please bear with me if it was already discussed.
From the first few posts of the thread and a couples of the last ones it seemed that listeners should provide a preferrence based ranking list or at least name the version preferred as ?best?. And to use the preference (if there is one) as indication of difference. Am i wrong?
I am wondering that some are using the ABX test, as it is a test for difference and the ranking task isn´t covered by it at all.
ABC/HR is indeed much better for this sort of task, if one wants to do it per a controlled blind test, but then it would be better to include a "straight wire" sort of reference to compare with.
A/B paired comparisons (forced choice preference) are possible too, but transitivity is not a given in case of multidimensional stimuli (aka music).
From the first few posts of the thread and a couples of the last ones it seemed that listeners should provide a preferrence based ranking list or at least name the version preferred as ?best?. And to use the preference (if there is one) as indication of difference. Am i wrong?
I am wondering that some are using the ABX test, as it is a test for difference and the ranking task isn´t covered by it at all.
ABC/HR is indeed much better for this sort of task, if one wants to do it per a controlled blind test, but then it would be better to include a "straight wire" sort of reference to compare with.
A/B paired comparisons (forced choice preference) are possible too, but transitivity is not a given in case of multidimensional stimuli (aka music).
Jakob, I think Pavel would be happy to receive any type of response from anyone willing to listen.
Detailed responses by PM messaging or email to PMA hopefully will provide him with the most useful information. Voting alone provides minimal information.
What I did was sort the files in order of how forward they sounded to me (or maybe it could be described as how bright they sound, but the effect is rather small).
Maybe I invented a new test besides ABX, I don't know. It's easier for me than ABX (at least, it is easier than the Foobar implementation of ABX), whether new or not. I described my method a few pages back in this post: http://www.diyaudio.com/forums/ever...6-24-listening-test-opamps-5.html#post5111943 I would add that Reaper has a 60-day free trial period and after than it only has a nag screen at startup to remind users they should buy a license, therefore I would say there is no financial reason not to give the method a try if there is any interest in doing so.
Detailed responses by PM messaging or email to PMA hopefully will provide him with the most useful information. Voting alone provides minimal information.
What I did was sort the files in order of how forward they sounded to me (or maybe it could be described as how bright they sound, but the effect is rather small).
Maybe I invented a new test besides ABX, I don't know. It's easier for me than ABX (at least, it is easier than the Foobar implementation of ABX), whether new or not. I described my method a few pages back in this post: http://www.diyaudio.com/forums/ever...6-24-listening-test-opamps-5.html#post5111943 I would add that Reaper has a 60-day free trial period and after than it only has a nag screen at startup to remind users they should buy a license, therefore I would say there is no financial reason not to give the method a try if there is any interest in doing so.
Last edited:
Definitely Mark, thank you for your comment. But it is getting better today
, I have already received opinion of 5 voters total, so I can start to put it into spreadsheet . Very interesting, though!!
I have reoptimized my soft and now the differences are as evident as before, with 20 mVrms. Off course, the same ranking.
I passed the test again (with speakers) after detecting improvement in the reproduction of a fantastic work that I have heard very much, confirming it.
Ahmad Jamal - Ahmad's Blues (1959) [FLAC]
-> https://www.amazon.com/Ahmads-Blues...e=avp_only_reviews&sortBy=recent&pageNumber=1
I passed the test again (with speakers) after detecting improvement in the reproduction of a fantastic work that I have heard very much, confirming it.
Ahmad Jamal - Ahmad's Blues (1959) [FLAC]
-> https://www.amazon.com/Ahmads-Blues...e=avp_only_reviews&sortBy=recent&pageNumber=1
Definitely Mark, thank you for your comment. But it is getting better today
Pavel, does that include the message 'I can not hear a difference'? I believe that is a valid vote that should also be used.
Jan
Jan, You make a fair point I think. If hearing differences was thought to be impossible, and in a listening experiment 100% of listeners had no problem hearing any differences, that might suggest a problem with the experiment. Maybe we should expect that most participants would likely not hear any differences.
Where I am not sure if we agree might be at to whether or not a listening impression report should be deemed to constitute a "vote" of some kind. For one example, I believe I heard differences in all of the files, but since I did not have a preference for any one of them, I did not intend to vote for one as preferred. If my comments are in some way to be interpreted as having voted, then I guess my vote should be tallied the same as someone who didn't hear any difference at all.
Where I am not sure if we agree might be at to whether or not a listening impression report should be deemed to constitute a "vote" of some kind. For one example, I believe I heard differences in all of the files, but since I did not have a preference for any one of them, I did not intend to vote for one as preferred. If my comments are in some way to be interpreted as having voted, then I guess my vote should be tallied the same as someone who didn't hear any difference at all.
Last edited:
Yes, but I am afraid it is impossible to edit the existing poll and to add a new question, at least I do not know how to do it.
Mark (and PMA), reflecting some more, it probably wouldn't be possible to include an 'I don't hear a difference' in the score. If you read some of the listening tests in the past they often do a 'dry run' first and then cull those participants that did not hear a difference, by their own account.
For the reason that there is no category for them in a preference listening test like this one. You could include it in a test where participants were asked if they could hear a difference, but not if you ask for prreferences.
In itself it is interesting to log those very honest participants, but you can't include them in the number score for preferences.
Jan
For the reason that there is no category for them in a preference listening test like this one. You could include it in a test where participants were asked if they could hear a difference, but not if you ask for prreferences.
In itself it is interesting to log those very honest participants, but you can't include them in the number score for preferences.
Jan
Jan, the option "can not hear a difference" was added at your suggestion
I know, but that was yesterday. I got smarter overnight
But seriously, I was wondering how to turn such a vote into a number that could be included in the other vote numbers. I don't think it can.
Still, very much looking forward to the results. It seems to be quite nicely blind this time.
Jan
- Status
- This old topic is closed. If you want to reopen this topic, contact a moderator using the "Report Post" button.