@Jakob: That doesn't make sense. Can you explain how you can still be double blind with only two people?
How clean does the signal source need to be to perform these measurements? Are you using a source better than -160 dBc THD or is that not necessary due to the cancellation/null nature of the test circuit? Does the cancellation/null nature of the test circuit cancel/null the input THD also and then the output THD is dominated by the non-linearity of the resistors?
The cleanest sources I have are the PCM1794 and ES9038Q2M DACs themselves with OPA1612A in the I/V and filter. So something in the range of -130 dB and but beyond my AK5394A ADC.
On the DAC side I have a very low noise 60 dB LNA (3 parallel 2SB737 and OPA228) followed by the AK5394A.
How clean does my source need to be to perform such tests?
Jan.didden offered an assertion but no explanation, therefore I tried a brief explanation why the assertion wasn't correct. 🙂
Second point; exactly, as long as the "switching" guy doesn't know, which resistor is located at which switch position it can be a "double blind" test.
@billshurv,
listener does the solder work, but doesn't see the switching, while the switching guy did not see the solder work, applies some randomly choosen markings and switches following a random series. "Deblinding" at the end.
But why? 😕
Basically, if the switcher does not know about the specific resistor "behind" the switch position and the listener does not know about the switch position used, it is by definition "double blind".
Of course, kind of beer talk, as we do know next to nothing about what actually happened, but we're just talkiing about if it still can be "double blind".
That said, could be that it was an approach to qualitative testing, which would have been not a bad idea....
Basically, if the switcher does not know about the specific resistor "behind" the switch position and the listener does not know about the switch position used, it is by definition "double blind".
Of course, kind of beer talk, as we do know next to nothing about what actually happened, but we're just talkiing about if it still can be "double blind".
That said, could be that it was an approach to qualitative testing, which would have been not a bad idea....
But if the listener knows which resistors are there he is surely by definition not blind even if he doesn't know which of them is selected?
It could be blind or not depending on the experimental question being asked. For example, is the test subject even supposed to know they are being tested? If they are allowed to know they are participating in a test, are they allowed to know anything about it? That they will receive an experimental drug for a medical condition they have or else a placebo?
It has been done. More commonly test subjects are told they are going to be participating in an experiment, but the purpose (and possibly other details) of the experiment may or may not be revealed to them. In some cases they are told the experiment is for something else in order to distract them from figuring out the real purpose. In that case the test subjects are typically debriefed immediately after the experiment and the real purpose revealed before they are allowed to leave. That is usually considered sufficiently ethical, but it may depend on an independent ethics review panel. The panel has to consider what harm could be done to the test subjects by temporarily misleading them. Also, any need for keeping the real test purpose concealed should be considered -- it is really essential for the question being investigated or not.
Last edited:
If being asked to identify the resistor behind the positions in each trial, the listener surely would still be blinded.
Same, if a more qualitative description was asked in each trial according to a predefined vocabular to check for consistency.
Isn´t not knowing about the selected DUT the definition of being "blind"? 😕
For true blind, the test subject should not even know if there is a switching action done between trials or not.
Jan
Jan
@Jan, maybe or maybe not. It depends on the experiment. For example, I had a set of files the with same audio recorded through different unity gain opamp buffers (files prepared and level matched by our own PMA). I did not know which opamps were used. I tried to sort the files in order of distortion. I knew when I was changing the sort order, but that was necessary in order to be able to sort them at all. I still didn't have any idea of which opamp was which. Of course what I was doing was not a typical ABX protocol experiment.
Last edited:
But if you knew that at each trial there certainly was a different opamp in the system, that makes it not truly blind. You were sure that there could be a difference.
Jan
Jan
Changing the subject a bit, I think we may be getting too picky about the definition of blind. I think one needs to look at the experimental question being asked. Suppose we allow a listener to train by listening to several resistors. We tell the test subject that after a period of training they will be asked to identify the resistors blind. Okay, they know the resistors they trained with. The experiment allowed for that condition as a part of the training phase, so nothing wrong with it. It would just mean that experimental results mostly applies to cases where a set of resistors is used for training. Other experimental questions and other test conditions may give different results.
Thanks. I can see how being deceived about the true test could be useful.
A member in another thread said the 4IAX test is better than ABX (PDF) A Thurstonian model for the dual pair (4IAX) discrimination method
And you were sure there were differences between those two identical files...
@jan.didden,
it depends on the task in each trial; obviously, asking "same/different" doesn't make sense if the participants knows that something indeed is different, and surely it is not "blind".
But if asked which one brings more of "what ever" or asked to identify the two different DUTs is different and the same participant would be correctly "blinded".
Think of it like a food test example, the participants gets two lemonades (and knows the two are different) and is asked which one is sweeter (directional test).
@jazzman,
we have discussed a lot of these topics quite often in great detail in other threads already, so I don't know if we should restart it in this thread, but there is evidence that different protocols have impact on the performance of the participants and especially the ABX-protocol seems to be more difficult for (not sufficiently trained) people than others.
But beside that, the specific design of any sensory experiment depends on the hypothesis to be examined.
As Markw4 already mentioned, it might matter if participants know about the EUT, it might matter, if they know being under test, the test protocol itself matters and so much more.
it depends on the task in each trial; obviously, asking "same/different" doesn't make sense if the participants knows that something indeed is different, and surely it is not "blind".
But if asked which one brings more of "what ever" or asked to identify the two different DUTs is different and the same participant would be correctly "blinded".
Think of it like a food test example, the participants gets two lemonades (and knows the two are different) and is asked which one is sweeter (directional test).
@jazzman,
we have discussed a lot of these topics quite often in great detail in other threads already, so I don't know if we should restart it in this thread, but there is evidence that different protocols have impact on the performance of the participants and especially the ABX-protocol seems to be more difficult for (not sufficiently trained) people than others.
But beside that, the specific design of any sensory experiment depends on the hypothesis to be examined.
As Markw4 already mentioned, it might matter if participants know about the EUT, it might matter, if they know being under test, the test protocol itself matters and so much more.
Last edited:
- Status
- Not open for further replies.