Yeah it's hard to call the ABX protocol the Boogeyman here in comparison to so many other confounding factors. One could use the files to do a different method but would require building some scripts to at least duplicate and rename files to keep it blinded. Also I would contend that ultra short switching is too contrived a test to have applicability beyond getting really good at detecting differences when samples are switched quickly. (Much like brain games make you good at brain games and nothing else)
Some random notes on what I do and associated thoughts:
Use Reaper here. But, anything with certain functionality will do. Try hard to use ASIO drivers if on Windows.
I set it up so I can try looping difference sections of the audio to find one or more segments for testing.
Reaper is set up by me for one-click switching between two tracks, either sighted or blind. For searching for sound segments and for initial training I used sighted, at least in difficult cases.
Once I figure out what there is to listen for, I switch to blind for more training or for testing. Blind testing merely involves randomizing mouse clicks so which track is playing is unknown. It's possible to have some one else randomize mouse clicks and/or turn off the monitor if desired. It is important to be able to check how you are doing whenever you want during training until proficient.
Once the mouse has been randomized, then I count clicks to know if I am listening to A or B, although still not knowing which one is which file due to the initial mouse click randomization.
Once I can nail it every time, only then would I make a claim here.
Volume level is the maximum that results in no or minimal ear fatigue, maybe 70dB SPL plus or minus. If more volume is needed, sessions may need to be shorter to recover. If very loud, maybe only one trial at a time is possible without loosing sensitivity.
One can remember that one heard a difference without remembering exactly what that difference is well enough to pass another trial. Curiously, similar thing happens with pain in humans. One can remember one that one was in too much pain to walk to the bathroom, but cannot re-experience the exact pain sensation from memory.
As I see it the main problems with ABX are probably that looping isn't allowed, and that training is dictated by what ABX will let you do, not what whatever may work best for you.
Seems to me looping can be valid in some cases depending on what one would like to know from the experiment.
IIRC foobar ABX does allow comparing segments of audio but not looping. However, adjusting segment boundaries is highly inconvenient and detracts from getting and staying focused on listening. Reaper OTOH makes it super easy.
EDIT: Manual looping in foobar ABX by repeated clicking is sort of possible, but it detracts from staying focused on listening IMHO. It interferes with the most sensitive tests, at least I think so.
Use Reaper here. But, anything with certain functionality will do. Try hard to use ASIO drivers if on Windows.
I set it up so I can try looping difference sections of the audio to find one or more segments for testing.
Reaper is set up by me for one-click switching between two tracks, either sighted or blind. For searching for sound segments and for initial training I used sighted, at least in difficult cases.
Once I figure out what there is to listen for, I switch to blind for more training or for testing. Blind testing merely involves randomizing mouse clicks so which track is playing is unknown. It's possible to have some one else randomize mouse clicks and/or turn off the monitor if desired. It is important to be able to check how you are doing whenever you want during training until proficient.
Once the mouse has been randomized, then I count clicks to know if I am listening to A or B, although still not knowing which one is which file due to the initial mouse click randomization.
Once I can nail it every time, only then would I make a claim here.
Volume level is the maximum that results in no or minimal ear fatigue, maybe 70dB SPL plus or minus. If more volume is needed, sessions may need to be shorter to recover. If very loud, maybe only one trial at a time is possible without loosing sensitivity.
One can remember that one heard a difference without remembering exactly what that difference is well enough to pass another trial. Curiously, similar thing happens with pain in humans. One can remember one that one was in too much pain to walk to the bathroom, but cannot re-experience the exact pain sensation from memory.
As I see it the main problems with ABX are probably that looping isn't allowed, and that training is dictated by what ABX will let you do, not what whatever may work best for you.
Seems to me looping can be valid in some cases depending on what one would like to know from the experiment.
IIRC foobar ABX does allow comparing segments of audio but not looping. However, adjusting segment boundaries is highly inconvenient and detracts from getting and staying focused on listening. Reaper OTOH makes it super easy.
EDIT: Manual looping in foobar ABX by repeated clicking is sort of possible, but it detracts from staying focused on listening IMHO. It interferes with the most sensitive tests, at least I think so.
Last edited:
I might disagree a little. In the Toccata test, I found the more detailed files more emotionally pleasing, and I found the correlation with small differences very interesting. The objective differences were small, but the pleasure was greater out of proportion. But to pass ABX or any objective test, I can't rely on my emotions, doesn't work.
I would think as an equipment designer I would probably want to go with design choices that sound better even if I have to fast-switch to help nail them down. The result of such choices might result in a better sounding and more enjoyable product, at least to some very sensitive or otherwise picky customers. Helps to measure too, obviously.
EDIT: Don't know about brain games, but video games have found some application in military use.
Last edited:
Problem is Mark, that the marketing bumph is more likely to make it enjoyable to the average picky customer, you know the sort who extols the virtues of a new product by quoting the user manual and confirming he can hear everything they said he could. (Note I do not count you in this group).
Have we confirmed whether you thought the less distorted sounded better yet? This is always the risk with subjective tuning.
Have we confirmed whether you thought the less distorted sounded better yet? This is always the risk with subjective tuning.
Haha, good question 😀. I would leave it open for some time yet. However, from the previous files under test and another tests performed before I am afraid you might deduce the answer.
Thanks Mark. One problem I've always had with Foobar ABX, and some similar protocols, is they have no real time feedback. I don't know until the end of the test how I've done. A test that rewards or punishes me immediately would make the learning of differences much faster and more reliable - I think.
Whether that's fair or not, I don' know - but it would go a long way toward learning to hear subtle differences.
Whether that's fair or not, I don' know - but it would go a long way toward learning to hear subtle differences.
In the older versions of the ABX utility of foobar, you did know the result after every single trial. Later, foobar have changed the SW to hide intermediate results from the tester. I think they are following some AES standards in DBT methodology this way.
I still have the older version on my XP computer. But it is unable to generate digital signature for remote verification.
I have another objection to the ABX utility. It generates audible click when switching between A-B, X-Y etc. It should be seamless, maybe with defined fade-out, fade-in.
It'd be useful insofar as training*, but when it comes down to the *actual* test, you want no feedback along the way. Zero, nein, nada.
*training, at least as far as I've read/learned, is actually really important for the reasons you define, both to feel comfortable with the test protocol as to semi-unblinded let you learn the differences in sound. This should also be done completely unblinded. Only the end ABX "counts" but all the build up matters to give you the best chance of having a positive result (rather than unfamiliarity causing the null). Unfortunately, this all requires a good deal of time and concentration. I'd also like to highlight if one looks through older posts, I have changed my view here, as faced with new-to-me info.
Mark -- my point was more to specificity, specificity, specificity. The brain games I was citing are things like Lumosity, et al, that market that their curriculum helps with general brain function. When you actually read their methods sections, you'll notice not only a low N, but also that the pre- and post-training tests are also brain games. Studies that look at more general cognitive ability (in elderly where staving off dementia is so absolutely important) don't show any improvement from brain games. Video game simulations, whether combat or flight or whatever, if well done, will certainly help with the task at hand. But not much outside of the task at hand.
The other question I'd have before you try doing the super-short switching is to see your preference over a 12-15 second test. Because once you pull the genie out of the bottle, other cognitive processes are going to have a strong say. (not to say that you won't be consistent between the super short and the more "standard" 12-15 second switch time.)
Daniel - Point taken about brain games.
Regarding switch time, it's loop time that seems to matter. I don't mind looping 12 seconds before switching, but loop length seems like it might depend on how much there is to remember, or if its something simple or complex.
Something simple might be to compare the sound of a cymbal when it is struck relative to at some point in its decay. Two data points, and maybe its kind of easy to hear it to begin with.
Complex would be more this Toccata test where differences are many, small and complex.
Let me try this: Most people can hold the equivalent information of 6 digits in working memory without forgetting. If more information can be encoded in the 6 digits, that's fine, so long the digits are limited to 6. Some people can do more, and more digits happen to be associated with higher IQ. Its called "working memory" capacity.
With Toccata it happens my digits get full of details pretty quick, they come by fast and detailed. It takes effort to hold it in working memory. Fiddling with ABX buttons to do my own looping is killing me. I need fewer digits or more IQ, something like that. Shorter loops and less distraction.
EDIT: Pavel, if testing with noise perhaps System 1 processes take over more of the detection workload. Working memory is a limitation of conscious awareness.
Regarding switch time, it's loop time that seems to matter. I don't mind looping 12 seconds before switching, but loop length seems like it might depend on how much there is to remember, or if its something simple or complex.
Something simple might be to compare the sound of a cymbal when it is struck relative to at some point in its decay. Two data points, and maybe its kind of easy to hear it to begin with.
Complex would be more this Toccata test where differences are many, small and complex.
Let me try this: Most people can hold the equivalent information of 6 digits in working memory without forgetting. If more information can be encoded in the 6 digits, that's fine, so long the digits are limited to 6. Some people can do more, and more digits happen to be associated with higher IQ. Its called "working memory" capacity.
With Toccata it happens my digits get full of details pretty quick, they come by fast and detailed. It takes effort to hold it in working memory. Fiddling with ABX buttons to do my own looping is killing me. I need fewer digits or more IQ, something like that. Shorter loops and less distraction.
EDIT: Pavel, if testing with noise perhaps System 1 processes take over more of the detection workload. Working memory is a limitation of conscious awareness.
Last edited:
This thread has been from the outset about high order harmonics created by crossover distortion and whether they are audible, and if they are, how do you perceive it.
I've listened intently to the Bach via my PC and headphones and am completely unable to differentiate the two files reliably. Even when I think I can, I am not really basing that difference on something detracting from the music, I am listening for audible cues in the background noise or trying to hear some 'false' tone somewhere... and its not working 🙁 They are to close to call.
I've listened intently to the Bach via my PC and headphones and am completely unable to differentiate the two files reliably. Even when I think I can, I am not really basing that difference on something detracting from the music, I am listening for audible cues in the background noise or trying to hear some 'false' tone somewhere... and its not working 🙁 They are to close to call.
Exactly. I would only add that the thread was also about the actual level of the high order harmonics. I am sure that if they were of 40dB higher, they would be definitely audible. If they were of 20dB higher, then I am not sure.
Karl, would it be possible for you to take one of the Toccata files (makes no difference which one) and pass it through your Texan simulation? This might be interesting as well. I do not have any real amplifier with distortion high enough.
Sure. But the experiments have been to try to audibly differentiate two files, whatever that means for the thread topic. I did say I got more emotional enjoyment out of the more detailed version of Toccata, even though I don't know if the so-called detail is an artifact of distortion.
I will see what I can come up with and also try and run a stereo simulation. It takes time though, lots and lots of CPU cycles needed 🙂
Edit... which of course I don't need for the Bach 🙂 I also have a feeling that LT will only accept 16/44.1 files for some reason. I'll look into that.
Had a chance today to resurrect the old Rotel amplifier and have actually been having a play with that and making a couple of test recordings, one at zero bias and the other at normal bias. The results are interesting and of course the differences here are purely down to crossover distortion.
At some point if you are interested I can share the result but it needs documenting accurately so that the method and results are clear.
- Status
- Not open for further replies.