No gymnastics, just statement of the statistical/scientific realities of the ABX test.
dave
When people think something is audible, its usually due to one of two reasons:
(1) It was audible under the conditions and at the time it occurred.
(2) At the time of listening the person was focusing attention on one part of the total sound, and when listening to something else, attention was focused on some other aspect of the sound. It happens without the person's knowledge, System 1 processes in the brain make it happen that way. So, what they hear, that is to say, what makes it to conscious awareness is a different listening experience in both cases. They did hear a difference, but not for the reason we are interested in.
In the first case (1) if they can't tell a difference under test, there is probably a problem with the test, even if the test giver doesn't know what it is and doesn't care.
In the second case (2), under blind conditions the person has lost the cues that allowed unwitting focusing of attention on particular aspects of sound, so now the mental experience is that the sounds no longer sound different.
(1) It was audible under the conditions and at the time it occurred.
(2) At the time of listening the person was focusing attention on one part of the total sound, and when listening to something else, attention was focused on some other aspect of the sound. It happens without the person's knowledge, System 1 processes in the brain make it happen that way. So, what they hear, that is to say, what makes it to conscious awareness is a different listening experience in both cases. They did hear a difference, but not for the reason we are interested in.
In the first case (1) if they can't tell a difference under test, there is probably a problem with the test, even if the test giver doesn't know what it is and doesn't care.
In the second case (2), under blind conditions the person has lost the cues that allowed unwitting focusing of attention on particular aspects of sound, so now the mental experience is that the sounds no longer sound different.
Last edited:
I call them small if they are, but most people don't. When enough attention is focused on something small it can seem much bigger than it is.
That isn’t actually correct, the NH does in fact indicate the gear was indistinguishable to these participants under these test conditions.
You could argue that others participants would have been able to identify differences, or that these participants would have been able to identify differences under different test conditions.
But common sense typically prevails.
As i said previously the only thing you can take away is that test that day and nothing else.
dave
All humans are constantly influenced by cognitive biases and heuristics.
Here is the list:
List of cognitive biases - Wikipedia
Right, and until it's accounted for, there's nothing of value in any subjective opinion whatsoever with regards to audible information.
We certainly shouldn't entertain the idea that a blind listening test is flawed because there might be someone in this thread who didn't participate in it and we can't know their results, or it's too stressful, or..whatever else anyone might say that has a vested interest in casting doubt at ears-only testing.
I have foo abx, I've done ABX testing to compare files with different levels of compression, and it does exactly what it's supposed to to.
I'll say it tells me what bit rates I need to encode something at so that audible compression isn't present, and if the only counter argument is "it only says that for you", then that's the same old tired argument that my ears are broken, or my gear sucks, and frankly, I don't want to hear about what someone else's results might have been in comparison to mine unless they're prepared to present them.
If we're going to say the results aren't valid because a person from this thread might have heard something under the same conditions, or on a different day, or when it's cold out, or if it was in your listening room, or because the button was red, we're throwing the baby out with the bathwater.
"It might be different under different conditions" doesn't mean anything until some positive results are achieved. "Under any conditions currently tried to date" has to be good enough, otherwise, it starts to sound a lot like people being hopelessly pedantic because they haven't been able to get a result they like, most likely with some "thing" they like that they claim to be able to readily tell apart from something similar. Or even better, the great mother of all bias injectors, something they conceived themselves.
I'm late to this thread, so it's likely already been discussed, but I want to add that in mastering work, when reviewing the effect of a compressor, EQ or level change (if mixing stems) sighted A/B listening is very common.
Having done this A/B ritual on the daily for months at a time, I've taken away how long my mind can remember what it just heard. When I first started mixing my ability to do this wasn't very high, but as I continued for several years it started to become something where I could accurately remember what just played and in an even more difficult psycho-acoustic feat, compare the memory to the current rendition and make adjustments.. This is why mastering engineers train for so many years before graduating to solo positions (well, most good ones anyway.) So what is the point behind this?
1.) Qualified listeners. Why? if 99% of end users aren't qualified does it matter? Because unqualified listeners will still form an impression over time, even though their actual ability to test-out will be low the enjoyment a good system can bring will be slightly improved by jitter reduction among other things.
But I think this second point is more important>>
2.) Acoustic memory is very fleeting. When doing A/B changes in mixing/mastering it is common to loop a very small section of the track and then adjust (seamlessly flipping back and forth between the changes made).
The finer the detail of the move the shorter amount of temporal data we give our minds to hold onto in order to have more ability to compare and less competing cognitive changes. To not be distracted by the song we might loop a measure or several measures (ad the edge of boring, but at the beginning of interesting). Now this would drive a lot of music listeners crazy, but this is a very simple way to stop hearing the song and start hearing the transient details, the spectral balance, the dynamics, etc. When I listen to more than say ~4 seconds I find that my ability to remember (at the detail level needed for accurate comparison is) compromised significantly. Perhaps 10x worse just by adding another 4 seconds.
Anyway, a last thought that dovetails with the above:
a.) The time needed to switch between the two sources/dacs/EQ/etc should be instantaneous and without distraction. A simple button click without the music starting/stopping or changing temporal position within the track. Any other change will distract and overwrite the persons memory. They should ideally be able to control when it changes, thus allowing the person to choose the listening frame of evaluation.
BTW, single driver systems without crossovers are much more appropriate for AB testing, and oh, head in vice is a real protocol. Our physical position is often much more meaningful than THD in the DAC. Just moving your head off axis 2deg creates a huge change in FR compared to the differences between the DACs we are comparing.
I'm not sure about the qualifications of the people doing the testing, but for the mixing/mastering educated like myself these are all normal parameters you consider if you are taking data.
I'm sure this has been said 20x now, but lets add another voice:
Modern DACs are pretty good compared to speakers.
Last last last,
After writing this, the phrase "He has golden ears" doesn't seem appropriate anymore. Something more like "He has unmatched tempo-acoustic perception and recall". UTAPR? ugh.
Having done this A/B ritual on the daily for months at a time, I've taken away how long my mind can remember what it just heard. When I first started mixing my ability to do this wasn't very high, but as I continued for several years it started to become something where I could accurately remember what just played and in an even more difficult psycho-acoustic feat, compare the memory to the current rendition and make adjustments.. This is why mastering engineers train for so many years before graduating to solo positions (well, most good ones anyway.) So what is the point behind this?
1.) Qualified listeners. Why? if 99% of end users aren't qualified does it matter? Because unqualified listeners will still form an impression over time, even though their actual ability to test-out will be low the enjoyment a good system can bring will be slightly improved by jitter reduction among other things.
But I think this second point is more important>>
2.) Acoustic memory is very fleeting. When doing A/B changes in mixing/mastering it is common to loop a very small section of the track and then adjust (seamlessly flipping back and forth between the changes made).
The finer the detail of the move the shorter amount of temporal data we give our minds to hold onto in order to have more ability to compare and less competing cognitive changes. To not be distracted by the song we might loop a measure or several measures (ad the edge of boring, but at the beginning of interesting). Now this would drive a lot of music listeners crazy, but this is a very simple way to stop hearing the song and start hearing the transient details, the spectral balance, the dynamics, etc. When I listen to more than say ~4 seconds I find that my ability to remember (at the detail level needed for accurate comparison is) compromised significantly. Perhaps 10x worse just by adding another 4 seconds.
Anyway, a last thought that dovetails with the above:
a.) The time needed to switch between the two sources/dacs/EQ/etc should be instantaneous and without distraction. A simple button click without the music starting/stopping or changing temporal position within the track. Any other change will distract and overwrite the persons memory. They should ideally be able to control when it changes, thus allowing the person to choose the listening frame of evaluation.
BTW, single driver systems without crossovers are much more appropriate for AB testing, and oh, head in vice is a real protocol. Our physical position is often much more meaningful than THD in the DAC. Just moving your head off axis 2deg creates a huge change in FR compared to the differences between the DACs we are comparing.
I'm not sure about the qualifications of the people doing the testing, but for the mixing/mastering educated like myself these are all normal parameters you consider if you are taking data.
I'm sure this has been said 20x now, but lets add another voice:
Modern DACs are pretty good compared to speakers.
Last last last,
After writing this, the phrase "He has golden ears" doesn't seem appropriate anymore. Something more like "He has unmatched tempo-acoustic perception and recall". UTAPR? ugh.
Hi, thanks for a sane post
I wonder, as someone in the "business", do you consider yourself to be more a musician than an audiophile?If you carefully read what we said you will see that we left this possibility open:Markw4 said:
That does not necessarily imply that the abilities always vanish when eyes are covered - although we know that this often happens.DrDyna said:
This does not say that golden ears always lose their powers.DF96 said:
You introduced the word "moron". On the contrary, it is the clever who are more likely to be fooled by their own ears because they think they are too smart for this to happen.Markw4 said:
Very small differences don't matter very much, and very small differences will not be accurately measured by any hearing tests -simply because they are very small. The issue here is that allegedly large differences (sighted) cannot be heard under ABX.
The null result means the devices were indistinguishable (in that test, on that day, by those people etc. etc.). People may then wrongly say 'the same' as shorthand for 'indistinguishable'. This mostly upsets people who believe, rightly or wrongly, that the two devices are hugely different and therefore easily distinguishable. Of course, you could do measurements (or calculations) to determine how different the devices are.planet10 said:
No. ABX shows whether two devices are distinguishable. It is then a matter of logic to deduce that if they are distinguishable (in that test, on that day etc.) then in some way they must be different. If they are not found to be distinguishable (in that test, on that day etc.) then they are indistinguishable. There is no third option. People objecting to the word "same" are often playing semantic games to divert attention from the word 'indistinguishable' when that is the clear result of the test.planet10 said:
Genuine large differences hardly need an ABX test, except to demonstrate that they genuinely are large. Small differences benefit from the statistical analysis of ABX. Very small differences can be expected to be inaudible, and therefore lead to indistinguishability in ABX.
There is a third reason:Markw4 said:
(3) they think it ought to be audible and they have enough clues from non-auditory input to support this - which is why tests of audibility have to be blind.
Anthony, Thanks for all that you wrote. It happens that I did live sound for 7 years, then a break, then several years of mixing. I couldn't agree with you more. I have found that listening skills are learned and have often used mastering engineers as examples of what some people can do.
Unfortunately, there had been a strong bias in this forum that if people have trouble with Foobar ABX, the problem is with the person. I did some experiments with blind testing using the Reaper DAW, set it up for single button click comparison, and found that looping a very short section of music, particularly a revealing section, allows extremely low level discrimination of distortion, but that operating Foobar ABX is so distracting that fleeting auditory memory is easily lost. Also, re-doing a very low level discrimination test over and over again 20 times or a 100 times to get statistics is unbearably taxing. It can be hard to latch onto the small details of a sound again and again to achieve recognition.
People who haven't experienced these things tend to be very skeptical. They think of some untrained and unpracticed audiophiles they know and assume that's all there is, nobody can actually hear any fine details of sound.
Last edited:
Thanks for the feedback Anthony, I was beginning to fear that I was getting an elitist complex as I gain listening skills working with my projects here over the past 4 years. Glad it’s probably not that but more like real skills instead.
It makes it more rewarding to have a background and some time in that’s for sure. I couldn’t have made the decisions and determinations that I make now when I first started to actively participate in this hobby.
I use speakers w/out crossovers on my test bench that are always the first listen, have shut down more than a few modifications...
It makes it more rewarding to have a background and some time in that’s for sure. I couldn’t have made the decisions and determinations that I make now when I first started to actively participate in this hobby.
I use speakers w/out crossovers on my test bench that are always the first listen, have shut down more than a few modifications...
- Status
- This old topic is closed. If you want to reopen this topic, contact a moderator using the "Report Post" button.