What kind of evidence do you consider as sufficient?

Status
Not open for further replies.
This seems to remain a one sided discussion. The average consumer/user of limited means, time, and ability is to be disarmed of their "forum ABX" tests while the industry plods on making claims that are supported by even less credible means. Look at MQA for instance. I feel like I'm arguing with an industry shill.

Try to just address the points rather than flinging out ad-homs?

I'm bringing a balance to the viewpoints by showing what the corollary of the viewpoint expressed in the post (it wasn't your post) - it's called showing both sides of the argument but if you only want one viewpoint expressed then why not state that?

You seem not to want to face up to the many problems with forum ABX tests & constantly plead for it to be left alone & allow people to use it - I haven't seen you actually deal with any of the flaws pointed out about such a test?

I heard a demo of MQA & was very suspicious of it's administration with the demonstrator picking tracks using a remote - in the time I was there, no audience member was allowed do this - made me suspicious & I came away with a less than happy feeling about it. However, I reserve my opinion about MQA until I have had extended listening to it but I'm not rushing out to equip myself to hear this.
 
Ironically many musicians aren't that concerned about Hi-Fi reproduction

which could be a good thing for lack of bias. Might make it difficult to drag them along to a test though..

I don't fully agree - really depends on the individual as some very much are concerned with sound if not specifically "hi-fi", or at least into subtle sonic differences (of deliberate colouration, instrument tone etc) but as a slice of the average population who also aren't concerned, then as a generalisation it's probably fairly true.
 
Last edited:
No-one knows if they've heard a difference or not, only that they've perceived one. You won't either know if the ABX null result is setting straight your false reality or if it is moving you away from a previously more accurate one.
Correct. I use the word "hear" to actually mean "perceive" as there is no hearing without auditory perception. What I'm attempting to illustrate in some of my posts is that an understanding of auditory perception is fundamental to this hobby as it prevents us making many basic mistakes. Essentially, we are processing machines analyzing a very incomplete set of signals & making a pretty good guess (using all sorts of techniques & experiences). So yes, we have to use our auditory perception to judge all of this. I work on the basis that we get pretty good performance from our auditory perception throughout our daily lives so why would I suddenly begin to think that it is failing me when I listen to audio playback systems?

Therefore, essentially it is the attitude of the participant that is doing all of the power of suggestion upon themselves. If they have a bais against ABX testing for example or an ego that can't allow itself to be contradicted etc

The better approach would be to acknowledge your own experience without conclusion it was correct or false and to let people take the result of the ABX to do with as they please. I don't understand why folks can't take that approach - I guess it is many other factors in the history of their lives, part of which is their brain's constructed self-story and image. One can still make strong decisions without necessariy knowing the absolute truth (afterall, that's how we all vote!) .

p.s. for clarity, I consider "hear" to mean you could detect that they heard something if one could intercept and inturpret the signals between sensary organs and the brain. "perceive" meaning it is part of the story the brain has concocted to represent reality.
I define it differently above - to me "hear" is just a short way of saying "perceive"
 
You seem not to want to face up to the many problems with forum ABX tests & constantly plead for it to be left alone & allow people to use it

Why don't we just fix ABX rather than keep arguing about it? My own experiments suggest it shouldn't be very hard to do. No one seems interested enough to bother trying out the ideas though. Easier to argue than do work to try to make some progress?
 
Last edited:
No-one knows if they've heard a difference or not, only that they've perceived one. You won't either know if the ABX null result is setting straight your false reality or if it is moving you away from a previously more accurate one.

Therefore, essentially it is the attitude of the participant that is doing all of the power of suggestion upon themselves. If they have a bais against ABX testing for example or an ego that can't allow itself to be contradicted etc

The better approach would be to acknowledge your own experience without conclusion it was correct or false and to let people take the result of the ABX to do with as they please. I don't understand why folks can't take that approach - I guess it is many other factors in the history of their lives, part of which is their brain's constructed self-story and image. One can still make strong decisions without necessariy knowing the absolute truth (afterall, that's how we all vote!) .

p.s. for clarity, I consider "hear" to mean you could detect that they heard something if one could intercept and inturpret the signals between sensary organs and the brain. "perceive" meaning it is part of the story the brain has concocted to represent reality.

A well known audio designer who regularly posts here in an eponymous thread, often expresses negative opinions about ABX and blind "ears only" listening in general. His complaint is that he has participated in double-blind and ABX tests and he has "failed" them. This is a very strange thing to say. I assume that he means that he has failed to differentiate equipment differences in blind conditions, which he regularly "hears" under sighted conditions. But saying that he "failed" the test implies that HE was the DUT. You cannot pass or fail a blind listening test, you can only report your results. That which is being measured is the existence or audibility of differences between two (or more) devices, not the hearing acuity of the tester. Of couse, his attitude and choice of words imply a deep bias and belief that his perceptions in sighted listening always reflect a difference in the signal produced by the DUT, so much that a failure to perceive the same differences unsighted can ONLY be explained by a fault in the test, otherwise it would imply a failure on his part.
 
Why don't we just fix ABX rather than keep arguing about it? My own experiments suggest it shouldn't be very hard to do. No one seems interested enough to bother trying out the ideas though. Easier to argue than do work to try to make some progress?

To my mind, the only way to be able to judge the validity of "forum ABX tests" is to have suitable controls which calibrate the whole setup, including the participants - there are so may variables in home run tests.

I posted this example of a guy describing his real-world ABX test on the other thread which gives you some idea of what's involved - don't want to pollute this thread any more than I have already ;)
 
Look at this thread for the first time, two things immediately come to mind from cognitive psychology research into bias and human nature. (1) There there is very interesting research showing that when people have firmly made up their minds, they are closed to thinking in truth seeking mode. (2) Debating is for winning, not for truth seeking. Both effects are very much in evidence in some of the posts. No point in talking to people who are already firmly decided and freely use debating techniques to win.

Yeah, I have to divorce myself here for the reasons you stated. I can't stand certain argument styles, as evidenced here.
 
mmerrill99, What if we just got it so that persons being tested agreed the test is fair and accurate from their perspective? That would at least be progress? By that I mean that the tests should be fair and agreeable no matter who is judging them. Those who demand more testing, and those who are to be tested should both find the testing to be fair and honest. To get to that point we will need to make a few small changes to the tests, and then see if there is anything else we can do to help improve them even more. This should be no big deal. Work on it, fix it, make it better. Don't we do that with amplifiers? I hope we do.
 
Last edited:
I'm not sure what side of MQA you fall on, but what I've read/listened to about what MQA does and how it works and the research gone into it, it all come across as logical.

It's technically worse than standard 24 bit audio signals compressed with FLAC or other open source lossless compression algorithms. By that I mean mqa is both lossy and larger file size. It's purely a licencing money grab.

We can argue the merits of hd audio, but mqa as an implementation is awful.

Back to the op.
 
Last edited:
mmerrill99, What if we just got it so that persons being tested agreed the test is fair and accurate from their perspective? That would at least be progress? By that I mean that the tests should be fair and agreeable no matter who is judging them. Those who demand more testing, and those who are to be tested should both find the testing to be fair and honest. To get to that point we will need to make a few small changes to the tests, and then see if there is anything else we can do to help improve them even more. This should be no big deal. Work on it, fix it, make it better. Don't we do that with amplifiers? I hope we do.

Your heart's in the right place but what really has to be faced up to is that perceptual testing is not suitable for running by inexperienced people who know nothing about perceptual testing - there are just too many factors that need to be addressed for any results of value. Using controls within these tests would quickly show how useless these tests are & people are just fooling themselves with the help of the many enablers that are scattered throughout audio forums
 
Using controls within these tests would quickly show how useless these tests are & people are just fooling themselves with the help of the many enablers that are scattered throughout audio forums

That sounds a lot like a construction you your part. It is a part of human nature to predict with certainty that which is in reality unknowable.

I'm sure you are aware of people who regularly predict the end of the world, and do so with great certainty. Or, how about from History Central, 'Federalist newspapers claimed that the election of Jefferson would cause the "teaching of murder robbery, rape, adultery and incest".'

As far as I am concerned, fear of what might happen does not justify failing to try to make any progress to improve testing now.
 
That which is being measured is the existence or audibility of differences between two (or more) devices, not the hearing acuity of the tester.

Maybe more accurate to say, "That which is being measured is the existence or audibility of differences between two (or more) devices under the specific conditions in effect at the time of the test."

@Nezblue - He is correct - what is under examination in ABX blind tests is the auditory perceptual ability of the participants in differentiating between A & B.. "under specific conditions in effect at the time of the test"
 
A well known audio designer who regularly posts here in an eponymous thread, often expresses negative opinions about ABX and blind "ears only" listening in general. His complaint is that he has participated in double-blind and ABX tests and he has "failed" them. This is a very strange thing to say. I assume that he means that he has failed to differentiate equipment differences in blind conditions, which he regularly "hears" under sighted conditions. But saying that he "failed" the test implies that HE was the DUT. You cannot pass or fail a blind listening test, you can only report your results.

Interesting - I think often people do feel it is a test of themselves and then are caught up in whether they are being believed that they did/didn't hear somthing when the ABX gives a different result.

Part of why I feel an ABX test can be better when the participants are not that worried about the subject of the test in the first place (and maybe are getting paid to participate? Would that help?) . Ony problem is it's harder to get those you presume will be better practiced at listening.

Often in psychology, they perform another test which is a just a front whilst they observe some other behaviour. Could something like that be worked out?
 
Only if a participant has previously claimed the difference to be audible in sighted 'tests'.
If no such claims were made the only thing tested for is the audibility of any differences.

If the group of testers are changed, the result could change based on those testers. It's not logical to extrapolate and claim the same result for any group of people. Therefore one can only claim the result for that particular set of "conditions", one being the group.

Depends on how subtle one thinks the differences will be and how they could be influenced by external factors (e.g. mains interference in the building that day, some kind of electromagnetic interference blah blah) that one can't control for.

I mean ... if the differences are expected to be subtle and you're testing human perception AND you can't control very possible external conditions that could effect the result - because it's not a mega-epensive physics experiement - in many ways what is the point of the testing in the first place? Too many logical reasons why differences could be masked by noise or differences created by random changes overlaying those made deliberately .

Alll you can end up saying is "this is the result for these specific conditions and we are not fully aware what those specific conditions were... "

"Just a bit of funn"
 
Status
Not open for further replies.