How can music be off topic? It is after all the reason all of us mess around over optimising audio systems (or at least it should be!)
I am making discreet enquiries about acquiring a turntable, as I haven’t listened to music on vinyl in my own home for 25 years. So I’m with you on this one, John
Sunset shooting out rays of light here. ToS
Asking people to think about the subject is not a challenge (except for those who don't want to think)The statement of vacuphile's "If an auditory difference exists, a blind test can be designed to bring it out." is exactly what needs to be thought about - you might consider it a challenge?
I already asked why this great need for blind ABX testing - I suggested blind preference testing was a far more useful test & one that is more like our normal listening. The biggest problem with blind ABX is the requirement to find a specific aspect of the sound that can be quickly A/Bed & that has an audible difference - this usually means a freq or amplitude difference at a specific part of a track. Most people approach ABX testing in this way by trying to find some specific part of the track that is audibly different.
So here's the dilemma, in our casual listening we often have a preference for one audio playback over another but when we try to nail down exactly what is the difference we find it escapes us - yet we still prefer one device playback over another. So why not do a blind preference test, not a blind ABX test? If we get a statistically significant preference for one device over another it is real even if we can't say what the exact difference is. Maybe someone else can find the exact audible difference & when pointed out to us we can then do blind ABX without difficulty.
I often hear people say on audio forums that a blind preference test can only be done after a blind ABX test has shown an audible difference is identifiable - shows a distinct lack of understanding about these perceptual tests.
Last edited:
hehe
You have a very wrong image of the constraints of an ABX test, in particular with "The biggest problem with blind ABX is the requirement to find a specific aspect of the sound that can be quickly A/Bed & that has an audible difference - this usually means a freq or amplitude difference at a specific part of a track". There is no requirement for a quick decision, you can switch back and forth at your preferred pace, you can train sighted as much as you would like, etc... BTW, a DUT that changes frequency and amplitude only in certain parts of a track surely looks like being haunted by Maxwell audiophile daemons.
But ok, preference blind test it is; please describe the exact procedure to implement a blind preference test to your satisfaction. In general, please describe exactly a whatever test procedure you think it's relevant, I have only one condition: no peeking.
Still not holding my breath for a positive contribution 😀.
Ok, you asked for it
Sure, I've heard this before "no requirement for quick A/B switching (I never said you had to make a quick decision) but the whole design of Foobar ABX is around quick A/B comparisons of specific sounds in the tracks being compared - this was based on the concept that auditory memory (echoic memory) has a short lived duration during which two sounds can be best compared. Hence the function built-in to Foobar ABX to switch from one track to the exact same playing position in the other track. This revisionism is simply because Foobar ABX has provided the exact tool that supports the belief system of a certain group of people by providing the necessary null results which are then interpreted as no audible difference (yes, this also has has revisionism applied- now it's said that a null result itself is does not mean no audible difference but an accumulation of null results .........)
OK, so you are happy not to insist on a blind ABX test before considering a blind preference test, then? Maybe you can tell others this & why you have decided this?
So what is it you are pleading for now, exactly? How to do a blind preference test?
I just received my BenchMark DAC3 from The Upgrade Co. out of Henderson, NV/USA.
David Schulte did the "Signature Edition Upgrade" and RFI/EMI riddance, also.
Nothing changed except everything, every which way was better, clearer, more accurate.
www upgrade company.com
THx-RNMarsh
David Schulte did the "Signature Edition Upgrade" and RFI/EMI riddance, also.
Nothing changed except everything, every which way was better, clearer, more accurate.
www upgrade company.com
THx-RNMarsh
Would you be willing to remove the lid and take a nice clear picture of the new insides?
Thx
Mark
Who really cares about this endless debate - just enjoy your hobby, the music, the equipment, whatever gives you pleasure.
Some random thoughts on the discussion of listening tests.
Sorry, this thread moves way too fast for me. Sorry if I repeat, or have missed important posts.
One important weakness of preference testing, using preference scores, is one you uncovered with Olive's speaker test: the same preference score does *not* mean the same sound. I may give the same 5/5 preference score for a well-made chateaubriand and well-made eggs Benedict, but, trust me, they taste different (no ABX proof to offer, though).
There are specific questions that *can* be answered with a classic, unmodified ABX.
A note on "blind" vs. "sighted" that I'm sure many here already know... sorry, ... but it is also clear some don't: sighted/blind is not directly related to vision/eyes. It is related to knowledge of the identity of the DUT, which may or may not be obtained through vision. It is child's play to create a "blind" test with the eyes wide open to see everything in the room (except what's blocked by a screen, for example). It's equally easy to create a "sighted" test where the eyes see nothing. The purpose of blind testing is not to avoid visual bias or visual distraction.
ITU BS.1116
An important part of it (as they mention), is that it is a guide, not a cookbook, subject to competent modification, depending of the specific goal(s).
There are specific goal(s) for which the classic ABX is good.
Sorry, this thread moves way too fast for me. Sorry if I repeat, or have missed important posts.
Preference tests and discrimination tests have different purposes. If I ask you what's better, a hammer or a screwdriver, don't you first want to know what my goal or task is? It may be that I need *both* for my task!
One important weakness of preference testing, using preference scores, is one you uncovered with Olive's speaker test: the same preference score does *not* mean the same sound. I may give the same 5/5 preference score for a well-made chateaubriand and well-made eggs Benedict, but, trust me, they taste different (no ABX proof to offer, though).
There are specific questions that *can* be answered with a classic, unmodified ABX.
A note on "blind" vs. "sighted" that I'm sure many here already know... sorry, ... but it is also clear some don't: sighted/blind is not directly related to vision/eyes. It is related to knowledge of the identity of the DUT, which may or may not be obtained through vision. It is child's play to create a "blind" test with the eyes wide open to see everything in the room (except what's blocked by a screen, for example). It's equally easy to create a "sighted" test where the eyes see nothing. The purpose of blind testing is not to avoid visual bias or visual distraction.
Sorry, but I missed yours posts where you contributed something useful to this topic. Can you point me to any?
Obviously many do care, otherwise the debate would not be endless... tiresome though it may be.
Step one is *always* define your goal(s)/question(s) and what outcomes you will accept. For listening tests that yield preference scores, this is a great start:
ITU BS.1116
An important part of it (as they mention), is that it is a guide, not a cookbook, subject to competent modification, depending of the specific goal(s).
There are specific goal(s) for which the classic ABX is good.
Last edited:
Always questions, never answers.mmerrill99 said:
Of course, you are in logically a strong position as no amount of tests (however excellent) can prove that two items will never be distinguished by another test or another listener. Therefore you can claim that all items are distinguishable and be confident that nobody can prove you wrong. However in experimental science we do not look for proof - we leave that to the mathematicians - but instead look for overwhelming evidence.
All this attacking of blind ABX may be a smokescreen, just like those who attack THD when they really don't want any measurements at all. Pointing out the flaws in a particular test/measurement becomes a proxy for rejecting all tests/measurements of a similar type - of course it is hoped that onlookers don't spot the difference between rejecting ABX/THD and rejecting all sensible blind tests/measurements. This smokescreen can sometimes be blown away by asking what tests/measurements the critic would accept. The anti-THD camp then usually woffle about using ears - which are known to be easily fooled. The anti-ABX camp usually woffle about the advantages of sighted long-term testing, or maybe continue to pick holes in any specific proposal from others while remaining shy about proposing any tests themselves. Thus the FUD spreads.
But SAM, what is always being contested in an audio forum that calls for ABX 'proof' is that someone states one is better sound than another - amplifier A Vs Amp B or DAC A Vs DAC B (it never arise over speakers) - it is never a case of DAC A Vs Amp B, the device in question are both peforming the same function.
So I'm not with your example of steak Vs eggs? Actually, I'm not with you on the Olive preference test either?
In a preference test of DAC1 Vs DAC2 if the results show a statistically significant preference for DAC2 then we have de facto established there is an audible difference, no but also which is performing its function in a more audibly correct manner (when judged by auditory perception).
Now there are some who might say that euphonic distortion will always be chosen over accuracy. If it is that obvious a difference, demands for ABX 'proof' would not be heard, anyway.
All I asked vacuphile was how he verified that his test was going to be what he said it should be "blind test that will not bring an auditory difference out"?
It seems rather strange, given your citing of the experimental science that you should think my answer is somehow different from the scientific approach?
Ah, that explains it - your (& many here) paranoia & bias about FUD & filthy lucre seems to have twisted your idea of what is a scientific experimental approach into a strange position of - yea, let's use a test that gives us predominantly null results & let'snot ever question how valid/accurate/sensitive this test really is - we have a good debating team that will deal with that sort of scientific curiosity 😉
Sorry part of that last post should read
There is plenty of FUD in this thread. One does not need to be paranoid to detect it.
My bias is simple: if conventional psychoacoustics and conventional physics say that some effect is likely to be inaudible then I am not surprised if a blind test finds it to be probably inaudible. I am similarly not surprised if a sighted test finds it to be audible. I am not surprised if a few people find it to be audible in a blind test.
My bias is simple: if conventional psychoacoustics and conventional physics say that some effect is likely to be inaudible then I am not surprised if a blind test finds it to be probably inaudible. I am similarly not surprised if a sighted test finds it to be audible. I am not surprised if a few people find it to be audible in a blind test.
@ SaM,
always good to emphasize that a clearly stated hypothesis/question should be the first step. Makes two of us doing it (scottjoplin will surely love the repetition too 🙂 ).
Preference tests can be misleading, if segmentation takes place (DUTs under tests are equally liked or dislikes), but if a preference is established (provided it was a sound experiment) then it provides corrobation for the existence of a difference.
As you've said, tests for preference and difference exist for different purposes, iirc I've tried to explain that and why in a orthodox product development process (or product change process) usually a test for difference comes first, but in the context of our discussions in these threads there is often no need for the traditional orthodox approach.
@ syn08,
At least we both knew that it is not true; "no peeking" isn't the only condition that you have.
You've already provided an example to the contrary in case of the Oohashi et al. publication (2000).
"No peeking" took place but you were more than happy to dismiss the whole thing just for the reason that something could have happened, while not being able to provide any real evidence and without consideration of the evidence that contradicts your assumptions.
Btw, you're still sitting on the fence... 😉
@ mmerrill99,
Imo vacuphile meant the statement in a general sense that it is possible to design a controlled listening test (including the "blind" property) so that an audible difference will be most probably detected. For example by considering all the informations we have collected in this and other threads. 😉
always good to emphasize that a clearly stated hypothesis/question should be the first step. Makes two of us doing it (scottjoplin will surely love the repetition too 🙂 ).
Preference tests can be misleading, if segmentation takes place (DUTs under tests are equally liked or dislikes), but if a preference is established (provided it was a sound experiment) then it provides corrobation for the existence of a difference.
As you've said, tests for preference and difference exist for different purposes, iirc I've tried to explain that and why in a orthodox product development process (or product change process) usually a test for difference comes first, but in the context of our discussions in these threads there is often no need for the traditional orthodox approach.
@ syn08,
At least we both knew that it is not true; "no peeking" isn't the only condition that you have.
You've already provided an example to the contrary in case of the Oohashi et al. publication (2000).
"No peeking" took place but you were more than happy to dismiss the whole thing just for the reason that something could have happened, while not being able to provide any real evidence and without consideration of the evidence that contradicts your assumptions.
Btw, you're still sitting on the fence... 😉
@ mmerrill99,
Imo vacuphile meant the statement in a general sense that it is possible to design a controlled listening test (including the "blind" property) so that an audible difference will be most probably detected. For example by considering all the informations we have collected in this and other threads. 😉
Well, that settles that!
We are now all free to discuss other things - game on!
Thank you DF96
Because the quote obviously does not contain evidence for your claim from 9th of June.
Further, none of the quotes that you've provided, contained any evidence for your claim from 9th of June.
The thread has > 1550 messages , so you have to be more specific which messages in this thread provide evidence for your claim from 9th of June.
As said above, the thread title itself does not.
- Status
- Not open for further replies.