There is a world of difference between determining exactly what happens when you put two chemicals in a test tube, and determining what a random (trained or untrained) listener can hear in a particular environment with particular music played through particular equipment on a particular day.mmerrill99 said:
I have some sympathy with this view, but FUD sometimes needs defusing - even when wearing 'scientific' or 'statistical' clothes.Markw4 said:
I continue to note that tests showing indistinguishability are criticised, and tests showing distinguishability are either uncritically accepted or offered in evidence - yet we are asked to believe that the criticism is solely based on test protocols and not on the outcome.
Did you read what JonBocani wrote, BEFORE the test test I referenced?
If so, please show how what I have said is false. If not, how did you arrive at your conclusion?
I know Jon for a long time. I can assure you no agenda drive him. This is what he wrote before the test ( in French ):
LE test des convertisseurs (DAC) - Hedonistes
Translation: " According to my experiences, no DAC under 200$-300$ can cause a surprise."
I accept the test findings that those people on that day using that equipment in that environment with that music have honestly reported that they could not distinguish the DACs, despite at least some of them being confident before the test that they would be able to distinguish them. If that is what you mean by "agree" then yes.Doppler9000 said:
If someone else on a different day with different equipment etc. found the opposite result I would equally accept that. I would not conclude, as some appear to do, that one test was faulty and the other test was fine.
I agree that the outcome of a test is not evidence that it was either valid or invalid. We must look elsewhere for insight.
The 1980s Stereo Review 'tests' undertaken by David L. Clark, which "proved", among other things that a $239 Pioneer receiver sounded the same as a $12,000 OTL amp, a Mark Levinson amp, and others, was widely regarded as dispositive, though there were critics.
A null result, which the OP doesn't understand, by the way, is less clear than the opposite, by its nature.
Sighted, I perceived better decay, more resolution, better extension and micro dynamics. I try to always focus on a small portion of the track that enhance these characteristics. Presentation in general appear to be immersive, lots of details thanks to the Raal tweeter and proper micro-dynamics.
Blind, those characteristics disappears. Attacks and decays are no longer discernible. Soundstage and image appears exactly identical.
Sorry this again shows your failure to understand methodology. You were already told by Jakob about controlling variables in any experiment. What is it you fail to understand in this simple idea?
So far you have rejected the concept of controls in the test, failed completely to understand what blind preference testing is & stated you have no interest in learning about perceptual testing & yet suggest that people should be free to run a 'test' in whatever way they see fit as you don't care to evaluate the validity of a test.
I'm wondering if you would take Mark's advice as you seem to have dug yourself into a hole & you keep digging deeper?
JonBocani is almost always "surprised", over and over, by his null results, even though he has repeatedly said that human hearing is weak, "massively" and "vastly" overestimated.
In the "test" I referenced above, he predicted that the test subjects would be unable to hear differences, which would cause great consternation for audiophiles. He predicted the outcome, correctly, before he undertook the test. This, on its own, calls into question the tests themselves.
If you saw test results that showed that smoking wasn't very harmful, would you believe the results equally if they were published by the NIH or by a tobacco industry group?
I'm not sure that the issue is as you describe. Many people here have criticized the claims that have been made about what can be inferred from tests, not so much the tests themselves.
The problem is when four people are tested and fail to hear something, and then a claim is made the test has proven that no person in the entire world could have heard anything. DF96, you know that can't be right don't you? The claim doesn't logically follow from the experiment, completely aside from whether the experiment was a good one or not.
EDIT: Suppose for a moment the original experiment in the thread was a good one. What could be logically claimed from the result? It seems to me it would logically follow that a minimum of four people in the world could not hear something under the conditions of the test. That's it.
It the test was a bad one, then nothing could be concluded from it.
If instead of just applying logic, we wanted to go into the world of biostatistics, then if the test were a good one we could produce some statistics. But, with only 4 test subjects, a sample size of 4 subjects out of a population of 7.6 billion people in the world the statistics would be meaningless. If we had a 100 or 1,000 test subject chosen randomly from the population (and they must be selected randomly, not volunteers), then we could maybe start talking about trends for many or most people in the population. Even then we couldn't say what would be impossible for every single human on earth based on the statistics.
Last edited:
Again, you are wrong. What is the part that you don't understand?
LE test des convertisseurs (DAC) - Page 4 - Hedonistes
Thanks!
So instead of trying to identify if X is A or B, what would you think of just doing an A/B preference test, blind?
This was my fundamental complaint, as well - the generalization of the result. When someone makes this sort of general conclusion, there will be more scrutiny on the test and on the tester.
The OP deflected much of the methodological critique and resorted to ad hominem attacks. He ignored counter-evidence. He demonstrated that he doesn't understand what a null test can prove and what it can't.
The defense of the OP seems to rest on a narrative about the results, and a critique of people's attachments to their ideas of audibility.
Please explain to me how it is possible to have a preference when A and B are indiscernible? I certainly can't.
That's the point - you are trying to consciously discern a difference & you can't - this is one of the pitfalls with ABX, it relies on actually spotting a specific difference
In A/B preference testing, don't try to spot a specific difference - just pick what you genuinely think sounds best on each trial. Don't be biased by your results in ABX testing that there is no difference between DACs - do give it a real shot & ask yourself on each trial which do I prefer even if you can't say why.
if a significant number of trials shows you favouring one over another, you may think it was pure luck so try again.
But again don't try to consciously spot a difference - rather get a feel for one presentation Vs another
Who knows, it may be null again but what if it consistently favours one device?
The point is that it's a bigger ask to be able to consistently (or statistically significantly) spot a difference than it is to just state a preference without being expected to name the difference & why you prefer one over the other - it may be that you can't say why you preferred one over another (at least initially). Being expected to spot a difference is cognitively more onerous & effects perceptual abilities.
Last edited:
that's gonna be so much fun to organize, this test...
audiocopy of a whole boutique system, which will include so much bulls*** mumbo jumbo AT THE SAME TIME
They'll be like:
''What do you think went wrong ?''
- ...ehm.. i think the Shunyata power cable was not quite broke-in yet
''oh no, you didnt!? I told you, Berthold! 1000 hours minimum!''
- I'm truly sorry, Oliver, *sobbing* i failed, it's all my fault!! *sobbing*
'' You're a shame for Audiophilia, Berthold, a bloody shame!!
An externally hosted image should be here but it was not working when we last tested it.
You seem confused here - I am not speaking about the DAC test. I am referring to an earlier test undertaken by JonBocani (August, 2016).
JonBocani wrote the word above about a test he was about to undertake. He wrote his conclusion BEFORE he started the test.
He had an obvious (null result) preconception of the outcome, and its effect, before he even started the test.
I would fail because my preferences are based on the same criteria ( decays, attacks, micro dynamics, soundstage etc... ) that I try to use to identify A from B.
My favourite is "Speak not what you know not of". Not your favourite, I'm guessing.
Please explain how a prior history of prejudging the outcome of a test, pre-concluding a null result, from someone claiming a generalizable null result from a new ABX test is "off topic".
OK, you always listen analytically even with casual listening to music?
That's all I'm saying to do - casual listening for enjoyment, not a 'test' - switch off the analytical approach & listen to the music & the say which gave you more pleasure, no any forensic analysis, forget about 'test'
- Status
- This old topic is closed. If you want to reopen this topic, contact a moderator using the "Report Post" button.