Ah, I see - better to receive lots of probably invalid results & consider them all valid rather than make any attempt at some quality assurance? Of course, once the results satisfy your bias, what's the harm?
Most everyone knows that sighted listening is just a bit of fun & people need to listen for themselves o their own systems. Only people trying to defend a slap-dash listening test point to other slap-dash tests as justification for their approach.
What does it tell?
If it's 'just fun' then forget about ABX testing with 16 boring trials (not most people's idea of fun) - forget about statistical analysis (most people do not consider this fun).
So what's the point in having these elements, then?
Look, I'm not getting at you (although it might seem I am) but rather at the double speak that goes with these forum run blind tests & the pained attempts to make of them something more than they are.
i like the pun connotation that this misspelling invokes...i wonder if Pistollero would mind if i made it my signature line?
FWIW I don't put too much stock in them either, but I would love it if someone would post the details of some "approved" ones rather than endless specifications of what they have to entail and how they have to be interpreted. It does not encourage me that the ITU spec often quoted comes with undithered 16/44.1 test samples. They're so laden with artifacts how could any subtle effects be tested for?
Last edited:
I dunno ... I didn't use any fancy software ... just iTunes for playback and a Marsh Class-A headphone amp into ATH-M50X phones and I could clearly hear the difference. Avocado had much more detail in mids and highs and the bass "seemed" less but was actually tighter than Apricot. Apricot was overly warm to me and sounded like it was running through an unmodified 60's vintage tube amp. Then again I am told by all my audiophile friends I have a "Golden Ear" ... <shrugs>
Sorry, I don't get it. Please lay out your test protocol for a valid ABX test among forum users. How would you do it better? What have we learned well down the road that we didn't know at the beginning of the test?
This afternoon I learned that it's tricky to measure headphones. Actually I knew that already, but just jumped thru a few hoops to find out what might be going on with this test. After a number of tests and setups, I find that my DAC/AMP/Heaphone setup can resolve the distortion overtones, but with altered levels. 2K shows up 10dB below the 1K sine. Why? I don't know. FR sweeps do not show that difference. So a dual tone with 1K+2K at 100:1 ratio should show 2kHz at 40 dB down. But it consistently shows as 50 dB down. How are users supposed to know that going into the test? And is that just a test artifact?
EDIT: Two new sweeps show 2 kHz at about -10 from 1 Khz in my setup. Is it really that low at my eardrum? Don't know.
Last edited:
Please PMA ( 🙂 ), the first question was:
"But in this case did you realize the importance of the assumption?"
and you answered that.
The second question was:
"The assumption of random guessing means that each participants could just have thrown a coin each time, so ask yourself would have problems to accept combining the results from coin flips done in different locations by different humans? (Each one using a perfect coin, that is the assumption)"
An answer to that is still missing.
Please Jakob 🙂,
the second question was answered yesterday in this post:¨
Can you tell original file from tube amp record? - test
(#325)
Imo it is a bit ironic that guys who were unwilling/unable to tell what kind of evidence they would accept (if the evidence is contrary to their prior beliefs), think it is easy to find/explain a "one size fits it all" test approach, distributable to arbitrarily choosen locations, using arbitrarily choosen equipment to get results of good quality. 😉
Taken at face value that statement is misleading, as the "ITU spec" we are referring to in the discussions about good controlled listening tests is ITU-R BS.1116-x and it comes without any test samples.
P.S. Please remember the various requirements that were allegedly mandatory like,
-) must include "anti cheating provisions"
-) zero financial interest by all parties
and we have to add, that
-) should be preferable a software solution freely distributable
-) should have kind of a audio routing quality check to ensure that all participants at least really get exactly the same data send to the end devices
-) should preferable have several different test protocols to choose from so that each participant might be able to find the individually best fit
-) a plethora of additional things to think about
At the moment we have just started to check the available packages and are trying to work something out.
So far i´m quite sure that we can´t do all from the list.......
This is something I do not understand why they do it so. Is it just inertia? Everything taking so long time when it is a subject of standardization? I remember old CD test disc were undithered, but hoped it is not the case nowadays. Undithered low level tones in 16/44.1, like -60dBFS or -80dBFS 1kHz, have very audible artifacts.
I´d say that is the answer to the first question of the cited post?! 🙂
but i´m interested in this one:
"...so ask yourself would (you) have problems to accept combining the results from coin flips done in different locations by different humans? (Each one using a perfect coin, that is the assumption)"
(the bold "you" now inserted)
as it is the crucial point when you expressed your concerns about the "summing of different test results".
So, would you have problems combinging the results in that case or wouldn´t you have?
You are pretty persistent, but that's fine 😀. I hope you were able to read between the lines and deduce from what is happening and posted 🙂.
I would have big problems if someone is reluctant to test his system and post the result. For this reason, I have put the sine test with a pure dithered tone and the distorted sine with profile almost same as the amp tested and asked participants if they hear the difference. The distortion of the distorted tone is by all means above audibility threshold. So if someone does not pass this test, to me, his result on music sample is irrelevant. It was only Pano who was willing to test his system and found he has intrinsic -46dB H2 distortion, so the system cannot be used to get a valid result (to me). Is that enough?
The sine test was in 96/16 (because I had the test tone already prepared), maybe I should provide the same test in 44.1/16 to have the same format as apricot/avocado, however it all takes time, more than writing on keyboard, with questionable output from audience.
As Mooly was the only one who passed the sine test with 8/8 (though I would prefer 16/16) and posted a valid ABX record, he is the only one whose test result on music samples I take serious. Am I clear enough?
Last edited:
It would be of great interest to know if someone passed the test using foobar2000, with or without the ABX plugin. And Output mode too.
*********** ************ *************
I already had problems with him in the old test of the op-amps, which easily surpassed with JRMC.
That with foobar2000 had a different choice in the tube test has been the lace.
*********** ************ *************
I already had problems with him in the old test of the op-amps, which easily surpassed with JRMC.
That with foobar2000 had a different choice in the tube test has been the lace.
Yes, with foobar2000 + ABX, post #198
Can you tell original file from tube amp record? - test
Do you want for me to post 50/50? To spend time for nothing?
Your persistent accusation of foobar flaws is boring. Show me a loopback measurement of your system with foobar2000 playing sig1 signal, or at least Rightmark report.