That's generally acceptable, anything else like distortion is going to be hard, the research is all over the place and there are numerous variables like order and magnitude. Tests where the distortion is magnified to establish relative thresholds tell little about absolute threshold at small levels of distortion. Testing myself I need at least 1-2% (at low orders) to even feel something is maybe off.
Right so you have hidden controls in the ABX test - in other words in some random trials there will a 1dB difference between A & B. If, in these trials, X isn't correctly recognized as A or B then what does it tell us. It tells us that a known audible differences hasn't been perceived (false negative) so can we say that the rest of the trials are accurately going to identify audible differences if they actually exist between A & B?
As I said in blind tests a post-testing stage is used when controls are employed & those tests are rejected whose results show the failed perception of hidden controls
As I said in blind tests a post-testing stage is used when controls are employed & those tests are rejected whose results show the failed perception of hidden controls
Last edited:
But no actual case of you asking about control when someone makes claim about audible difference between audio cables, DACs or amps after subjective casual comparison. That's what I thought.
You still can't grasp the point and keep missing it.
You have once again confirmed that you've never participated in audio DBT despite your claim of having done so a few pages back.
This is all well and good but as Count Bayesie says...
Using Bayes' Factor to Build a Voight-Kampff Test! — Count Bayesie
In other words will we ever come up with a test that you and Mark will participate in? Everyone else has chickened out for one reason or another. Mr Marsh considers it an insult to past work and a waste of his time (the masses have spoken). John Curl admits that when blind the differences disappear.
Using Bayes' Factor to Build a Voight-Kampff Test! — Count Bayesie
In other words will we ever come up with a test that you and Mark will participate in? Everyone else has chickened out for one reason or another. Mr Marsh considers it an insult to past work and a waste of his time (the masses have spoken). John Curl admits that when blind the differences disappear.
Last edited:
Which 'Mark' are you referring to? If me, then sure. Not hard at all to come up with such a test. That is exactly what I would like to see Jakob2 doing, instead of asking people what it would take to convince them. Come up with a fair and accurate test and get people to agree that is exactly what it is. As a start it could be very, very similar to Foobar ABX, some kind of a file comparison test. In addition to what Foobar ABX allows, it would need to allow looping and one-touch hot-key switching of A/B files so most listening could be done without unnecessary distraction, and without looking if so desired by the test subject. Then start testing people who aren't afraid and who are good candidates to maybe show some interesting findings.
EDIT: Standard test setup should include something like Benchmark DAC-3. A typical computer sound card is not up to the task in terms of reproduction accuracy, IMHO.
Last edited:
Me too. And given that as far as I can tell, if someone uses "groove" or "vibe" you have to ask for clarification, and will not get the same answer every time (I got conflicting answers from a guitarist and a keyboard player) , they are more or less useless terms.
Are there any links to a well defined set of words - if so they should be in a sticky thread in here and mandated in the forum rules! could save a lot of bother, and it's certainly a start point for your original query Jakob - like any science one of the first prerequisites is defined terms so all can understand.
- Graham (correct, DPH!
)It´s a good example for the difficulties of communication/discussion via internet forums. I simply missed (or didn´t understand) that you proposed to develop a "test" that all will/could agree with.
(I´m wondering if you actually proposed that in plain words or if it was a matter of interpretation)
I would not agree to the "not hard at all..." line though, as it indeed depends on the objective of the experiment to develop/configure an appropriate test.
If you add the demands for "no financial interests" , "no ego involved" , "extraordinary evidence if ...." and nonacceptance of positive controls and so on, i think "not hard" is a serious underestimation.
In another forum i´ve seen demand for a supervising authority (i guess the poster had someone like Randi in mind) and of course the demands on the reproduction system used in a test will give reason for discussions as well.
I thought collecting the various demands first, then to discuss what is usually done, what could be added and what would be a (nearly) out of reality could be a good idea as it would be possible to mention and discuss the basics of sensory tests.
I think the discussions in this thread show clearly that - despite all the threads we had about it - there´s a lot still not part of common knowledge.
Of course i can be wrong, so why not start a new thread to try your approach?
It might not happen often if at all, but if it did or does, I'm going to work at finding one to have a go at it. That's the kind of product I would be very interested in hearing. I would suggest that all three are sometimes found amongst the less consistent audio products we still have in the market ... loudspeakers, certainly, and things like headphones, phono cartridges, even turntables and tonearms.
Most of the time I do what everyone else does ... I seek out products based on need and absolutely must hear it to buy it. But that doesn't address the OP's question; a personal choice, mine or another's, is not "evidence".
Last edited:
Why do you NOT consider listening to a product at home in your setup, as evidence?
No source of funding for a programmer. I had someone volunteer by PM to take it on for free, but when they realized it would be more work for them than they originally thought they dropped it.
Since Foobar ABX is closed source and nobody knows how to contact the programmer, it would take somebody who knows how to code Foobar plugins or something equivalent.
I am old and retired. It isn't a project for to me to take on. It would be a good project for someone who wants to show what some people are capable of in terms of careful listening. That should be enough to start changing some minds on the subject. Some people will never change their firmly held opinions though, so no point in worrying about them. Eventually they will disappear and people that come along later will think more in accordance with the new and better scientific evidence, all IMHO, of course.
If somebody else wants to get a test together, I would be happy to discuss how it might work, act as guinea pig for trying it out, and or present myself for testing when it is ready.
Of course everything understandable, but wasn´t your suggestion to lay out first the conditions of a experiment (including test protocol) - maybe/surely discuss it - let (hopefully) all agree to it and only then start with any kind of realization?
Edit: So what i basically meant, why don´t you start another thread in which we propose an experiment, try to explain why certain elements would be used?
Last edited:
I hope you are open to the possibility that that IS an opinion and not a fact (for everyone).
IMO BTW a condition like that ends all possibility of any traction on this. I also have very little faith in web based file swapping getting anywhere, too many uncontrolled variables (not the least of which the past history of easy cheats).
I was suggesting live in person with a trained referee on the same equipment.
Finally, as I have mentioned to Jakob in the past, I don't have a lot of interest in this except with respect to extraordinary claims. By extraordinary I mean those that put audio outside of the rest of the engineering/scientific community.
Nevertheless you are participating in this thread and as i remember that you were talking about "not seeing .... doing hundreds of hours of ?ABX? trials" (not an exact quote, more something equivalent iirc) it would be imo fair if you´d disclose what kind of positive result from "these" tests you would be willing to accept as evidence although your prior belief was different.
Again i have to ask what qualifies as "extraordinary" because "outside of the rest of the engineering/scientific community" is unspecific. Is it based on a majority decision or based on available evidence from other well executed experiments or just belief of those "engineers and scientists" ?
From my viewpoint a lot of these engineers do know next to nothing about sensory evaluation, don´t really understand statistics and therefore don´t know what experimental results (including wrt thresholds of hearing) do mean and what they don´t, but nevertheless they express strong beliefs about what can be or can´t be audible.
We have no disagreement there. But, the test equipment needs to be good enough for what is to be measured, that's all.
EDIT: By the way, I don't claim any extraordinary abilities. In another thread I have been incrementally designing a dac without resort to much test equipment. I can hear when it is get closer or farther away from DAC-3. If you can accept that then I don't think we have anything to disagree about. I certainly don't hear cables, or anything of that nature, and I don't know why you would ask when 'Mark' will agree to a test.
EDIT 2: Could be you were referring to my efforts to convince Mr. Merrill that a good test should be nothing to fear. In that case, he is now on my ignore list so I don't have much hope he will come around anytime soon. If Jakob2 could start publishing some good listening test results however, then Mr. M. might find it reassuring that some people can do quite well on such a test, better than he might expect. I personally do not anticipate that anyone will hear wooden cable lifters though.
Last edited:
My comment has nothing to do with sensory evaluation, I am talking rewriting basic principles like conservation of energy or charge, super-luminal propagation, Maxwell's demon, reversal of entropy, hidden variables/information, etc. Frankly I think you are a little stubborn on this point.
If you want to test whether putting a photo of your preamp in the freezer improves the sound, you can waste your time don't ask me to.
- Status
- Not open for further replies.