What kind of evidence do you consider as sufficient?

Status
Not open for further replies.
@ gpauk & DPH,

alignment of vocabulary is an important point, and we addressed it accordingly several times in the past (the last time in June in the Blowtorch thread), where for example the work done by Choisel & Wickelmaier for selecting and using listener panels and useful descriptors as well was mentioned.

Or to quote Bech/Zacharov:
The so-called consensus vocabulary techniques are characterised
by a process in which a group of subjects develops and agrees on the
attributes and their meaning.
(Soren Bech, Nick Zacharov. Perceptual Audio Evaluation–Theory, Method and Application. Wiley & Sons 2006, page 45 .)

But pointing to the importance of alignment/adjustment is imo a different thing than to mark some attributes as useless.
 
What is being used in auditory research is Overall Listening Experience (OLE) as a measure which includes the audio quality as well as all other factors involved in the listening experience

It illustrates the point that there are usually a lot of external factors contributing to the overall experience of listeners and further that it is mainly impossible to control everything within a experiment or otoh that total control of the experimental conditions would lead to results that lack practical relevance because of the totally artificial environment.
 
Sorry, i don´t know why you should be "assimilated" but i´d be interested to know where i have used circular logic. Be specific and i can correct it, throwing a catchphrase in does not help.

It´s a matter of logic, if you like to call it "lip service" , well it´s up to you.
A positive control is a difference that must give a positive result in a test.

This was your original sentence:

Please be more specific; when is a reference sufficiently authoritative so that you will change your prior belief? That´s not a rhetorical question.

What do you mean by "supporting this method of defining positive controls"?
I suggested to use a _sound_ _difference_ as a positive control in test for _sound_ _differences_

It´s more a certain type of lazyness; shortly after the ITU made their recommendations available for free, i began promoting the links to the various publications(means for several years now); MUSHRA and ABC/HR (the latter denotes the test protocol used described in ITU-R BS.1116) are well known methods but you even refused to invest a little effort to use a search engine and preferred to post about word dropping. That does not help either.


Wrong question, to start with. You need to define a hypothesis before looking into a) what kind of test would fit and b) what would be considered proof (or disproof) of the hypothesis. There is no one size fits all approach.

Maybe, but im just asking for the personal criteria that members want to see being met before accepting any "result/procedure" as evidence although they believed something different.

And a few posts later.

There is no test that will ever guarantee the outcome will be universally valid. That is, if the result is "no difference was detected", this doesn't mean there isn't a Buryat shaman in Siberia that would be able to statistically reliable detect a difference.

The ABX results should be understood as "with X% probability, a difference cannot be identified" where X essentially depends on the auditory sample size. The larger the sample, the more certitude (in a probabilistic sense) but X it will never reach 100%, unless you are willing to somehow test the entire world population.

So any ABX test with a negative outcome will not exclude the hypothesis of one or more Golden Ears able to identify a difference. It will only say that if such a Golden Ear exists, it's unlikely that his abilities could be extrapolated to the rest of the mortals.

Reason why ABX tests are in general tilted toward implementing negative controls is that the opposite negative hypothesis is much easier to accept; that is, if a difference is detected, then it is easy to accept that a deaf individual, not hearing a damn thing, exists. This is another way to say that "exceptional claims require exceptional proof" and could easily mathematically put in terms of p and 1-p probabilities.

(...)
No, in general it shouldn´t be understood that way, but especially not given the way you worded your demand/proposal, as nothing indicated that it was meant to examine (or estimate) parameters for the underlying population.
(...)

And when I said your question doesn’t make any sense unless the hypothesis under test is precisely and completely identified you dismissed the idea. Now you are questioning the hypothesis in my example and call me biased.

Obviously i did not dismiss your idea, but tried to politely hint that the level of abstraction wasn´t sufficient.

Rejecting an argument ("proof" can't be defined without specifying an hypothesis) only to conveniently use it later is the best proof of circular logic. And when confronted with a rigorous approach, asking for "personal criteria", only to blame "personal criteria" a few posts later.

Plus that you were unable to produce the slightest authoritative reference (and having the nerve to ask what is that, although it was stated a few posts upstream) to an ABX test that uses "positive controls" as you define them, and unrelated to the hypothesis under test. No amount of verbosity or name dropping is going to change this situation.


And to ask what constitutes "exceptional claims" when examples were given in black and white again a few posts upstream, and in direct reply to you:


I am talking (and I suspect you know it) about "I can hear a difference between mains cable A and mains cable B", "99.999999% pure silver wire is, sound wise, so much better than ordinary copper", "Bybees have improved my system sound, the silence is now black", "I can hear the effect of my new myrtle block raisers", "after rolling the opamp X the bass is so much better", etc... If you think it's worth debating, spending money and designing experiments to prove (or disprove) such stances, then please go ahead and do it; my life is to short to waste it on such junk.


Obviously this discussion no longer makes any sense.
 
@ gpauk & DPH,

alignment of vocabulary is an important point, and we addressed it accordingly several times in the past (the last time in June in the Blowtorch thread), where for example the work done by Choisel & Wickelmaier for selecting and using listener panels and useful descriptors as well was mentioned.

Or to quote Bech/Zacharov:

(Soren Bech, Nick Zacharov. Perceptual Audio Evaluation–Theory, Method and Application. Wiley & Sons 2006, page 45 .)

But pointing to the importance of alignment/adjustment is imo a different thing than to mark some attributes as useless.

Thanks, Jakob (and apologies for earlier misspelling your name as Jacob), for the links. I'll give them a read.

Fair enough on calling me out for my "baby and the bathwater" dismissal; albeit I'm still no closer to what "groove and vibe" mean, especially within the confines of what Dan had to say.
 
It illustrates the point that there are usually a lot of external factors contributing to the overall experience of listeners
Yeah, like the power of suggestion, confirmation bias or bling factor.

and further that it is mainly impossible to control everything within a experiment
What do you mean impossible? When someone claims hearing difference between audio cables, DACs or amps while subjectively comparing in their own room, does the same comparison but level matched double blind in their own room, what control problem is there?

or otoh that total control of the experimental conditions would lead to results that lack practical relevance because of the totally artificial environment.
Recreated sound through electronics is artificial in and of itself. It's like a bird flying through air complaining that if it wasn't for this air drag, it would go faster, when in fact the air is the reason it can fly from the first place.
 
Last edited:
What constitutes "opinions" and "facts" is already at the meta-debate level and is not something that could be addressed here. Descartes and Locke already debated this centuries ago.

Everything is fine, as long as Locke fans don't attempt to stuff "all ideas come from sensation and reflection and that all knowledge is founded on experience" down my throat.

LMAO yourself, BTW.

IDGAF about what Descartes and Locke did centuries ago.

Everything is fine as long as you don't take all of this seriously as opinions are like ********, everybody has one. LMAO....but everyone should have the right to voice there opinion no matter how absurd it may be....If you cant handle opinions you shouldn't be on any sight that allows it and just remain alone in your safe place where no one can offend you. LOL

BTW I am LMAO to my self because it does NOT bother me at all but it probably does bother you. SMDH :rolleyes::rolleyes::rolleyes:
 
Last edited:
....
Plus that you were unable to produce the slightest authoritative reference (and having the nerve to ask what is that, although it was stated a few posts upstream) to an ABX test that uses "positive controls" as you define them, and unrelated to the hypothesis under test. No amount of verbosity or name dropping is going to change this situation.

Of course you miss the whole point of controls - ABX testing is only concerned with one type of 'proof' - ensuring that false positives are eliminated - it has absolutely no concern with ensuring there are no false negatives.

Hence known differences that should normally be audible can easily be missed

What you don't understand is that ABX does NOT have controls within the test for false negatives so the test has unknown sensitivity.

Asking to be shown a ABX test that has such controls is asking for what we are saying is glaringly missing from ABX tests
 
..
What do you mean impossible? When someone claims hearing difference between audio cables, DACs or amps while subjectively comparing in their own room, does the same comparison but level matched double blind in their own room, what control problem is there?
It has been pointed out many times already that there is far more to auditory perception than this simplistic view & many factors need controlling - it is impossible to control all factors

The use of internal controls is one way to get a handle on how sensitive the test/tster was i.e. this is a post-test analysis of results & may mean certain results should be ignored - those that don't recognize the audible differences in the control trials.


Recreated sound through electronics is artificial in and of itself. It's like a bird flying through air complaining that if it wasn't for this air drag, it would go faster, when in fact the air is the reason it can fly from the first place.
Oh dear!!
 
It has been pointed out many times already that there is far more to auditory perception than this simplistic view & many factors need controlling
You never ask about control when someone makes claim about audible difference between audio cables, DACs or amps after subjective casual comparison, right?

- it is impossible to control all factors
You missed the point. The example I wrote is about the variance between subjective casual audio comparison and level matched audio DBT in one's own room. Only variance is level matching and not peeking at the component during test. The rest is equal.

The use of internal controls is one way to get a handle on how sensitive the test/tster was i.e. this is a post-test analysis of results & may mean certain results should be ignored - those that don't recognize the audible differences in the control trials.
Flip back a few pages and look up pre game warm up I posted.
You should get yourself experienced in actual audio DBT. Then your questions and doubts will be cleared.
 
Which audio DBT did that? Can you cite it?

I'll call and raise on that one. "Known differences" is that a belief or a mindset that might be changed?

Are both of you for real? There are known audible differences that can be introduced into one sample - a volume difference, for instance. For god's sake, have you guys ever really done anything with audio testing or even thought about it anything that is posted? Are you too keen to get your retort in to stop & think for even a minute?

I mean this is getting really tiresome & uneeded.
 
You never ask about control when someone makes claim about audible difference between audio cables, DACs or amps after subjective casual comparison, right?
Do you read my posts? I said it many times before - casual, sighted listening does not purport to be anything more than anecdotal but you guys want us to take ABX as a higher standard than this - "a gold standard", yadda, yadda

Pity you can't face up to the requirements needed for such "gold standard" claims

You missed the point. The example I wrote is about the variance between subjective casual audio comparison and level matched audio DBT in one's own room. Only variance is level matching and not peeking at the component during test. The rest is equal.
And this is where you show that you have no idea what you're talking about. Are you talking about ABX test? How many trials which are correct will you accept as 'proof' that an audible difference exists?

Flip back a few pages and look up pre game warm up I posted.
You should get yourself experienced in actual audio DBT. Then your questions and doubts will be cleared.
"pre game warm up" has a name - has a recognized term in blind testing literature - it's called training & pre-testing. Certain people will be excused from the test based on this training & pre-testing That's only one factor that is recommended practice for blind testing.

And I said "The use of internal controls is one way to get a handle on how sensitive the test/tester was i.e. this is a post-test analysis of results & may mean certain results should be ignored - those that don't recognize the audible differences in the control trials."

Just because someone passes pre-testing doesn't mean they are then focused during the many trials that make up the test.

Rather than telling me to "get yourself experienced in actual audio DBT." you need to find out what a valid DBT entails instead of the half-baked notions you clearly suffer from !!
 
Yes, it's not known as audible until a threshold is established. Please provide a reference to any studies.

You don't seem to be able to follow this:
- a control is a trial that has either A or B adjusted by a factor which is known to be audible (according to the established & accepted thresholds of audibility) - it may be a vol difference, a particular distortion difference or whatever.

As I said this is well known in blind testing & answering these same questions over & over is tiresome. Take some time out to think about this rather than rush to reply
 
Last edited:
You don't seem to be able to follow this:
- a control is a trial that has either A or B adjusted by a factor which is known to be audible (according to the established & accepted thresholds of audibility) - it may be a vol difference, a particular distortion difference or whatever.

As I said this is well known in blind testing & answering these same questions over & over is tiresome. Take some time out to think about this rather than rush to reply

OK so we stick with the usual 1dB, just curious. If you make this too hard we won't all get to play.
 
Status
Not open for further replies.