What kind of evidence do you consider as sufficient?

Status
Not open for further replies.
I'm pretty sure Jacob was alluding to the pressure to perform by someone who believes something is audible under non-testing conditions, then "failing" to discern that effect under testing.

I'm pretty sure that the posts to which he was responding were about that (including mine), but I was fascinated by his use of a false dichotomy which distinguished perception from results in tests of perception. If results in blind auditory perception tests are to be distinguished from auditory perception, then, once again, what are we talking about?
 
Can you please explain the difference between "get a positive ABX result" vs "perceives it so"? Are you suggesting that ABX (or other blind protocol) results are predicated on something other than auditory perception?

The usual tests have to rely on the answers of the participants. In this regard any test result depends on the subjective internal jugdement process.

Oohahsi et al. (experiments for the so-called Hypersonic Brain Effect) tried to get additional more (hopefully) objective informations by usage of EEG and PET/Scans, the more recent approach is to use fMRI.

Having that said, let´s remember that we only count hits (or misses) to the trial question but don´t know if that really reflects what is/was percepted, because we can´t look into the heads/brains of our listeners.

We know that testing the same sensory difference in different test protocols using the same participants leads to different results, so obviously the results are not just reflecting the "raw" auditory perception.
To keep it short, the assumption that "blind" test schemes somehow ensure that the results only depend on the auditory perception is most likely wrong.

In an another thread we already discussed the plethora of possible bias effect still at work in "blind" tests.

I'm pretty sure that the posts to which he was responding were about that (including mine), but I was fascinated by his use of a false dichotomy which distinguished perception from results in tests of perception.

I hope you realize by now that it isn´t a "false dichotomy".....

If results in blind auditory perception tests are to be distinguished from auditory perception, then, once again, what are we talking about?

Talking about the difficulties to do tests that are _really_ objective, valid and reliable, furthermore about which way to do better experiments, about what is considered to be sufficiental evidence and obviously about a lot more that would be better done in other threads, but that´s (forum)life. :)
 
<snip> As far as I'm concerned this whole discussion of not being able to sample exactly the wavefront impinging on your ears is a red herring. At some point the audio information is contained in two 2D signals (amplitude vs time) and the information they can contain is limited by fundamental principles.

Could it be that you´ve forgot for a moment about the last 40 years of research on HTRFs and binaural recording? :confused:

So, what we listen to is already a mechanically processed version of the orginal sound field and it is further processed by our auditory "apparatus".

Our brain is used to work with this individually preprocessed material and that´s the reason why binaural recordings done with individualized dummy heads/torsi are so stunningly "realistic" .

In the usual recording setup none of that is used (beside maybe the spacing) and it was imo one of the most surprising detections that similar perception is possible although the reproduced sound field (two channel stereophonic setup) was vastly different from the original sound field.

It works because our brain constantly tries to construct a perception from the cues it gets from the reproduction that is compatible to our experiences.
 
Back to the subject of the so far 'hypothetical' ABX testing protocol there are many considerations that would serve to drive null results when testing for 'minor' differences.

Included in this would be BG noises like HVAC, and the fact of multiple other subjects in the room, all breathing and the mere presence of other subjects altering the sound field.....also, who gets to sit in the optimal spot ?.
The time of day would be another factor...BG acoustic/seismic noise, AC supply noise and general RF noise all change according to the time of day.
Also the listening mode of the subject is variable according to time of day....ever noticed how late night listening can reveal system nuances that are not noticed during daytime listening.

Anybody care to detail their experiences of formal ABX testing ?.

Dan.

I gave an example of a real world ABX test on the other thread to show what's involved in such testing & bear in mind the participant was an audio engineer. He found it extremely difficult but well worth reading his experience which I'll reproduce just his summary post f his experience here:
"Keeping my attention focused for a proper aural listening posture is brutal. It is VERY easy to drift into listening for frequency domains–which is usually the most productive approach when recording and mixing. Instead I try to focus on depth of the soundstage, the sound picture I think I can hear. The more 3D it seems, the better."

Program material is crucial. Anything that did not pass through the air on the way to the recording material, like ITB synth tracks, I'm completely unable to detect; only live acoustic sources give me anything to work with. So for lots of published material, sample rates really don't matter–and they surely don't matter to me for that material. However, this result is also strong support for a claim that I'm detecting a phenomenon of pure sample rate/word length difference, and not just incidental coloration induced by processing. The latter should be detectable on all program material with sufficient freq content.

Also, these differences ARE small, and hard to detect. I did note that I was able to speed up my decision process as time went on, but only gradually. It's a difference that's analogous to the difference between a picture just barely out of focus, and one that's sharp focused throughout–a holistic impression. For casual purposes, a picture that focused “enough” will do–in Marketing, that's ‘satisficing’. But of course I always want more.
It took me a **lot** of training. I listened for a dozen wrong things before I settled on the aspects below.

The difference I hear is NOT tonal quality (I certainly don't claim to hear above 22 kHz). I would describe it as spatial depth, spatial precision, spatial detail. The higher resolution file seems to me to have a dimensional soundstage that is in *slightly* better focus. I have to actively concentrate on NOT looking for freq balance and tonal differences, as those will lead you astray every time. I actively try to visualize the entire soundstage and place every musical element in it. When I do that, I can get the difference. It's *very* easy to drift into mix engineer mode and start listening for timbres–this ruins the series every time. Half the battle is just concentrating on spatial perception ONLY

I initially found training my ears to find a difference very difficult. It's *very* easy to go toward listening for tonal changes, which does not help. I get reliable results only when trying to visualize spatial detail and soundstage size, and I tend to get results in streaks. I get distracted by imaginary tonal differences, and have to get back on track by concentrating only on the perceived space and accuracy of the soundstage image.

Again, remember this guy is an sound engineer so works with & relies on his hearing every day - I presume he has more listening experience than the average audiophile.

He also achieved some positive ABX tests for differentiating jitter levels & had to train for a totally different audible cue
 
....I gave an example of a real world ABX test on the other thread to show what's involved in such testing & bear in mind the participant was an audio engineer.....
Thanks for that. I agree, listening for precision and extent (clarity) of 3D position/depth information is the first thing to listen for. Another is noting the groove/vibe/flow of the reproduced audio and how it affects oneself...self involvement/enjoyment/relaxedness etc. This is probably fairly difficult in a group situation.

Dan.
 
@scottjoplin,

<snip> Anything else that is going on is in the listener's head, and I'm not interested in what goes on in people's heads, usually because they don't know, so how can I? ;) That does not mean I'm not interested in psychoacoustics, I am, I just don't think it's particularly relevant

As the more modern definition of the term "Psychoacoustics" coined by Blauert in the 1960s is to examine if a sound event leads to an auditory impression (in German it´s Hörereignis, so maybe audiotory event is a better translation?) and if different sound events lead to different audiotory impressions. That covers all from JNDs to DLs on every possible sound event.

So psychoacoustics is relevant especially as we are talking about the instruments to find out about listening impressions. :)

Traditionally psychoacoustic was more interested in the basic mechanisms and did not use multidimensional stimuli (aka music) to examine the hearing sense; today that has changed.

I'm obviously a bit dense, where is the truth if not in an accurate reproduction of the signal?

That´s another long ongoing discussion, as there is no universal agreement if the "accurate reproduction" should be related to the original event or to an arbitrarily choosen (a bit exaggerated of course) intermediate stadium (i.e. the recorded content that nobody really knows)
 
Last edited:
As the more modern definition of the term "Psychoacoustics" coined by Blauert in the 1960s is to examine if a sound event leads to an auditory impression (in German it´s Hörereignis, so maybe audiotory event is a better translation?) and if different sound events lead to different audiotory impressions. That covers all from JNDs to DLs on every possible sound event.

So psychoacoustics is relevant especially as we are talking about the instruments to find out about listening impressions. :)

It always strikes me that what people mean who think psychoacoustics is irrelevant to this audio hobby, is that psychoacoustics is what people 'imagine' they hear :rolleyes:
 
As the more modern definition of the term "Psychoacoustics" coined by Blauert in the 1960s is to examine if a sound event leads to an auditory impression (in German it´s Hörereignis, so maybe audiotory event is a better translation?) and if different sound events lead to different audiotory impressions. That covers all from JNDs to DLs on every possible sound event.

So psychoacoustics is relevant especially as we are talking about the instruments to find out about listening impressions. :)

Traditionally psychoacoustic was more interested in the basic mechanisms and did not use multidimensional stimuli (aka music) to examine the hearing sense; today that has changed.
Thank you
 
It is not possible to argue and debate the issues in this thread to an agreeable conclusion. Everybody knows that, right? And everybody should know that people on both sides who have already firmly made up their minds are not going to be persuaded to change here and now, perhaps not ever, perhaps no matter what experimental evidence were to be produced in the future. So, what is the point of arguing?
 
@Jakob2, Maybe you are putting the cart before the horse by asking what people would need to see to be convinced. You are asking people to play a game of imagination that people tend to be very bad at. There is research showing that people are very bad at imagining the future and how they will feel or react if this or that comes to pass.

Nice picture of the horse not pulling but trying to shove.....

I think at least the poster who demand that "golden ears" should provide evidence by doing "blind" tests must know what they would accept as evidence. I have assumed to see the "extraordinary claims need extraordinary proof/evidence" argument for example.

In a broader sense it is imo worth to think about what could constitute sufficient evidence.

As you´ve mentioned Kahneman & Tversky, it´s a good example how important it is to remember that we are always more willing to accept confirming information than to accept contradictionary. Kahneman himself pointed out that he should have known better but was nevertheless trapped in the priming case.

<snip> We also ought to look into what kind if hearing mistakes people make when they think they hear a difference and in reality there isn't one. My bet would be that the errors are mostly systematic, that is, everyone tends to make the exact same errors.

That´s an interesting question as it points directly to bias effects. As usual there are already some experimental results (although again mainly from food sensory tests) where the participants where asked for their preferences when in reality testing the exact same stimulus twice or even more times.

If a "no preference/difference" answer option is included, the proportion of correct answers (means number of correct no preference/difference answers) is always the lowest when probing the exact same stimulus. Stable phenomenon over different product categories (and well known in audio tests) and even a crossnational effect.

But it is mainly noticed in multidimensional evaluations not so in one-dimensional experiments.
 
...
That´s an interesting question as it points directly to bias effects. As usual there are already some experimental results (although again mainly from food sensory tests) where the participants where asked for their preferences when in reality testing the exact same stimulus twice or even more times.

If a "no preference/difference" answer option is included, the proportion of correct answers (means number of correct no preference/difference answers) is always the lowest when probing the exact same stimulus. Stable phenomenon over different product categories (and well known in audio tests) and even a crossnational effect.

But it is mainly noticed in multidimensional evaluations not so in one-dimensional experiments.

Yes, if a test setup is testing the difference between A & B, the participants naturally assume that there is a difference & they are trying to find it - so even if A is really the same as B, they will be psychologically primed to hear differences (psychologically, we want to be right) - when the option of no difference is included as a possible answer, it changes this stress/pressure
 

TNT

Member
Joined 2003
Paid Member
You seem to lack the overall system view to discuss the real issues and constantly end up in irrelevant details e.g. the one you refer to below.

You do realize that the frequency spectrum at the eardrum is different for everybody due to different pinna & ear canals shapes causing different resonances?

You would do well to read the research of David Greisinger on binaural reproduction?
Have a look at "Accurate reproduction of non-individual binaural recordings without head tracking through individual headphone equalization"

Look at slides 11 onwards to see the how different are the frequency spectrum measurements taken at the eardrums of different 'normal hearing' participants!

Yes, we all have a different frequency spectrum at the eardrums & our analytic engine (auditory processing) equalizes this difference so that we coalesce to a similar relationship with the auditory world.
 
You seem to lack the overall system view to discuss the real issues and constantly end up in irrelevant details e.g. the one you refer to below.
Please educate me - what are the "real issues"?

I was imply pointing out Greisinger's research into using dummy heads for binaural recordings as being less than optimal for a realistic eperience

You quoted planet10's post in your post & seemed to be replying to this but maybe you weren't?
 
Yes, if a test setup is testing the difference between A & B, the participants naturally assume that there is a difference & they are trying to find it - so even if A is really the same as B, they will be psychologically primed to hear differences (psychologically, we want to be right) - when the option of no difference is included as a possible answer, it changes this stress/pressure

Isn't that the advantage of ABX over AB?
 
Isn't that the advantage of ABX over AB?

As usual it depends, as you don´t need to include a tie (i.e. the no difference/no preference option) option in an A/B test - but if you do you get another/an additional analysis choice.

Basically the ABX test is a pure test for difference so you can´t use it for preference tests.
I have to check if there were comparisons between ABX and A/B "same/different" test protocols already done.
 
Status
Not open for further replies.