What kind of evidence do you consider as sufficient?

Status
Not open for further replies.
Yes you are still arguing & after interminable posts misinterpreting what I clearly said & arguing as a result, you are now moving into stage 2 argumentum

Does anyone else agree with this? I asked a lot of clarification questions and several times for you or someone else to propose a process. I said take the ITU spec as gospel if you want but do the work, I'm not. I even said you make the rules, but take the test.

If you take offense because I said I trust the spec precludes peeking or cheating that's your problem.
 
Does anyone else agree with this? I asked a lot of clarification questions and several times for you or someone else to propose a process. I said take the ITU spec as gospel if you want but do the work, I'm not. I even said you make the rules, but take the test.

If you take offense because I said I trust the spec precludes peeking or cheating that's your problem.

Maybe it's your communication style or maybe it's my reading of your posts but they generally seem to me confrontational & usually peripheral.

I believe Jakob has told you what the process is for a specific request you made of him?

What more do you want? This request for a process - what does that mean?
 
Will you guys please stop the word salad and propose something tangible and in detail. In case you have not figured it out no one is arguing anymore. You can discard my results ahead of time I'm only interested in seeing others claim's substantiated, I'll even let you set the rules.

Thanks for the friendly comment.

This was your question...err ....request:

So I ask with respect to the two DAC's set to level match to .01dB what do you propose as controls and how are they used?

and i replied:

I assume you want to test the two DACs with music samples; so for the normal trials you use the same music sample as stimulus for both DACs, but for the control trials you prepare music samples with a slightly higher level (around 0.2 dB and lower) which you are using in the control trials. One DAC gets fed with the music sample of the normal level while the other DAC gets fed with the music sample of slightly higher level. But your listeners will still answer the same question as before.

There are two possibilities to choose from; as usual both have their advantages and disadvantages but principially you can use control trials within your normal test routine (requires to use more trials overall) or you can run a completely seperated test (with the same number of trials as you normally would use) with the control stimulus/stimuli against the normal/unaltered stimulus.

The information that you requested is included as i proposed a specific control and told you how to use it (even gave you a choice).

Not "tangible"?? Not detailed enough? Ok let´s see, in a following answer to mmerrill99 i added:

So i would choose to include controls when sort of (hedonic) rating is used and several stimuli will be rated, but use seperated tests when only two items are under test and the same stimulus is used in every trial.
If different stimuli are used, the controls can be used again within the "normal" trials.

Still not "tangible" and not detailed? I am at a loss.

But let´s see what you replied to my first answer to your request:
That leaves me out from the start. BTW I don't see a 0.2dB level change as fitting on their 5 point annoyance scale.

WHAT? That leaves you out from the start, but since when was it about you and how could anybody have known that it is?
And who did tell you that you have to use the ABC/HR?
Why are you talking later about "peeking" and "cheating"?

This is about controlled listening tests including the "double blind property" ....

May i politely ask for returning the favour and answering my question:
if an ABX test is used to test a given fixed sensory difference but the proportion of correct answers in the ABX is significantly lower than in an A/B test used to test the same given fixed sensory difference, could it mean (everything else equal, especially the number of trials) that the ABX has a hidden flaw?
 
Last edited:
I just took a close listen to Foobar ABX.
I have two files, '1' and '2' stored on a thumbdrive, or on laptop HD which playback as 'A' and 'B' respectively when running Foobar ABX.
Now, with ABX comparator running but paused/stopped, it is possible to listen to 1 or 2 from the playlist and then switch to the comparator applet and select A or B playback.
The kicker I find is that A is different to 1, and B is different to 2, but the relative difference between 1 and 2, or A and B remains.
It seems that the 'ABX preparing files...' routine adds subtle noise thereby somewhat altering the test files playback quality.
Despite this I was able to score 8/8, and then 9/10 and 9/10....I got bored and went to bed.


So, this Foobar ABX quirk does alter the sensitivity of the test, but not necessarily the outcome.
Is there another ABX freeware test that does not rewrite the sample files to HD, ie plays directly from stored files ?.


Dan.
 
WHAT? That leaves you out from the start, but since when was it about you and how could anybody have known that it is?

You deleted my quote of the ITU spec about expert listeners, I'm not, that leaves me out and yes it's not about me.

Let me clarify, pick a selection of appropriate music to test DAC's and the length of each selection (I have no idea how to do this). Pick your control, I disagree on a 0.2dB uniform level change as an impairment in the spirit of the spec but, so what, use it anyway. Choose some protocol for the trials, in other words propose a complete flow chart for this test. I'm sorry if I was not clear, I don't mean to question or challenge most of what you propose just put it in an actionable form. You choose, you make the rules, but take the test.

BTW as an impairment I would rather suggest a serious down sampling to fewer bits or something that actually removes information.

As to answering your question I don't know, I would have to defer to people for whom the answer is critical. Unfortunately the industry is in a sad state if I can believe the MQA crowd never did any critical listening tests as the folklore states. I can say our listening room has not been used in years, and sorry to disappoint the content is 99% of the importance for me.
 
Last edited:
....I'm sorry if I was not clear, I don't mean to question or challenge most of what you propose just put it in an actionable form. You choose, you make the rules, but take the test.

I think it might be your posting style but this certainly seems to be a challenge to me.

AFAIK, this thread was a discussion about what evidence is considered sufficient - not a challenge to do tests?

What is it you now understand about positive controls & why are you challenging Jakob to take a test?
 
...

As to answering your question I don't know, I would have to defer to people for whom the answer is critical. Unfortunately the industry is in a sad state if I can believe the MQA crowd never did any critical listening tests as the folklore states. I can say our listening room has not been used in years, and sorry to disappoint the content is 99% of the importance for me.

I see you're dodging the question asked. As I said if you are not interested in the topic or in truthfully engaging on this thread, why not leave?
 
I think it might be your posting style but this certainly seems to be a challenge to me.

AFAIK, this thread was a discussion about what evidence is considered sufficient - not a challenge to do tests?

Oh is that all, I consider it sufficient evidence if a participant freely accepts all the parameters of the test and then takes it. I have stated this for years even for casual tests if the person making a claim honestly feels they don't know. If the peeking/cheating aspect bothers you, you have not been around.

If you take a simple challenge as a personal attack I don't know what to say. Is this thread a simple intellectual exercise or is there a goal to rid the web of the "forum ABX tests" and replace them with something less controversial?
 
Oh is that all, I consider it sufficient evidence if a participant freely accepts all the parameters of the test and then takes it. I have stated this for years even for casual tests if the person making a claim honestly feels they don't know. If the peeking/cheating aspect bothers you, you have not been around.

If you take a simple challenge as a personal attack I don't know what to say. Is this thread a simple intellectual exercise or is there a goal to rid the web of the "forum ABX tests" and replace them with something less controversial?
Again, what are you talking about?
You stated you don't mean to challenge & in the next breath you do challenge :confused:

What are you talking about - where did I say this was a personal attack? You seem to be ranting now :yikes:

If you want to set up a challenge why not start a thread to do so - this thread is discussing what's considered sufficient evidence !
 
I see you're dodging the question asked. As I said if you are not interested in the topic or in truthfully engaging on this thread, why not leave?

I don't choose to, you have the ignore list. I'm very interested in the topic, I would love to have a test accepted by everyone here for, "I changed my crossover caps to $400 teflon ones and my wife ran out of the kitchen in surprise".
 
Please do a search on Audacity and 24 bit audio. I have helped numerous people here discover that many programs do not support 24/96 over USB and give no warnings of such.

Who's being instantly hostile now?

You need to calm down!

This is a bullet point on the front page of the software
  • Will play high sample rates and bit-depths (limited by your audio system only) but make sure your operating system isn't downsampling. E.g. in Windows you must set your fancy sound interface as the default interface, and you need to choose the default sample rate / bit depth to use for the device. Lacinato ABX/Shootouter will not change the sample rate or bit depth as it plays: it is at the mercy of your operating system. You can set your OS to the highest rate you will be testing, but if you play files of a lower sampling rate, upsampling will occur.
 
I don't choose to, you have the ignore list. I'm very interested in the topic, I would love to have a test accepted by everyone here for, "I changed my crossover caps to $400 teflon ones and my wife ran out of the kitchen in surprise".

yep, it's obvious you are on a campaign with your agenda fully on display


Unbelievable, so Mr. Wurcer has a hidden agenda to discourage using $400 teflon capacitors in crossovers.


I need to sleep on this :rofl:.
 
Again, what are you talking about?
You stated you don't mean to challenge & in the next breath you do challenge :confused:

What challenge? Lets leave it at sufficient evidence = some participants take a test for which they agree on all parameters. If the test is not taken there is no sufficient evidence.

I've agreed to all your conditions, use the ITU spec, pick any controls you want, no complaints from me.
 
Status
Not open for further replies.