I didn't know that 6/10 in 5 blocks is the same as a single 30/50 in terms of probability theory. It would be possible to make a sufficiently long run of the single ABX test, say 1 day or more, and see if one gets 30/50. I am sorry but I am not convinced that it is the same case as to repeat the 10 attempts 5 times.
I was more trying to make the example of have someone repeat the same experiment of a 10-trial ABX each day for 5 days, versus, say, having said person have a break between each 10-trial and do them in one day (where fatigue/etc would be a real concern).
If the trials are truly independent (they aren't perfectly, but that's a big difficulty in characterizing how well someone can do in such a test), then each of the trials is cumulative. I.e. 5x 10-trials = 50 independent trials.
Last edited:
I think this is why a change is often perceived as an improvement. How is one ever to tell if something is better and not merely a preference, isn't a preference all any kind of listening test can ultimately do, after showing a difference? I thought the idea of this discussion was to both measure and listen to find the missing link?
-RNM
As I wrote before, I am not sure that 6/10 over 5 runs is the same as 30/50. However, we have a guy who scored 20/30 in a single run.
Do you like tube distortion? - listening test
I think this is why a change is often perceived as an improvement. How is one ever to tell if something is better and not merely a preference, isn't a preference all any kind of listening test can ultimately do, after showing a difference? I thought the idea of this discussion was to both measure and listen to find the missing link?
The idea is to both measure and listen. However, I don't have as much test equipment as would be optimal. It's a hobby, after all. Fortunately, I don't like the sound of distortion, except for electric guitars (and maybe one other exception).
In the case of replacing DAC-1 with DAC-3, I already knew the rated distortion was much lower for DAC-3, so confusion not likely. In cases when I don't have specs for something and I am stuck with mostly relying on listening, such as when I work on my dac project, I still have DAC-3 as a reference to work towards. In addition, I listen to things like cymbals, drums, guitars, synths, vocals, etc., and I happen to have all those things right here to listen to. Makes it pretty easy to know if changes are in the right direction.
What is interesting, IME, is that some distortion, in some circumstances, can make brass horns sound much more real. Technically speaking, it sounds to me like part of the sound of the instruments is being reproduced, and part is being synthesized. The result can be a very true to life sound which is very enjoyable to listen to. The clue that there is something wrong is when vocals and other instruments come in. It can be fun while the illusion lasts though.
My point in saying all the above is that usually things all work out to the correct end. Since you asked about it, I thought maybe you would find a more detailed answer interesting. Sorry if I rambled on too long.
Last edited:
It is not the newness of ideas which is the problem. It is the obvious daftness which is the problem. Some of these 'new' ideas are not even new, but merely recycling old nonsense such as 'military secret' or 'trade secret'.john curl said:
Obviously some people will feel personally hurt if they publish an idea, and then see it shot down in flames on the grounds that it is obviously false. What should happen then is that they either admit they made a mistake, or they should come back with reasoned arguments to put the critics straight. What is more likely to happen is that they throw a tantrum and get banned, or insult their critics (too poor, too deaf, too stupid etc.) while continuing to peddle their nonsense and confusing newbies; some of the newbies turn into fanboys and encourage them.
@PMA,
could be or could be not. How could you know if not being able to "look into the listener´s head"?
The analysis/result just tells you that probability to get such a result by random guessing is quite large, so the result is compatible with the null hypothesis.
But if the listener was really just randomly guessing is an open question.
From the analysis point of view it indeed doesn´t matter.
You are analysing the observed results with a statistical does _under_ the assumption that the null hypothesis is true. The null hypothesis is "random guessing" or more formally p = 0.5 .
Random guessing is in this statistical sense like flipping a perfect coin.
It doesn´t matter if you flipp it 30 times in a row today or if you flip it 10 times today and another 20 times next week; a coin flip is a coin flip, each trial (and its result) is completely independent from the next one.
As DPH already mentioned, if these model assumption are true in reality is often an interesting question. From a practical or operational point of view results from longer time spans might carry more weight, but it depends on the environmental/confounding factors and it must be ensured that the DUT don´t change their parameters over time.
could be or could be not. How could you know if not being able to "look into the listener´s head"?
The analysis/result just tells you that probability to get such a result by random guessing is quite large, so the result is compatible with the null hypothesis.
But if the listener was really just randomly guessing is an open question.
From the analysis point of view it indeed doesn´t matter.
You are analysing the observed results with a statistical does _under_ the assumption that the null hypothesis is true. The null hypothesis is "random guessing" or more formally p = 0.5 .
Random guessing is in this statistical sense like flipping a perfect coin.
It doesn´t matter if you flipp it 30 times in a row today or if you flip it 10 times today and another 20 times next week; a coin flip is a coin flip, each trial (and its result) is completely independent from the next one.
As DPH already mentioned, if these model assumption are true in reality is often an interesting question. From a practical or operational point of view results from longer time spans might carry more weight, but it depends on the environmental/confounding factors and it must be ensured that the DUT don´t change their parameters over time.
Last edited:
I´m not sure it these both approaches are contradictionary, as it depends on the goals.
Sometimes efficiency plays a role so using a fast switching/listening approach needs less time to get a result. That such test scheme does not reflect the "usual/casul" listening condition should be a given.
If the prediciton from a model is that no difference can be percepted but a listener feels differently by using his normal listening approach and is able to corrobate it by a fast switching test - otherwise good planned and executed - (even to prelicate it) then there is hard evidence to reject the null hypothesis
and to accept the alternative hypothesis (i.e. a difference is perceptable).
But beside these considerations there might exist individual differences, some people might easier accomodate to the long term listening approach, while others might like the fast procdedure.
If otherwise results from fast procedures really reflect the practical relevance is another question; i think we´ve already mentionend the importance of external validity and practical relevance in discussions/threads before.
Our preamplifier test was more a longer time span approach, as the listeners used their samples for something between two days to roughly two weeks.
Fast switching tests were not possible as both units had to be rewired manually, so any "switch" would usually take a minimum of 30 - 60s.
What preamplifier test was that? Fast switching could have been arranged if the will was there, I think Mark had a few suggestions a while back.
I have an experience with the preamp test method as you are describing. There was a similar test made here, with 3 preamps and a group of listeners, who all were convinced that the preamps do sound different. Then, there was a blind test with preamps switched (exchanged) manually. There was an absolute null result of this test, participants were unable to tell which of the preamps was connected.
I, personally, am deeply convinced that the Foobar ABX test with the files, as I am posting in another threads, is much more revealing and has much more resolution to a real difference than the "physical exchange" based DBT. The listener may take as much time as he wants and may concentrate on a short passage, if he wants, which is impossible under the test conditions you are mentioning and with which I do have quite a lot of experience.
Are you saying that a physically swapping DBT of 3 preamplifiers showed a they could be distinguished?
If not then you are not comparing apples with apples - Jakob2 did a long term DBT (up to two weeks) with people which gave consistent (but maybe not statistically significant) results.
Have you ever done such a test?
Someone on Facebook turned me on to a U-tube presentation of John Atkinson at RMAF this year. Unfortunately, I missed it when I was there, but in listening to it yesterday it gave me hope! Yes, other people hear what I hear. Measure what I measure, etc. I recommend that everybody listen to this talk which is only about one month old. I would put up the link, but I don't know how to find it on my regular computer. I would hope someone would help me with this if it is easy enough to do.
I have read quite a lot of this thread - a couple of observations ........
Yes, there are some real experts here, or shall I say experts in their own narrow field of endeavour who consistently fail to recognise that real knowledge comes from confluence flowing in from what may appear at first to be wholly unrelated matters, but eventually, the penny drops. This is why for me, I seldom get into arguments, no point, no fun and life is short. But some people like to argue for arguments sake - let them.
The other thing is, how many here have actually made a musical instrument with their own hands, and then gone on to learn how to play it? It is an experience that changes everything about any argument involving fidelity of sound.
Yes, there are some real experts here, or shall I say experts in their own narrow field of endeavour who consistently fail to recognise that real knowledge comes from confluence flowing in from what may appear at first to be wholly unrelated matters, but eventually, the penny drops. This is why for me, I seldom get into arguments, no point, no fun and life is short. But some people like to argue for arguments sake - let them.
The other thing is, how many here have actually made a musical instrument with their own hands, and then gone on to learn how to play it? It is an experience that changes everything about any argument involving fidelity of sound.
Last edited:
I think he is saying that an unsighted swap made them indistinguishable, even though the participants were confidant beforehand that they would be distinguishable.mmerrill99 said:
On the contrary, it is an experience which is almost entirely irrelevant to the issue of faithful sound reproduction. You seem to be confusing the painter (and his brush maker) with the photographer. Both have important skills, but very different and used for different purposes.tapestryofsound said:
How ironic. I have spent my whole life as a working professional photographer and am now an optical painter in my own right.
Your analogy makes no sense at all. None.
Completely and utterly wrong again.
I am considered to be a master photographer who has done absolutely everything considered humanly possible with the medium over a lifetime. Not an idle boast, but a fact.
That's well in excess of 30,000 hours of direct hands on experience, and I can assure you that there is a considerable crossover of knowledge between the worlds of sound and vision.
Keep calm, ask me a question - any question, and I will carry on.
- Status
- Not open for further replies.