Nonsence.
You can do perfectly scientificaly valid ABX test on your own with the propper software (E.g. Foobar2000 with ABX). Then when you get say 9 out of 10, you can say: Hey guy´s I got a positive result when ABXing this device, can other people verify this?
If you're referring to the BBE discussion, that's a very interesting interpretation. 🙂
Ta, we do our best to keep you all happy.
I've highligted the bit above that I feel that this is a very important point in this thread. I don't 100% agree with the rest of the quote, but didn't want to quote out of context 🙂
again I think a very good point. I think that the great intellectual pride and self delusion is a little over the top, but that's me 🙂
As many others have also said working together as a team to understand the whole picture, warts and all is the only way for progress to be made. There will always be the lunatic fringe on both sides, that is unavoidable. It is the coming together of those who are open to possibilities (and are not afraid to find out that they may be wrong) that will ultimately bridge the divide between the two camps.
I can still remember to this day an author for Electronics Australia who continually harped on about how ALL CD players sounded the same because they were all digital and all perfectly reproduced 20-20Khz, so there was no point in looking at anything but the cheapest model. I was young and naive at the time, didn't know anything about how the things worked, so took it at face value. Man did I get a shock when I bought a new Denon CD player for my parents for Christmas (a few generations newer than my pioneer PD5100) It blew it into the dust. and I still up until that point believed that all CD players sounded the same.
Oh and one other thing. I think many a time an objective leaning persons request for evidence is taken as meaning "If you can't provide evidence then it doesn't exist" which is not I believe (in the majority of cases) the intent at all. More likely it means "if you can't produce evidence, I'll accept that you *might* hear something, but I will not accept it as fact".
Tony.
Last edited:
If you're going to be rude to Fran, at least get the spelling of your pejorative right.
Guys let's keep things civil..I would like tot propose an experiment that might satisfy both objectivists and subjectivists. Unfortunately it requires a rather complex hardware arrangement.....
What if we would take a supposingly perfect amp such as e.g. John Curls Blowtorch.
We will have to use the exactly same amp, but dress it up as (first test) a Blowtorch, then (secondl test) dress up the Blowtorch as an el cheapo Technics amps of sorts, than (third test) as something in between quality wise. To meet the criticism on ABX of the subjectivists, the reviewer may take a very long time, think several weeks,to assess the amp. This is to meet the "stress of choice"argument.
I would be curious to see whether the reviewer would be able to detect that he/she has been listening to the very same amp for all those weeks/months.
Ay ay what a pity such a setup will probably never be realized....
Eelco
What if we would take a supposingly perfect amp such as e.g. John Curls Blowtorch.
We will have to use the exactly same amp, but dress it up as (first test) a Blowtorch, then (secondl test) dress up the Blowtorch as an el cheapo Technics amps of sorts, than (third test) as something in between quality wise. To meet the criticism on ABX of the subjectivists, the reviewer may take a very long time, think several weeks,to assess the amp. This is to meet the "stress of choice"argument.
I would be curious to see whether the reviewer would be able to detect that he/she has been listening to the very same amp for all those weeks/months.
Ay ay what a pity such a setup will probably never be realized....
Eelco
Nope, don't even know what a BBE is!!
But seriously, as I read more and more of this thread, it seems that most accept that neither measurements nor ears tell the whole story - you must combine most together to see the whole.
Now, if most here would agree with that statement, then maybe we could prevent a bad situation occurring:
Dirk95100: no doubt for some things, there is software available. But I don't think that will suit in every situation - how can you run a SW test for say a new output cap in a phonostage? I'm not going to build 2 phonostages, with different output caps am I? No, I'm going to listen to one, swap, then listen again with at least time to desolder and resolder in between. What about a new tonearm, or platter, or shunt vs LM317 PSU?
I don't disagree with you, I just don't think its as easy to do as you suggest.
Fran
Hi,
Well, I used to modify/build my creations into commercial cases, usually of stuff was cheap and/or old and throw-away. I turned up with that for any number of tests and shoot-outs.
It was fun to take them to dealers, compare to very expensive gear and have the cheapo beat (even to the dealers ears) their High Quid stuff...
The dealers never kew what was really in there.
Ciao T
Well, I used to modify/build my creations into commercial cases, usually of stuff was cheap and/or old and throw-away. I turned up with that for any number of tests and shoot-outs.
It was fun to take them to dealers, compare to very expensive gear and have the cheapo beat (even to the dealers ears) their High Quid stuff...
The dealers never kew what was really in there.
Ciao T
Hi,
Really? Interesting.
Now, please, could you elaborate why you would suggest that scientific validity is produced by "9 in 10"?
And does "scientifically valid" mean "having statistical power to allow generalisation"?
Ciao T
Really? Interesting.
Now, please, could you elaborate why you would suggest that scientific validity is produced by "9 in 10"?
And does "scientifically valid" mean "having statistical power to allow generalisation"?
Ciao T
Well 5 out 5 is a coin flip, so this means no audible difference.
9 out of 10 good means better than a coin flip. And it means that you have a high probabuility that you actualy realy hear a difference. But 100 out of a 100 would be better. I´m pretty sure you can do the math yourself.
I have never made that claim.
But if other people can reproduce the results we are a step further.
Questions:
What would be a scientificaly valid listening test ording to you?
Do you know any scientificaly valid listening test (ording to you) that, for instance, 2 cd players that have simular measurements (just an example, pick anything you like) and have shown a clear audible difference?
I´m not able to eedit my comment so I do it like this.
ording should be acording.
simular measurements should be all simular significant measurements. (the temerature is not of much interest)
ording should be acording.
simular measurements should be all simular significant measurements. (the temerature is not of much interest)
Interesting topic,
What is "Good" sound? What one person likes another will not like!
I don't think this is just HIFI related. Some people like a "soft" sound less treble or more bass etc.
I think its interesting when someone buys some HIFI gear and then takes it back because they can't live with it....It's two harsh etc.
I think it would be interesting to hear someone play live music, and then play the recording back on the same stage....Lets see how close a system that measures "perfect" sounds to the real thing. I bet it's not even close!
Could you tell the two apart in a blind test....
Then play another "perfect" system and do the same. Does one violin sound the same as another on the HIFI or can you tell who is playing?
Theoretically if a measurement can set the bench mark output of an amp IE perfect, they would all be made and sound the same or they would all sound the same with different circuits..I wonder how many would not like it? There is something wrong here, we do not know how to measure perfect sound! Have you done blind tests with blind people? Do sighted people make good listeners?
Regards
M. Gregg
What is "Good" sound? What one person likes another will not like!
I don't think this is just HIFI related. Some people like a "soft" sound less treble or more bass etc.
I think its interesting when someone buys some HIFI gear and then takes it back because they can't live with it....It's two harsh etc.
I think it would be interesting to hear someone play live music, and then play the recording back on the same stage....Lets see how close a system that measures "perfect" sounds to the real thing. I bet it's not even close!
Could you tell the two apart in a blind test....
Then play another "perfect" system and do the same. Does one violin sound the same as another on the HIFI or can you tell who is playing?
Theoretically if a measurement can set the bench mark output of an amp IE perfect, they would all be made and sound the same or they would all sound the same with different circuits..I wonder how many would not like it? There is something wrong here, we do not know how to measure perfect sound! Have you done blind tests with blind people? Do sighted people make good listeners?
Regards
M. Gregg
Hi,
Hmm, so "a good deal better than a coinflip" is the new scientific criteria?
What about 7 in 10? That is a good deal better than a coin flip as well.
Actually, I CAN do the math. I took statistics as part of my second degree.
I was looking for evidence that YOU can do so and are aware of the implications of your test criteria.
What if the result is a null result and it is replicated?
One that has equal risk of Type I and Type II errors and is free from expectation bias. That is for the test.
For the statistical analysis I would expect a confidence interval. Basically sensible science and sensible statistics.
I am unaware of any such publication covering two CD Players. Scientifically valid tests are extremely rare in audio, simply due to the practical problems of setting up such. The one applied to distortion metrics referenced earlier is a good example.
I am aware of some such tests in the context of perceptual coding research, others have been spotted in acoustic perception research (look under the "ultrasonic effect" section in the various papers).
There are many published small size AB Tests that where done blind that lack statistical power and scientific validity due to poor experimental design and very poor statistics applied, which show no difference. Sadly, due to their flaws we have no choice than to throw them out.
Actually, I once took all the "null result" tests (disregardless of what was tested) published by the ABX Mafia and analysed them as a "block" (it was a bit of "fun with stats").
As this was a fairly substantial dataset making it it is easy to have a reasonable confidence that a substantial amount of the null tests actually had a stimulus present, which they failed to detect, as the whole lot together certainly showed a statistically significant difference.
In other words, the 30 or 40 so tests lumped together showed a statistically significant difference. Of course, it says not much about actual audibility, but much about the tests themselves, their aims and the lack of credible statistics in them.
Ciao T
Hmm, so "a good deal better than a coinflip" is the new scientific criteria?
What about 7 in 10? That is a good deal better than a coin flip as well.
Actually, I CAN do the math. I took statistics as part of my second degree.
I was looking for evidence that YOU can do so and are aware of the implications of your test criteria.
What if the result is a null result and it is replicated?
One that has equal risk of Type I and Type II errors and is free from expectation bias. That is for the test.
For the statistical analysis I would expect a confidence interval. Basically sensible science and sensible statistics.
I am unaware of any such publication covering two CD Players. Scientifically valid tests are extremely rare in audio, simply due to the practical problems of setting up such. The one applied to distortion metrics referenced earlier is a good example.
I am aware of some such tests in the context of perceptual coding research, others have been spotted in acoustic perception research (look under the "ultrasonic effect" section in the various papers).
There are many published small size AB Tests that where done blind that lack statistical power and scientific validity due to poor experimental design and very poor statistics applied, which show no difference. Sadly, due to their flaws we have no choice than to throw them out.
Actually, I once took all the "null result" tests (disregardless of what was tested) published by the ABX Mafia and analysed them as a "block" (it was a bit of "fun with stats").
As this was a fairly substantial dataset making it it is easy to have a reasonable confidence that a substantial amount of the null tests actually had a stimulus present, which they failed to detect, as the whole lot together certainly showed a statistically significant difference.
In other words, the 30 or 40 so tests lumped together showed a statistically significant difference. Of course, it says not much about actual audibility, but much about the tests themselves, their aims and the lack of credible statistics in them.
Ciao T
@M. Gregg - there is another interesting thread that pertains to your question http://www.diyaudio.com/forums/everything-else/191488-source-problem-souless-sound.html where SY's system sounds to him a perfect reproduction of the mike feed (I haven't seen any agree with him yet) A logical conclusion would be that either his recording equipment is not up to snuff OR the issue is in the capture stage.
I think we are listening to the equipment.....we cannot listen to the music because we have never heard it for real... unless we were their when it was recorded!
So how can you tell which system sounds "Good"?
I think we are working to limited parameters that can only have a variation of close sound..Can a speaker sound like a trumpet...for real?
Each variation is close, however the other system is close in a different way! We choose which is closest sound to us with comfort of listening.
How much closer are we now than in the early days of HIFI some want to use old equipment, some say digital sounds wrong and record / tape is better….the fact that each is recorded differently means it should in theory sound different…and it does. Can you tell LP from cd of the same album. I wonder how our systems sound to animals?
Regards
M. Gregg
Last edited:
Just for interest,
I am not saying any system is better than another....
Someone asked me if I had an old valve radio....I asked why!
He said he had a dairy farm and the valve radio in the shed was broken.
He had put a transistor radio in the shed and the milk production had gone down...this he had measured!
I said I think you are mistaken...and gave him an old one I had in the attic.
I laughed when I saw him and said how are the cows? He said the milk was back to normal and could I fix his old one as a spare?
I thought he was joking....he was serious!
Regards
M. Gregg
I am not saying any system is better than another....
Someone asked me if I had an old valve radio....I asked why!
He said he had a dairy farm and the valve radio in the shed was broken.
He had put a transistor radio in the shed and the milk production had gone down...this he had measured!
I said I think you are mistaken...and gave him an old one I had in the attic.
I laughed when I saw him and said how are the cows? He said the milk was back to normal and could I fix his old one as a spare?
I thought he was joking....he was serious!
Regards
M. Gregg
Last edited:
- Status
- Not open for further replies.