@ Steve_Eddy,
your last post doesn´t help anybody and imho it is unacceptable in a forum like this.
Please ask yourself if you really did never wrote anything "crappy" yourself or never did remember something wrong......
Even if RNmarsch was wrong that does not excuse something like that.
@ jan.didden,
We can only compare the methodology in this way and of course a test done "blind/double blind" is therefore methodologically better then a test done "sighted" .
But, as we don´t try to get points for methodology, but want to do good which means objective, valid and reliable tests, the only way to evaluate the quality of any test/experiment are the results.
Thats why - what did you call it .... "calibration" - using positive controls on appropriate sensitivity levels are so important (and using negative controls as well) .
If you ask for the influences; yes in sighted tests these bias mechanims could lead to false positives while in controlled tests the same bias mechanisms might lead to false negatives.
But it depends on the specific test conditions; see for example an paired comparison done as "same/different" test (means "A" "B" "AA" "BB" were presented). The numbers from well documented controlled tests show
that the rate of wrong answers in the "AA" and "BB" trials reach up to ~80% , while it should be roughly 50% in the case of guessing.
Another example came from the SDT (signal detection theory), in controlled tests participants changed their answering habits due to external motivation, which means the balance between real hits and false alarms changed.
(insight from that would be to avoid prize money without special preparation)
If you think sighted tests must be wrong because the particpants couldn´t work against their expectance that maybe amplifiers should sound different, you might be right.
But what about a listener in a controlled test (including "blinding") who was told before numerous times that he surely will not be able to detect a difference?
your last post doesn´t help anybody and imho it is unacceptable in a forum like this.
Please ask yourself if you really did never wrote anything "crappy" yourself or never did remember something wrong......
Even if RNmarsch was wrong that does not excuse something like that.
@ jan.didden,
We can only compare the methodology in this way and of course a test done "blind/double blind" is therefore methodologically better then a test done "sighted" .
But, as we don´t try to get points for methodology, but want to do good which means objective, valid and reliable tests, the only way to evaluate the quality of any test/experiment are the results.
Thats why - what did you call it .... "calibration" - using positive controls on appropriate sensitivity levels are so important (and using negative controls as well) .
If you ask for the influences; yes in sighted tests these bias mechanims could lead to false positives while in controlled tests the same bias mechanisms might lead to false negatives.
But it depends on the specific test conditions; see for example an paired comparison done as "same/different" test (means "A" "B" "AA" "BB" were presented). The numbers from well documented controlled tests show
that the rate of wrong answers in the "AA" and "BB" trials reach up to ~80% , while it should be roughly 50% in the case of guessing.
Another example came from the SDT (signal detection theory), in controlled tests participants changed their answering habits due to external motivation, which means the balance between real hits and false alarms changed.
(insight from that would be to avoid prize money without special preparation)
If you think sighted tests must be wrong because the particpants couldn´t work against their expectance that maybe amplifiers should sound different, you might be right.
But what about a listener in a controlled test (including "blinding") who was told before numerous times that he surely will not be able to detect a difference?
Last edited:
Richard's point here is perfectly valid..ie 16 bits rec/pb system incurs significant distortions at mid levels which is where 'correctly' recorded orchestral, jazz, old school pop/rock etc music inhabits most of the time.
By 'correctly', I mean levels that comply with the old standards..ie -18dBFS average level which generally leaves sufficient peaks headroom for the above mentioned music genres. (much modern music is highly compressed/peak limited even when played live in studio so may not conform to/require the 18dB+ headroom spec historically used).
So, when playing back 'correctly' recorded programme on 16 bit systems, the playback DA convertor stage is indeed running at average levels way below (10% of) the maximum output levels that measure close to perfect and arguably inaudible.
So, straight away 20dB of noise and distortion 'perfection' is 'thrown away' in order to capture peak level without incurring distortion.
Tape systems and vinyl systems do not 'fall apart' when running at -20dB average level in the manner that 16 bit digital does.
Digital becomes more distorted as level decreases, analog systems do not become more distorted as level decreases, but do suffer masking due to intrinsic wideband noise.
That said the nature of analog system wideband noise is generally white/pink in nature, and is easily discounted or 'listened through' by the ear.
So, really a digital system needs to be 10x better resolution than 16 bit to have a hope of matching the distortion characteristics of analog at -18dB average levels.
Maybe somebody can do the maths, but on simple reckoning that means 21 bits real resolution is required for digital systems to compare to analog in the real world.
Never mind that 24 bits real resolution is pretty much unattainable in real world consumer PB systems, but 21 bits real resolution is probably what is generally attained in current 24 bit claimed consumer systems.
In order for digital systems to compete with analog/vinyl systems, music distribution does indeed need to be at least 24 bit for the typical consumer to correctly hear music as it was intended.
Sampling rate is another issue, but at least one of the benefits of higher Fs is slower/lower DAC output filtering roll off slope.
This lower slope/phase change does provide sonically beneficial results.
With any digital system, the amplification chain must have HF capability to ensure lack of LSB production.
LSB production is one factor that ruins sound on many systems.
Dan.
Last edited:
Do not conflate distortion with noise, then call noise in one case "distortion" and in the other case "noise." Or are you ignoring dither again?
Well if someone would run a test like that, i.e. strongly suggesting what the 'right' answer would or would not be, that's a useless test of course.
Can we lean back and overlook the landscape? In the light of all what you have so eloquently explained, what would be the chances that a naive listener, who tries to listen to the audible difference between two types of caps, in a fully sighted test with all his biases and prejudices present, gets a meaningful result?
Such a result can be selfserving in the sense that it reinforces the subject's wellbeing about a job well done, and that's as good a reason as any, but it is quite useless for anyone else.
Jan
Last night, my wife (who was not in the kitchen) and I watched a delightfully goofy demonstration on Penn & Teller's TV series, the name of which is automatically censored by the forum software. It had nothing to do with audio, it was about organic food. They set up a booth at a farmers' market and had a side by side blind comparison of so-called organic versus non-organic produce.
Scroll to 2:45.
https://www.youtube.com/watch?v=8Zqe4ZV9LDs
Language warning for the sensitive- they talk pretty much the way I do. The adjectives are amazingly familiar, as are the excuses.
Scroll to 2:45.
https://www.youtube.com/watch?v=8Zqe4ZV9LDs
Language warning for the sensitive- they talk pretty much the way I do. The adjectives are amazingly familiar, as are the excuses.
You have heard of oversampling? Only a small cult of 'NOS' believers don't run at a multiple of Fs in the DAC. Brickwall filters on DACs went out in the 80s.
In this case, I believe that he does understand the difference. The intended audience may not.
"Amateur" versus "pro" does not mean the same as "ignorant" versus "knowledgeable." I posted an anecdote a few days ago about a pro who maintained (I believe sincerely) that there were audible differences between WAV and FLAC files. His demo of this was very similar to the banana demo in the video I linked a couple posts back. The one guy in the room who understood that this was totally incorrect, as well as why, and that the pro had completely gone off the rails was... an amateur.
"Amateur" versus "pro" does not mean the same as "ignorant" versus "knowledgeable." I posted an anecdote a few days ago about a pro who maintained (I believe sincerely) that there were audible differences between WAV and FLAC files. His demo of this was very similar to the banana demo in the video I linked a couple posts back. The one guy in the room who understood that this was totally incorrect, as well as why, and that the pro had completely gone off the rails was... an amateur.
By definition digital systems exhibit increasing lack of resolution at lowering levels which translates to THD.
Theoretically SD digital systems should not produce Johnson noise, R/2R DACs by definition will output Johnson noise according to the values of the R/2R networking.
Regarding dither, it depends on the nature of the dither.
'Flat' dithering adds a white noise nature/masking that can be 'listened through', akin to analog noise and gives an apparent/subjective improvement due to white noise type spreading of distortion products.
The distortion is still there, it is just not quite so apparent due to masking effects.
Shaped dithering subjectively to my ear sucks big time.
In my experience shaped dithering causes a skewing of the low level distortion products that in longer term becomes reasily identifiable and wholly unnatural to the point of irritation and displeasure.
Dithering is a band aid, but not a cure.
Dan.
Last night, my wife (who was not in the kitchen) and I watched a delightfully goofy demonstration on Penn & Teller's TV series, the name of which is automatically censored by the forum software. It had nothing to do with audio, it was about organic food. They set up a booth at a farmers' market and had a side by side blind comparison of so-called organic versus non-organic produce.
Scroll to 2:45.
https://www.youtube.com/watch?v=8Zqe4ZV9LDs
Language warning for the sensitive- they talk pretty much the way I do. The adjectives are amazingly familiar, as are the excuses.
Well at the end Amanda (?) showed a rare open mind...
Jan
Ok, point made.
Perhaps I need to do some more recent fine comparisons, but I do know that I don't like Panasonic MASH, and Sony is less objectionable but still does not sound musically right to my ear.
Dan.
Last edited:
So what does lack of resolution translate to please ?....IMD, THD, what ??.
Sure, for single/pure sine wave, flat dithering will give improvement in measured thd/noise.
Dithering is addition of noise/uncertainty which improves the measured resolution but is in itself audible upon close listen.
Shaped dithering means addition of a frequency dependent noise/uncertainty which is readily discenable/identifiable.
Have a play with Cool Edit/Audition dithering modes and you ought to hear what I'm saying.
Dan.
Last edited:
Define your terms. You're using them interchangeably and in a loose and sloppy manner. Perhaps some time spent with a basic text on signal processing might be useful.
I could be wrong, but as way back as the Otala/Lohstroh IEEE paper, they did note that IMD was much more objectionable to the human hearing and could be detected at much lower levels than THD.
I was speaeking about THD, not IM, even if they are intertwined.
I haven't tried what you note Pavel, but I will take your word for it.
If you really want to know someting, the requirements of a test would be that it has to be valid, objective and reliable.
In most cases it is impossible to show that a sighted test meet these requirements, but you can´t show it neither for a controlled test if you didn´t use positive and negative controls.
A test works for two possible purposes, first you´re using a known difference to evaluate listeners or you use listeners with known detection capabilities to evaluate a difference.
The usual controlled audio test use detectors (means listeners) of unknown detection capabilities to evaluate an unknown difference (unkown perceptionswise). Obviously it can´t work that way.
Funny that you mentioned the case of capacitors. We started with some modifications of a preamp back in the beginning of the 80s. Exchanging a coupling electrolytic in the phone stage and doing sighted listening (never thought or heard about blind audio tests at that time) led to "whow what a difference" . Of course we exchanged the next coupling elctrolytics as well and went back to sighted listening (me and a colleague listenening).
Result was "not a difference worth the effort"
After that i read an article from Dan Shanefield about blind listening tests and his approach was plausible, so it was time for the next experiment. This time again a coupling capacitor in a phono stage but two film materials polyester vs. polypropylen from different manufacturers. First sighted listeníng with the result "whow big difference" , lets do a blind test.
Very simple, amateurish tests; my colleague would start, 5 trials, in each trial positive identification was needed (we soldered the capacitors in and out to get into the same time frame although one capacitor could be presented two or more times in a row). Surprsing result was that he failed, while i passed the test, although according to his description of the difference he obviously did hear the same things that i did.
And i noticed the difficulties under test conditions myself too, because in the first four trials he did not really swap the caps and i nevertheless thought everytime it was a different one. After that he presented in the fifth trial the other one and yes the difference was there, so i knew the first four times it was the same, but the other one, and got 5 correct identifications.
It was a very interesting experience and started my interest in sensory tests (or controlled testing / design of experiments).
Yes sometimes we are too smart for our own good. Reminds me of the Lipshitz/Tiefenbrunn experiment where L scored 100% because he locked onto the difference in back ground noise but not the fact whether there was /was not an AD/DA in the chain.
jan
- Status
- Not open for further replies.