Hi,
I've been trying to measure THD of several DACs and then compare the sound difference.
So I have about 10 different DACs (from ES9038Pro based to PC embedded), measured their harmonic spectra (at 1kHz 0db with notch filter) and then compare their sounds in pairs (on the same signal source and the same amplifier / speakers / headphones, with switching DACs "without looking", a couple of my friends were involved into the listening)
The measurements-to-listening results were generally predictable and matched the theory, some curious things were observed though. However there is a riddle I'm trying to solve.
The matched the theory and expected results
- The higher THD the better sound (the most obvious and expected of course)
- The sound quality is determined rather by a highest harmonic absolute level than by THD (RMS of all harmonics)
- Higher (5,7,11-th) harmonics are definitely more noticeable than (2-3rd)
- All the DACs with highest harmonic level below -100db sounded nearly equally
The curious observations
- Left and Right channels of the same DAC can have different harmonic levels. The difference can be up to 8-9 db
- Some DAC (devices) based on not too highly rated DAC chips can easily compete with award wining DAC chips based devices producing very good results.
The puzzle is with a single USB sound card.
Its sound is very noticeably worse the "competitors", but its FFT looks decent enough.
It's obviously not the best, high harmonics are at -110db, low harmonics are at -96db, but several other DACs with the same parameters sound definitely better.
Most probably 1kHz 0db FFT is not too representative, so I tried different frequencies, different levels, IM measurements etc. None of them revealed what's bad with the DAC,
but its sound is definitely worse DACs with even worse THD.
What should be measured to reveal the DAC defect tehcnically, not just to hear it.
Using a dynamic signal anyhow maybe? Actually all the known measurements techniques are based on static signals. I'm lost in conjectures and need an advice
Thank you,
Serge
I've been trying to measure THD of several DACs and then compare the sound difference.
So I have about 10 different DACs (from ES9038Pro based to PC embedded), measured their harmonic spectra (at 1kHz 0db with notch filter) and then compare their sounds in pairs (on the same signal source and the same amplifier / speakers / headphones, with switching DACs "without looking", a couple of my friends were involved into the listening)
The measurements-to-listening results were generally predictable and matched the theory, some curious things were observed though. However there is a riddle I'm trying to solve.
The matched the theory and expected results
- The higher THD the better sound (the most obvious and expected of course)
- The sound quality is determined rather by a highest harmonic absolute level than by THD (RMS of all harmonics)
- Higher (5,7,11-th) harmonics are definitely more noticeable than (2-3rd)
- All the DACs with highest harmonic level below -100db sounded nearly equally
The curious observations
- Left and Right channels of the same DAC can have different harmonic levels. The difference can be up to 8-9 db
- Some DAC (devices) based on not too highly rated DAC chips can easily compete with award wining DAC chips based devices producing very good results.
The puzzle is with a single USB sound card.
Its sound is very noticeably worse the "competitors", but its FFT looks decent enough.
It's obviously not the best, high harmonics are at -110db, low harmonics are at -96db, but several other DACs with the same parameters sound definitely better.
Most probably 1kHz 0db FFT is not too representative, so I tried different frequencies, different levels, IM measurements etc. None of them revealed what's bad with the DAC,
but its sound is definitely worse DACs with even worse THD.
What should be measured to reveal the DAC defect tehcnically, not just to hear it.
Using a dynamic signal anyhow maybe? Actually all the known measurements techniques are based on static signals. I'm lost in conjectures and need an advice
Thank you,
Serge
Very interesting topic, I've been interested in this question for rather a long time and still haven't come to any firm conclusion. I do reckon low-frequency noise is important though in a DAC as I recently got a worthwhile subjective improvement in moving from a single multibit DAC to many in parallel. These are 16bit DACs though so don't start out particularly low noise. I don't have any measurements of noise which correlate with this improvement.
One caveat when dealing with USB-fed soundcards is CM noise - this has the effect of 'bleaching out' tonal colours. If possible use balanced connections to PC-fed DACs to mitigate this.
One caveat when dealing with USB-fed soundcards is CM noise - this has the effect of 'bleaching out' tonal colours. If possible use balanced connections to PC-fed DACs to mitigate this.
I think this is a classic example of getting hung up on the mostly insignificant stats and completely missing the big picture. As indicated in your test the D/A chip itself makes almost no difference in the results and this should not be a surprise as this process has been essentially perfected for decades now, but all the other parts around it and particularly the analog parts in the signal chain can have a major impact on SQ. And just to be clear I'm not talking about the other parts of the listening system I'm referring to the analog parts included with the DAC.
To that end did you measure the frequency response of all the DACs? They should all be flat but subtle difference here will be noticeable. I bet the computer's included DAC measures the worst here by a long shot and again not a surprise, this is an add-on feature not a core component so little effort or money would be spent on it's design.
To that end did you measure the frequency response of all the DACs? They should all be flat but subtle difference here will be noticeable. I bet the computer's included DAC measures the worst here by a long shot and again not a surprise, this is an add-on feature not a core component so little effort or money would be spent on it's design.
> ;-)
Honestly I suspected such reaction ;-)
Obviously human preferences vary significantly. That means "the lower distortion the better" is not unconditional. Many people prefer tube or R2R sound that distortions are higher, but specific.
Needless to say that R2Rs or (especially) tubes together with distortions add a lot of other "effects". Preferring a tube device (for instance) there is no way to separate non-linear distortions from other "effects". So it's easily could that a "tube amateur" actually prefers the other effects, not the pure distortions.
Trying to realize that I have created a tool (Sound Distorter THD controlling)
that adds just and only non-linear distortions (all the other parameters are kept intact) and allows to verify the distortion related preferences/limits of perceptibility.
>One caveat when dealing with USB-fed soundcards is CM noise - this has the effect of 'bleaching out' tonal colours.
Hmm.... The frequency response was absolutely flat,
The noise level is about -140db, so could hardly be heard
Maybe I just don't understand in full what you meant.
Could you be a bit more descriptive?
I just have to say that the card sound is very noticeably ugly, similar to PC embedded cards, in spite of -100db of high harmonics and -95-96 db of the second and third.
I believe -100db is the lower limit a human can hear at the non-destructive (for ears) 0 db.
In the sound card spectrum I did see some non-multiple components but at -130 db.
Not sure it could affect anyhow. Could you suggest a specific experiment to reveal the influence?
>If possible use balanced connections to PC-fed DACs to mitigate this.
Tried, no effect.
Balanced connections typically add more distortion as more analog circuits OPAMP parts) are involved though.
>Were the volume levels matched at the speaker output?
Sure. if a DAC output was different at 0db, it was adjusted by analog adjustment, before the master volume, the amplifier allows that..
>The analog parts in the signal chain can have a major impact
Undoubtedly, but I measured/compared entire devices, not chips inside.
A middle level DAC chip actually + perfect analog parts can be better than a perfect DAC + bad analog parts.
However it's rather an exception. I saw that only once.
>To that end did you measure the frequency response of all the DACs?
Sure, the FRs were strictly flat accurate within with the available measurement
I believe +/- 0.2-0.3db would be detected.
>They should all be flat but subtle difference here will be noticeable.
Good idea, thank you! I will try to determine what minimal frequency response deviation can be heard.
>I bet the computer's included DAC measures the worst here
You are right.
I included computer's DAC just to cover the entire possible diapason.
All in all the question is still what to measure beyond FFT to predict a DAC quality.
Frequency response tiny deviations should not a genuine reason for my the particular DAC "surprise" , I will try to measure though.
Honestly I suspected such reaction ;-)
Obviously human preferences vary significantly. That means "the lower distortion the better" is not unconditional. Many people prefer tube or R2R sound that distortions are higher, but specific.
Needless to say that R2Rs or (especially) tubes together with distortions add a lot of other "effects". Preferring a tube device (for instance) there is no way to separate non-linear distortions from other "effects". So it's easily could that a "tube amateur" actually prefers the other effects, not the pure distortions.
Trying to realize that I have created a tool (Sound Distorter THD controlling)
that adds just and only non-linear distortions (all the other parameters are kept intact) and allows to verify the distortion related preferences/limits of perceptibility.
>One caveat when dealing with USB-fed soundcards is CM noise - this has the effect of 'bleaching out' tonal colours.
Hmm.... The frequency response was absolutely flat,
The noise level is about -140db, so could hardly be heard
Maybe I just don't understand in full what you meant.
Could you be a bit more descriptive?
I just have to say that the card sound is very noticeably ugly, similar to PC embedded cards, in spite of -100db of high harmonics and -95-96 db of the second and third.
I believe -100db is the lower limit a human can hear at the non-destructive (for ears) 0 db.
In the sound card spectrum I did see some non-multiple components but at -130 db.
Not sure it could affect anyhow. Could you suggest a specific experiment to reveal the influence?
>If possible use balanced connections to PC-fed DACs to mitigate this.
Tried, no effect.
Balanced connections typically add more distortion as more analog circuits OPAMP parts) are involved though.
>Were the volume levels matched at the speaker output?
Sure. if a DAC output was different at 0db, it was adjusted by analog adjustment, before the master volume, the amplifier allows that..
>The analog parts in the signal chain can have a major impact
Undoubtedly, but I measured/compared entire devices, not chips inside.
A middle level DAC chip actually + perfect analog parts can be better than a perfect DAC + bad analog parts.
However it's rather an exception. I saw that only once.
>To that end did you measure the frequency response of all the DACs?
Sure, the FRs were strictly flat accurate within with the available measurement
I believe +/- 0.2-0.3db would be detected.
>They should all be flat but subtle difference here will be noticeable.
Good idea, thank you! I will try to determine what minimal frequency response deviation can be heard.
>I bet the computer's included DAC measures the worst here
You are right.
I included computer's DAC just to cover the entire possible diapason.
All in all the question is still what to measure beyond FFT to predict a DAC quality.
Frequency response tiny deviations should not a genuine reason for my the particular DAC "surprise" , I will try to measure though.
Can you clarify this point please? Are you saying that people preferred units with more overall measurable distortion or just more high order distortion?
I'm one with a preference for multibit DACs (not R2R in particular) but it may well turn out I just have a preference to avoid CMOS opamps in the output stages of my DACs. In that I haven't tried my I/V stage with any S-D Iout DACs.
Seems that way, I wouldn't expect CM noise pollution to affect either.
CM noise is otherwise called 'leakage currents'- they are ultrasonic at the switching frequency of the SMPSU. Here is a technical treatment :
https://www.eetimes.com/power-tip-47-tame-conducted-common-mode-emissions-in-isolated-switchers-part-1/#
They'll not show up in the audio band (because they're usually >40kHz) but they will intermodulate with audio to give IMD products which appears as noise modulation.
Was the input stage (bal - unbal) an opamp? In which case it could have added in the noise modulation itself. Just an idea.
Digital low pass filter Implementation makes a huge difference in sound quality. I don't know how to measure the difference. My casual measurement software (RMAA) can't detect any meaningful difference between different FIR filters.
>Can you clarify this point please? Are you saying that people preferred units with more overall measurable distortion or just more high order distortion?
People are always emotional first of all,
So "good sound" and "I like the sound" are emotional categories.
They are generally not a measurement (FFT/etc) dependent at all.
"Live", "warm", "soft" are rather not technical categories.
Sometimes visual appearance of an apparatus can be decisive.
So technical perfection can be irrelevant at all.
This doesn't mean distortions are good
(sometimes high level even harmonics can make the sound "warmer" at first glance though).
This just means there are other more emotionally important, the correct conclusion suppressing (and generally unpredictable) factors to think "the sound is good" far beyond the technical perfections.
For instance, last week I tried to demonstrate room equalization to my very old friend.
Took a microphone, Room Equalization Wizard and Equalizer APO,
after his room was equalized, he said "it's terrible" "it's impossible to listen"
Just a real sample.....;-)
>CM noise is otherwise called 'leakage currents'- they are ultrasonic at the switching frequency of the SMPSU. Here is a technical treatment :
Thank you for the explanation!
>they will intermodulate with audio to give IMD products which appears as noise modulation.
Shouldn't that be visible in FFT?
I saw some irregular components in FFT , but -126db level.
That level can't be heard I think, Right?
>Was the input stage (bal - unbal) an opamp?
Unbalanced every time, for the DACs pairs audio comparison.
And only one DAC from the set caused me doubting there is something missed in the measurement scenario.
>In which case it could have added in the noise modulation itself. Just an idea.
For the FFT I used -50 db (at 1 kHz test signal) notch filter
just to suppress possible effects of the measurement instrument (ADC + software FFT analyzer) and (I believe) modulation as well.
>Digital low pass filter Implementation makes a huge difference in sound quality
Undoubtedly, but I tried to compare entire devices (as "black boxes" with inputs and outputs),
not individual components inside the devices.
Generally the higher THD the better sound, excepting a single "ugly" sounding DAC with relatively good THD. This makes my thinking that THD is not enough to estimate
The question is "what else"
>How closely were the volumes matched between devices tested?
I understand the logic, but there was about 10 DAC pairs participated in the experiment.
The volume levels were adjusted quality and aurally for all the pairs. And only one DAC (in spite of a pair) makes be doubting that THD is not enough to estimate the quality.
I'm starting to think the problem may be in dynamic parameters, rate of signal rise ( as in ancient amplifiers) or so. Anyway all the measurements were made on static signal, but aural comparing was made on music.
People are always emotional first of all,
So "good sound" and "I like the sound" are emotional categories.
They are generally not a measurement (FFT/etc) dependent at all.
"Live", "warm", "soft" are rather not technical categories.
Sometimes visual appearance of an apparatus can be decisive.
So technical perfection can be irrelevant at all.
This doesn't mean distortions are good
(sometimes high level even harmonics can make the sound "warmer" at first glance though).
This just means there are other more emotionally important, the correct conclusion suppressing (and generally unpredictable) factors to think "the sound is good" far beyond the technical perfections.
For instance, last week I tried to demonstrate room equalization to my very old friend.
Took a microphone, Room Equalization Wizard and Equalizer APO,
after his room was equalized, he said "it's terrible" "it's impossible to listen"
Just a real sample.....;-)
>CM noise is otherwise called 'leakage currents'- they are ultrasonic at the switching frequency of the SMPSU. Here is a technical treatment :
Thank you for the explanation!
>they will intermodulate with audio to give IMD products which appears as noise modulation.
Shouldn't that be visible in FFT?
I saw some irregular components in FFT , but -126db level.
That level can't be heard I think, Right?
>Was the input stage (bal - unbal) an opamp?
Unbalanced every time, for the DACs pairs audio comparison.
And only one DAC from the set caused me doubting there is something missed in the measurement scenario.
>In which case it could have added in the noise modulation itself. Just an idea.
For the FFT I used -50 db (at 1 kHz test signal) notch filter
just to suppress possible effects of the measurement instrument (ADC + software FFT analyzer) and (I believe) modulation as well.
>Digital low pass filter Implementation makes a huge difference in sound quality
Undoubtedly, but I tried to compare entire devices (as "black boxes" with inputs and outputs),
not individual components inside the devices.
Generally the higher THD the better sound, excepting a single "ugly" sounding DAC with relatively good THD. This makes my thinking that THD is not enough to estimate
The question is "what else"
>How closely were the volumes matched between devices tested?
I understand the logic, but there was about 10 DAC pairs participated in the experiment.
The volume levels were adjusted quality and aurally for all the pairs. And only one DAC (in spite of a pair) makes be doubting that THD is not enough to estimate the quality.
I'm starting to think the problem may be in dynamic parameters, rate of signal rise ( as in ancient amplifiers) or so. Anyway all the measurements were made on static signal, but aural comparing was made on music.
They have to be level matched within 0.1 db in order to ensure the objectivity in comparison. If not, you (& other listeners) are likely to hear difference arising from volume level difference. For what it's worth, you can hear a difference from the same DAC just by playing at different level. Not only that, you can even fool consumers to buy your DAC (if you sell) by doing a level unmatched comparison, a.k.a. sales pitch.
Theory has long held that the ear cannot detect changes of 1dB or less, how do you arrive at your number of 0.1dB.
Sure, with an FFT that goes up high enough in frequency. Such as with this DAC - scroll down and compare 'noise floor with USB galvanic isolation' to 'noise floor without USB galvanic isolation' FFTs : exaSound Audio Design > Products > e32 Mark II DAC
IME people prefer lower distortion and higher fidelity. An interesting experiment is to use an ESS Sabre dac and get into the I2C registers to change the harmonic distortion compensation registers. One can adjust distortion without changing any other variable in the dac. IME lower distortion always sounds subjectively better in that case.
However, there can be some confounders: (1) poor recordings can sometimes be made to sound subjectively better with some added distortion, (2) modern music is sometimes intentionally distorted as a part of the loudness wars, and kids growing up listening to that music have learned to like distortion as a result, (3) distortion can sometimes be used to mask or otherwise compensate for other problems in a compromised reproduction system, ... and so on.
Despite confounders, when people are given an opportunity to listen to a great reproduction system they are usually blown away. Adding more distortion does not make it sound 'better' if the overall system is good enough and accurate enough.
EDIT: By the way, it has been well known for a very long time that higher order harmonics are more noticeable than 2nd or 3rd order. IIRC the sound quality metric proposed by Earl Geddes weighted distortion according to harmonic order.
EDIT 2: It is also well known that noise floor modulation in sigma delta dacs is one factor affecting perceived sound quality that does not appear on an FFT.
However, there can be some confounders: (1) poor recordings can sometimes be made to sound subjectively better with some added distortion, (2) modern music is sometimes intentionally distorted as a part of the loudness wars, and kids growing up listening to that music have learned to like distortion as a result, (3) distortion can sometimes be used to mask or otherwise compensate for other problems in a compromised reproduction system, ... and so on.
Despite confounders, when people are given an opportunity to listen to a great reproduction system they are usually blown away. Adding more distortion does not make it sound 'better' if the overall system is good enough and accurate enough.
EDIT: By the way, it has been well known for a very long time that higher order harmonics are more noticeable than 2nd or 3rd order. IIRC the sound quality metric proposed by Earl Geddes weighted distortion according to harmonic order.
EDIT 2: It is also well known that noise floor modulation in sigma delta dacs is one factor affecting perceived sound quality that does not appear on an FFT.
Last edited:
My 2 cents ...I have 2 almost identical dacs (THD measurements) very small changes to PCB and yet I am able to find one "better" . It came down to RF energy (and I am talking over over 20Mhz-200Mhz) on i2s lines. Almost no change in THD between the 2 , yet the DAC with tamed RF sounds better
Last edited:
>They have to be level matched within 0.1 db in order to ensure the objectivity in comparison.
Maybe you are right, but achieving that is hardly possible.
On the other hand, why the level tiny discrepancy was critically significant for the only DAC in spite of a pair it participated.
All the other DAC pairs with the same discrepancy were compared with predictable results.
>you can even fool consumers to buy your DAC (if you sell) by doing a level unmatched comparison, a.k.a. sales pitch.
I don't sell anything. I'm just trying to understand for myself.
Actually if you spoke to a HiFi/End salesmen, they would probably tell there is a lot of much more reliable methods to "fool" unprepared listeners.
Moreover HiFi/End salesmen would never want to measure anything.
It's just against their intention to sell. They can have no a bit better or a bit worse equipment at all. The equipment can be just amazing, brilliant or a bit more budgetary, but never worse.
So mentality of salesmen is completely different, rather funny and curious for technicians.
>When and by whom was that theory presented?
Equal-loudness contour - Wikipedia for instance describes minimum hearable levels. So, taking into account we talk about music that shouldn't damage ears at the highest level, the dynamic range is about 100db as maximum on selected frequencies (in spite of whether harmonics or a main tone)
>How about practise?
We are discussing that. Right?
>>Shouldn't that be visible in FFT?
>Sure, with an FFT that goes up high enough in frequency.
However I saw just a relatively normal FFT of the device, but the device sound was "unacceptably" bad.
So what else should be measured to find the reason,
just FFT looks not enough? There is probably a specific defect to reveal.
>An interesting experiment is to use an ESS Sabre dac and get into the I2C registers to change the harmonic distortion compensation registers. One can adjust distortion without changing any other variable in the dac. IME lower distortion always sounds subjectively better in that case.
Might I ask you for more information/links. I'm very curious regardless of this theme.
>However, there can be some confounders:
Oh yes, you are right, completely right!
>higher order harmonics are more noticeable than 2nd or 3rd order.
Undoubtedly. But how you could explain the "tube phenomena"?
Tubes' (transformers') 2nd and 3rb harmonics are typically high but the entire sound is often perceived as good, without noticing specific harmonics as defects. Just masked by the others?
> It is also well known that noise floor modulation in sigma delta dacs is one factor affecting perceived sound quality that does not appear on an FFT.
Maybe it's exactly what I was missing in my measurements.
The "surprising" DAC is AK4396 based.
(please don't tell me it's not good, I know that ;-) I have no purpose to choose or use it, I would like just to explain unmatched technical and aural comparing results)
Is there any chance to measure it or see at least technically?
If it's not FFT detectable there should be something else.
Do you have an idea?
>It came down to RF energy
Should I just use a (hardware) oscilloscope connected to the "surprising" DAC output? It seems reasonable to check. Thanks!
Maybe you are right, but achieving that is hardly possible.
On the other hand, why the level tiny discrepancy was critically significant for the only DAC in spite of a pair it participated.
All the other DAC pairs with the same discrepancy were compared with predictable results.
>you can even fool consumers to buy your DAC (if you sell) by doing a level unmatched comparison, a.k.a. sales pitch.
I don't sell anything. I'm just trying to understand for myself.
Actually if you spoke to a HiFi/End salesmen, they would probably tell there is a lot of much more reliable methods to "fool" unprepared listeners.
Moreover HiFi/End salesmen would never want to measure anything.
It's just against their intention to sell. They can have no a bit better or a bit worse equipment at all. The equipment can be just amazing, brilliant or a bit more budgetary, but never worse.
So mentality of salesmen is completely different, rather funny and curious for technicians.
>When and by whom was that theory presented?
Equal-loudness contour - Wikipedia for instance describes minimum hearable levels. So, taking into account we talk about music that shouldn't damage ears at the highest level, the dynamic range is about 100db as maximum on selected frequencies (in spite of whether harmonics or a main tone)
>How about practise?
We are discussing that. Right?
>>Shouldn't that be visible in FFT?
>Sure, with an FFT that goes up high enough in frequency.
However I saw just a relatively normal FFT of the device, but the device sound was "unacceptably" bad.
So what else should be measured to find the reason,
just FFT looks not enough? There is probably a specific defect to reveal.
>An interesting experiment is to use an ESS Sabre dac and get into the I2C registers to change the harmonic distortion compensation registers. One can adjust distortion without changing any other variable in the dac. IME lower distortion always sounds subjectively better in that case.
Might I ask you for more information/links. I'm very curious regardless of this theme.
>However, there can be some confounders:
Oh yes, you are right, completely right!
>higher order harmonics are more noticeable than 2nd or 3rd order.
Undoubtedly. But how you could explain the "tube phenomena"?
Tubes' (transformers') 2nd and 3rb harmonics are typically high but the entire sound is often perceived as good, without noticing specific harmonics as defects. Just masked by the others?
> It is also well known that noise floor modulation in sigma delta dacs is one factor affecting perceived sound quality that does not appear on an FFT.
Maybe it's exactly what I was missing in my measurements.
The "surprising" DAC is AK4396 based.
(please don't tell me it's not good, I know that ;-) I have no purpose to choose or use it, I would like just to explain unmatched technical and aural comparing results)
Is there any chance to measure it or see at least technically?
If it's not FFT detectable there should be something else.
Do you have an idea?
>It came down to RF energy
Should I just use a (hardware) oscilloscope connected to the "surprising" DAC output? It seems reasonable to check. Thanks!
You can ask ESS for a Sabre dac data sheet, but you will have to sign an NDA to get anything from them. They may or may not answer questions you might have. Or, you might get lucky doing a web search, who knows.
Anyway, most or all of what exists in print is in data sheets. How distortion compensation registers are used in practice is up to the whoever is programming the dac. A more technical discussion of dac design/configuration is going in the 'ES9038Q2M Board' thread, in case you would like to read/participate in that.
It's not. Use voltmeter at the speaker terminal.
As I've already explained, comparing any audio electronic device at mismatched levels can result in the observation of level difference, not the claims of improved sound due to some superior parts used.
What about salesmen posting on internet forums? Have you spotted any on this thread?