Digital Distortion Compensation for Measurement Setup
 User Name Stay logged in? Password
 Home Forums Rules Articles diyAudio Store Blogs Gallery Wiki Register Donations FAQ Calendar Search Today's Posts Mark Forums Read Search

 Equipment & Tools From test equipment to hand tools

 Please consider donating to help us continue to serve you. Ads on/off / Custom Title / More PMs / More album space / Advanced printing & mass image saving
phofman
diyAudio Member

Join Date: Apr 2005
Location: Pilsen
Quote:
 Originally Posted by QAMAtt The phase shift from the LPF is a very clever idea to try and constrain the system, but try as I might, I cannot derive the equations to unconditionally constrain.
The two equations are in nonlinear-compensation/vdlpEqs.m at master * pavhofman/nonlinear-compensation * GitHub , I posted the link here recently.

Quote:
 However, if you first measure the combined ADC and DAC distortions using the non-inverting input of an analyzer, and then make another combined measurement of ADC and DAC distortions going into the inverting input of an amplifier, then you get the following simple math: Dd + Da = X (measurement into non-inverting input) -Dd + Da = Y (measurement in inverting input) Then the math simplifies to Da = (Y+X)/2 and Dd = X-Da. The above are complex quantities. The inversion is key.
Have you actually tried to compute some results in your software? It should be quite easy to modify, IMO.

Da is derived from the fundament. The fundament is inverted by the invertor too, along with Dd. IMO the invertor equation is:

-Dd - Da = Y

Which is again dependent on the first one.

Quote:
 Without the inversion, you have two unknowns, but no matter what with the LPF you don't get two distinct equations it seems to me.
One equation for VD, another equation for LPF.

VD:

Dd + Da = X

LPF:

K * Dd + L * Da = Y

Where K is complex transfer of LPF at Dd freq and L is complex transfer of LPF at fundamental freq, adjusted for Da freq.

For calculation I do not use complex numbers because the
nonlin_curvefit function of octave does not like them. I fit pre-calculated sine series in time domain, results should be the same.

QAMAtt
diyAudio Member

Join Date: Nov 2017
Location: Seattle, WA, USA
Quote:
 Originally Posted by phofman Which is again dependent on the first one.
Ah, yes, you are right. The cleverness of the LPF for breaking these apart becomes even more clever I'll continue to study
__________________
www.QuantAsylum.com

 2nd May 2019, 05:01 PM #173 phofman   diyAudio Member   Join Date: Apr 2005 Location: Pilsen I am thinking about split-calibrating the dual-tone signal (joint-sides dual-tone compensation works perfect, for frequencies both divisable by their difference - 10+12kHz OK, 11+13kHz not OK). In this case the fundamentals behind VD and LPF differ as LPF attenuates the second tone more than the first one. Different fundamentals mean different distortion profiles. The distortion fitting works only for fundamentals of same level and structure. I think the dual-tone splitting could be calculated by using four equations for four distortions. Let's assume fundamentals F2 > F1, all at -7dB: VD filter just scales both fundamentals: (1) VD filter: F1@-7 + F2@-7dB -> (VD@-6dB) -> F1@-13 + F2@-13dB LP filter attenuates F2 by -2dB below F1: (2) LP filter: F1@-7 + F2@-7dB -> (LPF) -> F1@-13 + F2@-15dB DAC generates F2 attenuated by -2dB, so that output of VD produces the same fundamentals as in (2) with the same ADC distortion profile: (3) VD filter: F1@-7 + F2@-9dB -> (VD@-6dB) -> F1@-13 + F2@-15fB Obtaining output of (1) for input of (3) requires attenuating the first fundamental F1 by -2dB - we need a high-pass filter: (4) HP filter: F1@-7 + F2@-9dB -> (HPF + some VD scaliing) -> F1@-13 + F2@-13dB Here we have 2 fundamental combinations on DA side, 2 fundamental combinations on AD side, 4 measured joint-sides distortions, 4 equations using all the 4 fundamental combinations (searched-for values). VD/LPF/HPF params for each distortion frequency are known - measured precisely in advance - just like for single-tone. IMO these four equations should be mutually independent and fitting them the same way as for single-tone (distortions adjusted for time and amplitude difference) should yield the four distortion values (DA@-7+-7, DA@-7+-9, AD@-13+-13, AD@-13+-15) for any/all distortion freqs. Out of these four results only the DAC distortion for eq (1) would be used for compensating the DAC at the measurement signal -7+-7dB. The corresponding ADC distortion would be precisely measured consequently by single-side calibration of AD side, with level set close to the last-measured DUT output level by the VD (as already performed by the current tool version for single-tone signal). The key would be creating the HP filter with inverse effect of the LP filter on the fundamentals. IMO if the RC LP filter worked at the -6dB/oct region, then using another RC HP filter at the +6dB/oct region + correct scaling with a voltage divider to obtain the same input values as in the corresponding equation should do the trick. Perhaps so :-) Last edited by phofman; 2nd May 2019 at 05:12 PM.
QAMAtt
diyAudio Member

Join Date: Nov 2017
Location: Seattle, WA, USA
Yesterday afternoon I attempted a simplified first try at building what phofman has built and the results are, to me, staggering.

In this implementation the Dd and Da terms are calculated from the two measurements X (pass through) and Y (through LPF), but all the cancellation is currently put in the DAC. As a result, re-inserting the LPF doesn't hold. But I'll fix that in the coming week.

The LPF is a 3.3K and 33nF NP0. The filter was swept and exported. System was configured for pass through, a single cycle was run. And then that measurement was captured as "phase 1". System was configured as LPF, and then another single cycle was run and that was captured as "phase 2" using the previously exported filter data.

In the attached, you can see 3 plots. Plot 1 is the QA401 at 0.5 dB away from max output limit and thus harmonics are growing. Harmonics are around -100 dB. Plot 2 is with cancellation active. Harmonics are -138 and lower. Plot 3 is with cancellation and melt active. Melt is an experimental mode on the QA401 that was prompted by a discussion with Demian years back after he'd sent a paper on paralleling ADCs. It basically sums N coherent time-domain acquistions, and then performs FFT on the average. This drives noise to zero. If you wait long enough, the THD+N will converge to THD because you have "melted" the noise. Here, it's useful to peel back the noise and see where the harmonics are.

Some interesting observations to bolster what phofman has seen. First, the filter only needed to be swept once. That one capture has been used over and over to reproduce this. Second, the results held over night. IOW, the coeffs determined 12 hours ago still "work". There is a fairly large temperature sensitivity, however. If I hit the DAC or ADC with cold spray, all is lost: the harmonics come back up and don't ever really go back to where they were. The input/output opamps don't seem nearly as sensitive though.

Implementation details:

This is all polar math and ham-fisted at this point. Phase 1 is capturing pass through value of X = Dd+Da. Phase 2 is capturing with LPF inserted Y = LPF*Dd + Da. This is a slight simplification of phofman's solution that will only hold (I think) if there's no gain in DUT.

For each step, the waveform is aligned to give the same alignment as the DAC output buffer. Post DAC, there are a few sources of delay, on the order of 10's of microseconds. The shfiting is done by a well-known FFT property such that you can take a time series, convert to freq domain, add a linear phase to every point, and convert back to time domain. This permits precise sub-sample shifting if needed. With Phase 1, it's shifted to the correct location and then the amplitude and phases of each harmonic are noted. Amplitudes are stored as relative to fundamental. Phases are absolute. In phase 2, the same is done. At this point, Dd and Da for each point can be computed and you get the values needed in both the ADC and DAC required to offset the offenders. From that point on, the DAC and ADC corrections are applied ot the DAC output. As noted above, that needs to be split in order for anything other than passthrough to work.

I cannot emphasize enough how clever I think the measurement through the LPF is to allow these the dac and adc components to be learned. Phofman, that exploit you shared is awesome.
Attached Images
 HofmanCancellation.png (145.8 KB, 179 views)
__________________
www.QuantAsylum.com

Last edited by QAMAtt; 4th May 2019 at 06:28 PM.

phofman
diyAudio Member

Join Date: Apr 2005
Location: Pilsen
Quote:
 Originally Posted by QAMAtt Yesterday afternoon I attempted a simplified first try at building what phofman has built and the results are, to me, staggering.
I am glad it works for you. Honestly, I still do not understand why such solution has not been used for many years. Computer-assisted analyzers have been produced for decades. To use LPF was not a random idea, it was the first obvious answer to the question how to change harmonics in a controlled manner against the fundamental so that two independent equations can be obtained and solved.

Quote:
 First, the filter only needed to be swept once. That one capture has been used over and over to reproduce this.
Yes, measuring the LPF and VD at all required frequencies precisely takes some time. In my latest version the transfer measurements are separated from the much faster splitting calibration.

Quote:
 Second, the results held over night. IOW, the coeffs determined 12 hours ago still "work".
I have the same experience.

Quote:
 There is a fairly large temperature sensitivity, however. If I hit the DAC or ADC with cold spray, all is lost: the harmonics come back up and don't ever really go back to where they were.
That is why the actual calibration must be fast and easily repeatable at any moment of measurement.

Quote:
 Phase 1 is capturing pass through value of X = Dd+Da. Phase 2 is capturing with LPF inserted Y = LPF*Dd + Da.
I did not understand exactly your measurement procedure, but Da is rotated and scaled by the LPF transfer of the fundament frequency:

Y = LPF@Dfreq*Dd + LPF@FundFreq*Da

Your use of compensation on one side only is what I call the joint-side compensation. It always works very nice. You used two components, but the X = Dd + Da equation keeps the results equal to the joint-side value. It works for VD/direct loop. Upon inserting LPF, the distortions shot up again if the LPF equation is not correct.

Also take into account the phase difference between each channel and phase shift of the voltage divider, once you start using it (non-negligible at higher freqs).

Quote:
 This is a slight simplification of phofman's solution that will only hold (I think) if there's no gain in DUT.
DUT gain is actually unimportant when splitting the distortion.

1) Split-calibrate distortions at the required DAC level (which is determined by the generator plus changes rarely) (two FFT measurements), use only the DAC part to clean the output signal

2) Measure remaining distortions at ADC side at the DUT level (using voltage divider) (one FFT measurement), compensate

When the DUT output level changes, just repeat step 2), there is no change on DAC side.

Quote:
 For each step, the waveform is aligned to give the same alignment as the DAC output buffer. Post DAC, there are a few sources of delay, on the order of 10's of microseconds. The shfiting is done by a well-known FFT property such that you can take a time series, convert to freq domain, add a linear phase to every point, and convert back to time domain. This permits precise sub-sample shifting if needed.
You use probably the same procedure. FFT directly provides phase shift = time offset, the compensation signal can be calculated with cosine at that time offset. Any phase offset can be calculated at any precision related to the actual sample time.

Quote:
 Amplitudes are stored as relative to fundamental.
You will learn the distortion profile works only for small range around the fundamental amplitude, on both sides. You may want to keep absolute values, you will need them when generating the compensation signal. That is why I use interpolation of distortions for calibrated fundamental levels around the current level.

Quote:
 I cannot emphasize enough how clever I think the measurement through the LPF is to allow these the dac and adc components to be learned. Phofman, that exploit you shared is awesome.
Again, I am glad it works.

Last edited by phofman; 4th May 2019 at 07:59 PM.

 5th May 2019, 02:32 AM #176 Techland   diyAudio Member   Join Date: Aug 2015 > Honestly, I still do not understand why such solution has not been used for many years. The answer may lie in the obvious limitations of this 'trick', caused by the imperfection of the available DACs (and ADCs). When QAMATt plays more with this solution he will most probably notice that it needs a recalibration for every single level he wants to use. That is not only cumbersome, it also prevents the use of level sweeps. And a usage with frequency sweeps seems unrealistic as well. I also expect termic problems as soon as you want to use this solution to its fullest and need to measure below -130 dB THD. But I don't want to bash the idea and the execution, it's both ingenious!
phofman
diyAudio Member

Join Date: Apr 2005
Location: Pilsen
Quote:
 Originally Posted by Techland it needs a recalibration for every single level he wants to use.
Yes, the distortion profile holds for very narrow margin of the current level. Interpolation of the profile calibrated at a series of levels works, but of course the key is automated calibration - the closer the calibration points are, the more precise profile the interpolation provides.

Quote:
 And a usage with frequency sweeps seems unrealistic as well.
Obviously the profile is for a specific frequency. Again automated splitting the sweep into series of frequencies is possible. It will make the measurement much longer though.

The usage I assume is that compensation is not to be used all the time, for all measurements. It is for temporary "zooming" into distortion of a device below native capabilities of the measurement device. Once the measurement (no sweeps) reveals a need for higher distortion resolution, push a calibration button and obtain the zoomed-in measurement.

Quote:
 I also expect termic problems as soon as you want to use this solution to its fullest and need to measure below -130 dB THD.
Termic problems are unavoidable without cooling, not much to do about them. Yet my ordinary soundcard at virtual balanced mode with both channels compensated measured -131dB THD at output -0.1dBFS/input -0.3dBFS Digital Distortion Compensation for Measurement Setup . I believe a good HW can go lower.

I am surprised how much of the distortions are due to non-random issues. IMO that is a good reason to try to do something about it, to the extent and situations allowed by physics.

 12th May 2019, 08:08 PM #178 phofman   diyAudio Member   Join Date: Apr 2005 Location: Pilsen I have finally managed to compile a working playrec on Win10 for ASIO. No WASAPI/the other many windows sound systems yet, There are constant problems with header files of Win10SDK/of mingw coming with windows port of octave/of mingw coming with portaudio. ASIO itself would be OK, but I could not find any ASIO-based loopback tool. So no go for now anyway. I really hate this dumb inconsistent OS.
phofman
diyAudio Member

Join Date: Apr 2005
Location: Pilsen
Finally managed to compile playrec with ASIO + WASAPI + MME:

Code:
>> test_playrec
Playrec was built with the following defines: DEBUG
Playrec was built with the following host API: MME (6 devices), ASIO (1 devices), Windows WASAPI (4 devices)

Available output devices:
-1) No Device
3) Microsoft Sound Mapper - Output (MME) 2 channels
4) Speakers (2 - High Definition A (MME) 2 channels
5) CABLE Input (VB-Audio Virtual C (MME) 2 channels
6) ASIO4ALL v2 (ASIO) 8 channels
7) CABLE Input (VB-Audio Virtual Cable) (Windows WASAPI) 2 channels
8) Speakers (2 - High Definition Audio Device) (Windows WASAPI) 2 channels
Next step was testing Arta + Virtual Cable VB-Audio Virtual Apps

The device is configured to run at 192/24 - see the settings screenshot. Arta is configured the same - see the screenshot

The digital loopback spectrum - see the screenshot. Complete fail, distortion at -120dB, far from bit-perfect.

I tried hooking Asio4All to the virtual cable device and configuring arta to use Asio4All - no change in spectrum. IMO the Virtual Cable is doing something to the signal, no idea what.

At 192kHz the virtual cable fails even more - there is a -40dB peak at 95kHz exactly.

For comparison - the last two screenshots are linux alsa loopback at 96/24 and 192/24 - bit perfect path, as it should be. I am not used to fighting the infrastructure - it is supposed to work 100% correctly.

Should someone try the virtual cable loopback in arta and share the results, I would be very grateful. Perhaps my windows arta is the culprit, I do not know.

Another option for the loopback would be compiling the open source virtual audio wire GitHub - HSpear/virtual-audio-wire: The Virtual Audio Wire (VAW) is an open source Virtual Audio Device (VAD) project its function is to provide Virtual Audio Devices for multimedia applications. . Unfortunately that is outside of my options since I do not own/have never used the required MS Visual Studio (playrec is compiled directly in octave by the included mingw compiler).
Attached Images
 arta-setup.png (16.9 KB, 100 views) vc-settings.png (15.6 KB, 98 views) arta-virtual-cable-loopback.png (106.3 KB, 101 views) arta-virtual-cable-192-24.png (107.3 KB, 100 views) arta-aloop.png (15.4 KB, 24 views) arta-aloop-192-24.png (16.0 KB, 23 views)

Markw4
Sin Bin

Join Date: Jul 2016
Location: California
The default behavior of Windows sound engine is to resample any audio it wants to without warning. For each sound device, in Windows control panel there is a default bit-depth and sample rate. If they exactly match your audio then it shouldn't be resampled. Please see attached jpg.

The only way to let an application control the bit-depth and sample rate is to use an ASIO driver AND make sure that the sound device you want to use does not show a green circle with a check mark in it next to the sound device name in Windows control panel. A check mark would indicate it is a default device for some purpose. Best to make some unused sound device the default sound device for every purpose (right click for menu), and then only use an ASIO driver for some other device which is the device you want to use.
Attached Images
 WinSoundConfig.jpg (67.1 KB, 34 views)

Last edited by Markw4; 13th May 2019 at 11:35 PM.

 Posting Rules You may not post new threads You may not post replies You may not post attachments You may not edit your posts BB code is On Smilies are On [IMG] code is On HTML code is Off Forum Rules
 Forum Jump User Control Panel Private Messages Subscriptions Who's Online Search Forums Forums Home Site     Site Announcements     Forum Problems Amplifiers     Solid State     Pass Labs     Tubes / Valves     Chip Amps     Class D     Power Supplies     Headphone Systems Source & Line     Analogue Source     Analog Line Level     Digital Source     Digital Line Level     PC Based Loudspeakers     Multi-Way     Full Range     Subwoofers     Planars & Exotics Live Sound     PA Systems     Instruments and Amps Design & Build     Parts     Equipment & Tools     Construction Tips     Software Tools General Interest     Room Acoustics & Mods     Music     diyAudio.com Articles     Car Audio     Everything Else Member Areas     Introductions     The Lounge     Clubs & Events     In Memoriam The Moving Image Commercial Sector     Swap Meet     Group Buys     The diyAudio Store     Vendor Forums         Vendor's Bazaar         Sonic Craft         Apex Jr         Audio Sector         Acoustic Fun         Chipamp         DIY HiFi Supply         Elekit         Elektor         Mains Cables R Us         Parts Connexion         Planet 10 hifi         Quanghao Audio Design         Siliconray Online Electronics Store         Tubelab     Manufacturers         AKSA         Audio Poutine         Musicaltech         Holton Precision Audio         CSS         exaDevices         Feastrex         GedLee         Head 'n' HiFi - Walter         Heatsink USA         miniDSP         SITO Audio         Twin Audio         Twisted Pear         Wild Burro Audio

 Similar Threads Thread Thread Starter Forum Replies Last Post lup31337 Software Tools 4 6th May 2016 09:33 PM akis Equipment & Tools 5 12th October 2012 07:26 PM Sanon Multi-Way 3 26th July 2011 11:58 AM thadman Multi-Way 7 7th June 2009 07:29 PM SimontY Multi-Way 65 26th April 2007 06:05 PM

 New To Site? Need Help?

All times are GMT. The time now is 02:17 AM.