Measuring Distortion on the Cheap

[tomchr]

So, you have a burning desire to measure harmonic distortion and other audio parameters but don't have $40k burning a holes in your pockets so you can buy an APx555B. I have news for you: There are cheaper options.

I just finished testing three such options:

• QuantAsylum QA403 Audio Analyzer ($600)
• Focusrite Scarlett 2i2, 4th Generation ($200)
• Focusrite Scarlett Solo, 3rd Generation (Refurbished from Focusrite: $80)

The results will probably surprise you. For example, I was surprised that the third Generation Solo performed better than the fourth generation 2i2 with single-ended input. I also think you get incredible bang for your buck with the QA403.

Unfortunately ultrasonic noise limits the maximum usable frequency for THD+N vs frequency sweeps on the Scarlett sound cards. Basically the noise renders the 192 kHz sampling rate useless for audio measurements. 96 kHz works fine, but then you're limited to a maximum test frequency of 15 kHz if you want two harmonics (H2 and H3) included in the THD measurement.

I've included the loopback plots below. I have tons more measurements available here: https://neurochrome.com/pages/measuring-distortion-on-the-cheap

I also yap about it here:

QA403:

Scarlett 2i2 (4th Gen):

Scarlett Solo (3rd Gen):

Tom

 

[restorer-john]

Hey Tom, can confirm your results are consistent with my unit.

Example.

Yours:

Mine:

 

[rsavas]

Well done Tom. Last night I was reading Bob Cordell’s writeup on his evaluation and tutorial on QA-403 testing the BC-1 power amp design. Nice piece of test gear for sure. Maybe your feedback on the GUI will get back to Matt at QA and find improvements.
Cheers
Rick

 

[tomchr]

Hey Tom, can confirm your results are consistent with my unit.

Excellent. That's actually pretty remarkable unit-to-unit consistency.

Last night I was reading Bob Cordell’s writeup on his evaluation and tutorial on QA-403 testing the BC-1 power amp design.

That's a good place to start.

Nice piece of test gear for sure. Maybe your feedback on the GUI will get back to Matt at QA and find improvements.

The QA403 offers amazing value for the money, especially if you don't need an AP logo on the graphs. You can go lower in price with the sound cards, though I'd try to find one that has a dedicated line level input rather than a combo input. I'm told the Focusrite Scarlett 4i4 has better inputs, but I haven't tested it. That's also $350, so by the time you add up the cost of attenuators, adapters, etc. that you need to get performance that approaches the QA403 you've quickly spent the better part of the $250 you save by going with the sound card instead of the QA403.

Add a precision oscillator like Victor's and you should be able to measure THD (no +N) down around -130 dBc with the QA403.

I hope that Matt & Co will hear my feedback. It's up to them if they want to take action. If they can define a user interface that makes sense I doubt it would cost that much to have someone overseas code it up. Also, QA publishes the source code, so it could turn into a community effort too.

Tom

 

[tktran303]

Thank you Tom!

Your measurements of the Scarlett gen 3 could explain why I’ve avoided use of measurements beyond 88.2KHz with my 2i2gen3.

Bruno talks about the IMD test at 18.5/19.5KHz as an altenative, though I haven’t conducted this.

“The two tall poles 18.5 kHz and 19.5 kHz are the test tones. Even-order distortion components are visible at 1kHz and multiples, while odd-order components show up at 1kHz intervals from 17.5kHz down. So, despite using only audio-band signals and only looking at the audio-band outcome, this test is perfectly revealing of what an amplifier can do at the top of the audible range…

Even for linear amplifiers, I consider this intermodulation (IMD) test more meaningful than a non-band limited THD test with 20kHz”

Reference:
https://hometheaterhifi.com/30th-an...esent-and-future-of-class-d-audio-amplifiers/

At one point I discovered that that my long standing hiccups with my Scarlett 2i2 was related to some software incompatibilities; so I had to look for an alternative.

The gen4 has been discovered to be slightly worse than gen 3, and I was tempted to get the QuantAsylum QA403.

Is it true that it doesn’t come with ASIO drivers?

What is the test environment? I was hoping to run REW with it, but I hadn’t heard any reports of people doing that.

I would like to benchmark the world’s first 24bit multichannel external audio interface (USB1.0) and see how far we’ve come in 25 years…

 

[BFNY]

Below is the plot of 1kHz loopback, single ended, with a 2nd generation 2i2. These go for 1/3 to 1/2 new bought used, so meet the cheap goal. These also have analog input pots.
I like ARTA, and use the win 10 snipping tool to save images like below.

But does anyone know of software that can do 2 channel bode plots, i.e. Frequency Response Function (H1) using a soundcard? Especially looking for stuff like real, imaginary, phase, group delay etc. - what you get in a "real" FFT box from HP? Nano VNA works from 10kHz up...
Mostly for FM MPX filter development. I think the old Sound Technology FFT package could, but it's really old now.

 

[restorer-john]

I think the old Sound Technology FFT package could, but it's really old now.

There are people out there still using the ST software. I saw it referred to recently. That has to be the Win95 days, surely?

Have to agree with the 2i2v2 being a bargain, but once you've bought a QA403, you'll only use the 2i2 for listening to music.

In answer to @tktran303 (Hi), there is an QA ASIO project done by edechamps here:
https://forum.quantasylum.com/t/qa403-asio-driver-when/1224/15

IMO, Tom is being overly hard on the QA's software, especially as he perhaps didn't take the time to a) read the manual or b) read the forums before diving in.

He says:
The user interface is laid out as the front panel of a physical real-time analyzer. That works reasonably well for controls such as pushbuttons but not that well for number entry. Thus, tasks such as changing the generator amplitude or frequency, the FFT length, and the number of averages requires multiple clicks on up/down buttons instead of using a Windows-native pull-down menu or number entry field. Some aspects of the software are just flat-out unintuitive. For example, it took me a lot of random clicking and right-clicking to finally figure out how to change the axis settings on the graph. Hint: Right-click on the greyed out numbers in the Y PRESET section of the AXIS panel. Similarly a right click on many of the pushbuttons will reveal a dialog that allows for direct entry of the settings. That's not exactly intuitive.

Control+1 (Ctrl 1) or 2 brings up the direct entry for each generator.

Ctrl + up or down arrow adjusts on the fly amplitude.
Ctrl + up/down averages jumps 5 at a time.
1 or 2 turns on or off generators.
etc.
It's all on page 38 and 39 of the QA manual which is one click away on the help menu.

When you get the unit, right click on every tile to see what options there are. It takes all of 10 minutes and you'll know what is what. After using the unit for a day, you'll know where everything is.

I can't recommend the QA-403 highly enough for what I do. It has obsoleted all my vintage audio analyzers and much other legacy gear that is now taking up space. There's at least 5 large, heavy pieces of gear in my lab that haven't even been switched on in over a year since the QA arrived.

We'd all like an APX555B, it's a magnificent piece of gear, but I don't want the fan noise (the QA is silent- no fan), the massive hole in my finances and the prospect of being locked out of future upgrades unless I cough up huge sums either up front, or drip fed down the track. I don't trust the direction Axiometrix (AP) is going.

 

[tktran303]

Well, I should have watched Tom's video on YT! It's very good.
It answered all my questions above.

Lovely to see a APx555B x vs alternatives.

Thank for your work!

 

[GKTAUDIO]

Excellent presentation, Tom! Very informative and useful to me, since I have all of the test instruments you discuss, plus more.

 

[Jacruzer787]

Great work Tom, read the article. Looks like a scarlet solo 3 gen is in my lost of things to buy.

 

[tomchr]

Control+1 (Ctrl 1) or 2 brings up the direct entry for each generator.

Great, but not intuitive.

This wasn't intuitive either:

You can also right click the GEN button. Unless I remember incorrectly, the dialog you show is different from the one that comes up with a right click on GEN1. Why? Again, not intuitive.

When you get the unit, right click on every tile to see what options there are. It takes all of 10 minutes and you'll know what is what.

True. But that's still not intuitive. It's not intuitive that I have to right-click on a seemingly inactive field that says "20-180", or something like that, to set the Y-axis settings. How about a button labeled Y-AXIS or something?

After using the unit for a day, you'll know where everything is.

True. Until next time you need to use it.

Tom

 

[tomchr]

Bruno talks about the IMD test at 18.5/19.5KHz as an altenative, though I haven’t conducted this.

Honestly, I think the 18+19 kHz (or 18.5+19.5 kHz) IMD and multi-tone IMD are more relevant than a THD+N vs frequency sweep, but the THD+N vs freq is still a very common measurement. So it's a bit of a bummer with the ultrasonic noise. That said, I'd say the sound cards are good enough for basic DIY design validation. For the full picture you need something better.

Tom

 

[restorer-john]

How about a button labeled Y-AXIS or something?

Like this one that says "Axis Y Preset"? The one where hovering the mouse pointer on it, tells you what to do?

You mentioned a progress indicator for automated tests. We have one...

You're coming from the AP and have been using it for years. Anything else will seem to be not "intuitive" to you.

Personally, I love keyboard shortcuts, Alt keys, function keys and not having my hand on a mouse all the time. The fact that the QA guys are so responsive to requests and features is also unique. You can operate the QA403 with just a keyboard for most tests, or a touchscreen all-in-one which is pretty cool.

We've had 15 software updates since September 2023, many with added useful features and improvements. Amazing. One of my previous analyzers had just one release in 25 years!

 

[tomchr]

ou're coming from the AP and have been using it for years. Anything else will seem to be not "intuitive" to you.

That's flat-out false. I base my opinion on my educational background, which includes human factors engineering, i.e., how humans and machines, including software, work together. But it is still my opinion. I do believe it's an informed opinion, however.

If I had to right-click every item in MS Word, Final Cut Pro, Photoshop, or any other program I use on a regular basis in order to figure out if it had some hidden function I'd go mad. Figuring out how to use a simple piece of software like the one for the QA403 shouldn't be a game of right-click whack-a-mole. But it seems like we disagree on that. That's fine. I was able to learn the APx500 software without ever reading the manual, using the help function, or any of the hover help items. I was going within seconds of launching the software. That's an example of software that's intuitive to use.

I simply don't have the patience to deal with crappy software. And I have high standards. I mention both in the video. As I also say, you can decide whether what I grumble about would bother you. You have clearly decided that it doesn't bother you, so great!

I see lots of room for improvement in the QA software. I'm not here to slam QuantAsylum. I just think it would be nice if the quality of the software matched the quality of the hardware.

You mentioned a progress indicator for automated tests. We have one...

That's great! I missed it because it's buried with a bunch of completely irrelevant information. Had it been in the prominent dialog box I would have seen it.

And, as I also point out both on my website and in the video: It's not like I could not get anything done. In fact, once I figured out the quirks and oddities of the QA software I could make it do what I needed to do within the limitations of the hardware. I wish the graphics were closer to production/publication quality, but that's something I'm willing to trade off at the $600 price point.

We've had 15 software updates since September 2023, many with added useful features and improvements. Amazing. One of my previous analyzers had just one release in 25 years!

AP has several releases of their software over the years. I started in 2014 on version 4.6 and am now running 7.1. Latest is 9.something. They too have implemented many new features, including a bunch of tests for speaker testing, including of smart speakers. I have no use for those tests which is why I haven't upgraded.

Tom

 

[tktran303]

I think it depends on context. As much as the lower cost alternatives are nice, I don't think those a replacement for the AP. Hear me out....

If I was a working professional in the field of electronics/acoustics, I would want a reliable tool that just works. Every time I pick it up it needs to work, is both accurate and precise. Your review shows that AP just works, with no fuss/mess and good support from Audio Precision.

On the other hand, as a hobbyist of 20 years, I have been through a variety of soundcards, including but not limited to- Creative Labs, Advance Gravis, Apogee, Focusrite. All these are mass produced consumer cards, with various quirks and no support from the manufacturer.

For instance, the ultrasonic noise was causing all kinds of confusion for me when I was testing at high sample rates, but without a reference ADC, I didn't know what was going on. All one can do is a loopback but then problem is still there- you can't tell if it's the DAC or ADC without another audio interface.
Or the ADC? The 2nd problem I have had from day one was the intermittent pops and blips. I had no idea where it was coming from, eventually I tracked it down to the interface by forcing a sweep that lasted 1 and half minutes- Turns out it's a system/driver incompatibility with my system. Dropping Focusrite's ASIO driver and switching to ASIO4ALL helped, and increasing the buffer also helped, reducing the frequency of the pops/clicks. But it still happens intermittently.

This kind of stress testing and to discover problems like this is just annoying. Glitchy intermittent problems are frustrating. Dead things are better to fix.
Eventually I just had to change audio interfaces.

I think a good middle ground like the QuantAsylum QA403 Audio Analyzer is a minimum for any professional who values their time and does paid work.
Either that or the Prism (now Spectral) dScope M1. Pocket the difference between the AP and put the money towards a TTC or PTC...

 

[tomchr]

I think a good middle ground like the QuantAsylum QA403 Audio Analyzer is a minimum for any professional who values their time and does paid work.

I agree. I would go even further that the QA403 is a solid choice for any hobbyist who values their time. There are options that can get you close to the performance of the QA403, at least when operated at their sweet spot. Someone on ASR suggested a combination of a separate ADC and DAC but that adds complexity, cost, and removes the ability to have the ADC and DAC sync'ed to the same clock which is handy for FFTs.

I've often found that cheap tools end up costing more in the long run than just buying the expensive tool up front. That's mostly because I hate using the cheap tool or it wrecks the workpiece and then I end up buying the expensive tool anyway. That's how I feel about getting a sound card versus a QA403 for audio measurements.

If I had a QA403 I'd probably try the ASIO401 driver and REW. I found REW pretty intuitive to work with. I don't know how the ASIO driver handles access to the attenuator. That'd probably be my only concern.

Tom

 

[anatech]

Hi Tom,
I use the RTX 6001. A solid tool. I've looked at the AP offerings. I'm sorry, way too much money for what they are and do. Compare to the price of a car ... really? No way!

Keysight is no better (U8903B), but I trust them more for long term support. The only intelligent alternative I see would be the QA403. Someone needs to teach these folks the audio testing hardware isn't worth what most of the market can't bear.

 

[thermionic]

I am amazed that the U8903B remains in production. The -110dB THD+N figure quoted by Keysight is below that of the QA403, no? (not sure which standard Keysight use). I really wish one or two of the big players such as Keysight or R+S would compete with AP. I get the distinct impression that AP think they have a monopoly on the market and can charge accordingly. Their trade-in offer against a new AA expires halves in 2025 to 10%. I've been in dialogue about trading the 2722 against a 555. I wonder if they jacked up the price of the 555 (£35K GBP / $42K for base model), just to offer 20% off against old AAs. They are even offering 10% against a UPV (massive rip-off IMO, as a working UPV is worth way more than 10% of a 555).

Aside from Sam Groener's article that suggests the 2722 oscillator outperforms the 555 (the rest of the 555 will nuke the 2722 - its software does everything apart from sit up and beg), their 'trade in' offer is in the form of a discount on the 555, not an actual deposit. This means that for a company trying to finance it, the interest rate goes right up. A third concern would be that the 555 has been out for over 10 years now (5 in case of B model), and I have to wonder if a more affordable model might come out that has similar performance. The new 516B (with external oscillator) isn't that far off the 555, and its price tag is competitive by AP standards. My UPV has a couple of sucker punches that I don't see on the 555 for everday testing (the B3 oscillator is uber useful) and I worry that I will only use the 555 as I do the 2722, which is to spec things (rare) or to impress clients. I have an associate with several 555s and he still uses the UPVs. R+S still service the UPV...for how long?

The DScope we have hasn't been switched on in years.... Its THD+N spec is way below the QA403. Unlike the Dscope, using the 403 won't give you service anxiety (the DScope isn't that valuable and I doubt it'll be economic to repair one if, say, a DSP IC failed...).

 

[tomchr]

Compare to the price of a car ... really? No way!

... or to the downpayment for a house if you load it up. I was shocked to see that AP has cranked the price of the APx555B from $30k to $40k. I thought $30k was a lot of money. Trust me. It was not lost on me that I just spent about 2/3rds of the average annual income for a single Canadian when I bought my APx555. Unfortunately for me, the 555 is the only instrument on the market that can measure the performance of my circuits reliably. And, frankly, the AP logo in the top corner of the graph lends credibility and fairy dust. When I bought my 525 years ago, I joked with the salesman that AP could probably make pretty good money by selling a stamp with their logo on it.

I'm also a bit surprised that AP maintains the 515. It's no better than the QA403 at its sweet spot, but the AP probably has more sweet spots. It's also faster. That is a big deal for those looking to screen lots of parts for THD. Think semiconductor manufacturers and such.

Keysight is no better (U8903B)

I'm surprised that's even in the market. I haven't looked at the specs for the past ten years, so I could be off base here, but I don't remember it being dramatically better than the original HP 8903B. Maybe 5-6 dB. Nothing earth-shaking. At the time it was about the same cost as the APx525 (which is now up from $12k to $17k) so getting the AP was a no-brainer.

Then again. What you tend to get with the instruments from R&S, HP/Agilent/Keysight/Whatever-Name-This-Week, Fluke, AP, etc. is good protection circuits on the inputs and outputs along with good technical support.

I once connected the input of my HP 8903A to the screen grid instead of the input grid on a tube. All that happened with the 8903 is that it's 62.5 mA (!) input fuse blew. I swapped the fuse and ran it through the performance test outlined in the service manual. It still met all the specs. I doubt the QA403 would survive that. A sound card certainly wouldn't and a mishap like that could potentially destroy the PC connected to the sound card as well.

My only gripe with AP is that they now want money for software upgrades that used to be free. Basically, when you drop $40k on an audio analyzer you get to choose between a perpetual software license ($0 extra) or a subscription plan (about $1200/year with a 3-year minimum, if I remember right). The perpetual license gives you the current software plus one major revision update.
Now, don't get me wrong. I do believe that companies deserve to be compensated for their development efforts. AP has, undoubtedly, put a lot of hours into the development of the new feature in their software and they should charge for that. But it still seems a bit over the top.
Their software is based on Microsoft .NET and the update from .NET 4.8 to 4.8.1 broke AP software version 7.1 and below. So whenever I update my test PC I have to uninstall the update that contains .NET 4.8.1. That also means I can't run the AP software on Windoze 11. That's pretty disappointing. I'd have to spend $3-4k to get the latest software (and probably updates for a while after). But all I need is version 7.1 updated to the current .NET. I don't need the new features. Yeah. No dice! So for now my test PC lives on its own VLAN and has no internet access. The only reason its network cable is plugged in is so it can figure out what time it is (I run my own NTP server).

That approach is not terribly uncommon in the test & measurement world. We had several instruments at TI that ran on Windoze. Some ran Windoze XP or NT4. At some point our fancy $50k oscilloscope had a virus. That was great. Then the IT folks wisely decided to keep these instruments disconnected from the network. I also wonder what will happen to the Agilent E5052. That runs Windoze too. It may turn into a $150k unmaintainable doorstop pretty soon, if it hasn't already.

Tom

 

[phase]

Am very surprised that a company like that networked their lab computers(?)