No compentent electronics engineer engaged in design and engineering of a high quality audio amplifier works this way. Never has.
Sinewave testing is a basic test. It tells you basic and fundamental things about how a circuit is working. If an amplifier fails sinewave testing, it is no good. No point in testing any further.
Square wave testing is a basic test. It tells you other basic and fundamental things about how a circuit is working. If an amplifier fails squarewave testing, its no good. No point in testing any further.
THD (Total Harmonic Distortion) is a another basic test. It tells the engineer usefull things when an amplifier is ok on a sinewave test but not quite working right. It's not actually the percent distortion figure that matters, it's looking at the residual output from the THD set that tells the engineer usefull things. As D Self and a lot of others not so famous in audiphile circles have explained, a good engineer can tell where is the circuit the problems are occurring, so he can objectively and scientifically fix them. If an amplifier fails a THD test, it is time to make it pass. Dot not pass go, do not collect $200. Stop and fix it. But when it passes, it may not be a good amplifier. More testing is required.
2-tone & 3-tone IMD (intermodulation distortion testing) gives a nurmerical result that has much better correlation with how good an amplifier sounds, compared to THD, which has a poor correlation. Thus, it has been used in the professional audio industry (radio stations, motion picture sound, recording studios) for over 60 years to specify performance. If (say) the Columbia Broadcasting Company wants to buy 100 mixer consoles, they what a spec written in the contract. The spec that gets written is 2-tone IMD. Unfortunately, while IMD can tell you is an amp is in spec or out of spec, it is nowhere as good as THD at telling a design engineer what's going wrong in the circuit. Despite its value in contract specifications, an amplifier that passes IMD testing may not be a good amplifier. So IMD is more a customer spec, not a design engineer's diagnostic test. There are various ways of testing IMD, some have merits and caveats that other don't, so doing more than one test has merit.
Slotted noise testing has long been used in the telecommunications industry, where linearity is absolutely crucial. It is the best test, in that if there is anything wrong with an amplifier at all, slotted noise testing will show it up. But it has two big diasadvantages compared to THD and IMD: 1) it doesn't give you a go/no-go answer, and 2) you can't tell much of anything at all what's wrong with an amplifier - just that there is something. Various engineers over the years have proposed slotted noise testing for audio equipment, but it has never caught on for the above reasons.
Good engineers have always given their latest creation an ear test. Thta's the last test we do, after 1) sinewave testing, b) square wave testing, and c) THD testing, and (if we are really good) slotted noise testing.
From time to time, engineers and other researchers have looked what the ear can detect that THD and IMD cannot, and vice versa. And proposed new types of instrument test that will pickup what the ear can detect. But, in the consumer manufacturing game, two things happened: first, the rise in products based on semiconductor manufacturer's application reports, then, the advent of amplifier-on-a-chip integrated circuits. So, the real audio engineers are just a handful of chaps working for the major semiconductor manufacturers.
Testing with a real loudspeaker can and often will show up amplifier faults that testing with a resitor load will not. I recall a kit amp known as the "Twin 25" (solid state) described in the magazine Electronics Australia in the 1970's. It used an unusual common emitter output configuration. It's performance in THD testing. sine, and square testing into a resistor load was quite good. But it was woefull on loudspeakers not very well damped. It was a popular kit amp - thousands sold. Quite a few techs and trainee techs brought their Twin-25's to me thinking they were faulty, there was so much audible distortion. Some were completely happy - it depended on what speakers they had. These amps could also give distorted sound if certain transistors were installed C-E reversed, or if ferrite beads were not installed in the emitter leads. I found a simple way to show trainees where the problem lay - put an 8 ohm resistor in parallel with the speaker. Sometimes the drop in distortion was quite noticable. If it wasn't, look for reversed transistors & missing beads. It it was, change the amp or change the speakers.
Of course, none of this is a secret. Competent engineers have always know that speakers are reactive loads. Competent engineers have always known that speakers can push energy back into the amp - sometimes just when the amp can't cope with it. No problem - good engineers do yet another test: with a speaker "model" comprising and inductance and resistance in series. And test with capacitance as well.
I could write a whole book about established ways that good engineers use to test their latest creations. But its already written (by others) The above is just a summary.
Audio DiffMaker
Nice piece of software.
> If we can hear a difference, but can't measure a difference, then we need
> to improve our methods of measurement.
Earl Geddes did that and got a metric which is correlated to listening test results. Basically he found out that sound quality is correlated with the higher order derivative of the transfer function around zero, which is the same as saying that a kink in the transfer function close to zero is bad, like crossover distortion. Personally I think that, since the ear is a mechanical analog system, it will have distortion, but rather low-order, like a smoothly curved transfer function. So it makes sense that the brain would compensate and ignore that kind of distortion, while detecting higher order distortion as anomalous.
Noone cared about Geddes' article. I feel sad for him since that paper should have been printed in big characters and taped to the front panel of every Audio Precision analyzer sold since then, with a big red "read me first".
People like to measure what is measurable instead of what is relevant. Some time ago I was thinking about that, so I wrote a little python script. It sends a signal to the soundcard which is a sum of a low frequency, high amplitude sinewave (like 100Hz) and a high frequency, low amplitude sinewave (like 10kHz). The soundcard is connected to an amplifier with a dummy load, the output is digitized. Then it multiplies the output signal and the HF sinewave, both in phase and in quadrature, and detects the HF component amplitude and phase. Its variations, via a simple calculation, yield the output impedance of the amplifier. The low frequency sinewave determines the current. So it plots the output stage Gm (or output impedance) versus current, which looks like a wing, this diagram appears in Self's books.
And since this process involves no FFT and temporal averaging, I could also plot the variation of output stage Gm in time as the output transistors warm up and cool, and the "optimally biased" class B stage explores various bias points.
My point being, that plotting a FFT of a 1kHz signal tells you very little about actual performance, there is a lot of temporal averaging going on, and the plot throws away the phase of the harmonics which would allow reconstruction of the distortion residual, which contains all the interesting stuff.
So you don't need any special hardware to get an amplifier output stage Gm(I) plot, a soundcard and some digital processing will do. And the result will be much better than a slow DC sweep, since the slow sweep will be spoiled by self heating effects.
It is all about designing the right test signal, and the right DSP calculation to extract the juicy bits from the results. Noone does that, I wonder why, it's interesting though...
> I have convinced myself that a speaker is a much more unpredictable
> load than most people believe,
I have measured the current drawn by a loudspeaker (bookshelf 4 ohm Triangle brand), it has a HUGE amount of harmonics, very very ugly, especially in the bass. So the cable resistance will turn this ugly current into voltage, and therefore, into sound. And the audiophiles will spend hours talking about cables, while the objectivists will tell them that there is no difference since it only alters the frequency response by a small fraction of a dB, while at the same time it is a fact that higher cable resistance will increase THD measurably because of the nonlinear current, but noone bothers to measure that.
Newell and Holland did do just that - Loudspeakers: Effects of amplifiers and cables - Part 5 | EE Times
No compentent electronics engineer engaged in design and engineering....
I was not referring to designing an amp. I am talking about attempting to quantify the differences in two or more amps that have already been tested by conventional methods and pass with extremely good numbers yet sound different.
I have used it and it works. A differential scope will too, but you have to know what you are looking fore, and where in time the errors are.
A loudspeaker also has a mechanical motor - generator function which can not be modeled by purely passive electronics. A 15 inch coaxial high efficiency speaker (Hawthorne Silver Iris) can kick back some considerable EMF which is delayed in time from the transient that caused it. This is shunted by the amps output impedance and the DCR of the OPT in a tube amp. The resulting voltage is dumped into the feedback loop (in a GNFB amp) resulting in a response to an error that was not created inside the amp.
You may be right....at least that's what it sounds like. I'll go away now. When I get back to this stuff next year, I will post it on my web site.
I was not referring to designing an amp. I am talking about attempting to quantify the differences in two or more amps that have already been tested by conventional methods and pass with extremely good numbers yet sound different.
I have used it and it works. A differential scope will too, but you have to know what you are looking fore, and where in time the errors are.
A loudspeaker also has a mechanical motor - generator function which can not be modeled by purely passive electronics. A 15 inch coaxial high efficiency speaker (Hawthorne Silver Iris) can kick back some considerable EMF which is delayed in time from the transient that caused it. This is shunted by the amps output impedance and the DCR of the OPT in a tube amp. The resulting voltage is dumped into the feedback loop (in a GNFB amp) resulting in a response to an error that was not created inside the amp.
You may be right....at least that's what it sounds like. I'll go away now. When I get back to this stuff next year, I will post it on my web site.
Same comments apply really. If you want to quatify the differences between two amplifiers, then use the methods the pros use.
I did simplify things to keep my previous post to a reasonable length. I did say there was more to it. Having said that, testing with an L+R load will show up things that testing with only R will not. Delayed effects certainly occur in loudspeakers, however if L+R testing is good, it is quite unlikely in practice that delayed speaker responses will cause a problem. An amplifier does not "know" a response is delayed - it's feedback loop just knows there is a response and tries to suppress it. The output stage must be able to absorb power as well as supply power as the feedback loop demands. It either does that adequately or it doesn't.
If the loudspeaker gives back a delayed response, then yes the neg feedback (both global AND local around the output stage) will react to it. It will suppress it, which is mostly good, but not entirely - at least in theory. The thing is: nothing is perfect in the world. Neg feedback has a lot of pro and a bit of con. In balance, you are better off with it. A well engineered loudspeaker is perfectly acceptable if it offers a reactive response, and perfectly acceptable if it offers a delayed electrical response. Amplifiers must accept that. It is the role of an amplifier engineer to ensure that it does.
But a well engineered loudspeaker produces no significant acoustic response to the amplifier's negative feedback response to the delayed electrical response. The easy way to do that of course is to make the loudspeaker well damped. The loudspeaker engineer has a role to play too!
There is nothing wrong with using semiconductors in a tube amp. It doesn't need an analogy, if the part or design works better then you use it. Somewhere along the way these imaginary rules were put into place by "purists". The time for those notions clearly came to an end some time ago.
Will you be sharing your python script?
Will you be sharing your python script?
All of the above leads me to believe that I should stick with my current test equipment.
1 set of "standard" ears + the processing engine between them.
I built a HiFI system for each of the 6 nieces. All of them got Baby Huey Tube Amps as part of their systems but each got different speakers (what I could find 2nd hand at reasonable prices). All of the 6 Baby Huey Power Amps were different in terms of the amount of local and global feedback used. Each was tuned to its speaker set using that standard set of ears. The differences required by the speakers were quite stark.
Since the amplifier is designed to be listened to, then perhaps we should focus on listening to it.
I also built and heavily experimented with the Doug Self "Blameless Amplifier" using the same test equipment. The version I enjoyed most was using the Bootstrapped VAS. Everytime I changed that to a Current Source Load or even a compromised Current Source load the amp lost its "emotional engagement". This told me that I really did'nt want a "Blameless Amp".
During a time of all amps on the bench being modified I bought a top of the line ROTEL with 0.001% THD + Noise. It was the most cold, sterile and boring thing I'd ever heard. I sold it to a Shreader Guitar Player who used it with a Line 6 POD DSP preamp. He bridged the outputs for >400W "ear bleeding" sound levels, that is, it was used for reproducing distortion - poetic justice!!
No "Blameless Amp" for me thanks just the same.
Cheers,
Ian
1 set of "standard" ears + the processing engine between them.
I built a HiFI system for each of the 6 nieces. All of them got Baby Huey Tube Amps as part of their systems but each got different speakers (what I could find 2nd hand at reasonable prices). All of the 6 Baby Huey Power Amps were different in terms of the amount of local and global feedback used. Each was tuned to its speaker set using that standard set of ears. The differences required by the speakers were quite stark.
Since the amplifier is designed to be listened to, then perhaps we should focus on listening to it.
I also built and heavily experimented with the Doug Self "Blameless Amplifier" using the same test equipment. The version I enjoyed most was using the Bootstrapped VAS. Everytime I changed that to a Current Source Load or even a compromised Current Source load the amp lost its "emotional engagement". This told me that I really did'nt want a "Blameless Amp".
During a time of all amps on the bench being modified I bought a top of the line ROTEL with 0.001% THD + Noise. It was the most cold, sterile and boring thing I'd ever heard. I sold it to a Shreader Guitar Player who used it with a Line 6 POD DSP preamp. He bridged the outputs for >400W "ear bleeding" sound levels, that is, it was used for reproducing distortion - poetic justice!!
No "Blameless Amp" for me thanks just the same.
Cheers,
Ian
If the recording lacks emotional engagement, that is the problem of the people who made it, they should have turned up the emotional engagement knob on the mixing desk more.
On testing, it doesn't matter what test procedure you use, as long as it exposes the weaknesses of the circuit. High frequency THD and IMD measurements will do this just as effectively as real music signals, if not more so.
For marketing purposes you do the opposite, publish the results of only those tests that play to your circuit's strengths. This is how objective measurements got a bad name in audio (see the Crowhurst article mentioned earlier)
On testing, it doesn't matter what test procedure you use, as long as it exposes the weaknesses of the circuit. High frequency THD and IMD measurements will do this just as effectively as real music signals, if not more so.
For marketing purposes you do the opposite, publish the results of only those tests that play to your circuit's strengths. This is how objective measurements got a bad name in audio (see the Crowhurst article mentioned earlier)
100% true.
Actually, that is not always true. Some types of transient intermodulation do not show up in THD testing, even when done at high frequencies and at high level. IMD testing may show it up to a certain extent.
There are two aspects of amplifier behavior that degrade the quality on music that do not show up on THD and IMD testing at all: behavour during clipping, and recovery from clipping. There have been some amplifiers whose behaviour in these aspects was extremely audible, but undetected in THD and IMD testing. It is interesting that solid state has more ways to go wrong in regard to behaviour during clipping, and vacuum tubes have more ways to go wrong in regard to recovery from clipping.
Example for SS: In common emitter stages, under overdrive, the base collector junction can be forward biased. Iif the previous stage can supply a fair bit of current, and especially if there is an emitter resistor, the loop gain reverses! Even if that does not provoke oscillation, it sounds horrible. Very obviously horrible. Undetectable in THD testing.
Example for tube circuits: During overdrive, the power stage draws grid current. This charges the grid coupling capacitor such that after the overload ceases, the power stage grid is temporarily overbiased and partly or wholley parallized.
Most of these design faults not detectable by THD and IM testing can be detected by slotted noise testing.
It is no good saying don't overdrive your amp. The statistical nature of music is such that just about every amplifier, even high power amps, spend some fraction of time overdriven.
As I said previously, in my post about amplifier testing, I simplified it to keep it to reasonable length. Competent engineers whose job it is to originate and engineer quality audio amplifiers have another test in their bag of tricks: tone burst testing. Sine waves are applied in bursts of typically 5 cycles at high level interspersed with 10 to 20 cycles at low level. Design errors show up as distortion of the low level parts or even complete blocking of the low level parts.
For any instrument test you care to name, I, and any experienced engineer who has speciallised in high quality audio, can nominate a type of audible amplifier impairment that that particular test cannot detect (or at least not detect very easily), And for any real (not imagined or invented by a magazine reviewer) audible impairment, nominate an instrument test that will detect and quantify it. That's the way it is.
In any case THD in particular, and IMD to a certain extent, do not correlate to perceived quality, even for the amplifier impairments these tests readilay detect. For instance, these days you might do THD testing on two amplifiers A & B. A has THD not exceeding 0.1% at all frequencies and levels below clipping, and amplifier B 0.002%. Will amplifier B sound better? No. You may have A & B both testing 0.3%. Will they sound equally as good? Or equallly bad? No, not necessarily. Even if the power rating is the same, tone burst testing is equallly good, etc.
True. That and magazine reviews done by unqualified idiots. And magazines who feel the need to keep in their advertiser's good books.
Last edited:
Can you explain what these types of intermodulation are?
I had terrible trouble with Douglas Self's beta-enhanced VAS in this respect. When the amp clips on negative half cycles, it saturates heavily and takes forever to recover. To get acceptable clipping behaviour, I had to add a Baker clamp diode in parallel with the compensation capacitor. From the point of view of non-linear junction capacitances, this is about the worst place you could put a diode, but it fixed the clipping issues with only a modest increase in HF THD.
This might be true if you only listen to audiophile grade classical music. Any modern popular music (arguably since the Beatles) is compressed and limited to hell, such that the RMS level becomes painfully loud before the peaks start to bother your power amp.
If higher order harmonics are weighted according to the work done by Shorter, Geddes, Lee etc. then the correlation is quite good. Hence my insistence that high frequency THD is a good predictor of subjective sound quality in typical solid-state amps.
this has been confirmed with solid state into commercial passive crossover speakers.
It's why we end up with current peaks on fast transients that regularly exceed twice the current expected and sometimes approaches 5times the non reactive load current.
Last edited:
Wouldn't this show up, or at least give clues as to its existence with high level pink noise measurements with software like ARTA/LIMP?
Actually, recorded music has always been compressed. The further back you go, the more it was compressed. Music was issued on 78 RPM in parallel with 45 RPM 33&1/3 for several years. That meant music had to be mixed and engineerred to suite the restricted dynamic range of 78's. Quite a few recording studios did not update until 33&1/3 low noise pressings and high compliance pickups were well established.
The worst CD I have in my collection, from the point of view of showing up amplifier clipping limitations, is not any of my classical collection, it's The Buddy Holly Story (London Cast). On anything less than 100W with reasonably efficient speakers, clipping is audible even at moderate volume settings.
Ideas about weighting harmonics based on harmonic order are almost as old as electronic audio amplification. Noted and respected author Marcus Scrogglie (aka 'Cathode Ray') proposed it in the English magazine Wireless World in the 1940's. And he got the idea from even earlier workers.
I'v never been comfortable with the idea though. And BBC researchers and others in the 1960's more or less debunked it. The concept has been "re-invented" from time to time.
Scroggie's reasoning was faulty and was essentially this: You can have two amplifiers, A & B. Both test at the same THD level, say 2% (not untypical for domestic radios at the time). The one whose 2% distortion comes from Class B cross-over distortion spounds worse. There must be a reason for this. From wave analysis, we know that more higher order harmonics are pressent in Class B amplifiers if negative feedback is used. Therefore it must be that the ear is more sensitive to higher order harmonics. So we should weight them.
It's faulty reasoning. The fly in the ointment is that an amplifier with cross-over distortion will also sound worse than one whose distortion, equal on a THD test, is not crossover even if no negative feedback is used. And if there is no neg feedback, little high order harmonics are generated. Note that we are talking about tube amps, whose Class B crossover is a more gentle slope change than with bipolar transistors.
The problem with cross-over distortion is not high order harmonics. The problem is the cross-over occurs at zero crossings, at which the probability of a signal crossing is greatest. In Class H solid state toplogy used by Hitachi in the 1970's cross-over occurs when the signal passes thru the auxilary rail voltage, something that occurs with a much lower probaility. It is inaudible.
Last edited:
The classic type of transient intermodulation distortion (TID) came to light in the 1960s' with the early transistor amplifiers. Economic reasons compelled designers to use very slow output transistors. Some the early germanium power transistors started to roll off as low as 4 kHz. No use was made of VAS lag compenstion capacitors and the like, as the very slow output transistors made the amp completely stable anyway. Heaps of neg fedabck was used, not just to get the distortion down, but to flatten out the frequency response. However, in reacting to a transient, the neg feedback overloaded the VAS stage trying to force the slow output transistors to react. Because the output transistors were an effective low pass filter, they attenuatted the VAS overload harmonics and the THD wasn't too bad, even if done at high level and at highish frequencies.
When fast silicon power transistors became available, compensation became necesary for stability. The easy way is a lag capacitor around the VAS. If whatever drives the VAS cannot supply the charging or discharging current of the compensation capacitor, when the VAS is traversing the full rail-to-rail supply voltage range, the VAS rate of change is limitted and there will be a form of transient intermodulation distortion. This distortion can be detected in THD testing if you go looking for it. Do this by testing at near full power at the maximum frequency the amplifier input can possibly see. However, it seems to be a convention to not do this, and just test over the nominal audio range - not the same thing.
Yes. This is why in my highest quality amplifier designs, I used a cascode VAS. You get fast recovery without introducing the problems the baker diode causes. It totally eliminates Early Effect distortion. And you can use a low power fast recovery transistor for the lower one, and a cheap medium power upper transistor without sacrificing anything in performance. Commercially, it has a big disadvantage though. Unless you have a supplementary power rail for the VAS, you can't fully drive the output stage and max power output will be quite a bit less than you would expect for the supply voltage used. That means, in order to meet a given power spec, you'll need a bigger power supply and quite a bit more heatsinking. And company bosses hate spending dollars on something the clots can't hear anyway.
In Self-like topologies, there is a compromise with the input stage. Make it too weak and you get TID because it can't slew the VAS capacitor fast enough. Make it too strong and you get slow VAS recovery from overdrive. Doug focussed on the first and not on the second. Pick the wrong transistor to use as the VAS and the two problems overlap too much.
Last edited:
The problem with crossover distortion is high order nonlinearity. This gives rise to high order harmonics and high order IM. The fact that this high order nonlinearity is near the zero crossing makes it more noticeable on small signals; most domestic amps spend most of their time handling small signals.Keit said:
I believe the Gedlee metric also weights kinks near the zero crossing more heavily to account for this, though Shorter did ignore it as you say. I account for this by testing high frequency THD at several power levels. If it is clean at (say) 10kHz and 0.1, 1, 10 and 100W, then I think there is nothing to worry about.
You just described SID, TIM or whatever it is called. I test my solid-state amps at full power at 20kHz. I am confident that if I get low distortion under this test condition, there is no problem with SID. No music signal has a slew rate anywhere near that required for full power at 20kHz.
This is a somewhat of a common misconception. To see why, consider the following:-
The worst possible cross-over distortion is a (theoretical) case where the transistors do not start to conduct at all until a finite voltage - let's say +.- 0.5 volt. Under sinewave drive, this results in the output +ve half cycles being held bodily down by 0.5V, and similar for teh -ve half cycles.
This can be modelled to a first approximation by a sinwave with a 1V amplitude square wave added to it in inverse phase.
We know from Fourier that a square wave consists of the fundamental at unit amplitude plus 1/3 amplitude 3rd harmonic, 1/5 5th harmonic, 1/7 7th harmonic, 1/9 9th harmonic and so on, for as high as you like.
We need to improve the approximation. Instead of adding an inverse phase square wave, we can add a trapeziodal wave, with the rising and falling slope equal to the slope of the sinewave at the zero crossings. This will a) reduce the amplitude of all harmonics; b) produce a greater reduction in higher order harmonics over that of a square wave; and c) effectively impose a limit on the number of harmonics. Since the slope is the max slope of the sinewave, the highest harmonic number is as to the ratio of the trapezoide height vs the sinwave height but cannot be particularly high.
We can improve the approximation still further. In vacuum tubes especially, and ven with transistors, the chage in gm in cross-over is not an abrupt switch from zero to full gain. It is a gradually transition - in bipolar transistors the hfe varies as to teh log of the b-e voltage. This gradual transistion introduces a marked curvature to the rising and falling slope of our trapezoidal approximation. The result is a further large reduction in the highest harmonic generated, and a marked reduction in a harmonic or two below that.
All up, it means that with bipolar transistors in Class B, crossover distortion is limitted for practical purposes to harmonics below the 5th or 7th. With vacuum tubes, the main active device we are concerned with here, the picture is even better.
The situation changes when negative feedback is used - especially with bipolar transistors. The feedback takes the 3rd and 5th in the output and mixes it with the fundamental at the input, so that non-linearity produces sum and difference frequencies: f1 +,- f3, f3 +,- f5 and so on. In this way, negative feedback causes (low level) harmonics that would not be present without neg feedback.
I believe that the misconception that cross-ver distortion is necessarily high order comes from sloppy writing by Marcus Scroggie, a quite famous engineer and an author prolific in in the 1940's & 1950's. Sloppy writing is something that Scroggie was noted for NOT doing, but we all make mistakes occaisionally.
Last edited:
Any problem which affects just a small part of the total in/out function necessarily requires a high order nonlinearity (this is just a matter of algebra). Low order nonlinearites affect a larger part of the function.
In almost all practical cases the higher order terms created by re-entrant distortion are similar in size or smaller than the high order stuff already present without feedback. Perhaps the only common case where this is not true is a source follower.
As you say, Scroggie rarely made mistakes. I don't believe this was a mistake. As it happens, I don't get my understanding of crossover distortion from Scroggie but from thinking about algebra. It is always good to go back to first principles.
In almost all practical cases the higher order terms created by re-entrant distortion are similar in size or smaller than the high order stuff already present without feedback. Perhaps the only common case where this is not true is a source follower.
As you say, Scroggie rarely made mistakes. I don't believe this was a mistake. As it happens, I don't get my understanding of crossover distortion from Scroggie but from thinking about algebra. It is always good to go back to first principles.
- Status
- Not open for further replies.