The dynamic range of 16 bits

Status
This old topic is closed. If you want to reopen this topic, contact a moderator using the "Report Post" button.
I suggested it once and feared for the future of my employment past the end of the week. I got about the same looks that one would expect after suggesting building a house of cards bird house for a woodpecker .....

(For Fred)
Factiously:
Those playing cards don’t look like they are NEBS-3 certified, and if we do try to deploy a woodpecker, then NCON will have to be updated, the cutoff for the next revision from Telcordia was last week and I don’t have the million in the budget to perform the RGS/BRD work. Net-Ops has never worked with a woodpecker before so that also would require a whole new training cycle before they will even consider writing the MPS, and is a woodpecker ISO layer 2 or layer 3 functionality?
 
Konnichiwa,

Werner said:
Tried the same now with CE Pro. Its attenuator works differently.

Generating the source signal in a 32 bit domain, and attenuating there with 96dB together with the addition of a DC offset of the equivalent of 1/2 LSB(16bit), and then quantizing to 16 bit (without dither), yields the same as my lowest trace above: the representation of a sine wave with a signal-to-quantisation-noise ratio of almost 0dB. This is the limit level that is still being encoded, 96dB below full scale.

However you do it, all wave-edit/analysis software calls the resulting -90.3dbfs. We can argue endlessly, even tricking the system into something that most programs try not to let you do you end up with the signal being recognised as -90.3db, not -96db.

Now we can endlessy argue about dithering, the best use of dither to linearise a given system and so on, however in terms of the raw underlying system you hit the fundamental brickwall at -90.3dbfs, according to anything I have at hand to analyse and generate.

Please note again that the fact that there is "no life below -90.3dbfs" does not invalidate the "2^16 = 96db" position. Except, it is not applicable of course to using a 16-Bit system for bipolar audio signals.

Sayonara
 
Ok, one more time from the top.

FLAME ON!

Er, I mean DITHER OFF!

An externally hosted image should be here but it was not working when we last tested it.


1kHz sinewave at 0dBFS

An externally hosted image should be here but it was not working when we last tested it.


The FFT of this 1kHz sinewave at 0dBFS

An externally hosted image should be here but it was not working when we last tested it.


The statststics of this 1kHz sinewave at 0dBFS

An externally hosted image should be here but it was not working when we last tested it.


Same 1kHz sinewave with -90.3dB attenuation

An externally hosted image should be here but it was not working when we last tested it.


And the FFT of that waveform

An externally hosted image should be here but it was not working when we last tested it.


And the stats of that waveform

An externally hosted image should be here but it was not working when we last tested it.


Same 1kHz sinewave with -96dB attenuation

An externally hosted image should be here but it was not working when we last tested it.


The FFT

An externally hosted image should be here but it was not working when we last tested it.


And the stats

An externally hosted image should be here but it was not working when we last tested it.


It should be pretty clear what's happening here.

As the signal level drops from -90.3dB to -96dB, even though both signals are quantized to plus or minus 1LSB, at the 96dB level, there are fewer samples quantized at plus and minus 1LSB and more quantized at 0.

The effect of this is to reduce the amount of energy at 1kHz and thus the signal is attenuated.

Some point to the increased quantization noise as the signal is attenuated and say this can't possibly be high fidelity and the fact that the energy at 1kHz has been attenuated below the claimed limit is moot.

However those who say this are neglecting the fact that in an ideal undithered digital system, THERE IS NO NOISE IN THE ABSENCE OF A SIGNAL compared to an analog system where there is ALWAYS NOISE IN THE ABSENCE OF A SIGNAL.

In an undithered digital system, the only noise that can exist is that which is due to the signal and as we decrease the level of the signal, we increase the amount of noise with the result being that the ratio of the signal to the noise decreases as you decrease the signal level.

AND THIS IS EXACTLY WHAT HAPPENS IN AN ANALOG SYSTEM. Since an analog system has a fixed noise floor even in the absence of a signal, as you reduce the signal level, you reduce the ratio of signal to noise.

The ONLY differences are that in an undithered digital system, the noise is correlated to the signal whereas in an analog system the noise is random and uncorrelated which is why quantization noise sounds worse than random analog noise. And that an undithered digital system cannot encode anything below -96dB as it simply gets quantized to 0 whereas an analog system can encode information below its noise level.

Now let's take it all the way down just at the "noise floor" at +/-1/2LSB level with -96.3dB attenuation

An externally hosted image should be here but it was not working when we last tested it.


Again, the number of sample points at plus and minus 1LSB diminishes further which again has the result of reducing the energy at 1kHz while increasing it elsewhere as we can see here:

The FFT

An externally hosted image should be here but it was not working when we last tested it.


And the stats

An externally hosted image should be here but it was not working when we last tested it.



Finally, for those who are grumbing about those FFT plots of the undithered digital waveform, let's see what an FFT of an analog system with a 1kHz sinewave recorded just at the noise floor might look like:

An externally hosted image should be here but it was not working when we last tested it.


:bigeyes: :bigeyes: :bigeyes: :bigeyes:

Now if you'll excuse me, I have to deliver a Bose Wave Radio to the local senior citizens center.

se
 
Konnichiwa,

Steve Eddy said:
Now let's take it all the way down just at the "noise floor" at +/-1/2LSB level with -96.3dB attenuation

Please post the wave files. I cannot get ANY of the wave programs I have to generate anything that looks at all like the files you show. Care to describe how you produced them so, others can verify your results?

Sayonara
 
Kuei Yang Wang said:
Please post the wave files. I cannot get ANY of the wave programs I have to generate anything that looks at all like the files you show. Care to describe how you produced them so, others can verify your results?

I can't post the WAV files because I'm using a trial version of CoolEdit 2000 which doesn't allow you to save anything.

How I produced them is pretty simple.

Turned dithering off.

Went to Generate and generated a 0dBFS 1kHz sine wave.

Then I went to Transform/Amplitude/Amplify, set the level to either -90.3dB, -96dB or -96.3dB depending on which one I was doing.

I zoomed in horizontally and vertically to get down to the point where you can see the individual samples and took screen shots of the waveform.

Then I went to Analyze/Frequency Analysis for the FFT and did a screen shot of those.

Then I went to Analyze/Statistics for the statistics and did a screen shot of those.

Simple.

Grab the trial version of CoolEdit 2000 and try it yourself.

I'd try it with CoolEdit Pro but I'm running on a regular Pentium system with Windows95. But I don't see why you would get any different results. How it's working in CoolEdit 2000 is simply how undithered 16-bit digital audio works.

se
 
Fred Dieckmann said:

I am agreement with that and the interesting thing is whether the smallest analog voltage change that the DAC can make is really a representation of the smallest signal still representative of the full scale sine wave from the same DAC.
It must be least argued that the signal is non-monotonic and differs in harmonic content to the extent the even the mathematicians should squirm at calling this an attenuated version of a sign wave. Using the juxtaposition of looking at this smallest signal possible from the DAC and trying deduce the full scale waveform leaves one with some real ambiguities as to possible signals it could be.

Not sure i follow you here. Are you saying the smallest output
signal we accept as a sine should be possible to multiply by
a constant (amplifying) and then be identical to the largest
sine we can get from the DAC? I am sure you know as well
as me that that is impossible with a quantised output, so
I must be misunderstanding you (and I really mean that, it
is not sarcasm).

I agree that i also find it a bit controversial whether the 1-bit
square wave should be accepted as the smallest sine or not,
but at the same time I think, that at least in my case, it is
probably the eyes that are trying to tell me, no that is not a
sine. Thinking rationally about it, we can represent sine waves
of various amplitudes, with varying degree of distorsion, the
smaller the amplitude, the higher the distorsion, which we
also agree on. Where do we draw the borderline? I am fully
aware that there is a difference between theory and practice,
and whatever figure for dynamics we agree on as the
theoretical limit, we will not be able to achieve this value in
practice. Howver, as I previously said to Frank, unless we know
what is theoretically possible, we don't know how well we
are doing in practice. So where is the borderline? On the one
hand we have the mathematical theory telling us what is
the smallest signal that can be represented and reconstructed,
assuming a theretically perfecct DAC, and a theoretically
perfect low-pass filter, both of which can only be approximated
in practice. We cannot achieve this in practice, but when
faced with a real implementation, we can measure and compare
to the theoretical value and know how close to perfect we are.
Isn't it a little bit like analog noise? If we know the theory on
thermal noise (we only need the usual formula in practice) we
can know in advance what the thermal noise is going to be
for a certain resistance so we know what the theoretical limit
is and that we may be able to get close, but never quite get
there. I am sure you made many designs where you have been
faced with getting as close to this theoretical limit as possible,
and that you would have had severe difficulties if you hadn't
been able to calculate this theoretical limit in advance. BTW,
I am sure Philips and Sony were very much interested in
knowing the theoretical limit of the CD medium when they
developed it.

Then, what is theory and what is practice? The mathematics is
the theory and the measured is practice, right? Hm, but what
matters in the end, in audio, is what we hear, and maybe the
measured values as just theory, not telling us much about the
practical matters of what we hear. It has certainly been
advocated by many on this forum that measured THD figures
are not of much practical use for the judging an amplifier (true
or not), so from this perspective, what was hand-on practical
in one case, is suddenly theory.
 
Kuei Yang Wang said:
Konnichiwa,



However you do it, all wave-edit/analysis software calls the resulting -90.3dbfs. We can argue endlessly, even tricking the system into something that most programs try not to let you do you end up with the signal being recognised as -90.3db, not -96db.

And I tell you again, that you should not trust the programs
to be right. Do you have any reliable information that any of
these programs were designed after a specification that
strictly adhered to the PCM standard? Did they even have
something worth a design specification? Probably not. Has
the final program been verified to implement the specification?
This is a next to impossible task and is extremely tedious to
do, for the lack of sufficient theoretical knowledge in the
area of program verification. Usually only programs in highly
safety-critical applications are verified to any degree worth
the name. Finally, is the compiler verified to be correct? Most
of them aren't you know. All Ada compilers must be certified by
DOD to be accepted as Ada compilers. Yet, I have experienced
a very puzzling bug that could hardly be explain as anything
else than a compiler error. Other compilers are usulle even
worse. I tried a Borland compiler that was supposed to adhere
to the latest C++ standard. It didn't even accept my program
as valid, while the excellent freeware GCC compiler did and
produced the expected result from the compiled program.
i know of at least one reserach group working on program
verification that do not verify the source code against the spec,
but instead verify the compiled executable against the spec,
since they mean that it would too dangerous to rely on the
compiler being correct.

This does not say that the programs are necessarily wrong, but
we cannot be sure.
 
Fred Dieckmann said:
'Also given that the average competence of programmers is scaring"

First rule of Telecom design:

Thou shalt not touch the software!

Which is a very sensible rule, unless one takes steps to get
a reasonable degree of knowledge of programming.

It is really scaring, that people who have taken one or two
introductory courses in progamming (usually no computer
science at all, rather practically oriented progamming) can
get well-paid jobs as programmers. Gee, I must have taken
about three times as many EE courses at university so by
that standard I should be able to apply for and get a top-job
at AD or BB!!! Of course I know that the courses I took are
not sufficient to qualify as an EE, and I wouldn't dream of
applying for a job in analog design based on those merits.

People pay high prices for lousy software often written by
programmers with no proper education for the job. Often
even trusting the program to run the companys financial
routines. Yet, nobody in his right minds would seek
treatment for an illness from somone who has taken two introductory courses in medicine.
 
Wait a minute........

It was accepted that the Sterophile measured wave form at -90.3 dB was the smallest analog signal that was smallest signal that that the DAC could swing about ground. The rational for a -96 dB signal was for a transistion from zero to either the smallest positive step or conversely between the smallest negative voltage step and zero volts. Either of these cases would not swing between positive and negative voltages but would
contain a DC component and no swing to the voltage of the opposite polarity.

You show a bipolar waveform which is an impossibility under the constraints above
as logically outlined by SY for a signal at this level.

How is this possible?

SY?
 
Re: Wait a minute........

Fred Dieckmann said:
It was accepted that the Sterophile measured wave form at -90.3 dB was the smallest analog signal that was smallest signal that that the DAC could swing about ground. The rational for a -96 dB signal was for a transistion from zero to either the smallest positive step or conversely between the smallest negative voltage step and zero volts. Either of these cases would not swing between positive and negative voltages but would
contain a DC component and no swing to the voltage of the opposite polarity.

You show a bipolar waveform which is an impossibility under the constraints above
as logically outlined by SY for a signal at this level.

How is this possible?

SY?

Not sure if you were referring to me or SY or both? My program
used the 1/2-bit offset suggested by SY, so it does not swing
symmetrically around zero, but around half the smallest output
voltage step. Whether this should be accepted as valid
PCM or not, I don't know. It is to the best of my knowledge
a valid PCM-encoded Wave file. It it complies with the Red Book,
I don't know, since I haven't read it. SY seems to be of the
opinion that this is valid, unless I have misunderstood him.

I have since seen Steves recently posted waveforms and see
that he gets a quite different result, actually getting three
different levels at -96.3 dB, which I haven't figured out yet
how that can be possible. Maybe my program doesn't do the
correct waveform generation, or the offset gives a different
result???
 
32 bit origin/Twos compliment

If SY was starting with a 32 bit signal, take the twos compliment for DSP manipulation without over running the 32 bit scale while making gain adjustments, reduce the signal level 96 dB... convert back to 16 bits, but maintain the twos compliment (easy to do since its just software, and we don’t really know what’s going on under the hood) and it is possible to have a square a bipolar square wave with just one a one bit change to signify its level.

In the twos compliment the MSB will determine weather the waveform has a positive or negative voltage about the output reference.

Watkinson/Rumsy have a nice explanation of this in "Digital Audio Interface Handbook". Its at home now and I'm not and therefore my explanation and "valuable" insight are both limited. Watkinson was doing digital interface and format design at for companies like Ampex before becoming an iconoclast.
 
The EE graduates aren't much better

"Of course I know that the courses I took are not sufficient to qualify as an EE, and I wouldn't dream of applying for a job in analog design based on those merits."

Courses for the EE are abstract enough that they don't have feel for real devices and circuits. I spent a long morning explaining how a diode works to a BSEE that was well on his way to earning a MSEE. He wasn't an idiot, they just don't deal with the real world in the limited time they have to teach a discipline that has become so broad in scope that there is no time for details required for use in the real world. I get blank staresfrom new EEs in response to questions that a trade school trained electronics technician could answer. If my audio mania had not supplemented my BSEE before I started work, the first year would have been pretty nonproductive. Articles by Nelson Pass and Erno Borbely probably saved my butt........ Thanks Nelson!
 
Re: Re: Wait a minute........

Christer said:
I have since seen Steves recently posted waveforms and see
that he gets a quite different result, actually getting three
different levels at -96.3 dB, which I haven't figured out yet
how that can be possible. Maybe my program doesn't do the
correct waveform generation, or the offset gives a different
result???

Forget about the offset. It's irrelevant. Going to a symmetrical bipolar scheme only loses 1 quantization step out of the 65,536 and the difference is only a tiny fraction of a dB in terms of dynamic range.

se
 
Sorry, you guys all type alike to me......

"I have since seen Steves recently posted waveforms and see that he gets a quite different result, actually getting three
different levels at -96.3 dB, which I haven't figured out yet how that can be possible. Maybe my program doesn't do the correct waveform generation, or the offset gives a different result???

It can't. Spice programs can even lie to you about analog waveforms. I would never rely on computer results that are not easy to correlate with what is happening in the real world. I call this the difference between computer added design and computer aided guessing. I hate those scenes with someone saying that can't be right because computer won't let you .......... and end with the discovery that some someone made a mistake in a database 4 years ago and you were the one to finally get burned by it.
 
WHAT

"Forget about the offset. It's irrelevant."

You must be kidding right? The difference between three possible output voltage levels and two and ignoring the difference between a positive and a negative voltage, is way too big a load of dirt to sweep under any carpet. I have never seen such skill in shooting oneself in the foot. Which toe were you trying to blow off?

"the difference is only a tiny fraction of a dB in terms of dynamic range."

No it is the difference between the 96dB and 90.3 dB that this riot stated over! I can hardly wait to see forum the reaction to this one.
 
Lowest level of the signal that can be properly made by SpectraLab’s utility “save signal to wave file” is -90.20dB. Going lower it works taking from the time and giving to the amplitude domain. This however does not last too long; in the case of the 1kHz signal, anything under -90.22dB produces an empty file.

Fred Dieckmann said:
"the difference is only a tiny fraction of a dB in terms of dynamic range."

No it is the difference between the 96dB and 90.3 dB that this riot stated over! I can hardly wait to see forum the reaction to this one.
IIRC, MBK pointed this a few pages back.

Pedja
 

Attachments

  • 1khz_sinewave.zip
    2.1 KB · Views: 35
Re: WHAT

Fred Dieckmann said:
"Forget about the offset. It's irrelevant."

You must be kidding right? The difference between three possible output voltage levels and two and ignoring the difference between a positive and a negative voltage, is way too big a load of dirt to sweep under any carpet. I have never seen such skill in shooting oneself in the foot. Which toe were you trying to blow off?

It's not a difference between three possible output voltge levels and two. It's about a positive level and a negative level about the 0 axis.

"the difference is only a tiny fraction of a dB in terms of dynamic range."

No it is the difference between the 96dB and 90.3 dB that this riot stated over! I can hardly wait to see forum the reaction to this one.

No, the 1/2LSB offset is not the difference between 96dB and 90.3dB. You're not losing your LSB with the 1/2LSB offset. You're only losing one quantization level.

So instead of being able to define the waveform with 65,536 discrete levels, you're defininig it with 65,535 discrete levels. If you lost your LSB, which WOULD make the difference between 96dB and 90.3dB, you'd have to lose 32,768 quantization levels. But you're not losing 32,768 quantization levels with the 1/2LSB offset.

Look, it's quite simple. 2 to the 16 is 65,536. If you use 65,636 discrete levels to make a PERFECTLY symmetrical bipolar system, then you'd have to have an offset of 1/2 bit because your 0 point takes away one quantization level leaving you with 65,535 to split on either side of zero.

But 65,535 is an odd number so you can't have an equal number of integer steps on each side of the zero point. You'd have to have 32,767.5 steps on one side and 32,767.5 steps on the other side.

So you need to lose one quantization step in order to get an even number. And that even number will be 65,534. Split that in two and we get 32,767 for each side of the zero.

And going back to what's been said numerous times already, your maximum error for the positive and negative halves will be +/-1/2LSB.

So for our dynamic range, we've got 20 x log (0.5/32,767) = 96.3dB.

se
 
Re: The EE graduates aren't much better

Fred Dieckmann said:
[BCourses for the EE are abstract enough that they don't have feel for real devices and circuits. I spent a long morning explaining how a diode works to a BSEE that was well on his way to earning a MSEE. He wasn't an idiot, they just don't deal with the real world in the limited time they have to teach a discipline that has become so broad in scope that there is no time for details required for use in the real world. I get blank staresfrom new EEs in response to questions that a trade school trained electronics technician could answer. If my audio mania had not supplemented my BSEE before I started work, the first year would have been pretty nonproductive. Articles by Nelson Pass and Erno Borbely probably saved my butt........ Thanks Nelson! [/B]

I was a bit surprised yesterday, actually, when reading that
excellent TI paper on PCB layout and ground planes that both
you and jcarr linked to in another thread. At one point the
author expressed a deep worry about most EEs not knowing
that the film in a film resistors usual is spiral-formed. I mean,
if he brings such a marginal thing up, he must be concerned.
Gee, I am not an EE, but I knew that already as a teenager
and we learnt it in high-school and in the introductory EE course
at university. It is just one isolated detail, and I do not in any
way mean to make any implications or compare american and
Swedish EEs, but it is somewhat puzzling. Maybe swedish
EE courses have become even less practical since I took them,
I don't know.
 
Re: Sorry, you guys all type alike to me......

Fred Dieckmann said:

It can't. Spice programs can even lie to you about analog waveforms. I would never rely on computer results that are not easy to correlate with what is happening in the real world. I call this the difference between computer added design and computer aided guessing. I hate those scenes with someone saying that can't be right because computer won't let you .......... and end with the discovery that some someone made a mistake in a database 4 years ago and you were the one to finally get burned by it.

I fully agree. I am a computer scientist, and I know how difficult
it is to be certain that a program does the right thing, not to
speak of erroneous data, as you suggested. Hence, I am always
careful when interpreting computer results. Spice a good example.
I haven't bothered to learn about the simulation algorithm, so
if the simulation won't work I fiddle with various step lenghts
etc. to see if I can get it running. Sometimes it works, but is
the results reliable?? Would never bet on that.

Also note my previous reply to Kuei on this matter.
 
Here, let's use a simple 4-bit system to illustrate how utterly insnigificant the 1/2LSB offset is.

In a 4-bit system you have 2 to the 4, or 16 possible values. They are:

0 = 0000
1 = 0001
2 = 0010
3 = 0011
4 = 0100
5 = 0101
6 = 0110
7 = 0111
8 = 1000
9 = 1001
10 = 1010
11 = 1011
12 = 1100
13 = 1101
14 = 1110
15 = 1111

Now let's turn this into a signed number system about the zero point using two's compliment:

+7 = 0111
+6 = 0110
+5 = 0101
+4 = 0100
+3 = 0011
+2 = 0010
+1 = 0001
0 = 0000
-1 = 1111
-2 = 1110
-3 = 1101
-4 = 1100
-5 = 1011
-6 = 1010
-7 = 1001
-8 = 1000

As you can see, we're not perfectly symmetrical about the zero point and a symmetrical waveform will only be able to swing +/-7 without clipping on the positive half.

So let's just drop one quantization level, -8 and we've got:

+7 = 0111
+6 = 0110
+5 = 0101
+4 = 0100
+3 = 0011
+2 = 0010
+1 = 0001
0 = 0000
-1 = 1111
-2 = 1110
-3 = 1101
-4 = 1100
-5 = 1011
-6 = 1010
-7 = 1001

Now we're perectly symmetrical.

With all 16 levels to describe the waveform, dynamic range would be 20 x log 16 = 24.0824dB.

But with just 15 levels to describe the waveform, dynamic range will be 20 x log 15 = 23.522dB.

For a loss of a mere 0.56dB (24.082 - 23.522) of dynamic range in just a 4-bit system.

With a signed 16 bit system, the calculation's the same. Instead of 65,536 discrete levels to describe the waveform, you've got 65,535 discrete levels.

With 65,536 levels we've got a dynamic range of 20 x log 65,536 = 96.329599dB.

With 65,535 levels, we've got a dynamic range of 20 x log 65,535 = 96.329466dB.

For a loss of 96.329599 - 96.329466 = 0.000133dB.

I think I can safey sweep that under the carpet.

se
 
Status
This old topic is closed. If you want to reopen this topic, contact a moderator using the "Report Post" button.