SY said:
= 1 Hartley.
Occasionally you'll see someone refer to a bit as a "Shannon", but that never caught on.
As far as I know, Shannon used the term "entropy" for information content as kind of a pun, just because the formula looked a lot like the one familiar from thermodynamics. It was only later that it turned out not to be a coincidence at all.
Anyone who's interested in digital communication (and *definitely* anyone who teaches CS!) ought to take a look at Shannon's original papers. They're really quite amazing, and available online.
http://cm.bell-labs.com/cm/ms/what/shannonday/paper.html
DAC: digtal analog converter
"patent on his newly invented three-value bits"
Please tell me this is an attempt at humor and that you know the difference between the analog output voltage of a DAC and the digital input. If you are honestly that confused, I can only assume your are joking about the Ph.D. or need to do a lot of reading before even attempting to offer an opinion here. This is really not meant to funny, its just sad..........
"patent on his newly invented three-value bits"
Please tell me this is an attempt at humor and that you know the difference between the analog output voltage of a DAC and the digital input. If you are honestly that confused, I can only assume your are joking about the Ph.D. or need to do a lot of reading before even attempting to offer an opinion here. This is really not meant to funny, its just sad..........
nats!
SY
don't forget to make that Q reversible.
I don't see anything wrong with Shannon's def of entropy of a probability distribution p:
S(p)=-Sigma
pn log(pn)
this discussion reminds me of my discussions with my Korean postdoc colleague that sits at its desk reading articles and then comes up with 'new ideas' at the group meeting ideas we tried years ago and that didn't work. He force me to be the Fred of the situation and re-explain to my boss why stuff didn't work and why we shouldn't pursue this crap further, half the stuff I myself can't remember anymore.
What I am getting back from the guy is a bunch of according to this paper blah blah blah, according to that paper blah blah blah. The guy hasn't grasped the simple concept that papers, books and other academic treaties are often the result of the need for grant approval, tenure promotion, sometimes genuine intellectual curiosity, and are very rarely generally applicable.
SY
don't forget to make that Q reversible.
I don't see anything wrong with Shannon's def of entropy of a probability distribution p:
S(p)=-Sigma
pn log(pn)this discussion reminds me of my discussions with my Korean postdoc colleague that sits at its desk reading articles and then comes up with 'new ideas' at the group meeting ideas we tried years ago and that didn't work. He force me to be the Fred of the situation and re-explain to my boss why stuff didn't work and why we shouldn't pursue this crap further, half the stuff I myself can't remember anymore.
What I am getting back from the guy is a bunch of according to this paper blah blah blah, according to that paper blah blah blah. The guy hasn't grasped the simple concept that papers, books and other academic treaties are often the result of the need for grant approval, tenure promotion, sometimes genuine intellectual curiosity, and are very rarely generally applicable.
grataku: Let's just say that my post-doc experience was not too different
Qrev, yes, but easier to write k ln w.
all: Anyone who doubts that an undithered sampling system can resolve a +/-1/2 bit sinewave should look at Vanderkooy and Lipshitz's JAES paper (March 1984). The paper's purpose was to quantitate the distortion-reducing effect of dither, but it shows waveforms and spectra of undithered for comparison. The spectrum shows a 1kHz fundamental and a lot of harmonics, just like Fred's helpful Stereophile graph. They didn't find it remarkable that their control data were exactly what anyone who understands sampling theory would expect, like someone measuring a current through a resistor and finding that it followed Ohm's Law.
Apparently they know how to use their software. And understand the theory.
Qrev, yes, but easier to write k ln w.
all: Anyone who doubts that an undithered sampling system can resolve a +/-1/2 bit sinewave should look at Vanderkooy and Lipshitz's JAES paper (March 1984). The paper's purpose was to quantitate the distortion-reducing effect of dither, but it shows waveforms and spectra of undithered for comparison. The spectrum shows a 1kHz fundamental and a lot of harmonics, just like Fred's helpful Stereophile graph. They didn't find it remarkable that their control data were exactly what anyone who understands sampling theory would expect, like someone measuring a current through a resistor and finding that it followed Ohm's Law.
Apparently they know how to use their software. And understand the theory.
SY said:
It rather boggles the mind, doesn't it?
These are the ABSOLUTE limits of the CD Format. Below this (-90.3db) or above this (0db) there is no way to record any Information.
What is so difficult to understand that a signal with a magnitude of at least +/-1/2LSB will indeed be recorded and quantized to +/-1LSB with an error of +/-1/2LSB?
se
Posted by:SY
That one really rang a bell and yet I'd never read Ken's book. Jeepers Ive wasted the whole evening digging.
"In a system using n bit words, there will be 2ee
quanitising intervals. The largest sinusoidal which can fit without clipping will have this peak-to-peak amplitude. The peak amplitude will be half as great i.e. 2ee-Q and the r.m.s. amplitude will be this value divided by 2ee(.5).The quanitising error has an amplitude of 1/2Q peak which is the equivalent of Q/(12ee(.5)) r.m.s. The signal to noise ratio for the large signal case is then given by:
20log10 (12ee(.5) x 2ee
-1) / 2ee(.5) dB=20log10 (6ee(.5) x 2ee
-1) dB(Second class phone wipers mathematical simplification)
=6.02n + 1.76dB
By way of example a 16 bit system will offer around 98.1 dB SNR."
-John Watkinson, "The Art of Digital Audio" Second edition (autographed copy)
"The Expression above is only valid if the probability density of the quanitisation error is uniform. Unfortunately at low levels and with pure or simple waveforms, this simple is not the case"
-IBid
"The signal to noise ratio derived above has no relevance to practical audio shaping applications as it will be modified by the dither and the noise shaping used"
-Ibid
And now for a few quick words about dither: John paraphrases a number of seminal papers on non subtractive dither published in the mid 80’s
· Rectangular p.d.f. dither 6.02n-1.24dB about 95.1 dB but with much greater linearity as quanitsation errors are minimized.
· Triangular p.d.f. dither 6.02n-3dB about 93.3 dB perfect linearity and unchanging noise floor.
· Gaussian p.d.f. dither 6.02(n-1)+1.76dB about 92.1dB SNR.
· Analogue dither is not necessary as all ADC contain gaussian thermal noise
· Over sampling and or noise shaping will significantly change the meaningful SNR.
Of course all this defining and arguing reminds me that the JND for SPL in most humans is measured at around 3dB. So with a 16 bit signal we can sample and recreate an audible waveform with a SNR in the mid to high 90’s. Not too shabby but still audibly less than we should like it. 24 bits, now there’s something that will consistently sound better than the ½ track Ampexes at 15 IPS that I used to drag around for location recording.
This was actualy an ATR 700 which was not too stable.
Have you seen the 1" 1/2 track ATR 102's that some of the mastering studios mave been using? I understand they will typicly run them at 30 IPS.
I know and love the 350/1, wish I could think of a reason for keeping one.
BTW: Fred, Whats this Aggie DC your trying to sell us?
Have you seen the 1" 1/2 track ATR 102's that some of the mastering studios mave been using? I understand they will typicly run them at 30 IPS.
I know and love the 350/1, wish I could think of a reason for keeping one.
BTW: Fred, Whats this Aggie DC your trying to sell us?
Attachments
Re: DAC: digtal analog converter
Of course it was a joke, as I hope you have understood I have
merely been trying to be sarcastic towards you in the same
way that you so often are to others. You did, however not
understand the concept LSB, or you made a very confused
explanation earlier. I am happy to see that you can say
something crisp and clear, as you just did, instead of just
make obscure insinuations by posting things with no clue as
to how you yourself interpret it or how others are to interpret
it.
BTW, on those few occasions I have mentioned that I have
a PhD, it has not been in any way to show-off, or claim that
I am in any way superior to you or anybody else. A PhD title
does not impress me, and shouldn't impress anybody else
either.
Fred Dieckmann said:
Of course it was a joke, as I hope you have understood I have
merely been trying to be sarcastic towards you in the same
way that you so often are to others. You did, however not
understand the concept LSB, or you made a very confused
explanation earlier. I am happy to see that you can say
something crisp and clear, as you just did, instead of just
make obscure insinuations by posting things with no clue as
to how you yourself interpret it or how others are to interpret
it.
BTW, on those few occasions I have mentioned that I have
a PhD, it has not been in any way to show-off, or claim that
I am in any way superior to you or anybody else. A PhD title
does not impress me, and shouldn't impress anybody else
either.
To the rescue of Fred ? (no pun, no hard feelings)
Fred,
I did get an idea today. If it is the case that you truly do believe
you were right about what you said, then could it be that analog
designers commonly use the terminology in a quite different
way from what digital designers, computers scientists,
mathematicians etc. do, so that, with a different interpretation
of the terminology, your explanation actually does make sense?
I actually even have an idea of how your thinking might go,
but it is not yet clear enough to see if it would give a
sensible reading.
To get it straigh, with no puns, no ill intentions. In digital
electronic design (I actually do have professional experience
from that area, and know my terminology to be 100% consistent
with all others that worked at the same company) and computer
science (my current area of profession), to "toggle" a bit, means
that you change its value. If it is zero, you make it a one and if
it is one you make it a zero. If you toggle a bit twice, you get the
original value. Now, let's have a 16-bit DAC. You may choose
whatever binary input x you wish, which results in an output
voltage Vx. Now, I let you choose any bit you wish, out the
16 possible. You can choose the LSB, but you don't have to.
Toggle that bit. Note here, that toggling a bit is a strictly local
operation. None of the other 15 bits will change their value,
only the one you have choosen. This will result in a new input
y, which differs in one bit from x (ie. the Hamming distance
between x and y is 1). The input y gives the output voltage
Vy, which is different from Vx (at least for the theoretical DAC).
Now toggle the same bit again and you will get the input x
and the output voltage Vx. There is no way you can get a third
distinct input value z and a corresponding third distinct output
voltage Vz by just toggling one bit.
This is in accordance with my terminology (and most certainly
SY's). I think it should make it clear to you if we are using the
same terminology or not. If the above does not fit your
terminology, then you could very well be right according to
some other, non-standard, interpretation of the terminology.
Fred,
I did get an idea today. If it is the case that you truly do believe
you were right about what you said, then could it be that analog
designers commonly use the terminology in a quite different
way from what digital designers, computers scientists,
mathematicians etc. do, so that, with a different interpretation
of the terminology, your explanation actually does make sense?
I actually even have an idea of how your thinking might go,
but it is not yet clear enough to see if it would give a
sensible reading.
To get it straigh, with no puns, no ill intentions. In digital
electronic design (I actually do have professional experience
from that area, and know my terminology to be 100% consistent
with all others that worked at the same company) and computer
science (my current area of profession), to "toggle" a bit, means
that you change its value. If it is zero, you make it a one and if
it is one you make it a zero. If you toggle a bit twice, you get the
original value. Now, let's have a 16-bit DAC. You may choose
whatever binary input x you wish, which results in an output
voltage Vx. Now, I let you choose any bit you wish, out the
16 possible. You can choose the LSB, but you don't have to.
Toggle that bit. Note here, that toggling a bit is a strictly local
operation. None of the other 15 bits will change their value,
only the one you have choosen. This will result in a new input
y, which differs in one bit from x (ie. the Hamming distance
between x and y is 1). The input y gives the output voltage
Vy, which is different from Vx (at least for the theoretical DAC).
Now toggle the same bit again and you will get the input x
and the output voltage Vx. There is no way you can get a third
distinct input value z and a corresponding third distinct output
voltage Vz by just toggling one bit.
This is in accordance with my terminology (and most certainly
SY's). I think it should make it clear to you if we are using the
same terminology or not. If the above does not fit your
terminology, then you could very well be right according to
some other, non-standard, interpretation of the terminology.
I thought of that myself and asked a pretty prominent analog designer. He looked at me like I was funning him. The answer, limited to my sample-of-one survey, is no. That's backed up by the Lipshitz/Vanderkooy reference and Pohlmann.
I think that Fred is a smart and capable guy, but this is one case where I believe he's incorrect. News flash: Fred's human!
- Status
- This old topic is closed. If you want to reopen this topic, contact a moderator using the "Report Post" button.