Chris,
I really do not understand your need to complicate a very simple issue. How the files were extracted and whether they were bit perfect is completely irrelevant. The only thing that matters is that both files are identical and this can be verified by a bit-to-bit file compare or by checksum.
I really do not understand your need to complicate a very simple issue. How the files were extracted and whether they were bit perfect is completely irrelevant. The only thing that matters is that both files are identical and this can be verified by a bit-to-bit file compare or by checksum.
Hi Hugo,
The individual errors would probably be very short spikes that the reconstruction filter would decrease in amplitude. I'm not familiar with that software at all. I still have trouble considering that a perfect copy from an audio CD could be commonly made. Is it possible that, knowing perfect copies are very unlikely, there is a fudge factor in the comparison program. Is it only comparing CRC data, or the actual audio file? I know what they are telling you, but is it correct?
My professional experience with CD players is that they are most commonly not properly set up. The new servo-ed types (they adjust to the CD) are never as close to correct as a hand aligned unit. Not only that, but there is most commonly mechanical mis-alignment in new drives. The original NEC die cast transports allowed the head top be adjusted completely in every axis. These had about the most error free retrieval I have ever seen on any transport- ever. As CD transports have evolved, they have become cheaper with noisier and less well defined RF patterns.
The number one thing that must be concentrated on is the quality of the RF pattern. It is from this that the digital information is extracted from and the more noise there is, the more errors you will have. The amount of horizontal jitter (on a 'scope) is another defect that will increase read errors as well.
One thing that amazes me is that a cheap CDROM drive (cost what? $20?) is expected to read more accurately than a carefully designed, purpose built CD drive. Not only that, but we expect it to do this at much higher read rates where eccentricity of the CD itself becomes a huge problem, plus vertical warping (affects focus). Just considering those things I can't see why anyone expects accurate data from an audio CD.
The industry built in tolerance for imperfect data reads. They knew there would be errors. They knew that uncorrectable errors will occur (why else would they designed the C2 flag then?). Everyone was happy until the concept of "bit perfect" reads was advanced. Now what are they going to do? Tell the truth and admit the shortcomings of audio CDs? I don't think so.
My advice? Data errors do not bother anyone if they are kept to a minimum. As long as people think everything is okay, they will continue to be happy with the CDs. This is a problem that you simply can not win with. Physics is not on the side of perfect data transfer.
Hey! LP records pop, click and carry on. We still love those, so what is the problem with imperfect CD data files? We were all very happy for many years until recently. Accept reality and just enjoy the music.
-Chris 🙂
The individual errors would probably be very short spikes that the reconstruction filter would decrease in amplitude. I'm not familiar with that software at all. I still have trouble considering that a perfect copy from an audio CD could be commonly made. Is it possible that, knowing perfect copies are very unlikely, there is a fudge factor in the comparison program. Is it only comparing CRC data, or the actual audio file? I know what they are telling you, but is it correct?
My professional experience with CD players is that they are most commonly not properly set up. The new servo-ed types (they adjust to the CD) are never as close to correct as a hand aligned unit. Not only that, but there is most commonly mechanical mis-alignment in new drives. The original NEC die cast transports allowed the head top be adjusted completely in every axis. These had about the most error free retrieval I have ever seen on any transport- ever. As CD transports have evolved, they have become cheaper with noisier and less well defined RF patterns.
The number one thing that must be concentrated on is the quality of the RF pattern. It is from this that the digital information is extracted from and the more noise there is, the more errors you will have. The amount of horizontal jitter (on a 'scope) is another defect that will increase read errors as well.
One thing that amazes me is that a cheap CDROM drive (cost what? $20?) is expected to read more accurately than a carefully designed, purpose built CD drive. Not only that, but we expect it to do this at much higher read rates where eccentricity of the CD itself becomes a huge problem, plus vertical warping (affects focus). Just considering those things I can't see why anyone expects accurate data from an audio CD.
The industry built in tolerance for imperfect data reads. They knew there would be errors. They knew that uncorrectable errors will occur (why else would they designed the C2 flag then?). Everyone was happy until the concept of "bit perfect" reads was advanced. Now what are they going to do? Tell the truth and admit the shortcomings of audio CDs? I don't think so.
My advice? Data errors do not bother anyone if they are kept to a minimum. As long as people think everything is okay, they will continue to be happy with the CDs. This is a problem that you simply can not win with. Physics is not on the side of perfect data transfer.
Hey! LP records pop, click and carry on. We still love those, so what is the problem with imperfect CD data files? We were all very happy for many years until recently. Accept reality and just enjoy the music.
-Chris 🙂
Hi analog_sa,
Look at this another way. If you believe that all mechanisms and servo systems can equally extract perfect data, then the least expensive CD player would work as well as an expensive one. CDROMs are after all the cheapest transport / servo systems out there - right?
I did end off with a statement running along the lines of, "accept the problems and be happy".
-Chris
I'm not complicating anything. I am merely pointing out the reality of audio CDs. I have a very strong dislike for "facts" that are incorrect.
I am sure then that we disagree. The extraction is fundamental to the basic concept of "bit perfect". With all the errors present, I find it amazing that anyone would accept this premise to begin with.
Look at this another way. If you believe that all mechanisms and servo systems can equally extract perfect data, then the least expensive CD player would work as well as an expensive one. CDROMs are after all the cheapest transport / servo systems out there - right?
And this comment is exactly where I believe the problem is. I do not believe that the checking methods actually tell the truth. In fact, I would believe that the program(s) might report perfect agreement between files as long as each consecutive block errors are below some threshold. It's not like we can hear individual samples, especially after going through the reconstruction filters in a CD player.
I did end off with a statement running along the lines of, "accept the problems and be happy".
-Chris
File bit comparisons have a threshold?! I guess they have but its under one bit.
Or is yours a more philosophical view towards files: no two can ever be quite the same?
I have a suspicion that you know too much about the cd reading process, which is sadly, completely irrelevant in this thread and too little about files.
Or is yours a more philosophical view towards files: no two can ever be quite the same?
I have a suspicion that you know too much about the cd reading process, which is sadly, completely irrelevant in this thread and too little about files.
I am interested in reading this... EDIT: Must be http://ham-radio.com/k6sti/xdr-f1hd.htmscott wurcer said:
anatech said:
The situation is CRC and MD5 algorithms operating on WAV files. The algorithms don't know or care what sort of data they're working on. In any case, the checksums are just a human-readable convenience -- I am confident that a bit-by-bit binary diff will show that the files are identical, in the strictest sense of the word.
I don't have your expertise about CD playing mechanisms, but as a programmer who uses and depends on perfect CD copying of executable computer code, on regular computer CD or DVD drives, on which a single bit lost will likely make a computer program diverge off to insanity and crashes... I find that really hard to believe. It seems like imperfect copies are what would be extremely unlikely. Do the disk copy programs operate differently when they are copying audio data as when they are copying the **completely unforgiving** executable data?
Also, your version of bit-perfect comparison makes no sense at all. It is trivial to take the audio data (and/or other data) files from CDs and do the extremely simple operations of XOR or SUBTRACTION on a byte-per-byte basis. I've programmed that function, it takes very little time to do or to execute, and think that if any programmer were going to the trouble of digging out the CRCs from an audio disk rather than just comparing the actual bits, well he really needs to find another profession!
Maybe that's because CDs are used out in the wild where over time they get scuffed, scratched and covered in finger prints? For CD data copies, that causes a lot more problems than possible data copying errors. Data copying errors do happen, those are the ones that cause the "copy failed" message from the verification process on burner programs that makes the disk go into the can.
bwaslo said:
Indeed they do. In order to save space and reach 60 minutes playing time there is simply not enough redundant data on the cd to correct all errors. For a while i thought this was really a problem but in practice it isn't. Most transports manage to read correctly practically all the data of a cd unless some sillly digital manupulation is done as in one infamous philips player. It is quite easy to read cd data off an spdif output and compare this to data extracted by EAC. Remarkably, once aligned the data files are practically identical. So, for me the idea that cd transports read different data is mostly a red herring. How transports handle this data and specifically the spdif circuitry is a completely different issue.
This is generally interesting but is not what this thread is all about.
anatech,
You clearly know a lot about how CDs are read, but please try to read what the people here are actually saying. The commentary so far is not anything to do with reading CDs. At all. It's about files which already exist on a hard drive and are being read out to a playback system. There isn't a CD in sight. So when people have said it doesn't matter about how well the CD is read, they're not telling you that they don't think the odd bit error is worth worrying about. What they're saying is it's not relevant to the discussion at hand.
You clearly know a lot about how CDs are read, but please try to read what the people here are actually saying. The commentary so far is not anything to do with reading CDs. At all. It's about files which already exist on a hard drive and are being read out to a playback system. There isn't a CD in sight. So when people have said it doesn't matter about how well the CD is read, they're not telling you that they don't think the odd bit error is worth worrying about. What they're saying is it's not relevant to the discussion at hand.
Okay fellas,
I am assuming the audio file originates with a CD. Could be an MP3 or data file. You can not assume that a file from an audio CD is correct. End of message.
BTW, mako1138, the way data CDs are encoded is different to the way a CD is encoded. Where a single bit lost on a program is catastrophic, it isn't on an audio CD. Look it up.
Hi analog_sa,
My intention was to simply call attention to the fact that some pretty big assumptions are being made here.
My experience with digital audio runs from CD encoding and reproduction and also multi-track digital recording. I know what is, and what is not guaranteed to be capable of exact digital copies. I am also aware of what happens with VoIP files over the PSTN or local network.
Errors are a part of life. Get use to it and be suspicious of any claim of perfection. Besides, apart from programs and data, you probably will not notice these errors anyway.
-Chris
I am assuming the audio file originates with a CD. Could be an MP3 or data file. You can not assume that a file from an audio CD is correct. End of message.
BTW, mako1138, the way data CDs are encoded is different to the way a CD is encoded. Where a single bit lost on a program is catastrophic, it isn't on an audio CD. Look it up.
Hi analog_sa,
A data file comparison is an exact comparison. I use these programs a fair bit <intended> myself. I am commenting on programs that might compare audio CDs. I'm just not sure and I have learned not to trust things concerning media files in general.
My intention was to simply call attention to the fact that some pretty big assumptions are being made here.
My experience with digital audio runs from CD encoding and reproduction and also multi-track digital recording. I know what is, and what is not guaranteed to be capable of exact digital copies. I am also aware of what happens with VoIP files over the PSTN or local network.
Errors are a part of life. Get use to it and be suspicious of any claim of perfection. Besides, apart from programs and data, you probably will not notice these errors anyway.
-Chris
If you follow the reasoning in one of the older TAS articles, bit error should not be responsible for different sound signatures in digital playback:
http://www.audiostereo.pl/zalaczniki/1205149_1.jpg
http://www.audiostereo.pl/zalaczniki/1205149_2.jpg
http://www.audiostereo.pl/zalaczniki/1205149_1.jpg
http://www.audiostereo.pl/zalaczniki/1205149_2.jpg
anatech said:
The debate in this thread is about comparing data files originating from audio CDs. Once the audio CDs have been ripped, with unrecoverable errors or not, we have data files. Agree/disagree?
A data file comparison is an exact comparison as you say. We look at each bit and see if it matches. All bits need to match, else the data files are not identical. Agree/disagree?
Next I present two 16-bit audio samples in binary form:
0100011101010110
0100011101010110
As we can readily see they are identical. Now let's suppose the samples came from two different audio CDs through the long, error prone journey from the optical disc to the computer hard drive. The samples are still identical and there is no ambiguity about it! Agree/disagree?
breez said:
This first sequence definitely has a more defined, open sound stage whereas the second is rather indistinct.
🙂
Hi breez,
We had moved on already. It is blindingly obvious that a real bit to bit comparison would should any error.
My premise was that I do not trust programs designed to compare audio CD's as I do not believe they actually compare bit to bit. More likely, they compare CRC values in each file for each block of data. Even then, they may accept a number of concurrent errors before the error is judged to be audible and therefore needs to be flagged.
It would take a long time to run a bit by bit comparison, as anyone who has ever compared data files down to the bit level can attest.
In the end, your statements are overly simplified and ignore the earlier point I made about this. You are making an assumption with nothing to back up your statement. You are assuming the comparison program is indeed checking each bit. That is where our views part ways.
Hi Peter,
-Chris
We had moved on already. It is blindingly obvious that a real bit to bit comparison would should any error.
My premise was that I do not trust programs designed to compare audio CD's as I do not believe they actually compare bit to bit. More likely, they compare CRC values in each file for each block of data. Even then, they may accept a number of concurrent errors before the error is judged to be audible and therefore needs to be flagged.
It would take a long time to run a bit by bit comparison, as anyone who has ever compared data files down to the bit level can attest.
In the end, your statements are overly simplified and ignore the earlier point I made about this. You are making an assumption with nothing to back up your statement. You are assuming the comparison program is indeed checking each bit. That is where our views part ways.
Hi Peter,
With that I will completely agree with you. There are more than enough examples for equipment that uses the same chip sets and transports, and yet sound completely different. How the audio signal is treated after D to A conversion (from any data source) that makes all the difference in the world. Of course, this puts us back in the audio realm again and wouldn't have much to do with bit errors as long as they remain low.
-Chris
Hi Scott,
-Chris
True, but then I have more trust in an open concept of software. I will be putting another Linux machine up to run my gear on the bench. Living with windows sucks a lot of time fixing the system instead of doing real work.
-Chris
anatech, the more I read through your posts, the more confused I get about where the disconnect lies. You seem to be talking about low-level CRC checks during the CD reading process, but everyone else is talking about checksums run on already-ripped files.
If your point is that data from audio CDs can't be trusted 100%, due to the limitations of the format, I certainly agree.
If your point is that data from audio CDs can't be trusted 100%, due to the limitations of the format, I certainly agree.
anatech,
in the thread the OP linked to, sandyK created md5 checksums comparing the files which clearly showed they are absolutely identical, and then claimed there were consistent audible differences depending on what drive they came off of.
What everyone is saying is this is impossible. (Me too.)
Checksum collisions are incredibly unlikely. Basically, not all hashes are cryptographically secure (you can create a collision if you really want to and have the computing power) but even with CRC32, the odds of a hash collision at random are astronomical. And even the most basic hash algorithms (CRC32) really do check every single bit.
As you say, if you actually can't verify that the files are identical, then they could sound different. Though I've yet to see anyone show EAC consistently producing actual different output (except for tiny time offsets) when there aren't C2 errors -- With ANY kind of drive.
in the thread the OP linked to, sandyK created md5 checksums comparing the files which clearly showed they are absolutely identical, and then claimed there were consistent audible differences depending on what drive they came off of.
What everyone is saying is this is impossible. (Me too.)
Checksum collisions are incredibly unlikely. Basically, not all hashes are cryptographically secure (you can create a collision if you really want to and have the computing power) but even with CRC32, the odds of a hash collision at random are astronomical. And even the most basic hash algorithms (CRC32) really do check every single bit.
As you say, if you actually can't verify that the files are identical, then they could sound different. Though I've yet to see anyone show EAC consistently producing actual different output (except for tiny time offsets) when there aren't C2 errors -- With ANY kind of drive.
As I said before, NO IT WOULDN'T AND NO IT DOESN'T. Even in Windows! It takes a miniscule bit longer to subtract the data words from those on the HDD file than it takes to just read the CD. It uses about the simplest math there is for the processor. With a drive that reads at an average of 32x, it would be about 2 minutes for an entire CD, if it were full. You don't compare each bit individually, you compare entire words, 32 bits at a time.
It isn't hard to find code or pre-written DLLs to rip audio data from CD to WAV. But I have never seen code or DLLs (though I'm sure they exist) to read the CRCs, that would require quite low level accessing of the drive. A programmer will do the first unless he is getting special brownie points and has a big budget for fooling around with CRCs.
When NERO does a data verification, it takes pretty much just as long as it takes to read the disk into a buffer to make the copy. If it is reading just CRCs then I don't know what it would be doing with the rest of that time.
Rescue Toaster said:
Data is reclocked several times inside a computer, so if there are time offsets they will not depend on the source material.
ionomolo said:
Sorry, I wasn't trying to imply that jitter is somehow stored (that is insane). Just that two CD-ROM drives might see the start of an audio track in different places. ie one might have a tiny bit of silence at the start. At least I think that's what the offset setting is for in EAC. So even if two .wav files may not be identical, the actual audio data still should be.
And yeah, there's like 30 asynchronous buffers between EAC ripping a CD-ROM and a soundcard. Not least of which is inside the CD-ROM itself. Data is clocked out of the drive by the ATA or SATA clocks, not the CD-ROM master clock.
- Status
- Not open for further replies.