As has been said series coupling capacitors are there to block DC between stages and to form high pass elements.
If you have circuits that operate at unity gain and are all ground centred, ie bi-polar, then coupling caps aren't really necessary. Where you get issues is in circuits that have gain (power amplifiers) or between two circuits that operate at different bias points.
In a unity configuration no gain is applied, so if a small amount of offset is present on the output of one stage feeding another, then that offset will be apparent on the output of the final stage, ignoring that stages own DC offsets. If you've got a gain element within the second stage then the offset applied to it will be amplified by the gain factor. This could become extremely significant and cause massive issues. This is why power amplifiers use DC coupling on their inputs. Some do not but these use active circuitry to compensate for any DC that may appear on the output.
In other cases you will have DACs or (usually) single supply circuits that require a DC bias be applied to work properly with audio signals. Typically audio can be represented as a sine wave, that is a signal that swings positive and negative about a common zero point in the middle. Ideally circuits approach this by having both a positive and negative supply, like so they can swing both above and below the common ground reference point (zero volts) and viola we have audio. But if your circuit has to be powered from a single supply, it cannot swing both directions relative to zero. The way these device get around this is by adding a DC bias to their circuitry. This is usually set to the supply voltage divided by 2, or something very similar. This bias is now used to define a new operating point around which the sine wave can function. Zero has now been redefined as supply/2 so the signal can now effectively swing between zero and supply, resting in the middle at supply/2.
Obviously the DC bias applied to the circuit (whatever supply/2 is) cannot be applied to anything before or after said circuit so DC coupling caps will always be used to ensure that the inputs and outputs have zero volts present on them.
They are called coupling caps because they are used to couple one stage to another. Technically they do decouple different DC operating points between stages, but decoupling caps are used to describe caps that sit between a power rail and ground as a way of reducing voltage ripple from the power rail, provide a limited low impedance source of energy for said circuit and to help keep the power rail free from noise.