Audio Codecs for Voice AI

What are Audio Codecs for Voice AI?

An audio codec is the format and method used to encode voice audio into digital data, and to decode it back into sound. In voice AI, the codec and sample rate a system uses directly affect audio quality, bandwidth usage, and how accurately downstream speech recognition and voice generation perform. The two codecs that come up constantly in telephony and voice AI are PCM and mu-law, along with the sample rate each is typically paired with.

PCM (Pulse-Code Modulation)

PCM is an uncompressed way of representing audio: it takes a sound wave and measures its amplitude at regular intervals, storing each measurement as a number. Linear PCM at 16-bit depth is the standard used by most speech recognition and text-to-speech models, since it preserves the full resolution of the audio signal without the quality loss that comes from compression. The trade-off is size: uncompressed PCM audio takes up meaningfully more bandwidth and storage than a compressed format carrying the same duration of speech.

Mu-law and A-law (G.711)

Mu-law and A-law are companding algorithms defined under the ITU-T G.711 standard, and they’re the default audio codecs used across most of the world’s telephone networks, mu-law in North America and Japan, A-law almost everywhere else. Instead of storing full-resolution amplitude values like linear PCM, companding compresses the dynamic range non-linearly, allocating more precision to quieter sounds and less to louder ones, which mirrors how human hearing perceives loudness. This lets G.711 represent voice-quality audio in 8 bits per sample instead of 16, roughly halving the data size compared to linear PCM at the same sample rate, at the cost of some audio fidelity.

Why sample rate matters just as much as the codec

Sample rate is how many times per second the audio is measured, expressed in Hertz (Hz). Traditional telephony runs at 8kHz, sometimes called narrowband, which is sufficient to capture the frequency range of the human voice needed for intelligibility (roughly 300Hz to 3,400Hz) but noticeably strips out the fuller tone a caller hears in person. Voice AI systems, particularly speech recognition and text-to-speech models, often perform meaningfully better at 16kHz or higher, sometimes called wideband, since that extra frequency range carries information that improves transcription accuracy and makes synthesized speech sound more natural.

Where this creates friction in practice

A live phone call typically arrives at a voice AI system as 8kHz mu-law or A-law audio, matching standard telephony infrastructure, while many speech recognition and generation models are trained on 16kHz or higher linear PCM audio. Bridging that gap requires transcoding, converting from one codec and sample rate to another, and resampling, converting from one sample rate to another. Done well, this conversion is nearly transparent to call quality; done poorly, it introduces audible artifacts or subtly degrades transcription accuracy, which is one of the less visible reasons two voice AI systems can sound noticeably different even when using similar underlying models.

Use cases

• Telephony-to-AI bridging, converting 8kHz mu-law call audio arriving over a SIP trunk into the PCM format an STT or LLM pipeline expects.
• Voice AI vendor evaluation, comparing providers partly on what sample rate and codec their pipeline actually processes audio at, not just their advertised model quality.
• Voice streaming integrations, where the codec and sample rate used in the streamed audio format need to match what the receiving STT or TTS engine expects.

Benefits of getting codec and sample rate right

• Higher transcription accuracy: speech recognition models generally perform better on audio close to their training sample rate, rather than heavily downsampled or poorly transcoded audio.
• More natural-sounding synthesized speech: text-to-speech output at a higher sample rate carries more of the tonal detail that makes a voice sound human rather than compressed.
• Lower unnecessary bandwidth use: choosing a codec appropriate to the use case, compressed G.711 for standard telephony, higher-fidelity PCM where quality matters more, avoids wasting bandwidth without a corresponding quality benefit.

FAQs

What’s the difference between mu-law and A-law?
They’re both G.711 companding algorithms with the same underlying idea but different formulas and regional adoption: mu-law is standard in North America and Japan, while A-law is used across most of Europe and the rest of the world. A call crossing between regions using different standards typically needs to be transcoded.

Why don’t voice AI systems just always use the highest possible sample rate?
Higher sample rates mean more data per second of audio, which increases bandwidth use and processing load. Since standard telephony audio arrives at 8kHz regardless, there’s often limited benefit to processing at a much higher rate unless the original audio genuinely carries that additional detail.

Does codec choice affect voice AI latency?
Indirectly, yes. Heavier transcoding and resampling steps add small amounts of processing time, and using a codec that closely matches what the AI pipeline expects natively reduces the number of conversion steps needed before the audio can be processed.