Making Computer Voices Sound More Expressive from Your Speech

This patent describes how a computer listens to your spoken words, figures out which parts you emphasized, and then makes its own computer-generated voice emphasize those same parts when it speaks back.

What it covers

This patent describes a method for generating computer speech that mimics the emphasis from a user's original spoken input. First, the system receives input audio data representing speech and uses Automatic Speech Recognition (ASR) to convert it into text, referred to as "input data" (Claim 1). Then, it analyzes the input audio to determine "paralinguistic feature data" (Claim 1), which includes characteristics like pitch, volume, or duration of specific words. This paralinguistic data is used to identify which parts of the text should be emphasized. For example, if a user says "I *really* want coffee," the system detects the emphasis on "really." The system then prepares "output data" (Claim 1), potentially by adding a Speech Synthesis Markup Language (SSML) tag (Claim 4) to the emphasized portion. Finally, Text-to-Speech (TTS) processing is performed on this output data, generating speech where the identified portion, like "really," is spoken with corresponding emphasis (Claim 2), making the computer's response sound more natural.

What it doesn't cover

• —Does not cover text-to-speech emphasis based solely on linguistic rules or grammar without analyzing the user's original spoken input for paralinguistic features.
• —Does not cover systems that only transcribe speech to text without generating new, emphasized speech output.
• —Does not cover emphasizing computer-generated speech based purely on pre-written text where no input audio was analyzed for emphasis cues.
• —Does not cover systems where emphasis is determined solely by user preferences or profile data without considering the paralinguistic features of the current input audio.
• —Does not cover emphasis derived from non-audio inputs, such as visual cues or typed instructions, without corresponding analysis of spoken audio.

The clever bit

The clever part is directly linking the *way* a person speaks (their emphasis, pitch, or duration, called "paralinguistic feature data") from their input audio to how a computer then generates its own speech. Instead of just guessing which words to emphasize, the system uses the actual human vocal cues to make its response sound more natural.

Why it matters

This patent is important for making interactions with voice assistants and other spoken interfaces feel more natural and intuitive. By understanding *how* a user speaks, not just *what* they say, systems can respond with more appropriate vocal emphasis. This improves the user experience, making computer-generated speech less robotic and more engaging, which is crucial for widespread adoption of voice-controlled devices.

Real-world examples

• 1.Amazon Alexa
• 2.Google Assistant
• 3.Apple Siri
• 4.Microsoft Cortana
• 5.Voice-enabled smart home devices