How to Automatically Generate Musical Harmonies from Audio

This 2012 patent describes a system that listens to music and automatically generates harmony notes to accompany a melody, even detecting and ignoring accidental strums on stringed instruments.

What it covers

This patent details an apparatus that can create musical harmonies. It takes a melody signal and an accompaniment signal, analyzes their sound content, and then figures out what harmony notes would sound good. Specifically, it identifies the current melody note and looks at the 'spectral content' (basically, the mix of frequencies) of the accompaniment music. Using this information, it generates at least one harmony note. A clever part is its ability to detect and ignore 'unintentional strums' from stringed instruments, like when a guitarist accidentally brushes strings during a chord change, so it doesn't mess up the generated harmony. It can even output these harmonies as MIDI data or mix them with the original sounds for real-time performance.

What it doesn't cover

• —Generating harmonies without analyzing the spectral content of an accompaniment signal
• —Generating harmonies without identifying a current melody note
• —Harmony generation that does not suppress determination based on an unintentional strum
• —Methods that do not involve receiving at least one polyphonic electrical signal from a multi-stringed instrument
• —Detecting unintentional strums without comparing received notes to templates based on open string tuning

The clever bit

The innovation lies in the system's ability to not only generate harmonies but also to intelligently filter out unwanted musical 'noise,' specifically unintentional strums on stringed instruments, ensuring the generated harmony remains musically coherent.

Why it matters

This patent addresses a core challenge in music production: creating convincing instrumental accompaniments and harmonies automatically. It provides a technical solution for software and hardware that can assist musicians, from hobbyists to professionals, in arranging and performing music by intelligently adding harmonic layers.

Real-world examples

• 1.Music production software (DAWs) with auto-harmony features
• 2.Digital audio workstations (DAWs)
• 3.Live performance accompaniment systems
• 4.Virtual instrument plugins