Noam M. Shazeer

How AI Models Understand Language Using 'Attention'

This patent describes a neural network architecture, known as a Transformer, that uses a "self-attention" mechanism to process sequences of information, like words in a sentence, by weighing the importance of different parts of the input.