Lukasz Mieczyslaw Kaiser

How AI Models Understand Language Using Self-Attention

This patent describes a neural network architecture, known as the Transformer, that uses a "self-attention" mechanism to process sequences of information, like words in a sentence, by weighing the importance of different parts of the input to generate an output sequence.