79. The Attention Mechanism: Focus on Important Parts
The word "it" in this sentence: "The animal didn't cross the street because it was too tired." What does "it" refer to? The animal or the street? A human reads this and immediately knows: the animal was tired. The street cannot be tired. For a model processing this word by word, the word "it" arr...