BERT
BERT (Bidirectional Encoder Representations from Transformers) is a Transformer-based natural language understanding model developed by Google that reads each word together with the context on both sides, capturing even subtle meaning-shifting elements such as prepositions. It was applied to Google Search in 2019, allowing the engine to interpret the intent behind conversational and long-tail queries far more accurately.
- BERT stands for Bidirectional Encoder Representations from Transformers, a natural language understanding model that interprets each word using context from both directions at once.
- Google brought BERT to Search in 2019 and said at launch that it affected roughly one in ten English-language searches in the United States.
- Because it understands words that subtly change meaning, such as prepositions and word order, it is especially strong at reading the intent of conversational and long-tail queries.
- Unlike RankBrain, a machine learning system that helps rank content, BERT focuses on understanding the language of the query itself.
- From an SEO standpoint, the key in the BERT era is to write clearly and naturally for people rather than stuffing in keywords.
Overview
BERT (Bidirectional Encoder Representations from Transformers) is a Transformer-based natural language understanding (NLP) model introduced by Google's AI researchers. Where earlier language models read a sentence in a single direction, such as left to right, BERT grasps meaning bidirectionally by weighing the context before and after a word at the same time. When Google brought the technology to Search in 2019, it described the change as "the biggest leap forward in the past five years, and one of the biggest leaps forward in the history of Search."
The crux is that BERT understands small but decisive words, like prepositions and word order, that can completely change what a query means. In the query "2019 brazil traveler to usa need a visa," for example, the preposition "to" signals that a Brazilian is traveling to the United States. Before BERT, Search could miss that relationship and surface results for the opposite direction.
How It Works
BERT is pre-trained on large amounts of unlabeled text using a masked language model approach. Some words in a sentence are hidden, and the model learns to predict them using the full surrounding context on both sides, so it can interpret a word in relation to all of its neighbors. This bidirectional context handling lets it distinguish which sense of the same word is intended within a given sentence.
In a real search example, for the query "do estheticians stand a lot at work," BERT understands that "stand" refers to physically standing. Earlier systems sometimes tied it incorrectly to unrelated expressions such as "stand-alone."
Impact on Search
When Google announced the rollout, it stated that BERT affected about one in ten English-language searches in the United States. The areas of greatest impact include the following.
- Conversational queries: It more accurately interprets the intent behind long, sentence-style searches that people type the way they speak.
- Long-tail and specific queries: It captures the detailed context, such as who, in which direction, and in what situation, that a few keywords alone could not convey.
- Featured snippets: Google improved featured snippet quality with BERT across 24 countries and reported especially large gains in Korean, Hindi, and Portuguese.
Difference from RankBrain
BERT is often mentioned alongside RankBrain, but the two play different roles. RankBrain is closer to a machine learning system that adjusts the ranking of search results, while BERT focuses on understanding the meaning of the language within the query itself. The two are not competitors; they are components in the same family of language understanding that complement different stages as Google interprets queries and arranges results.
SEO Implications
BERT neutralizes the practice of mechanically packing in keywords. Because the model understands context and intent, content that is written clearly and naturally for readers performs better than content that repeats keywords awkwardly. In practical terms, the following are recommended.
- Write in the question forms that real users actually pose, both conversational and long-tail.
- Sharpen sentences so that prepositions and modifying relationships are unambiguous.
- Cover a topic with enough context to answer the query intent precisely.
- Prioritize satisfying intent and readability over optimizing keyword density.
Evidence
The technical foundation of BERT is laid out in Devlin et al. (2018), "BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding" (arXiv:1810.04805), which set state-of-the-art results on 11 natural language processing benchmarks, including GLUE, at the time of its release. The figures on its search application, namely one in ten English-language searches in the United States and featured snippet improvements across 24 countries, come from Google's official blog post "Understanding searches better than ever before" (2019).