22988 — Rar

It can still understand "raar" by breaking it down into parts it recognizes.

Text classification with BERT: tokenizers.ipynb - Colab - Google 22988 rar

This system is why AI has become so much better at understanding us. By using subwords like , the model can: It can still understand "raar" by breaking it

If a model encounters a word it doesn't know, it breaks it into smaller chunks it does recognize. For example: The word "rarity" might be split into rar + ##ity . The word "unrar" might become un + ##rar . For example: The word "rarity" might be split

Next time you use a search engine or talk to an AI, remember that under the hood, your words are being dissolved into a sea of numbers. Somewhere in that digital soup, is working hard to make sense of the world, one "rar" at a time.