#tokenization

Articles tagged with #tokenization

Vector Embedding
Think of every Bollywood actor.When a casting director chooses them for a role, they don’t just remember their name — they remember their qualities. For example: Shah Rukh Khan → (romance: 10, drama: 9, comedy: 6) Akshay Kumar → (action: 9, comedy:...
Aug 12, 20251 min read1
Tokenization - For Freshers
Imagine you have a big storybook .If you try to read the whole thing in one breath, you’ll pass out faster than India chasing 350 without Kohli.So you split it into words and sentences. Now you can read it bit by bit.That’s exactly what tokenization ...
Aug 12, 20251 min read4