In this new post from Ian Barber he takes a look at something that can come in very handy when you need something a bit more complex than the standard search results - term weighting.

The term weighting and ranking function is at the core of any information retrieval system. The vector space model with the cosine similarity is maybe the best known and most widely used, but there are plenty of alternatives. We're looking at two here, the BM25 function based around a probabilistic model, and a function based around language modeling.

He's put together a few examples on some basic weighting practices - simple string evaluation based on word occurrence, using the Okapi/BM25 method and language modeling with a little bit of probability and scoring involved.