On search, part 4 of 4

Function Score and Boosting

At Quote Catalog we recommended quotes from the tags of the quotes a reader had liked, then folded in what the whole site was liking. Relevance finds what’s similar. Boosts and function_score decide what’s worth showing, and every number in them is a policy someone has to own.

At Quote Catalog I built a recommendation engine on Elasticsearch, and the idea behind it was simple. Readers liked quotes. Every quote they liked carried tags, the topics, moods and people our editors had attached to it over the years. So we took the quotes a reader had liked, pulled their content, collected their tags, and recommended other quotes with the same tags. Then we added popularity, sitewide, so that among all the quotes that fit a reader’s taste, the ones everyone else was liking rose to the top. We didn’t use profile data. A reader’s likes were the whole signal.

The first half of that is relevance, and Elasticsearch is good at relevance. The second half is everything relevance doesn’t know. A quote that shares three tags with a reader’s likes might be one nobody else has ever liked. A quote that shares two might be one of the most liked on the site. Tags can’t tell those apart, so the relevance score can’t either.

Elasticsearch gives you two ways to fold that knowledge in. Boosting changes how much a clause counts toward relevance. function_score goes further and recomputes the score with functions of the document itself, like how many likes it has. We used both. The exact weights we shipped are long gone, and reconstructed ones wouldn’t be worth trusting, so the numbers below are illustrative. The mechanics are what’s worth knowing, along with the warning that comes with them.

The last part was about deciding who’s in, and about ordering results with a sort. This one is about the other way to order them, a score you shape yourself.

From Likes to a Query

The query starts with the reader. We looked up the quotes they’d liked, pulled each one’s content, and collected the tags. Then the tags became the query. Quotes that share more of them match more clauses and score higher, which is relevance doing exactly what it should:

{
  "bool": {
    "should": [
      { "terms": { "tags": ["moving-on", "letting-go", "heartbreak", "self-love"] } }
    ],
    "must_not": [
      { "ids": { "values": ["q_1042", "q_2207", "q_5530"] } }
    ]
  }
}

The must_not takes out the quotes the reader already liked, since recommending them back would be pointless. And the should has no filter or must beside it, so at least one tag has to match, which is the rule from the last part working in our favor for once.

Boosting

Not every tag a reader has liked says as much about them. A tag that shows up on ten of their likes is a taste. A tag that shows up once is an accident. Any clause can carry a boost, so the tags a reader keeps coming back to can count for more than the ones they touched once:

"should": [
  { "terms": { "tags": ["moving-on", "letting-go"], "boost": 3 } },
  { "terms": { "tags": ["heartbreak", "self-love"] } }
]

The same idea works on fields. When a search runs across several, “individual fields can be boosted with the caret (^) notation” (Elastic), so a match in a quote’s tags can outweigh a match on a word in its text, as in "fields": ["tags^3", "text"].

And there’s a third kind, the boosting query, which demotes instead of excluding. It “returns documents matching a positive query while reducing the relevance score of documents that also match a negative query,” with a negative_boost between 0 and 1.0 that sets how hard to push them down (Elastic). For a recommendation that’s often the honest answer. Another quote from an author a reader has already liked five times isn’t wrong to show, it’s just less useful than a voice they haven’t found yet, so it should sink, not vanish.

The Same Idea at LabRecruit

Boosting isn’t only for recommendations. In 2019 I built a pipeline for LabRecruit, a London recruiting agency focused on functional programming, that found developers through their public work and indexed them in Elasticsearch. Searching those candidates was the same problem in a different domain. A search started from a candidate hiring profile, built from survey data the client gave us about the role they were hiring for, and the parts of that profile weren’t equal. Programming languages and industry terms were boosted differently, because for a functional programming agency, a candidate who writes Haskell or Scala is a different match than one who mentions fintech once.

Here’s how that works with an example. This isn’t LabRecruit’s actual survey, but it’s the shape of one. Five questions about a role, covering the things we actually ranked on, proximity, languages, years of experience, salary and industry:

1. Where is the role, and how far could someone commute?   London, about 50 km       → decay on location
2. Which languages must they write?                        Scala                     → filter
   Which languages would be a plus?                        Haskell, OCaml            → boost
3. How many years of experience?                           around 6                  → decay on years
4. What’s the salary budget?                               £90k to £110k             → decay on salary
5. What industry is the role in?                           fintech, payments         → boost

Each answer becomes part of one query, and the survey’s wording tells you which part. A hard requirement is a filter. A nice-to-have is a boosted should. And anything with a target, a place, a number of years, a salary, becomes a decay function, which scores a document “with a function that decays depending on the distance” from an origin (Elastic). Distance there doesn’t have to mean kilometres. It works on any numeric, date or geo field, so a candidate with eight years of experience is “two years away” from a role that wants six, and one asking for £120k is £10k past the top of the budget.

Put together, the five answers become a single search score:

{
  "query": {
    "function_score": {
      "query": {
        "bool": {
          "filter": [ { "terms": { "languages": ["scala"] } } ],
          "should": [
            { "terms": { "languages": ["haskell", "ocaml"], "boost": 3 } },
            { "terms": { "industry_terms": ["fintech", "payments"], "boost": 2 } }
          ]
        }
      },
      "functions": [
        { "gauss": { "location": { "origin": "51.51,-0.13", "offset": "10km", "scale": "50km" } } },
        { "gauss": { "years_experience": { "origin": 6, "offset": 1, "scale": 3 } } },
        { "gauss": { "expected_salary": { "origin": 100000, "offset": 10000, "scale": 20000 } } }
      ],
      "score_mode": "multiply",
      "boost_mode": "multiply"
    }
  }
}

Read it the way the survey reads. Scala is required, so it’s a filter, and anyone without it is out. Haskell, OCaml and the industry are optional, and because the query has a filter, minimum_should_match defaults to zero, so they raise a candidate’s score without excluding anyone. The trap from the last part is a feature when the should clauses really are optional. Then the three decay functions each score how close a candidate is to what the role wants. Anyone within 10 km of the office gets full credit for location, and the credit falls off past that. Anyone from five to seven years of experience gets full credit, and more or fewer years cost a little, then a lot. Anyone asking for £90k to £110k gets full credit for salary, and the further outside the budget, the less.

Each decay function is one of three curves, and Elastic documents exactly what each computes. The Gaussian, the one in the query above, is

S(d) = \exp\left(-\frac{\max(0,\ |d - \text{origin}| - \text{offset})^2}{2\sigma^2}\right), \qquad \sigma^2 = -\frac{\text{scale}^2}{2\ln(\text{decay})}

which is a long way of saying full credit inside the offset, then a bell curve that passes through exactly decay, 0.5 by default, at offset + scale (Elastic). The other two shapes fall off differently past that point, and the difference is the whole choice. Move the sliders:

Decay: how much credit falls off with distance

offset scale decay

Gauss, exp and linear decay computed from Elastic’s documented formulas, for distance from the office in kilometres. Every curve hits the decay value at offset plus scale; what differs is how forgiving each is beyond it.

Gauss is gentle near the target and harsh far from it, which suits a commute. Exp is harsh early and never quite lets go, which suits recency, where an old quote should fade but not vanish. Linear goes to zero and stays there, which is how you’d express a hard edge without writing a filter.

Multiplying the three functions is the strict choice, and it’s deliberate. A candidate who is perfect on languages but lives a continent away, or wants twice the budget, shouldn’t be saved by their other strengths, because any one of those is a deal-breaker for a recruiter who has to place them. Summing would let a strong match on two axes paper over a disqualifying miss on the third.

The boosts and the decay scales are where the hiring profile’s priorities live. A role where Haskell is nearly required and the industry is incidental is a different query from one where both matter equally. A fully remote role has no location function at all. Change the survey’s answers and the query changes with them, without anyone writing a new search.

Adding Popularity

Boosts only reshape relevance. Popularity isn’t relevance at all, it’s a number on the document, and that’s what function_score is for. It “allows you to modify the score of documents that are retrieved by a query” (Elastic). You give it the query, which decides what’s eligible and how relevant each quote is, and a list of functions, each of which computes its own score from the quote. Then it combines them.

For sitewide popularity at Quote Catalog, the function is field_value_factor, which lets you “use a field from a document to influence the score.” Our field was a quote’s total likes across the whole site:

{
  "query": {
    "function_score": {
      "query": {
        "bool": {
          "should": [
            { "terms": { "tags": ["moving-on", "letting-go"], "boost": 3 } },
            { "terms": { "tags": ["heartbreak", "self-love"] } }
          ],
          "must_not": [ { "ids": { "values": ["q_1042", "q_2207", "q_5530"] } } ]
        }
      },
      "functions": [
        { "field_value_factor": { "field": "likes", "modifier": "log1p", "missing": 0 } }
      ],
      "boost_mode": "multiply"
    }
  }
}

The modifier is the most important word in that query. Raw counts are dangerous, because one quote with a hundred thousand likes would beat everything that matched a reader’s taste, forever, and every reader would get the same recommendation. log1p takes the logarithm of the count plus one, so going from 10 likes to 100 matters about as much as going from 10,000 to 100,000. Popular quotes still rise, but they can’t drown out the tags. Elastic is explicit about why it’s log1p and not the plain logarithm: “The log and ln modifiers will produce negative values if used on values between 0 and 1,” and “it is recommended to use log1p instead.” The missing value covers a brand-new quote with no likes yet.

In numbers, with log1p meaning the common logarithm of one plus the count, as Elastic defines it, the multiplier for a quote with n likes is

\text{score} = \text{relevance} \times \log_{10}(1 + n)

so 40 likes is worth about 1.6, and 50,000 likes about 4.7. Here’s what that does to a strong match against a viral one:

Popularity: does one viral quote drown out a better match?

none sqrt log1p

Two illustrative quotes. The strong match shares three of a reader’s tags (relevance 3.0) and has 40 likes; the weak match shares one (relevance 1.0) and has 50,000. Switch the modifier and watch which one ranks first.

How the Scores Combine

function_score has two settings for combining scores, and they’re easy to confuse. score_mode combines the functions with each other, and boost_mode combines that result with the query’s relevance score. Both default to multiply.

With one popularity function, boost_mode: multiply is the right choice. Relevance from the tags stays in charge of what counts as a good match, and popularity scales it, so a well-matched quote that a lot of people like beats a well-matched quote that nobody does. boost_mode: replace would throw the tags away entirely and rank every eligible quote by likes, which turns a recommendation into a leaderboard.

Once there are several functions, score_mode starts to matter. Multiplying them is strict, because if any one scores near zero, the whole quote does. Summing lets a quote make up for weakness on one signal with strength on another. And there are more functions to choose from than popularity. Decay functions score a document “with a function that decays depending on the distance” from an origin, which can be a date, so newer quotes can count for more. A function with a filter and a weight “acts as a function that simply returns the weight,” which is how an editor’s pick could get a fixed bump. And random_score adds a little variety, though its default randomness is “not reproducible since documents might be renumbered by merges,” so it needs a seed and a field if a reader should see the same list twice.

Two more settings are worth knowing. max_boost caps how much the functions can add, and its default is effectively unlimited. min_score drops anything under a threshold, but every matching document has to be scored first, so it’s a filter applied after the expensive part, not instead of it.

Every Number Is a Policy

Here’s the warning. Who decided that a repeated tag should count three times as much as a single one, or that a language should count twice as much as an industry? Every number in those queries is a decision. The boost on tags is a decision about how much a reader’s habits should outweigh their curiosity. The log on likes is a decision about how much popularity should be allowed to compound, and whether the site should converge on the same few quotes. The boost on languages is a decision about what a client really needs. None of those come from the data. Someone picks them, and someone else will eventually ask why a result showed up where it did.

That’s the same argument I made for tiers in the last part, from the other side. A strict sort can be explained in a sentence. A tuned score can’t, but it can be inspected. Elasticsearch’s explain API will “get information about why a specific document matches, or doesn’t match, a query” (Elastic), clause by clause and function by function, which is how you answer the question when it comes.

Takeaways

If I were building recommendations again, I’d start the same way, with the tags of what a reader already liked, because a reader’s own likes are the most honest signal there is. I’d exclude what they’ve already liked and boost the tags they keep returning to. I’d add popularity through function_score with log1p, because raw counts turn every recommendation into the same handful of hits. I’d multiply by relevance so the tags stay in charge. I’d add any other function one at a time, checking what each one changes before adding the next. And I’d write down why every number is what it is, because the explain API will tell someone what the score was, not why we chose it.

Relevance knows what’s similar. Everything else is a policy, and policies need owners.