Intuition vs. data: why your beliefs often fail you

If you’ve used Textio for a little while, chances are you’ve come across some recommendations that surprise you a little bit. For instance, you might wonder why Textio recommends the word dedicated in describing someone’s work style but rejects loyal, or why it encourages the use of build but not craft.

Sometimes you can concoct a theory about the guidance you’re getting: “Of course no one likes the word stakeholders!” But other times, the recommendations you see surprise you, or may even feel counterintuitive. There’s no better evidence than this that the system is working as designed! Textio’s predictive engine has uncovered millions of language patterns that wouldn’t be possible to find without augmented writing software.

The algorithms that power Textio’s predictive engine are not based on academic research or opinions. They are based on hiring data from millions of actual humans who have applied to actual job posts and responded to real recruiting emails — in fact, 95% of our customers contribute their own hiring data. There is nothing hand-wavy or hypothetical in Textio’s data models. As you write in Textio, you are tapping into a global community of writers and using their hiring outcomes to improve yours.

So, when a phrase gets flagged as an “orange word,” it’s because fewer candidates have applied to job posts or responded to emails that contain it, as compared statistically to the baseline expectation. Likewise, for phrases flagged as “green words” it just means that more candidates have applied to job posts or responded to emails that contain that phrase.

I spent years working on the Bing search engine, and there are some similarities behind the scenes. Here is the basic way a search engine works: the more popular the search result, the more it gets clicked on, and the higher it will move in the rankings. A similar thing happens with Textio — the language patterns that are “ranked” highest are the ones that job seekers “click on” when they see the job ad!

Sometimes you’ll find cases where you’re thinking, “Duh, this word should obviously turn people away,” and yet it’s not highlighted in orange. For example, I would never use synergy in a job post — to my ear, it sounds tedious and corporate. But it turns out that not everyone agrees with me; job seekers in San Francisco “click on” synergy all the time, and that word actually helps companies fill roles faster in the SF market! Most of the time these cases simply expose the hidden personal biases you didn’t even know you were harboring.

Sometimes, a word isn’t flagged because there have not been enough instances of the phrase to make a difference statistically. But more often, it’s because your intuitions about language that is “masculine,” “feminine,” “positive,” or “negative” are actually misplaced. When we measure over a vast data set, we find the truth, rather than just our stereotyped beliefs about what might be true.