I used to love the internet. It was fun to explore. I think it was because Google was a better company. They would return exactly what I wanted. Now it's all ads and SEO crap. Often I go to social media like twitter or reddit to get good content. This article explains why:
https://dkb.io/post/google-search-is-dying
Google Search Is Dying
2022-02-15
None
<div><div><p><em>There is good discussion on this article on <a href="https://news.ycombinator.com/item?id=30347719">Hacker News</a> and <a href="https://www.reddit.com/r/technology/comments/st9ri1/google_search_is_dying/">Reddit</a></em></p>
<p>Reddit is currently the most popular search engine. The only people who don’t know that are the team at Reddit, who can’t be bothered to build a decent search interface. So instead we resort to using Google, and appending the word “reddit” to the end of our queries.</p>
<p>Paul Graham thinks this image means Reddit as a social media site “still hasn’t peaked”. What it actually means is that the amount of people using Reddit as a search engine is growing.</p>
<p></p>
<p>Why are people searching Reddit specifically? The short answer is that Google search results are clearly dying. The long answer is that most of the web has become too inauthentic to trust.</p>
<h2>How do we know Google is dying?</h2>
<p>If you’ve tried to search for a recipe or product review recently, I don’t need to tell you that Google search results have gone to shit. You would have already noticed that the first few non-ad results are SEO optimized sites filled with affiliate links and ads.</p>
<p>Google still gives decent results for many other categories, especially when it comes to factual information. You might think that Google results are pretty good for you, and you have no idea what I’m talking about.</p>
<p>What you don’t realize is that you’ve been self-censoring yourself from searching most of the things you would have wanted to search. You already know subconsciously that Google isn’t going to return a good result.</p>
<p>I’m far from the only one who thinks Google is dying:</p>
<p><a href="https://dcgross.com/a-new-google">Daniel Gross</a></p>
<blockquote>
<p>In 2000, Google got popular because hackers realized it was better than Lycos or Excite. This effect is happening again. Early adopters aren’t using Google anymore.</p>
</blockquote>
<p><a href="https://news.ycombinator.com/item?id=29392702">Ask HN: Has Google search become quantitatively worse?</a></p>
<blockquote>
<p>The results keep getting "refined" so as to suit the popular 80% of queries, while getting much worse for any technical or obscure queries. Forced synonyms and "people also searched for" are typically useless and almost infuriating. Once you get off the first or second page, the results get even worse-- with pages entirely unrelated to the query (e.g. not even containing the searched phrases). They are probably testing/already implemented some sort of multi armed bandit type optimization like on Youtube's search results where they just show any popular pages (ignoring relevancy) to see if they yield a click.</p>
</blockquote>
<p><a href="https://twitter.com/mwseibel/status/1477701120319361026">Michael Seibel</a>:</p>
<blockquote>
<p>A recent small medical issue has highlighted how much someone needs to disrupt Google Search. Google is no longer producing high quality search results in a significant number of important categories.</p>
</blockquote>
<blockquote>
<p>Health, product reviews, recipes are three categories I searched today where top results featured clickbait sites riddled with crappy ads. I’m sure there are many more.</p>
</blockquote>
<p><a href="https://twitter.com/paulg/status/1477760548787920901">Paul Graham</a></p>
<blockquote>
<p>This may not just be a problem with Google but possibly also the recipe for beating Google. A startup usually has to start with a niche market. Why not try writing a search engine specifically for some category dominated by SEO spam?</p>
</blockquote>
<h2>Why is Google dying?</h2>
<h3>Ads</h3>
<p>It is obvious that serving ads creates misaligned incentives for search engines. The founders of Google pointed this out themselves when they were just getting started.</p>
<p><a href="http://infolab.stanford.edu/~backrub/google.html">Sergey Brin and Lawrence Page (1998)</a></p>
<blockquote>
<p>“Currently, the predominant business model for commercial search engines is advertising. The goals of the advertising business model do not always correspond to providing quality search to users…we expect that advertising funded search engines will be inherently biased towards the advertisers and away from the needs of the consumers…Furthermore, advertising income often provides an incentive to provide poor quality search results.”</p>
</blockquote>
<p>Unfortunately, these thoughts on the failings of ad-based search engines read like an instruction manual for what Google did next.</p>
<p>They’ve dialed it up to the max recently to squeeze out every last cent before their inevitable collapse.</p>
<p></p>
<h3>SEO</h3>
<p>There are tons of people whose sole job is to game their way to the top of Google, so it shouldn’t be surprising when search results deteriorate in quality. To be fair, this would probably be an issue with any search engine, but you’d expect Google to be able to come up with a less gameable algorithm.</p>
<p>Here’s a <a href="https://alexskra.com/blog/the-mermaid-is-taking-over-google-search-in-norway/">fun story</a> about how one website gamed their way to the top of Google in Norway.</p>
<h3>AI</h3>
<p>Google increasingly does not give you the results for what you typed in. It tries to be “smart” and figure out what you “really meant”, in addition to personalizing things for you. If you really meant exactly what you typed, then all bets are off.</p>
<p>Even the exact match query operator (“ ”) <a href="https://news.ycombinator.com/item?id=27052634">doesn’t give exact matches anymore</a>, which is quite bizarre.</p>
<h2>Why are people appending “reddit” to their queries?</h2>
<p>There’s a fun conspiracy theory that popped up recently called the <a href="https://www.theatlantic.com/technology/archive/2021/08/dead-internet-theory-wrong-but-feels-true/619937/">Dead Internet Theory</a>. The claim is basically that most of the internet is bots. There aren’t real people here anymore.</p>
<p><a href="https://forum.agoraroad.com/index.php?threads/dead-internet-theory-most-of-the-internet-is-fake.3011/">IlluminatiPirate</a>:</p>
<blockquote>
<p>TLDR: Large proportions of the supposedly human-produced content on the internet are actually generated by artificial intelligence networks in conjunction with paid secret media influencers in order to manufacture consumers for an increasing range of newly-normalised cultural products.</p>
</blockquote>
<p>This isn’t true (yet), but it reflects some general sense that the authentic web is gone. The SEO marketers gaming their way to the top of every Google search result might as well be robots. Everything is commercialized. Someone’s always trying to sell you something. Whether they’re a bot or human, they are decidedly fake.</p>
<p>So how can we regain authenticity? What if you want to know what a genuine real life human being thinks about the latest Lenovo laptop?</p>
<p>You append “reddit" to your query (or hacker news, or stack overflow, or some other community you trust).</p>
<p>Google is dead.</p>
<p>Long live Google + “site:reddit.com”.</p>
<h2>Appendix 1: response from Google.</h2>
<h3>Google claims the exact matching feature is merely unintuitive, not broken</h3>
<p><a href="https://twitter.com/dannysullivan/status/1493702054673485826">Danny Sullivan (Google's public Search Liaison)</a>:</p>
<blockquote>
<p>I work for Google Search, passed your feedback along, thanks. You said in the post that quotes don't give exact matches. They really do. Honest. Put a word or phrase in quotes, that's what we'll match. If anyone has an example where they feel it doesn't, please let me know...</p>
</blockquote>
<blockquote>
<p>Here's why people often think quoting with Google doesn't work when it really does (I've looked a huge number of these reports).</p>
<ol>
<li>We match ALT text</li>
<li>We match text not readily visible, such as in a menu or small text</li>
<li>Page has changed since we indexed it</li>
<li>Punctuation...</li>
</ol>
</blockquote>
<blockquote>
<p>Punctuation comes into play if you did a quoted search like "dog cat" and there's text that says "dog, cat" then we'll see that without the punctuation. That doesn't seem a major issue but we're looking at if we could improve there.</p>
</blockquote>
<h3>And Google seems to be right so far</h3>
<p>Every example of exact matching not working so far can be traced back to issues with punctuation, alt text, etc.</p>
<p>For example, one proposed <a href="https://news.ycombinator.com/item?id=30355528">failing query</a> was "quotes don't give". But HN user <a href="https://news.ycombinator.com/item?id=30356152">saalweachter</a> has pointed out that it is a very intricate punctuation problem.</p>
<blockquote>
<p>For instances, on the ["quotes don't give"] example, the first result I get is
https://www.goodreads.com/quotes/tag/never-give-up</p>
</blockquote>
<blockquote>
<p>If I do a find-in-page for "quotes don't give", I get zero results. Oh no! Perfidy!</p>
</blockquote>
<blockquote>
<p>... but, if you look more closely, you'll find this string waaaaay down at the bottom:</p>
</blockquote>
<blockquote>
<p>tags: don-t-give-up, don-t-give-up-on-your-dreams, don-t-give-up-on-yourself, don-t-give-up-quotes, don-t-give-up-the-fight, encouragement, ...</p>
</blockquote>
<blockquote>
<p>Thanks to the wonders of tokenization, that "don-t-give-up-quotes, don-t-give-up-the-fight" gives you the string of tokens, "don t give up quotes don t give up the fight", which contains the exact phrase "quotes don t give", which is the tokenization of the phrase "quotes don't give".</p>
</blockquote>
<p><a href="https://news.ycombinator.com/item?id=30356382">Danny Sullivan (Google's public Search Liaison)</a>:</p>
<blockquote>
<p>And...as noted in another reply below, in the first result, if you look at the cached page that shows the ENTIRE page that we indexed (rather than the paginated version you land on), there's this text:</p>
</blockquote>
<blockquote>
<p>quotes, don-t-give-up-the-fight</p>
</blockquote>
<blockquote>
<p>with when you remove the punctation, is the match:</p>
</blockquote>
<blockquote>
<p>quotes dont give</p>
</blockquote>
<blockquote>
<p>And I get that this can be frustrating, that we don't consider punctation in a quoted search. That's not a new change, however. It's been that way for ages. But as I also said, it's something we might revisit.</p>
</blockquote>
<h2>Appendix 2: Google just had record profits, you are obviously wrong, and Google isn't dying.</h2>
<p>I never said that Google wasn't making money. In fact, if Google really is squeezing every last cent out of ads without regard to search quality, I would expect their revenue to be at an all time high.</p>
<p>I am saying that the quality of search results are declining. This may eventually lead to a decrease in revenue, but has not yet.</p>
<h2>Appendix 3: Seriously, what are you talking about? My search results are perfect.</h2>
<p>If you think your search results are perfect (without appending reddit), then you're probably right. If every single person agreed that Google search results were trash, then Google would already be bankrupt.</p>
<p>Perhaps it is more likely that 80% of people think Google is good enough, and 20% think Google sucks.</p>
<p>I do suspect that the 20% will be growing in number though.</p>
<h2>Appendix 4: *Yawn*, this is the 87th time someone has claimed that Google search is dying in the last 20 years. This is a big meme in the SEO world.</h2>
<p><em>"The reports of my death are greatly exaggerated" - Google, probably</em></p>
<p>You're right, there's been a new article bashing Google every few months for the last 20 years straight. It's probably nothing.</p>
<p>Still, it is a bit interesting that this short and simple post is now one of the most upvoted things of all time on Hacker News. There must be a lot of people who resonate with it this time around.</p>
<p>Hard to tell if something significant has changed.</p>
<h2>Appendix 5: Random redditor explains it succinctly</h2>
<p><a href="https://www.reddit.com/r/technology/comments/st9ri1/comment/hx3zubc/">u/a_latvian_potato</a>:</p>
<blockquote>
<p>I think I understand what this article is trying to say. It's not saying that Google's search technology is worse or that people don't use Google to search. It's saying that people trust less of the results Google shows compared to seeing discussions of it on Reddit.</p>
</blockquote>
<blockquote>
<p>For instance, if I'm looking to see reviews of the Honda Civic 2022 or whatever, I actually do find myself typing "Honda Civic review reddit" instead of "Honda Civic review". This is because I want to see what real people and enthusiasts (on r/cars or whatever) are talking about the car, rather than the top results at Google which are basically just paid reviews advertising the car anyway.</p>
</blockquote>
<blockquote>
<p>Even though I kinda know people in Reddit are just as capable of spouting BS that are completely wrong, I find the discussions more authentic anyway than the corporate speak the "big websites" have on their articles that Google shows me.</p>
</blockquote>
<h2>Appendix 6: A fun comic by <a href="https://twitter.com/zenacomics/">@zenacomics</a></h2>
<p></p>
<h2>Appendix 7: More thoughts on organizing information on the internet</h2>
<p>If you want some more in-depth thoughts on this subject, you should check out <a href="/post/organize-the-world-information">this post I wrote a year ago</a> about all the different ways we could organize information usefully on the internet.</p>
</div>||||I|||| * Home
 * About
 * RSS



 Google Search Is Dying

 February 15, 2022 Last updated: February 18, 2022
 # search

 There is good discussion on this article on Hacker News and Reddit

 Reddit is currently the most popular search engine. The only people who don’t know that are the team at Reddit, who can’t be bothered to build a decent search interface. So instead we resort to using Google, and appending the word “reddit” to the end of our queries.

 Paul Graham thinks this image means Reddit as a social media site “still hasn’t peaked”. What it actually means is that the amount of people using Reddit as a search engine is growing.

 Why are people searching Reddit specifically? The short answer is that Google search results are clearly dying. The long answer is that most of the web has become too inauthentic to trust.

 How do we know Google is dying?

 If you’ve tried to search for a recipe or product review recently, I don’t need to tell you that Google search results have gone to shit. You would have already noticed that the first few non-ad results are SEO optimized sites filled with affiliate links and ads.

 Google still gives decent results for many other categories, especially when it comes to factual information. You might think that Google results are pretty good for you, and you have no idea what I’m talking about.

 What you don’t realize is that you’ve been self-censoring yourself from searching most of the things you would have wanted to search. You already know subconsciously that Google isn’t going to return a good result.

 I’m far from the only one who thinks Google is dying:

 Daniel Gross

 In 2000, Google got popular because hackers realized it was better than Lycos or Excite. This effect is happening again. Early adopters aren’t using Google anymore.

 Ask HN: Has Google search become quantitatively worse?

 The results keep getting "refined" so as to suit the popular 80% of queries, while getting much worse for any technical or obscure queries. Forced synonyms and "people also searched for" are typically useless and almost infuriating. Once you get off the first or second page, the results get even worse-- with pages entirely unrelated to the query (e.g. not even containing the searched phrases). They are probably testing/already implemented some sort of multi armed bandit type optimization like on Youtube's search results where they just show any popular pages (ignoring relevancy) to see if they yield a click.

 Michael Seibel:

 A recent small medical issue has highlighted how much someone needs to disrupt Google Search. Google is no longer producing high quality search results in a significant number of important categories.

 Health, product reviews, recipes are three categories I searched today where top results featured clickbait sites riddled with crappy ads. I’m sure there are many more.

 Paul Graham

 This may not just be a problem with Google but possibly also the recipe for beating Google. A startup usually has to start with a niche market. Why not try writing a search engine specifically for some category dominated by SEO spam?

 Why is Google dying?

 Ads

 It is obvious that serving ads creates misaligned incentives for search engines. The founders of Google pointed this out themselves when they were just getting started.

 Sergey Brin and Lawrence Page (1998)

 “Currently, the predominant business model for commercial search engines is advertising. The goals of the advertising business model do not always correspond to providing quality search to users…we expect that advertising funded search engines will be inherently biased towards the advertisers and away from the needs of the consumers…Furthermore, advertising income often provides an incentive to provide poor quality search results.”

 Unfortunately, these thoughts on the failings of ad-based search engines read like an instruction manual for what Google did next.

 They’ve dialed it up to the max recently to squeeze out every last cent before their inevitable collapse.

 SEO

 There are tons of people whose sole job is to game their way to the top of Google, so it shouldn’t be surprising when search results deteriorate in quality. To be fair, this would probably be an issue with any search engine, but you’d expect Google to be able to come up with a less gameable algorithm.

 Here’s a fun story about how one website gamed their way to the top of Google in Norway.

 AI

 Google increasingly does not give you the results for what you typed in. It tries to be “smart” and figure out what you “really meant”, in addition to personalizing things for you. If you really meant exactly what you typed, then all bets are off.

 Even the exact match query operator (“ ”) doesn’t give exact matches anymore, which is quite bizarre.

 Why are people appending “reddit” to their queries?

 There’s a fun conspiracy theory that popped up recently called the Dead Internet Theory. The claim is basically that most of the internet is bots. There aren’t real people here anymore.

 IlluminatiPirate:

 TLDR: Large proportions of the supposedly human-produced content on the internet are actually generated by artificial intelligence networks in conjunction with paid secret media influencers in order to manufacture consumers for an increasing range of newly-normalised cultural products.

 This isn’t true (yet), but it reflects some general sense that the authentic web is gone. The SEO marketers gaming their way to the top of every Google search result might as well be robots. Everything is commercialized. Someone’s always trying to sell you something. Whether they’re a bot or human, they are decidedly fake.

 So how can we regain authenticity? What if you want to know what a genuine real life human being thinks about the latest Lenovo laptop?

 You append “reddit" to your query (or hacker news, or stack overflow, or some other community you trust).

 Google is dead.

 Long live Google + “site:reddit.com”.

 Appendix 1: response from Google.

 Google claims the exact matching feature is merely unintuitive, not broken

 Danny Sullivan (Google's public Search Liaison):

 I work for Google Search, passed your feedback along, thanks. You said in the post that quotes don't give exact matches. They really do. Honest. Put a word or phrase in quotes, that's what we'll match. If anyone has an example where they feel it doesn't, please let me know...

 Here's why people often think quoting with Google doesn't work when it really does (I've looked a huge number of these reports).

 1. We match ALT text
 2. We match text not readily visible, such as in a menu or small text
 3. Page has changed since we indexed it
 4. Punctuation...

 Punctuation comes into play if you did a quoted search like "dog cat" and there's text that says "dog, cat" then we'll see that without the punctuation. That doesn't seem a major issue but we're looking at if we could improve there.

 And Google seems to be right so far

 Every example of exact matching not working so far can be traced back to issues with punctuation, alt text, etc.

 For example, one proposed failing query was "quotes don't give". But HN user saalweachter has pointed out that it is a very intricate punctuation problem.

 For instances, on the ["quotes don't give"] example, the first result I get is https://www.goodreads.com/quotes/tag/never-give-up

 If I do a find-in-page for "quotes don't give", I get zero results. Oh no! Perfidy!

 ... but, if you look more closely, you'll find this string waaaaay down at the bottom:

 tags: don-t-give-up, don-t-give-up-on-your-dreams, don-t-give-up-on-yourself, don-t-give-up-quotes, don-t-give-up-the-fight, encouragement, ...

 Thanks to the wonders of tokenization, that "don-t-give-up-quotes, don-t-give-up-the-fight" gives you the string of tokens, "don t give up quotes don t give up the fight", which contains the exact phrase "quotes don t give", which is the tokenization of the phrase "quotes don't give".

 Danny Sullivan (Google's public Search Liaison):

 And...as noted in another reply below, in the first result, if you look at the cached page that shows the ENTIRE page that we indexed (rather than the paginated version you land on), there's this text:

 quotes, don-t-give-up-the-fight

 with when you remove the punctation, is the match:

 quotes dont give

 And I get that this can be frustrating, that we don't consider punctation in a quoted search. That's not a new change, however. It's been that way for ages. But as I also said, it's something we might revisit.

 Appendix 2: Google just had record profits, you are obviously wrong, and Google isn't dying.

 I never said that Google wasn't making money. In fact, if Google really is squeezing every last cent out of ads without regard to search quality, I would expect their revenue to be at an all time high.

 I am saying that the quality of search results are declining. This may eventually lead to a decrease in revenue, but has not yet.

 Appendix 3: Seriously, what are you talking about? My search results are perfect.

 If you think your search results are perfect (without appending reddit), then you're probably right. If every single person agreed that Google search results were trash, then Google would already be bankrupt.

 Perhaps it is more likely that 80% of people think Google is good enough, and 20% think Google sucks.

 I do suspect that the 20% will be growing in number though.

 Appendix 4: *Yawn*, this is the 87th time someone has claimed that Google search is dying in the last 20 years. This is a big meme in the SEO world.

 "The reports of my death are greatly exaggerated" - Google, probably

 You're right, there's been a new article bashing Google every few months for the last 20 years straight. It's probably nothing.

 Still, it is a bit interesting that this short and simple post is now one of the most upvoted things of all time on Hacker News. There must be a lot of people who resonate with it this time around.

 Hard to tell if something significant has changed.

 Appendix 5: Random redditor explains it succinctly

 u/a_latvian_potato:

 I think I understand what this article is trying to say. It's not saying that Google's search technology is worse or that people don't use Google to search. It's saying that people trust less of the results Google shows compared to seeing discussions of it on Reddit.

 For instance, if I'm looking to see reviews of the Honda Civic 2022 or whatever, I actually do find myself typing "Honda Civic review reddit" instead of "Honda Civic review". This is because I want to see what real people and enthusiasts (on r/cars or whatever) are talking about the car, rather than the top results at Google which are basically just paid reviews advertising the car anyway.

 Even though I kinda know people in Reddit are just as capable of spouting BS that are completely wrong, I find the discussions more authentic anyway than the corporate speak the "big websites" have on their articles that Google shows me.

 Appendix 6: A fun comic by @zenacomics

 Appendix 7: More thoughts on organizing information on the internet

 If you want some more in-depth thoughts on this subject, you should check out this post I wrote a year ago about all the different ways we could organize information usefully on the internet.



There is good discussion on this article on Hacker News and Reddit  
Reddit is currently the most popular search engine. The only people who don’t know that are the team at Reddit, who can’t be bothered to build a decent search interface. So instead we resort to using Google, and appending the word “reddit” to the end of our queries.
Paul Graham thinks this image means Reddit as a social media site “still hasn’t peaked”. What it actually means is that the amount of people using Reddit as a search engine is growing.
 
Why are people searching Reddit specifically? The short answer is that Google search results are clearly dying. The long answer is that most of the web has become too inauthentic to trust.
How do we know Google is dying?
If you’ve tried to search for a recipe or product review recently, I don’t need to tell you that Google search results have gone to shit. You would have already noticed that the first few non-ad results are SEO optimized sites filled with affiliate links and ads.
Google still gives decent results for many other categories, especially when it comes to factual information. You might think that Google results are pretty good for you, and you have no idea what I’m talking about.
What you don’t realize is that you’ve been self-censoring yourself from searching most of the things you would have wanted to search. You already know subconsciously that Google isn’t going to return a good result.
I’m far from the only one who thinks Google is dying:
 Daniel Gross 
In 2000, Google got popular because hackers realized it was better than Lycos or Excite. This effect is happening again. Early adopters aren’t using Google anymore.
 Ask HN: Has Google search become quantitatively worse? 
The results keep getting "refined" so as to suit the popular 80% of queries, while getting much worse for any technical or obscure queries. Forced synonyms and "people also searched for" are typically useless and almost infuriating. Once you get off the first or second page, the results get even worse-- with pages entirely unrelated to the query (e.g. not even containing the searched phrases). They are probably testing/already implemented some sort of multi armed bandit type optimization like on Youtube's search results where they just show any popular pages (ignoring relevancy) to see if they yield a click.
 Michael Seibel:
A recent small medical issue has highlighted how much someone needs to disrupt Google Search. Google is no longer producing high quality search results in a significant number of important categories.
Health, product reviews, recipes are three categories I searched today where top results featured clickbait sites riddled with crappy ads. I’m sure there are many more.
 Paul Graham 
This may not just be a problem with Google but possibly also the recipe for beating Google. A startup usually has to start with a niche market. Why not try writing a search engine specifically for some category dominated by SEO spam?
Why is Google dying?
Ads
It is obvious that serving ads creates misaligned incentives for search engines. The founders of Google pointed this out themselves when they were just getting started.
 Sergey Brin and Lawrence Page (1998) 
“Currently, the predominant business model for commercial search engines is advertising. The goals of the advertising business model do not always correspond to providing quality search to users…we expect that advertising funded search engines will be inherently biased towards the advertisers and away from the needs of the consumers…Furthermore, advertising income often provides an incentive to provide poor quality search results.”
Unfortunately, these thoughts on the failings of ad-based search engines read like an instruction manual for what Google did next.
They’ve dialed it up to the max recently to squeeze out every last cent before their inevitable collapse.
 
SEO
There are tons of people whose sole job is to game their way to the top of Google, so it shouldn’t be surprising when search results deteriorate in quality. To be fair, this would probably be an issue with any search engine, but you’d expect Google to be able to come up with a less gameable algorithm.
Here’s a fun story about how one website gamed their way to the top of Google in Norway.
AI
Google increasingly does not give you the results for what you typed in. It tries to be “smart” and figure out what you “really meant”, in addition to personalizing things for you. If you really meant exactly what you typed, then all bets are off.
Even the exact match query operator (“ ”) doesn’t give exact matches anymore, which is quite bizarre.
Why are people appending “reddit” to their queries?
There’s a fun conspiracy theory that popped up recently called the Dead Internet Theory. The claim is basically that most of the internet is bots. There aren’t real people here anymore.
 IlluminatiPirate:
TLDR: Large proportions of the supposedly human-produced content on the internet are actually generated by artificial intelligence networks in conjunction with paid secret media influencers in order to manufacture consumers for an increasing range of newly-normalised cultural products.
This isn’t true (yet), but it reflects some general sense that the authentic web is gone. The SEO marketers gaming their way to the top of every Google search result might as well be robots. Everything is commercialized. Someone’s always trying to sell you something. Whether they’re a bot or human, they are decidedly fake.
So how can we regain authenticity? What if you want to know what a genuine real life human being thinks about the latest Lenovo laptop?
You append “reddit" to your query (or hacker news, or stack overflow, or some other community you trust).
Google is dead.
Long live Google + “site:reddit.com”.
Appendix 1: response from Google.
Google claims the exact matching feature is merely unintuitive, not broken
 Danny Sullivan (Google's public Search Liaison):
I work for Google Search, passed your feedback along, thanks. You said in the post that quotes don't give exact matches. They really do. Honest. Put a word or phrase in quotes, that's what we'll match. If anyone has an example where they feel it doesn't, please let me know...
Here's why people often think quoting with Google doesn't work when it really does (I've looked a huge number of these reports).
We match ALT text
We match text not readily visible, such as in a menu or small text
Page has changed since we indexed it
Punctuation...
Punctuation comes into play if you did a quoted search like "dog cat" and there's text that says "dog, cat" then we'll see that without the punctuation. That doesn't seem a major issue but we're looking at if we could improve there.
And Google seems to be right so far
Every example of exact matching not working so far can be traced back to issues with punctuation, alt text, etc.
For example, one proposed failing query was "quotes don't give". But HN user saalweachter has pointed out that it is a very intricate punctuation problem.
For instances, on the ["quotes don't give"] example, the first result I get is
https://www.goodreads.com/quotes/tag/never-give-up
If I do a find-in-page for "quotes don't give", I get zero results. Oh no! Perfidy!
... but, if you look more closely, you'll find this string waaaaay down at the bottom:
tags: don-t-give-up, don-t-give-up-on-your-dreams, don-t-give-up-on-yourself, don-t-give-up-quotes, don-t-give-up-the-fight, encouragement, ...
Thanks to the wonders of tokenization, that "don-t-give-up-quotes, don-t-give-up-the-fight" gives you the string of tokens, "don t give up quotes don t give up the fight", which contains the exact phrase "quotes don t give", which is the tokenization of the phrase "quotes don't give".
 Danny Sullivan (Google's public Search Liaison):
And...as noted in another reply below, in the first result, if you look at the cached page that shows the ENTIRE page that we indexed (rather than the paginated version you land on), there's this text:
quotes, don-t-give-up-the-fight
with when you remove the punctation, is the match:
quotes dont give
And I get that this can be frustrating, that we don't consider punctation in a quoted search. That's not a new change, however. It's been that way for ages. But as I also said, it's something we might revisit.
Appendix 2: Google just had record profits, you are obviously wrong, and Google isn't dying.
I never said that Google wasn't making money. In fact, if Google really is squeezing every last cent out of ads without regard to search quality, I would expect their revenue to be at an all time high.
I am saying that the quality of search results are declining. This may eventually lead to a decrease in revenue, but has not yet.
Appendix 3: Seriously, what are you talking about? My search results are perfect.
If you think your search results are perfect (without appending reddit), then you're probably right. If every single person agreed that Google search results were trash, then Google would already be bankrupt.
Perhaps it is more likely that 80% of people think Google is good enough, and 20% think Google sucks.
I do suspect that the 20% will be growing in number though.
Appendix 4: *Yawn*, this is the 87th time someone has claimed that Google search is dying in the last 20 years. This is a big meme in the SEO world.
 "The reports of my death are greatly exaggerated" - Google, probably 
You're right, there's been a new article bashing Google every few months for the last 20 years straight. It's probably nothing.
Still, it is a bit interesting that this short and simple post is now one of the most upvoted things of all time on Hacker News. There must be a lot of people who resonate with it this time around.
Hard to tell if something significant has changed.
Appendix 5: Random redditor explains it succinctly
 u/a_latvian_potato:
I think I understand what this article is trying to say. It's not saying that Google's search technology is worse or that people don't use Google to search. It's saying that people trust less of the results Google shows compared to seeing discussions of it on Reddit.
For instance, if I'm looking to see reviews of the Honda Civic 2022 or whatever, I actually do find myself typing "Honda Civic review reddit" instead of "Honda Civic review". This is because I want to see what real people and enthusiasts (on r/cars or whatever) are talking about the car, rather than the top results at Google which are basically just paid reviews advertising the car anyway.
Even though I kinda know people in Reddit are just as capable of spouting BS that are completely wrong, I find the discussions more authentic anyway than the corporate speak the "big websites" have on their articles that Google shows me.
Appendix 6: A fun comic by @zenacomics 
 
Appendix 7: More thoughts on organizing information on the internet
If you want some more in-depth thoughts on this subject, you should check out this post I wrote a year ago about all the different ways we could organize information usefully on the internet.
Google Search Is Dying
https://dkb.io/post/google-search-is-dying
https://vicki.substack.com/p/google-and-the-nothing
Google and the Nothing
2020-08-14
Vicki Boykis
<div><div><article><div><a href="https://substackcdn.com/image/fetch/f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fbucketeer-e05bbc84-baa3-437e-9518-adb32be77984.s3.amazonaws.com%2Fpublic%2Fimages%2F3f6ac7c2-12c7-4ecb-935e-89bcabf7035f_1025x1600.png"><div></div></a><p><em><strong>Art: Friendship, Pablo Picasso, 1908</strong></em></p><p><em>Vicki’s Note: I realize the irony of complaining about paywalls in a semi-paywalled newsletter, so as of today, I’m working on opening up all the paywalled Normcore archives. I’ve gotten up to April of this month so far (it takes like 5 clicks to ungate each post.) Only the latest week’s paid subscriber post will be locked, and then after a weekish passes, I’ll unlock it. </em></p><p>Google, for better or worse, has become our closest friend, our confidant, our therapist, the place where we go to whisper our secret fears at 3 am, hoping it will answer them.  And Google knew this as early as 2009, when it made this ad: </p><div><p></p></div><p>Ever since, we have been drawn to Google for advice, for solace, just to know that someone else is out there with the same problems as us. We have used the power of hundreds of thousands of machines<a href="https://vicki.substack.com/p/data-centers-are-the-new-oil"> churning away quietly in Utah, Paris, and around the world</a>, just to connect to a single other person offering us the advice we need. </p><p><strong>Your Worst Best Friend</strong></p><p>Off the top of my head, here’s a list of things that I’ve needed someone with more experience than me to weigh in on very quickly over the past couple months, sometimes also at 3 in the morning. </p><ul><li><p>Baby fever teething</p></li><li><p><a href="https://vicki.substack.com/p/how-do-you-fake-a-tornado">Tornado watch Philadelphia</a></p></li><li><p>COVID symptoms</p></li><li><p>Pandemic crafts 5 year old</p></li><li><p>COVID testing near me</p></li><li><p>Can you transmit COVID even if you don’t have it</p></li><li><p>Getting rid of ants under deck</p></li><li><p>Can ants catch COVID</p></li></ul><p>I don’t remember the exact searches I did, because<a href="https://myactivity.google.com/?continue=https://myactivity.google.com/myactivity"> I disabled search history, which Google studiously tracks, sometime after 2014</a>. It’s probably for the best, anyway. My last search recorded search results strings included,, inexplicably, in a row “roman gods”, “Philadelphia man cheese” and “zip -e unix.”  </p><p> Right around the same time that I disabled search history was when my love affair with Google started unravelling. </p><p>As was the case for many of us, Google was my confidant and best friend. But, after the <a href="https://vicki.substack.com/p/edward-snowden-and-our-great-cross">revelations of Edward Snowden,</a> it became a gaping hole that all of my extremely personal data went into, only to come out some other end in an NSA Hive cluster, or worse, targeted and resold to advertisers who now also all knew about my ant problem and my baby teething. </p><p>What didn’t help is that Google itself was becoming less of a friendly, quirky search friend, but a big, grown-up corporation. And, as it grew, it became colder, <a href="https://vicki.substack.com/p/i-wanna-be-in-the-black-box-where">more strategic</a>. Which led the webpages populating the internet to also become more strategic so that they could get the Google algorithm’s attention. </p><p>Whereas before we could search Google and get direct access to information, now every single thing that we Google for produces not blogs or homemade sites, actual news, but manufactured, packaged search results with predetermined and prefiltered results that lead us to click through to sites that are so packed full of SEO, spam, ads, and cruft, that it’s impossible to squeeze the last bit of content from the vast internet. </p><p>In a way, the progress of content on the internet has reminded me of the old children’s classic, Neverending Story (highly recommend the book as well as the movie), where the land of Fantasia is disappearing as it is consumed by <a href="https://theneverendingstory.fandom.com/wiki/The_Nothing">The Nothing</a>, a nameless, shapeless void that consumes Fantasia bit by bit. All Fantasians consumed by The Nothing are reborn as lies in the human world. </p><blockquote><p><strong>G'mork:</strong><br /> Foolish boy. Don't you know anything about Fantasia? It's the world of human fantasy. Every part, every creature of it, is a piece of the dreams and hopes of mankind. Therefore, it has no boundaries.</p><p><strong>Atreyu:</strong><br /> But why is Fantasia dying, then?</p><p><strong>G'mork:</strong><br /> Because people have begun to lose their hopes and forget their dreams. So the Nothing grows stronger.</p><p><strong>Atreyu:</strong><br /> What is the Nothing?</p><p><strong>G'mork:</strong><br /> It's the emptiness that's left. It's like a despair, destroying this world. And I have been trying to help it.</p><p><strong>Atreyu:</strong><br /> But why?</p><p><strong>G'mork:</strong><br /> Because people who have no hopes are easy to control. And whoever has control has the Power.</p></blockquote><p>Why is this happening? Really, there are two separate issues. </p><p><strong>The First Problem: The Google Search Result Page</strong></p><p>Google’s results page used to be so easy. You search for “roman gods”, it sends you back some links to Wikipedia, some to random Latin class pages built by university professors, and maybe a link to a site in Italy. </p><p>These days, Google, who has before stated that they don’t want to be an arbiter of truth, is an arbiter of truth with <a href="https://developers.google.com/search/docs/guides/search-gallery">Google Cards and Snippets</a>, which are usually the first results for any search. The reason they say they do this, of course, is that it’s more efficient. But really, they don’t want you to leave Google, which has resulted <a href="https://www.theverge.com/2020/8/11/21363692/google-genius-lyrics-lawsuit-scraping-copyright-yelp-antitrust-competition">in a dubious practice</a> that verges but slightly skirts web scraping, and a results page that looks like a billboard instead of a text search. </p><a href="https://substackcdn.com/image/fetch/f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fbucketeer-e05bbc84-baa3-437e-9518-adb32be77984.s3.amazonaws.com%2Fpublic%2Fimages%2F6a6b6bf8-d950-4cdd-9cac-eafd07aaa6ee_1600x1022.png"><div></div></a><p>Then, there is <a href="https://www.trendmicro.com/vinfo/us/security/news/cybercrime-and-digital-threats/cybercriminals-using-paid-ads-top-cloud-providers-customers">keyword highjacking</a>. </p><p>Then, there is the way the Google homepage was <a href="https://www.theverge.com/2020/1/24/21080424/google-search-result-ads-desktop-favicon-redesign-backtrack-controversial-experiment">recently redesigned</a>, and promptly rolled back, </p><blockquote><p>Google made one of the<a href="https://www.theverge.com/tldr/2020/1/23/21078343/google-ad-desktop-design-change-favicon-icon-ftc-guidelines"> biggest changes to how it displays search results</a> in the company’s history earlier this month, with the changes taking effect over the course of the last week. It involved a visual overhaul that makes it more difficult to differentiate between advertising and organic search results with the removal of color overlays and the introduction of small branded iconography, known on the web as favicons, next to non-ad results. </p></blockquote><p>The search results page that comes up in the commercial from 2009 seems laughably naive, cute. It’s just way, way too functional. </p><a href="https://substackcdn.com/image/fetch/f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fbucketeer-e05bbc84-baa3-437e-9518-adb32be77984.s3.amazonaws.com%2Fpublic%2Fimages%2F24ea9bf9-4b70-4277-9b7d-3beea99b5e49_1600x845.png"><div></div></a><p>Everything that Google has done to make people stay on the page and make results more accessible has resulted in a set of editorial decisions that mean we’re no longer getting straight search results that take us away from the page, but are in some kind of Upside Down Google Funhouse Internet where the content is brought directly to us, and the internet that is brought to us is stripped of context, of the richness of the pages on their own pages, wrapped up in Google’s business logic. </p><p>The text and content that works best in this new Funouse Internet is content that plays by the rules: it’s easy to scrape, it’s made up of videos and pictures that catch the algorithm’s attention, and it self-links in hundreds of ways to boost SEO.</p><p>It’s the opposite of actual, good content that people want to consume. </p><p>As Visa says, </p><p><strong>The Second Problem: Building for Google</strong></p><p>The second issue is that sites now understand that they’re competing with this Upside Down Internet. It’s up there on their list of worries, along with a shrinking public attention span, shrinking ad revenues, an uncertain market, and everything that makes the internet the beautiful, <a href="https://vicki.substack.com/p/our-beautiful-broken-internet">broken thing it is today</a>. </p><p>And, by the way, all of this has been going on for years. <a href="https://news.ycombinator.com/item?id=1753134">This comment</a>, from Hacker News’s prolific <a href="https://twitter.com/patio11">patio11</a>, was posted in 2010, only a few years after that initial Paris ad.</p><a href="https://substackcdn.com/image/fetch/f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fbucketeer-e05bbc84-baa3-437e-9518-adb32be77984.s3.amazonaws.com%2Fpublic%2Fimages%2F5b52d284-ced1-4452-977d-2a885e6a568e_1416x384.png"><div></div></a><p>And so these sites are turning to the only strategies they know that will keep people on their own sites, which are dark anti-patterns - more propogation of The Nothing. </p><p>I’m going to pick on Business Insider here because they’re one of the worst offenders, and also because every time I research something business-related for a Normcore post, they’re one of the first sites that come up.  Not only is everything ridiculously paywalled, but this is the mobile experience. And there’s a lot more dark patterns where <a href="https://old.reddit.com/r/assholedesign/search?q=business+insider&amp;restrict_sr=on&amp;sort=relevance&amp;t=all">that comes from</a>. </p><p>Every single piece of content is designed to funnel people to ads, to the worst possible UX patterns, because that’s what makes money according to Google right now. My favorite recent piece on these patterns has been “<a href="https://daringfireball.net/2017/06/medium_dickbars">Medium and the Scourge of the Persistent Sharing Dickbars</a>,” which Medium ended up removing, but the ...medium...has gotten worse and worse. </p><p>Literally any search I’ve done lately results in either pages of results from sketchy websites, or a paywall, or an article that repeats the phrase over and over again for SEO’s sake and doesn’t get to the meat of the question I’m looking for. I tried finding a specific example, but there are too many to list.</p><p>A search for baby teething symptoms leads to WebMD, <a href="https://mashable.com/article/google-facebook-tracking-online-health-activity-webmd/">which sells your data to Google</a>, or Mayo Clinic, which might be ok, <a href="https://www.mayoclinic.org/documents/mayo-seeks-to-dominate-with-data/doc-20139628">but it’s been on the Big Data bandwagon for a long time</a>, so who knows what they’re doing with front-end tracking data. By the way, if you go down the Mayo Clinic rabbit hole for too long,<a href="https://www.beckershospitalreview.com/healthcare-information-technology/mayo-clinic-sells-data-center-to-epic-for-46m.html"> you get to this article</a> where they’re collaborating with Epic, the really weird but bigtime healthcare record company whose leadership is forcing employees <a href="https://twitter.com/vboykis/status/1290825429549539329">back to the office during a pandemic</a>. </p><p>Or, if you’re looking for recipes, where it takes at least a page or two to get to the actual content because it’s better for sites to show videos and newsletter popups. </p><p>Here we are thinking that we were in control of our websites, but what’s happened is that Google (and, later, its social friends), has been in the driver’s seat for how internet content is developed. </p><p>I recently went back through my blog archives, and it looks like I’ve been struggling with this shrinking content since 2016, when I scraped a <a href="http://veekaybee.github.io/2016/04/07/content-is-dead/">Wired webpage to get at the text</a>. </p><blockquote><p>It’s 2016. I shouldn’t have to write a scraping script to extract the information I need from the webpage of a magazine article. But that’s exactly what I had to do in February when I needed information on switching between Android and iOS.</p></blockquote><p><strong>The Tangled Web</strong></p><p>And here we have the real fundamental problem of today’s internet:  We are awash in in websites, but the amount of information we’re getting from these websites, from all websites really, is exponentially shrinking. The Nothing is winning. </p><p> Lately, it seems like everyone else is thinking about this, as well. </p><p>And, in a <a href="https://www.currentaffairs.org/2020/08/the-truth-is-paywalled-but-the-lies-are-free">very very good article on the news paywall piece</a>, Nathan Robinson writes that much of the news media we consume today is paywalled, but more sensational sources are free, relying on clicks against Google’s algorithm for traffic. The problem is even worse in the misinformation sphere. </p><blockquote><p>A white supremacist on YouTube will<a href="https://www.youtube.com/watch?v=T0KKc6GbeNo"> tell you all about race and IQ</a> but if you want to read a careful scholarly refutation, obtaining a legal PDF<a href="https://content.apa.org/record/2005-00117-006"> from the journal publisher would cost you $14.95</a>, a price nobody in their right mind would pay for one article if they can’t get institutional access. (I recently gave up on trying to access a scholarly article because I could not find a way to get it for less than $39.95, though in that case the article was<a href="https://www.currentaffairs.org/2020/07/how-to-pretend-that-you-are-smart"> garbage rather than gold.</a>) </p></blockquote><p>It’s really a very dangerous problem, as we can see by the rise of misinformation. The more we’ve given to Google and friends, the more they’ve taken, to the point where it’s impossible to find any genuine content that’s not algorithmed away. </p><p>In the words of Marshall McLuhan, the medium has really become the message, and the message is that money matters. But the second part of what McLuhan <a href="https://web.mit.edu/allanmc/www/mcluhan.mediummessage.pdf">said after that famous quote is was very interesting</a>, as well, “This is merely to say that the personal and social consequences of any medium—that is, of any extension of our-selves—result from the new scale that is introduced into our affairs by each extension of ourselves, or by any new technology.”</p><p><strong>How do we push back the Nothing? </strong></p><p>It’s true that if you try to solve this problem today, you play by the rules of Google’s internet . Even Substack, the platform this newsletter is hosted on, which is fighting against the shrinkage of content, is impacted by Google.  </p><p><a href="https://support.substack.com/hc/en-us/articles/360038866571-Why-does-it-say-Email-too-long-in-the-post-editor-">If you send too much data in a Substack newsletter</a>, Gmail will truncate the email, which they warn you about with concern. </p><blockquote><p>Gmail will truncate emails that are more than 102KB (the "Message clipped" notice you might've seen on other people's emails), which can affect your email analytics, because they also truncate the tracking code. We display a warning after you've exceeded Gmail's size limit.</p></blockquote><p>Which means that many times, people will feel compelled to cut content so that the email gets delivered. Many other companies who deliver email also rely entirely on the good graces of being outside of Google’s spam filter. </p><p>As I type, the warning button is coming up on my Substack editor, leading me to think about what I need to cut in this draft. </p><a href="https://substackcdn.com/image/fetch/f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fbucketeer-e05bbc84-baa3-437e-9518-adb32be77984.s3.amazonaws.com%2Fpublic%2Fimages%2Fc0099db4-0378-47c4-91d9-bf057810da52_384x130.png"><div></div></a><p>It’s a real doozy. Everything we do online to create content these days is reliant on Google, the company, as an arbiter of what makes content “good”, which is really the decision of maybe a few hundred PMs and developers (now at home) in California, and entirely dependent on what’s going to continue to make Google profitable instead of what’s actually good for the internet. </p><p>When I think about what I can personally do to fight decontification, I’m at a loss. I did the research for this post using Google, writing it with Substack which is impacted by Gmail delivery mechanisms, writing the draft of it in Google Docs <a href="https://vicki.substack.com/p/google-drive-is-production">which is production</a>), impacted by a million other things going on under the covers, algorithms and infrastructure I can’t even begin to list out. </p><p>Sure, I can continue to write good content on my own platform. I can continue to use adblock and fight clicking on dubious links, to tell my family not to read clickbait, <a href="https://twitter.com/vboykis/status/1074138338574168064">to strip UTMs</a>, to not use Google’s AMP service <a href="https://markosaric.com/google-amp/">to click through to links</a>, don’t use Chrome, etc. But at this point, it’s an uphill battle.</p><p>In The Neverending Story, the way Balthazar won against the Nothing was by recreating Fantasia from his imagination and giving the Childlike Empress a new name. We need to stop Google from having this power to create more Nothing. </p><p>It’s not clear that it will happen from a regulatory perspective anytime soon, and my personal opinion is that “breaking up Google” is not a realistic battle cry, because what does that mean? It’s very hostile, abstract, and not entirely actionable. Does Microsoft get Google Search product and Dropbox gets Google Docs? But who will get the Snippets? </p><p>While regulators continue to catch up on this issue and we continue to think about solutions, the only thing we can do as individuals with a stake in access to content, I think, is to keep talking about it and documenting it, to keep creating open, free content, to use DuckDuckGo or other alternatives, sidestep AMP, and to spread knowledge of the Nothing, in the hopes that, as a starting point, awareness of the problem will start to drive it back. </p><p><strong>What I’m reading lately:</strong></p><ol><li><p><a href="https://www.mcsweeneys.net/articles/sharpies">Sharpies </a>- going back to school in America</p></li><li><p>Randy on <a href="https://counting.substack.com/p/id-like-more-people-to-join-the-broader">getting more people</a> into the data community</p></li><li><p>A <a href="https://savewithandrew.substack.com/p/a-really-boring-way-to-save-50month">really boring way to save money</a></p></li><li><p>My first time hearing <a href="https://decksanddiapers.substack.com/p/concept-of-the-w-curve">about the W curve</a></p></li><li><p><a href="https://twitter.com/getdbt/status/1293565835144044545">On Dashboards</a></p></li><li><p><a href="https://themargins.substack.com/p/twitter-george-soros-and-porn">Twitter, George Soros, and Porn</a></p></li><li><p><a href="https://healeycodes.com/privacy-focused-analytics-from-scratch/?utm_source=Pointer&amp;utm_campaign=e1d3c18c96-ISSUE_199&amp;utm_medium=email&amp;utm_term=0_6ba2b83261-e1d3c18c96-592102573">Privacy-focused analytics from scratch</a></p></li><li><p>Mozilla thread </p></li></ol><p><strong>The Newsletter:</strong></p><p>This newsletter’s M.O. is takes on tech news that are rooted in humanism, nuance, context, rationality, and a little fun. It goes out once a week to free subscribers, and once more to paid subscribers. If you like it, forward it to friends and tell them to subscribe! </p><p><strong>Swag: </strong><a href="https://normcore-tech.myshopify.com/">Stickers. </a><a href="https://www.redbubble.com/people/veekaybee/works/43009292-untitled?p=mug&amp;style=standard&amp;asc=u">Mug</a>. <a href="http://www.redbubble.com/people/veekaybee/works/43009292-untitled?p=spiral-notebook&amp;asc=u">Notepad</a>. </p><p><strong>The Author: <br /></strong>I’m a machine learning engineer. Most of my free time is spent wrangling a preschooler and a baby, reading, and <a href="https://twitter.com/vboykis/status/1077024069126668289">writing bad tweets</a>. Find out more <a href="http://vickiboykis.com/">here</a> or follow me <a href="https://twitter.com/vboykis">on Twitter</a>. </p></div></article></div>||||I|||| Normcore Tech

 SubscribeSign in
 Share this post

 Google and the Nothing

 vicki.substack.com
 Copy link
 Twitter
 Facebook
 Email

 Google and the Nothing

 How Google's sprawl has shaped modern content

 Vicki Boykis
 Aug 14, 2020
 17
 1
 Share this post

 Google and the Nothing

 vicki.substack.com
 Copy link
 Twitter
 Facebook
 Email

 Art: Friendship, Pablo Picasso, 1908

 Vicki’s Note: I realize the irony of complaining about paywalls in a semi-paywalled newsletter, so as of today, I’m working on opening up all the paywalled Normcore archives. I’ve gotten up to April of this month so far (it takes like 5 clicks to ungate each post.) Only the latest week’s paid subscriber post will be locked, and then after a weekish passes, I’ll unlock it.

 Google, for better or worse, has become our closest friend, our confidant, our therapist, the place where we go to whisper our secret fears at 3 am, hoping it will answer them. And Google knew this as early as 2009, when it made this ad:

 Ever since, we have been drawn to Google for advice, for solace, just to know that someone else is out there with the same problems as us. We have used the power of hundreds of thousands of machines churning away quietly in Utah, Paris, and around the world , just to connect to a single other person offering us the advice we need.

 Your Worst Best Friend

 Off the top of my head, here’s a list of things that I’ve needed someone with more experience than me to weigh in on very quickly over the past couple months, sometimes also at 3 in the morning.

 * Baby fever teething

 * Tornado watch Philadelphia

 * COVID symptoms

 * Pandemic crafts 5 year old

 * COVID testing near me

 * Can you transmit COVID even if you don’t have it

 * Getting rid of ants under deck

 * Can ants catch COVID

 I don’t remember the exact searches I did, because I disabled search history, which Google studiously tracks, sometime after 2014 . It’s probably for the best, anyway. My last search recorded search results strings included,, inexplicably, in a row “roman gods”, “Philadelphia man cheese” and “zip -e unix.”

 Right around the same time that I disabled search history was when my love affair with Google started unravelling.

 As was the case for many of us, Google was my confidant and best friend. But, after the revelations of Edward Snowden, it became a gaping hole that all of my extremely personal data went into, only to come out some other end in an NSA Hive cluster, or worse, targeted and resold to advertisers who now also all knew about my ant problem and my baby teething.

 What didn’t help is that Google itself was becoming less of a friendly, quirky search friend, but a big, grown-up corporation. And, as it grew, it became colder, more strategic . Which led the webpages populating the internet to also become more strategic so that they could get the Google algorithm’s attention.

 Whereas before we could search Google and get direct access to information, now every single thing that we Google for produces not blogs or homemade sites, actual news, but manufactured, packaged search results with predetermined and prefiltered results that lead us to click through to sites that are so packed full of SEO, spam, ads, and cruft, that it’s impossible to squeeze the last bit of content from the vast internet.

 In a way, the progress of content on the internet has reminded me of the old children’s classic, Neverending Story (highly recommend the book as well as the movie), where the land of Fantasia is disappearing as it is consumed by The Nothing , a nameless, shapeless void that consumes Fantasia bit by bit. All Fantasians consumed by The Nothing are reborn as lies in the human world.

 G'mork:
 Foolish boy. Don't you know anything about Fantasia? It's the world of human fantasy. Every part, every creature of it, is a piece of the dreams and hopes of mankind. Therefore, it has no boundaries.

 Atreyu:
 But why is Fantasia dying, then?

 G'mork:
 Because people have begun to lose their hopes and forget their dreams. So the Nothing grows stronger.

 Atreyu:
 What is the Nothing?

 G'mork:
 It's the emptiness that's left. It's like a despair, destroying this world. And I have been trying to help it.

 Atreyu:
 But why?

 G'mork:
 Because people who have no hopes are easy to control. And whoever has control has the Power.

 Why is this happening? Really, there are two separate issues.

 The First Problem: The Google Search Result Page

 Google’s results page used to be so easy. You search for “roman gods”, it sends you back some links to Wikipedia, some to random Latin class pages built by university professors, and maybe a link to a site in Italy.

 These days, Google, who has before stated that they don’t want to be an arbiter of truth, is an arbiter of truth with Google Cards and Snippets , which are usually the first results for any search. The reason they say they do this, of course, is that it’s more efficient. But really, they don’t want you to leave Google, which has resulted in a dubious practice that verges but slightly skirts web scraping, and a results page that looks like a billboard instead of a text search.

 Then, there is keyword highjacking .

 Then, there is the way the Google homepage was recently redesigned , and promptly rolled back,

 Google made one of the biggest changes to how it displays search results in the company’s history earlier this month, with the changes taking effect over the course of the last week. It involved a visual overhaul that makes it more difficult to differentiate between advertising and organic search results with the removal of color overlays and the introduction of small branded iconography, known on the web as favicons, next to non-ad results.

 The search results page that comes up in the commercial from 2009 seems laughably naive, cute. It’s just way, way too functional.

 Everything that Google has done to make people stay on the page and make results more accessible has resulted in a set of editorial decisions that mean we’re no longer getting straight search results that take us away from the page, but are in some kind of Upside Down Google Funhouse Internet where the content is brought directly to us, and the internet that is brought to us is stripped of context, of the richness of the pages on their own pages, wrapped up in Google’s business logic.

 The text and content that works best in this new Funouse Internet is content that plays by the rules: it’s easy to scrape, it’s made up of videos and pictures that catch the algorithm’s attention, and it self-links in hundreds of ways to boost SEO.

 It’s the opposite of actual, good content that people want to consume.

 As Visa says,

 youtube.com/visakanv (💜 if you subscribe!) @visakanv
 One of my convictions: if you over-optimize for accessibility, you get flooded with low-quality/low-effort engagement, which crowds out high-quality/high-effort engagement. There are contexts where this bargain makes sense for strategic reasons, but I’m broadly averse to it
 10:47 AM ∙ Aug 14, 2020

 The Second Problem: Building for Google

 The second issue is that sites now understand that they’re competing with this Upside Down Internet. It’s up there on their list of worries, along with a shrinking public attention span, shrinking ad revenues, an uncertain market, and everything that makes the internet the beautiful, broken thing it is today .

 And, by the way, all of this has been going on for years. This comment , from Hacker News’s prolific patio11 , was posted in 2010, only a few years after that initial Paris ad.

 And so these sites are turning to the only strategies they know that will keep people on their own sites, which are dark anti-patterns - more propogation of The Nothing.

 I’m going to pick on Business Insider here because they’re one of the worst offenders, and also because every time I research something business-related for a Normcore post, they’re one of the first sites that come up. Not only is everything ridiculously paywalled, but this is the mobile experience. And there’s a lot more dark patterns where that comes from .

 Every single piece of content is designed to funnel people to ads, to the worst possible UX patterns, because that’s what makes money according to Google right now. My favorite recent piece on these patterns has been “ Medium and the Scourge of the Persistent Sharing Dickbars ,” which Medium ended up removing, but the ...medium...has gotten worse and worse.

 Literally any search I’ve done lately results in either pages of results from sketchy websites, or a paywall, or an article that repeats the phrase over and over again for SEO’s sake and doesn’t get to the meat of the question I’m looking for. I tried finding a specific example, but there are too many to list.

 A search for baby teething symptoms leads to WebMD, which sells your data to Google , or Mayo Clinic, which might be ok, but it’s been on the Big Data bandwagon for a long time , so who knows what they’re doing with front-end tracking data. By the way, if you go down the Mayo Clinic rabbit hole for too long, you get to this article where they’re collaborating with Epic, the really weird but bigtime healthcare record company whose leadership is forcing employees back to the office during a pandemic .

 Or, if you’re looking for recipes, where it takes at least a page or two to get to the actual content because it’s better for sites to show videos and newsletter popups.

 Here we are thinking that we were in control of our websites, but what’s happened is that Google (and, later, its social friends), has been in the driver’s seat for how internet content is developed.

 I recently went back through my blog archives, and it looks like I’ve been struggling with this shrinking content since 2016, when I scraped a Wired webpage to get at the text .

 It’s 2016. I shouldn’t have to write a scraping script to extract the information I need from the webpage of a magazine article. But that’s exactly what I had to do in February when I needed information on switching between Android and iOS.

 The Tangled Web

 And here we have the real fundamental problem of today’s internet: We are awash in in websites, but the amount of information we’re getting from these websites, from all websites really, is exponentially shrinking. The Nothing is winning.

 Lately, it seems like everyone else is thinking about this, as well.

 Anil Dash @anildash
 I know this makes me a cranky old man, but when I look up how to do stuff on the Internet, I want to read an article that’s maybe 600 words that I can skim. I definitely do not want to watch a YouTube video that’s been padded out to 10 minutes to feed the monetization algorithm.
 12:49 AM ∙ Aug 3, 2020
 29,656 Likes 2,826 Retweets

 And, in a very very good article on the news paywall piece , Nathan Robinson writes that much of the news media we consume today is paywalled, but more sensational sources are free, relying on clicks against Google’s algorithm for traffic. The problem is even worse in the misinformation sphere.

 A white supremacist on YouTube will tell you all about race and IQ but if you want to read a careful scholarly refutation, obtaining a legal PDF from the journal publisher would cost you $14.95 , a price nobody in their right mind would pay for one article if they can’t get institutional access. (I recently gave up on trying to access a scholarly article because I could not find a way to get it for less than $39.95, though in that case the article was garbage rather than gold. )

 It’s really a very dangerous problem, as we can see by the rise of misinformation. The more we’ve given to Google and friends, the more they’ve taken, to the point where it’s impossible to find any genuine content that’s not algorithmed away.

 In the words of Marshall McLuhan, the medium has really become the message, and the message is that money matters. But the second part of what McLuhan said after that famous quote is was very interesting , as well, “This is merely to say that the personal and social consequences of any medium—that is, of any extension of our-selves—result from the new scale that is introduced into our affairs by each extension of ourselves, or by any new technology.”

 How do we push back the Nothing?

 It’s true that if you try to solve this problem today, you play by the rules of Google’s internet . Even Substack, the platform this newsletter is hosted on, which is fighting against the shrinkage of content, is impacted by Google.

 If you send too much data in a Substack newsletter , Gmail will truncate the email, which they warn you about with concern.

 Gmail will truncate emails that are more than 102KB (the "Message clipped" notice you might've seen on other people's emails), which can affect your email analytics, because they also truncate the tracking code. We display a warning after you've exceeded Gmail's size limit.

 Which means that many times, people will feel compelled to cut content so that the email gets delivered. Many other companies who deliver email also rely entirely on the good graces of being outside of Google’s spam filter.

 As I type, the warning button is coming up on my Substack editor, leading me to think about what I need to cut in this draft.

 It’s a real doozy. Everything we do online to create content these days is reliant on Google, the company, as an arbiter of what makes content “good”, which is really the decision of maybe a few hundred PMs and developers (now at home) in California, and entirely dependent on what’s going to continue to make Google profitable instead of what’s actually good for the internet.

 When I think about what I can personally do to fight decontification, I’m at a loss. I did the research for this post using Google, writing it with Substack which is impacted by Gmail delivery mechanisms, writing the draft of it in Google Docs which is production ), impacted by a million other things going on under the covers, algorithms and infrastructure I can’t even begin to list out.

 Sure, I can continue to write good content on my own platform. I can continue to use adblock and fight clicking on dubious links, to tell my family not to read clickbait, to strip UTMs , to not use Google’s AMP service to click through to links , don’t use Chrome, etc. But at this point, it’s an uphill battle.

 In The Neverending Story, the way Balthazar won against the Nothing was by recreating Fantasia from his imagination and giving the Childlike Empress a new name. We need to stop Google from having this power to create more Nothing.

 Pinboard @Pinboard
 This situation—a single company controlling web standards—is what everyone freaked out about in the Windows 95 era, but the emergence of Google as de facto regulatory authority for the web doesn't seem to rankle people as much as it should.
 Miguel de Icaza @migueldeicaza
 @james_clark @robinberjon Realistically, I think that we are headed to a single engine world based on Chromium. The only thing preventing this is Apple’s requirement for WebKit on iOS.
 7:01 PM ∙ Aug 12, 2020
 523 Likes 235 Retweets

 It’s not clear that it will happen from a regulatory perspective anytime soon, and my personal opinion is that “breaking up Google” is not a realistic battle cry, because what does that mean? It’s very hostile, abstract, and not entirely actionable. Does Microsoft get Google Search product and Dropbox gets Google Docs? But who will get the Snippets?

 While regulators continue to catch up on this issue and we continue to think about solutions, the only thing we can do as individuals with a stake in access to content, I think, is to keep talking about it and documenting it, to keep creating open, free content, to use DuckDuckGo or other alternatives, sidestep AMP, and to spread knowledge of the Nothing, in the hopes that, as a starting point, awareness of the problem will start to drive it back.

 What I’m reading lately:

 1. Sharpies - going back to school in America

 2. Randy on getting more people into the data community

 3. A really boring way to save money

 4. My first time hearing about the W curve

 5. On Dashboards

 6. Twitter, George Soros, and Porn

 7. Privacy-focused analytics from scratch

 8. Mozilla thread

 Ted Mielczarek @TedMielczarek
 I'm convinced that the biggest problem Mozilla had was that the business model we stumbled into (ad revenue sharing from search providers) gave us a firehose of money that was mostly disconnected from our execution no matter how you measure things.
 4:12 PM ∙ Aug 12, 2020
 1,671 Likes 227 Retweets

 The Newsletter:

 This newsletter’s M.O. is takes on tech news that are rooted in humanism, nuance, context, rationality, and a little fun. It goes out once a week to free subscribers, and once more to paid subscribers. If you like it, forward it to friends and tell them to subscribe!

 Swag: Stickers. Mug . Notepad .

 The Author:
 I’m a machine learning engineer. Most of my free time is spent wrangling a preschooler and a baby, reading, and writing bad tweets . Find out more here or follow me on Twitter .

 1
 Share this post

 Google and the Nothing

 vicki.substack.com
 Copy link
 Twitter
 Facebook
 Email
 1 Comment
 atec 
 Aug 15, 2020 
 
 Zero is Nothing 
 
 not even One 
 
 from Zero to One 
 
 Expand full comment
 ReplyCollapse 
 
 Top New Community

 No posts

 Ready for more?

 Subscribe
 © 2023 Vicki Boykis
 Privacy ∙ Terms ∙ Collection notice
 Start WritingGet the app
 Substack is the home for great writing
 This site requires JavaScript to run correctly. Please turn on JavaScript or unblock scripts
Art: Friendship, Pablo Picasso, 1908    Vicki’s Note: I realize the irony of complaining about paywalls in a semi-paywalled newsletter, so as of today, I’m working on opening up all the paywalled Normcore archives. I’ve gotten up to April of this month so far (it takes like 5 clicks to ungate each post.) Only the latest week’s paid subscriber post will be locked, and then after a weekish passes, I’ll unlock it.   Google, for better or worse, has become our closest friend, our confidant, our therapist, the place where we go to whisper our secret fears at 3 am, hoping it will answer them.  And Google knew this as early as 2009, when it made this ad:      Ever since, we have been drawn to Google for advice, for solace, just to know that someone else is out there with the same problems as us. We have used the power of hundreds of thousands of machines churning away quietly in Utah, Paris, and around the world, just to connect to a single other person offering us the advice we need.   Your Worst Best Friend  Off the top of my head, here’s a list of things that I’ve needed someone with more experience than me to weigh in on very quickly over the past couple months, sometimes also at 3 in the morning.    Baby fever teething    Tornado watch Philadelphia    COVID symptoms   Pandemic crafts 5 year old   COVID testing near me   Can you transmit COVID even if you don’t have it   Getting rid of ants under deck   Can ants catch COVID   I don’t remember the exact searches I did, because I disabled search history, which Google studiously tracks, sometime after 2014. It’s probably for the best, anyway. My last search recorded search results strings included,, inexplicably, in a row “roman gods”, “Philadelphia man cheese” and “zip -e unix.”    Right around the same time that I disabled search history was when my love affair with Google started unravelling.  As was the case for many of us, Google was my confidant and best friend. But, after the revelations of Edward Snowden, it became a gaping hole that all of my extremely personal data went into, only to come out some other end in an NSA Hive cluster, or worse, targeted and resold to advertisers who now also all knew about my ant problem and my baby teething.  What didn’t help is that Google itself was becoming less of a friendly, quirky search friend, but a big, grown-up corporation. And, as it grew, it became colder, more strategic. Which led the webpages populating the internet to also become more strategic so that they could get the Google algorithm’s attention.  Whereas before we could search Google and get direct access to information, now every single thing that we Google for produces not blogs or homemade sites, actual news, but manufactured, packaged search results with predetermined and prefiltered results that lead us to click through to sites that are so packed full of SEO, spam, ads, and cruft, that it’s impossible to squeeze the last bit of content from the vast internet.  In a way, the progress of content on the internet has reminded me of the old children’s classic, Neverending Story (highly recommend the book as well as the movie), where the land of Fantasia is disappearing as it is consumed by The Nothing, a nameless, shapeless void that consumes Fantasia bit by bit. All Fantasians consumed by The Nothing are reborn as lies in the human world.    G'mork:  Foolish boy. Don't you know anything about Fantasia? It's the world of human fantasy. Every part, every creature of it, is a piece of the dreams and hopes of mankind. Therefore, it has no boundaries.  Atreyu:  But why is Fantasia dying, then?  G'mork:  Because people have begun to lose their hopes and forget their dreams. So the Nothing grows stronger.  Atreyu:  What is the Nothing?  G'mork:  It's the emptiness that's left. It's like a despair, destroying this world. And I have been trying to help it.  Atreyu:  But why?  G'mork:  Because people who have no hopes are easy to control. And whoever has control has the Power.  Why is this happening? Really, there are two separate issues.   The First Problem: The Google Search Result Page  Google’s results page used to be so easy. You search for “roman gods”, it sends you back some links to Wikipedia, some to random Latin class pages built by university professors, and maybe a link to a site in Italy.  These days, Google, who has before stated that they don’t want to be an arbiter of truth, is an arbiter of truth with Google Cards and Snippets, which are usually the first results for any search. The reason they say they do this, of course, is that it’s more efficient. But really, they don’t want you to leave Google, which has resulted in a dubious practice that verges but slightly skirts web scraping, and a results page that looks like a billboard instead of a text search.      Then, there is keyword highjacking.  Then, there is the way the Google homepage was recently redesigned, and promptly rolled back,   Google made one of the biggest changes to how it displays search results in the company’s history earlier this month, with the changes taking effect over the course of the last week. It involved a visual overhaul that makes it more difficult to differentiate between advertising and organic search results with the removal of color overlays and the introduction of small branded iconography, known on the web as favicons, next to non-ad results.   The search results page that comes up in the commercial from 2009 seems laughably naive, cute. It’s just way, way too functional.      Everything that Google has done to make people stay on the page and make results more accessible has resulted in a set of editorial decisions that mean we’re no longer getting straight search results that take us away from the page, but are in some kind of Upside Down Google Funhouse Internet where the content is brought directly to us, and the internet that is brought to us is stripped of context, of the richness of the pages on their own pages, wrapped up in Google’s business logic.  The text and content that works best in this new Funouse Internet is content that plays by the rules: it’s easy to scrape, it’s made up of videos and pictures that catch the algorithm’s attention, and it self-links in hundreds of ways to boost SEO. It’s the opposite of actual, good content that people want to consume.  As Visa says,   The Second Problem: Building for Google  The second issue is that sites now understand that they’re competing with this Upside Down Internet. It’s up there on their list of worries, along with a shrinking public attention span, shrinking ad revenues, an uncertain market, and everything that makes the internet the beautiful, broken thing it is today.  And, by the way, all of this has been going on for years. This comment, from Hacker News’s prolific patio11, was posted in 2010, only a few years after that initial Paris ad.     And so these sites are turning to the only strategies they know that will keep people on their own sites, which are dark anti-patterns - more propogation of The Nothing.  I’m going to pick on Business Insider here because they’re one of the worst offenders, and also because every time I research something business-related for a Normcore post, they’re one of the first sites that come up.  Not only is everything ridiculously paywalled, but this is the mobile experience. And there’s a lot more dark patterns where that comes from.  Every single piece of content is designed to funnel people to ads, to the worst possible UX patterns, because that’s what makes money according to Google right now. My favorite recent piece on these patterns has been “Medium and the Scourge of the Persistent Sharing Dickbars,” which Medium ended up removing, but the ...medium...has gotten worse and worse.  Literally any search I’ve done lately results in either pages of results from sketchy websites, or a paywall, or an article that repeats the phrase over and over again for SEO’s sake and doesn’t get to the meat of the question I’m looking for. I tried finding a specific example, but there are too many to list. A search for baby teething symptoms leads to WebMD, which sells your data to Google, or Mayo Clinic, which might be ok, but it’s been on the Big Data bandwagon for a long time, so who knows what they’re doing with front-end tracking data. By the way, if you go down the Mayo Clinic rabbit hole for too long, you get to this article where they’re collaborating with Epic, the really weird but bigtime healthcare record company whose leadership is forcing employees back to the office during a pandemic.  Or, if you’re looking for recipes, where it takes at least a page or two to get to the actual content because it’s better for sites to show videos and newsletter popups.  Here we are thinking that we were in control of our websites, but what’s happened is that Google (and, later, its social friends), has been in the driver’s seat for how internet content is developed.  I recently went back through my blog archives, and it looks like I’ve been struggling with this shrinking content since 2016, when I scraped a Wired webpage to get at the text.   It’s 2016. I shouldn’t have to write a scraping script to extract the information I need from the webpage of a magazine article. But that’s exactly what I had to do in February when I needed information on switching between Android and iOS.   The Tangled Web  And here we have the real fundamental problem of today’s internet:  We are awash in in websites, but the amount of information we’re getting from these websites, from all websites really, is exponentially shrinking. The Nothing is winning.   Lately, it seems like everyone else is thinking about this, as well.  And, in a very very good article on the news paywall piece, Nathan Robinson writes that much of the news media we consume today is paywalled, but more sensational sources are free, relying on clicks against Google’s algorithm for traffic. The problem is even worse in the misinformation sphere.   A white supremacist on YouTube will tell you all about race and IQ but if you want to read a careful scholarly refutation, obtaining a legal PDF from the journal publisher would cost you $14.95, a price nobody in their right mind would pay for one article if they can’t get institutional access. (I recently gave up on trying to access a scholarly article because I could not find a way to get it for less than $39.95, though in that case the article was garbage rather than gold.)   It’s really a very dangerous problem, as we can see by the rise of misinformation. The more we’ve given to Google and friends, the more they’ve taken, to the point where it’s impossible to find any genuine content that’s not algorithmed away.  In the words of Marshall McLuhan, the medium has really become the message, and the message is that money matters. But the second part of what McLuhan said after that famous quote is was very interesting, as well, “This is merely to say that the personal and social consequences of any medium—that is, of any extension of our-selves—result from the new scale that is introduced into our affairs by each extension of ourselves, or by any new technology.”  How do we push back the Nothing?   It’s true that if you try to solve this problem today, you play by the rules of Google’s internet . Even Substack, the platform this newsletter is hosted on, which is fighting against the shrinkage of content, is impacted by Google.    If you send too much data in a Substack newsletter, Gmail will truncate the email, which they warn you about with concern.   Gmail will truncate emails that are more than 102KB (the "Message clipped" notice you might've seen on other people's emails), which can affect your email analytics, because they also truncate the tracking code. We display a warning after you've exceeded Gmail's size limit.  Which means that many times, people will feel compelled to cut content so that the email gets delivered. Many other companies who deliver email also rely entirely on the good graces of being outside of Google’s spam filter.  As I type, the warning button is coming up on my Substack editor, leading me to think about what I need to cut in this draft.      It’s a real doozy. Everything we do online to create content these days is reliant on Google, the company, as an arbiter of what makes content “good”, which is really the decision of maybe a few hundred PMs and developers (now at home) in California, and entirely dependent on what’s going to continue to make Google profitable instead of what’s actually good for the internet.  When I think about what I can personally do to fight decontification, I’m at a loss. I did the research for this post using Google, writing it with Substack which is impacted by Gmail delivery mechanisms, writing the draft of it in Google Docs which is production), impacted by a million other things going on under the covers, algorithms and infrastructure I can’t even begin to list out.  Sure, I can continue to write good content on my own platform. I can continue to use adblock and fight clicking on dubious links, to tell my family not to read clickbait, to strip UTMs, to not use Google’s AMP service to click through to links, don’t use Chrome, etc. But at this point, it’s an uphill battle. In The Neverending Story, the way Balthazar won against the Nothing was by recreating Fantasia from his imagination and giving the Childlike Empress a new name. We need to stop Google from having this power to create more Nothing.  It’s not clear that it will happen from a regulatory perspective anytime soon, and my personal opinion is that “breaking up Google” is not a realistic battle cry, because what does that mean? It’s very hostile, abstract, and not entirely actionable. Does Microsoft get Google Search product and Dropbox gets Google Docs? But who will get the Snippets?  While regulators continue to catch up on this issue and we continue to think about solutions, the only thing we can do as individuals with a stake in access to content, I think, is to keep talking about it and documenting it, to keep creating open, free content, to use DuckDuckGo or other alternatives, sidestep AMP, and to spread knowledge of the Nothing, in the hopes that, as a starting point, awareness of the problem will start to drive it back.   What I’m reading lately:     Sharpies - going back to school in America   Randy on getting more people into the data community   A really boring way to save money    My first time hearing about the W curve     On Dashboards     Twitter, George Soros, and Porn     Privacy-focused analytics from scratch    Mozilla thread     The Newsletter:  This newsletter’s M.O. is takes on tech news that are rooted in humanism, nuance, context, rationality, and a little fun. It goes out once a week to free subscribers, and once more to paid subscribers. If you like it, forward it to friends and tell them to subscribe!   Swag:  Stickers.  Mug. Notepad.   The Author:  I’m a machine learning engineer. Most of my free time is spent wrangling a preschooler and a baby, reading, and writing bad tweets. Find out more here or follow me on Twitter.
Google and the Nothing
https://vicki.substack.com/p/google-and-the-nothing
https://www.surgehq.ai/blog/is-google-search-deteriorating-measuring-search-quality-in-2022
Is Google Search Deteriorating? Measuring Google's Search Quality in 2022
2022-01-10
Edwin Chen
<div><div><p>There's been <a href="https://news.ycombinator.com/item?id=29772136">a lot of discussion</a> on Hacker News recently about the quality of Google Search, and whether itâs deteriorated.<br /></p><figure><p></p></figure><p>Anecdotally, Iâve noticed this myself. For example, a friend and I were chatting about Databricks last week, and we searched <a href="https://www.google.com/search?q=databricks+series+b+valuation">"databricks series b valuation"</a> in order to figure out what their Series B valuation was. Unfortunately, Google doesn't understand what "series b" means (it seems to confuse "b" and "billions"), so the first search result is irrelevant. I don't even get any information about their Series B until below the fold!<br /></p><figure><p></p><figcaption>"databricks series b valuation" on Google</figcaption></figure><p>In contrast, <strong>Bing's search results page is much better.</strong> Information about the Series B is right in the expanded first search result (it doesnât contain valuation information, but thatâs expected because the Series B valuation isnât public), and the right-hand sidebar is also quite helpful.<br /></p><figure><p></p><figcaption>"databricks series b valuation" on Bing</figcaption></figure><p>So why might Google Search be deteriorating? A couple plausible reasons:<br /></p><ol><li><strong>Google has been prioritizing short-term ad revenue over search quality.</strong> Interestingly, Google has a <a href="https://research.google/pubs/pub43887/">well-known paper</a> explaining why focusing on the long-term is better for users and their business!<br /></li></ol><ol><li><strong>Information is moving beyond traditional webpages.</strong> These days, content often lives on Twitter, Facebook, YouTube, Medium, Reddit, etc. The Internet today is very different from the Internet that Google Search was born in!<br /></li></ol><ol><li><strong>Historically, Google Search contained little ML.</strong> From what I've heard, this has changed in recent years, due to changes in leadership and<a href="https://research.google/pubs/pub43887/"> improvements in AI</a>. Is it possible that ML is inadvertently making quality worse?<br /></li></ol><ol><li><strong>Crucially, measuring search quality is a very difficult problem.</strong> Naively, for example, you might think that a better search algorithm leads to more clicks: when I search for "databricks series b valuation", you might think that I want to click on a website containing the information. But ideally I might never click on a website at all! The ideal SERP may be one that displays the valuation at the top of SERP itself. Whatâs more, clicking is often a bad sign: I might click on Google's first search result about the Series H, because I mistakenly think it contains information about the Series B too.<br /></li></ol><p>So is Google Search actually deteriorating? How good is it these days, and how does it compare to its competitors?</p><p>I used to work on Search Measurement at YouTube, Twitter, and Microsoft, and it's one of the major customer use cases <a href="https://www.surgehq.ai/">Surge AI</a> provides. So letâs play around and analyze just how good Google Search is in 2022!</p><h2>Background: Human Evaluation</h2><p>First of all, how do you even measure the quality of a search engine in a rigorous way?Â As mentioned above, it's very difficult to measure search quality using traditional metrics.</p><ul><li><strong>Clicks</strong> aren't necessarily something you want to optimize for, for the reasons above.</li><li>Neither is <strong>time spent</strong> searching: is a short session a good thing (perhaps you found your answer immediately) or a bad thing (the search results were so bad you quickly gave up)?</li><li>Perhaps you can measure <strong>reformulations</strong>: if your initial search query failed, you may rewrite your query and try again, so an increase in reformulations could be viewed as a bad thing. But many people will give up instead of reformulating, and how do you tell whether a query is a reformulation anyways?</li><li>Maybe <strong>long-term metrics</strong> are the solution. Happy Google Searchers will continue searching on Google. But running long A/B tests is painful if you want to quickly iterate, and even if you're unhappy with Google, is it likely that you'll switch to a competitor?<br /></li></ul><p>What's a search engine to do? One alternative that Google pioneered is the idea of <strong>human evaluation</strong>: in order to measure search quality, why don't you simply ask human raters how good your search results are? In other words, you give human raters a set of search queries and search results, and ask them to rate how well each search result satisfies the intent behind the query.<br /></p><p>There are many nuances to this approach. For example: how do raters know the intent behind the query?, do you rate search results individually or the SERP as a whole?, where do you get these raters? But overall, it's my favored approach as well.</p><h2>Example: Google Search Quality</h2><p>So in order to measure the quality of Google Search, here was my process:</p><ul><li>I leveraged a set of human raters from Surge AI. (Weâre a new kind of data labeling platform with <strong>high-skill</strong> human raters, built with <em>quality</em> as our top focus â whether you need savvy social media users tuned into USÂ politics to help <a href="https://www.surgehq.ai/blog/the-ai-bottleneck-high-quality-human-powered-data">clean up the Internet</a>, <a href="https://openai.com/blog/improving-factual-accuracy/">computer science graduates to train AI to answer how neural networks work</a>Â (many even from Ivy League schools!), high school teachers to train educational question-answering AIÂ systems for students, or Fortnite players versed in platform jargon to build gaming NLP.)</li><li>In order to get search queries to be rated, I asked 250 raters to look up a recent search in their browser history, and to use that as the search query to rate. This is an example of a <em>personalized </em>search evaluation (as opposed to one where you sample random search queries and raters guess the intent behind each one). So these are actual queries representative of the usage patterns of the broad USÂ Internet population.</li><li>Each rater then explained their original intent, rated how well the Google SERP satisfied that intent on a 1-5 scale, and explained their judgment.<br /></li></ul><p>Here are the results:<br /></p><figure><p></p></figure><p>Hereâs an examples of a search result Google performed poorly on.<br /></p><h3>Example</h3><p><strong>Search Query:</strong> <em>tim lee vlogger age</em></p><p><strong>Intent:</strong> <em>I wanted to know how old the YouTube vlogger Tim Lee is</em></p><p><strong>Rating:</strong> Bad</p><p><strong>Rater Explanation:</strong> <em>Only some of the results were about the right person, and I couldnât find his age from the results at all.</em><br /></p><figure><p></p></figure><p>(UPDATE:Â thanks to helpful readers who pointed out that <a href="https://en.wikipedia.org/wiki/Tim_Lee_(comedian)">this Wikipedia page</a> is not in fact the Tim Lee mentioned above! Google Search:Â +1, Editor:Â 0)<br /></p><h2>Example: Google vs. Bing</h2><p>In addition to asking raters to rate the Google SERP, I also asked them to compare it against Bing. This is a <strong>side-by-side</strong> eval: rather than making absolute judgments (how good is Google's SERP? how good is Bing's SERP?), sometimes it's easier to compare them (which SERP is better?).<br /></p><p>Here are the results:<br /></p><figure><p></p></figure><p>So Google does outperform Bing (the difference is statistically significant), although it's interesting to see the places where Google returns worse results. For example:<br /></p><h3>Example #1</h3><p><strong>Search Query:</strong> <em>Natural ways to heal cats who have allergies</em></p><p><strong>Intent:</strong> <em>My cat is suffering from a stuffy nose, and I wanted to find out if there natural remedies or any products that I could buy to cure it.</em><br /></p><h4><strong>Google SERP</strong></h4><p><strong>Rating:</strong> Okay</p><p><strong>Rater Explanation:</strong> <em>The Google results page was not focused on my topic. The results were confused in that half of the content pertained to allergies in cats, and the rest were for general pet allergies or allergies to cats in humans.</em><br /></p><figure><p></p></figure><figure><p></p></figure><h4><strong>Bing SERP</strong><br /></h4><p><strong>Rating:</strong> Amazing</p><p><strong>Rater Explanation:</strong> <em>Bing's results were spot on. First, they provided products to buy in the advertisement section, then they offered a specifically targeted article that suggested how to treat and prevent allergies in cats utilizing home remedies.</em></p><figure><p></p></figure><p><strong>Overall: Bing was Much Better.</strong> <em>Bing was clearly the better search result because it answered the search in every possible way that a user may want with ads, articles, images, and how-tos. Google's page offered products to buy and some interesting articles, but their search results were based on a misunderstanding of the query.</em><br /></p><h3>Example #2</h3><p><strong>Search Query:</strong> <em>what is message blocking on iphone</em></p><p><strong>Intent:</strong> <em>I received a message on my phone after texting a relative with a line saying "message blocking." I wasn't fighting with that cousin and we're always in good standing, so I thought it was odd. I also thought it was odd because I've never heard of an automatic message telling you your message was blocked, usually companies try to make that unknown and discreet, so I looked it up</em><br /></p><h4><strong>GoogleÂ SERP</strong></h4><p><strong>Rating:</strong> Okay</p><p><strong>Rater Explanation:</strong> <em>The first and third search results misunderstood my query and thought I was asking how to block other people. The second result, however, was helpful.<br /></em></p><figure><p></p></figure><h3><strong>Bing SERP</strong><br /></h3><p><strong>Rating:</strong> Good</p><p><strong>Explanation:</strong> <em>The website search results all understood what I was looking for, and were related to the message I received.</em><br /></p><figure><p></p></figure><p><strong>Overall: Bing was Much Better.</strong> <em>Google really didn't understand the question I was asking and gave me unhelpful answers. Bing understood that I was questioning why I was getting that message.</em><br /></p><h3>Example #3</h3><p><strong>Search Query:</strong> <em>Indianapolis free COVID PCR tests</em></p><p><strong>Intent:</strong> <em>I was attempting to find free PCR tests where I live in Indianapolis. Ideally, a listing or map of all resources in the area would be desirable.</em><br /></p><h4><strong>GoogleÂ SERP</strong></h4><p><strong>Rating:</strong> Okay</p><p><strong>Rater Explanation:</strong> <em>I would have liked to have seen the ISDH result sooner than I did in the search results. Or at least a map sooner in the search results of available testing options in the city. Instead, I got a page full of ads.</em>â<em><br /></em></p><figure><p></p></figure><h3><strong>BingÂ SERP</strong><br /></h3><p><strong>Rating:</strong> Good</p><p><strong>Rater Explanation:</strong> <em>The second search result was a map with a listing of testing resources, their locations, and when they were open. This was helpful information to me!</em></p><figure><p></p></figure><p><strong>Overall: Bing was Much Better.</strong> <em>I thought I would have had to click on a link in the search results first before getting a map, but I got one as soon as I searched with Bing. </em><br /></p><h2>Beating Google</h2><p>This human evaluation method can also be a useful way to find patterns of deficiencies in Google Search. For example, I hate searching for recipes on Google, since the results favor Pinterest-style blog posts with endless narrative and ads before you get to the recipe itself. So what are other categories of deficiencies where competitors could arise?<br /></p><p>We'll cover that next, as well as a comparison of Google vs. DuckDuckGo! <br /></p></div><div><p></p><div><h3>Data Labeling 2.0 for Rich, Creative AI</h3><p>Superintelligent AI, meet your human teachers. Our data labeling platform is designed from the ground up to train the next generation of AI âÂ whether itâs systems that can code in Python, summarize poetry, or detect the subtleties of toxic speech. Use our powerful data labeling workforce and tools to build the rich, human-powered datasets you need today.</p></div></div>||||I|||| Data Labeling
 for the Richness of AI
 Go to Homepage
 Large Language Models
 Customers
 Use Cases
 Latest Case Study
 Use Cases
 Use Cases
 Language Models

 Electrify your large language models

 Search Evaluation

 Power your search engine with human evaluation

 Content Moderation

 Moderate content like never before

 Other Use Cases
 Instruct and RLHF Training
 Hate Speech Detection
 Customer Support
 Financial Categorization
 GPT Fine Tuning
 Hundreds of other uses
 Latest Case Study
 Search Ranking with Neeva
 To succeed, Neeva needs to understand what users think of its search engine and how its capabilities stack up against incumbents....
 Read more
 Blog
 About
 DatasetsPricingFAQCompanyContact
 Schedule Call
 Sign in
 Sign up
 Sign up
 Home
 /
 Blog
 /
 Human Evaluation

 Is Google Search Deteriorating? Measuring Google's Search Quality in 2022

 Edwin Chen
 Jan 10, 2022

 Has Google's Search Quality deteriorated in recent years? This post measures Google Search using human evaluation.

 There's been a lot of discussion on Hacker News recently about the quality of Google Search, and whether itâs deteriorated.

 Anecdotally, Iâve noticed this myself. For example, a friend and I were chatting about Databricks last week, and we searched "databricks series b valuation" in order to figure out what their Series B valuation was. Unfortunately, Google doesn't understand what "series b" means (it seems to confuse "b" and "billions"), so the first search result is irrelevant. I don't even get any information about their Series B until below the fold!

 "databricks series b valuation" on Google

 In contrast, Bing's search results page is much better. Information about the Series B is right in the expanded first search result (it doesnât contain valuation information, but thatâs expected because the Series B valuation isnât public), and the right-hand sidebar is also quite helpful.

 "databricks series b valuation" on Bing

 So why might Google Search be deteriorating? A couple plausible reasons:

 1. Google has been prioritizing short-term ad revenue over search quality. Interestingly, Google has a well-known paper explaining why focusing on the long-term is better for users and their business!
 1. Information is moving beyond traditional webpages. These days, content often lives on Twitter, Facebook, YouTube, Medium, Reddit, etc. The Internet today is very different from the Internet that Google Search was born in!
 1. Historically, Google Search contained little ML. From what I've heard, this has changed in recent years, due to changes in leadership and improvements in AI. Is it possible that ML is inadvertently making quality worse?
 1. Crucially, measuring search quality is a very difficult problem. Naively, for example, you might think that a better search algorithm leads to more clicks: when I search for "databricks series b valuation", you might think that I want to click on a website containing the information. But ideally I might never click on a website at all! The ideal SERP may be one that displays the valuation at the top of SERP itself. Whatâs more, clicking is often a bad sign: I might click on Google's first search result about the Series H, because I mistakenly think it contains information about the Series B too.

 So is Google Search actually deteriorating? How good is it these days, and how does it compare to its competitors?

 I used to work on Search Measurement at YouTube, Twitter, and Microsoft, and it's one of the major customer use cases Surge AI provides. So letâs play around and analyze just how good Google Search is in 2022!

 Background: Human Evaluation

 First of all, how do you even measure the quality of a search engine in a rigorous way?Â As mentioned above, it's very difficult to measure search quality using traditional metrics.

 * Clicks aren't necessarily something you want to optimize for, for the reasons above.
 * Neither is time spent searching: is a short session a good thing (perhaps you found your answer immediately) or a bad thing (the search results were so bad you quickly gave up)?
 * Perhaps you can measure reformulations: if your initial search query failed, you may rewrite your query and try again, so an increase in reformulations could be viewed as a bad thing. But many people will give up instead of reformulating, and how do you tell whether a query is a reformulation anyways?
 * Maybe long-term metrics are the solution. Happy Google Searchers will continue searching on Google. But running long A/B tests is painful if you want to quickly iterate, and even if you're unhappy with Google, is it likely that you'll switch to a competitor?

 What's a search engine to do? One alternative that Google pioneered is the idea of human evaluation: in order to measure search quality, why don't you simply ask human raters how good your search results are? In other words, you give human raters a set of search queries and search results, and ask them to rate how well each search result satisfies the intent behind the query.

 There are many nuances to this approach. For example: how do raters know the intent behind the query?, do you rate search results individually or the SERP as a whole?, where do you get these raters? But overall, it's my favored approach as well.

 Example: Google Search Quality

 So in order to measure the quality of Google Search, here was my process:

 * I leveraged a set of human raters from Surge AI. (Weâre a new kind of data labeling platform with high-skill human raters, built with quality as our top focus â whether you need savvy social media users tuned into USÂ politics to help clean up the Internet, computer science graduates to train AI to answer how neural networks workÂ (many even from Ivy League schools!), high school teachers to train educational question-answering AIÂ systems for students, or Fortnite players versed in platform jargon to build gaming NLP.)
 * In order to get search queries to be rated, I asked 250 raters to look up a recent search in their browser history, and to use that as the search query to rate. This is an example of a personalized search evaluation (as opposed to one where you sample random search queries and raters guess the intent behind each one). So these are actual queries representative of the usage patterns of the broad USÂ Internet population.
 * Each rater then explained their original intent, rated how well the Google SERP satisfied that intent on a 1-5 scale, and explained their judgment.

 Here are the results:

 Hereâs an examples of a search result Google performed poorly on.

 Example

 Search Query: tim lee vlogger age

 Intent: I wanted to know how old the YouTube vlogger Tim Lee is

 Rating: Bad

 Rater Explanation: Only some of the results were about the right person, and I couldnât find his age from the results at all.

 (UPDATE:Â thanks to helpful readers who pointed out that this Wikipedia page is not in fact the Tim Lee mentioned above! Google Search:Â +1, Editor:Â 0)

 Example: Google vs. Bing

 In addition to asking raters to rate the Google SERP, I also asked them to compare it against Bing. This is a side-by-side eval: rather than making absolute judgments (how good is Google's SERP? how good is Bing's SERP?), sometimes it's easier to compare them (which SERP is better?).

 Here are the results:

 So Google does outperform Bing (the difference is statistically significant), although it's interesting to see the places where Google returns worse results. For example:

 Example #1

 Search Query: Natural ways to heal cats who have allergies

 Intent: My cat is suffering from a stuffy nose, and I wanted to find out if there natural remedies or any products that I could buy to cure it.

 Google SERP

 Rating: Okay

 Rater Explanation: The Google results page was not focused on my topic. The results were confused in that half of the content pertained to allergies in cats, and the rest were for general pet allergies or allergies to cats in humans.

 Bing SERP

 Rating: Amazing

 Rater Explanation: Bing's results were spot on. First, they provided products to buy in the advertisement section, then they offered a specifically targeted article that suggested how to treat and prevent allergies in cats utilizing home remedies.

 Overall: Bing was Much Better. Bing was clearly the better search result because it answered the search in every possible way that a user may want with ads, articles, images, and how-tos. Google's page offered products to buy and some interesting articles, but their search results were based on a misunderstanding of the query.

 Example #2

 Search Query: what is message blocking on iphone

 Intent: I received a message on my phone after texting a relative with a line saying "message blocking." I wasn't fighting with that cousin and we're always in good standing, so I thought it was odd. I also thought it was odd because I've never heard of an automatic message telling you your message was blocked, usually companies try to make that unknown and discreet, so I looked it up

 GoogleÂ SERP

 Rating: Okay

 Rater Explanation: The first and third search results misunderstood my query and thought I was asking how to block other people. The second result, however, was helpful.

 Bing SERP

 Rating: Good

 Explanation: The website search results all understood what I was looking for, and were related to the message I received.

 Overall: Bing was Much Better. Google really didn't understand the question I was asking and gave me unhelpful answers. Bing understood that I was questioning why I was getting that message.

 Example #3

 Search Query: Indianapolis free COVID PCR tests

 Intent: I was attempting to find free PCR tests where I live in Indianapolis. Ideally, a listing or map of all resources in the area would be desirable.

 GoogleÂ SERP

 Rating: Okay

 Rater Explanation: I would have liked to have seen the ISDH result sooner than I did in the search results. Or at least a map sooner in the search results of available testing options in the city. Instead, I got a page full of ads.â

 BingÂ SERP

 Rating: Good

 Rater Explanation: The second search result was a map with a listing of testing resources, their locations, and when they were open. This was helpful information to me!

 Overall: Bing was Much Better. I thought I would have had to click on a link in the search results first before getting a map, but I got one as soon as I searched with Bing.

 Beating Google

 This human evaluation method can also be a useful way to find patterns of deficiencies in Google Search. For example, I hate searching for recipes on Google, since the results favor Pinterest-style blog posts with endless narrative and ads before you get to the recipe itself. So what are other categories of deficiencies where competitors could arise?

 We'll cover that next, as well as a comparison of Google vs. DuckDuckGo!

 Related Articles

 I wanted burritos. Facebook's search engine sent me to a dead restaurant 45 minutes away.

 Edwin Chen

 Edwin oversees Surge AI's Engineering and Research teams â whether it's helping customers train large language models on human feedback, building content moderation algorithms to detect hate speech and spam, or scaling up an elite data labeling workforce. He previously led AI, Data Science, and Human Computation teams at Google, Facebook, and Twitter, and studied mathematics and linguistics at MIT.

 Data Labeling 2.0 for Rich, Creative AI

 Superintelligent AI, meet your human teachers. Our data labeling platform is designed from the ground up to train the next generation of AI âÂ whether itâs systems that can code in Python, summarize poetry, or detect the subtleties of toxic speech. Use our powerful data labeling workforce and tools to build the rich, human-powered datasets you need today.

 Receive Updates
 Subscribe
 Thank you! Your submission has been received!
 Oops! Something went wrong while submitting the form.

 Related articles

 Show all articles

 How Surge AI Built OpenAI's GSM8K Dataset of 8,500 Math Problems

 Holy $#!t: Are popular toxicity models simply profanity detectors?

 We asked 100 humans to draw the DALLÂ·E prompts

 I wanted burritos. Facebook's search engine sent me to a dead restaurant 45 minutes away.

 Data Labeling for the
 Richness of AI

 GetÂ Started

 Want to learn more? Schedule Call

 Build powerful AI datasets using our global labeling force and platform.

 Get Started Â Â Â Â
 Stay tuned

 Subscribe to our newsletter and never miss our latest news.

 Subscribe
 Thanks!
 Oops! Something went wrong while submitting the form.
 Surge AI is SOC 2 Compliant. Learn More
 Menu
 Home
 Use Cases
 FAQ
 Pricing
 About
 Careers
 Blog
 Datasets
 Contact
 Documentation
 API Documentation
 Python SDK
 Support
 Connect
 Instagram
 Twitter
 Linkedin
 Github
 Case Studies
 Redwood Research
 Content Moderation for Social Media
 Search Ranking with Neeva
 Get in touch
 hello@surgehq.ai
 careers@surgehq.ai
 2022 Â© Surge AI. All Rights Reserved
 Privacy Policy
 Terms of Service
 Platform
 * Schedule Call
 * Pricing
 * Use Cases
 * Customers
 Developers
 * Python SDK
 * API Documentation
 * Support
 * Datasets
 * FAQs
 Case Studies
 * Adversarial Data Labeling
 * Content Moderation
 * Search Ranking
 Company
 * Home
 * Blog
 * About
 * Careers
 * Contact
 Social
 * Linkedin
 * Instagram
 * Twitter
 * Github
 2022 Â© Surge AI. All Rights Reserved
 Sitemap
 |
 Terms of Service
 |
 Privacy Policy
 Follow Surge AI for the latest in AI, language, and data labeling!
<!â Replacing Submit Button â>
There's been a lot of discussion on Hacker News recently about the quality of Google Search, and whether itâs deteriorated.      Anecdotally, Iâve noticed this myself. For example, a friend and I were chatting about Databricks last week, and we searched "databricks series b valuation" in order to figure out what their Series B valuation was. Unfortunately, Google doesn't understand what "series b" means (it seems to confuse "b" and "billions"), so the first search result is irrelevant. I don't even get any information about their Series B until below the fold!     "databricks series b valuation" on Google  In contrast, Bing's search results page is much better. Information about the Series B is right in the expanded first search result (it doesnât contain valuation information, but thatâs expected because the Series B valuation isnât public), and the right-hand sidebar is also quite helpful.     "databricks series b valuation" on Bing  So why might Google Search be deteriorating? A couple plausible reasons:    Google has been prioritizing short-term ad revenue over search quality. Interestingly, Google has a well-known paper explaining why focusing on the long-term is better for users and their business!     Information is moving beyond traditional webpages. These days, content often lives on Twitter, Facebook, YouTube, Medium, Reddit, etc. The Internet today is very different from the Internet that Google Search was born in!     Historically, Google Search contained little ML. From what I've heard, this has changed in recent years, due to changes in leadership and improvements in AI. Is it possible that ML is inadvertently making quality worse?     Crucially, measuring search quality is a very difficult problem. Naively, for example, you might think that a better search algorithm leads to more clicks: when I search for "databricks series b valuation", you might think that I want to click on a website containing the information. But ideally I might never click on a website at all! The ideal SERP may be one that displays the valuation at the top of SERP itself. Whatâs more, clicking is often a bad sign: I might click on Google's first search result about the Series H, because I mistakenly think it contains information about the Series B too.   So is Google Search actually deteriorating? How good is it these days, and how does it compare to its competitors? I used to work on Search Measurement at YouTube, Twitter, and Microsoft, and it's one of the major customer use cases Surge AI provides. So letâs play around and analyze just how good Google Search is in 2022! Background: Human Evaluation First of all, how do you even measure the quality of a search engine in a rigorous way?Â As mentioned above, it's very difficult to measure search quality using traditional metrics.   Clicks aren't necessarily something you want to optimize for, for the reasons above. Neither is time spent searching: is a short session a good thing (perhaps you found your answer immediately) or a bad thing (the search results were so bad you quickly gave up)? Perhaps you can measure reformulations: if your initial search query failed, you may rewrite your query and try again, so an increase in reformulations could be viewed as a bad thing. But many people will give up instead of reformulating, and how do you tell whether a query is a reformulation anyways? Maybe long-term metrics are the solution. Happy Google Searchers will continue searching on Google. But running long A/B tests is painful if you want to quickly iterate, and even if you're unhappy with Google, is it likely that you'll switch to a competitor?   What's a search engine to do? One alternative that Google pioneered is the idea of human evaluation: in order to measure search quality, why don't you simply ask human raters how good your search results are? In other words, you give human raters a set of search queries and search results, and ask them to rate how well each search result satisfies the intent behind the query.  There are many nuances to this approach. For example: how do raters know the intent behind the query?, do you rate search results individually or the SERP as a whole?, where do you get these raters? But overall, it's my favored approach as well. Example: Google Search Quality So in order to measure the quality of Google Search, here was my process:  I leveraged a set of human raters from Surge AI. (Weâre a new kind of data labeling platform with high-skill human raters, built with quality as our top focus â whether you need savvy social media users tuned into USÂ politics to help clean up the Internet, computer science graduates to train AI to answer how neural networks workÂ (many even from Ivy League schools!), high school teachers to train educational question-answering AIÂ systems for students, or Fortnite players versed in platform jargon to build gaming NLP.) In order to get search queries to be rated, I asked 250 raters to look up a recent search in their browser history, and to use that as the search query to rate. This is an example of a personalized search evaluation (as opposed to one where you sample random search queries and raters guess the intent behind each one). So these are actual queries representative of the usage patterns of the broad USÂ Internet population. Each rater then explained their original intent, rated how well the Google SERP satisfied that intent on a 1-5 scale, and explained their judgment.   Here are the results:      Hereâs an examples of a search result Google performed poorly on.  Example  Search Query: tim lee vlogger age   Intent: I wanted to know how old the YouTube vlogger Tim Lee is   Rating: Bad  Rater Explanation: Only some of the results were about the right person, and I couldnât find his age from the results at all.       (UPDATE:Â thanks to helpful readers who pointed out that this Wikipedia page is not in fact the Tim Lee mentioned above! Google Search:Â +1, Editor:Â 0)  Example: Google vs. Bing In addition to asking raters to rate the Google SERP, I also asked them to compare it against Bing. This is a side-by-side eval: rather than making absolute judgments (how good is Google's SERP? how good is Bing's SERP?), sometimes it's easier to compare them (which SERP is better?).  Here are the results:      So Google does outperform Bing (the difference is statistically significant), although it's interesting to see the places where Google returns worse results. For example:  Example #1  Search Query: Natural ways to heal cats who have allergies   Intent: My cat is suffering from a stuffy nose, and I wanted to find out if there natural remedies or any products that I could buy to cure it.    Google SERP   Rating: Okay  Rater Explanation: The Google results page was not focused on my topic. The results were confused in that half of the content pertained to allergies in cats, and the rest were for general pet allergies or allergies to cats in humans.            Bing SERP    Rating: Amazing  Rater Explanation: Bing's results were spot on. First, they provided products to buy in the advertisement section, then they offered a specifically targeted article that suggested how to treat and prevent allergies in cats utilizing home remedies.       Overall: Bing was Much Better. Bing was clearly the better search result because it answered the search in every possible way that a user may want with ads, articles, images, and how-tos. Google's page offered products to buy and some interesting articles, but their search results were based on a misunderstanding of the query.   Example #2  Search Query: what is message blocking on iphone   Intent: I received a message on my phone after texting a relative with a line saying "message blocking." I wasn't fighting with that cousin and we're always in good standing, so I thought it was odd. I also thought it was odd because I've never heard of an automatic message telling you your message was blocked, usually companies try to make that unknown and discreet, so I looked it up    GoogleÂ SERP   Rating: Okay  Rater Explanation: The first and third search results misunderstood my query and thought I was asking how to block other people. The second result, however, was helpful.        Bing SERP    Rating: Good  Explanation: The website search results all understood what I was looking for, and were related to the message I received.        Overall: Bing was Much Better. Google really didn't understand the question I was asking and gave me unhelpful answers. Bing understood that I was questioning why I was getting that message.   Example #3  Search Query: Indianapolis free COVID PCR tests   Intent: I was attempting to find free PCR tests where I live in Indianapolis. Ideally, a listing or map of all resources in the area would be desirable.    GoogleÂ SERP   Rating: Okay  Rater Explanation: I would have liked to have seen the ISDH result sooner than I did in the search results. Or at least a map sooner in the search results of available testing options in the city. Instead, I got a page full of ads.â         BingÂ SERP    Rating: Good  Rater Explanation: The second search result was a map with a listing of testing resources, their locations, and when they were open. This was helpful information to me!       Overall: Bing was Much Better. I thought I would have had to click on a link in the search results first before getting a map, but I got one as soon as I searched with Bing.    Beating Google This human evaluation method can also be a useful way to find patterns of deficiencies in Google Search. For example, I hate searching for recipes on Google, since the results favor Pinterest-style blog posts with endless narrative and ads before you get to the recipe itself. So what are other categories of deficiencies where competitors could arise?  We'll cover that next, as well as a comparison of Google vs. DuckDuckGo!        Data Labeling 2.0 for Rich, Creative AI Superintelligent AI, meet your human teachers. Our data labeling platform is designed from the ground up to train the next generation of AI âÂ whether itâs systems that can code in Python, summarize poetry, or detect the subtleties of toxic speech. Use our powerful data labeling workforce and tools to build the rich, human-powered datasets you need today.
Is Google Search Deteriorating? Measuring Google's Search Quality in 2022
https://www.surgehq.ai/blog/is-google-search-deteriorating-measuring-search-quality-in-2022
https://future.com/the-future-of-search-is-boutique
The Future of Search Is Boutique
2022-05-02
Sari Azout
<div><div>
 <article>
	<p><i>This is an edited version of a post that originally ran </i><a href="https://sariazout.mirror.xyz/7gSSTJ96SEyvXeljymglO3zN4H6DCgVnrNZq8_2NX1A"><i>here</i></a><i>.</i></p>
<hr />
<p>For most queries, Google search is pretty underwhelming these days. Google is great at answering questions with an objective answer, like “# of billionaires in the world” or “What is the population of Iceland?” It’s pretty bad at answering questions that require judgment and context like “What do NFT collectors think about NFTs?”</p>
<p>The evidence is everywhere. These days, I find myself suppressing the garbage Internet by searching on Google for “Substack + future of learning” to find the best takes on education. We hack Twitter with the “<a href="https://twitter.com/jackbutcher/status/1435299942235615236">what is the best</a>” posts over and over again. When I’m researching a new product, I type “X item reddit” into Google. I find enormous value in small, niche, often forgotten sites like <a href="https://spaghetti.directory/">Spaghetti Directory</a>.</p>
<p>There’s an emergence of tools like Notion, Airtable, and Readwise where people are aggregating content and resources, reviving the <i>curated web</i>. But at the moment these are mostly solo affairs — hidden in private or semi-private corners of the Internet, fragmented, poorly indexed, and unavailable for public use. We haven’t figured out how to make them multiplayer. In cases where we’ve made them public and collaborative — here is<a href="https://docs.google.com/spreadsheets/d/16IDgIyPcfwJGG-zmXeMAenYbePQVHkc2P6WCwKEZgpk/htmlview#gid=0"> a great example</a> — these projects are often short-lived and poorly maintained.</p>
<p>The stated mission of a company worth almost two trillion dollars is to “organize the world’s information” and yet the Internet remains poorly organized. Or, stated differently, in a world of infinite information,<b> it’s no longer enough to organize the world’s information. It becomes important to organize the world’s </b><b><i>trustworthy</i></b><b> information.</b></p>
<p>How did we get here?</p>
<p>It’s hard to believe, but one of Google’s main problems, once it got going, was that there just wasn’t much to see online. Having a great search engine is useless if somebody types in “how to grow an herb garden” and the answer doesn’t exist online. With the advent of Google AdWords, it became profitable to put out low-quality content that passed as informative and filled Google’s search engine results. The end result is that the websites at the top of Google are not necessarily the highest-quality ones, but rather the ones that put the most effort into SEO. What started as a well-intentioned way to organize the world’s information has turned into a business focusing most of its resources on monetizing clicks to support advertisers, rather than focusing on delivering trusted search results for people.</p>
<p>The problem, now so drastically different from a decade ago, is not <i>what</i> to read/buy/eat/watch/etc., but<b> figuring out the </b><b><i>best thing</i></b><b> to read/buy/eat/watch/etc. with my limited time and attention.</b></p>
<p>Audacious teams, like <a href="https://duckduckgo.com/">DuckDuckGo</a> and <a href="https://neeva.com/">Neeva</a>, are trying to compete with Google head-on by building massive horizontal search engines. Rather than crawling and indexing things their own way, they sit on top of existing data sources and position themselves as privacy-focused alternatives to Google. But protection of privacy is not a compelling enough reason to leave Google. For the vast majority of people, allowing them to “control their own data” is not a selling point, especially if it requires paying for something they’re used to getting for free.</p>
<p>I believe the opportunity in search is not to attack Google head-on with a massive, one-size-fits-all horizontal aggregator, but instead to<b> build boutique search engines that index, curate, and organize things in new ways.</b></p>
<figure><a href="https://future.com/wp-content/uploads/2022/04/Boutique-Search-Venn-Diagram.jpg"></a></figure>
<h2>Vertical search aggregators</h2>
<p>Google is a great example of how the internet enabled scale and speed: every page on the web returned in an instant. But, increasingly, we’re seeing that this scale is at odds with a fundamental human need: <b>relevance</b>. Someone who wants to find the best freelance designer, or the best sushi restaurant, or the best NFT to buy will not find the answer on Google.</p>
<p>There is no search architecture that will work universally across all categories. It’s hard to imagine wanting to use the same UX to search for recipes as to search for freelancers. Whereas Google’s product begins and ends with a search bar, trading off functionality for simplicity, vertical search players like Yelp, Expedia, Zillow, and Behance emerged to fill functionality and relevancy gaps using structured data specific to their industries. With strong opinions on how to organize information, reflected in their choice of filters, vertical search aggregators have distinct advantages that horizontal software can never achieve.</p>
<p>But here, too, relevance depends on the sociology of the current moment. For example, on<a href="https://www.behance.net/search"> Behance</a>, the online creative community, school and location are featured prominently as filters — implying that where you live and where you went to school is an important indicator of the quality of your design portfolio. In a world where talent is being decoupled from credentialism and geography, those filters are losing relevance. </p>
<p>As signals evolve, new ways of indexing and surface areas for innovation emerge. If Behance were designed today, I’d argue neither “location” nor “school” would be filters.</p>
<figure><a href="https://future.com/wp-content/uploads/2022/04/Boutique-Search-Behance-Search.png"></a></figure>
<p>On Yelp, a search for “electricians in Miami” begins with a page titled “The Best 10 Electricians in Miami, FL” with text underneath that indicates the majority of these are sponsored results. </p>
<figure><a href="https://future.com/wp-content/uploads/2022/05/Boutique-Search-Yelp-Search.png"></a></figure>
<p><strong>When you monetize via ads, curation takes a backseat to featuring advertisers</strong> because there is just less digital real estate available to curate your own recommendations. So these platforms end up making ethically dubious design choices that generate massive <a href="https://www.usv.com/writing/2019/05/trusted-brands/">trust gaps</a>.</p>
<p>Additionally, across vertical search aggregators like Yelp, Zillow, LinkedIn, and Behance, anyone can have a profile. A combination of irrelevant filters, ad-based business models, and unconstrained supply has overwhelmed consumers and made it hard to find signals in these platforms. </p>
<p>Vertical search aggregators work when you know exactly what you want. But knowing what you want isn’t usually the starting point, which creates an <b>opportunity to help the overwhelmed consumer</b> with better discovery and curation along the funnel.</p>
<h2>Curators, curators</h2>
<p>It has become popular to say we live in the information age, and we need “curation” to help us sort through the mess. But thus far, <b>the conversation around curation has been too focused on the content and not enough on the structure</b>. We seem to have accepted the job of the curator as providing a product review, a list of links, or a song recommendation — all inside linear structures and chronological feeds designed to surface the ideas of the last 24 hours, not to accumulate and surface knowledge as needed.</p>
<p>A daily email with the top five Alibaba products feels fun and gimmicky as a side project, but it doesn’t help when you’re trying to find the best crib for your baby. Inevitably, you’ll want a way to search through a curator’s archives.</p>
<p>Curation, when thought of in the context of sharing bite-sized, isolated bits in feed-like architectures, is predominantly about <em>entertainment</em>, not <em>utility</em>. It’s not wrong to say there is a market for this kind of curation. What people miss, though, is that this market is already captured by Twitter, Facebook, and TikTok.</p>
<p>These entertainment giants offer curation that demands our attention, but they don’t offer curation on demand. <b>The opportunity is in moving curated content feeds away from their never-ending-now orientation and toward more goal-oriented interfaces.</b> People should be able to find whatever content they want on their terms and not be beholden to when the curator decides to publish.</p>
<h2>Boutique search engines are next-level curation</h2>
<p><i>All curation grows until it requires search, and all search grows until it requires curation. </i></p>
<p><i>—Ben Evans</i></p>
<p>Applying Ben Evans’ framework, it becomes clear that while the vertical search players have become too large and need curation, the curation feeds have become too long to browse and require search and structured data. The solution is better search and better curation, all wrapped in a better business model — a combination I call <b>boutique search engines</b>.</p>
<p>Searchable, curated interfaces will help us move away from ephemeral, time-bound feeds into contextual, high-signal, trustworthy knowledge spaces. Because searchable interfaces are densely linked, an explorer can follow multiple trails through the content, rather than being dumped into a “most recent” feed.</p>
<p>With such a tight relationship between curation and search, the real question is not <em>whether</em> you need curation or search but <em>at what point</em> it comes in, and <em>how</em>:</p>
<ul>
<li>Spotify doesn’t curate what songs make it to their platform. Instead, it takes the entire universe of music and finds endless ways to discover and search across its library, including a mix of manual curation (via playlists curated by its in-house team of curators and its users) and algorithms (like Discover Weekly).</li>
<li>Wirecutter does not review every product. It manually curates the top products, and then uses search and other discovery tools to help you find what you need.</li>
<li><a href="https://thingtesting.com/">Thingtesting</a> is not automatically scraping all Consumer Packaged Goods brands on the Internet. Someone from its team or community went out of their way to add a brand to the database.</li>
<li>If you’re searching the <a href="https://beondeck.com/">On Deck</a> member database, you know everyone has applied, been vetted, and paid a fee to participate in the program.</li>
<li>If you’re reading through transcripts on <a href="https://www.tegus.co/">Tegus</a>, you know the content comes from experts that have been handpicked by its team.</li>
</ul>
<p>Across all these examples, the value is in <i>what they exclude</i> as much as <i>what they include</i>. The friction in the supply side is what generates the signal.</p>
<p>On top of the signals, these businesses have built strong search engines. OnDeck, for instance, has built an opinionated graph that allows you to discover talent in unique ways. For example, you can filter by people with “software engineering” skills whose current status is “open to new ideas.” As a founder looking for engineering talent, I’ll take this curated dataset over LinkedIn’s any day.</p>
<figure><a href="https://future.com/wp-content/uploads/2022/04/Boutique-Search-LinkedIn-Search.png"></a></figure>
<p>Unlike vertical search aggregators, <b>boutique search engines feel less like the Yellow Pages and more like texting your friends to ask for a recommendation.</b> They have constrained supply, which is the foundation for their biggest moat: <b>trust</b>. Importantly, boutique search engines also introduce new business models that don’t rely on advertising.</p>
<figure><a href="https://future.com/wp-content/uploads/2022/04/Boutique-Search-Table.png"></a></figure>
<h2>Questions remain, because search is hard </h2>
<p>Building boutique search engines requires countless nuanced product choices. Getting the right mix of curation, search, algorithms, and business model will likely prove massively difficult and massively valuable. Here’s a non-exhaustive list of questions we’re pondering as we build <a href="https://beta.startupy.world/">Startupy</a>, our attempt at a boutique search engine for qualitative insights.</p>
<h3>If the value proposition is signal over noise, how do you scale the signal? </h3>
<p>Over and over again, curation sites fall into an existential trap. They start with high-quality, curated recommendations. As they grow, they scale with crowdsourcing, often filling the gap with scraping. Over time, the content goes from great to good. At that point, vertical search aggregators like Yelp offer more utility. Yahoo, for example, became too big to be browsed, lost its signaling power, and reached the point where Google was better. The line between curator, compiler, and cataloguer is thin and there is a natural <a href="https://www.eugenewei.com/blog/2018/5/21/invisible-asymptotes">invisible asymptote</a> — <strong>diminishing returns on more data over time. </strong></p>
<h3>What is the business model for this new wave of search engines? </h3>
<p>On the surface, vertical search engines are simple — content is the supply, and eyeballs are the demand. But the last wave of vertical search engines was built atop ad-based business models, which made things trickier. In an ad-driven marketplace, the eyeballs are on the supply side. Their attention is what the demand side — advertisers — want. The downside to this ad-driven model is that advertisers and content producers are competing for the same attention, which is why these sites end up feeling like marketing blogs. </p>
<p>This is why subscriptions present an opportunity. Subscription simplifies the network effects to two sides: content as supply, a paying audience as demand. But subscription itself is not a panacea, especially when the use cases are not frequent enough. How often are you needing to find a freelancer? An investor? If use cases are not frequent enough, the utility of a search engine won’t translate to a sustainable business model and you’ll have to come up with your own flavor of “come for the search, stay for something else.”</p>
<p>Moreover, as my friend Joey points out in <a href="https://ff.mirror.xyz/Rf3fOSJmT4mD1AobLmZ0wM6lmea5gEUDqfcW91Iv6XI">this article</a>, any product where you’re spending a lot of time using it in incognito mode has a pretty big UX problem. How many <i>New York Times</i> accounts did you create before finally giving in to the paywall? With today’s subscription models, customers have no real incentive to help the platform grow. <b>Nascent token-based business models show early signs of promise.</b> By giving ownership to stakeholders and allowing subscribers to benefit from future upside, startups can overcome the cold start problem. </p>
<p>Though appealing, a playbook for tokenized business models has not yet emerged. I suspect this will change in the coming years, and I’m excited to improve my understanding of the subject.</p>
<h3>Who curates the curators? </h3>
<p>Platforms like Twitter delegate this responsibility to their users, who have to go through a long and arduous process of following a huge number of people to ultimately arrive at a self-curated timeline that mimics their interests. Some centralize their curation — at OnDeck, you trust that they are doing the work of selecting who can join their network. Yet others stay away from curation in favor of more traditional crowdsourcing. The spectrum is wide. </p>
<h3>How do you find the search engine in the first place? </h3>
<p>I started this piece arguing that Google needs to be unbundled. That’s a catchy headline but, in truth, I believe that until you can build habitual recall into your product, Google will be an important part of how your engine gets discovered in the first place. Zillow and Airbnb are examples of search companies that enjoy a good amount of direct traffic, but SEO was a big part of their early strategy. By being among the first to create the definitive page for a home, they benefited from an SEO land grab that has been hard to displace since.</p>
<p>We are far from achieving the grand vision of the Internet. The project of human knowledge, as it stands today, is a vast ocean of ephemeral and fragmented information and ideas, with the best sources near-impossible to find. <strong>We need more interfaces with a point of view on what information is missing, how it needs to be organized, and at what point of the value chain the curation has to happen.</strong></p>
 
 <p>
 Posted <time>May 2, 2022</time>
 </p>
 	 <ul>
 <li>
 <p><a href="https://future.com/author/sari-azout/">
 	 	 </a>
 </p>
 
 </li>
 </ul>
 
 
 
 <div>
 <p>Views expressed in “posts” (including articles, podcasts, videos, and social media) are those of the individuals quoted therein and are not necessarily the views of AH Capital Management, L.L.C. (“a16z”) or its respective affiliates. Certain information contained in here has been obtained from third-party sources, including from portfolio companies of funds managed by a16z. While taken from sources believed to be reliable, a16z has not independently verified such information and makes no representations about the enduring accuracy of the information or its appropriateness for a given situation.</p>
<p>This content is provided for informational purposes only, and should not be relied upon as legal, business, investment, or tax advice. You should consult your own advisers as to those matters. References to any securities or digital assets are for illustrative purposes only, and do not constitute an investment recommendation or offer to provide investment advisory services. Furthermore, this content is not directed at nor intended for use by any investors or prospective investors, and may not under any circumstances be relied upon when making a decision to invest in any fund managed by a16z. (An offering to invest in an a16z fund will be made only by the private placement memorandum, subscription agreement, and other relevant documentation of any such fund and should be read in their entirety.) Any investments or portfolio companies mentioned, referred to, or described are not representative of all investments in vehicles managed by a16z, and there can be no assurance that the investments will be profitable or that other investments made in the future will have similar characteristics or results. A list of investments made by funds managed by Andreessen Horowitz (excluding investments for which the issuer has not provided permission for a16z to disclose publicly as well as unannounced investments in publicly traded digital assets) is available at <a href="https://a16z.com/investments/">https://a16z.com/investments/</a>.</p>
<p>Charts and graphs provided within are for informational purposes solely and should not be relied upon when making any investment decision. Past performance is not indicative of future results. The content speaks only as of the date indicated. Any projections, estimates, forecasts, targets, prospects, and/or opinions expressed in these materials are subject to change without notice and may differ or be contrary to opinions expressed by others. Please see <a href="https://a16z.com/disclosures">https://a16z.com/disclosures</a> for additional important information.</p>
 </div>
 </article>
 </div>||||I|||| * Tech Trends
 + Crypto & Web3
 + Marketplaces
 + Creator Economy
 + SaaS
 + Fintech
 + Infrastructure
 + Social
 + Games
 + Bio & Science
 + Data
 + See all Tech Trends
 See all Tech Trends
 * Company Building
 + Capital & Fundraising
 + Product, Design & Engineering
 + Metrics & KPIs
 + Sales & Go To Market
 + Growth & Marketing
 + Innovation & Creativity
 + Leadership & Management
 + Work Life
 + See all Company Building
 See all Company Building
 * Special Reports
 Marketplace 100
 Data50
 * Podcasts
 Top Suggestions
 * ai
 * community
 * nft
 * info diet
 * marketing platform
 * metaverse
 * dao
 * internet
 * gpt
 * data
 1. Choosing Your North Star Metric

 Lenny Rachitsky

 2. The Marketplace 100: 2022

 Olivia Moore and Brandon Barros

 3. Data50: The World’s Top 50 Data Startups

 Jennifer Li, Sarah Wang, and Jamie Sullivan

 4. A Guide to DeSci, the Latest Web3 Movement

 Sarah Hamburg

 5. Go-to-Market in Web3: New Mindsets, Tactics, Metrics

 Maggie Hsu

 See More Results
 Creator Economy

 The Future of Search Is Boutique

 Sari Azout
 share
 * Twitter
 * LinkedIn
 * Facebook
 * Hacker News
 * WhatsApp
 * Flipboard
 * Reddit
 * Copy Link
 * Send Email

 This is an edited version of a post that originally ran here .

 For most queries, Google search is pretty underwhelming these days. Google is great at answering questions with an objective answer, like “# of billionaires in the world” or “What is the population of Iceland?” It’s pretty bad at answering questions that require judgment and context like “What do NFT collectors think about NFTs?”

 The evidence is everywhere. These days, I find myself suppressing the garbage Internet by searching on Google for “Substack + future of learning” to find the best takes on education. We hack Twitter with the “ what is the best ” posts over and over again. When I’m researching a new product, I type “X item reddit” into Google. I find enormous value in small, niche, often forgotten sites like Spaghetti Directory .

 There’s an emergence of tools like Notion, Airtable, and Readwise where people are aggregating content and resources, reviving the curated web . But at the moment these are mostly solo affairs — hidden in private or semi-private corners of the Internet, fragmented, poorly indexed, and unavailable for public use. We haven’t figured out how to make them multiplayer. In cases where we’ve made them public and collaborative — here is a great example — these projects are often short-lived and poorly maintained.

 The stated mission of a company worth almost two trillion dollars is to “organize the world’s information” and yet the Internet remains poorly organized. Or, stated differently, in a world of infinite information, it’s no longer enough to organize the world’s information. It becomes important to organize the world’s trustworthy information.

 How did we get here?

 It’s hard to believe, but one of Google’s main problems, once it got going, was that there just wasn’t much to see online. Having a great search engine is useless if somebody types in “how to grow an herb garden” and the answer doesn’t exist online. With the advent of Google AdWords, it became profitable to put out low-quality content that passed as informative and filled Google’s search engine results. The end result is that the websites at the top of Google are not necessarily the highest-quality ones, but rather the ones that put the most effort into SEO. What started as a well-intentioned way to organize the world’s information has turned into a business focusing most of its resources on monetizing clicks to support advertisers, rather than focusing on delivering trusted search results for people.

 The problem, now so drastically different from a decade ago, is not what to read/buy/eat/watch/etc., but figuring out the best thing to read/buy/eat/watch/etc. with my limited time and attention.

 Audacious teams, like DuckDuckGo and Neeva , are trying to compete with Google head-on by building massive horizontal search engines. Rather than crawling and indexing things their own way, they sit on top of existing data sources and position themselves as privacy-focused alternatives to Google. But protection of privacy is not a compelling enough reason to leave Google. For the vast majority of people, allowing them to “control their own data” is not a selling point, especially if it requires paying for something they’re used to getting for free.

 I believe the opportunity in search is not to attack Google head-on with a massive, one-size-fits-all horizontal aggregator, but instead to build boutique search engines that index, curate, and organize things in new ways.

 Vertical search aggregators

 Google is a great example of how the internet enabled scale and speed: every page on the web returned in an instant. But, increasingly, we’re seeing that this scale is at odds with a fundamental human need: relevance . Someone who wants to find the best freelance designer, or the best sushi restaurant, or the best NFT to buy will not find the answer on Google.

 There is no search architecture that will work universally across all categories. It’s hard to imagine wanting to use the same UX to search for recipes as to search for freelancers. Whereas Google’s product begins and ends with a search bar, trading off functionality for simplicity, vertical search players like Yelp, Expedia, Zillow, and Behance emerged to fill functionality and relevancy gaps using structured data specific to their industries. With strong opinions on how to organize information, reflected in their choice of filters, vertical search aggregators have distinct advantages that horizontal software can never achieve.

 But here, too, relevance depends on the sociology of the current moment. For example, on Behance , the online creative community, school and location are featured prominently as filters — implying that where you live and where you went to school is an important indicator of the quality of your design portfolio. In a world where talent is being decoupled from credentialism and geography, those filters are losing relevance.

 As signals evolve, new ways of indexing and surface areas for innovation emerge. If Behance were designed today, I’d argue neither “location” nor “school” would be filters.

 On Yelp, a search for “electricians in Miami” begins with a page titled “The Best 10 Electricians in Miami, FL” with text underneath that indicates the majority of these are sponsored results.

 When you monetize via ads, curation takes a backseat to featuring advertisers because there is just less digital real estate available to curate your own recommendations. So these platforms end up making ethically dubious design choices that generate massive trust gaps .

 Additionally, across vertical search aggregators like Yelp, Zillow, LinkedIn, and Behance, anyone can have a profile. A combination of irrelevant filters, ad-based business models, and unconstrained supply has overwhelmed consumers and made it hard to find signals in these platforms.

 Vertical search aggregators work when you know exactly what you want. But knowing what you want isn’t usually the starting point, which creates an opportunity to help the overwhelmed consumer with better discovery and curation along the funnel.

 Curators, curators

 It has become popular to say we live in the information age, and we need “curation” to help us sort through the mess. But thus far, the conversation around curation has been too focused on the content and not enough on the structure . We seem to have accepted the job of the curator as providing a product review, a list of links, or a song recommendation — all inside linear structures and chronological feeds designed to surface the ideas of the last 24 hours, not to accumulate and surface knowledge as needed.

 A daily email with the top five Alibaba products feels fun and gimmicky as a side project, but it doesn’t help when you’re trying to find the best crib for your baby. Inevitably, you’ll want a way to search through a curator’s archives.

 Curation, when thought of in the context of sharing bite-sized, isolated bits in feed-like architectures, is predominantly about entertainment, not utility. It’s not wrong to say there is a market for this kind of curation. What people miss, though, is that this market is already captured by Twitter, Facebook, and TikTok.

 These entertainment giants offer curation that demands our attention, but they don’t offer curation on demand. The opportunity is in moving curated content feeds away from their never-ending-now orientation and toward more goal-oriented interfaces. People should be able to find whatever content they want on their terms and not be beholden to when the curator decides to publish.

 Boutique search engines are next-level curation

 All curation grows until it requires search, and all search grows until it requires curation.

 —Ben Evans

 Applying Ben Evans’ framework, it becomes clear that while the vertical search players have become too large and need curation, the curation feeds have become too long to browse and require search and structured data. The solution is better search and better curation, all wrapped in a better business model — a combination I call boutique search engines .

 Searchable, curated interfaces will help us move away from ephemeral, time-bound feeds into contextual, high-signal, trustworthy knowledge spaces. Because searchable interfaces are densely linked, an explorer can follow multiple trails through the content, rather than being dumped into a “most recent” feed.

 With such a tight relationship between curation and search, the real question is not whether you need curation or search but at what point it comes in, and how :

 * Spotify doesn’t curate what songs make it to their platform. Instead, it takes the entire universe of music and finds endless ways to discover and search across its library, including a mix of manual curation (via playlists curated by its in-house team of curators and its users) and algorithms (like Discover Weekly).
 * Wirecutter does not review every product. It manually curates the top products, and then uses search and other discovery tools to help you find what you need.
 * Thingtesting is not automatically scraping all Consumer Packaged Goods brands on the Internet. Someone from its team or community went out of their way to add a brand to the database.
 * If you’re searching the On Deck member database, you know everyone has applied, been vetted, and paid a fee to participate in the program.
 * If you’re reading through transcripts on Tegus , you know the content comes from experts that have been handpicked by its team.

 Across all these examples, the value is in what they exclude as much as what they include . The friction in the supply side is what generates the signal.

 On top of the signals, these businesses have built strong search engines. OnDeck, for instance, has built an opinionated graph that allows you to discover talent in unique ways. For example, you can filter by people with “software engineering” skills whose current status is “open to new ideas.” As a founder looking for engineering talent, I’ll take this curated dataset over LinkedIn’s any day.

 Unlike vertical search aggregators, boutique search engines feel less like the Yellow Pages and more like texting your friends to ask for a recommendation. They have constrained supply, which is the foundation for their biggest moat: trust . Importantly, boutique search engines also introduce new business models that don’t rely on advertising.

 Questions remain, because search is hard

 Building boutique search engines requires countless nuanced product choices. Getting the right mix of curation, search, algorithms, and business model will likely prove massively difficult and massively valuable. Here’s a non-exhaustive list of questions we’re pondering as we build Startupy , our attempt at a boutique search engine for qualitative insights.

 If the value proposition is signal over noise, how do you scale the signal?

 Over and over again, curation sites fall into an existential trap. They start with high-quality, curated recommendations. As they grow, they scale with crowdsourcing, often filling the gap with scraping. Over time, the content goes from great to good. At that point, vertical search aggregators like Yelp offer more utility. Yahoo, for example, became too big to be browsed, lost its signaling power, and reached the point where Google was better. The line between curator, compiler, and cataloguer is thin and there is a natural invisible asymptote — diminishing returns on more data over time.

 What is the business model for this new wave of search engines?

 On the surface, vertical search engines are simple — content is the supply, and eyeballs are the demand. But the last wave of vertical search engines was built atop ad-based business models, which made things trickier. In an ad-driven marketplace, the eyeballs are on the supply side. Their attention is what the demand side — advertisers — want. The downside to this ad-driven model is that advertisers and content producers are competing for the same attention, which is why these sites end up feeling like marketing blogs.

 This is why subscriptions present an opportunity. Subscription simplifies the network effects to two sides: content as supply, a paying audience as demand. But subscription itself is not a panacea, especially when the use cases are not frequent enough. How often are you needing to find a freelancer? An investor? If use cases are not frequent enough, the utility of a search engine won’t translate to a sustainable business model and you’ll have to come up with your own flavor of “come for the search, stay for something else.”

 Moreover, as my friend Joey points out in this article , any product where you’re spending a lot of time using it in incognito mode has a pretty big UX problem. How many New York Times accounts did you create before finally giving in to the paywall? With today’s subscription models, customers have no real incentive to help the platform grow. Nascent token-based business models show early signs of promise. By giving ownership to stakeholders and allowing subscribers to benefit from future upside, startups can overcome the cold start problem.

 Though appealing, a playbook for tokenized business models has not yet emerged. I suspect this will change in the coming years, and I’m excited to improve my understanding of the subject.

 Who curates the curators?

 Platforms like Twitter delegate this responsibility to their users, who have to go through a long and arduous process of following a huge number of people to ultimately arrive at a self-curated timeline that mimics their interests. Some centralize their curation — at OnDeck, you trust that they are doing the work of selecting who can join their network. Yet others stay away from curation in favor of more traditional crowdsourcing. The spectrum is wide.

 How do you find the search engine in the first place?

 I started this piece arguing that Google needs to be unbundled. That’s a catchy headline but, in truth, I believe that until you can build habitual recall into your product, Google will be an important part of how your engine gets discovered in the first place. Zillow and Airbnb are examples of search companies that enjoy a good amount of direct traffic, but SEO was a big part of their early strategy. By being among the first to create the definitive page for a home, they benefited from an SEO land grab that has been hard to displace since.

 We are far from achieving the grand vision of the Internet. The project of human knowledge, as it stands today, is a vast ocean of ephemeral and fragmented information and ideas, with the best sources near-impossible to find. We need more interfaces with a point of view on what information is missing, how it needs to be organized, and at what point of the value chain the curation has to happen.

 Posted May 2, 2022

 * Sari Azout is a founder of startupy.world, a search engine for tech and culture insights. She was previously a partner at Level Ventures, and occasionally publishes her writing at sariazout.substack.com .

 Follow Twitter Website
 Related Articles

 Why Developers Are Building So Many Side Projects

 Ben Stokes

 A Practical Guide to NFT Memberships for Creators

 Peter Yang

 The Overlooked Levels of the Creator Economy

 Dan Runcie

 What’s Next for Creator Platforms? Learning to Sell

 Nathan Baschez

 The Music Industry Is Built on Artists, but Shuns Creators

 Dave Edwards

 Views expressed in “posts” (including articles, podcasts, videos, and social media) are those of the individuals quoted therein and are not necessarily the views of AH Capital Management, L.L.C. (“a16z”) or its respective affiliates. Certain information contained in here has been obtained from third-party sources, including from portfolio companies of funds managed by a16z. While taken from sources believed to be reliable, a16z has not independently verified such information and makes no representations about the enduring accuracy of the information or its appropriateness for a given situation.

 This content is provided for informational purposes only, and should not be relied upon as legal, business, investment, or tax advice. You should consult your own advisers as to those matters. References to any securities or digital assets are for illustrative purposes only, and do not constitute an investment recommendation or offer to provide investment advisory services. Furthermore, this content is not directed at nor intended for use by any investors or prospective investors, and may not under any circumstances be relied upon when making a decision to invest in any fund managed by a16z. (An offering to invest in an a16z fund will be made only by the private placement memorandum, subscription agreement, and other relevant documentation of any such fund and should be read in their entirety.) Any investments or portfolio companies mentioned, referred to, or described are not representative of all investments in vehicles managed by a16z, and there can be no assurance that the investments will be profitable or that other investments made in the future will have similar characteristics or results. A list of investments made by funds managed by Andreessen Horowitz (excluding investments for which the issuer has not provided permission for a16z to disclose publicly as well as unannounced investments in publicly traded digital assets) is available at https://a16z.com/investments/.

 Charts and graphs provided within are for informational purposes solely and should not be relied upon when making any investment decision. Past performance is not indicative of future results. The content speaks only as of the date indicated. Any projections, estimates, forecasts, targets, prospects, and/or opinions expressed in these materials are subject to change without notice and may differ or be contrary to opinions expressed by others. Please see https://a16z.com/disclosures for additional important information.

 Popular

 Choosing Your North Star Metric

 Lenny Rachitsky

 The Marketplace 100: 2022

 Olivia Moore and Brandon Barros

 Data50: The World’s Top 50 Data Startups

 Jennifer Li, Sarah Wang, and Jamie Sullivan
 * Terms of Use & Privacy
 * About a16z
 * RSS
 © 2023 Future. All Rights Reserved.
 * 
 * 
 * 
This is an edited version of a post that originally ran   here  . 
For most queries, Google search is pretty underwhelming these days. Google is great at answering questions with an objective answer, like “# of billionaires in the world” or “What is the population of Iceland?” It’s pretty bad at answering questions that require judgment and context like “What do NFT collectors think about NFTs?”
The evidence is everywhere. These days, I find myself suppressing the garbage Internet by searching on Google for “Substack + future of learning” to find the best takes on education. We hack Twitter with the “what is the best” posts over and over again. When I’m researching a new product, I type “X item reddit” into Google. I find enormous value in small, niche, often forgotten sites like Spaghetti Directory.
There’s an emergence of tools like Notion, Airtable, and Readwise where people are aggregating content and resources, reviving the curated web. But at the moment these are mostly solo affairs — hidden in private or semi-private corners of the Internet, fragmented, poorly indexed, and unavailable for public use. We haven’t figured out how to make them multiplayer. In cases where we’ve made them public and collaborative — here is a great example — these projects are often short-lived and poorly maintained.
The stated mission of a company worth almost two trillion dollars is to “organize the world’s information” and yet the Internet remains poorly organized. Or, stated differently, in a world of infinite information, it’s no longer enough to organize the world’s information. It becomes important to organize the world’s   trustworthy   information. 
How did we get here?
It’s hard to believe, but one of Google’s main problems, once it got going, was that there just wasn’t much to see online. Having a great search engine is useless if somebody types in “how to grow an herb garden” and the answer doesn’t exist online. With the advent of Google AdWords, it became profitable to put out low-quality content that passed as informative and filled Google’s search engine results. The end result is that the websites at the top of Google are not necessarily the highest-quality ones, but rather the ones that put the most effort into SEO. What started as a well-intentioned way to organize the world’s information has turned into a business focusing most of its resources on monetizing clicks to support advertisers, rather than focusing on delivering trusted search results for people.
The problem, now so drastically different from a decade ago, is not what to read/buy/eat/watch/etc., but figuring out the   best thing   to read/buy/eat/watch/etc. with my limited time and attention. 
Audacious teams, like DuckDuckGo and Neeva, are trying to compete with Google head-on by building massive horizontal search engines. Rather than crawling and indexing things their own way, they sit on top of existing data sources and position themselves as privacy-focused alternatives to Google. But protection of privacy is not a compelling enough reason to leave Google. For the vast majority of people, allowing them to “control their own data” is not a selling point, especially if it requires paying for something they’re used to getting for free.
I believe the opportunity in search is not to attack Google head-on with a massive, one-size-fits-all horizontal aggregator, but instead to build boutique search engines that index, curate, and organize things in new ways. 
   
Vertical search aggregators
Google is a great example of how the internet enabled scale and speed: every page on the web returned in an instant. But, increasingly, we’re seeing that this scale is at odds with a fundamental human need: relevance. Someone who wants to find the best freelance designer, or the best sushi restaurant, or the best NFT to buy will not find the answer on Google.
There is no search architecture that will work universally across all categories. It’s hard to imagine wanting to use the same UX to search for recipes as to search for freelancers. Whereas Google’s product begins and ends with a search bar, trading off functionality for simplicity, vertical search players like Yelp, Expedia, Zillow, and Behance emerged to fill functionality and relevancy gaps using structured data specific to their industries. With strong opinions on how to organize information, reflected in their choice of filters, vertical search aggregators have distinct advantages that horizontal software can never achieve.
But here, too, relevance depends on the sociology of the current moment. For example, on Behance, the online creative community, school and location are featured prominently as filters — implying that where you live and where you went to school is an important indicator of the quality of your design portfolio. In a world where talent is being decoupled from credentialism and geography, those filters are losing relevance. 
As signals evolve, new ways of indexing and surface areas for innovation emerge. If Behance were designed today, I’d argue neither “location” nor “school” would be filters.
   
On Yelp, a search for “electricians in Miami” begins with a page titled “The Best 10 Electricians in Miami, FL” with text underneath that indicates the majority of these are sponsored results. 
   
 When you monetize via ads, curation takes a backseat to featuring advertisers because there is just less digital real estate available to curate your own recommendations. So these platforms end up making ethically dubious design choices that generate massive trust gaps.
Additionally, across vertical search aggregators like Yelp, Zillow, LinkedIn, and Behance, anyone can have a profile. A combination of irrelevant filters, ad-based business models, and unconstrained supply has overwhelmed consumers and made it hard to find signals in these platforms. 
Vertical search aggregators work when you know exactly what you want. But knowing what you want isn’t usually the starting point, which creates an opportunity to help the overwhelmed consumer with better discovery and curation along the funnel.
Curators, curators
It has become popular to say we live in the information age, and we need “curation” to help us sort through the mess. But thus far, the conversation around curation has been too focused on the content and not enough on the structure. We seem to have accepted the job of the curator as providing a product review, a list of links, or a song recommendation — all inside linear structures and chronological feeds designed to surface the ideas of the last 24 hours, not to accumulate and surface knowledge as needed.
A daily email with the top five Alibaba products feels fun and gimmicky as a side project, but it doesn’t help when you’re trying to find the best crib for your baby. Inevitably, you’ll want a way to search through a curator’s archives.
Curation, when thought of in the context of sharing bite-sized, isolated bits in feed-like architectures, is predominantly about entertainment, not utility. It’s not wrong to say there is a market for this kind of curation. What people miss, though, is that this market is already captured by Twitter, Facebook, and TikTok.
These entertainment giants offer curation that demands our attention, but they don’t offer curation on demand. The opportunity is in moving curated content feeds away from their never-ending-now orientation and toward more goal-oriented interfaces. People should be able to find whatever content they want on their terms and not be beholden to when the curator decides to publish.
Boutique search engines are next-level curation
 All curation grows until it requires search, and all search grows until it requires curation.  
 —Ben Evans 
Applying Ben Evans’ framework, it becomes clear that while the vertical search players have become too large and need curation, the curation feeds have become too long to browse and require search and structured data. The solution is better search and better curation, all wrapped in a better business model — a combination I call boutique search engines.
Searchable, curated interfaces will help us move away from ephemeral, time-bound feeds into contextual, high-signal, trustworthy knowledge spaces. Because searchable interfaces are densely linked, an explorer can follow multiple trails through the content, rather than being dumped into a “most recent” feed.
With such a tight relationship between curation and search, the real question is not whether you need curation or search but at what point it comes in, and how:
Spotify doesn’t curate what songs make it to their platform. Instead, it takes the entire universe of music and finds endless ways to discover and search across its library, including a mix of manual curation (via playlists curated by its in-house team of curators and its users) and algorithms (like Discover Weekly).
Wirecutter does not review every product. It manually curates the top products, and then uses search and other discovery tools to help you find what you need.
 Thingtesting is not automatically scraping all Consumer Packaged Goods brands on the Internet. Someone from its team or community went out of their way to add a brand to the database.
If you’re searching the On Deck member database, you know everyone has applied, been vetted, and paid a fee to participate in the program.
If you’re reading through transcripts on Tegus, you know the content comes from experts that have been handpicked by its team.
Across all these examples, the value is in what they exclude as much as what they include. The friction in the supply side is what generates the signal.
On top of the signals, these businesses have built strong search engines. OnDeck, for instance, has built an opinionated graph that allows you to discover talent in unique ways. For example, you can filter by people with “software engineering” skills whose current status is “open to new ideas.” As a founder looking for engineering talent, I’ll take this curated dataset over LinkedIn’s any day.
   
Unlike vertical search aggregators, boutique search engines feel less like the Yellow Pages and more like texting your friends to ask for a recommendation. They have constrained supply, which is the foundation for their biggest moat: trust. Importantly, boutique search engines also introduce new business models that don’t rely on advertising.
   
Questions remain, because search is hard 
Building boutique search engines requires countless nuanced product choices. Getting the right mix of curation, search, algorithms, and business model will likely prove massively difficult and massively valuable. Here’s a non-exhaustive list of questions we’re pondering as we build Startupy, our attempt at a boutique search engine for qualitative insights.
If the value proposition is signal over noise, how do you scale the signal? 
Over and over again, curation sites fall into an existential trap. They start with high-quality, curated recommendations. As they grow, they scale with crowdsourcing, often filling the gap with scraping. Over time, the content goes from great to good. At that point, vertical search aggregators like Yelp offer more utility. Yahoo, for example, became too big to be browsed, lost its signaling power, and reached the point where Google was better. The line between curator, compiler, and cataloguer is thin and there is a natural invisible asymptote — diminishing returns on more data over time.  
What is the business model for this new wave of search engines? 
On the surface, vertical search engines are simple — content is the supply, and eyeballs are the demand. But the last wave of vertical search engines was built atop ad-based business models, which made things trickier. In an ad-driven marketplace, the eyeballs are on the supply side. Their attention is what the demand side — advertisers — want. The downside to this ad-driven model is that advertisers and content producers are competing for the same attention, which is why these sites end up feeling like marketing blogs. 
This is why subscriptions present an opportunity. Subscription simplifies the network effects to two sides: content as supply, a paying audience as demand. But subscription itself is not a panacea, especially when the use cases are not frequent enough. How often are you needing to find a freelancer? An investor? If use cases are not frequent enough, the utility of a search engine won’t translate to a sustainable business model and you’ll have to come up with your own flavor of “come for the search, stay for something else.”
Moreover, as my friend Joey points out in this article, any product where you’re spending a lot of time using it in incognito mode has a pretty big UX problem. How many New York Times accounts did you create before finally giving in to the paywall? With today’s subscription models, customers have no real incentive to help the platform grow. Nascent token-based business models show early signs of promise. By giving ownership to stakeholders and allowing subscribers to benefit from future upside, startups can overcome the cold start problem. 
Though appealing, a playbook for tokenized business models has not yet emerged. I suspect this will change in the coming years, and I’m excited to improve my understanding of the subject.
Who curates the curators? 
Platforms like Twitter delegate this responsibility to their users, who have to go through a long and arduous process of following a huge number of people to ultimately arrive at a self-curated timeline that mimics their interests. Some centralize their curation — at OnDeck, you trust that they are doing the work of selecting who can join their network. Yet others stay away from curation in favor of more traditional crowdsourcing. The spectrum is wide. 
How do you find the search engine in the first place? 
I started this piece arguing that Google needs to be unbundled. That’s a catchy headline but, in truth, I believe that until you can build habitual recall into your product, Google will be an important part of how your engine gets discovered in the first place. Zillow and Airbnb are examples of search companies that enjoy a good amount of direct traffic, but SEO was a big part of their early strategy. By being among the first to create the definitive page for a home, they benefited from an SEO land grab that has been hard to displace since.
We are far from achieving the grand vision of the Internet. The project of human knowledge, as it stands today, is a vast ocean of ephemeral and fragmented information and ideas, with the best sources near-impossible to find. We need more interfaces with a point of view on what information is missing, how it needs to be organized, and at what point of the value chain the curation has to happen. 
 
 
 Posted May 2, 2022
 
 	 
 
  
 	 	 
 
 
 
 
 
 
 
 
 Views expressed in “posts” (including articles, podcasts, videos, and social media) are those of the individuals quoted therein and are not necessarily the views of AH Capital Management, L.L.C. (“a16z”) or its respective affiliates. Certain information contained in here has been obtained from third-party sources, including from portfolio companies of funds managed by a16z. While taken from sources believed to be reliable, a16z has not independently verified such information and makes no representations about the enduring accuracy of the information or its appropriateness for a given situation.
This content is provided for informational purposes only, and should not be relied upon as legal, business, investment, or tax advice. You should consult your own advisers as to those matters. References to any securities or digital assets are for illustrative purposes only, and do not constitute an investment recommendation or offer to provide investment advisory services. Furthermore, this content is not directed at nor intended for use by any investors or prospective investors, and may not under any circumstances be relied upon when making a decision to invest in any fund managed by a16z. (An offering to invest in an a16z fund will be made only by the private placement memorandum, subscription agreement, and other relevant documentation of any such fund and should be read in their entirety.) Any investments or portfolio companies mentioned, referred to, or described are not representative of all investments in vehicles managed by a16z, and there can be no assurance that the investments will be profitable or that other investments made in the future will have similar characteristics or results. A list of investments made by funds managed by Andreessen Horowitz (excluding investments for which the issuer has not provided permission for a16z to disclose publicly as well as unannounced investments in publicly traded digital assets) is available at https://a16z.com/investments/.
Charts and graphs provided within are for informational purposes solely and should not be relied upon when making any investment decision. Past performance is not indicative of future results. The content speaks only as of the date indicated. Any projections, estimates, forecasts, targets, prospects, and/or opinions expressed in these materials are subject to change without notice and may differ or be contrary to opinions expressed by others. Please see https://a16z.com/disclosures for additional important information.
The Future of Search Is Boutique
https://future.com/the-future-of-search-is-boutique/
https://kagi.com/faq
What is Kagi?
2022-01-01
None
<div><div> <div> <h2> <a href="#What-is-Kagi">What is Kagi?</a> </h2> <div> <p>Kagi is a company created with the mission to humanize the web. Our goal is amplify the web of human knowledge, creativity and self-expression and provide the user tools to fight against the web of greed, ad-tech and user tracking.</p> <p>Kagi is currently Kagi Search, a fast, private search engine, and <a href="https://browser.kagi.com/">Orion Browser</a>, a fast, zero-telemetry browser. We plan to release an email service as well, creating a platform to experience the web in a private, secure and unbiased way.</p> </div> </div> <div> <h2> <a href="#pronounce">How do you pronounce Kagi?</a> </h2> <p>Kagi is pronounced as Kah-gee. If still not sure, listen to it <a href="https://www.youtube.com/watch?v=ig4VTr0rt4Q">here</a>.</p> </div> <div> <h2> <a href="#What-is-Kagi-Search">What is Kagi Search?</a> </h2> <div> <p>Kagi Search is a quick, user-centric, 100% privacy-respecting search engine with results augmented by non-commercial indexes and personalized searches. It has a clean, high-performance user interface with only the most relevant results and no telemetry, no ads and no collection of private information. Read more about its <a href="https://blog.kagi.com/kagi-features">unique features</a>.</p> <p>We hope it will help users amplify their abilities and make them more creative and smarter as well as more productive. </p> </div> </div> <div> <h2> <a href="#public-beta">What does it mean that youâre in public beta?</a> </h2> <p> While we're in public beta, we don't feel like our vision is fully realized and ready for prime time. But we also don't want to build something in isolation. We want to build it with you, our users. During this time we plan to fix bugs, introduce more core feature and improve stability of the product. You can read more about this in our public beta <a href="https://blog.kagi.com/kagi-orion-public-beta">announcement post</a>. </p> </div> <div> <h2> <a href="#cost">What are the costs to deliver a superior ad free search product?</a> </h2> <div> <p>All search engines have: Search costs, Development costs and Administrative costs. Most search engines cover this by advertising, tracking, and selling your data. We attempt to cover it with a simple subscription of USD $10 a month.</p> <p>Our proposed price is dictated by the fact that search itself has a non-zero cost. In fact, it costs us about $1 to process 80 searches (wherever in the world you search from). So a user searching 8 times a day would perform about 240 searches a month, costing us $3 in search cost. But an average Kagi beta user is actually searching about 30 times a day. At USD $10/month, the price does not even cover our cost for average use, and we are basically betting that in the future we will be able to improve the product to reduce average number of searches needed to find something and reduce search cost by optimizations, to not only cover search cost but make room to pay for other costs as well. </p> <p>Our goal is to find the minimum price at which we can sustain the business. USD $10/mo is a price-point that we lowered as much as we could hoping it will allow us to have a chance of hitting sustainability at some point. If it turns out that we have more room we will decrease it. But it can also be that we may need to increase it.</p> <p>The cost of running free accounts is also substantial and it also has to be covered by paying customers, which makes the above math even harder. Tips are a way to help offset some of that cost.</p> <p>All of the above has to be set up in a way that after subtracting search cost we are still having left over to pay for the development costs (salaries) and the administrative costs. </p> </div> </div> <div> <h2> <a href="#future">How will Kagi ensure its principles survive the future?</a> </h2> <div> <p>Kagi has been driven by a vision of a more humane web and funded completely out of founder's pocket. You can read more about the financials in our <a href="https://blog.kagi.com/status-update-first-three-months#financials">first update</a> after the public beta launch.</p><p>This has allowed Kagi to follow only one agenda and that is the agenda of user-centric products.</p><p>The only thing that could change this (potentially) is if it suddenly got powerful shareholders that could sway the path (typically VCs). However we are strategically avoding that and planning to <a href="https://blog.kagi.com/status-update-first-three-months#invest">offer our users to be investors In Kagi instead</a>.</p><p>This in our opinion will further extends the guarantee the company will stay on its track.</p> </div> </div> <div> <h2> <a href="#How-do-I-set-Kagi-as-my-default-search-engine">How do I set Kagi as my default search engine?</a> </h2> <div> <p>We have a browser extension for all major browsers. The extensions sets Kagi as the default search engine and allows you to search with Kagi even in a private window.</p> <ul> <li><b>Chromium</b> based browsers (Chrome, Edge, Brave, Opera, Vivaldi...) <a href="https://chrome.google.com/webstore/detail/cdglnehniifkbagbbombnjghhcihifij">download here</a>.</li> <li><b>Firefox</b> based browsers <a href="https://addons.mozilla.org/en-US/firefox/addon/kagi-search-for-firefox/">download here</a>. If you use Firefox on Android, check this <a href="https://blog.mozilla.org/addons/2020/09/29/expanded-extension-support-in-firefox-for-android-nightly/">guide</a> (it doesn't set Kagi as the default search engine, but it allows search in private mode to work normally as expected).</li> <li><b>Safari</b> macOS <a href="https://apps.apple.com/app/kagi-search-for-safari/id1622835804">download here</a>, Safari iOS <a href="https://apps.apple.com/app/kagi-search-for-safari/id1607766153">download here</a>. Some users use <a href="https://apps.apple.com/us/app/xsearch-for-safari/id1579902068">xSearch</a> and <a href="https://apps.apple.com/us/app/hyperweb/id1581824571">HyperWeb</a> extensions for Safari which also support Kagi.</li> </ul> <p>If you use macOS/iOS consider trying <a href="https://browser.kagi.com/">Orion browser</a>, made by same folks who made Kagi Search - us! Orion is a WebKit-based browser for Mac, with native integration with Kagi Search.</p> </div> </div> <div> <h2> <a href="#Why-use-Kagi-instead-of-Google">Why use Kagi instead of Google?</a> </h2> <div> <p>Googleâs search engine has indeed changed the world and the way we access information, and we value this contribution. Weâre grateful to have access to Google's search technology and infrastructure for Kagi. </p> <p>But instead of trying to create a search product for billions of people, we want to develop a refined search experience for sophisticated customers who value high-quality results, privacy and speed.</p> <p>Kagi will allow you to discover well-written articles from lesser-known blogs and use features like Lenses to personalize results. Ad-supported search must avoid this kind of depth and flexibility to stay profitable. Read more about Kagi's <a href="https://blog.kagi.com/kagi-features">unique features</a>, many of which can never be replicated in ad-supported search engine.</p> <p> </p></div> </div> <div> <h2> <a href="#Google-killer">Is Kagi a "Google killer"?</a> </h2> <div> <p>We do not see Kagi as the Google killer. Google's scale and reach is enourmous and Google serves a purpose in the world - it did help enable the modern society to exist, with all its marvels and flaws. Heck, it even enables Kagi to exist!</p> <p>Think of Kagi as a small, premium brand, providing a very different, tailor-made, search experience for people who need and appreciate that.</p> </div> </div> <div> <h2> <a href="#Why-use-Kagi-instead-of-DuckDuckGo">Why use Kagi instead of DuckDuckGo?</a> </h2> <div> <p>DuckDuckGo has shown the world that a privacy-first search engine is possible, and we respect this contribution. But its innovation has slowed in the past decade. And Ad-supported business models will always force a company to make compromises and balance between serving users and advertisers. At the end, DuckDuckGo's search product is just âgood enoughâ (by our standard, sorry DuckDuckGo!) and has been stagnant years without any ground-breaking feature development. </p> <p>In contrast, Kagi search does not need to compromise on user experience. Everything we do is user-centric. Kagi Search already has many <a href="https://blog.kagi.com/kagi-features">unique features</a>, like personalized results and "Lenses". And because we depend only on our users for revenue, Kagi can and will always offer a much richer search experience for the user. </p> <p> </p></div> </div> <div> <h2>Does Kagi support !bangs? </h2> <p> Yes, Kagi supports all DuckDuckGo-style <a href="https://duckduckgo.com/bang">bangs</a>. </p> </div> <div> <h2> <a href="#Why-pay-for-search">Why pay for a search engine?</a> </h2> <div> <p>You should pay for your search engine to ensure that the incentives of the information provider are aligned with whatâs best for you, not whatâs best for the advertisers.</p> <p>Every âfreeâ search engine out there comes with a hidden cost. Your private details pass through your searches, and search results influenced by advertising spends can influence your thoughts and actions. Businesses that monetize your data owe you more than just a "free" account! Our stand against ad-supported business models has created a 100% ad-free search experience, focused entirely on our users.</p> <p>If you want a search experience tailored to you and your needs, one that could also make the world a slightly better place, then you need to break free from the ruts that advertisers and corporations have guided us into. Investing in Kagi is investing in a more humane web, your online future and the future of your children, if you believe in them growing in a world where they are not bombarded with ads from an early age. </p> <p> </p></div> </div> <div> <h2> <a href="#Why-paid">Why ad-supported search engines do not work for the future of the web?</a> </h2> <div> <p>We will let no other than the founders of Google explain this. The year is 1998, and this is how they describe why it is imperative for search engines to be ad-free:</p> <blockquote>"Currently, the predominant business model for commercial search engines is advertising. The goals of the advertising business model do not always correspond to providing quality search to users. For example, in our prototype search engine one of the top results for cellular phone is "The Effect of Cellular Phone Use Upon Driver Attention", a study which explains in great detail the distractions and risk associated with conversing on a cell phone while driving. This search result came up first because of its high importance as judged by the PageRank algorithm, an approximation of citation importance on the web [Page, 98].<p> It is clear that a search engine which was taking money for showing cellular phone ads would have difficulty justifying the page that our system returned to its paying advertisers. For this type of reason and historical experience with other media [Bagdikian 83], we expect that advertising funded search engines will be inherently biased towards the advertisers and away from the needs of the consumers."</p></blockquote> <p><a href="http://infolab.stanford.edu/pub/papers/google.pdf">The Anatomy of a Large-Scale Hypertextual Web Search Engine</a>, Sergey Brin and Lawrence Page, 1998</p> <p>They were right in recognizing early on that ad-supported search engines will be biased towards the advertisers. What they probably could not foresee is how much damage to the web adopting the very model they originally criticised will ultimately make.</p> </div> </div> <div> <h2> <a href="#Where-are-your-results-coming-from">Where are your results coming from?</a> </h2> <div> <p>We use heuristics and deep learning to understand query intent, select the best information sources, query them directly using APIs, and rank the results. You can think of Kagi as a "search client," working like an email client, connecting to indexes and sources to find relevant results and package them into a superior, secure and privacy-respecting search experience for you.</p> <p>Our searching includes anonymized requests to traditional search indexes like Google and Bing as well as vertical sources like Wikipedia and DeepL or other APIs. We also have our own non-commercial index (Teclis), news index (TinyGem), and an AI for instant answers.</p> <p>Teclis and TinyGem are a result of our crawl through millions of domains, focusing primarily on non-commercial, high-quality content. Our unique results help you discover some of the best content you can possibly find online, sometimes in the quieter places on the web.</p> <p>And of course, we answer quick queries like âHow far is the sun from the earth?â or â10kg in lbsâ with our instant-answer systems that use dozens of sources and APIs, all connected to you quickly yet transparently. </p> </div> </div> <div> <h2> <a href="#How-does-Kagi-achieve-superior-search-quality">How does Kagi achieve superior search quality?</a> </h2> <div> <p>Kagi can surface unique, high-quality, non-commercial content while avoiding results bloated by ads and tracking. Kagi's non-commercial index (Teclis) and non-commercial news index (TinyGem) as well as instant answers (IA) offer high-quality, relevant and authentic results for the user. Clicking "Info" in search results will show a percentage of these unique Kagi results in search results.</p> <p>Take a look at Kagiâs responses to a âsteve jobsâ query.</p> <p></p><p><i>Actual Kagi search results<i></i></i></p> <p>These results are authentic to Kagi. To provide them, we even crawl the <a href="https://archive.org/">Wayback Machine</a> to dig up great content that no longer exists in its original state online. In every search, Kagi will always display unique results when possible as well as some of the typical fare. </p> <p>With Kagi, you also get to influence search results. You can mute or favor domains, giving you an unprecedented level of personalization and refinement. And finally with Kagiâs "Lenses" you can define filters to narrow the scope of certain searches. A "Programming" or "Recipes" Lens can provide results only from sites you trust. .</p> <p> </p></div> </div> <div> <h2>Why does it appear that Kagi has fewer search results compared to Google? </h2> <div> <p> We believe the name of the game in search is quality, not quantity. Search engine's job is to provide the best answer fast and within the reach of one or two pages of results. If the user needs to go to page 17 to find what they were looking for, we would have failed anyway. </p> <p> Also note that one page of Kagi results will show 20-30 organic results compared to just 10 results typically shown on other search engines. So two pages of Kagi results are actually an equivalent of about six pages of results in other search engines! </p> <p> Other search engines may also show an inflated number of results available. The reality is they do not really have that many which you can check yourself by clicking through pages of their results, which one point abruptly stop. So between deceiving users with provisional high number and showing the best results we think there are, we chose the latter. </p> <p> We feel our approach is in line with our values. In the unlikely event of not finding what you are looking for, you can always modify the query slightly to get new results. </p> </div> </div> <div> <h2> <a href="#censoring">Does Kagi censor results?</a> </h2> <div> <p>No! Some results from traditional sources will reflect biases beyond our control. Also, we have built product features to help with bias reduction. For example, our "News 360" Lens includes articles from respectable media outlets across the globe.</p> <p>We are very careful not to introduce a precedent with any type of bias. This sometimes leads to constructive debate with our community, like <a href="https://kagifeedback.org/d/865-suicide-results-should-probably-have-a-dont-do-that-widget-like-google">this one</a>.</p> <p>What we do have is a strict search-quality bias. We penalize sites that bloated with ads and trackers. Simple as that.</p> </div> </div> <div> <h2> <a href="#ranking">How does Kagi rank results?</a> </h2> <div> <p> Our algorithm primarily focuses on relevancy and user intent. When it makes sense, we try to prioritize non-commercial sources. A highly relevant answer to your query from a monetized site will fall lower in your results than the same answer from an educational site. Of course, if a site has a naturally high rank, Kagi factors that in as well. And if you personalize your searches, domains you mute will lose ground to domains you prefer. </p> <p> </p></div> </div> <div> <p>Nope. Say goodbye to those paid entries at the top of your results page, because Kagi only shows the organic results to your queries.</p> </div> <div> <h2> <a href="#maps">Will Kagi have maps with local businesses?</a> </h2> <p>Yes. Weâre developing our Maps app, already available in Kagi, and weâll soon add map results to the main search results page. </p> </div> <div> <p>Yes. We know our users love these features, and so do we! Theyâre next on our development agenda.</p> </div> <div> <h2> <a href="#privacy">How does Kagi protect my privacy?</a> </h2> <div> <p> âFreeâ search engines have two ways to make money â get more users or sell more ads per user. The easiest, fastest way is to mine more data to sell ads more effectively. Many search-engine companies have buckled under investor pressure, leading us to the online privacy issues we have today.</p> <p>Paid models like Kagi only make money when users pay us, and users only pay when they receive great value. We offer superior result quality, features, speed and of course privacy. If we compromise on any of these points, we lose users. Therefore we have to offer a better product that includes the best possible privacy.</p> <p>We designed our product to give you a good answer and get you on your way instead of forcing you to wade through a swamp of results that have leeches lurking under the surface. </p> </div> </div> <div> <h2> <a href="#What-data-does-Kagi-collect">What data does Kagi collect?</a> </h2> <p>Only bare necessities to run the service. Please see our <a href="https://kagi.com/privacy">privacy policy</a> for more information.</p> </div> <div> <h2> <a href="#worried-privacy">I am still worried about my private information and searches being linked together. Why should I trust you?</a> </h2> <div> <p>Let us reiterate that we do not log searches or in any way tie them to an account. Our business model is to sell subscriptions, not user data.</p> <p>While it is virtually impossible for us to prove that, nor it is impossible to charge for the service without people having an account, we simply have no incentive to do it.</p> <p>Kagiâs business model is unlike any other search engine. This means we donât need or want your personal information (it would just be an unwelcome liability). In fact the only data we take is the email address, which can be any email address really, as long as you can use it to recover your account. There is no other personal information shared with Kagi. By choosing this business model we know that we are going to have far less users than mainstream search engine for example, but we have also removed any incentive to misuse any information you have shared with us.</p> </div> </div> <div> <h2> <a href="#using-login-token">How can I use Kagi in private browser sessions?</a> </h2> <div> <p>The simplest way is by using an official extension listed <a href="https://kagi.com/faq#How-do-I-set-Kagi-as-my-default-search-engine">here</a>.</p> <p>Basically, Kagi features a 'login token' which is simply a URL parameter that you can use to automatically log you into your existing Kagi session, from anywhere, including using search within private browser sessions. You can visit <a href="https://kagi.com/settings?p=user_details">Account settings</a> to get your login token.</p> <p>This token is attached to the session it was copied from. You can log out of that session if you need to invalidate it for any reason.</p> </div> </div> <div> <h2> <a href="#How-fast-is-Kagi">How fast is Kagi?</a> </h2> <div> <p>Ahhh... speed. Weâre obsessed with increasing speed and lowering latency, and we currently use three approaches. First, we optimized our technology stack to increase code execution speed and decrease connection latency. </p> <p></p><p>Then we reduced data transfer between Kagi and the browser, in some cases as much as 20x less compared to some of our competitors! This reduction has the neat side-effect of reducing CO2 emissions. Using Kagi Search will benefit the environment as well as you! </p> <table> <thead> <tr> <td>Product</td> <td>SERP Size</td> <td>CO2</td> <td>Load Time</td> </tr> </thead> <tbody> <tr> <td>Kagi</td> <td>0.76 MB</td> <td>0.43 g</td> <td>0.4 s</td> </tr> <tr> <td>Ecosia</td> <td>1.55 MB</td> <td>0.89 g</td> <td>1.2 s</td> </tr> <tr> <td>Bing</td> <td>1.94 MB</td> <td>1.11 g</td> <td>0.8 s</td> </tr> <tr> <td>Google</td> <td>2.43 MB</td> <td>1.39 g</td> <td>2.4 s</td> </tr> <tr> <td>DuckDuckGo</td> <td>2.48 MB</td> <td>1.42 g</td> <td>2.1 s</td> </tr> </tbody> </table> <p><i>Data from ecoping.earth<i></i></i></p> <p>Finally, our infrastructure is global, so youâll always connect automatically to the Kagi node closest to you. </p> </div> </div> <div> <h2> <a href="#comparison">How Kagi compares vs competition?</a> </h2> <div> <p>The table below should give you a general idea about how Kagi stacks up compared to some of the most popular search engines around.</p> <div> <table> <thead> <tr> <td></td> <td>Kagi</td> <td>Google/Bing</td> <td>DDG/StartPage</td> <td>Brave</td> <td>Neeva</td> </tr> </thead> <tbody> <tr> <td>Has own search results</td> <td>Yes</td> <td>Yes</td> <td>No</td> <td>Yes</td> <td>Yes</td> </tr> <tr> <td>Builds own Maps</td> <td>Yes</td> <td>Yes</td> <td>No</td> <td>Yes</td> <td>No</td> </tr> <tr> <td>User can block sites</td> <td>Yes</td> <td>No</td> <td>No</td> <td>No</td> <td>No</td> </tr> <tr> <td>Question answering</td> <td>Yes</td> <td>Yes</td> <td>No</td> <td>Yes</td> <td>No</td> </tr> <tr> <td>Ad-free business model</td> <td>Yes</td> <td>No</td> <td>No</td> <td>No</td> <td>Yes</td> </tr> <tr> <td>Anonymises user data</td> <td>Yes</td> <td>No</td> <td>Yes</td> <td>Yes</td> <td>No</td> </tr> <tr> <td>Zero search telemetry</td> <td>Yes</td> <td>No</td> <td>No</td> <td>No</td> <td>No</td> </tr> </tbody> </table> </div> </div> </div> <div> <p>Certainly! We take pride in our index of high-quality, non-commercial results. Check âInteresting Findsâ for relevant yet slightly unusual results or âBlast from the Pastâ for answers that exist only in the historical archives of the internet.</p> <p></p><p>You can also get instant answers to direct questions from Kagiâs AI, like â5*123,â â10 ft in metersâ or âtranslate hello to spanish.â</p> <p>In Kagi, you can use typical search operators as well as the search bangs popularized by DuckDuckGo. For example, if you want to find TVs on Amazon, you can enter â!a tvâ and go right to Amazonâs TV selections.</p> <p>If you know CSS, you can customize your search results page into nearly any style. This feature is an example of our power-user settings, with more on the horizon.</p> <p>Finally, Kagi Lenses allows you to specify what sites you see in your results. We provide a couple of Lenses to get you started, such as one to search only online discussions and forums. </p> </div> <div> <h2> <a href="#How-do-I-set-Kagi-as-my-default-search-engine-manually">How do I set Kagi as my default search engine manually?</a> </h2> <div> <div><p><b>Chromium based browsers (Chrome, Edge, Brave, Opera, Vivaldi...)</b></p><p> Desktop:</p></div><ol> <li>Open this URL in your search bar: <a>chrome://settings/searchEngines</a></li> <li>Click Add and fill: <br /> * Search engine: Kagi Search<br /> * Keyword: kagi<br /> * Query URL: https://kagi.com/search?q=%s </li> <li>Click the three dots</li> <li>Click "Make default"</li> </ol> <p></p><p> Mobile:</p><ol> <li>Open https://kagi.com/, login and submit a search.</li> <li>Open your browserâs settings. Choose âSearch engineâ.</li> <li>Select Kagi Search from the Recently Visited section</li> </ol> <p><b>Firefox based browsers</b><br /> </p><ol> <li>Open <a href="/">https://kagi.com</a>, then right click browser address/search bar and select "Add Kagi Search".</li> <li>Open this URL in your search bar: <a>about:preferences#search</a></li> <li>Scroll to "Default Search Engine" and select Kagi from the dropdown. </li></ol><p> To set up a <b>private session link</b> in Firefox, use <a href="https://addons.mozilla.org/en-US/firefox/addon/add-custom-search-engine/">this extension</a> with:<br /> <code>https://kagi.com/search?token=TOKEN&amp;q=%s</code> </p> <p><b>Safari</b><br /> </p><p><b>Desktop</b><br /> </p><ol> <li>Install the <a href="https://github.com/marcocebrian/kagisearchsafari">Kagi Search for Safari</a> extension from one of our community members.</li> <li>Activate the extension in Safari -&gt; Preferences -&gt; Extensions. </li> <li>Enjoy Kagi search from your Safari address bar!</li> </ol> <p>But a much better alternative is <a href="https://browser.kagi.com/">Orion browser</a>, made by same folks who made Kagi Search - us! Orion is a WebKit-based browser for Mac, with native integration with Kagi Search.</p> <div><p><b>Custom use</b><br /> For search URL use:<br /> <code>https://kagi.com/search?q={searchTerms}</code></p><p> and for autosuggestions use:<br /> <code>https://kagi.com/api/autosuggest?q={searchTerms}</code></p></div> <div><p><b>Private session link</b></p><p> To use Kagi Search inside a private browser session you can use the <a href="#using-login-token">session link</a>. </p></div> </div> </div> <div> <h2> <a href="#reduce-motion">Can I turn off animations in Kagi?</a> </h2> <div> <p>Yes. Kagi respects "Reduce motion" accessibility setting in your OS (name can differ). Here is where to find it:</p> <ul> <li>In macOS: System Preferences &gt; Accessibility &gt; Display &gt; Reduce motion.</li> <li>In iOS: Settings &gt; General &gt; Accessibility &gt; Reduce Motion.</li> <li>In Windows 11: Settings &gt; Accessibility &gt; Visual Effects &gt; Animation Effects</li> <li>In Windows 10: Settings &gt; Ease of Access &gt; Display &gt; Show animations in Windows.</li> <li>In Windows 7: Control Panel &gt; Ease of Access &gt; Make the computer easier to see &gt; Turn off all unnecessary animations (when possible).</li> <li>In Android 9+: Settings &gt; Accessibility &gt; Remove animations.</li> <li>In GTK/GNOME: GNOME Tweaks &gt; General tab (or Appearance, depending on version) &gt; Animations is turned off. </li> <li>In Plasma/KDE: System Settings &gt; Workspace Behavior -&gt; General Behavior &gt; "Animation speed" is set all the way to right to "Instant".</li> </ul> </div> </div> <div> <h2> <a href="#What-is-the-name-of-Kagi-mascot">What is the name of Kagi mascot?</a> </h2> <p>We simply call him âdoggoâ for now.</p> </div> <div> <h2> <a href="#kagi-shortcuts">What are search operators supported by Kagi?</a> </h2> <p>If you press ? on your keyboard (shift + /) while in search results, Kagi will open a help menu with search operators and other keyboard shortcuts that we support.</p> </div> <div> <h2> <a href="#kagi-logo">The logo is a "g" which is in the Google ballpark. Why not "k"??</a> </h2> <p>The designer insisted that Kagi logo is "anchored" in the letter "g" and we had to agree. Besides, can't let Google own letter "g" too, can we? ;)</p> </div> <div> <h2> <a href="#kagi-old">Are you affiliated with the legendary Kagi shareware platform?</a> </h2> <p>No. That Kagi went bankrupt in an unfortunate turn of events. We liked the name and acquired it when we got the chance.</p> </div> <div> <h2> <a href="#Are-you-hiring">Are you hiring?</a> </h2> <div> <p> It depends. Do you have advanced Crystal (backend) and/or vanilla JavaScript (frontend) knowledge? Can you solve difficult problems and code exceptionally well? (Weâll send a test project.) Send your CV to <a href="mailto:vlad@kagi.com">vlad@kagi.com</a>. </p> <p>Fun fact: entire Kagi Search runs with no JavaScript enabled in the browser. We see JavaScript as a way to enhance product experience, not create it.</p> </div> </div> <div> <h2>What if I find a security issue? </h2> <p> If you discover potential security issues on our platform, please notify our <a href="mailto:vlad@kagi.com">security contact</a>. </p> </div> <div> <h2>How do I join Kagiâs Discord server? </h2> </div> <div> <h2>Where can I find examples of Custom CSS created by other users? </h2> <p> The Kagi <a href="/discord">Discord</a> server has an #appearance channel where people are sharing custom CSS examples. A community member also started a <a href="https://www.reddit.com/r/KagiUserCSS/">subreddit</a>, which would be nice to see people contributing to. </p> </div> <div> <h2>Does Kagi have an assets page?</h2> <div> <p>Yes! You can go to <a href="https://kagi.com/assets">Kagi Assets</a> to view all of our assets and download them!</p></div> </div> <div> <h2>How do I contact support?</h2> </div> <div> <h2> <a href="#About-Kagi">About</a> </h2> <div> <p> Kagi's founder is <a href="https://vladimir.prelovac.com/">Vladimir Prelovac</a>: <i>"With Kagi and Orion, weâre striving to build better ways to experience the web, ways that are safe yet fun for everyone, including my own kids."</i></p> <p> Kagi was created in 2018 and is running on tight budget, bootstrapped by the founder. Kagi is based in SF Bay Area, and is being built by an international team. </p> <p>We are also building <a href="https://browser.kagi.com/">Orion</a>, an insanely fast, privacy-respecting browser, currently in beta for Mac.</p> <p>We hope that our work impacts people's lives in some small way for the better. Thank you very much for your interest in being a part of this!</p> <p>Kagi Inc.<br /> Palo Alto, CA<br /> email: vlad@kagi.com<br /> phone: (650) 535 2700<br /> </p> </div> </div></div>||||I|||| Help Settings
 Sign In
 Frequently Asked Questions

 What is Kagi?

 Kagi is a company created with the mission to humanize the web. Our goal is amplify the web of human knowledge, creativity and self-expression and provide the user tools to fight against the web of greed, ad-tech and user tracking.

 Kagi is currently Kagi Search, a fast, private search engine, and Orion Browser, a fast, zero-telemetry browser. We plan to release an email service as well, creating a platform to experience the web in a private, secure and unbiased way.

 How do you pronounce Kagi?

 Kagi is pronounced as Kah-gee. If still not sure, listen to it here.

 What is Kagi Search?

 Kagi Search is a quick, user-centric, 100% privacy-respecting search engine with results augmented by non-commercial indexes and personalized searches. It has a clean, high-performance user interface with only the most relevant results and no telemetry, no ads and no collection of private information. Read more about its unique features.

 We hope it will help users amplify their abilities and make them more creative and smarter as well as more productive.

 What does it mean that youâre in public beta?

 While we're in public beta, we don't feel like our vision is fully realized and ready for prime time. But we also don't want to build something in isolation. We want to build it with you, our users. During this time we plan to fix bugs, introduce more core feature and improve stability of the product. You can read more about this in our public beta announcement post.

 What are the costs to deliver a superior ad free search product?

 All search engines have: Search costs, Development costs and Administrative costs. Most search engines cover this by advertising, tracking, and selling your data. We attempt to cover it with a simple subscription of USD $10 a month.

 Our proposed price is dictated by the fact that search itself has a non-zero cost. In fact, it costs us about $1 to process 80 searches (wherever in the world you search from). So a user searching 8 times a day would perform about 240 searches a month, costing us $3 in search cost. But an average Kagi beta user is actually searching about 30 times a day. At USD $10/month, the price does not even cover our cost for average use, and we are basically betting that in the future we will be able to improve the product to reduce average number of searches needed to find something and reduce search cost by optimizations, to not only cover search cost but make room to pay for other costs as well.

 Our goal is to find the minimum price at which we can sustain the business. USD $10/mo is a price-point that we lowered as much as we could hoping it will allow us to have a chance of hitting sustainability at some point. If it turns out that we have more room we will decrease it. But it can also be that we may need to increase it.

 The cost of running free accounts is also substantial and it also has to be covered by paying customers, which makes the above math even harder. Tips are a way to help offset some of that cost.

 All of the above has to be set up in a way that after subtracting search cost we are still having left over to pay for the development costs (salaries) and the administrative costs.

 How will Kagi ensure its principles survive the future?

 Kagi has been driven by a vision of a more humane web and funded completely out of founder's pocket. You can read more about the financials in our first update after the public beta launch.

 This has allowed Kagi to follow only one agenda and that is the agenda of user-centric products.

 The only thing that could change this (potentially) is if it suddenly got powerful shareholders that could sway the path (typically VCs). However we are strategically avoding that and planning to offer our users to be investors In Kagi instead.

 This in our opinion will further extends the guarantee the company will stay on its track.

 How do I set Kagi as my default search engine?

 We have a browser extension for all major browsers. The extensions sets Kagi as the default search engine and allows you to search with Kagi even in a private window.

 * Chromium based browsers (Chrome, Edge, Brave, Opera, Vivaldi...) download here.
 * Firefox based browsers download here. If you use Firefox on Android, check this guide (it doesn't set Kagi as the default search engine, but it allows search in private mode to work normally as expected).
 * Safari macOS download here, Safari iOS download here. Some users use xSearch and HyperWeb extensions for Safari which also support Kagi.

 If you use macOS/iOS consider trying Orion browser, made by same folks who made Kagi Search - us! Orion is a WebKit-based browser for Mac, with native integration with Kagi Search.

 Why use Kagi instead of Google?

 Googleâs search engine has indeed changed the world and the way we access information, and we value this contribution. Weâre grateful to have access to Google's search technology and infrastructure for Kagi.

 But instead of trying to create a search product for billions of people, we want to develop a refined search experience for sophisticated customers who value high-quality results, privacy and speed.

 Kagi will allow you to discover well-written articles from lesser-known blogs and use features like Lenses to personalize results. Ad-supported search must avoid this kind of depth and flexibility to stay profitable. Read more about Kagi's unique features, many of which can never be replicated in ad-supported search engine.

 Is Kagi a "Google killer"?

 We do not see Kagi as the Google killer. Google's scale and reach is enourmous and Google serves a purpose in the world - it did help enable the modern society to exist, with all its marvels and flaws. Heck, it even enables Kagi to exist!

 Think of Kagi as a small, premium brand, providing a very different, tailor-made, search experience for people who need and appreciate that.

 Why use Kagi instead of DuckDuckGo?

 DuckDuckGo has shown the world that a privacy-first search engine is possible, and we respect this contribution. But its innovation has slowed in the past decade. And Ad-supported business models will always force a company to make compromises and balance between serving users and advertisers. At the end, DuckDuckGo's search product is just âgood enoughâ (by our standard, sorry DuckDuckGo!) and has been stagnant years without any ground-breaking feature development.

 In contrast, Kagi search does not need to compromise on user experience. Everything we do is user-centric. Kagi Search already has many unique features, like personalized results and "Lenses". And because we depend only on our users for revenue, Kagi can and will always offer a much richer search experience for the user.

 Does Kagi support !bangs?

 Yes, Kagi supports all DuckDuckGo-style bangs.

 Why pay for a search engine?

 You should pay for your search engine to ensure that the incentives of the information provider are aligned with whatâs best for you, not whatâs best for the advertisers.

 Every âfreeâ search engine out there comes with a hidden cost. Your private details pass through your searches, and search results influenced by advertising spends can influence your thoughts and actions. Businesses that monetize your data owe you more than just a "free" account! Our stand against ad-supported business models has created a 100% ad-free search experience, focused entirely on our users.

 If you want a search experience tailored to you and your needs, one that could also make the world a slightly better place, then you need to break free from the ruts that advertisers and corporations have guided us into. Investing in Kagi is investing in a more humane web, your online future and the future of your children, if you believe in them growing in a world where they are not bombarded with ads from an early age.

 Why ad-supported search engines do not work for the future of the web?

 We will let no other than the founders of Google explain this. The year is 1998, and this is how they describe why it is imperative for search engines to be ad-free:

 "Currently, the predominant business model for commercial search engines is advertising. The goals of the advertising business model do not always correspond to providing quality search to users. For example, in our prototype search engine one of the top results for cellular phone is "The Effect of Cellular Phone Use Upon Driver Attention", a study which explains in great detail the distractions and risk associated with conversing on a cell phone while driving. This search result came up first because of its high importance as judged by the PageRank algorithm, an approximation of citation importance on the web [Page, 98].

 It is clear that a search engine which was taking money for showing cellular phone ads would have difficulty justifying the page that our system returned to its paying advertisers. For this type of reason and historical experience with other media [Bagdikian 83], we expect that advertising funded search engines will be inherently biased towards the advertisers and away from the needs of the consumers."

 The Anatomy of a Large-Scale Hypertextual Web Search Engine, Sergey Brin and Lawrence Page, 1998

 They were right in recognizing early on that ad-supported search engines will be biased towards the advertisers. What they probably could not foresee is how much damage to the web adopting the very model they originally criticised will ultimately make.

 Where are your results coming from?

 We use heuristics and deep learning to understand query intent, select the best information sources, query them directly using APIs, and rank the results. You can think of Kagi as a "search client," working like an email client, connecting to indexes and sources to find relevant results and package them into a superior, secure and privacy-respecting search experience for you.

 Our searching includes anonymized requests to traditional search indexes like Google and Bing as well as vertical sources like Wikipedia and DeepL or other APIs. We also have our own non-commercial index (Teclis), news index (TinyGem), and an AI for instant answers.

 Teclis and TinyGem are a result of our crawl through millions of domains, focusing primarily on non-commercial, high-quality content. Our unique results help you discover some of the best content you can possibly find online, sometimes in the quieter places on the web.

 And of course, we answer quick queries like âHow far is the sun from the earth?â or â10kg in lbsâ with our instant-answer systems that use dozens of sources and APIs, all connected to you quickly yet transparently.

 How does Kagi achieve superior search quality?

 Kagi can surface unique, high-quality, non-commercial content while avoiding results bloated by ads and tracking. Kagi's non-commercial index (Teclis) and non-commercial news index (TinyGem) as well as instant answers (IA) offer high-quality, relevant and authentic results for the user. Clicking "Info" in search results will show a percentage of these unique Kagi results in search results.

 Take a look at Kagiâs responses to a âsteve jobsâ query.

 Actual Kagi search results

 These results are authentic to Kagi. To provide them, we even crawl the Wayback Machine to dig up great content that no longer exists in its original state online. In every search, Kagi will always display unique results when possible as well as some of the typical fare.

 With Kagi, you also get to influence search results. You can mute or favor domains, giving you an unprecedented level of personalization and refinement. And finally with Kagiâs "Lenses" you can define filters to narrow the scope of certain searches. A "Programming" or "Recipes" Lens can provide results only from sites you trust. .

 Why does it appear that Kagi has fewer search results compared to Google?

 We believe the name of the game in search is quality, not quantity. Search engine's job is to provide the best answer fast and within the reach of one or two pages of results. If the user needs to go to page 17 to find what they were looking for, we would have failed anyway.

 Also note that one page of Kagi results will show 20-30 organic results compared to just 10 results typically shown on other search engines. So two pages of Kagi results are actually an equivalent of about six pages of results in other search engines!

 Other search engines may also show an inflated number of results available. The reality is they do not really have that many which you can check yourself by clicking through pages of their results, which one point abruptly stop. So between deceiving users with provisional high number and showing the best results we think there are, we chose the latter.

 We feel our approach is in line with our values. In the unlikely event of not finding what you are looking for, you can always modify the query slightly to get new results.

 Does Kagi censor results?

 No! Some results from traditional sources will reflect biases beyond our control. Also, we have built product features to help with bias reduction. For example, our "News 360" Lens includes articles from respectable media outlets across the globe.

 We are very careful not to introduce a precedent with any type of bias. This sometimes leads to constructive debate with our community, like this one.

 What we do have is a strict search-quality bias. We penalize sites that bloated with ads and trackers. Simple as that.

 How does Kagi rank results?

 Our algorithm primarily focuses on relevancy and user intent. When it makes sense, we try to prioritize non-commercial sources. A highly relevant answer to your query from a monetized site will fall lower in your results than the same answer from an educational site. Of course, if a site has a naturally high rank, Kagi factors that in as well. And if you personalize your searches, domains you mute will lose ground to domains you prefer.

 If Kagi gets results from Google or Bing, does it also display paid/sponsored results?

 Nope. Say goodbye to those paid entries at the top of your results page, because Kagi only shows the organic results to your queries.

 Will Kagi have maps with local businesses?

 Yes. Weâre developing our Maps app, already available in Kagi, and weâll soon add map results to the main search results page.

 Will Kagi show stocks/weather/sports scores in results?

 Yes. We know our users love these features, and so do we! Theyâre next on our development agenda.

 How does Kagi protect my privacy?

 âFreeâ search engines have two ways to make money â get more users or sell more ads per user. The easiest, fastest way is to mine more data to sell ads more effectively. Many search-engine companies have buckled under investor pressure, leading us to the online privacy issues we have today.

 Paid models like Kagi only make money when users pay us, and users only pay when they receive great value. We offer superior result quality, features, speed and of course privacy. If we compromise on any of these points, we lose users. Therefore we have to offer a better product that includes the best possible privacy.

 We designed our product to give you a good answer and get you on your way instead of forcing you to wade through a swamp of results that have leeches lurking under the surface.

 What data does Kagi collect?

 Only bare necessities to run the service. Please see our privacy policy for more information.

 I am still worried about my private information and searches being linked together. Why should I trust you?

 Let us reiterate that we do not log searches or in any way tie them to an account. Our business model is to sell subscriptions, not user data.

 While it is virtually impossible for us to prove that, nor it is impossible to charge for the service without people having an account, we simply have no incentive to do it.

 Kagiâs business model is unlike any other search engine. This means we donât need or want your personal information (it would just be an unwelcome liability). In fact the only data we take is the email address, which can be any email address really, as long as you can use it to recover your account. There is no other personal information shared with Kagi. By choosing this business model we know that we are going to have far less users than mainstream search engine for example, but we have also removed any incentive to misuse any information you have shared with us.

 How can I use Kagi in private browser sessions?

 The simplest way is by using an official extension listed here.

 Basically, Kagi features a 'login token' which is simply a URL parameter that you can use to automatically log you into your existing Kagi session, from anywhere, including using search within private browser sessions. You can visit Account settings to get your login token.

 This token is attached to the session it was copied from. You can log out of that session if you need to invalidate it for any reason.

 How fast is Kagi?

 Ahhh... speed. Weâre obsessed with increasing speed and lowering latency, and we currently use three approaches. First, we optimized our technology stack to increase code execution speed and decrease connection latency.

 Then we reduced data transfer between Kagi and the browser, in some cases as much as 20x less compared to some of our competitors! This reduction has the neat side-effect of reducing CO2 emissions. Using Kagi Search will benefit the environment as well as you!

 Product SERP Size CO2 Load Time
 Kagi 0.76 MB 0.43 g 0.4 s 
 Ecosia 1.55 MB 0.89 g 1.2 s 
 Bing 1.94 MB 1.11 g 0.8 s 
 Google 2.43 MB 1.39 g 2.4 s 
 DuckDuckGo 2.48 MB 1.42 g 2.1 s 
 
 Data from ecoping.earth

 Finally, our infrastructure is global, so youâll always connect automatically to the Kagi node closest to you.

 How Kagi compares vs competition?

 The table below should give you a general idea about how Kagi stacks up compared to some of the most popular search engines around.

 Kagi Google/Bing DDG/StartPage Brave Neeva
 Has own search results Yes Yes No Yes Yes 
 Builds own Maps Yes Yes No Yes No 
 User can block sites Yes No No No No 
 Question answering Yes Yes No Yes No 
 Ad-free business model Yes No No No Yes 
 Anonymises user data Yes No Yes Yes No 
 Zero search telemetry Yes No No No No 
 

 Can you share a few pro tips for using Kagi?

 Certainly! We take pride in our index of high-quality, non-commercial results. Check âInteresting Findsâ for relevant yet slightly unusual results or âBlast from the Pastâ for answers that exist only in the historical archives of the internet.

 You can also get instant answers to direct questions from Kagiâs AI, like â5*123,â â10 ft in metersâ or âtranslate hello to spanish.â

 In Kagi, you can use typical search operators as well as the search bangs popularized by DuckDuckGo. For example, if you want to find TVs on Amazon, you can enter â!a tvâ and go right to Amazonâs TV selections.

 If you know CSS, you can customize your search results page into nearly any style. This feature is an example of our power-user settings, with more on the horizon.

 Finally, Kagi Lenses allows you to specify what sites you see in your results. We provide a couple of Lenses to get you started, such as one to search only online discussions and forums.

 How do I set Kagi as my default search engine manually?

 Chromium based browsers (Chrome, Edge, Brave, Opera, Vivaldi...)

 Desktop:

 1. Open this URL in your search bar: chrome://settings/searchEngines
 2. Click Add and fill:
 * Search engine: Kagi Search
 * Keyword: kagi
 * Query URL: https://kagi.com/search?q=%s
 3. Click the three dots
 4. Click "Make default"


 Mobile:
 1. Open https://kagi.com/, login and submit a search.
 2. Open your browserâs settings. Choose âSearch engineâ.
 3. Select Kagi Search from the Recently Visited section

 Firefox based browsers

 1. Open https://kagi.com, then right click browser address/search bar and select "Add Kagi Search".
 2. Open this URL in your search bar: about:preferences#search
 3. Scroll to "Default Search Engine" and select Kagi from the dropdown.
 To set up a private session link in Firefox, use this extension with:
 https://kagi.com/search?token=TOKEN&q=%s

 Safari

 Desktop

 1. Install the Kagi Search for Safari extension from one of our community members.
 2. Activate the extension in Safari -> Preferences -> Extensions.
 3. Enjoy Kagi search from your Safari address bar!

 But a much better alternative is Orion browser, made by same folks who made Kagi Search - us! Orion is a WebKit-based browser for Mac, with native integration with Kagi Search.

 iOS
 Use Orion iOS browser (still in beta) which has native support for Kagi Search.

 If you prefer to use Safari, you can use Kagi Search for Safari.

 Custom use
 For search URL use:
 https://kagi.com/search?q={searchTerms}

 and for autosuggestions use:
 https://kagi.com/api/autosuggest?q={searchTerms}

 Private session link

 To use Kagi Search inside a private browser session you can use the session link.

 Can I turn off animations in Kagi?

 Yes. Kagi respects "Reduce motion" accessibility setting in your OS (name can differ). Here is where to find it:

 * In macOS: System Preferences > Accessibility > Display > Reduce motion.
 * In iOS: Settings > General > Accessibility > Reduce Motion.
 * In Windows 11: Settings > Accessibility > Visual Effects > Animation Effects
 * In Windows 10: Settings > Ease of Access > Display > Show animations in Windows.
 * In Windows 7: Control Panel > Ease of Access > Make the computer easier to see > Turn off all unnecessary animations (when possible).
 * In Android 9+: Settings > Accessibility > Remove animations.
 * In GTK/GNOME: GNOME Tweaks > General tab (or Appearance, depending on version) > Animations is turned off.
 * In Plasma/KDE: System Settings > Workspace Behavior -> General Behavior > "Animation speed" is set all the way to right to "Instant".

 What is the name of Kagi mascot?

 We simply call him âdoggoâ for now.

 What are search operators supported by Kagi?

 If you press ? on your keyboard (shift + /) while in search results, Kagi will open a help menu with search operators and other keyboard shortcuts that we support.

 The logo is a "g" which is in the Google ballpark. Why not "k"??

 The designer insisted that Kagi logo is "anchored" in the letter "g" and we had to agree. Besides, can't let Google own letter "g" too, can we? ;)

 Are you affiliated with the legendary Kagi shareware platform?

 No. That Kagi went bankrupt in an unfortunate turn of events. We liked the name and acquired it when we got the chance.

 Are you hiring?

 It depends. Do you have advanced Crystal (backend) and/or vanilla JavaScript (frontend) knowledge? Can you solve difficult problems and code exceptionally well? (Weâll send a test project.) Send your CV to vlad@kagi.com.

 Fun fact: entire Kagi Search runs with no JavaScript enabled in the browser. We see JavaScript as a way to enhance product experience, not create it.

 What if I find a security issue?

 If you discover potential security issues on our platform, please notify our security contact.

 How do I join Kagiâs Discord server?

 The Kagi beta Discord server is a great way to provide feedback and have direct communication with our team.

 You can join this server here.

 For feature requests and bug reports, please submit to kagifeedback.org instead.

 Where can I find examples of Custom CSS created by other users?

 The Kagi Discord server has an #appearance channel where people are sharing custom CSS examples. A community member also started a subreddit, which would be nice to see people contributing to.

 Does Kagi have an assets page?

 Yes! You can go to Kagi Assets to view all of our assets and download them!

 How do I contact support?

 Please email support@kagi.com for customer support.

 About

 Kagi's founder is Vladimir Prelovac: "With Kagi and Orion, weâre striving to build better ways to experience the web, ways that are safe yet fun for everyone, including my own kids."

 Kagi was created in 2018 and is running on tight budget, bootstrapped by the founder. Kagi is based in SF Bay Area, and is being built by an international team.

 We are also building Orion, an insanely fast, privacy-respecting browser, currently in beta for Mac.

 We hope that our work impacts people's lives in some small way for the better. Thank you very much for your interest in being a part of this!

 Kagi Inc.
 Palo Alto, CA
 email: vlad@kagi.com
 phone: (650) 535 2700

 * Upgrade to Premium
 * About
 * Help
 * Changelog
 * Blog
 * Privacy & Terms
 * Status
 * Â© 2022 Kagi. Humanize the Web.
 Kagi includes data from Apple, Bing, Google, Wikipedia and others.
 Keyboard shortcuts
 ? Toggle open/close this keyboard shortcut help screen 
 j / arrowdown Move highlight down a result  
 k / arrowup Move highlight up a result  
 h / arrowleft Move left within horizontal inline content, or change boost/ban status in site info modal 
 l / arrowright Move right within horizontal inline content, or change boost/ban status in site info modal 
 enter Go to highlighted result  
 (or open first result, which would be highlighted) 
 / Focus the search bar  
 Escape Reset highlight scroll state to first result, or close site info modal 
 Shift + ~ Open control center  
 s Open site info modal on the currently highlighted result if applicable, or close it if already open.
 w Open web search tab  
 i Open images tab  
 v Open videos tab  
 m Open maps tab  
 n Open news tab  
 ] Cycle forward between navigation tabs 
 [ Cycle backwards between navigation tabs 
 
 Search operators
 filetype: Returns only search results that match a particular file extension 
 site: Returns only search results from a particular website 
 inurl: Finds webpages whose URL contains the terms or phrases 
 intitle: Finds webpages whose title contains the terms or phrases 
 âwordsâ Finds the exact words in a phrase 
 () Finds or excludes webpages that contain a group of words. 
 AND Finds webpages that contain either of the terms or phrases, for example âfood (cat OR dog)â.
 + and - Finds webpages that contain or exclude a term or phrase for example âfood +cat -dogâ. 
 OR Finds webpages that contain either of the terms or phrases for example âfood (cat OR dog)â. 
 * Matches any word for example âfirst * arrayâ. Try it! 
 
 Query Shortcuts & Widgets
 !bang We support all of DuckDuckGo's bangs.  
 \query Opens the first result of the search.  
 query !   
 !k Bang that performs a Kagi Search  
 !i Bang that performs a Kagi Image search  
 !v Bang that performs a Kagi Video search  
 !n Bang that performs a Kagi News search  
 !m Bang that performs a Kagi Maps search  
 calc Open Computer widget. Performs math calculations, unit, and currency conversions. Will open automatically when you search for a calculation. Try it!
 ip address Check your IP address. Try it!  
 timer / stopwatch Open Timer & Stopwatch widget. Try it!  
 translate Open language translation widget. Try it!  
 
What is Kagi?   Kagi is a company created with the mission to humanize the web. Our goal is amplify the web of human knowledge, creativity and self-expression and provide the user tools to fight against the web of greed, ad-tech and user tracking. Kagi is currently Kagi Search, a fast, private search engine, and Orion Browser, a fast, zero-telemetry browser. We plan to release an email service as well, creating a platform to experience the web in a private, secure and unbiased way.     How do you pronounce Kagi?  Kagi is pronounced as Kah-gee. If still not sure, listen to it here.    What is Kagi Search?   Kagi Search is a quick, user-centric, 100% privacy-respecting search engine with results augmented by non-commercial indexes and personalized searches. It has a clean, high-performance user interface with only the most relevant results and no telemetry, no ads and no collection of private information. Read more about its unique features. We hope it will help users amplify their abilities and make them more creative and smarter as well as more productive.      What does it mean that youâre in public beta?   While we're in public beta, we don't feel like our vision is fully realized and ready for prime time. But we also don't want to build something in isolation. We want to build it with you, our users. During this time we plan to fix bugs, introduce more core feature and improve stability of the product. You can read more about this in our public beta announcement post.     What are the costs to deliver a superior ad free search product?   All search engines have: Search costs, Development costs and Administrative costs. Most search engines cover this by advertising, tracking, and selling your data. We attempt to cover it with a simple subscription of USD $10 a month. Our proposed price is dictated by the fact that search itself has a non-zero cost. In fact, it costs us about $1 to process 80 searches (wherever in the world you search from). So a user searching 8 times a day would perform about 240 searches a month, costing us $3 in search cost. But an average Kagi beta user is actually searching about 30 times a day. At USD $10/month, the price does not even cover our cost for average use, and we are basically betting that in the future we will be able to improve the product to reduce average number of searches needed to find something and reduce search cost by optimizations, to not only cover search cost but make room to pay for other costs as well.  Our goal is to find the minimum price at which we can sustain the business. USD $10/mo is a price-point that we lowered as much as we could hoping it will allow us to have a chance of hitting sustainability at some point. If it turns out that we have more room we will decrease it. But it can also be that we may need to increase it. The cost of running free accounts is also substantial and it also has to be covered by paying customers, which makes the above math even harder. Tips are a way to help offset some of that cost. All of the above has to be set up in a way that after subtracting search cost we are still having left over to pay for the development costs (salaries) and the administrative costs.      How will Kagi ensure its principles survive the future?   Kagi has been driven by a vision of a more humane web and funded completely out of founder's pocket. You can read more about the financials in our first update after the public beta launch. This has allowed Kagi to follow only one agenda and that is the agenda of user-centric products. The only thing that could change this (potentially) is if it suddenly got powerful shareholders that could sway the path (typically VCs). However we are strategically avoding that and planning to offer our users to be investors In Kagi instead. This in our opinion will further extends the guarantee the company will stay on its track.     How do I set Kagi as my default search engine?   We have a browser extension for all major browsers. The extensions sets Kagi as the default search engine and allows you to search with Kagi even in a private window.   Chromium based browsers (Chrome, Edge, Brave, Opera, Vivaldi...) download here.  Firefox based browsers download here. If you use Firefox on Android, check this guide (it doesn't set Kagi as the default search engine, but it allows search in private mode to work normally as expected).  Safari macOS download here, Safari iOS download here. Some users use xSearch and HyperWeb extensions for Safari which also support Kagi.  If you use macOS/iOS consider trying Orion browser, made by same folks who made Kagi Search - us! Orion is a WebKit-based browser for Mac, with native integration with Kagi Search.     Why use Kagi instead of Google?   Googleâs search engine has indeed changed the world and the way we access information, and we value this contribution. Weâre grateful to have access to Google's search technology and infrastructure for Kagi.  But instead of trying to create a search product for billions of people, we want to develop a refined search experience for sophisticated customers who value high-quality results, privacy and speed. Kagi will allow you to discover well-written articles from lesser-known blogs and use features like Lenses to personalize results. Ad-supported search must avoid this kind of depth and flexibility to stay profitable. Read more about Kagi's unique features, many of which can never be replicated in ad-supported search engine.       Is Kagi a "Google killer"?   We do not see Kagi as the Google killer. Google's scale and reach is enourmous and Google serves a purpose in the world - it did help enable the modern society to exist, with all its marvels and flaws. Heck, it even enables Kagi to exist! Think of Kagi as a small, premium brand, providing a very different, tailor-made, search experience for people who need and appreciate that.     Why use Kagi instead of DuckDuckGo?   DuckDuckGo has shown the world that a privacy-first search engine is possible, and we respect this contribution. But its innovation has slowed in the past decade. And Ad-supported business models will always force a company to make compromises and balance between serving users and advertisers. At the end, DuckDuckGo's search product is just âgood enoughâ (by our standard, sorry DuckDuckGo!) and has been stagnant years without any ground-breaking feature development.  In contrast, Kagi search does not need to compromise on user experience. Everything we do is user-centric. Kagi Search already has many unique features, like personalized results and "Lenses". And because we depend only on our users for revenue, Kagi can and will always offer a much richer search experience for the user.       Does Kagi support !bangs?   Yes, Kagi supports all DuckDuckGo-style bangs.     Why pay for a search engine?   You should pay for your search engine to ensure that the incentives of the information provider are aligned with whatâs best for you, not whatâs best for the advertisers. Every âfreeâ search engine out there comes with a hidden cost. Your private details pass through your searches, and search results influenced by advertising spends can influence your thoughts and actions. Businesses that monetize your data owe you more than just a "free" account! Our stand against ad-supported business models has created a 100% ad-free search experience, focused entirely on our users. If you want a search experience tailored to you and your needs, one that could also make the world a slightly better place, then you need to break free from the ruts that advertisers and corporations have guided us into. Investing in Kagi is investing in a more humane web, your online future and the future of your children, if you believe in them growing in a world where they are not bombarded with ads from an early age.        Why ad-supported search engines do not work for the future of the web?   We will let no other than the founders of Google explain this. The year is 1998, and this is how they describe why it is imperative for search engines to be ad-free: "Currently, the predominant business model for commercial search engines is advertising. The goals of the advertising business model do not always correspond to providing quality search to users. For example, in our prototype search engine one of the top results for cellular phone is "The Effect of Cellular Phone Use Upon Driver Attention", a study which explains in great detail the distractions and risk associated with conversing on a cell phone while driving. This search result came up first because of its high importance as judged by the PageRank algorithm, an approximation of citation importance on the web [Page, 98]. It is clear that a search engine which was taking money for showing cellular phone ads would have difficulty justifying the page that our system returned to its paying advertisers. For this type of reason and historical experience with other media [Bagdikian 83], we expect that advertising funded search engines will be inherently biased towards the advertisers and away from the needs of the consumers."   The Anatomy of a Large-Scale Hypertextual Web Search Engine, Sergey Brin and Lawrence Page, 1998 They were right in recognizing early on that ad-supported search engines will be biased towards the advertisers. What they probably could not foresee is how much damage to the web adopting the very model they originally criticised will ultimately make.     Where are your results coming from?   We use heuristics and deep learning to understand query intent, select the best information sources, query them directly using APIs, and rank the results. You can think of Kagi as a "search client," working like an email client, connecting to indexes and sources to find relevant results and package them into a superior, secure and privacy-respecting search experience for you. Our searching includes anonymized requests to traditional search indexes like Google and Bing as well as vertical sources like Wikipedia and DeepL or other APIs. We also have our own non-commercial index (Teclis), news index (TinyGem), and an AI for instant answers. Teclis and TinyGem are a result of our crawl through millions of domains, focusing primarily on non-commercial, high-quality content. Our unique results help you discover some of the best content you can possibly find online, sometimes in the quieter places on the web. And of course, we answer quick queries like âHow far is the sun from the earth?â or â10kg in lbsâ with our instant-answer systems that use dozens of sources and APIs, all connected to you quickly yet transparently.      How does Kagi achieve superior search quality?   Kagi can surface unique, high-quality, non-commercial content while avoiding results bloated by ads and tracking. Kagi's non-commercial index (Teclis) and non-commercial news index (TinyGem) as well as instant answers (IA) offer high-quality, relevant and authentic results for the user. Clicking "Info" in search results will show a percentage of these unique Kagi results in search results. Take a look at Kagiâs responses to a âsteve jobsâ query.    Actual Kagi search results    These results are authentic to Kagi. To provide them, we even crawl the Wayback Machine to dig up great content that no longer exists in its original state online. In every search, Kagi will always display unique results when possible as well as some of the typical fare.  With Kagi, you also get to influence search results. You can mute or favor domains, giving you an unprecedented level of personalization and refinement. And finally with Kagiâs "Lenses" you can define filters to narrow the scope of certain searches. A "Programming" or "Recipes" Lens can provide results only from sites you trust. .      Why does it appear that Kagi has fewer search results compared to Google?    We believe the name of the game in search is quality, not quantity. Search engine's job is to provide the best answer fast and within the reach of one or two pages of results. If the user needs to go to page 17 to find what they were looking for, we would have failed anyway.   Also note that one page of Kagi results will show 20-30 organic results compared to just 10 results typically shown on other search engines. So two pages of Kagi results are actually an equivalent of about six pages of results in other search engines!   Other search engines may also show an inflated number of results available. The reality is they do not really have that many which you can check yourself by clicking through pages of their results, which one point abruptly stop. So between deceiving users with provisional high number and showing the best results we think there are, we chose the latter.   We feel our approach is in line with our values. In the unlikely event of not finding what you are looking for, you can always modify the query slightly to get new results.      Does Kagi censor results?   No! Some results from traditional sources will reflect biases beyond our control. Also, we have built product features to help with bias reduction. For example, our "News 360" Lens includes articles from respectable media outlets across the globe. We are very careful not to introduce a precedent with any type of bias. This sometimes leads to constructive debate with our community, like this one. What we do have is a strict search-quality bias. We penalize sites that bloated with ads and trackers. Simple as that.     How does Kagi rank results?    Our algorithm primarily focuses on relevancy and user intent. When it makes sense, we try to prioritize non-commercial sources. A highly relevant answer to your query from a monetized site will fall lower in your results than the same answer from an educational site. Of course, if a site has a naturally high rank, Kagi factors that in as well. And if you personalize your searches, domains you mute will lose ground to domains you prefer.       Nope. Say goodbye to those paid entries at the top of your results page, because Kagi only shows the organic results to your queries.    Will Kagi have maps with local businesses?  Yes. Weâre developing our Maps app, already available in Kagi, and weâll soon add map results to the main search results page.    Yes. We know our users love these features, and so do we! Theyâre next on our development agenda.    How does Kagi protect my privacy?    âFreeâ search engines have two ways to make money â get more users or sell more ads per user. The easiest, fastest way is to mine more data to sell ads more effectively. Many search-engine companies have buckled under investor pressure, leading us to the online privacy issues we have today. Paid models like Kagi only make money when users pay us, and users only pay when they receive great value. We offer superior result quality, features, speed and of course privacy. If we compromise on any of these points, we lose users. Therefore we have to offer a better product that includes the best possible privacy. We designed our product to give you a good answer and get you on your way instead of forcing you to wade through a swamp of results that have leeches lurking under the surface.      What data does Kagi collect?  Only bare necessities to run the service. Please see our privacy policy for more information.    I am still worried about my private information and searches being linked together. Why should I trust you?   Let us reiterate that we do not log searches or in any way tie them to an account. Our business model is to sell subscriptions, not user data. While it is virtually impossible for us to prove that, nor it is impossible to charge for the service without people having an account, we simply have no incentive to do it. Kagiâs business model is unlike any other search engine. This means we donât need or want your personal information (it would just be an unwelcome liability). In fact the only data we take is the email address, which can be any email address really, as long as you can use it to recover your account. There is no other personal information shared with Kagi. By choosing this business model we know that we are going to have far less users than mainstream search engine for example, but we have also removed any incentive to misuse any information you have shared with us.     How can I use Kagi in private browser sessions?   The simplest way is by using an official extension listed here. Basically, Kagi features a 'login token' which is simply a URL parameter that you can use to automatically log you into your existing Kagi session, from anywhere, including using search within private browser sessions. You can visit Account settings to get your login token. This token is attached to the session it was copied from. You can log out of that session if you need to invalidate it for any reason.     How fast is Kagi?   Ahhh... speed. Weâre obsessed with increasing speed and lowering latency, and we currently use three approaches. First, we optimized our technology stack to increase code execution speed and decrease connection latency.    Then we reduced data transfer between Kagi and the browser, in some cases as much as 20x less compared to some of our competitors! This reduction has the neat side-effect of reducing CO2 emissions. Using Kagi Search will benefit the environment as well as you!     Product SERP Size CO2 Load Time     Kagi 0.76 MB 0.43 g 0.4 s   Ecosia 1.55 MB 0.89 g 1.2 s   Bing 1.94 MB 1.11 g 0.8 s   Google 2.43 MB 1.39 g 2.4 s   DuckDuckGo 2.48 MB 1.42 g 2.1 s     Data from ecoping.earth    Finally, our infrastructure is global, so youâll always connect automatically to the Kagi node closest to you.      How Kagi compares vs competition?   The table below should give you a general idea about how Kagi stacks up compared to some of the most popular search engines around.       Kagi Google/Bing DDG/StartPage Brave Neeva     Has own search results Yes Yes No Yes Yes   Builds own Maps Yes Yes No Yes No   User can block sites Yes No No No No   Question answering Yes Yes No Yes No   Ad-free business model Yes No No No Yes   Anonymises user data Yes No Yes Yes No   Zero search telemetry Yes No No No No        Certainly! We take pride in our index of high-quality, non-commercial results. Check âInteresting Findsâ for relevant yet slightly unusual results or âBlast from the Pastâ for answers that exist only in the historical archives of the internet.   You can also get instant answers to direct questions from Kagiâs AI, like â5*123,â â10 ft in metersâ or âtranslate hello to spanish.â In Kagi, you can use typical search operators as well as the search bangs popularized by DuckDuckGo. For example, if you want to find TVs on Amazon, you can enter â!a tvâ and go right to Amazonâs TV selections. If you know CSS, you can customize your search results page into nearly any style. This feature is an example of our power-user settings, with more on the horizon. Finally, Kagi Lenses allows you to specify what sites you see in your results. We provide a couple of Lenses to get you started, such as one to search only online discussions and forums.     How do I set Kagi as my default search engine manually?     Chromium based browsers (Chrome, Edge, Brave, Opera, Vivaldi...)   Desktop:   Open this URL in your search bar: chrome://settings/searchEngines  Click Add and fill:  * Search engine: Kagi Search * Keyword: kagi * Query URL: https://kagi.com/search?q=%s  Click the three dots Click "Make default"     Mobile:  Open https://kagi.com/, login and submit a search. Open your browserâs settings. Choose âSearch engineâ. Select Kagi Search from the Recently Visited section   Firefox based browsers    Open https://kagi.com, then right click browser address/search bar and select "Add Kagi Search". Open this URL in your search bar: about:preferences#search  Scroll to "Default Search Engine" and select Kagi from the dropdown.    To set up a private session link in Firefox, use this extension with: https://kagi.com/search?token=TOKEN&amp;q=%s   Safari    Desktop    Install the Kagi Search for Safari extension from one of our community members. Activate the extension in Safari -&gt; Preferences -&gt; Extensions.  Enjoy Kagi search from your Safari address bar!  But a much better alternative is Orion browser, made by same folks who made Kagi Search - us! Orion is a WebKit-based browser for Mac, with native integration with Kagi Search.   Custom use  For search URL use: https://kagi.com/search?q={searchTerms}   and for autosuggestions use: https://kagi.com/api/autosuggest?q={searchTerms}     Private session link   To use Kagi Search inside a private browser session you can use the session link.       Can I turn off animations in Kagi?   Yes. Kagi respects "Reduce motion" accessibility setting in your OS (name can differ). Here is where to find it:  In macOS: System Preferences &gt; Accessibility &gt; Display &gt; Reduce motion. In iOS: Settings &gt; General &gt; Accessibility &gt; Reduce Motion. In Windows 11: Settings &gt; Accessibility &gt; Visual Effects &gt; Animation Effects In Windows 10: Settings &gt; Ease of Access &gt; Display &gt; Show animations in Windows. In Windows 7: Control Panel &gt; Ease of Access &gt; Make the computer easier to see &gt; Turn off all unnecessary animations (when possible). In Android 9+: Settings &gt; Accessibility &gt; Remove animations. In GTK/GNOME: GNOME Tweaks &gt; General tab (or Appearance, depending on version) &gt; Animations is turned off.  In Plasma/KDE: System Settings &gt; Workspace Behavior -&gt; General Behavior &gt; "Animation speed" is set all the way to right to "Instant".      What is the name of Kagi mascot?  We simply call him âdoggoâ for now.    What are search operators supported by Kagi?  If you press ? on your keyboard (shift + /) while in search results, Kagi will open a help menu with search operators and other keyboard shortcuts that we support.    The logo is a "g" which is in the Google ballpark. Why not "k"??  The designer insisted that Kagi logo is "anchored" in the letter "g" and we had to agree. Besides, can't let Google own letter "g" too, can we? ;)    Are you affiliated with the legendary Kagi shareware platform?  No. That Kagi went bankrupt in an unfortunate turn of events. We liked the name and acquired it when we got the chance.    Are you hiring?    It depends. Do you have advanced Crystal (backend) and/or vanilla JavaScript (frontend) knowledge? Can you solve difficult problems and code exceptionally well? (Weâll send a test project.) Send your CV to vlad@kagi.com.  Fun fact: entire Kagi Search runs with no JavaScript enabled in the browser. We see JavaScript as a way to enhance product experience, not create it.    What if I find a security issue?   If you discover potential security issues on our platform, please notify our security contact.    How do I join Kagiâs Discord server?    Where can I find examples of Custom CSS created by other users?   The Kagi Discord server has an #appearance channel where people are sharing custom CSS examples. A community member also started a subreddit, which would be nice to see people contributing to.    Does Kagi have an assets page?  Yes! You can go to Kagi Assets to view all of our assets and download them!    How do I contact support?    About    Kagi's founder is Vladimir Prelovac: "With Kagi and Orion, weâre striving to build better ways to experience the web, ways that are safe yet fun for everyone, including my own kids."   Kagi was created in 2018 and is running on tight budget, bootstrapped by the founder. Kagi is based in SF Bay Area, and is being built by an international team.  We are also building Orion, an insanely fast, privacy-respecting browser, currently in beta for Mac. We hope that our work impacts people's lives in some small way for the better. Thank you very much for your interest in being a part of this! Kagi Inc. Palo Alto, CA email: vlad@kagi.com phone: (650) 535 2700
What is Kagi?
https://kagi.com/faq
https://dkb.io/post/the-next-google
The Next Google
2022-04-05
None
<div><div><p>DuckDuckGo and Bing are not true alternatives – they’re just worse versions of Google.</p>
<p>The next Google can’t just be an input box that spits out links. We need new thinking to create something much better than what came before.</p>
<p>In the last few years, different groups of people came to the same conclusion, and started working on the next generation of search engines.</p>
<p>For this new generation, privacy is necessary, and invasive ads are not an option. But that’s where the commonalities end. Beyond that, they’ve all taken the idea of a search engine in very different directions.</p>
<h2>Kagi – The Customization Engine</h2>
<p></p>
<p>Why should everyone have the same search experience? We all have our own preferences about how things should look and work.</p>
<p><a href="https://kagi.com/">Kagi</a> is the most customizable search engine ever. You can change everything, from surface level appearances, to the final ranking of results.</p>
<blockquote>
<p>“Everybody has different preferences of how they want a search engine to look and feel. Our goal is to provide tools and empower users to do that instead of trying to be smart and creating an average search engine for an average user. There’s no average human.” - Vladimir Prelovac, Founder of Kagi</p>
</blockquote>
<h2>Visual Customization</h2>
<p>On the visual side, you can modify everything about the way things look, even being able to write your own custom CSS.</p>
<p></p>
<h2>Search Customization</h2>
<p>More importantly, you can customize the way the search engine itself works. Do you want search suggestions? How detailed should the snippets be?</p>
<p></p>
<p>There’s also a list of widgets you can toggle like “Instant Answers”, “Inline Discussions” and “Listicles”. These widgets are groupings of specific types of results which you can choose to include in the results page. If you never want to see another listicle again in your life, just toggle the listicle widget off.</p>
<p>
But you can take the search customization even further than that, down to the level of individual websites. You can create your own domain rankings to tell Kagi what websites should be boosted, and what should be down ranked.</p>
<p>You can even outright block sites you never want to see again (looking at you pinterest), and pin sites if you want them always on top (hello reddit).</p>
<p></p>
<blockquote>
<p>“Nobody has their home screen on iPhone looking exactly the same. I think people deserve that in search, especially being such an important part of our lives. It’s one of the most used products out there, and it’s astonishing how little customization you can do.” - Vladimir Prelovac, Founder of Kagi</p>
</blockquote>
<h2>Lenses</h2>
<p>Kagi also allows you to create website filters called lenses. In any given lens, you can include or exclude specific sites, and more.</p>
<p></p>
<p></p>
<p>When searching you can turn on different lenses to see results that match those filters. Kagi comes with pre-built lenses for things like Discussions, Programming Help and Recipes.</p>
<h2>Non-commercial Results</h2>
<p>Another thing that Kagi focuses on is delivering non-commercial results. Kagi has an entire index of non-commercial websites, which they’ve also made available separately as <a href="http://teclis.com/">Teclis</a>. This allows them to find results that you won’t find on Google and most other search engines.</p>
<p></p>
<p>For example, when searching for “steve jobs”, the top results are similar to Google with mostly informational stuff. But right after that, we’re greeted by an “Interesting Finds” section, which has a <a href="https://sive.rs/itunes">fun blog post by Derek Sivers</a>, an article filled with <a href="https://www3.nhk.or.jp/nhkworld/en/news/backstories/1622/">stories of Steve Jobs in Japan</a>, and some other cool things you can’t find on Google.</p>
<h2>Neeva – The Everything Engine</h2>
<p></p>
<p>Why should a search engine only index the web? A lot of the information you care about is scattered across your email, your note taking app, and other places.</p>
<p><a href="https://neeva.com/">Neeva</a> is the search engine that allows you to search <em>everything</em>.</p>
<p>Right now you can connect Google Apps, Dropbox, Notion and more.</p>
<p></p>
<p>No matter what you’re looking for, or where it is, Neeva will help you find it. Searching your personal documents is as simple as adding “@me” to the query. This gives everyone the ability to have a <a href="https://thesephist.com/posts/monocle/">personal search engine</a>.</p>
<p></p>
<p>This is one of many things that makes Neeva interesting.</p>
<h2>Instant Inline Results</h2>
<p>Since Google is an ads based business, they are incentivized to take you to the search results page, even when it’s obvious what you want.</p>
<p>If you typed in “amazon” hoping to get to amazon.com, Google would first take you to the search results page, so they can show you an ad, which amazon.com has to pay for just to be the first result for their own company name.</p>
<p>On the Neeva browser app, you get instant inline search results as you type, allowing you to go directly to the page you’re looking for. You can do in one click what Google will always want you to do in two clicks.</p>
<blockquote>
<p>“We have the opportunity to innovate on the browser in a lot of interesting ways because we’re not beholden to how many search results page views we get with ads on them. Our business model unlocks our ability to help people not have to see the search results page as much.” - Darin Fisher, Head of Neeva Browser</p>
</blockquote>
<p></p>
<p>And it’s not just a shortcut for navigating to domains. Type “best thai food” and immediately get a link to the relevant yelp page.</p>
<p></p>
<p>You can still go to the search results page if you need to, but this feature allows you to be twice as fast as Google.</p>
<h2>Similarity Search</h2>
<p>Another unique feature, only available on the Neeva browser app for now, is their similarity search functionality called NeevaScope.</p>
<p>Wherever you are on the web, you can tap the NeevaScope button to find similar and related pages to the one you’re currently on. This is powerful for exploration.</p>
<p></p>
<p>I tried it out on <a href="https://waitbutwhy.com/2017/04/neuralink.html">Tim Urban’s Neuralink post</a> and found some other interesting content, including one that criticized Tim Urban’s post.</p>
<h2>Filters</h2>
<p>Neeva recognizes that there are different categories of websites that may be of interest to you for a given query, and provides filters to narrow things down.</p>
<blockquote>
<p>“Google creates this view, I call it the flat-earth view, where every site is just like every other site. We go out of our way to tell you additional things you should know, and allow you to filter on these attributes.” - Sridhar Ramaswamy, CEO of Neeva</p>
</blockquote>
<p>For example, when searching for legal information, you can filter to “Legal Providers” to see results from companies that provide legal services, or “User Generated Content” to get discussion forums or blogs.</p>
<p></p>
<h2>Shopping Experience</h2>
<p>One type of query that Neeva does particularly well at is shopping. Neeva knows that reviews are important, so they make it easy to find them.</p>
<p></p>
<p>The interface is geared towards helping you quickly figure out what products are worth looking more into, and connecting you to good review sources.</p>
<p></p>
<h2>You.com – The App Engine</h2>
<p></p>
<p>When you search a coding problem on Google, you’ll find a Stack Overflow answer. When you search on <a href="https://you.com/">You.com</a>, the Code Complete app will <em>generate the code snippet from scratch using a language model</em>.</p>
<p></p>
<p>You.com sees the web as a collection of apps, and their goal is to match your query with the most relevant apps. For coding questions, there is an AI Code Generator app, but Stack Overflow is also an “app” which you can open in the sidebar and easily copy code from.</p>
<p>The Quick Facts app can give you short factual information, and the Medium app allows you to easily read articles in the sidebar.</p>
<p></p>
<p>Eventually people will be able to build their own You.com apps with unique interactions, and publish them to the You.com platform.</p>
<h2>Summaries</h2>
<p>For many apps, you’ll be able to open a sidebar that has a summary of the important content from that page.</p>
<p>For example, if searching for food recipes, the All Recipes app and the New York Times Cooking app both allow you to open the recipe in the sidebar. This summarized view gives you exactly the information you want – a list of ingredients, and the instructions.</p>
<p></p>
<p>For coding, the Stack Overflow app and W3 Schools app allow you to click a “copy to clipboard” button to get the code.</p>
<p></p>
<h2>Filters</h2>
<p>Filters in You.com are a collection of apps particularly suited to some task. For example, the “YouCode” filter will give you apps like Stack Overflow, Code Complete, and Github, while the “Social” filter allows you to search social media apps.</p>
<p></p>
<h2>YouWrite</h2>
<p>This isn’t exactly a search feature, but it highlights the core strength of You.com, which is their AI capabilities. YouWrite is an AI writing assistant. You tell it what to write about, and it generates some paragraphs for you.</p>
<p>I told YouWrite to write about why everyone should watch anime, and this is what it gave me:</p>
<blockquote>
<p>There are plenty of reasons why everyone should watch anime. For one, anime is a great form of entertainment. It can be exciting, thrilling, and suspenseful – just like any good movie or TV show. But what makes anime so unique is that it often incorporates elements of Japanese culture and folklore that you won’t find in other forms of entertainment.</p>
</blockquote>
<blockquote>
<p>Another reason to watch anime is because it can be educational. Anime has been known to teach people about different cultures, history, and even science! There are also many documentaries available that focus on various aspects of the anime industry – such as how certain shows are made or the history behind specific characters or plot lines.</p>
</blockquote>
<blockquote>
<p>Lastly, watching anime can help you learn Japanese! While not all shows are dubbed into English (or any other language), there are many websites where you can find translations for free. And if you’re feeling really ambitious, there are plenty of courses available online which will help you learn to read and write in Japanese too!</p>
</blockquote>
<p>Something like this has the potential to become a powerful research tool. Instead of searching for “World War 2” and trying to find good sources, you can have YouWrite generate an essay about World War 2, and research the arguments it makes as a starting point.</p>
<p>As an aside, if you were wondering whether the <a href="https://www.theatlantic.com/technology/archive/2021/08/dead-internet-theory-wrong-but-feels-true/619937/">Dead Internet Theory</a> had any chance of being true, here’s your answer.</p>
<h2>Andi – The Conversation Engine</h2>
<p></p>
<p>Do search engines have to be so impersonal? We’re used to just typing our query and getting back a list of links, but <a href="https://andisearch.com/">Andi</a> flips the script on this.</p>
<p>Andi gives you a conversational search experience that’s more like “messaging with a smart friend who answers questions and sends you useful links”.</p>
<p>Andi’s focus is on answering specific questions by extracting an answer from the best source.</p>
<p>For example, searching “what is the gdp per capita of china vs new zealand” on Google returns a badly formatted table without the answer.</p>
<p></p>
<p>While searching the same thing on Andi will extract the right information for you.</p>
<p></p>
<h2>Dynamic Results</h2>
<p>The way that Andi generates search results is different from the other search engines on this list. It focuses on understanding the intent and topics of your query, then tries to predict the best sources for your answer. After that, it queries the relevant APIs and vertical search engines directly, generating results in realtime.</p>
<p>There is no index.</p>
<h2>Reader View</h2>
<p>Another feature of Andi worth mentioning is the reader view, which allows you to view the content of pages without having to open them. This allows you to have a consistent browsing experience, and avoid ads and clutter.</p>
<p></p>
<p></p>
<h2>Brave Search – The Independent Engine</h2>
<p>It’s time to address the elephant in the room. Every search engine I’ve mentioned so far, no matter how many cool features they’re building, does not have a fully independent index.</p>
<p>They’re all getting some portion of their results from Google or Bing.</p>
<p>Building an independent index is a very hard problem given the current size of the web. You can count the number of search engines with a fully independent index on one hand.</p>
<p>Many search engines on this list aspire to have a fully independent index at some point, but none of them are more focused on this than <a href="https://search.brave.com/">Brave Search</a>.</p>
<p></p>
<p>For any search you do on Brave, you’ll see what percentage of the results came from their own index. They make this number loud and clear, because they want it to be 100%.</p>
<p></p>
<blockquote>
<p>“Brave Search can operate as stand-alone, the rest cannot as they rely on Google or Bing. Most search engines are not independent search engines, and while they may provide some value, they are qualitatively different from what Brave Search is doing. Independence is not something directly actionable, but it’s a fundamental property. Independence means that Brave Search would continue to work even if Google and Microsoft opposed it. Independence means choice and diversity.” - Josep M. Pujol, Chief of Search at Brave</p>
</blockquote>
<p>You won’t see much cool new functionality on Brave Search, because they are laser focused on the one thing they believe to be important to a search engine.</p>
<h2>Web Discovery Project</h2>
<p>The web discovery project is Brave’s solution to one of the major problems with privacy focused search engines. Google can provide good results because they collect a ton of data on everyone, and they use that to improve the search experience. Privacy focused search engines don’t collect that kind of data, which limits their potential.</p>
<p>With the <a href="https://support.brave.com/hc/en-us/articles/4409406835469-What-is-the-Web-Discovery-Project-">web discovery project</a>, users can opt-in to contribute anonymous search and web browsing data to help improve Brave’s results. This gives Brave a way to get the same kind of data that Google uses to optimize queries, but without comprising on user privacy.</p>
<p>If you don’t like it, you have nothing to worry about, because it’s off by default.</p>
<h2>Goggles – Collaborative Mini Search Engines</h2>
<p>The most important innovation of Brave is yet to be released. <a href="https://brave.com/static-assets/files/goggles.pdf">Goggles</a> allows users to create mini search engines within Brave. Anyone can create detailed ranking specifications that Brave will use when returning results. People can collaborate on goggles and share them.</p>
<p></p>
<p>Some examples of goggles that could be created are “top tech blogs”, “drop the top 1000 most popular domains” (which implements the general idea of an older search engine, <a href="https://millionshort.com/">MillionShort</a>), or “cooking sites my mom would recommend”.</p>
<p>Goggles are easily toggled on and off, so users can leave and enter different filter bubbles as they please. Unlike Google, these filter bubbles are intentionally created by groups of people, not decided by one centralized algorithm that’s working in the background. Goggles can be a useful tool to explore the web from different perspectives.</p>
<p>Goggles is effectively making search engine creation accessible to anyone. It’s very hard for a regular person to build an entire search engine from scratch, but it will be easy for anyone to create a goggle that represents their idea of a perfect search engine.</p>
<h2>This Is Just The Beginning</h2>
<p>This new wave of search engines is only just getting started. Many of them have only recently launched.</p>
<p>Even if they aren’t perfect yet, the paths they’re exploring can lead to promising new innovation in the stagnant search space.</p>
<blockquote>
<p>“The search technology was developed a decade ago. It’s a bet that no one else will come up with a better search technology. So you invest in Google because you’re betting against technological innovation in search.” - <a href="https://www.youtube.com/watch?v=2Q26XIKtwXQ">Peter Thiel, 2012</a></p>
</blockquote>
<p>Betting against innovation in search has been a great move for the last ten years. The next ten years look a little bit more hopeful.</p>
<hr />
<p>I’ll be posting interviews with the founders of each of these search engines soon. If you’re interested in getting a more in-depth look at their stories, and how they think about things, stay tuned by signing up for my <a href="/">newsletter</a>.</p>
<h2>Appendix 1: Interview with Brave Search</h2>
<p>I have now published a full length interview with Brave Search which you can read <a href="/post/brave-search-interview">here</a>.</p>
<h2>Appendix 2: Interview with Neeva</h2>
<p>You can read a full length interview with Neeva <a href="/post/neeva-interview">here</a>.</p>
<h2>Appendix 3: Interview with Kagi</h2>
<p>You can read a full length interview with Kagi <a href="/post/kagi-interview">here</a>.</p>
</div>||||I|||| * Home
 * About
 * RSS



 The Next Google

 April 5, 2022 Last updated: April 5, 2022
 # search

 DuckDuckGo and Bing are not true alternatives – they’re just worse versions of Google.

 The next Google can’t just be an input box that spits out links. We need new thinking to create something much better than what came before.

 In the last few years, different groups of people came to the same conclusion, and started working on the next generation of search engines.

 For this new generation, privacy is necessary, and invasive ads are not an option. But that’s where the commonalities end. Beyond that, they’ve all taken the idea of a search engine in very different directions.

 Kagi – The Customization Engine

 Why should everyone have the same search experience? We all have our own preferences about how things should look and work.

 Kagi is the most customizable search engine ever. You can change everything, from surface level appearances, to the final ranking of results.

 “Everybody has different preferences of how they want a search engine to look and feel. Our goal is to provide tools and empower users to do that instead of trying to be smart and creating an average search engine for an average user. There’s no average human.” - Vladimir Prelovac, Founder of Kagi

 Visual Customization

 On the visual side, you can modify everything about the way things look, even being able to write your own custom CSS.

 Search Customization

 More importantly, you can customize the way the search engine itself works. Do you want search suggestions? How detailed should the snippets be?

 There’s also a list of widgets you can toggle like “Instant Answers”, “Inline Discussions” and “Listicles”. These widgets are groupings of specific types of results which you can choose to include in the results page. If you never want to see another listicle again in your life, just toggle the listicle widget off.

 But you can take the search customization even further than that, down to the level of individual websites. You can create your own domain rankings to tell Kagi what websites should be boosted, and what should be down ranked.

 You can even outright block sites you never want to see again (looking at you pinterest), and pin sites if you want them always on top (hello reddit).

 “Nobody has their home screen on iPhone looking exactly the same. I think people deserve that in search, especially being such an important part of our lives. It’s one of the most used products out there, and it’s astonishing how little customization you can do.” - Vladimir Prelovac, Founder of Kagi

 Lenses

 Kagi also allows you to create website filters called lenses. In any given lens, you can include or exclude specific sites, and more.

 When searching you can turn on different lenses to see results that match those filters. Kagi comes with pre-built lenses for things like Discussions, Programming Help and Recipes.

 Non-commercial Results

 Another thing that Kagi focuses on is delivering non-commercial results. Kagi has an entire index of non-commercial websites, which they’ve also made available separately as Teclis. This allows them to find results that you won’t find on Google and most other search engines.

 For example, when searching for “steve jobs”, the top results are similar to Google with mostly informational stuff. But right after that, we’re greeted by an “Interesting Finds” section, which has a fun blog post by Derek Sivers, an article filled with stories of Steve Jobs in Japan, and some other cool things you can’t find on Google.

 Neeva – The Everything Engine

 Why should a search engine only index the web? A lot of the information you care about is scattered across your email, your note taking app, and other places.

 Neeva is the search engine that allows you to search everything.

 Right now you can connect Google Apps, Dropbox, Notion and more.

 No matter what you’re looking for, or where it is, Neeva will help you find it. Searching your personal documents is as simple as adding “@me” to the query. This gives everyone the ability to have a personal search engine.

 This is one of many things that makes Neeva interesting.

 Instant Inline Results

 Since Google is an ads based business, they are incentivized to take you to the search results page, even when it’s obvious what you want.

 If you typed in “amazon” hoping to get to amazon.com, Google would first take you to the search results page, so they can show you an ad, which amazon.com has to pay for just to be the first result for their own company name.

 On the Neeva browser app, you get instant inline search results as you type, allowing you to go directly to the page you’re looking for. You can do in one click what Google will always want you to do in two clicks.

 “We have the opportunity to innovate on the browser in a lot of interesting ways because we’re not beholden to how many search results page views we get with ads on them. Our business model unlocks our ability to help people not have to see the search results page as much.” - Darin Fisher, Head of Neeva Browser

 And it’s not just a shortcut for navigating to domains. Type “best thai food” and immediately get a link to the relevant yelp page.

 You can still go to the search results page if you need to, but this feature allows you to be twice as fast as Google.

 Similarity Search

 Another unique feature, only available on the Neeva browser app for now, is their similarity search functionality called NeevaScope.

 Wherever you are on the web, you can tap the NeevaScope button to find similar and related pages to the one you’re currently on. This is powerful for exploration.

 I tried it out on Tim Urban’s Neuralink post and found some other interesting content, including one that criticized Tim Urban’s post.

 Filters

 Neeva recognizes that there are different categories of websites that may be of interest to you for a given query, and provides filters to narrow things down.

 “Google creates this view, I call it the flat-earth view, where every site is just like every other site. We go out of our way to tell you additional things you should know, and allow you to filter on these attributes.” - Sridhar Ramaswamy, CEO of Neeva

 For example, when searching for legal information, you can filter to “Legal Providers” to see results from companies that provide legal services, or “User Generated Content” to get discussion forums or blogs.

 Shopping Experience

 One type of query that Neeva does particularly well at is shopping. Neeva knows that reviews are important, so they make it easy to find them.

 The interface is geared towards helping you quickly figure out what products are worth looking more into, and connecting you to good review sources.

 You.com – The App Engine

 When you search a coding problem on Google, you’ll find a Stack Overflow answer. When you search on You.com, the Code Complete app will generate the code snippet from scratch using a language model.

 You.com sees the web as a collection of apps, and their goal is to match your query with the most relevant apps. For coding questions, there is an AI Code Generator app, but Stack Overflow is also an “app” which you can open in the sidebar and easily copy code from.

 The Quick Facts app can give you short factual information, and the Medium app allows you to easily read articles in the sidebar.

 Eventually people will be able to build their own You.com apps with unique interactions, and publish them to the You.com platform.

 Summaries

 For many apps, you’ll be able to open a sidebar that has a summary of the important content from that page.

 For example, if searching for food recipes, the All Recipes app and the New York Times Cooking app both allow you to open the recipe in the sidebar. This summarized view gives you exactly the information you want – a list of ingredients, and the instructions.

 For coding, the Stack Overflow app and W3 Schools app allow you to click a “copy to clipboard” button to get the code.

 Filters

 Filters in You.com are a collection of apps particularly suited to some task. For example, the “YouCode” filter will give you apps like Stack Overflow, Code Complete, and Github, while the “Social” filter allows you to search social media apps.

 YouWrite

 This isn’t exactly a search feature, but it highlights the core strength of You.com, which is their AI capabilities. YouWrite is an AI writing assistant. You tell it what to write about, and it generates some paragraphs for you.

 I told YouWrite to write about why everyone should watch anime, and this is what it gave me:

 There are plenty of reasons why everyone should watch anime. For one, anime is a great form of entertainment. It can be exciting, thrilling, and suspenseful – just like any good movie or TV show. But what makes anime so unique is that it often incorporates elements of Japanese culture and folklore that you won’t find in other forms of entertainment.

 Another reason to watch anime is because it can be educational. Anime has been known to teach people about different cultures, history, and even science! There are also many documentaries available that focus on various aspects of the anime industry – such as how certain shows are made or the history behind specific characters or plot lines.

 Lastly, watching anime can help you learn Japanese! While not all shows are dubbed into English (or any other language), there are many websites where you can find translations for free. And if you’re feeling really ambitious, there are plenty of courses available online which will help you learn to read and write in Japanese too!

 Something like this has the potential to become a powerful research tool. Instead of searching for “World War 2” and trying to find good sources, you can have YouWrite generate an essay about World War 2, and research the arguments it makes as a starting point.

 As an aside, if you were wondering whether the Dead Internet Theory had any chance of being true, here’s your answer.

 Andi – The Conversation Engine

 Do search engines have to be so impersonal? We’re used to just typing our query and getting back a list of links, but Andi flips the script on this.

 Andi gives you a conversational search experience that’s more like “messaging with a smart friend who answers questions and sends you useful links”.

 Andi’s focus is on answering specific questions by extracting an answer from the best source.

 For example, searching “what is the gdp per capita of china vs new zealand” on Google returns a badly formatted table without the answer.

 While searching the same thing on Andi will extract the right information for you.

 Dynamic Results

 The way that Andi generates search results is different from the other search engines on this list. It focuses on understanding the intent and topics of your query, then tries to predict the best sources for your answer. After that, it queries the relevant APIs and vertical search engines directly, generating results in realtime.

 There is no index.

 Reader View

 Another feature of Andi worth mentioning is the reader view, which allows you to view the content of pages without having to open them. This allows you to have a consistent browsing experience, and avoid ads and clutter.

 Brave Search – The Independent Engine

 It’s time to address the elephant in the room. Every search engine I’ve mentioned so far, no matter how many cool features they’re building, does not have a fully independent index.

 They’re all getting some portion of their results from Google or Bing.

 Building an independent index is a very hard problem given the current size of the web. You can count the number of search engines with a fully independent index on one hand.

 Many search engines on this list aspire to have a fully independent index at some point, but none of them are more focused on this than Brave Search.

 For any search you do on Brave, you’ll see what percentage of the results came from their own index. They make this number loud and clear, because they want it to be 100%.

 “Brave Search can operate as stand-alone, the rest cannot as they rely on Google or Bing. Most search engines are not independent search engines, and while they may provide some value, they are qualitatively different from what Brave Search is doing. Independence is not something directly actionable, but it’s a fundamental property. Independence means that Brave Search would continue to work even if Google and Microsoft opposed it. Independence means choice and diversity.” - Josep M. Pujol, Chief of Search at Brave

 You won’t see much cool new functionality on Brave Search, because they are laser focused on the one thing they believe to be important to a search engine.

 Web Discovery Project

 The web discovery project is Brave’s solution to one of the major problems with privacy focused search engines. Google can provide good results because they collect a ton of data on everyone, and they use that to improve the search experience. Privacy focused search engines don’t collect that kind of data, which limits their potential.

 With the web discovery project, users can opt-in to contribute anonymous search and web browsing data to help improve Brave’s results. This gives Brave a way to get the same kind of data that Google uses to optimize queries, but without comprising on user privacy.

 If you don’t like it, you have nothing to worry about, because it’s off by default.

 Goggles – Collaborative Mini Search Engines

 The most important innovation of Brave is yet to be released. Goggles allows users to create mini search engines within Brave. Anyone can create detailed ranking specifications that Brave will use when returning results. People can collaborate on goggles and share them.

 Some examples of goggles that could be created are “top tech blogs”, “drop the top 1000 most popular domains” (which implements the general idea of an older search engine, MillionShort), or “cooking sites my mom would recommend”.

 Goggles are easily toggled on and off, so users can leave and enter different filter bubbles as they please. Unlike Google, these filter bubbles are intentionally created by groups of people, not decided by one centralized algorithm that’s working in the background. Goggles can be a useful tool to explore the web from different perspectives.

 Goggles is effectively making search engine creation accessible to anyone. It’s very hard for a regular person to build an entire search engine from scratch, but it will be easy for anyone to create a goggle that represents their idea of a perfect search engine.

 This Is Just The Beginning

 This new wave of search engines is only just getting started. Many of them have only recently launched.

 Even if they aren’t perfect yet, the paths they’re exploring can lead to promising new innovation in the stagnant search space.

 “The search technology was developed a decade ago. It’s a bet that no one else will come up with a better search technology. So you invest in Google because you’re betting against technological innovation in search.” - Peter Thiel, 2012

 Betting against innovation in search has been a great move for the last ten years. The next ten years look a little bit more hopeful.

 I’ll be posting interviews with the founders of each of these search engines soon. If you’re interested in getting a more in-depth look at their stories, and how they think about things, stay tuned by signing up for my newsletter.

 Appendix 1: Interview with Brave Search

 I have now published a full length interview with Brave Search which you can read here.

 Appendix 2: Interview with Neeva

 You can read a full length interview with Neeva here.

 Appendix 3: Interview with Kagi

 You can read a full length interview with Kagi here.



DuckDuckGo and Bing are not true alternatives – they’re just worse versions of Google.
The next Google can’t just be an input box that spits out links. We need new thinking to create something much better than what came before.
In the last few years, different groups of people came to the same conclusion, and started working on the next generation of search engines.
For this new generation, privacy is necessary, and invasive ads are not an option. But that’s where the commonalities end. Beyond that, they’ve all taken the idea of a search engine in very different directions.
Kagi – The Customization Engine
 
Why should everyone have the same search experience? We all have our own preferences about how things should look and work.
 Kagi is the most customizable search engine ever. You can change everything, from surface level appearances, to the final ranking of results.
“Everybody has different preferences of how they want a search engine to look and feel. Our goal is to provide tools and empower users to do that instead of trying to be smart and creating an average search engine for an average user. There’s no average human.” - Vladimir Prelovac, Founder of Kagi
Visual Customization
On the visual side, you can modify everything about the way things look, even being able to write your own custom CSS.
 
Search Customization
More importantly, you can customize the way the search engine itself works. Do you want search suggestions? How detailed should the snippets be?
 
There’s also a list of widgets you can toggle like “Instant Answers”, “Inline Discussions” and “Listicles”. These widgets are groupings of specific types of results which you can choose to include in the results page. If you never want to see another listicle again in your life, just toggle the listicle widget off.
But you can take the search customization even further than that, down to the level of individual websites. You can create your own domain rankings to tell Kagi what websites should be boosted, and what should be down ranked.
You can even outright block sites you never want to see again (looking at you pinterest), and pin sites if you want them always on top (hello reddit).
 
“Nobody has their home screen on iPhone looking exactly the same. I think people deserve that in search, especially being such an important part of our lives. It’s one of the most used products out there, and it’s astonishing how little customization you can do.” - Vladimir Prelovac, Founder of Kagi
Lenses
Kagi also allows you to create website filters called lenses. In any given lens, you can include or exclude specific sites, and more.
 
 
When searching you can turn on different lenses to see results that match those filters. Kagi comes with pre-built lenses for things like Discussions, Programming Help and Recipes.
Non-commercial Results
Another thing that Kagi focuses on is delivering non-commercial results. Kagi has an entire index of non-commercial websites, which they’ve also made available separately as Teclis. This allows them to find results that you won’t find on Google and most other search engines.
 
For example, when searching for “steve jobs”, the top results are similar to Google with mostly informational stuff. But right after that, we’re greeted by an “Interesting Finds” section, which has a fun blog post by Derek Sivers, an article filled with stories of Steve Jobs in Japan, and some other cool things you can’t find on Google.
Neeva – The Everything Engine
 
Why should a search engine only index the web? A lot of the information you care about is scattered across your email, your note taking app, and other places.
 Neeva is the search engine that allows you to search everything.
Right now you can connect Google Apps, Dropbox, Notion and more.
 
No matter what you’re looking for, or where it is, Neeva will help you find it. Searching your personal documents is as simple as adding “@me” to the query. This gives everyone the ability to have a personal search engine.
 
This is one of many things that makes Neeva interesting.
Instant Inline Results
Since Google is an ads based business, they are incentivized to take you to the search results page, even when it’s obvious what you want.
If you typed in “amazon” hoping to get to amazon.com, Google would first take you to the search results page, so they can show you an ad, which amazon.com has to pay for just to be the first result for their own company name.
On the Neeva browser app, you get instant inline search results as you type, allowing you to go directly to the page you’re looking for. You can do in one click what Google will always want you to do in two clicks.
“We have the opportunity to innovate on the browser in a lot of interesting ways because we’re not beholden to how many search results page views we get with ads on them. Our business model unlocks our ability to help people not have to see the search results page as much.” - Darin Fisher, Head of Neeva Browser
 
And it’s not just a shortcut for navigating to domains. Type “best thai food” and immediately get a link to the relevant yelp page.
 
You can still go to the search results page if you need to, but this feature allows you to be twice as fast as Google.
Similarity Search
Another unique feature, only available on the Neeva browser app for now, is their similarity search functionality called NeevaScope.
Wherever you are on the web, you can tap the NeevaScope button to find similar and related pages to the one you’re currently on. This is powerful for exploration.
 
I tried it out on Tim Urban’s Neuralink post and found some other interesting content, including one that criticized Tim Urban’s post.
Filters
Neeva recognizes that there are different categories of websites that may be of interest to you for a given query, and provides filters to narrow things down.
“Google creates this view, I call it the flat-earth view, where every site is just like every other site. We go out of our way to tell you additional things you should know, and allow you to filter on these attributes.” - Sridhar Ramaswamy, CEO of Neeva
For example, when searching for legal information, you can filter to “Legal Providers” to see results from companies that provide legal services, or “User Generated Content” to get discussion forums or blogs.
 
Shopping Experience
One type of query that Neeva does particularly well at is shopping. Neeva knows that reviews are important, so they make it easy to find them.
 
The interface is geared towards helping you quickly figure out what products are worth looking more into, and connecting you to good review sources.
 
You.com – The App Engine
 
When you search a coding problem on Google, you’ll find a Stack Overflow answer. When you search on You.com, the Code Complete app will generate the code snippet from scratch using a language model.
 
You.com sees the web as a collection of apps, and their goal is to match your query with the most relevant apps. For coding questions, there is an AI Code Generator app, but Stack Overflow is also an “app” which you can open in the sidebar and easily copy code from.
The Quick Facts app can give you short factual information, and the Medium app allows you to easily read articles in the sidebar.
 
Eventually people will be able to build their own You.com apps with unique interactions, and publish them to the You.com platform.
Summaries
For many apps, you’ll be able to open a sidebar that has a summary of the important content from that page.
For example, if searching for food recipes, the All Recipes app and the New York Times Cooking app both allow you to open the recipe in the sidebar. This summarized view gives you exactly the information you want – a list of ingredients, and the instructions.
 
For coding, the Stack Overflow app and W3 Schools app allow you to click a “copy to clipboard” button to get the code.
 
Filters
Filters in You.com are a collection of apps particularly suited to some task. For example, the “YouCode” filter will give you apps like Stack Overflow, Code Complete, and Github, while the “Social” filter allows you to search social media apps.
 
YouWrite
This isn’t exactly a search feature, but it highlights the core strength of You.com, which is their AI capabilities. YouWrite is an AI writing assistant. You tell it what to write about, and it generates some paragraphs for you.
I told YouWrite to write about why everyone should watch anime, and this is what it gave me:
There are plenty of reasons why everyone should watch anime. For one, anime is a great form of entertainment. It can be exciting, thrilling, and suspenseful – just like any good movie or TV show. But what makes anime so unique is that it often incorporates elements of Japanese culture and folklore that you won’t find in other forms of entertainment.
Another reason to watch anime is because it can be educational. Anime has been known to teach people about different cultures, history, and even science! There are also many documentaries available that focus on various aspects of the anime industry – such as how certain shows are made or the history behind specific characters or plot lines.
Lastly, watching anime can help you learn Japanese! While not all shows are dubbed into English (or any other language), there are many websites where you can find translations for free. And if you’re feeling really ambitious, there are plenty of courses available online which will help you learn to read and write in Japanese too!
Something like this has the potential to become a powerful research tool. Instead of searching for “World War 2” and trying to find good sources, you can have YouWrite generate an essay about World War 2, and research the arguments it makes as a starting point.
As an aside, if you were wondering whether the Dead Internet Theory had any chance of being true, here’s your answer.
Andi – The Conversation Engine
 
Do search engines have to be so impersonal? We’re used to just typing our query and getting back a list of links, but Andi flips the script on this.
Andi gives you a conversational search experience that’s more like “messaging with a smart friend who answers questions and sends you useful links”.
Andi’s focus is on answering specific questions by extracting an answer from the best source.
For example, searching “what is the gdp per capita of china vs new zealand” on Google returns a badly formatted table without the answer.
 
While searching the same thing on Andi will extract the right information for you.
 
Dynamic Results
The way that Andi generates search results is different from the other search engines on this list. It focuses on understanding the intent and topics of your query, then tries to predict the best sources for your answer. After that, it queries the relevant APIs and vertical search engines directly, generating results in realtime.
There is no index.
Reader View
Another feature of Andi worth mentioning is the reader view, which allows you to view the content of pages without having to open them. This allows you to have a consistent browsing experience, and avoid ads and clutter.
 
 
Brave Search – The Independent Engine
It’s time to address the elephant in the room. Every search engine I’ve mentioned so far, no matter how many cool features they’re building, does not have a fully independent index.
They’re all getting some portion of their results from Google or Bing.
Building an independent index is a very hard problem given the current size of the web. You can count the number of search engines with a fully independent index on one hand.
Many search engines on this list aspire to have a fully independent index at some point, but none of them are more focused on this than Brave Search.
 
For any search you do on Brave, you’ll see what percentage of the results came from their own index. They make this number loud and clear, because they want it to be 100%.
 
“Brave Search can operate as stand-alone, the rest cannot as they rely on Google or Bing. Most search engines are not independent search engines, and while they may provide some value, they are qualitatively different from what Brave Search is doing. Independence is not something directly actionable, but it’s a fundamental property. Independence means that Brave Search would continue to work even if Google and Microsoft opposed it. Independence means choice and diversity.” - Josep M. Pujol, Chief of Search at Brave
You won’t see much cool new functionality on Brave Search, because they are laser focused on the one thing they believe to be important to a search engine.
Web Discovery Project
The web discovery project is Brave’s solution to one of the major problems with privacy focused search engines. Google can provide good results because they collect a ton of data on everyone, and they use that to improve the search experience. Privacy focused search engines don’t collect that kind of data, which limits their potential.
With the web discovery project, users can opt-in to contribute anonymous search and web browsing data to help improve Brave’s results. This gives Brave a way to get the same kind of data that Google uses to optimize queries, but without comprising on user privacy.
If you don’t like it, you have nothing to worry about, because it’s off by default.
Goggles – Collaborative Mini Search Engines
The most important innovation of Brave is yet to be released. Goggles allows users to create mini search engines within Brave. Anyone can create detailed ranking specifications that Brave will use when returning results. People can collaborate on goggles and share them.
 
Some examples of goggles that could be created are “top tech blogs”, “drop the top 1000 most popular domains” (which implements the general idea of an older search engine, MillionShort), or “cooking sites my mom would recommend”.
Goggles are easily toggled on and off, so users can leave and enter different filter bubbles as they please. Unlike Google, these filter bubbles are intentionally created by groups of people, not decided by one centralized algorithm that’s working in the background. Goggles can be a useful tool to explore the web from different perspectives.
Goggles is effectively making search engine creation accessible to anyone. It’s very hard for a regular person to build an entire search engine from scratch, but it will be easy for anyone to create a goggle that represents their idea of a perfect search engine.
This Is Just The Beginning
This new wave of search engines is only just getting started. Many of them have only recently launched.
Even if they aren’t perfect yet, the paths they’re exploring can lead to promising new innovation in the stagnant search space.
“The search technology was developed a decade ago. It’s a bet that no one else will come up with a better search technology. So you invest in Google because you’re betting against technological innovation in search.” - Peter Thiel, 2012 
Betting against innovation in search has been a great move for the last ten years. The next ten years look a little bit more hopeful.
I’ll be posting interviews with the founders of each of these search engines soon. If you’re interested in getting a more in-depth look at their stories, and how they think about things, stay tuned by signing up for my newsletter.
Appendix 1: Interview with Brave Search
I have now published a full length interview with Brave Search which you can read here.
Appendix 2: Interview with Neeva
You can read a full length interview with Neeva here.
Appendix 3: Interview with Kagi
You can read a full length interview with Kagi here.
The Next Google
https://dkb.io/post/the-next-google
https://alexskra.com/blog/the-mermaid-is-taking-over-google-search-in-norway/
The mermaid is taking over Google search in Norway - ALEXSKRA
2021-07-29
Alex
<div><div>
	
<p>Recently I’ve started seeing a lot of spam in Google Search in Norwegian. I’m not talking about a bad result here and there that ranks terrible. No, I’m talking about a large-scale spam operation that I’ve noticed more and more in recent days.</p>
<p>It’s so bad that I’m convinced that this one spam domain is getting a large cut of all Google Search traffic in Norway. I can search for basically anything and find it in the first few pages with a very high probability.</p>
<p>The domain they use right now is a danish one, havfruen4220.dk. (“Havfruen”, meaning “the mermaid”)</p>
<p>You’re greeted with this image if you visit it directly:</p>
<figure>
<figcaption>It seems like whoever is behind the site has been expecting us.</figcaption></figure>
<p>So I found a site with a silly domain and a silly image that ranks for things on Google Search. Do I have some examples? I sure do.</p>
<h2>Example searches</h2>
<p>Let me demonstrate the scale of the issue with some example searches.</p>
<p>A search for one of my brands? Yep, the spam site ranks as number 10. A large IT consultant firm in Norway(and some other countries)? Number 11. Local newspaper? Top of page 3.</p>
<p>Let’s try another one. Let’s try “REMA 1000” (the largest grocery store chain in Norway). Sure enough, on top of page 5, we have this:</p>
<figure>
<figcaption>A Google search result for “REMA 1000” that points to the spammer’s domain.</figcaption></figure>
<p>Let’s try something completely different and random. Maybe something people are wondering about. Let’s try “How often” and let Google pick a thing for us.</p>
<figure>
<figcaption>Google suggestions for “How often” in Norwegian.</figcaption></figure>
<p>So the first suggested result is “how often should you shower” in Norwegian. Let’s try it.</p>
<figure>
<figcaption>Google search result for “How often should you shower”.</figcaption></figure>
<p>Sure enough, it’s on the first page.</p>
<p>What about “How to calculate percentage” in Norwegian(Google’s top suggestion for when starting typing “Hvordan”)</p>
<figure>
<figcaption>“How to calculate percentage” is Google’s top suggestion for “How”.</figcaption></figure>
<p>Of course, on the first page, we get:</p>
<figure>
<figcaption>The mermaid got content on calculating percentages too.</figcaption></figure>
<p>What about “How often does apple update iOS”(“hvor ofte oppdaterer apple iOS”)?</p>
<p>Result nine and ten:</p>
<figure>
<figcaption>The mermaid even got content about iOS.</figcaption></figure>
<p>Let’s take a look at results from just the mermaid:</p>
<figure>
<figcaption>Just a casual 9,95 million pages, according to Google.</figcaption></figure>
<p>There is a lot of pages. Most of them seem to be relatively new and created in the last few days.</p>
<h2>How is the content generated?</h2>
<p>Just by looking at the results, it doesn’t make much sense. It’s scraped from a bunch of different places. Something is from Twitter, something is from news sites, and something is from other random websites. Content seems to be combined from multiple sources. The page is served with Cloudflare.</p>
<p>Searching for exact strings reveals that there are more domains used. All of them use the TLD .dk.</p>
<p>The thing is, there is no actual content available to us if we visit the page. The page uses cloaking and probably only shows content if you’re visiting from Google crawler IPs.</p>
<p>If I pretend to be Google by changing my user agent I just get the silly image I showed you. If I remove the GoogleBot user agent there is one difference: Javascript is inserted that redirects the user to another page:</p>
<pre><code>var b = "https://havfruen4220.dk/3_5_no_14_-__1627506246/gotodate";
 (/google|yahoo|facebook|vk|mail|alpha|yandex|search|msn|DuckDuckGo|Boardreader|Ask|SlideShare|YouTube|Vimeo|Baidu|AOL|Excite/.test(document.referrer) &amp;&amp; location.href.indexOf(".") != -1) &amp;&amp; (top.location.href = b);</code></pre>
<h2>How do they profit from this?</h2>
<p>After the first redirect, that page redirects the user, again, to other scam domains. Some fake news sites pretend to be one of the most popular Norwegian news sites; others are basic “want to earn money fast online” sites.</p>
<figure>
<figcaption>This is one of the pages asking if I’ve ever earned money online and that registration is free right now. After a few “do you want to earn £25k a month” questions, you’re redirected to another domain.</figcaption></figure>
<figure>
<figcaption>This site is the one the previously shown one redirected me to. Clearly, they want me to purchase bitcoin with them, and that’s a great idea at it’s hitting $500K soon.</figcaption></figure>
<p>Other sites are news sites like this:</p>
<figure>
<figcaption>The website pretends to be “Dagbladet”(Norwegian news site).</figcaption></figure>
<p>The fake news article is usually something like “this Norwegian celebrity reveals how he got rich”. And it usually ends with some crypto scheme.</p>
<p>These scams are old, but they usually don’t rank well. I’ve never seen anything like this. It’s currently a top result for nearly anything on Google search in Norway.</p>
<h2>How are they ranking so well?</h2>
<p>We have all probably heard that Google’s ranking is advanced and that it’s pretty hard to fool it. Someone made it. I’m not going to pretend I know how they did this, but I think I have some ideas.</p>
<p>First of all, the content seems somewhat decent in google search at times, and I’ve clicked it multiple times myself when searching for things. When you click it, it does what it can to block browser navigation so you can’t return to google. The content is also so clearly a scam that I had to read some of it for fun.</p>
<p>I think that Google uses stats on whether the user continued checking more results for that specific search query to determine if the visited result answered the user.</p>
<p>When the website blocks you from going back, Google might think you found the thing you were looking for and use this as a positive signal. This way, the site ranks you even higher. I often forget my exact search query, so I usually don’t search again with that exact phrase if I’m blocked from going back.</p>
<h2>How can Google fix this?</h2>
<p>The simple solution would be to test sites regularly with an unknown IP and common user agent to check that a site isn’t just showing content to Google and gives real users something completely different. That would stop this.</p>
<p>Another thing is that an alarm should probably go off when a new domain takes off like this. The havfruen4220.dk domain is shown for basically anything, so it wouldn’t surprise me if it’s the most shown domain in Google search in Norway right now.</p>
<p>How would you make a profit if you could rank for basically everything on Google search?</p>
<p>Thank’s for reading this weird and messy blog post. I just wanted to share the weirdness. Have a great day! 😊</p>
	</div>||||I||||Skip to the content
 ALEXSKRA
 Web development, tech and thinkering
 Menu
 Close Menu
 Categories
 Other

 The mermaid is taking over Google search in Norway

 * Post author By Alex
 * Post date July 29, 2021
 * 20 Comments on The mermaid is taking over Google search in Norway

 Recently I’ve started seeing a lot of spam in Google Search in Norwegian. I’m not talking about a bad result here and there that ranks terrible. No, I’m talking about a large-scale spam operation that I’ve noticed more and more in recent days.

 It’s so bad that I’m convinced that this one spam domain is getting a large cut of all Google Search traffic in Norway. I can search for basically anything and find it in the first few pages with a very high probability.

 The domain they use right now is a danish one, havfruen4220.dk. (“Havfruen”, meaning “the mermaid”)

 You’re greeted with this image if you visit it directly:

 It seems like whoever is behind the site has been expecting us.

 So I found a site with a silly domain and a silly image that ranks for things on Google Search. Do I have some examples? I sure do.

 Example searches

 Let me demonstrate the scale of the issue with some example searches.

 A search for one of my brands? Yep, the spam site ranks as number 10. A large IT consultant firm in Norway(and some other countries)? Number 11. Local newspaper? Top of page 3.

 Let’s try another one. Let’s try “REMA 1000” (the largest grocery store chain in Norway). Sure enough, on top of page 5, we have this:

 A Google search result for “REMA 1000” that points to the spammer’s domain.

 Let’s try something completely different and random. Maybe something people are wondering about. Let’s try “How often” and let Google pick a thing for us.

 Google suggestions for “How often” in Norwegian.

 So the first suggested result is “how often should you shower” in Norwegian. Let’s try it.

 Google search result for “How often should you shower”.

 Sure enough, it’s on the first page.

 What about “How to calculate percentage” in Norwegian(Google’s top suggestion for when starting typing “Hvordan”)

 “How to calculate percentage” is Google’s top suggestion for “How”.

 Of course, on the first page, we get:

 The mermaid got content on calculating percentages too.

 What about “How often does apple update iOS”(“hvor ofte oppdaterer apple iOS”)?

 Result nine and ten:

 The mermaid even got content about iOS.

 Let’s take a look at results from just the mermaid:

 Just a casual 9,95 million pages, according to Google.

 There is a lot of pages. Most of them seem to be relatively new and created in the last few days.

 How is the content generated?

 Just by looking at the results, it doesn’t make much sense. It’s scraped from a bunch of different places. Something is from Twitter, something is from news sites, and something is from other random websites. Content seems to be combined from multiple sources. The page is served with Cloudflare.

 Searching for exact strings reveals that there are more domains used. All of them use the TLD .dk.

 The thing is, there is no actual content available to us if we visit the page. The page uses cloaking and probably only shows content if you’re visiting from Google crawler IPs.

 If I pretend to be Google by changing my user agent I just get the silly image I showed you. If I remove the GoogleBot user agent there is one difference: Javascript is inserted that redirects the user to another page:

 var b = "https://havfruen4220.dk/3_5_no_14_-__1627506246/gotodate";
 (/google|yahoo|facebook|vk|mail|alpha|yandex|search|msn|DuckDuckGo|Boardreader|Ask|SlideShare|YouTube|Vimeo|Baidu|AOL|Excite/.test(document.referrer) && location.href.indexOf(".") != -1) && (top.location.href = b);

 How do they profit from this?

 After the first redirect, that page redirects the user, again, to other scam domains. Some fake news sites pretend to be one of the most popular Norwegian news sites; others are basic “want to earn money fast online” sites.

 This is one of the pages asking if I’ve ever earned money online and that registration is free right now. After a few “do you want to earn £25k a month” questions, you’re redirected to another domain.

 This site is the one the previously shown one redirected me to. Clearly, they want me to purchase bitcoin with them, and that’s a great idea at it’s hitting $500K soon.

 Other sites are news sites like this:

 The website pretends to be “Dagbladet”(Norwegian news site).

 The fake news article is usually something like “this Norwegian celebrity reveals how he got rich”. And it usually ends with some crypto scheme.

 These scams are old, but they usually don’t rank well. I’ve never seen anything like this. It’s currently a top result for nearly anything on Google search in Norway.

 How are they ranking so well?

 We have all probably heard that Google’s ranking is advanced and that it’s pretty hard to fool it. Someone made it. I’m not going to pretend I know how they did this, but I think I have some ideas.

 First of all, the content seems somewhat decent in google search at times, and I’ve clicked it multiple times myself when searching for things. When you click it, it does what it can to block browser navigation so you can’t return to google. The content is also so clearly a scam that I had to read some of it for fun.

 I think that Google uses stats on whether the user continued checking more results for that specific search query to determine if the visited result answered the user.

 When the website blocks you from going back, Google might think you found the thing you were looking for and use this as a positive signal. This way, the site ranks you even higher. I often forget my exact search query, so I usually don’t search again with that exact phrase if I’m blocked from going back.

 How can Google fix this?

 The simple solution would be to test sites regularly with an unknown IP and common user agent to check that a site isn’t just showing content to Google and gives real users something completely different. That would stop this.

 Another thing is that an alarm should probably go off when a new domain takes off like this. The havfruen4220.dk domain is shown for basically anything, so it wouldn’t surprise me if it’s the most shown domain in Google search in Norway right now.

 How would you make a profit if you could rank for basically everything on Google search?

 Thank’s for reading this weird and messy blog post. I just wanted to share the weirdness. Have a great day! 😊

 * Tags google , SEO , spam
 ← Dynamic DNS(DDNS) with OpenWRT and Cloudflare

 20 replies on “The mermaid is taking over Google search in Norway”

 Josh says:
 July 29, 2021 at 04:30

 Why are you convinced that this is happening for a large portion of Norwegians and not just you or a small group?

 Reply
 Alex says:
 July 29, 2021 at 06:49

 It’s happening across devices, even when those devices are sharing no common identifiers or network. Considering that Google has no way of tying those sessions together, it’s either how Google search is now, or I’m extremely unlucky and get put into a weird Google every single time.

 There might be Google accounts with other behaviour, but those accounts I and others have tested have the same problem as anonymous users.

 So, there are no signs of this not being common behaviour.

 Reply By Post Author
 Luis says:
 July 29, 2021 at 23:15

 I live in Canada and to test this I used Google Norway (for the first time ever) in an incognito tab.

 This domain is extremely prevalent in searches formed in the way the author did, finding its way into first page results surprisingly frequent.

 If you believe that my use of Google Norway somewhat represents what ‘unpersonalized’ search results look like, then I think it’s a fair assumption assumption to make that most Norwegians are also seeing the Mermaid.

 Reply
 Mike Hernandez says:
 July 29, 2021 at 05:42

 Just put of curiosity, did you try on a different machine or a phone? Could it be that your computer is infected, and that is causing the domain to appear in your results?

 Reply
 Alex says:
 July 29, 2021 at 06:28

 Yeah, It’s definitely coming from Google. You might be able to see it yourself by changing your region in Google Search (I haven’t tested).

 Reply By Post Author
 Jorge López says:
 July 29, 2021 at 22:42

 I’m from Spain and staying in California for some months, I use Duckduckgo exclusively. I searched “hvor ofte oppdaterer apple iOS” in DDG with the google bang and the mermaid appeared in number 6.

 Reply
 Ivan Kurmanov says:
 July 29, 2021 at 06:59

 Just in case you’re interested, the first image you show above features several characters of this Russian animated series: https://en.wikipedia.org/wiki/Kikoriki

 Reply
 Ivan says:
 July 29, 2021 at 07:45

 The girl at the left corner is from some unofficial Russian ad, they guy at the right corner I can’t identify.

 Reply
 Jakob says:
 July 29, 2021 at 08:09

 The guy on the right is Adam Sandler

 Reply
 T says:
 July 29, 2021 at 08:10

 Adam Sandler

 Reply
 Ståle Freyer says:
 July 29, 2021 at 08:25

 I can confirm Google’s behaviour. I’ve exactly the same experience (from a Norwegian IP). Bloody irritating.

 Reply
 Underhill says:
 July 29, 2021 at 08:39

 (I posted this on HN as well, but perhaps better here.)

 Interesting, I’ve been seeing the same spam for my Norwegian searches, but with the domain nem-multiservice dot dk, or nem-varmepumper dot dk – presumably another legitimate business’ domain that expired and was grabbed by the scammers. Visiting those domains show the same graphic as shown in the article.

 Almost any Google search I do in Norwegian will have obvious scam sites like these in the top 10 results.

 Other domains part of the same scam that show up in my results today: mariesofie dot dk, bvosvejsogmontage dot dk

 I wonder if it is related to this: https://www.dk-hostmaster.dk/en/news/dk-hostmaster-takes-102-dk-domain-names-foreigner-suspected-bitcoin-fraud

 Reply
 Alex says:
 July 29, 2021 at 08:59

 I’ve seen a few other domains too. Some are .it, but most of them are .dk(probably 6-8 domains). All of them seems to be reused.

 Lets hope Google does something, cause this is adding a lot of spam to the Norwegian search.

 Reply By Post Author
 Tommy says:
 July 29, 2021 at 09:28

 The same thing in Denmark – mariannjohann dot dk.. It´s pretty easy to registrer .dk domains. so it is properly why they are used in Norway.

 It has been going on for a while. the best way is to report it to dk-hostmaster and google.

 Thx for the article 🙂

 Tommy

 Reply
 Runar says:
 July 29, 2021 at 10:00

 I’m also seeing a lot of scam-commerncials on YouTube, all served in norwegian, and from .dk-tlds.

 Reply
 Henrik says:
 July 29, 2021 at 14:59

 Same thing for Google Alerts. I keep getting spam alerts from this havfruen-site… I’m considering to remove all the alerts I have set up, as I keep getting new emails every single day.

 Reply
 Jon says:
 July 29, 2021 at 17:28

 There used to be an extension (ID efinmbicabejjhjafeidhfbojhnfiepj ) that allowed people to report spam in search results.

 Announcement (2010): https://developers.google.com/search/blog/2010/11/chrome-extension-for-reporting-webspam

 Now, it seems that the extension is gone & Google’s community moderators are locking questions about it without answering it.

 There’s now an extension by Google called “Suspicious Site Reporter” at https://chrome.google.com/webstore/detail/suspicious-site-reporter/jknemblkbdhdcpllfgbfekkdciegfboi

 The Search Console Webspam Report page is still up – https://www.google.com/webmasters/tools/spamreportform

 For something as intentional as the mermaid site, hopefully they’ll get a “manual action” for “pure spam” ( https://support.google.com/webmasters/answer/9044175#zippy= )

 Reply
 Ian Rosenfeldt says:
 July 29, 2021 at 20:57

 This is also the case in Denmark! Found a couple of new sites today: mariannjohann(dk) and havfruen4220(dk). This needs to stop!

 Reply
 Martin says:
 July 30, 2021 at 18:57

 Hey, there is a LOT of domains like this and I mean a lot. We did some SEO for Norwegian market and its unbelivable what this spammer did!
 Btw, this spammer made a whole network of sites like this accross all EU.

 Reply
 Robert says:
 February 20, 2022 at 13:16

 I recently read this post, and just today I noticed a similar zombie site: thelittleaussiebakery.com. Very strange, this bakery seems to have closed 7 years ago, and you can see at archive.org it used to be a normal webpage for a store. Now it is a mess of auto generated content specifically for SEO. I got to this site while looking up the definition of the mathmatical equivalency symbol…

 Reply

 Leave a Reply

 Your email address will not be published. Required fields are marked *

 Comment *

 Name *

 Email *

 Website

 Save my name, email, and website in this browser for the next time I comment.

 Δ

 Recent Posts

 * The mermaid is taking over Google search in Norway
 * Dynamic DNS(DDNS) with OpenWRT and Cloudflare
 * Ubuntu 22.04 with software RAID1 and UEFI
 * How to manage databases (MariaDB)
 * A clean Ubuntu 20.04 setup

 Recent Comments

 * Cal on Ubuntu 22.04 with software RAID1 and UEFI
 * Dana on Dynamic DNS(DDNS) with OpenWRT and Cloudflare
 * Robert on The mermaid is taking over Google search in Norway
 * KoVaGa on Ubuntu 22.04 with software RAID1 and UEFI
 * Alex on Ubuntu 22.04 with software RAID1 and UEFI

 Categories

 * Gaming
 * Hardware
 * Linux
 * Other
 * Windows

 © 2023 ALEXSKRA

 Powered by WordPress

 To the top ↑ Up ↑
Recently I’ve started seeing a lot of spam in Google Search in Norwegian. I’m not talking about a bad result here and there that ranks terrible. No, I’m talking about a large-scale spam operation that I’ve noticed more and more in recent days.
It’s so bad that I’m convinced that this one spam domain is getting a large cut of all Google Search traffic in Norway. I can search for basically anything and find it in the first few pages with a very high probability.
The domain they use right now is a danish one, havfruen4220.dk. (“Havfruen”, meaning “the mermaid”)
You’re greeted with this image if you visit it directly:
It seems like whoever is behind the site has been expecting us. 
So I found a site with a silly domain and a silly image that ranks for things on Google Search. Do I have some examples? I sure do.
Example searches
Let me demonstrate the scale of the issue with some example searches.
A search for one of my brands? Yep, the spam site ranks as number 10. A large IT consultant firm in Norway(and some other countries)? Number 11. Local newspaper? Top of page 3.
Let’s try another one. Let’s try “REMA 1000” (the largest grocery store chain in Norway). Sure enough, on top of page 5, we have this:
A Google search result for “REMA 1000” that points to the spammer’s domain. 
Let’s try something completely different and random. Maybe something people are wondering about. Let’s try “How often” and let Google pick a thing for us.
Google suggestions for “How often” in Norwegian. 
So the first suggested result is “how often should you shower” in Norwegian. Let’s try it.
Google search result for “How often should you shower”. 
Sure enough, it’s on the first page.
What about “How to calculate percentage” in Norwegian(Google’s top suggestion for when starting typing “Hvordan”)
“How to calculate percentage” is Google’s top suggestion for “How”. 
Of course, on the first page, we get:
The mermaid got content on calculating percentages too. 
What about “How often does apple update iOS”(“hvor ofte oppdaterer apple iOS”)?
Result nine and ten:
The mermaid even got content about iOS. 
Let’s take a look at results from just the mermaid:
Just a casual 9,95 million pages, according to Google. 
There is a lot of pages. Most of them seem to be relatively new and created in the last few days.
How is the content generated?
Just by looking at the results, it doesn’t make much sense. It’s scraped from a bunch of different places. Something is from Twitter, something is from news sites, and something is from other random websites. Content seems to be combined from multiple sources. The page is served with Cloudflare.
Searching for exact strings reveals that there are more domains used. All of them use the TLD .dk.
The thing is, there is no actual content available to us if we visit the page. The page uses cloaking and probably only shows content if you’re visiting from Google crawler IPs.
If I pretend to be Google by changing my user agent I just get the silly image I showed you. If I remove the GoogleBot user agent there is one difference: Javascript is inserted that redirects the user to another page:
 var b = "https://havfruen4220.dk/3_5_no_14_-__1627506246/gotodate";
 (/google|yahoo|facebook|vk|mail|alpha|yandex|search|msn|DuckDuckGo|Boardreader|Ask|SlideShare|YouTube|Vimeo|Baidu|AOL|Excite/.test(document.referrer) &amp;&amp; location.href.indexOf(".") != -1) &amp;&amp; (top.location.href = b); 
How do they profit from this?
After the first redirect, that page redirects the user, again, to other scam domains. Some fake news sites pretend to be one of the most popular Norwegian news sites; others are basic “want to earn money fast online” sites.
This is one of the pages asking if I’ve ever earned money online and that registration is free right now. After a few “do you want to earn £25k a month” questions, you’re redirected to another domain. 
This site is the one the previously shown one redirected me to. Clearly, they want me to purchase bitcoin with them, and that’s a great idea at it’s hitting $500K soon. 
Other sites are news sites like this:
The website pretends to be “Dagbladet”(Norwegian news site). 
The fake news article is usually something like “this Norwegian celebrity reveals how he got rich”. And it usually ends with some crypto scheme.
These scams are old, but they usually don’t rank well. I’ve never seen anything like this. It’s currently a top result for nearly anything on Google search in Norway.
How are they ranking so well?
We have all probably heard that Google’s ranking is advanced and that it’s pretty hard to fool it. Someone made it. I’m not going to pretend I know how they did this, but I think I have some ideas.
First of all, the content seems somewhat decent in google search at times, and I’ve clicked it multiple times myself when searching for things. When you click it, it does what it can to block browser navigation so you can’t return to google. The content is also so clearly a scam that I had to read some of it for fun.
I think that Google uses stats on whether the user continued checking more results for that specific search query to determine if the visited result answered the user.
When the website blocks you from going back, Google might think you found the thing you were looking for and use this as a positive signal. This way, the site ranks you even higher. I often forget my exact search query, so I usually don’t search again with that exact phrase if I’m blocked from going back.
How can Google fix this?
The simple solution would be to test sites regularly with an unknown IP and common user agent to check that a site isn’t just showing content to Google and gives real users something completely different. That would stop this.
Another thing is that an alarm should probably go off when a new domain takes off like this. The havfruen4220.dk domain is shown for basically anything, so it wouldn’t surprise me if it’s the most shown domain in Google search in Norway right now.
How would you make a profit if you could rank for basically everything on Google search?
Thank’s for reading this weird and messy blog post. I just wanted to share the weirdness. Have a great day! 😊
The mermaid is taking over Google search in Norway - ALEXSKRA
https://alexskra.com/blog/the-mermaid-is-taking-over-google-search-in-norway/
https://www.bloomberg.com/news/articles/2019-07-15/to-break-google-s-monopoly-on-search-make-its-index-public
To Break Google’s Monopoly on Search, Make Its Index Public
2019-07-15
Robert Epstein
<div><div><p>Recognition is growing worldwide that something big needs to be done about Big Tech, and fast.</p><p>More than <a href="https://www.bloomberg.com/news/articles/2019-03-20/google-fined-1-7-billion-by-eu-over-advertising-contracts">$8 billion in fines</a> have been levied against <a href="https://www.bloomberg.com/quote/GOOGL:US">Google</a> by the European Union since 2017. <a href="https://www.bloomberg.com/quote/FB:US">Facebook Inc.</a>, facing an onslaught of investigations, has dropped in reputation to almost rock bottom among the 100 most visible companies in the U.S. Former employees of Google and Facebook have warned that these companies are “<a href="https://www.theverge.com/2017/12/11/16761016/former-facebook-exec-ripping-apart-society">ripping apart the social fabric</a>” and can “<a href="http://www.tristanharris.com/2016/05/how-technology-hijacks-peoples-minds%E2%80%8A-%E2%80%8Afrom-a-magician-and-googles-design-ethicist/">hijack the mind</a>.”</p></div></div>||||I|||| Skip to content
 Bloomberg the Company & Its Products The Company & its Products Bloomberg Terminal Demo Request Bloomberg Anywhere Remote Login Bloomberg Anywhere Login Bloomberg Customer Support Customer Support

 * Bloomberg

 Connecting decision makers to a dynamic network of information, people and ideas, Bloomberg quickly and accurately delivers business and financial information, news and insight around the world

 For Customers

 + Bloomberg Anywhere Remote Login
 + Software Updates
 + Manage Products and Account Information

 Support

 Americas +1 212 318 2000

 EMEA +44 20 7330 7500

 Asia Pacific +65 6212 1000

 * Company

 + About
 + Careers
 + Diversity and Inclusion
 + Tech At Bloomberg
 + Philanthropy
 + Sustainability
 + Bloomberg London
 + Bloomberg Beta
 + Gender-Equality Index

 Communications

 + Press Announcements
 + Press Contacts

 Follow

 + Facebook
 + Instagram
 + LinkedIn
 + Twitter
 + YouTube

 * Products

 + Bloomberg Terminal
 + Data
 + Trading
 + Risk
 + Compliance
 + Indices

 Industry Products

 + Bloomberg Law
 + Bloomberg Tax
 + Bloomberg Government
 + BloombergNEF

 * Media

 + Bloomberg Markets
 + Bloomberg Technology
 + Bloomberg Pursuits
 + Bloomberg Politics
 + Bloomberg Opinion
 + Bloomberg Businessweek
 + Bloomberg Live Conferences
 + Bloomberg Radio
 + Bloomberg Television
 + News Bureaus

 Media Services

 + Bloomberg Media Distribution
 + Advertising

 * Company

 + About
 + Careers
 + Diversity and Inclusion
 + Tech At Bloomberg
 + Philanthropy
 + Sustainability
 + Bloomberg London
 + Bloomberg Beta
 + Gender-Equality Index

 Communications

 + Press Announcements
 + Press Contacts

 Follow

 + Facebook
 + Instagram
 + LinkedIn
 + Twitter
 + YouTube

 * Products

 + Bloomberg Terminal
 + Data
 + Trading
 + Risk
 + Compliance
 + Indices

 Industry Products

 + Bloomberg Law
 + Bloomberg Tax
 + Bloomberg Government
 + Bloomberg Environment
 + BloombergNEF

 * Media

 + Bloomberg Markets
 + Bloomberg
 Technology
 + Bloomberg Pursuits
 + Bloomberg Politics
 + Bloomberg Opinion
 + Bloomberg
 Businessweek
 + Bloomberg Live Conferences
 + Bloomberg Radio
 + Bloomberg Television
 + News Bureaus

 Media Services

 + Bloomberg Media Distribution
 + Advertising

 * Bloomberg

 Connecting decision makers to a dynamic network of information, people and ideas, Bloomberg quickly and accurately delivers business and financial information, news and insight around the world

 For Customers

 + Bloomberg Anywhere Remote Login
 + Software Updates
 + Manage Contracts and Orders

 Support

 Americas +1 212 318 2000

 EMEA +44 20 7330 7500

 Asia Pacific +65 6212 1000

 Sign InSubscribe
 Live TV
 * Markets
 * Economics
 * Industries
 * Tech
 * Politics
 * Businessweek
 * Opinion
 * More
 US Edition
 Subscribe
 Businessweek
 Remarks

 To Break Google’s Monopoly on Search, Make Its Index Public

 The tech giant doesn’t have to be dismantled. Sharing its crown jewel might reshape the internet.

 Photo illustration: 731; Photo: Getty Images

 Gift this article
 Have a confidential tip for our reporters? Get in Touch
 Before it’s here, it’s on the Bloomberg Terminal
 LEARN MORE
 By Robert Epstein
 July 15, 2019 at 6:00 AM EDT
 Updated on
 July 16, 2019 at 10:10 AM EDT
 Save
 This article is for subscribers only.

 Recognition is growing worldwide that something big needs to be done about Big Tech, and fast.

 More than $8 billion in fines have been levied against Google by the European Union since 2017. Facebook Inc., facing an onslaught of investigations, has dropped in reputation to almost rock bottom among the 100 most visible companies in the U.S. Former employees of Google and Facebook have warned that these companies are “ripping apart the social fabric” and can “hijack the mind.”

 Have a confidential tip for our reporters? Get in Touch
 Before it’s here, it’s on the Bloomberg Terminal
 LEARN MORE
 HomeBTV+Market DataOpinionAudioOriginalsMagazineEvents
 News
 MarketsEconomicsTechnologyPoliticsGreenCryptoAI
 Work & Life
 WealthPursuitsBusinessweekCityLabEqualityPursuitsWork Shift
 Market Data
 StocksCommoditiesRates & BondsCurrenciesFuturesSectorsEconomic Calendar
 Explore
 NewslettersExplainersThe Big TakeGraphicsSubmit a TipAbout Us
 Terms of ServiceDo Not Sell or Share My Personal InformationTrademarksPrivacy Policy
 CareersMade in NYCAdvertise
 Ad Choices
 Help © 2024 Bloomberg L.P. All Rights Reserved.
Recognition is growing worldwide that something big needs to be done about Big Tech, and fast. More than $8 billion in fines have been levied against Google by the European Union since 2017. Facebook Inc., facing an onslaught of investigations, has dropped in reputation to almost rock bottom among the 100 most visible companies in the U.S. Former employees of Google and Facebook have warned that these companies are “ripping apart the social fabric” and can “hijack the mind.”
To Break Google’s Monopoly on Search, Make Its Index Public
https://www.bloomberg.com/news/articles/2019-07-15/to-break-google-s-monopoly-on-search-make-its-index-public#xj4y7vzkg
https://datasetsearch.research.google.com/
Dataset Search
2023-03-06
None
<div><div><p><a href="https://datasetsearch.research.google.com/help">Learn more</a> about Dataset Search.</p></div></div>||||I|||| Help
 Feedback
 Sign in
 Dataset Search
 Try coronavirus covid-19 or water quality site:canada.ca.
 Learn more about Dataset Search.
 ‫العربية‬‪Deutsch‬‪English‬‪Español (España)‬‪Español (Latinoamérica)‬‪Français‬‪Italiano‬‪日本語‬‪한국어‬‪Nederlands‬Polski‬‪Português‬‪Русский‬‪ไทย‬‪Türkçe‬‪简体中文‬‪中文（香港）‬‪繁體中文‬
Help
 Feedback
 Sign in
 Dataset Search
 Try coronavirus covid-19 or water quality site:canada.ca.
 Learn more about Dataset Search.
 ‫العربية‬‪Deutsch‬‪English‬‪Español (España)‬‪Español (Latinoamérica)‬‪Français‬‪Italiano‬‪日本語‬‪한국어‬‪Nederlands‬Polski‬‪Português‬‪Русский‬‪ไทย‬‪Türkçe‬‪简体中文‬‪中文（香港）‬‪繁體中文‬
Dataset Search
https://datasetsearch.research.google.com/
https://twitter.com/mwseibel/status/1477701120319361026?t=RYjM9qxKd8hmfdLAlc-9AQ&s=19
None
Sun Jan 02 17:59:36 +0000 2022
mwseibel
<div>A recent small medical issue has highlighted how much someone needs to disrupt Google Search.  Google is no longer producing high quality search results in a significant number of important categories.| created_at: Sun Jan 02 17:59:36 +0000 2022 | favorite_count: 7156 | quote_count: 482 | reply_count: 359 | retweet_count: 1227 | is_quote_status: False | retweeted: False | lang: en </div><div>Health, product reviews, recipes are three categories I searched today where top results featured clickbait sites riddled with crappy ads.  I’m sure there are many more.  Feel free to reply to the thread with the categories where you no longer trust Google Search results.| created_at: Sun Jan 02 17:59:36 +0000 2022 | favorite_count: 1832 | quote_count: 35 | reply_count: 88 | retweet_count: 112 | is_quote_status: False | retweeted: False | lang: en </div><div>I’m pretty sure the engineers responsible for Google Search aren’t happy about the quality of results either.  I’m wondering if this isn’t really a tech problem but the influence of some suit responsible for quarterly ad revenue increases.| created_at: Sun Jan 02 18:26:28 +0000 2022 | favorite_count: 1063 | quote_count: 5 | reply_count: 18 | retweet_count: 51 | is_quote_status: False | retweeted: False | lang: en </div><div>What would a paid version of Google Search results look like - where Google can just try to give me the best possible results and not be worried about generating revenue?| created_at: Sun Jan 02 18:32:19 +0000 2022 | favorite_count: 1103 | quote_count: 30 | reply_count: 33 | retweet_count: 66 | is_quote_status: False | retweeted: False | lang: en </div><div>Also, how can a search category be SEO’d into ruin?  Isn’t search engine optimization supposed to produce “better results”?  Doesn’t Google exclusively control the results it displays….| created_at: Sun Jan 02 18:32:19 +0000 2022 | favorite_count: 653 | quote_count: 6 | reply_count: 17 | retweet_count: 33 | is_quote_status: False | retweeted: False | lang: en </div><div>The more I think about this, the more it looks like classic short term thinking.  Juice ad revenue in the short run.  Open the door to complete disruption in the long run…| created_at: Sun Jan 02 18:32:19 +0000 2022 | favorite_count: 701 | quote_count: 2 | reply_count: 6 | retweet_count: 25 | is_quote_status: False | retweeted: False | lang: en </div><div>This is why no software incumbent is truly protected from startup disruption.  Inevitably the rot of a bad incumbent product starts from within…| created_at: Sun Jan 02 18:32:19 +0000 2022 | favorite_count: 794 | quote_count: 10 | reply_count: 16 | retweet_count: 41 | is_quote_status: False | retweeted: False | lang: en </div><div>Does search have network effects?  Sure! Knowing what everyone is searching for (and which results they click on) allows Google to produce better results for me.| created_at: Sun Jan 02 18:41:45 +0000 2022 | favorite_count: 352 | quote_count: 0 | reply_count: 8 | retweet_count: 13 | is_quote_status: False | retweeted: False | lang: en </div><div>In another way it doesn’t - I get no direct benefit if you are using the same search engine as I am.  We don’t interact with each other directly through search.  So if the search results being displayed are no longer high quality… isn’t the network effect broken?| created_at: Sun Jan 02 18:41:45 +0000 2022 | favorite_count: 534 | quote_count: 3 | reply_count: 36 | retweet_count: 13 | is_quote_status: False | retweeted: False | lang: en </div><div>@mwseibel Sorry to hear this. We’re always looking to keep improving our results. If you’re comfortable sharing any of the queries you found disappointing, happy to pass those on so we can look into them…| created_at: Mon Jan 03 03:54:58 +0000 2022 | favorite_count: 89 | quote_count: 0 | reply_count: 43 | retweet_count: 4 | is_quote_status: False | retweeted: False | lang: en </div><div>@dannysullivan I purposely asked people to share their experience so I hope that provides many examples.  To be honest though - I’m sure all the people on your team have examples as well.| created_at: Mon Jan 03 16:50:26 +0000 2022 | favorite_count: 14 | quote_count: 0 | reply_count: 5 | retweet_count: 2 | is_quote_status: False | retweeted: False | lang: en </div>
A recent small medical issue has highlighted how much someone needs to disrupt Google Search.  Google is no longer producing high quality search results in a significant number of important categories.| created_at: Sun Jan 02 17:59:36 +0000 2022 | favorite_count: 7156 | quote_count: 482 | reply_count: 359 | retweet_count: 1227 | is_quote_status: False | retweeted: False | lang: en  Health, product reviews, recipes are three categories I searched today where top results featured clickbait sites riddled with crappy ads.  I’m sure there are many more.  Feel free to reply to the thread with the categories where you no longer trust Google Search results.| created_at: Sun Jan 02 17:59:36 +0000 2022 | favorite_count: 1832 | quote_count: 35 | reply_count: 88 | retweet_count: 112 | is_quote_status: False | retweeted: False | lang: en  I’m pretty sure the engineers responsible for Google Search aren’t happy about the quality of results either.  I’m wondering if this isn’t really a tech problem but the influence of some suit responsible for quarterly ad revenue increases.| created_at: Sun Jan 02 18:26:28 +0000 2022 | favorite_count: 1063 | quote_count: 5 | reply_count: 18 | retweet_count: 51 | is_quote_status: False | retweeted: False | lang: en  What would a paid version of Google Search results look like - where Google can just try to give me the best possible results and not be worried about generating revenue?| created_at: Sun Jan 02 18:32:19 +0000 2022 | favorite_count: 1103 | quote_count: 30 | reply_count: 33 | retweet_count: 66 | is_quote_status: False | retweeted: False | lang: en  Also, how can a search category be SEO’d into ruin?  Isn’t search engine optimization supposed to produce “better results”?  Doesn’t Google exclusively control the results it displays….| created_at: Sun Jan 02 18:32:19 +0000 2022 | favorite_count: 653 | quote_count: 6 | reply_count: 17 | retweet_count: 33 | is_quote_status: False | retweeted: False | lang: en  The more I think about this, the more it looks like classic short term thinking.  Juice ad revenue in the short run.  Open the door to complete disruption in the long run…| created_at: Sun Jan 02 18:32:19 +0000 2022 | favorite_count: 701 | quote_count: 2 | reply_count: 6 | retweet_count: 25 | is_quote_status: False | retweeted: False | lang: en  This is why no software incumbent is truly protected from startup disruption.  Inevitably the rot of a bad incumbent product starts from within…| created_at: Sun Jan 02 18:32:19 +0000 2022 | favorite_count: 794 | quote_count: 10 | reply_count: 16 | retweet_count: 41 | is_quote_status: False | retweeted: False | lang: en  Does search have network effects?  Sure! Knowing what everyone is searching for (and which results they click on) allows Google to produce better results for me.| created_at: Sun Jan 02 18:41:45 +0000 2022 | favorite_count: 352 | quote_count: 0 | reply_count: 8 | retweet_count: 13 | is_quote_status: False | retweeted: False | lang: en  In another way it doesn’t - I get no direct benefit if you are using the same search engine as I am.  We don’t interact with each other directly through search.  So if the search results being displayed are no longer high quality… isn’t the network effect broken?| created_at: Sun Jan 02 18:41:45 +0000 2022 | favorite_count: 534 | quote_count: 3 | reply_count: 36 | retweet_count: 13 | is_quote_status: False | retweeted: False | lang: en  @mwseibel Sorry to hear this. We’re always looking to keep improving our results. If you’re comfortable sharing any of the queries you found disappointing, happy to pass those on so we can look into them…| created_at: Mon Jan 03 03:54:58 +0000 2022 | favorite_count: 89 | quote_count: 0 | reply_count: 43 | retweet_count: 4 | is_quote_status: False | retweeted: False | lang: en  @dannysullivan I purposely asked people to share their experience so I hope that provides many examples.  To be honest though - I’m sure all the people on your team have examples as well.| created_at: Mon Jan 03 16:50:26 +0000 2022 | favorite_count: 14 | quote_count: 0 | reply_count: 5 | retweet_count: 2 | is_quote_status: False | retweeted: False | lang: en
https://twitter.com/mwseibel/status/1477701120319361026
https://brave.com/search-and-web-discovery/
Privacy-preserving Brave Search Replaces Google as the Default Search Engine in the Brave Browser | Brave Browser
2021-10-19
None
<div><div>
 <p><em>Brave also debuts its opt-in Web Discovery Project to anonymously crowdsource improvements to Brave Search</em></p>
 </div><div>
 <p>Starting today, new Brave users will have the search functionality in the Brave browser powered by <a href="/search/">Brave Search</a>, giving them the privacy and independence of a search/browser alternative to Big Tech. Brave Search is built on top of an independent index, and doesnât track users, their searches, or their clicks. Privacy-preserving Brave Search now replaces Google as the default search engine used in the address bar for new Brave users in the United States, Canada, and the United Kingdom; Brave Search as default also replaces Qwant in France, and DuckDuckGo in Germany, with more geographies to be added in the next several months. Existing Brave users will keep their chosen search engine default, and can <a href="https://search.brave.com/default">set Brave Search as the default</a> search engine in Brave or most other major browsers. </p>
 <p>Todayâs Brave desktop browser update (version 1.31), as well as the Brave Android app (version 1.31) and the Brave iOS app (version 1.32) all automatically offer Brave Search as the default for new users in these five countries, with fully localized versions in non-English geographies. Brave users can easily choose a different search option if they prefer by managing their search engine settings. Brave Search is also available in any other browser at <a href="https://search.brave.com/">search.brave.com</a>. </p>
 <p>Brave Search is currently not displaying ads, but the free version of Brave Search will soon be ad-supported. Brave Search will also offer an ad-free Premium version in the near future. </p>
 <p>âBrave Search has grown significantly since its release last June, with nearly 80 million queries per month. Our users are pleased with the comprehensive privacy solution that Brave Search provides against Big Tech by being integrated into our browser,â said Brendan Eich, CEO and co-founder of Brave. âAs we know from experience in many browsers, the default setting is crucial for adoption, and Brave Search has reached the quality and critical mass needed to become our default search option, and to offer our users a seamless privacy-by-default online experience.â</p>
 <p>Brave Search is also launching the <a href="/privacy/browser/#web-discovery-project">Web Discovery Project</a> (WDP), a privacy-preserving system for users to anonymously contribute data to improve Brave Search coverage and quality, and its independence and competitiveness against Big Tech alternatives. The WDP is a strictly opt-in feature and protects user privacy and anonymity by design, so that contributed data cannot be linked to individuals or their devices, and cannot be linked together for a user or set of users; this prevents deanonymization.</p>
 <p>The WDP represents a fundamental shift in how a search index is built to serve relevant results for users. It is a major step forward from the typical server-side aggregation used by the industry. Big Tech search providers collect data from users without asking or notifying users, to continuously grow their indexesâthe list of billions of web pages they draw from to deliver resultsâand to ensure results are relevant and never stale. This data can be, and often is, associated with users personally by an identifier or linkable records. Unlike those search providers, the Web Discovery Project is designed so that all data received is unlinkable, making it impossible to build profiles or sessions of Web Discovery Project contributors. This means thereâs no data for Brave to sell to advertisers, or lose to theft or hacking, or hand over to government agencies, allowing us to keep promises through technology rather than slogans. </p>
 <p>The system contributes anonymous search and browsing data made within the Brave browser from users who have opted in. This data helps build the Brave Search independent index, and ensures Brave Search shows results relevant to search queries. For a URL to be sent, it needs to be visited independently by a large number of people; this is achieved by using the novel <a href="https://arxiv.org/abs/2109.10074">STAR</a> cryptographic protocol. The WDPâs privacy-preserving methodology discards search queries that are too long or suspicious looking. It also discards odd URLs (such as <a href="https://w3ctag.github.io/presentations/reveal/capability-urls.html">capability URLs</a>), URLs of pages that the creator defined as non-indexable, and of course pages that are not public or that require any sort of authentication. The Web Discovery Project runs automatically while a user is browsing the web, so it requires no effort on the part of contributors. Users who have opted in but no longer wish to participate can opt out at any time.</p>
 <p>For more technical details, an overview of the Web Discovery Project <a href="https://github.com/brave/web-discovery-project/blob/main/modules/web-discovery-project/sources/README.md">can be found on Braveâs GitHub repo</a>.</p>
 <p>Brave Search is part of the suite of privacy-preserving tools offered by Brave, as millions of users are turning to alternatives to big tech. Brave also offers <a href="/brave-ads/">Brave Ads</a>, <a href="/announcing-brave-today/">Brave News</a>, a <a href="/brave-and-guardian/">Firewall+VPN service</a>, and the recently-released <a href="https://talk.brave.com/">Brave Talk</a> for unlimited, private video calls.</p>
 </div><div>
 <p>About Brave:</p>
 <p>Brave Softwareâs fast, privacy-oriented browser, combined with its blockchain-based digital advertising platform, is reinventing the Web for users, publishers, and advertisers. Users get a private, speedier web experience with much longer battery life, publishers increase their revenue share, and advertisers achieve better conversion. Users can opt into privacy-respecting ads that reward them with the Basic Attention Token (BAT), a frequent flyer-like token they can redeem or use to tip or contribute to publishers and other content creators. The Brave solution is a win-win for everyone who has a stake in the open Web and who is weary of giving up privacy and revenue to the ad-tech intermediaries. Brave currently has over 38 million monthly active users and over 1.2 million Verified Publishers. Brave Software was co-founded by Brendan Eich, creator of JavaScript and co-founder of Mozilla (Firefox), and Brian Bondy, formerly of Khan Academy and Mozilla.</p>
 <p>For more information, visit <a href="/">brave.com</a> or follow the company on Twitter <a href="https://www.twitter.com/brave/">@brave</a>.</p>
 </div>||||I|||| |||
 * Browser
 Main menu

 Downloads

 + 
 + Desktop
 + Android
 + iOS
 + Linux
 + Beta release
 + Nightly release
 + All downloads
 + 

 Compare

 + 
 + Comparison guides
 Browser

 o What's the best private browser?

 Many users rely on private browsers to keep their Internet habits away from prying eyes. But with so many options out there, what's the best private browser for you? In this article we examine the choices.

 o What's the most secure browser?

 In this article, we'll define what makes a secure browser, and look at some of the best secure browsers available today.

 o Whatâs the fastest browser for surfing the Web?

 In this article, we look at the fastest web browser and discuss some of the factors that affect browser speed, including the browser engine and javascript engine.

 See all Articles
 + Brave vs. Chrome
 + 
 * Products
 Main menu
 + 
 + Search
 + Rewards
 + Wallet
 + Firewall + VPN
 + Talk
 + News
 + Playlist
 + All browser features
 + Basic Attention Token
 + 
 * Privacy
 Main menu
 + 
 + Brave Shields
 + Advanced privacy
 + Privacy updates
 Privacy

 o Blocking annoying and privacy-harming cookie consent banners

 New versions of Brave will hideâand, where possible, completely blockâcookie consent notifications. Brave's approach is distinct and more â¦

 o Grab bag 4: privacy improvements for our iOS browsers make them best-in-class with leading protections

 Recent versions of Brave on iOS include many new privacy features, ensuring that Brave iOS users have the strongest available protections of â¦

 o STAR: Braveâs new system for privacy-preserving data collection

 Brave's new system STAR protects user privacy by ensuring the data users contribute are never unique to that user. This property, sometimes â¦

 See all Articles
 + Web standards
 + Privacy glossary
 + Our privacy policy
 + 
 * Resources
 Main menu
 + 
 + Brave blog
 Resources

 o Brave Wallet adds four new DApps as Wallet partners in January, announces new changes to the program

 o Privacy by Design: Published Proxy Config

 o Important Changes to Brave Rewards

 See all Articles
 + Web3 guides
 Resources

 o What is Web3?

 Web3 is a whole new philosophy for how the web should be managed and how users should access it. It's based on the idea of a version of the Internet that is decentralized and based on public blockchains.

 o Whatâs being built on Web3?

 Web3 relies on new technologies. You may have started to hear a lot more about blockchains, nodes, cryptocurrency and wallets recently? In this article, we'll explain what these core terms are and how they work in Web3.

 o An intro to blockchain and the technology thatâs powering Web3

 Blockchains, Nodes, Cryptocurrency, Walletsâ¦ In this article, weâll define some core Web3 terms, and scratch the surface of decentralized technologies.

 See all Articles
 + Our research
 + 

 Support

 + 
 + Help center home
 + Browser support
 + Rewards support
 + 

 Forums

 + 
 + Community home
 + Feature requests
 + 

 Release notes

 + 
 + Desktop
 + iOS
 + Android
 + 
 * Business
 Main menu

 Advertisers

 + 
 + Overview
 + Ads audience
 + Ad formats
 + Case studies
 Business

 o Verizon R/GA

 Achieved 23.8 million ad views, 8% blended CTR, 57% 10+ second landing visits, and over 1000 Tweets

 o Dentsu

 Engaging the New âAd Chooserâ: A co-authored whitepaper from Brave Software and dentsu international

 o eToro

 Realized 230% increase in new customer registrations and achieved a CTR of 13.8%

 See all Case studies
 + Podcast
 Business

 o The Urgency for Brands to Adopt Emerging Technology

 Thomas Webb, Founder of Worldwide Webb 3, discusses the real fear of missing out â¦

 o Monetizing in the Metaverse

 Justin Hochberg, CEO of the Virtual Brand Group, shares how heâs helping brands â¦

 o From a 2D to 3D Internet: How Marketing Strategies Must Shift

 Mark van Rijmenam, Strategic Futurist and Founder of the Digital Futures â¦

 See all Episodes
 + Advertiser resources
 Business

 o Sales One-Sheet

 Available ad units and high-level information about the Brave Ads platform.

 o Brave Ads Gallery

 A gallery of high impact Sponsored Images from top brands using Brave.

 See all Resources
 + 

 Creators

 + 
 + Overview
 + Creators media kit
 + 
 Download Brave Try Brave Search
 Download Brave Try Brave Search

 Privacy-preserving Brave Search Replaces Google as the Default Search Engine in the Brave Browser

 blog > New products & features

 Last updated Oct 19, 2021

 Brave also debuts its opt-in Web Discovery Project to anonymously crowdsource improvements to Brave Search

 Starting today, new Brave users will have the search functionality in the Brave browser powered by Brave Search , giving them the privacy and independence of a search/browser alternative to Big Tech. Brave Search is built on top of an independent index, and doesnât track users, their searches, or their clicks. Privacy-preserving Brave Search now replaces Google as the default search engine used in the address bar for new Brave users in the United States, Canada, and the United Kingdom; Brave Search as default also replaces Qwant in France, and DuckDuckGo in Germany, with more geographies to be added in the next several months. Existing Brave users will keep their chosen search engine default, and can set Brave Search as the default search engine in Brave or most other major browsers .

 Todayâs Brave desktop browser update (version 1.31), as well as the Brave Android app (version 1.31) and the Brave iOS app (version 1.32) all automatically offer Brave Search as the default for new users in these five countries, with fully localized versions in non-English geographies. Brave users can easily choose a different search option if they prefer by managing their search engine settings. Brave Search is also available in any other browser at search.brave.com .

 Brave Search is currently not displaying ads, but the free version of Brave Search will soon be ad-supported. Brave Search will also offer an ad-free Premium version in the near future .

 âBrave Search has grown significantly since its release last June, with nearly 80 million queries per month. Our users are pleased with the comprehensive privacy solution that Brave Search provides against Big Tech by being integrated into our browser,â said Brendan Eich, CEO and co-founder of Brave. âAs we know from experience in many browsers, the default setting is crucial for adoption, and Brave Search has reached the quality and critical mass needed to become our default search option, and to offer our users a seamless privacy-by-default online experience.â

 Brave Search is also launching the Web Discovery Project (WDP), a privacy-preserving system for users to anonymously contribute data to improve Brave Search coverage and quality, and its independence and competitiveness against Big Tech alternatives. The WDP is a strictly opt-in feature and protects user privacy and anonymity by design, so that contributed data cannot be linked to individuals or their devices, and cannot be linked together for a user or set of users; this prevents deanonymization.

 The WDP represents a fundamental shift in how a search index is built to serve relevant results for users. It is a major step forward from the typical server-side aggregation used by the industry. Big Tech search providers collect data from users without asking or notifying users, to continuously grow their indexesâthe list of billions of web pages they draw from to deliver resultsâand to ensure results are relevant and never stale. This data can be, and often is, associated with users personally by an identifier or linkable records. Unlike those search providers, the Web Discovery Project is designed so that a ll data received is unlinkable, making it impossible to build profiles or sessions of Web Discovery Project contributors. This means thereâs no data for Brave to sell to advertisers, or lose to theft or hacking, or hand over to government agencies, allowing us to keep promises through technology rather than slogans.

 The system contributes anonymous search and browsing data made within the Brave browser from users who have opted in. This data helps build the Brave Search independent index, and ensures Brave Search shows results relevant to search queries. For a URL to be sent, it needs to be visited independently by a large number of people; this is achieved by using the novel STAR cryptographic protocol. The WDPâs privacy-preserving methodology discards search queries that are too long or suspicious looking. It also discards odd URLs (such as capability URLs ), URLs of pages that the creator defined as non-indexable, and of course pages that are not public or that require any sort of authentication. The Web Discovery Project runs automatically while a user is browsing the web, so it requires no effort on the part of contributors. Users who have opted in but no longer wish to participate can opt out at any time.

 For more technical details, an overview of the Web Discovery Project can be found on Braveâs GitHub repo .

 Brave Search is part of the suite of privacy-preserving tools offered by Brave, as millions of users are turning to alternatives to big tech. Brave also offers Brave Ads , Brave News , a Firewall+VPN service , and the recently-released Brave Talk for unlimited, private video calls.

 About Brave:

 Brave Softwareâs fast, privacy-oriented browser, combined with its blockchain-based digital advertising platform, is reinventing the Web for users, publishers, and advertisers. Users get a private, speedier web experience with much longer battery life, publishers increase their revenue share, and advertisers achieve better conversion. Users can opt into privacy-respecting ads that reward them with the Basic Attention Token (BAT), a frequent flyer-like token they can redeem or use to tip or contribute to publishers and other content creators. The Brave solution is a win-win for everyone who has a stake in the open Web and who is weary of giving up privacy and revenue to the ad-tech intermediaries. Brave currently has over 38 million monthly active users and over 1.2 million Verified Publishers. Brave Software was co-founded by Brendan Eich, creator of JavaScript and co-founder of Mozilla (Firefox), and Brian Bondy, formerly of Khan Academy and Mozilla.

 For more information, visit brave.com or follow the company on Twitter @brave .

Related articles

 Brave Adds Support for Bridges in Private Windows with Tor

 Bridges are a way for users to access the Tor network even when their government is blocking connections to the network.

 Read this article â

 FrodoPIR: a new privacy-preserving approach for retrieving data

 Introducing FrodoPIR, a Private Information Retrieval scheme for a variety of use-cases, such as Safe Browsing, checking certificate revocation, passwords over breached databases, streaming, and more.

 Read this article â

 Brave News adds new features for customizing feeds, and support for more countries

 Today Brave News introduced integrated RSS subscriptions, locally personalized suggestions, new sources and content channels powered by Brave Search, and more.

 Read this article â

 Ready to Brave the new internet?

 Brave is built by a team of privacy focused, performance oriented pioneers of the web. Help us fix browsing together.

 Download Brave
 Download Brave

 Resources

 * F.A.Q
 * Help center
 * Community
 * Status
 * Transparency report
 * Learn

 Privacy policy

 * Brave Browser
 * Website & email
 * Publishers & creators
 * Advertisers

 Browser

 * Brave Release
 * Brave Beta
 * Brave Nightly

 Company

 * About
 * Research
 * Careers
 * Brand assets
 * Ads audience
 * Ad formats
 * GitHub
 * Blog
 * Tor Onion address

 Social media

 * Reddit
 * Twitter
 * Mastodon
 * Facebook
 * YouTube

 Products

 * Search
 * Rewards
 * Wallet
 * Firewall + VPN
 * Talk
 * News
 * Playlist
 * All browser features
 * Basic Attention Token

 Contact

 Support

 community.brave.com

 Advertising

 adsales@trybrave.com Please only use this email address if you are interested in purchasing advertising with Brave. For support, please visit community.brave.com.

 Business

 bizdev@brave.com

 Press

 press@brave.com

 Developer docs

 * Wallet docs

 Offices

 Brave San Francisco

 580 Howard St. Unit 402,
 San Francisco, CA 94105

 Choose a language
 English
 Terms of use | Report a security issue Â© 2015 - 2023 Brave Software, Inc. | All rights reserved

 Almost thereâ¦

 Youâre just 60 seconds away from the best privacy online

 If your download didnât start automatically, click here .

 1. Download Brave

 Click âSaveâ in the window that pops up, and wait for the download to complete.

 Wait for the download to complete (you may need to click âSaveâ in a window that pops up).

 2. Run the installer

 Click the downloaded file at the bottom left of your screen, and follow the instructions to install Brave.

 Click the downloaded file at the top right of your screen, and follow the instructions to install Brave.

 Click the downloaded file, and follow the instructions to install Brave.

 3. Import settings

 During setup, import bookmarks, extensions, & passwords from your old browser.

 Need help?

 Get better privacy. Everywhere!

 Download Brave mobile for privacy on the go.

 Get better privacy. Everywhere!

 Download Brave mobile for privacy on the go.

 Click this file to install Brave
 Click this file to install Brave
12
Brave also debuts its opt-in Web Discovery Project to anonymously crowdsource improvements to Brave Search 
  
 Starting today, new Brave users will have the search functionality in the Brave browser powered by Brave Search, giving them the privacy and independence of a search/browser alternative to Big Tech. Brave Search is built on top of an independent index, and doesnât track users, their searches, or their clicks. Privacy-preserving Brave Search now replaces Google as the default search engine used in the address bar for new Brave users in the United States, Canada, and the United Kingdom; Brave Search as default also replaces Qwant in France, and DuckDuckGo in Germany, with more geographies to be added in the next several months. Existing Brave users will keep their chosen search engine default, and can set Brave Search as the default search engine in Brave or most other major browsers. 
 Todayâs Brave desktop browser update (version 1.31), as well as the Brave Android app (version 1.31) and the Brave iOS app (version 1.32) all automatically offer Brave Search as the default for new users in these five countries, with fully localized versions in non-English geographies. Brave users can easily choose a different search option if they prefer by managing their search engine settings. Brave Search is also available in any other browser at search.brave.com. 
 Brave Search is currently not displaying ads, but the free version of Brave Search will soon be ad-supported. Brave Search will also offer an ad-free Premium version in the near future. 
 âBrave Search has grown significantly since its release last June, with nearly 80 million queries per month. Our users are pleased with the comprehensive privacy solution that Brave Search provides against Big Tech by being integrated into our browser,â said Brendan Eich, CEO and co-founder of Brave. âAs we know from experience in many browsers, the default setting is crucial for adoption, and Brave Search has reached the quality and critical mass needed to become our default search option, and to offer our users a seamless privacy-by-default online experience.â
 Brave Search is also launching the Web Discovery Project (WDP), a privacy-preserving system for users to anonymously contribute data to improve Brave Search coverage and quality, and its independence and competitiveness against Big Tech alternatives. The WDP is a strictly opt-in feature and protects user privacy and anonymity by design, so that contributed data cannot be linked to individuals or their devices, and cannot be linked together for a user or set of users; this prevents deanonymization.
 The WDP represents a fundamental shift in how a search index is built to serve relevant results for users. It is a major step forward from the typical server-side aggregation used by the industry. Big Tech search providers collect data from users without asking or notifying users, to continuously grow their indexesâthe list of billions of web pages they draw from to deliver resultsâand to ensure results are relevant and never stale. This data can be, and often is, associated with users personally by an identifier or linkable records. Unlike those search providers, the Web Discovery Project is designed so that all data received is unlinkable, making it impossible to build profiles or sessions of Web Discovery Project contributors. This means thereâs no data for Brave to sell to advertisers, or lose to theft or hacking, or hand over to government agencies, allowing us to keep promises through technology rather than slogans. 
 The system contributes anonymous search and browsing data made within the Brave browser from users who have opted in. This data helps build the Brave Search independent index, and ensures Brave Search shows results relevant to search queries. For a URL to be sent, it needs to be visited independently by a large number of people; this is achieved by using the novel STAR cryptographic protocol. The WDPâs privacy-preserving methodology discards search queries that are too long or suspicious looking. It also discards odd URLs (such as capability URLs), URLs of pages that the creator defined as non-indexable, and of course pages that are not public or that require any sort of authentication. The Web Discovery Project runs automatically while a user is browsing the web, so it requires no effort on the part of contributors. Users who have opted in but no longer wish to participate can opt out at any time.
 For more technical details, an overview of the Web Discovery Project can be found on Braveâs GitHub repo.
 Brave Search is part of the suite of privacy-preserving tools offered by Brave, as millions of users are turning to alternatives to big tech. Brave also offers Brave Ads, Brave News, a Firewall+VPN service, and the recently-released Brave Talk for unlimited, private video calls.
  
 About Brave:
 Brave Softwareâs fast, privacy-oriented browser, combined with its blockchain-based digital advertising platform, is reinventing the Web for users, publishers, and advertisers. Users get a private, speedier web experience with much longer battery life, publishers increase their revenue share, and advertisers achieve better conversion. Users can opt into privacy-respecting ads that reward them with the Basic Attention Token (BAT), a frequent flyer-like token they can redeem or use to tip or contribute to publishers and other content creators. The Brave solution is a win-win for everyone who has a stake in the open Web and who is weary of giving up privacy and revenue to the ad-tech intermediaries. Brave currently has over 38 million monthly active users and over 1.2 million Verified Publishers. Brave Software was co-founded by Brendan Eich, creator of JavaScript and co-founder of Mozilla (Firefox), and Brian Bondy, formerly of Khan Academy and Mozilla.
 For more information, visit brave.com or follow the company on Twitter @brave.
Privacy-preserving Brave Search Replaces Google as the Default Search Engine in the Brave Browser | Brave Browser
https://brave.com/search-and-web-discovery/
https://twitter.com/kashhill/status/1402986015107653647
None
Thu Jun 10 13:44:47 +0000 2021
kashhill
<div>Google has a new concept called "known victims" for revenge porn &amp; people serially attacked on slander sites. Once a person requests removal of these results from a search of their name, Google will automatically suppress similar content from resurfacing. https://www.nytimes.com/2021/06/10/technology/google-algorithm-known-victims.html| created_at: Thu Jun 10 13:44:47 +0000 2021 | favorite_count: 1005 | quote_count: 57 | reply_count: 23 | retweet_count: 348 | is_quote_status: False | retweeted: False | lang: en </div><div>One of the surprising things about working on the slander series is how few people in the field, even experts, know that Google voluntarily removes some search results. (No court order needed!) You have to visit this generic url: https://support.google.com/websearch/troubleshooter/3111061?hl=en https://twitter.com/kashhill/status/1402986015107653647/photo/1| created_at: Thu Jun 10 13:48:26 +0000 2021 | favorite_count: 573 | quote_count: 61 | reply_count: 9 | retweet_count: 227 | is_quote_status: False | retweeted: False | lang: en </div><div>Because so few people know about it, "reputation managers" are charging people like $500 a pop to "remove damaging information from Google results." And ALL THEY DO is fill out that form for free. Someone tried to hawk this service to my husband after he came under attack.| created_at: Thu Jun 10 13:50:23 +0000 2021 | favorite_count: 266 | quote_count: 3 | reply_count: 5 | retweet_count: 65 | is_quote_status: False | retweeted: False | lang: en </div><div>The reason Google changes are meaningful (assuming they work) is that when someone comes after you online, it tends to be repeatedly. Plus sites scrape each other, compounding the damage, so you fill out these forms again &amp; again &amp; again. "Known victims" theoretically fixes that.| created_at: Thu Jun 10 14:56:59 +0000 2021 | favorite_count: 148 | quote_count: 4 | reply_count: 7 | retweet_count: 22 | is_quote_status: False | retweeted: False | lang: en </div><div>Google blog post on the changes: "Improving Search to better protect people from harassment" https://blog.google/products/search/improving-search-better-protect-people-harassment/ 

It is WILD how much the internet has changed in a decade.| created_at: Thu Jun 10 19:18:03 +0000 2021 | favorite_count: 134 | quote_count: 3 | reply_count: 5 | retweet_count: 34 | is_quote_status: False | retweeted: False | lang: en </div><div>@kashhill Key word is “some.” All their policies (except csam) reserve their right to discretion. And as a result, they ignore most requests and deny w/o explanation or process| created_at: Fri Jun 11 12:53:10 +0000 2021 | favorite_count: 14 | quote_count: 0 | reply_count: 1 | retweet_count: 0 | is_quote_status: False | retweeted: False | lang: en </div><div>@kashhill And of course “no court order needed” suggests a reality where a court order is even an option. As in hassel v Byrd, you can’t get injunctive relief against Google b/c 230. And let’s say you only sue the abuser or defamer and get a court order saying yes this is defamation/abuse| created_at: Fri Jun 11 12:56:49 +0000 2021 | favorite_count: 7 | quote_count: 0 | reply_count: 1 | retweet_count: 0 | is_quote_status: False | retweeted: False | lang: en </div><div>@kashhill hmmm... i wonder if google would remove the reputation attack against me that's hosted on their own blogging platform| created_at: Thu Jun 10 13:56:36 +0000 2021 | favorite_count: 1 | quote_count: 0 | reply_count: 0 | retweet_count: 0 | is_quote_status: False | retweeted: False | lang: en </div><div>@kashhill I'm the director of Free Law Project, a nonprofit where we post millions of court records. We get a lot of removal requests which we diligently respond to. Unfortunately, Google lacks an API so we can ask for these pages to be removed. We mark them as private, but...| created_at: Thu Jun 10 14:29:59 +0000 2021 | favorite_count: 10 | quote_count: 0 | reply_count: 1 | retweet_count: 0 | is_quote_status: False | retweeted: False | lang: en </div><div>@kashhill @retheauditors I had no idea this existed. Thanks for spreading the word.| created_at: Thu Jun 10 18:36:37 +0000 2021 | favorite_count: 0 | quote_count: 0 | reply_count: 0 | retweet_count: 0 | is_quote_status: False | retweeted: False | lang: en </div><div>@kashhill I ❤️ your work. You’ve been instrumental in effecting change. We’re all indebted. 🙏🏼 But referring to Google acting “voluntarily” (w/o a court order) is misleading. We EXPECT every company to exercise judgment &amp; not facilitate harm. That’s SOP for any non-tech co.
@cagoldberglaw| created_at: Fri Jun 11 12:47:38 +0000 2021 | favorite_count: 2 | quote_count: 0 | reply_count: 0 | retweet_count: 1 | is_quote_status: False | retweeted: False | lang: en </div><div>@kashhill What they don't tell people is that all the website has to do to thwart that form is change the URL. When Google removed content from one of my websites years ago due to a court order, I wrote a script to change URLs on posts like that every few weeks. Worked great, still does!| created_at: Sun Jun 13 06:40:17 +0000 2021 | favorite_count: 0 | quote_count: 0 | reply_count: 0 | retweet_count: 0 | is_quote_status: False | retweeted: False | lang: en </div>
Google has a new concept called "known victims" for revenge porn &amp; people serially attacked on slander sites. Once a person requests removal of these results from a search of their name, Google will automatically suppress similar content from resurfacing. https://www.nytimes.com/2021/06/10/technology/google-algorithm-known-victims.html| created_at: Thu Jun 10 13:44:47 +0000 2021 | favorite_count: 1005 | quote_count: 57 | reply_count: 23 | retweet_count: 348 | is_quote_status: False | retweeted: False | lang: en  One of the surprising things about working on the slander series is how few people in the field, even experts, know that Google voluntarily removes some search results. (No court order needed!) You have to visit this generic url: https://support.google.com/websearch/troubleshooter/3111061?hl=en https://twitter.com/kashhill/status/1402986015107653647/photo/1| created_at: Thu Jun 10 13:48:26 +0000 2021 | favorite_count: 573 | quote_count: 61 | reply_count: 9 | retweet_count: 227 | is_quote_status: False | retweeted: False | lang: en  Because so few people know about it, "reputation managers" are charging people like $500 a pop to "remove damaging information from Google results." And ALL THEY DO is fill out that form for free. Someone tried to hawk this service to my husband after he came under attack.| created_at: Thu Jun 10 13:50:23 +0000 2021 | favorite_count: 266 | quote_count: 3 | reply_count: 5 | retweet_count: 65 | is_quote_status: False | retweeted: False | lang: en  The reason Google changes are meaningful (assuming they work) is that when someone comes after you online, it tends to be repeatedly. Plus sites scrape each other, compounding the damage, so you fill out these forms again &amp; again &amp; again. "Known victims" theoretically fixes that.| created_at: Thu Jun 10 14:56:59 +0000 2021 | favorite_count: 148 | quote_count: 4 | reply_count: 7 | retweet_count: 22 | is_quote_status: False | retweeted: False | lang: en  Google blog post on the changes: "Improving Search to better protect people from harassment" https://blog.google/products/search/improving-search-better-protect-people-harassment/ 
It is WILD how much the internet has changed in a decade.| created_at: Thu Jun 10 19:18:03 +0000 2021 | favorite_count: 134 | quote_count: 3 | reply_count: 5 | retweet_count: 34 | is_quote_status: False | retweeted: False | lang: en  @kashhill Key word is “some.” All their policies (except csam) reserve their right to discretion. And as a result, they ignore most requests and deny w/o explanation or process| created_at: Fri Jun 11 12:53:10 +0000 2021 | favorite_count: 14 | quote_count: 0 | reply_count: 1 | retweet_count: 0 | is_quote_status: False | retweeted: False | lang: en  @kashhill And of course “no court order needed” suggests a reality where a court order is even an option. As in hassel v Byrd, you can’t get injunctive relief against Google b/c 230. And let’s say you only sue the abuser or defamer and get a court order saying yes this is defamation/abuse| created_at: Fri Jun 11 12:56:49 +0000 2021 | favorite_count: 7 | quote_count: 0 | reply_count: 1 | retweet_count: 0 | is_quote_status: False | retweeted: False | lang: en  @kashhill hmmm... i wonder if google would remove the reputation attack against me that's hosted on their own blogging platform| created_at: Thu Jun 10 13:56:36 +0000 2021 | favorite_count: 1 | quote_count: 0 | reply_count: 0 | retweet_count: 0 | is_quote_status: False | retweeted: False | lang: en  @kashhill I'm the director of Free Law Project, a nonprofit where we post millions of court records. We get a lot of removal requests which we diligently respond to. Unfortunately, Google lacks an API so we can ask for these pages to be removed. We mark them as private, but...| created_at: Thu Jun 10 14:29:59 +0000 2021 | favorite_count: 10 | quote_count: 0 | reply_count: 1 | retweet_count: 0 | is_quote_status: False | retweeted: False | lang: en  @kashhill @retheauditors I had no idea this existed. Thanks for spreading the word.| created_at: Thu Jun 10 18:36:37 +0000 2021 | favorite_count: 0 | quote_count: 0 | reply_count: 0 | retweet_count: 0 | is_quote_status: False | retweeted: False | lang: en  @kashhill I ❤️ your work. You’ve been instrumental in effecting change. We’re all indebted. 🙏🏼 But referring to Google acting “voluntarily” (w/o a court order) is misleading. We EXPECT every company to exercise judgment &amp; not facilitate harm. That’s SOP for any non-tech co.
@cagoldberglaw| created_at: Fri Jun 11 12:47:38 +0000 2021 | favorite_count: 2 | quote_count: 0 | reply_count: 0 | retweet_count: 1 | is_quote_status: False | retweeted: False | lang: en  @kashhill What they don't tell people is that all the website has to do to thwart that form is change the URL. When Google removed content from one of my websites years ago due to a court order, I wrote a script to change URLs on posts like that every few weeks. Worked great, still does!| created_at: Sun Jun 13 06:40:17 +0000 2021 | favorite_count: 0 | quote_count: 0 | reply_count: 0 | retweet_count: 0 | is_quote_status: False | retweeted: False | lang: en
https://twitter.com/kashhill/status/1402986015107653647