Seeks Project: Let's take back Websearch
CC Attribution 2.0 Belgium:
You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor.
Benazera, Emmanuel
Conventional websearch is a solitary and locked place. Searchers are forced to interact with remote proprietary black boxes with no possible feedback, and are isolated from each others. There's an estimated billion queries issued everyday by several hundred millions of users. Those queries are grounded into real life events and activities common to many searchers. However, users cannot share their experience and build the results together. Seeks develops a decentralized architecture for enabling collaborative websearch. This short talk intends to outline the Seeks architecture, and to show in simple words how Seeks allows its users to issue queries and get answers from other users while not revealing the true query itself. Seeks is an open peer-to-peer architecture and a free software application for enabling collaborative websearch. Its specific purpose is to regroup users whose queries are similar so they can share both the query results and their experience on these results. On this basis, Seeks allows for true real-time, decentralized, websearch to emerge. In the long term, there is no need for web crawlers and third-party web indexes as users ca npush content directly to the groups of seekers. By hashing queries, Seeks offers a unique solution that allows to issue queries and get answers while not revealing the true query itself. In its present state, Seeks is a free software meta-search engine that can be deployed on local or remote machines. Seeks supports a fair set of options, from thumbnails to more advanced and experimental features such as content similarity analysis, automated clustering and personalization of results. While in its early phase, Seeks aims at demonstrate that there exists a path toward collaborative, efficient, free software powered websearch on the Internet.
 I'm going to give you a very short and brief overview of the six projects. And what you'll get is sort of a one path to possible take back of web search, the way Firefox took back the web. This take back is a long, long work. It's a hard work. It'll be difficult, but that's the path that I know and that's the path that I work with with a bunch of people. So this is what I'm going to show you. And maybe you'll be convinced that the first software community and us and hackers and developers, blah, blah, blah, we can do something about it. Oops. Okay. So as you know, we have something fantastic at our fingertips. We have the most connected human built network of recorded history. It's the internet. And on this network, we have basically a massive flow of data. And this data, we can categorize them in two categories. Human generated and human targeted data and machine to machine data. So in this context, what is search? Search is this basic operation for humans to look up some human targeted content throughout this massive amount of data. Search is this fascinating characteristic that we all depend on it. Most of the time we look at, we ask something to Google or Bing or whatever, search engine before we even ask our friends. So we're really dependent on these engines. And also the level of the technicality that is required and the usage that is required by users to use this engine is as low-virus. What it means is that whether you're my mother or you're a power searcher, so-called power searcher, the tools that you're using, the way you're using Google or any other engine, it's almost the same, right? The power searcher may be a little bit better, but it's almost the same. So what happens is we are all dependent on search and we all use it. And if you're lucky to not have understood or seen this before, as users we're locked up and we're locked up in several ways. First because we're interacting with black boxes, proprietary black boxes, which means that basically you've all experienced this, typing in some keywords, getting some results back, and are satisfied with the results, you change the keyword, you get another query and so on and so on, and you iterate that until you get satisfied. Sometimes you just get garbage, right? And the machine will never tell you that you got garbage because the machine will never tell you that she doesn't know, but it doesn't know actually. So you can give feedback. That's the first lockup. The second lockup is very important to me. It's basically we're all doing the same queries over and over again in our own corner. We're isolated. If I look for something and I read the form and I find at the end of the form that the answer to my question is not there, the hundred or the thousand people behind me who are going to do the same query, well, they'll do the same thing, right? They'll go through the form and read it again. So we're isolated. The third lockup is that we're all experts in something, right? We look things on, we type in queries to these search engines and we're all experts in something, but we distill our queries to these search engines and we keep nothing on our machines, right? So we don't build profiles of expertise that should be ours. So the point here is truly, web search is lagging behind and there are reasons for this. I'll try to tell you what. But the way I can convince you that it's lagging behind is that we're millions, even billions of users and every day there is an estimated one billion query being issued on the search engines. And of course these queries, they're rooted in reality, right? We're looking in the info sphere for things that are real, recipes, medicines, whatever, right? But we're looking for something real. And since we're looking for something real, well, many of us, we're looking for similar things, right? But we cannot interact. There's no feedback. We cannot share our queries like we can share something else. So what is 6? 6 is an open architecture and it's a free software application that tries to fill the gap in web search by making it collaborative, making it social. And the main idea is that as a social web search is inevitable. It will happen. Whether 6 succeeds or whether some other people come up with better ideas, it doesn't matter. In the end, we'll do search together, okay? The way we do some other things together. The idea behind 6, the main idea is very, very simple. It's very natural too. It is to regroup users who perform similar or identical queries so they can share and collaborate on the results. That's all. This simple idea has many interesting consequences. So I'll try to describe 6 with just this slide very briefly. Basically, what happens is 6 regroups people in real time in so-called search groups around similar queries, okay? So people who type similar queries, they get regrouped in search groups. These search groups, these people in the search groups, they can work together. So we give them collaborative filtering tools to work together. So these tools, they allow them to exchange results automatically or actively by clicking to recommend results, to toss results away, to do a lot of things. So basically, it gives the ability to these people to process data. And the third point is very important is we allow some users to push some more data to the search groups for them to process. So let me give you an example. If you have a blog and you write a new entry to your blog, what you're going to do is you're going to devise a few queries, at least one query that describes your new entry of your blog. You're going to do that, you know, type in that query into 6. You're going to be regrouped with people who did similar queries. But instead of looking at the results that you get from the search engine, you'll push your content. Okay? You'll say, hey, this is a potential new result for you guys. If you've been lying, it means if your query is not in any equation with the content, you'll get tossed. Okay? People will say, okay, that's garbage. We don't want it. If you do this, the groups, the filtering, and the publication to the groups, you can build a real-time collaborative and fair search engine. Fair because anyone can push some data. Okay. Here is the important thing. Why do we need to do it? If social web search, you know, social web search is a simple idea. Why doesn't it exist? You know, all these companies, they could do it. There's one reason. The business model of these companies, right? The existing engines, they feed on user personal data in exchange for the service that they give you. These data, they used to target advertising and generate revenues for these engines, right? So if you let users share their queries and, you know, be clustered into groups of people with similar interests. And if you let people push them some content, give them some more content and results, like advertisers, then these advertisers, they don't need to go through the existing paywalls, like AdWords or some others. They don't need to go to these centralized entities and pay the fee to be able to push their results. So basically what happens is that these existing search engines, because of the business models that they have, that work, but, you know, because these business models, they will never do it. And the proof, the fact is that today was things that you may like or don't like, doesn't matter, like Facebook and others, they're still lagging behind. They haven't done it, okay? And the lockup is this. It's because they simply cannot do it. Okay, six builds on a few principles. I will outline them very quickly. The first one is if you want to take back web search, you've got to give more control to users over the search algorithms. What SIX does is that it groups, it regroups the people so they're together. Once they're together, they can work together and they have more control over their results and the algorithms. The second thing is transparency. Transparency means that you're able to modify the equations, the ranking, the way the ranking is done in the engine freely. You do not have to rely on proprietary code and equations and let your results being ranked by them. So the way SIX does this is that it's a free software. It's one of the reasons why it's a free software. It's a free software and so it lets you do change the code as you want and this does not impact the other users. Also, the third principle is fairness and constant deployment. By letting people push some results to the search groups, to people who search similar content as they do, this removes the crawlers so we don't need to copy the web every five minutes again and again. So it removes the crawlers. We're pushing more data to the groups. But also, it's more fair because anyone can push some data. They're not dependent on the SEOs or if you have more money than others, you can make your website better. So it's well recognized by the existing crawlers. Finally, one of the most important points is that SIX takes the user privacy very seriously and for this reason, it builds a decentralized architecture. SIX relies on the peer-to-peer network. The way it works is that you install SIX on your machine. You don't need like a, it can be a desktop, a laptop, it can be a Shiva plug. It works on all these devices. You install SIX on your machine and it will learn from your behavior and navigation and also from your queries that you're typing in every day. And it will build a profile for you, a filter through which you look at the results from other search engines or whatever feeds you're using. These profiles over the peer-to-peer network, they exchange fragments of information through the search groups to help each other improve your results and get some more, some recommendation and so on. That's the way it looks. Sorry for the bad hour. Okay, that's it. Okay, actually. So it looks just like in other search engines, right? You can use it every day. You get no loss of results, like of quality of results. It uses existing engines to get results. You can use whatever other feeds you want and see the little stars and the recommendation on the, on the rights. They're like, their information that is shared by the users. Okay, so here's the roadmap. The first step was to build the, what seems in the final edge is the only free software meta search engine. Okay, and this is done. We can search for text, image, videos, tweets. There are some nice tools built in and we're done with this. This is something you can install on your, on your own laptop, on your desktop. The second step, which is coming up soon is the, the rising of a peer-to-peer net that connects all these six nodes together. Okay. So whatever you're doing on your machine, if you let it do so, it will have an impact to other people searching similar things as yours. And this impact will be proportional to the similarity of your profile to some other users' profile. Okay? The third step is that you let on this network, people push some more content to the search groups, freeing ourselves from the existing engines and becoming more real-time. The fourth step is like the step for the end of time that we'll probably never see. Okay? Well, it's interesting though to have it in mind. So when you wake up in the morning, you know where you're going. If you let the web servers index their own content and build queries that describe their content and hook them up on the search groups and push this information to the search groups, then you free yourself completely from the crawlers. Okay? So that's for the end of time. We can move gradually its long path toward this end. But this end should work. There's no technical reason why it wouldn't work. And in the end, we would not index, we index the entire web, but just be more active in the push or receive mode. Okay? If you're interested in the project, you can test it. You can test some public nodes, 6.fr, there's some others. You're going on the website, 6project.info. They have a bunch of like six nodes that are maintained by volunteers. And you can play with them on a day-to-day basis. I mean, I use it on a day basis and I'm a hardcore searcher and I have no problem. The quality of the results is, in my opinion, even better than other engines. You can get in touch with us on IRC. We have a bunch of interesting tasks that need to be addressed. So I'll give you an overview here. If you like distributed hash tables, if you like network security, there are a lot of interesting issues we need to address. Database also, replicated databases. We move slowly, but surely. And we need a lot of different competences. The second point is machine learning. If you like machine learning, if you want to learn more about machine learning, and if you're willing to participate, please come in. There are a lot of interesting tasks to be addressed. And eventually, what you can do with 6 is that, as it is a free software, you can build the user interface of your dream. You don't need to rely on the user interface that the existing engines give you a change whenever they want. You can build whatever you want. So I'm a C++ developer. I can't really design UIs. So I consider that this is a job for the community and users to do whatever they want, try whatever they want. If it fails, it fails. If it's good, it's good. And OK. So that's all. The point being, come and help us if you like the project, or try to use it and see if it's any good for you. Thank you. Thank you very much. Wow. Do I have time for questions or not? One. No. No. No. Amen.
 I'm going to give you a very short and brief overview of the six projects.  And what you'll get is sort of a one path to possible take back of web search, the way  Firefox took back the web.  This take back is a long, long work.  It's a hard work.  It'll be difficult, but that's the path that I know and that's the path that I work with  with a bunch of people.  So this is what I'm going to show you.  And maybe you'll be convinced that the first software community and us and hackers and  developers, blah, blah, blah, we can do something about it.  Oops.  Okay.  So as you know, we have something fantastic at our fingertips.  We have the most connected human built network of recorded history.  It's the internet.  And on this network, we have basically a massive flow of data.  And this data, we can categorize them in two categories.  Human generated and human targeted data and machine to machine data.  So in this context, what is search?  Search is this basic operation for humans to look up some human targeted content throughout  this massive amount of data.  Search is this fascinating characteristic that we all depend on it.  Most of the time we look at, we ask something to Google or Bing or whatever, search engine  before we even ask our friends.  So we're really dependent on these engines.  And also the level of the technicality that is required and the usage that is required  by users to use this engine is as low-virus.  What it means is that whether you're my mother or you're a power searcher, so-called power  searcher, the tools that you're using, the way you're using Google or any other engine,  it's almost the same, right?  The power searcher may be a little bit better, but it's almost the same.  So what happens is we are all dependent on search and we all use it.  And if you're lucky to not have understood or seen this before, as users we're locked  up and we're locked up in several ways.  First because we're interacting with black boxes, proprietary black boxes, which means  that basically you've all experienced this, typing in some keywords, getting some results  back, and are satisfied with the results, you change the keyword, you get another query  and so on and so on, and you iterate that until you get satisfied.  Sometimes you just get garbage, right?  And the machine will never tell you that you got garbage because the machine will never  tell you that she doesn't know, but it doesn't know actually.  So you can give feedback.  That's the first lockup.  The second lockup is very important to me.  It's basically we're all doing the same queries over and over again in our own corner.  We're isolated.  If I look for something and I read the form and I find at the end of the form that the  answer to my question is not there, the hundred or the thousand people behind me who are going  to do the same query, well, they'll do the same thing, right?  They'll go through the form and read it again.  So we're isolated.  The third lockup is that we're all experts in something, right?  We look things on, we type in queries to these search engines and we're all experts in something,  but we distill our queries to these search engines and we keep nothing on our machines,  right?  So we don't build profiles of expertise that should be ours.  So the point here is truly, web search is lagging behind and there are reasons for this.  I'll try to tell you what.  But the way I can convince you that it's lagging behind is that we're millions, even billions  of users and every day there is an estimated one billion query being issued on the search  engines.  And of course these queries, they're rooted in reality, right?  We're looking in the info sphere for things that are real, recipes, medicines, whatever,  right?  But we're looking for something real.  And since we're looking for something real, well, many of us, we're looking for similar  things, right?  But we cannot interact.  There's no feedback.  We cannot share our queries like we can share something else.  So what is 6?  6 is an open architecture and it's a free software application that tries to fill the  gap in web search by making it collaborative, making it social.  And the main idea is that as a social web search is inevitable.  It will happen.  Whether 6 succeeds or whether some other people come up with better ideas, it doesn't matter.  In the end, we'll do search together, okay?  The way we do some other things together.  The idea behind 6, the main idea is very, very simple.  It's very natural too.  It is to regroup users who perform similar or identical queries so they can share and  collaborate on the results.  That's all.  This simple idea has many interesting consequences.  So I'll try to describe 6 with just this slide very briefly.  Basically, what happens is 6 regroups people in real time in so-called search groups around  similar queries, okay?  So people who type similar queries, they get regrouped in search groups.  These search groups, these people in the search groups, they can work together.  So we give them collaborative filtering tools to work together.  So these tools, they allow them to exchange results automatically or actively by clicking  to recommend results, to toss results away, to do a lot of things.  So basically, it gives the ability to these people to process data.  And the third point is very important is we allow some users to push some more data to  the search groups for them to process.  So let me give you an example.  If you have a blog and you write a new entry to your blog, what you're going to do is you're  going to devise a few queries, at least one query that describes your new entry of your  blog.  You're going to do that, you know, type in that query into 6.  You're going to be regrouped with people who did similar queries.  But instead of looking at the results that you get from the search engine, you'll push  your content.  Okay?  You'll say, hey, this is a potential new result for you guys.  If you've been lying, it means if your query is not in any equation with the content, you'll  get tossed.  Okay?  People will say, okay, that's garbage.  We don't want it.  If you do this, the groups, the filtering, and the publication to the groups, you can  build a real-time collaborative and fair search engine.  Fair because anyone can push some data.  Okay.  Here is the important thing.  Why do we need to do it?  If social web search, you know, social web search is a simple idea.  Why doesn't it exist?  You know, all these companies, they could do it.  There's one reason.  The business model of these companies, right?  The existing engines, they feed on user personal data in exchange for the service that they  give you.  These data, they used to target advertising and generate revenues for these engines, right?  So if you let users share their queries and, you know, be clustered into groups of people  with similar interests.  And if you let people push them some content, give them some more content and results, like  advertisers, then these advertisers, they don't need to go through the existing paywalls,  like AdWords or some others.  They don't need to go to these centralized entities and pay the fee to be able to push  their results.  So basically what happens is that these existing search engines, because of the business models  that they have, that work, but, you know, because these business models, they will never do  it.  And the proof, the fact is that today was things that you may like or don't like, doesn't matter,  like Facebook and others, they're still lagging behind.  They haven't done it, okay?  And the lockup is this.  It's because they simply cannot do it.  Okay, six builds on a few principles.  I will outline them very quickly.  The first one is if you want to take back web search, you've got to give more control  to users over the search algorithms.  What SIX does is that it groups, it regroups the people so they're together.  Once they're together, they can work together and they have more control over their results  and the algorithms.  The second thing is transparency.  Transparency means that you're able to modify the equations, the ranking, the way the ranking  is done in the engine freely.  You do not have to rely on proprietary code and equations and let your results being ranked  by them.  So the way SIX does this is that it's a free software.  It's one of the reasons why it's a free software.  It's a free software and so it lets you do change the code as you want and this does  not impact the other users.  Also, the third principle is fairness and constant deployment.  By letting people push some results to the search groups, to people who search similar  content as they do, this removes the crawlers so we don't need to copy the web every five  minutes again and again.  So it removes the crawlers.  We're pushing more data to the groups.  But also, it's more fair because anyone can push some data.  They're not dependent on the SEOs or if you have more money than others, you can make your  website better.  So it's well recognized by the existing crawlers.  Finally, one of the most important points is that SIX takes the user privacy very seriously  and for this reason, it builds a decentralized architecture.  SIX relies on the peer-to-peer network.  The way it works is that you install SIX on your machine.  You don't need like a, it can be a desktop, a laptop, it can be a Shiva plug.  It works on all these devices.  You install SIX on your machine and it will learn from your behavior and navigation and  also from your queries that you're typing in every day.  And it will build a profile for you, a filter through which you look at the results from  other search engines or whatever feeds you're using.  These profiles over the peer-to-peer network, they exchange fragments of information through  the search groups to help each other improve your results and get some more, some recommendation  and so on.  That's the way it looks.  Sorry for the bad hour.  Okay, that's it.  Okay, actually.  So it looks just like in other search engines, right?  You can use it every day.  You get no loss of results, like of quality of results.  It uses existing engines to get results.  You can use whatever other feeds you want and see the little stars and the recommendation  on the, on the rights.  They're like, their information that is shared by the users.  Okay, so here's the roadmap.  The first step was to build the, what seems in the final edge is the only free software  meta search engine.  Okay, and this is done.  We can search for text, image, videos, tweets.  There are some nice tools built in and we're done with this.  This is something you can install on your, on your own laptop, on your desktop.  The second step, which is coming up soon is the, the rising of a peer-to-peer net that  connects all these six nodes together.  Okay.  So whatever you're doing on your machine, if you let it do so, it will have an impact  to other people searching similar things as yours.  And this impact will be proportional to the similarity of your profile to some other users'  profile.  Okay?  The third step is that you let on this network, people push some more content to the search  groups, freeing ourselves from the existing engines and becoming more real-time.  The fourth step is like the step for the end of time that we'll probably never see.  Okay?  Well, it's interesting though to have it in mind.  So when you wake up in the morning, you know where you're going.  If you let the web servers index their own content and build queries that describe their  content and hook them up on the search groups and push this information to the search groups,  then you free yourself completely from the crawlers.  Okay?  So that's for the end of time.  We can move gradually its long path toward this end.  But this end should work.  There's no technical reason why it wouldn't work.  And in the end, we would not index, we index the entire web, but just be more active in  the push or receive mode.  Okay?  If you're interested in the project, you can test it.  You can test some public nodes, 6.fr, there's some others.  You're going on the website, 6project.info.  They have a bunch of like six nodes that are maintained by volunteers.  And you can play with them on a day-to-day basis.  I mean, I use it on a day basis and I'm a hardcore searcher and I have no problem.  The quality of the results is, in my opinion, even better than other engines.  You can get in touch with us on IRC.  We have a bunch of interesting tasks that need to be addressed.  So I'll give you an overview here.  If you like distributed hash tables, if you like network security, there are a lot of  interesting issues we need to address.  Database also, replicated databases.  We move slowly, but surely.  And we need a lot of different competences.  The second point is machine learning.  If you like machine learning, if you want to learn more about machine learning, and  if you're willing to participate, please come in.  There are a lot of interesting tasks to be addressed.  And eventually, what you can do with 6 is that, as it is a free software, you can build  the user interface of your dream.  You don't need to rely on the user interface that the existing engines give you a change  whenever they want.  You can build whatever you want.  So I'm a C++ developer.  I can't really design UIs.  So I consider that this is a job for the community and users to do whatever they want, try whatever  they want.  If it fails, it fails.  If it's good, it's good.  And OK.  So that's all.  The point being, come and help us if you like the project, or try to use it and see if it's  any good for you.  Thank you.  Thank you very much.  Wow.  Do I have time for questions or not?  One.  No.  No.  No.  Amen.
seeks_CVhTvnHsE2I_1_0_131.png seeks_CVhTvnHsE2I_1_132_179.png seeks_CVhTvnHsE2I_1_180_227.png seeks_CVhTvnHsE2I_1_228_983.png seeks_CVhTvnHsE2I_1_984_1031.png seeks_CVhTvnHsE2I_1_1032_1091.png seeks_CVhTvnHsE2I_1_1092_1427.png seeks_CVhTvnHsE2I_1_1428_2903.png seeks_CVhTvnHsE2I_1_2904_3335.png seeks_CVhTvnHsE2I_1_3336_4595.png seeks_CVhTvnHsE2I_1_4596_5615.png seeks_CVhTvnHsE2I_1_5616_5795.png seeks_CVhTvnHsE2I_1_5796_6659.png seeks_CVhTvnHsE2I_1_6660_7391.png seeks_CVhTvnHsE2I_1_7392_7823.png seeks_CVhTvnHsE2I_1_7824_10127.png seeks_CVhTvnHsE2I_1_10128_12239.png seeks_CVhTvnHsE2I_1_12240_12887.png seeks_CVhTvnHsE2I_1_12888_13727.png seeks_CVhTvnHsE2I_1_13728_14627.png seeks_CVhTvnHsE2I_1_14628_15971.png seeks_CVhTvnHsE2I_1_15972_16643.png seeks_CVhTvnHsE2I_1_16644_17135.png seeks_CVhTvnHsE2I_1_17136_19163.png seeks_CVhTvnHsE2I_1_19164_20099.png seeks_CVhTvnHsE2I_1_20100_21731.png seeks_CVhTvnHsE2I_1_21732_21983.png seeks_CVhTvnHsE2I_1_21984_22319.png seeks_CVhTvnHsE2I_1_22320_22438.png