[Websearch] update (#427)

* Fix reuqest body

* update webSearchQueryPromptTemplate

* update generate google query parser

* Add today's date to google search query creator

* crawl top stories if exts; remove answer_box & knowledgeGraph

* Create paragraph chunks from top articles

* flattened paragprah chunks

* update status texts

* add gradio client

* call gradio app for RAG

* Web scrape only "p, li, span" els

* add MAX_N_CHUNKS

* gradio result typing

* parse only <p> elements

* rm dev change

* update typing WebSearch

* buld RAG prompt

* Rm dev change

* change websearch context msg from user to assisntat type

* use hosted gradio app

* fix lint

* prompt engineering

* more prompt engineering

* MAX_N_PAGES_SCRAPE = 10

* better error msg

* more prompt engineering

* revert websearch prompt to previous

* rm `top_stories` from websearch as the results are not good

* Stop using gradio client, use regular fetch

* chore

* Rm websearchsummary references as it is no longer used

* update readme

* Apply suggestions from code review

Co-authored-by: Julien Chaumond <[email protected]>

* Use tfjs to do embeddings in server node

* fix websearch component disapperar after finishing generation

* Show sources of closest embeddings used in RAG

* fix prompting and also add current date

* add comment

* comment for search query

* sources

* hide www

* using hostname direclty

* Show successful web pages instead of failed ones

* rm noisy messages

* google query generation using previous messaages as context

* handle falcon generation

* bring back Browsing webpage msg

---------

Co-authored-by: Julien Chaumond <[email protected]>
Co-authored-by: Victor Mustar <[email protected]>

README.md

@@ -155,7 +155,7 @@ You can change things like the parameters, or customize the preprompt to better
 
 By default the prompt is constructed using `userMessageToken`, `assistantMessageToken`, `userMessageEndToken`, `assistantMessageEndToken`, `preprompt` parameters and a series of default templates.
 
-However, these templates can be modified by setting the `chatPromptTemplate`, `webSearchSummaryPromptTemplate`, and `webSearchQueryPromptTemplate` parameters. Note that if WebSearch is not enabled, only `chatPromptTemplate` needs to be set. The template language is https://handlebarsjs.com. The templates have access to the model's prompt parameters (`preprompt`, etc.). However, if the templates are specified it is recommended to inline the prompt parameters, as using the references (`{{preprompt}}`) is deprecated.
+However, these templates can be modified by setting the `chatPromptTemplate` and `webSearchQueryPromptTemplate` parameters. Note that if WebSearch is not enabled, only `chatPromptTemplate` needs to be set. The template language is https://handlebarsjs.com. The templates have access to the model's prompt parameters (`preprompt`, etc.). However, if the templates are specified it is recommended to inline the prompt parameters, as using the references (`{{preprompt}}`) is deprecated.
 
 For example:
 
@@ -187,33 +187,14 @@ The following is the default `chatPromptTemplate`, although newlines and indenti
 
 When performing a websearch, the search query is constructed using the `webSearchQueryPromptTemplate` template. It is recommended that that the prompt instructs the chat model to only return a few keywords.
 
-The following is the default `webSearchQueryPromptTemplate`. Note that not all models supports consecutive user-messages which this template uses.
+The following is the default `webSearchQueryPromptTemplate`.
 
 ```
 {{userMessageToken}}
-  The following messages were written by a user, trying to answer a question.
+  My question is: {{message.content}}.
+  Based on the conversation history (my previous questions are: {{previousMessages}}), give me an appropriate query to answer my question for google search. You should not say more than query. You should not say any words except the query. For the context, today is {{currentDate}}
 {{userMessageEndToken}}
-{{#each messages}}
-  {{#ifUser}}{{@root.userMessageToken}}{{content}}{{@root.userMessageEndToken}}{{/ifUser}}
-{{/each}}
-{{userMessageToken}}
-  What plain-text english sentence would you input into Google to answer the last question? Answer with a short (10 words max) simple sentence.
-{{userMessageEndToken}}
-{{assistantMessageToken}}Query:
-```
-
-**webSearchSummaryPromptTemplate**
-
-The search-engine response (`answer`) is summarized using the following prompt template. However, when `HF_ACCESS_TOKEN` is provided, a dedicated summary model is used instead. Additionally, the model's `query` response to `webSearchQueryPromptTemplate` is also available to this template.
-
-The following is the default `webSearchSummaryPromptTemplate`. Note that not all models supports consecutive user-messages which this template uses.
-
-```
-{{userMessageToken}}{{answer}}{{userMessageEndToken}}
-{{userMessageToken}}
-  The text above should be summarized to best answer the query: {{query}}.
-{{userMessageEndToken}}
-{{assistantMessageToken}}Summary:
+{{assistantMessageToken}}
 ```
 
 #### Running your own models using a custom endpoint

package-lock.json

@@ -10,6 +10,7 @@
 			"dependencies": {
 				"@huggingface/hub": "^0.5.1",
 				"@huggingface/inference": "^2.2.0",
+				"@xenova/transformers": "^2.6.0",
 				"autoprefixer": "^10.4.14",
 				"aws4fetch": "^1.0.17",
 				"date-fns": "^2.29.3",
@@ -655,6 +656,60 @@
 			"integrity": "sha512-a5Sab1C4/icpTZVzZc5Ghpz88yQtGOyNqYXcZgOssB2uuAr+wF/MvN6bgtW32q7HHrvBki+BsZ0OuNv6EV3K9g==",
 			"dev": true
 		},
+		"node_modules/@protobufjs/aspromise": {
+			"version": "1.1.2",
+			"resolved": "https://registry.npmjs.org/@protobufjs/aspromise/-/aspromise-1.1.2.tgz",
+			"integrity": "sha512-j+gKExEuLmKwvz3OgROXtrJ2UG2x8Ch2YZUxahh+s1F2HZ+wAceUNLkvy6zKCPVRkU++ZWQrdxsUeQXmcg4uoQ=="
+		},
+		"node_modules/@protobufjs/base64": {
+			"version": "1.1.2",
+			"resolved": "https://registry.npmjs.org/@protobufjs/base64/-/base64-1.1.2.tgz",
+			"integrity": "sha512-AZkcAA5vnN/v4PDqKyMR5lx7hZttPDgClv83E//FMNhR2TMcLUhfRUBHCmSl0oi9zMgDDqRUJkSxO3wm85+XLg=="
+		},
+		"node_modules/@protobufjs/codegen": {
+			"version": "2.0.4",
+			"resolved": "https://registry.npmjs.org/@protobufjs/codegen/-/codegen-2.0.4.tgz",
+			"integrity": "sha512-YyFaikqM5sH0ziFZCN3xDC7zeGaB/d0IUb9CATugHWbd1FRFwWwt4ld4OYMPWu5a3Xe01mGAULCdqhMlPl29Jg=="
+		},
+		"node_modules/@protobufjs/eventemitter": {
+			"version": "1.1.0",
+			"resolved": "https://registry.npmjs.org/@protobufjs/eventemitter/-/eventemitter-1.1.0.tgz",
+			"integrity": "sha512-j9ednRT81vYJ9OfVuXG6ERSTdEL1xVsNgqpkxMsbIabzSo3goCjDIveeGv5d03om39ML71RdmrGNjG5SReBP/Q=="
+		},
+		"node_modules/@protobufjs/fetch": {
+			"version": "1.1.0",
+			"resolved": "https://registry.npmjs.org/@protobufjs/fetch/-/fetch-1.1.0.tgz",
+			"integrity": "sha512-lljVXpqXebpsijW71PZaCYeIcE5on1w5DlQy5WH6GLbFryLUrBD4932W/E2BSpfRJWseIL4v/KPgBFxDOIdKpQ==",
+			"dependencies": {
+				"@protobufjs/aspromise": "^1.1.1",
+				"@protobufjs/inquire": "^1.1.0"
+			}
+		},
+		"node_modules/@protobufjs/float": {
+			"version": "1.0.2",
+			"resolved": "https://registry.npmjs.org/@protobufjs/float/-/float-1.0.2.tgz",
+			"integrity": "sha512-Ddb+kVXlXst9d+R9PfTIxh1EdNkgoRe5tOX6t01f1lYWOvJnSPDBlG241QLzcyPdoNTsblLUdujGSE4RzrTZGQ=="
+		},
+		"node_modules/@protobufjs/inquire": {
+			"version": "1.1.0",
+			"resolved": "https://registry.npmjs.org/@protobufjs/inquire/-/inquire-1.1.0.tgz",
+			"integrity": "sha512-kdSefcPdruJiFMVSbn801t4vFK7KB/5gd2fYvrxhuJYg8ILrmn9SKSX2tZdV6V+ksulWqS7aXjBcRXl3wHoD9Q=="
+		},
+		"node_modules/@protobufjs/path": {
+			"version": "1.1.2",
+			"resolved": "https://registry.npmjs.org/@protobufjs/path/-/path-1.1.2.tgz",
+			"integrity": "sha512-6JOcJ5Tm08dOHAbdR3GrvP+yUUfkjG5ePsHYczMFLq3ZmMkAD98cDgcT2iA1lJ9NVwFd4tH/iSSoe44YWkltEA=="
+		},
+		"node_modules/@protobufjs/pool": {
+			"version": "1.1.0",
+			"resolved": "https://registry.npmjs.org/@protobufjs/pool/-/pool-1.1.0.tgz",
+			"integrity": "sha512-0kELaGSIDBKvcgS4zkjz1PeddatrjYcmMWOlAuAPwAeccUrPHdUqo/J6LiymHHEiJT5NrF1UVwxY14f+fy4WQw=="
+		},
+		"node_modules/@protobufjs/utf8": {
+			"version": "1.1.0",
+			"resolved": "https://registry.npmjs.org/@protobufjs/utf8/-/utf8-1.1.0.tgz",
+			"integrity": "sha512-Vvn3zZrhQZkkBE8LSuW3em98c0FwgO4nxzv6OdSxPKJIEKY2bGbHn+mhGIPerzI4twdxaP8/0+06HBpwf345Lw=="
+		},
 		"node_modules/@rollup/plugin-commonjs": {
 			"version": "24.0.1",
 			"resolved": "https://registry.npmjs.org/@rollup/plugin-commonjs/-/plugin-commonjs-24.0.1.tgz",
@@ -947,6 +1002,11 @@
 			"integrity": "sha512-wOuvG1SN4Us4rez+tylwwwCV1psiNVOkJeM3AUWUNWg/jDQY2+HE/444y5gc+jBmRqASOm2Oeh5c1axHobwRKQ==",
 			"dev": true
 		},
+		"node_modules/@types/long": {
+			"version": "4.0.2",
+			"resolved": "https://registry.npmjs.org/@types/long/-/long-4.0.2.tgz",
+			"integrity": "sha512-MqTGEo5bj5t157U6fA/BiDynNkn0YknVdh48CMPkTSpFTVmvao5UQmm7uEF6xBEo7qIMAlY/JSleYaE6VOdpaA=="
+		},
 		"node_modules/@types/marked": {
 			"version": "4.0.8",
 			"resolved": "https://registry.npmjs.org/@types/marked/-/marked-4.0.8.tgz",
@@ -1298,6 +1358,18 @@
 				"url": "https://opencollective.com/vitest"
 			}
 		},
+		"node_modules/@xenova/transformers": {
+			"version": "2.6.0",
+			"resolved": "https://registry.npmjs.org/@xenova/transformers/-/transformers-2.6.0.tgz",
+			"integrity": "sha512-k9bs+reiwhn+kx0d4FYnlBTWtl8D5Q4fIzoKYxKbTTSVyS33KXbQESRpdIxiU9gtlMKML2Sw0Oep4FYK9dQCsQ==",
+			"dependencies": {
+				"onnxruntime-web": "1.14.0",
+				"sharp": "^0.32.0"
+			},
+			"optionalDependencies": {
+				"onnxruntime-node": "1.14.0"
+			}
+		},
 		"node_modules/abab": {
 			"version": "2.0.6",
 			"resolved": "https://registry.npmjs.org/abab/-/abab-2.0.6.tgz",
@@ -1472,6 +1544,11 @@
 			"resolved": "https://registry.npmjs.org/aws4fetch/-/aws4fetch-1.0.17.tgz",
 			"integrity": "sha512-4IbOvsxqxeOSxI4oA+8xEO8SzBMVlzbSTgGy/EF83rHnQ/aKtP6Sc6YV/k0oiW0mqrcxuThlbDosnvetGOuO+g=="
 		},
+		"node_modules/b4a": {
+			"version": "1.6.4",
+			"resolved": "https://registry.npmjs.org/b4a/-/b4a-1.6.4.tgz",
+			"integrity": "sha512-fpWrvyVHEKyeEvbKZTVOeZF3VSKKWtJxFIxX/jaVPf+cLbGUSitjb49pHLqPV2BUNNZ0LcoeEGfE/YCpyDYHIw=="
+		},
 		"node_modules/balanced-match": {
 			"version": "1.0.2",
 			"resolved": "https://registry.npmjs.org/balanced-match/-/balanced-match-1.0.2.tgz",
@@ -1510,6 +1587,16 @@
 			"integrity": "sha512-u4cBQNepWxYA55FunZSM7wMi55yQaN0otnhhilNoWHq0MfOfJeQx0v0mRRpolGOExPjZcl6FtB0BB8Xkb88F0g==",
 			"optional": true
 		},
+		"node_modules/bl": {
+			"version": "4.1.0",
+			"resolved": "https://registry.npmjs.org/bl/-/bl-4.1.0.tgz",
+			"integrity": "sha512-1W07cM9gS6DcLperZfFSj+bWLtaPGSOHWhPiGzXmvVJbRLdG82sH/Kn8EtW1VqWVA54AKf2h5k5BbnIbwF3h6w==",
+			"dependencies": {
+				"buffer": "^5.5.0",
+				"inherits": "^2.0.4",
+				"readable-stream": "^3.4.0"
+			}
+		},
 		"node_modules/blueimp-md5": {
 			"version": "2.19.0",
 			"resolved": "https://registry.npmjs.org/blueimp-md5/-/blueimp-md5-2.19.0.tgz",
@@ -1579,6 +1666,29 @@
 				"node": ">=14.20.1"
 			}
 		},
+		"node_modules/buffer": {
+			"version": "5.7.1",
+			"resolved": "https://registry.npmjs.org/buffer/-/buffer-5.7.1.tgz",
+			"integrity": "sha512-EHcyIPBQ4BSGlvjB16k5KgAJ27CIsHY/2JBmCRReo48y9rQ3MaUzWX3KVlBa4U7MyX02HdVj0K7C3WaB3ju7FQ==",
+			"funding": [
+				{
+					"type": "github",
+					"url": "https://github.com/sponsors/feross"
+				},
+				{
+					"type": "patreon",
+					"url": "https://www.patreon.com/feross"
+				},
+				{
+					"type": "consulting",
+					"url": "https://feross.org/support"
+				}
+			],
+			"dependencies": {
+				"base64-js": "^1.3.1",
+				"ieee754": "^1.1.13"
+			}
+		},
 		"node_modules/buffer-crc32": {
 			"version": "0.2.13",
 			"resolved": "https://registry.npmjs.org/buffer-crc32/-/buffer-crc32-0.2.13.tgz",
@@ -1588,6 +1698,20 @@
 				"node": "*"
 			}
 		},
+		"node_modules/bufferutil": {
+			"version": "4.0.7",
+			"resolved": "https://registry.npmjs.org/bufferutil/-/bufferutil-4.0.7.tgz",
+			"integrity": "sha512-kukuqc39WOHtdxtw4UScxF/WVnMFVSQVKhtx3AjZJzhd0RGZZldcrfSEbVsWWe6KNH253574cq5F+wpv0G9pJw==",
+			"hasInstallScript": true,
+			"optional": true,
+			"peer": true,
+			"dependencies": {
+				"node-gyp-build": "^4.3.0"
+			},
+			"engines": {
+				"node": ">=6.14.2"
+			}
+		},
 		"node_modules/builtin-modules": {
 			"version": "3.3.0",
 			"resolved": "https://registry.npmjs.org/builtin-modules/-/builtin-modules-3.3.0.tgz",
@@ -1736,11 +1860,27 @@
 				"node": ">= 6"
 			}
 		},
+		"node_modules/chownr": {
+			"version": "1.1.4",
+			"resolved": "https://registry.npmjs.org/chownr/-/chownr-1.1.4.tgz",
+			"integrity": "sha512-jJ0bqzaylmJtVnNgzTeSOs8DPavpbYgEr/b0YL8/2GO3xJEhInFmhKMUnEJQjZumK7KXGFhUy89PrsJWlakBVg=="
+		},
+		"node_modules/color": {
+			"version": "4.2.3",
+			"resolved": "https://registry.npmjs.org/color/-/color-4.2.3.tgz",
+			"integrity": "sha512-1rXeuUUiGGrykh+CeBdu5Ie7OJwinCgQY0bc7GCRxy5xVHy+moaqkpL/jqQq0MtQOeYcrqEz4abc5f0KtU7W4A==",
+			"dependencies": {
+				"color-convert": "^2.0.1",
+				"color-string": "^1.9.0"
+			},
+			"engines": {
+				"node": ">=12.5.0"
+			}
+		},
 		"node_modules/color-convert": {
 			"version": "2.0.1",
 			"resolved": "https://registry.npmjs.org/color-convert/-/color-convert-2.0.1.tgz",
 			"integrity": "sha512-RRECPsj7iu/xb5oKYcsFHSppFNnsj/52OVTRKb4zP5onXwVF3zVmmToNcOfGC+CRDpfK/U584fMg38ZHCaElKQ==",
-			"dev": true,
 			"dependencies": {
 				"color-name": "~1.1.4"
 			},
@@ -1753,6 +1893,15 @@
 			"resolved": "https://registry.npmjs.org/color-name/-/color-name-1.1.4.tgz",
 			"integrity": "sha512-dOy+3AuW3a2wNbZHIuMZpTcgjGuLU/uBL/ubcZF9OXbDo8ff4O8yVp5Bf0efS8uEoYo5q4Fx7dY9OgQGXgAsQA=="
 		},
+		"node_modules/color-string": {
+			"version": "1.9.1",
+			"resolved": "https://registry.npmjs.org/color-string/-/color-string-1.9.1.tgz",
+			"integrity": "sha512-shrVawQFojnZv6xM40anx4CkoDP+fZsw/ZerEMsW/pyzsRbElpsL/DBVW7q3ExxwusdNXI3lXpuhEZkzs8p5Eg==",
+			"dependencies": {
+				"color-name": "^1.0.0",
+				"simple-swizzle": "^0.2.2"
+			}
+		},
 		"node_modules/combined-stream": {
 			"version": "1.0.8",
 			"resolved": "https://registry.npmjs.org/combined-stream/-/combined-stream-1.0.8.tgz",
@@ -1928,6 +2077,20 @@
 			"resolved": "https://registry.npmjs.org/decimal.js/-/decimal.js-10.4.3.tgz",
 			"integrity": "sha512-VBBaLc1MgL5XpzgIP7ny5Z6Nx3UrRkIViUkPUdtl9aya5amy3De1gsUUSB1g3+3sExYNjCAsAznmukyxCb1GRA=="
 		},
+		"node_modules/decompress-response": {
+			"version": "6.0.0",
+			"resolved": "https://registry.npmjs.org/decompress-response/-/decompress-response-6.0.0.tgz",
+			"integrity": "sha512-aW35yZM6Bb/4oJlZncMH2LCoZtJXTRxES17vE3hoRiowU2kWHaJKFkSBDnDR+cm9J+9QhXmREyIfv0pji9ejCQ==",
+			"dependencies": {
+				"mimic-response": "^3.1.0"
+			},
+			"engines": {
+				"node": ">=10"
+			},
+			"funding": {
+				"url": "https://github.com/sponsors/sindresorhus"
+			}
+		},
 		"node_modules/deep-eql": {
 			"version": "4.1.3",
 			"resolved": "https://registry.npmjs.org/deep-eql/-/deep-eql-4.1.3.tgz",
@@ -1940,6 +2103,14 @@
 				"node": ">=6"
 			}
 		},
+		"node_modules/deep-extend": {
+			"version": "0.6.0",
+			"resolved": "https://registry.npmjs.org/deep-extend/-/deep-extend-0.6.0.tgz",
+			"integrity": "sha512-LOHxIOaPYdHlJRtCQfDIVZtfw/ufM8+rVj649RIHzcm/vGwQRXFt6OPqIFWsm2XEMrNIEtWR64sY1LEKD2vAOA==",
+			"engines": {
+				"node": ">=4.0.0"
+			}
+		},
 		"node_modules/deep-is": {
 			"version": "0.1.4",
 			"resolved": "https://registry.npmjs.org/deep-is/-/deep-is-0.1.4.tgz",
@@ -1972,6 +2143,14 @@
 				"node": ">=8"
 			}
 		},
+		"node_modules/detect-libc": {
+			"version": "2.0.2",
+			"resolved": "https://registry.npmjs.org/detect-libc/-/detect-libc-2.0.2.tgz",
+			"integrity": "sha512-UX6sGumvvqSaXgdKGUsgZWqcUyIXZ/vZTrlRT/iobiKhGL0zL4d3osHj3uqllWJK+i+sixDS/3COVEOFbupFyw==",
+			"engines": {
+				"node": ">=8"
+			}
+		},
 		"node_modules/devalue": {
 			"version": "4.3.0",
 			"resolved": "https://registry.npmjs.org/devalue/-/devalue-4.3.0.tgz",
@@ -2036,6 +2215,14 @@
 			"resolved": "https://registry.npmjs.org/electron-to-chromium/-/electron-to-chromium-1.4.359.tgz",
 			"integrity": "sha512-OoVcngKCIuNXtZnsYoqlCvr0Cf3NIPzDIgwUfI9bdTFjXCrr79lI0kwQstLPZ7WhCezLlGksZk/BFAzoXC7GDw=="
 		},
+		"node_modules/end-of-stream": {
+			"version": "1.4.4",
+			"resolved": "https://registry.npmjs.org/end-of-stream/-/end-of-stream-1.4.4.tgz",
+			"integrity": "sha512-+uw1inIHVPQoaVuHzRyXd21icM+cnt4CzD5rW+NC1wjOUSTOs+Te7FOv7AhN7vS9x/oIyhLP5PR1H+phQAHu5Q==",
+			"dependencies": {
+				"once": "^1.4.0"
+			}
+		},
 		"node_modules/entities": {
 			"version": "4.5.0",
 			"resolved": "https://registry.npmjs.org/entities/-/entities-4.5.0.tgz",
@@ -2397,6 +2584,14 @@
 				"url": "https://github.com/sindresorhus/execa?sponsor=1"
 			}
 		},
+		"node_modules/expand-template": {
+			"version": "2.0.3",
+			"resolved": "https://registry.npmjs.org/expand-template/-/expand-template-2.0.3.tgz",
+			"integrity": "sha512-XYfuKMvj4O35f/pOXLObndIRvyQ+/+6AhODh+OKWj9S9498pHHn/IMszH+gt0fBCRWMNfk1ZSp5x3AifmnI2vg==",
+			"engines": {
+				"node": ">=6"
+			}
+		},
 		"node_modules/fast-deep-equal": {
 			"version": "3.1.3",
 			"resolved": "https://registry.npmjs.org/fast-deep-equal/-/fast-deep-equal-3.1.3.tgz",
@@ -2409,6 +2604,11 @@
 			"integrity": "sha512-xJuoT5+L99XlZ8twedaRf6Ax2TgQVxvgZOYoPKqZufmJib0tL2tegPBOZb1pVNgIhlqDlA0eO0c3wBvQcmzx4w==",
 			"dev": true
 		},
+		"node_modules/fast-fifo": {
+			"version": "1.3.2",
+			"resolved": "https://registry.npmjs.org/fast-fifo/-/fast-fifo-1.3.2.tgz",
+			"integrity": "sha512-/d9sfos4yxzpwkDkuN7k2SqFKtYNmCTzgfEpz82x34IM9/zc8KGxQoXg1liNC/izpRM/MBdt44Nmx41ZWqk+FQ=="
+		},
 		"node_modules/fast-glob": {
 			"version": "3.2.12",
 			"resolved": "https://registry.npmjs.org/fast-glob/-/fast-glob-3.2.12.tgz",
@@ -2507,6 +2707,11 @@
 				"node": "^10.12.0 || >=12.0.0"
 			}
 		},
+		"node_modules/flatbuffers": {
+			"version": "1.12.0",
+			"resolved": "https://registry.npmjs.org/flatbuffers/-/flatbuffers-1.12.0.tgz",
+			"integrity": "sha512-c7CZADjRcl6j0PlvFy0ZqXQ67qSEZfrVPynmnL+2zPc+NtMvrF8Y0QceMo7QqnSPc7+uWjUIAbvCQ5WIKlMVdQ=="
+		},
 		"node_modules/flatted": {
 			"version": "3.2.7",
 			"resolved": "https://registry.npmjs.org/flatted/-/flatted-3.2.7.tgz",
@@ -2538,6 +2743,11 @@
 				"url": "https://www.patreon.com/infusion"
 			}
 		},
+		"node_modules/fs-constants": {
+			"version": "1.0.0",
+			"resolved": "https://registry.npmjs.org/fs-constants/-/fs-constants-1.0.0.tgz",
+			"integrity": "sha512-y6OAwoSIf7FyjMIv94u+b5rdheZEjzR63GTyZJm5qh4Bi+2YgwLCcI/fPFZkL5PSixOt6ZNKm+w+Hfp/Bciwow=="
+		},
 		"node_modules/fs.realpath": {
 			"version": "1.0.0",
 			"resolved": "https://registry.npmjs.org/fs.realpath/-/fs.realpath-1.0.0.tgz",
@@ -2582,6 +2792,11 @@
 				"url": "https://github.com/sponsors/sindresorhus"
 			}
 		},
+		"node_modules/github-from-package": {
+			"version": "0.0.0",
+			"resolved": "https://registry.npmjs.org/github-from-package/-/github-from-package-0.0.0.tgz",
+			"integrity": "sha512-SyHy3T1v2NUXn29OsWdxmK6RwHD+vkj3v8en8AOBZ1wBQ/hCAQ5bAQTD02kW4W9tUp/3Qh6J8r9EvntiyCmOOw=="
+		},
 		"node_modules/glob": {
 			"version": "7.2.3",
 			"resolved": "https://registry.npmjs.org/glob/-/glob-7.2.3.tgz",
@@ -2672,6 +2887,11 @@
 			"integrity": "sha512-bzh50DW9kTPM00T8y4o8vQg89Di9oLJVLW/KaOGIXJWP/iqCN6WKYkbNOF04vFLJhwcpYUh9ydh/+5vpOqV4YQ==",
 			"dev": true
 		},
+		"node_modules/guid-typescript": {
+			"version": "1.0.9",
+			"resolved": "https://registry.npmjs.org/guid-typescript/-/guid-typescript-1.0.9.tgz",
+			"integrity": "sha512-Y8T4vYhEfwJOTbouREvG+3XDsjr8E3kIr7uf+JZ0BYloFsttiHU0WfvANVsR7TxNUJa/WpCnw/Ino/p+DeBhBQ=="
+		},
 		"node_modules/handlebars": {
 			"version": "4.7.8",
 			"resolved": "https://registry.npmjs.org/handlebars/-/handlebars-4.7.8.tgz",
@@ -2781,6 +3001,25 @@
 				"node": ">=0.10.0"
 			}
 		},
+		"node_modules/ieee754": {
+			"version": "1.2.1",
+			"resolved": "https://registry.npmjs.org/ieee754/-/ieee754-1.2.1.tgz",
+			"integrity": "sha512-dcyqhDvX1C46lXZcVqCpK+FtMRQVdIMN6/Df5js2zouUsqG7I6sFxitIC+7KYK29KdXOLHdu9zL4sFnoVQnqaA==",
+			"funding": [
+				{
+					"type": "github",
+					"url": "https://github.com/sponsors/feross"
+				},
+				{
+					"type": "patreon",
+					"url": "https://www.patreon.com/feross"
+				},
+				{
+					"type": "consulting",
+					"url": "https://feross.org/support"
+				}
+			]
+		},
 		"node_modules/ignore": {
 			"version": "5.2.4",
 			"resolved": "https://registry.npmjs.org/ignore/-/ignore-5.2.4.tgz",
@@ -2829,6 +3068,11 @@
 			"resolved": "https://registry.npmjs.org/inherits/-/inherits-2.0.4.tgz",
 			"integrity": "sha512-k/vGaX4/Yla3WzyMCvTQOXYeIHvqOKtnqBduzTHpzpQZzAskKMhZ2K+EnBiSM9zGSoIFeMpXKxa4dYeZIQqewQ=="
 		},
+		"node_modules/ini": {
+			"version": "1.3.8",
+			"resolved": "https://registry.npmjs.org/ini/-/ini-1.3.8.tgz",
+			"integrity": "sha512-JV/yugV2uzW5iMRSiZAyDtQd+nxtUnjeLt0acNdw98kKLrvuRVyB80tsREOE7yvGVgalhZ6RNXCmEHkUKBKxew=="
+		},
 		"node_modules/int53": {
 			"version": "0.2.4",
 			"resolved": "https://registry.npmjs.org/int53/-/int53-0.2.4.tgz",
@@ -2839,6 +3083,11 @@
 			"resolved": "https://registry.npmjs.org/ip/-/ip-2.0.0.tgz",
 			"integrity": "sha512-WKa+XuLG1A1R0UWhl2+1XQSi+fZWMsYKffMZTTYsiZaUD8k2yDAj5atimTUD2TZkyCkNEeYE5NhFZmupOGtjYQ=="
 		},
+		"node_modules/is-arrayish": {
+			"version": "0.3.2",
+			"resolved": "https://registry.npmjs.org/is-arrayish/-/is-arrayish-0.3.2.tgz",
+			"integrity": "sha512-eVRqCvVlZbuw3GrM63ovNSNAeA1K16kaR/LRY/92w0zxQ5/1YzwblUX652i4Xs9RwAGjW9d9y6X88t8OaAJfWQ=="
+		},
 		"node_modules/is-binary-path": {
 			"version": "2.1.0",
 			"resolved": "https://registry.npmjs.org/is-binary-path/-/is-binary-path-2.1.0.tgz",
@@ -3177,6 +3426,11 @@
 			"integrity": "sha512-0KpjqXRVvrYyCsX1swR/XTK0va6VQkQM6MNo7PqW77ByjAhoARA8EfrP1N4+KlKj8YS0ZUCtRT/YUuhyYDujIQ==",
 			"dev": true
 		},
+		"node_modules/long": {
+			"version": "4.0.0",
+			"resolved": "https://registry.npmjs.org/long/-/long-4.0.0.tgz",
+			"integrity": "sha512-XsP+KhQif4bjX1kbuSiySJFNAehNxgLb6hPRGJ9QsUr8ajHkuXGdrHmFUTUUXhDwVX2R5bY4JNZEwbUiMhV+MA=="
+		},
 		"node_modules/loupe": {
 			"version": "2.3.6",
 			"resolved": "https://registry.npmjs.org/loupe/-/loupe-2.3.6.tgz",
@@ -3314,6 +3568,17 @@
 				"node": ">=6"
 			}
 		},
+		"node_modules/mimic-response": {
+			"version": "3.1.0",
+			"resolved": "https://registry.npmjs.org/mimic-response/-/mimic-response-3.1.0.tgz",
+			"integrity": "sha512-z0yWI+4FDrrweS8Zmt4Ej5HdJmky15+L2e6Wgn3+iK5fWzb6T3fhNFq2+MeTRb064c6Wr4N/wv0DzQTjNzHNGQ==",
+			"engines": {
+				"node": ">=10"
+			},
+			"funding": {
+				"url": "https://github.com/sponsors/sindresorhus"
+			}
+		},
 		"node_modules/min-indent": {
 			"version": "1.0.1",
 			"resolved": "https://registry.npmjs.org/min-indent/-/min-indent-1.0.1.tgz",
@@ -3354,6 +3619,11 @@
 				"mkdirp": "bin/cmd.js"
 			}
 		},
+		"node_modules/mkdirp-classic": {
+			"version": "0.5.3",
+			"resolved": "https://registry.npmjs.org/mkdirp-classic/-/mkdirp-classic-0.5.3.tgz",
+			"integrity": "sha512-gKLcREMhtuZRwRAfqP3RFW+TK4JqApVBtOIftVgjuABpAtpxhPGaDcfvbhNvD0B8iD1oUr/txX35NjcaY6Ns/A=="
+		},
 		"node_modules/mlly": {
 			"version": "1.2.1",
 			"resolved": "https://registry.npmjs.org/mlly/-/mlly-1.2.1.tgz",
@@ -3457,6 +3727,11 @@
 				"node": "^14 || ^16 || >=18"
 			}
 		},
+		"node_modules/napi-build-utils": {
+			"version": "1.0.2",
+			"resolved": "https://registry.npmjs.org/napi-build-utils/-/napi-build-utils-1.0.2.tgz",
+			"integrity": "sha512-ONmRUqK7zj7DWX0D9ADe03wbwOBZxNAfF20PlGfCWQcD3+/MakShIHrMqx9YwPTfxDdF1zLeL+RGZiR9kGMLdg=="
+		},
 		"node_modules/natural-compare": {
 			"version": "1.4.0",
 			"resolved": "https://registry.npmjs.org/natural-compare/-/natural-compare-1.4.0.tgz",
@@ -3474,6 +3749,34 @@
 			"resolved": "https://registry.npmjs.org/neo-async/-/neo-async-2.6.2.tgz",
 			"integrity": "sha512-Yd3UES5mWCSqR+qNT93S3UoYUkqAZ9lLg8a7g9rimsWmYGK8cVToA4/sF3RrshdyV3sAGMXVUmpMYOw+dLpOuw=="
 		},
+		"node_modules/node-abi": {
+			"version": "3.47.0",
+			"resolved": "https://registry.npmjs.org/node-abi/-/node-abi-3.47.0.tgz",
+			"integrity": "sha512-2s6B2CWZM//kPgwnuI0KrYwNjfdByE25zvAaEpq9IH4zcNsarH8Ihu/UuX6XMPEogDAxkuUFeZn60pXNHAqn3A==",
+			"dependencies": {
+				"semver": "^7.3.5"
+			},
+			"engines": {
+				"node": ">=10"
+			}
+		},
+		"node_modules/node-addon-api": {
+			"version": "6.1.0",
+			"resolved": "https://registry.npmjs.org/node-addon-api/-/node-addon-api-6.1.0.tgz",
+			"integrity": "sha512-+eawOlIgy680F0kBzPUNFhMZGtJ1YmqM6l4+Crf4IkImjYrO/mqPwRMh352g23uIaQKFItcQ64I7KMaJxHgAVA=="
+		},
+		"node_modules/node-gyp-build": {
+			"version": "4.6.1",
+			"resolved": "https://registry.npmjs.org/node-gyp-build/-/node-gyp-build-4.6.1.tgz",
+			"integrity": "sha512-24vnklJmyRS8ViBNI8KbtK/r/DmXQMRiOMXTNz2nrTnAYUwjmEEbnnpB/+kt+yWRv73bPsSPRFddrcIbAxSiMQ==",
+			"optional": true,
+			"peer": true,
+			"bin": {
+				"node-gyp-build": "bin.js",
+				"node-gyp-build-optional": "optional.js",
+				"node-gyp-build-test": "build-test.js"
+			}
+		},
 		"node_modules/node-int64": {
 			"version": "0.4.0",
 			"resolved": "https://registry.npmjs.org/node-int64/-/node-int64-0.4.0.tgz",
@@ -3572,6 +3875,46 @@
 				"url": "https://github.com/sponsors/sindresorhus"
 			}
 		},
+		"node_modules/onnx-proto": {
+			"version": "4.0.4",
+			"resolved": "https://registry.npmjs.org/onnx-proto/-/onnx-proto-4.0.4.tgz",
+			"integrity": "sha512-aldMOB3HRoo6q/phyB6QRQxSt895HNNw82BNyZ2CMh4bjeKv7g/c+VpAFtJuEMVfYLMbRx61hbuqnKceLeDcDA==",
+			"dependencies": {
+				"protobufjs": "^6.8.8"
+			}
+		},
+		"node_modules/onnxruntime-common": {
+			"version": "1.14.0",
+			"resolved": "https://registry.npmjs.org/onnxruntime-common/-/onnxruntime-common-1.14.0.tgz",
+			"integrity": "sha512-3LJpegM2iMNRX2wUmtYfeX/ytfOzNwAWKSq1HbRrKc9+uqG/FsEA0bbKZl1btQeZaXhC26l44NWpNUeXPII7Ew=="
+		},
+		"node_modules/onnxruntime-node": {
+			"version": "1.14.0",
+			"resolved": "https://registry.npmjs.org/onnxruntime-node/-/onnxruntime-node-1.14.0.tgz",
+			"integrity": "sha512-5ba7TWomIV/9b6NH/1x/8QEeowsb+jBEvFzU6z0T4mNsFwdPqXeFUM7uxC6QeSRkEbWu3qEB0VMjrvzN/0S9+w==",
+			"optional": true,
+			"os": [
+				"win32",
+				"darwin",
+				"linux"
+			],
+			"dependencies": {
+				"onnxruntime-common": "~1.14.0"
+			}
+		},
+		"node_modules/onnxruntime-web": {
+			"version": "1.14.0",
+			"resolved": "https://registry.npmjs.org/onnxruntime-web/-/onnxruntime-web-1.14.0.tgz",
+			"integrity": "sha512-Kcqf43UMfW8mCydVGcX9OMXI2VN17c0p6XvR7IPSZzBf/6lteBzXHvcEVWDPmCKuGombl997HgLqj91F11DzXw==",
+			"dependencies": {
+				"flatbuffers": "^1.12.0",
+				"guid-typescript": "^1.0.9",
+				"long": "^4.0.0",
+				"onnx-proto": "^4.0.4",
+				"onnxruntime-common": "~1.14.0",
+				"platform": "^1.3.6"
+			}
+		},
 		"node_modules/openid-client": {
 			"version": "5.4.2",
 			"resolved": "https://registry.npmjs.org/openid-client/-/openid-client-5.4.2.tgz",
@@ -3790,6 +4133,11 @@
 				"pathe": "^1.1.0"
 			}
 		},
+		"node_modules/platform": {
+			"version": "1.3.6",
+			"resolved": "https://registry.npmjs.org/platform/-/platform-1.3.6.tgz",
+			"integrity": "sha512-fnWVljUchTro6RiCFvCXBbNhJc2NijN7oIQxbwsyL0buWJPG85v81ehlHI9fXrJsMNgTofEoWIQeClKpgxFLrg=="
+		},
 		"node_modules/postcss": {
 			"version": "8.4.23",
 			"resolved": "https://registry.npmjs.org/postcss/-/postcss-8.4.23.tgz",
@@ -3947,6 +4295,57 @@
 				"node": "^10 || ^12 || ^13.7 || ^14 || >=15.0.1"
 			}
 		},
+		"node_modules/prebuild-install": {
+			"version": "7.1.1",
+			"resolved": "https://registry.npmjs.org/prebuild-install/-/prebuild-install-7.1.1.tgz",
+			"integrity": "sha512-jAXscXWMcCK8GgCoHOfIr0ODh5ai8mj63L2nWrjuAgXE6tDyYGnx4/8o/rCgU+B4JSyZBKbeZqzhtwtC3ovxjw==",
+			"dependencies": {
+				"detect-libc": "^2.0.0",
+				"expand-template": "^2.0.3",
+				"github-from-package": "0.0.0",
+				"minimist": "^1.2.3",
+				"mkdirp-classic": "^0.5.3",
+				"napi-build-utils": "^1.0.1",
+				"node-abi": "^3.3.0",
+				"pump": "^3.0.0",
+				"rc": "^1.2.7",
+				"simple-get": "^4.0.0",
+				"tar-fs": "^2.0.0",
+				"tunnel-agent": "^0.6.0"
+			},
+			"bin": {
+				"prebuild-install": "bin.js"
+			},
+			"engines": {
+				"node": ">=10"
+			}
+		},
+		"node_modules/prebuild-install/node_modules/tar-fs": {
+			"version": "2.1.1",
+			"resolved": "https://registry.npmjs.org/tar-fs/-/tar-fs-2.1.1.tgz",
+			"integrity": "sha512-V0r2Y9scmbDRLCNex/+hYzvp/zyYjvFbHPNgVTKfQvVrb6guiE/fxP+XblDNR011utopbkex2nM4dHNV6GDsng==",
+			"dependencies": {
+				"chownr": "^1.1.1",
+				"mkdirp-classic": "^0.5.2",
+				"pump": "^3.0.0",
+				"tar-stream": "^2.1.4"
+			}
+		},
+		"node_modules/prebuild-install/node_modules/tar-stream": {
+			"version": "2.2.0",
+			"resolved": "https://registry.npmjs.org/tar-stream/-/tar-stream-2.2.0.tgz",
+			"integrity": "sha512-ujeqbceABgwMZxEJnk2HDY2DlnUZ+9oEcb1KzTVfYHio0UE6dG71n60d8D2I4qNvleWrrXpmjpt7vZeF1LnMZQ==",
+			"dependencies": {
+				"bl": "^4.0.3",
+				"end-of-stream": "^1.4.1",
+				"fs-constants": "^1.0.0",
+				"inherits": "^2.0.3",
+				"readable-stream": "^3.1.1"
+			},
+			"engines": {
+				"node": ">=6"
+			}
+		},
 		"node_modules/prelude-ls": {
 			"version": "1.2.1",
 			"resolved": "https://registry.npmjs.org/prelude-ls/-/prelude-ls-1.2.1.tgz",
@@ -4081,11 +4480,45 @@
 				"url": "https://github.com/chalk/ansi-styles?sponsor=1"
 			}
 		},
+		"node_modules/protobufjs": {
+			"version": "6.11.4",
+			"resolved": "https://registry.npmjs.org/protobufjs/-/protobufjs-6.11.4.tgz",
+			"integrity": "sha512-5kQWPaJHi1WoCpjTGszzQ32PG2F4+wRY6BmAT4Vfw56Q2FZ4YZzK20xUYQH4YkfehY1e6QSICrJquM6xXZNcrw==",
+			"hasInstallScript": true,
+			"dependencies": {
+				"@protobufjs/aspromise": "^1.1.2",
+				"@protobufjs/base64": "^1.1.2",
+				"@protobufjs/codegen": "^2.0.4",
+				"@protobufjs/eventemitter": "^1.1.0",
+				"@protobufjs/fetch": "^1.1.0",
+				"@protobufjs/float": "^1.0.2",
+				"@protobufjs/inquire": "^1.1.0",
+				"@protobufjs/path": "^1.1.2",
+				"@protobufjs/pool": "^1.1.0",
+				"@protobufjs/utf8": "^1.1.0",
+				"@types/long": "^4.0.1",
+				"@types/node": ">=13.7.0",
+				"long": "^4.0.0"
+			},
+			"bin": {
+				"pbjs": "bin/pbjs",
+				"pbts": "bin/pbts"
+			}
+		},
 		"node_modules/psl": {
 			"version": "1.9.0",
 			"resolved": "https://registry.npmjs.org/psl/-/psl-1.9.0.tgz",
 			"integrity": "sha512-E/ZsdU4HLs/68gYzgGTkMicWTLPdAftJLfJFlLUAAKZGkStNU72sZjT66SnMDVOfOWY/YAoiD7Jxa9iHvngcag=="
 		},
+		"node_modules/pump": {
+			"version": "3.0.0",
+			"resolved": "https://registry.npmjs.org/pump/-/pump-3.0.0.tgz",
+			"integrity": "sha512-LwZy+p3SFs1Pytd/jYct4wpv49HiYCqd9Rlc5ZVdk0V+8Yzv6jR5Blk3TRmPL1ft69TxP0IMZGJ+WPFU2BFhww==",
+			"dependencies": {
+				"end-of-stream": "^1.1.0",
+				"once": "^1.3.1"
+			}
+		},
 		"node_modules/punycode": {
 			"version": "2.3.0",
 			"resolved": "https://registry.npmjs.org/punycode/-/punycode-2.3.0.tgz",
@@ -4127,6 +4560,11 @@
 				}
 			]
 		},
+		"node_modules/queue-tick": {
+			"version": "1.0.1",
+			"resolved": "https://registry.npmjs.org/queue-tick/-/queue-tick-1.0.1.tgz",
+			"integrity": "sha512-kJt5qhMxoszgU/62PLP1CJytzd2NKetjSRnyuj31fDd3Rlcz3fzlFdFLD1SItunPwyqEOkca6GbV612BWfaBag=="
+		},
 		"node_modules/quick-lru": {
 			"version": "5.1.1",
 			"resolved": "https://registry.npmjs.org/quick-lru/-/quick-lru-5.1.1.tgz",
@@ -4138,6 +4576,28 @@
 				"url": "https://github.com/sponsors/sindresorhus"
 			}
 		},
+		"node_modules/rc": {
+			"version": "1.2.8",
+			"resolved": "https://registry.npmjs.org/rc/-/rc-1.2.8.tgz",
+			"integrity": "sha512-y3bGgqKj3QBdxLbLkomlohkvsA8gdAiUQlSBJnBhfn+BPxg4bc62d8TcBW15wavDfgexCgccckhcZvywyQYPOw==",
+			"dependencies": {
+				"deep-extend": "^0.6.0",
+				"ini": "~1.3.0",
+				"minimist": "^1.2.0",
+				"strip-json-comments": "~2.0.1"
+			},
+			"bin": {
+				"rc": "cli.js"
+			}
+		},
+		"node_modules/rc/node_modules/strip-json-comments": {
+			"version": "2.0.1",
+			"resolved": "https://registry.npmjs.org/strip-json-comments/-/strip-json-comments-2.0.1.tgz",
+			"integrity": "sha512-4gB8na07fecVVkOI6Rs4e7T6NOTki5EmL7TUduTs6bu3EdnSycntVJ4re8kgZA+wx9IueI2Y11bfbgwtzuE0KQ==",
+			"engines": {
+				"node": ">=0.10.0"
+			}
+		},
 		"node_modules/react-is": {
 			"version": "17.0.2",
 			"resolved": "https://registry.npmjs.org/react-is/-/react-is-17.0.2.tgz",
@@ -4152,6 +4612,19 @@
 				"pify": "^2.3.0"
 			}
 		},
+		"node_modules/readable-stream": {
+			"version": "3.6.2",
+			"resolved": "https://registry.npmjs.org/readable-stream/-/readable-stream-3.6.2.tgz",
+			"integrity": "sha512-9u/sniCrY3D5WdsERHzHE4G2YCXqoG5FTHUiCC4SIbr6XcLZBY05ya9EKjYek9O5xOAwjGq+1JdGBAS7Q9ScoA==",
+			"dependencies": {
+				"inherits": "^2.0.3",
+				"string_decoder": "^1.1.1",
+				"util-deprecate": "^1.0.1"
+			},
+			"engines": {
+				"node": ">= 6"
+			}
+		},
 		"node_modules/readdirp": {
 			"version": "3.6.0",
 			"resolved": "https://registry.npmjs.org/readdirp/-/readdirp-3.6.0.tgz",
@@ -4284,6 +4757,25 @@
 				"node": ">=6"
 			}
 		},
+		"node_modules/safe-buffer": {
+			"version": "5.2.1",
+			"resolved": "https://registry.npmjs.org/safe-buffer/-/safe-buffer-5.2.1.tgz",
+			"integrity": "sha512-rp3So07KcdmmKbGvgaNxQSJr7bGVSVk5S9Eq1F+ppbRo70+YeaDxkw5Dd8NPN+GD6bjnYm2VuPuCXmpuYvmCXQ==",
+			"funding": [
+				{
+					"type": "github",
+					"url": "https://github.com/sponsors/feross"
+				},
+				{
+					"type": "patreon",
+					"url": "https://www.patreon.com/feross"
+				},
+				{
+					"type": "consulting",
+					"url": "https://feross.org/support"
+				}
+			]
+		},
 		"node_modules/safer-buffer": {
 			"version": "2.1.2",
 			"resolved": "https://registry.npmjs.org/safer-buffer/-/safer-buffer-2.1.2.tgz",
@@ -4337,10 +4829,9 @@
 			}
 		},
 		"node_modules/semver": {
-			"version": "7.3.8",
-			"resolved": "https://registry.npmjs.org/semver/-/semver-7.3.8.tgz",
-			"integrity": "sha512-NB1ctGL5rlHrPJtFDVIVzTyQylMLu9N9VICA6HSFJo8MCGVTMW6gfpicwKmmK/dAjTOrqu5l63JJOpDSrAis3A==",
-			"dev": true,
+			"version": "7.5.4",
+			"resolved": "https://registry.npmjs.org/semver/-/semver-7.5.4.tgz",
+			"integrity": "sha512-1bCSESV6Pv+i21Hvpxp3Dx+pSD8lIPt8uVjRrxAUt/nbswYc+tK6Y2btiULjd4+fnq15PX+nqQDC7Oft7WkwcA==",
 			"dependencies": {
 				"lru-cache": "^6.0.0"
 			},
@@ -4365,6 +4856,28 @@
 			"integrity": "sha512-RVnVQxTXuerk653XfuliOxBP81Sf0+qfQE73LIYKcyMYHG94AuH0kgrQpRDuTZnSmjpysHmzxJXKNfa6PjFhyQ==",
 			"dev": true
 		},
+		"node_modules/sharp": {
+			"version": "0.32.5",
+			"resolved": "https://registry.npmjs.org/sharp/-/sharp-0.32.5.tgz",
+			"integrity": "sha512-0dap3iysgDkNaPOaOL4X/0akdu0ma62GcdC2NBQ+93eqpePdDdr2/LM0sFdDSMmN7yS+odyZtPsb7tx/cYBKnQ==",
+			"hasInstallScript": true,
+			"dependencies": {
+				"color": "^4.2.3",
+				"detect-libc": "^2.0.2",
+				"node-addon-api": "^6.1.0",
+				"prebuild-install": "^7.1.1",
+				"semver": "^7.5.4",
+				"simple-get": "^4.0.1",
+				"tar-fs": "^3.0.4",
+				"tunnel-agent": "^0.6.0"
+			},
+			"engines": {
+				"node": ">=14.15.0"
+			},
+			"funding": {
+				"url": "https://opencollective.com/libvips"
+			}
+		},
 		"node_modules/shebang-command": {
 			"version": "2.0.0",
 			"resolved": "https://registry.npmjs.org/shebang-command/-/shebang-command-2.0.0.tgz",
@@ -4398,6 +4911,57 @@
 			"integrity": "sha512-wnD2ZE+l+SPC/uoS0vXeE9L1+0wuaMqKlfz9AMUo38JsyLSBWSFcHR1Rri62LZc12vLr1gb3jl7iwQhgwpAbGQ==",
 			"dev": true
 		},
+		"node_modules/simple-concat": {
+			"version": "1.0.1",
+			"resolved": "https://registry.npmjs.org/simple-concat/-/simple-concat-1.0.1.tgz",
+			"integrity": "sha512-cSFtAPtRhljv69IK0hTVZQ+OfE9nePi/rtJmw5UjHeVyVroEqJXP1sFztKUy1qU+xvz3u/sfYJLa947b7nAN2Q==",
+			"funding": [
+				{
+					"type": "github",
+					"url": "https://github.com/sponsors/feross"
+				},
+				{
+					"type": "patreon",
+					"url": "https://www.patreon.com/feross"
+				},
+				{
+					"type": "consulting",
+					"url": "https://feross.org/support"
+				}
+			]
+		},
+		"node_modules/simple-get": {
+			"version": "4.0.1",
+			"resolved": "https://registry.npmjs.org/simple-get/-/simple-get-4.0.1.tgz",
+			"integrity": "sha512-brv7p5WgH0jmQJr1ZDDfKDOSeWWg+OVypG99A/5vYGPqJ6pxiaHLy8nxtFjBA7oMa01ebA9gfh1uMCFqOuXxvA==",
+			"funding": [
+				{
+					"type": "github",
+					"url": "https://github.com/sponsors/feross"
+				},
+				{
+					"type": "patreon",
+					"url": "https://www.patreon.com/feross"
+				},
+				{
+					"type": "consulting",
+					"url": "https://feross.org/support"
+				}
+			],
+			"dependencies": {
+				"decompress-response": "^6.0.0",
+				"once": "^1.3.1",
+				"simple-concat": "^1.0.0"
+			}
+		},
+		"node_modules/simple-swizzle": {
+			"version": "0.2.2",
+			"resolved": "https://registry.npmjs.org/simple-swizzle/-/simple-swizzle-0.2.2.tgz",
+			"integrity": "sha512-JA//kQgZtbuY83m+xT+tXJkmJncGMTFT+C+g2h2R9uxkYIrE2yy9sgmcLhCnw57/WSD+Eh3J97FPEDFnbXnDUg==",
+			"dependencies": {
+				"is-arrayish": "^0.3.1"
+			}
+		},
 		"node_modules/sirv": {
 			"version": "2.0.2",
 			"resolved": "https://registry.npmjs.org/sirv/-/sirv-2.0.2.tgz",
@@ -4508,6 +5072,23 @@
 				"node": ">=10.0.0"
 			}
 		},
+		"node_modules/streamx": {
+			"version": "2.15.1",
+			"resolved": "https://registry.npmjs.org/streamx/-/streamx-2.15.1.tgz",
+			"integrity": "sha512-fQMzy2O/Q47rgwErk/eGeLu/roaFWV0jVsogDmrszM9uIw8L5OA+t+V93MgYlufNptfjmYR1tOMWhei/Eh7TQA==",
+			"dependencies": {
+				"fast-fifo": "^1.1.0",
+				"queue-tick": "^1.0.1"
+			}
+		},
+		"node_modules/string_decoder": {
+			"version": "1.3.0",
+			"resolved": "https://registry.npmjs.org/string_decoder/-/string_decoder-1.3.0.tgz",
+			"integrity": "sha512-hkRX8U1WjJFd8LsDJ2yQ/wWWxaopEsABU1XfkM8A+j0+85JAGppt16cr1Whg6KIbb4okU6Mql6BOj+uup/wKeA==",
+			"dependencies": {
+				"safe-buffer": "~5.2.0"
+			}
+		},
 		"node_modules/strip-ansi": {
 			"version": "6.0.1",
 			"resolved": "https://registry.npmjs.org/strip-ansi/-/strip-ansi-6.0.1.tgz",
@@ -4865,6 +5446,26 @@
 				"postcss": "^8.0.9"
 			}
 		},
+		"node_modules/tar-fs": {
+			"version": "3.0.4",
+			"resolved": "https://registry.npmjs.org/tar-fs/-/tar-fs-3.0.4.tgz",
+			"integrity": "sha512-5AFQU8b9qLfZCX9zp2duONhPmZv0hGYiBPJsyUdqMjzq/mqVpy/rEUSeHk1+YitmxugaptgBh5oDGU3VsAJq4w==",
+			"dependencies": {
+				"mkdirp-classic": "^0.5.2",
+				"pump": "^3.0.0",
+				"tar-stream": "^3.1.5"
+			}
+		},
+		"node_modules/tar-stream": {
+			"version": "3.1.6",
+			"resolved": "https://registry.npmjs.org/tar-stream/-/tar-stream-3.1.6.tgz",
+			"integrity": "sha512-B/UyjYwPpMBv+PaFSWAmtYjwdrlEaZQEhMIBFNC5oEG8lpiW8XjcSdmEaClj28ArfKScKHs2nshz3k2le6crsg==",
+			"dependencies": {
+				"b4a": "^1.6.4",
+				"fast-fifo": "^1.2.0",
+				"streamx": "^2.15.0"
+			}
+		},
 		"node_modules/text-table": {
 			"version": "0.2.0",
 			"resolved": "https://registry.npmjs.org/text-table/-/text-table-0.2.0.tgz",
@@ -5023,6 +5624,17 @@
 			"integrity": "sha512-Xni35NKzjgMrwevysHTCArtLDpPvye8zV/0E4EyYn43P7/7qvQwPh9BGkHewbMulVntbigmcT7rdX3BNo9wRJg==",
 			"dev": true
 		},
+		"node_modules/tunnel-agent": {
+			"version": "0.6.0",
+			"resolved": "https://registry.npmjs.org/tunnel-agent/-/tunnel-agent-0.6.0.tgz",
+			"integrity": "sha512-McnNiV1l8RYeY8tBgEpuodCC1mLUdbSN+CYBL7kJsJNInOP8UjDDEwdk6Mw60vdLLrr5NHKZhMAOSrR2NZuQ+w==",
+			"dependencies": {
+				"safe-buffer": "^5.0.1"
+			},
+			"engines": {
+				"node": "*"
+			}
+		},
 		"node_modules/type-check": {
 			"version": "0.4.0",
 			"resolved": "https://registry.npmjs.org/type-check/-/type-check-0.4.0.tgz",

package.json

@@ -42,6 +42,7 @@
 	"dependencies": {
 		"@huggingface/hub": "^0.5.1",
 		"@huggingface/inference": "^2.2.0",
+		"@xenova/transformers": "^2.6.0",
 		"autoprefixer": "^10.4.14",
 		"aws4fetch": "^1.0.17",
 		"date-fns": "^2.29.3",

src/lib/buildPrompt.ts

@@ -3,6 +3,7 @@ import type { Message } from "./types/Message";
 import { collections } from "$lib/server/database";
 import { ObjectId } from "mongodb";
 import { authCondition } from "./server/auth";
+import { format } from "date-fns";
 /**
  * Convert [{user: "assistant", content: "hi"}, {user: "user", content: "hello"}] to:
  *
@@ -39,13 +40,16 @@ export async function buildPrompt({
 
 		if (!conversation) throw new Error("Conversation not found");
 
-		if (webSearch.summary) {
+		if (webSearch.context) {
+			const messagesWithoutLastUsrMsg = messages.slice(0, -1);
+			const lastUserMsg = messages.slice(-1)[0];
+			const currentDate = format(new Date(), "MMMM d, yyyy");
 			messages = [
+				...messagesWithoutLastUsrMsg,
 				{
-					from: "assistant",
-					content: `The following context was found while searching the internet: ${webSearch.summary}`,
+					from: "user",
+					content: `Please answer my question "${lastUserMsg.content}" using the supplied context below (paragrpahs from various websites). For the context, today is ${currentDate}: \n=====================\n${webSearch.context}\n=====================\nSo my question is "${lastUserMsg.content}"`,
 				},
-				...messages,
 			];
 		}
 	}

src/lib/components/chat/ChatMessage.svelte

@@ -13,7 +13,7 @@
 	import CarbonThumbsDown from "~icons/carbon/thumbs-down";
 	import { PUBLIC_SEP_TOKEN } from "$lib/constants/publicSepToken";
 	import type { Model } from "$lib/types/Model";
-	import type { WebSearchMessage } from "$lib/types/WebSearch";
+	import type { WebSearchMessage, WebSearchMessageSources } from "$lib/types/WebSearch";
 
 	import OpenWebSearchResults from "../OpenWebSearchResults.svelte";
 
@@ -100,6 +100,10 @@
 	$: webSearchIsDone =
 		webSearchMessages.length > 0 &&
 		webSearchMessages[webSearchMessages.length - 1].type === "result";
+
+	$: webSearchSources = (
+		webSearchMessages.filter(({ type }) => type === "sources")?.[0] as WebSearchMessageSources
+	)?.sources;
 </script>
 
 {#if message.from === "assistant"}
@@ -140,6 +144,26 @@
 					{/if}
 				{/each}
 			</div>
+			<!-- Web Search sources -->
+			{#if webSearchSources?.length}
+				<div class="mt-4 flex flex-wrap items-center gap-x-2 gap-y-1.5 text-sm">
+					<div class="text-gray-400">Sources:</div>
+					{#each webSearchSources as { link, title, hostname }}
+						<a
+							class="flex items-center gap-2 whitespace-nowrap rounded-lg border bg-white px-2 py-1.5 leading-none hover:border-gray-300  dark:border-gray-800 dark:bg-gray-900 dark:hover:border-gray-700"
+							href={link}
+							target="_blank"
+						>
+							<img
+								class="h-3.5 w-3.5 rounded"
+								src="https://www.google.com/s2/favicons?sz=64&domain_url={hostname}"
+								alt="{title} favicon"
+							/>
+							<div>{hostname.replace(/^www\./, "")}</div>
+						</a>
+					{/each}
+				</div>
+			{/if}
 		</div>
 		{#if isAuthor && !loading && message.content}
 			<div

src/lib/server/generateFromDefaultEndpoint.ts

@@ -29,7 +29,7 @@ export async function generateFromDefaultEndpoint(
 
 	if (randomEndpoint.host === "sagemaker") {
 		const requestParams = JSON.stringify({
-			...newParameters,
+			parameters: newParameters,
 			inputs: prompt,
 		});
 
@@ -56,7 +56,7 @@ export async function generateFromDefaultEndpoint(
 			},
 			method: "POST",
 			body: JSON.stringify({
-				...newParameters,
+				parameters: newParameters,
 				inputs: prompt,
 			}),
 			signal: abortController.signal,

src/lib/server/models.ts

@@ -1,9 +1,5 @@
 import { HF_ACCESS_TOKEN, MODELS, OLD_MODELS } from "$env/static/private";
-import type {
-	ChatTemplateInput,
-	WebSearchQueryTemplateInput,
-	WebSearchSummaryTemplateInput,
-} from "$lib/types/Template";
+import type { ChatTemplateInput, WebSearchQueryTemplateInput } from "$lib/types/Template";
 import { compileTemplate } from "$lib/utils/template";
 import { z } from "zod";
 
@@ -71,28 +67,14 @@ const modelsRaw = z
 						"{{/each}}" +
 						"{{assistantMessageToken}}"
 				),
-			webSearchSummaryPromptTemplate: z
-				.string()
-				.default(
-					"{{userMessageToken}}{{answer}}{{userMessageEndToken}}" +
-						"{{userMessageToken}}" +
-						"The text above should be summarized to best answer the query: {{query}}." +
-						"{{userMessageEndToken}}" +
-						"{{assistantMessageToken}}Summary: "
-				),
 			webSearchQueryPromptTemplate: z
 				.string()
 				.default(
 					"{{userMessageToken}}" +
-						"The following messages were written by a user, trying to answer a question." +
+						'My question is: "{{message.content}}". ' +
+						"Based on the conversation history (my previous questions are: {{previousMessages}}), give me an appropriate query to answer my question for google search. You should not say more than query. You should not say any words except the query. For the context, today is {{currentDate}}" +
 						"{{userMessageEndToken}}" +
-						"{{#each messages}}" +
-						"{{#ifUser}}{{@root.userMessageToken}}{{content}}{{@root.userMessageEndToken}}{{/ifUser}}" +
-						"{{/each}}" +
-						"{{userMessageToken}}" +
-						"What plain-text english sentence would you input into Google to answer the last question? Answer with a short (10 words max) simple sentence." +
-						"{{userMessageEndToken}}" +
-						"{{assistantMessageToken}}Query: "
+						"{{assistantMessageToken}}"
 				),
 			promptExamples: z
 				.array(
@@ -122,10 +104,6 @@ export const models = await Promise.all(
 		userMessageEndToken: m?.userMessageEndToken || m?.messageEndToken,
 		assistantMessageEndToken: m?.assistantMessageEndToken || m?.messageEndToken,
 		chatPromptRender: compileTemplate<ChatTemplateInput>(m.chatPromptTemplate, m),
-		webSearchSummaryPromptRender: compileTemplate<WebSearchSummaryTemplateInput>(
-			m.webSearchSummaryPromptTemplate,
-			m
-		),
 		webSearchQueryPromptRender: compileTemplate<WebSearchQueryTemplateInput>(
 			m.webSearchQueryPromptTemplate,
 			m

src/lib/server/websearch/generateQuery.ts

@@ -1,12 +1,45 @@
 import type { Message } from "$lib/types/Message";
+import { format } from "date-fns";
 import { generateFromDefaultEndpoint } from "../generateFromDefaultEndpoint";
 import { defaultModel } from "../models";
 
 export async function generateQuery(messages: Message[]) {
-	const promptSearchQuery = defaultModel.webSearchQueryPromptRender({ messages });
+	const currentDate = format(new Date(), "MMMM d, yyyy");
+	const userMessages = messages.filter(({ from }) => from === "user");
+	const previousUserMessages = userMessages.slice(0, -1);
+	const lastMessage = userMessages.slice(-1)[0];
+	const promptSearchQuery = defaultModel.webSearchQueryPromptRender({
+		message: lastMessage,
+		previousMessages: previousUserMessages.map(({ content }) => content).join(" "),
+		currentDate,
+	});
 	const searchQuery = await generateFromDefaultEndpoint(promptSearchQuery).then((query) => {
-		const arr = query.split(/\r?\n/);
-		return arr[0].length > 0 ? arr[0] : arr[1];
+		// example of generating google query:
+		// case 1
+		// user: tell me what happened yesterday
+		// LLM: google query is "news september 12, 2023"
+		// the regex below will try to capture the last "phrase" (i.e. words between quotes or double quotes or ticks)
+		// in this case, it is "news september 12, 2023"
+		// if there is no "phrase", we will just use the user query, which was "tell me what happened yesterday"
+		const regexLastPhrase = /("|'|`)((?:(?!\1).)+)\1$/;
+		let match = query.match(regexLastPhrase);
+		if (match) {
+			return match[2];
+		}
+
+		// case 2
+		// user: tell me what happened yesterday
+		// LLM: Here is a query: news september 12, 2023
+		// the regex below will try to capture the last sentences starting from :
+		// in this case, it is "news september 12, 2023"
+		// if there is no math, we will just use the user query, which was "tell me what happened yesterday"
+		const regexColon = /:\s(.*)$/;
+		match = query.match(regexColon);
+		if (match) {
+			return match[1];
+		}
+
+		return lastMessage.content;
 	});
 
 	return searchQuery;

src/lib/server/websearch/parseWeb.ts

@@ -1,32 +1,5 @@
 import { JSDOM, VirtualConsole } from "jsdom";
 
-function removeTags(node: Node) {
-	if (node.hasChildNodes()) {
-		node.childNodes.forEach((childNode) => {
-			if (node.nodeName === "SCRIPT" || node.nodeName === "STYLE") {
-				node.removeChild(childNode);
-			} else {
-				removeTags(childNode);
-			}
-		});
-	}
-}
-function naiveInnerText(node: Node): string {
-	const Node = node; // We need Node(DOM's Node) for the constants, but Node doesn't exist in the nodejs global space, and any Node instance references the constants through the prototype chain
-	return [...node.childNodes]
-		.map((childNode) => {
-			switch (childNode.nodeType) {
-				case Node.TEXT_NODE:
-					return node.textContent;
-				case Node.ELEMENT_NODE:
-					return naiveInnerText(childNode);
-				default:
-					return "";
-			}
-		})
-		.join("\n");
-}
-
 export async function parseWeb(url: string) {
 	const abortController = new AbortController();
 	setTimeout(() => abortController.abort(), 10000);
@@ -44,13 +17,16 @@ export async function parseWeb(url: string) {
 		virtualConsole,
 	});
 
-	const body = dom.window.document.querySelector("body");
-	if (!body) throw new Error("body of the webpage is null");
-
-	removeTags(body);
+	const { document } = dom.window;
+	const textElTags = "p";
+	const paragraphs = document.querySelectorAll(textElTags);
+	if (!paragraphs.length) {
+		throw new Error(`webpage doesn't have any "${textElTags}" element`);
+	}
+	const paragraphTexts = Array.from(paragraphs).map((p) => p.textContent);
 
-	// recursively extract text content from the body and then remove newlines and multiple spaces
-	const text = (naiveInnerText(body) ?? "").replace(/ {2}|\r\n|\n|\r/gm, "");
+	// combine text contents from paragraphs and then remove newlines and multiple spaces
+	const text = paragraphTexts.join(" ").replace(/ {2}|\r\n|\n|\r/gm, "");
 
 	return text;
 }

src/lib/server/websearch/searchWeb.ts

@@ -42,8 +42,6 @@ export async function searchWebSerper(query: string) {
 
 	return {
 		organic_results: data["organic"] ?? [],
-		knowledge_graph: data["knowledgeGraph"] ?? null,
-		answer_box: data["answerBox"] ?? null,
 	};
 }
 

src/lib/server/websearch/sentenceSimilarity.ts

@@ -0,0 +1,39 @@
+import type { Tensor } from "@xenova/transformers";
+import { pipeline, dot } from "@xenova/transformers";
+
+// see here: https://github.com/nmslib/hnswlib/blob/359b2ba87358224963986f709e593d799064ace6/README.md?plain=1#L34
+function innerProduct(tensor1: Tensor, tensor2: Tensor) {
+	return 1.0 - dot(tensor1.data, tensor2.data);
+}
+
+const extractor = await pipeline("feature-extraction", "Xenova/e5-small-v2");
+
+export async function findSimilarSentences(
+	query: string,
+	sentences: string[],
+	{ topK = 5 }: { topK: number }
+) {
+	// this preprocessing step is suggested for e5-small-v2 model
+	// see more: https://huggingface.co/intfloat/e5-small-v2/blob/main/README.md?code=true#L2631
+	const input = [`query: ${query}`, ...sentences.map((s) => `passage: ${s}`)];
+	const output: Tensor = await extractor(input, { pooling: "mean", normalize: true });
+
+	const queryTensor: Tensor = output[0];
+	const sentencesTensor: Tensor = output.slice([1, input.length - 1]);
+
+	const distancesFromQuery: { distance: number; index: number }[] = [...sentencesTensor].map(
+		(sentenceTensor: Tensor, index: number) => {
+			return {
+				distance: innerProduct(queryTensor, sentenceTensor),
+				index: index,
+			};
+		}
+	);
+
+	distancesFromQuery.sort((a, b) => {
+		return a.distance - b.distance;
+	});
+
+	// Return the indexes of the closest topK sentences
+	return distancesFromQuery.slice(0, topK).map((item) => item.index);
+}

src/lib/server/websearch/summarizeWeb.ts

@@ -1,39 +0,0 @@
-import { HF_ACCESS_TOKEN } from "$env/static/private";
-import { HfInference } from "@huggingface/inference";
-import { defaultModel } from "$lib/server/models";
-import type { BackendModel } from "../models";
-import { generateFromDefaultEndpoint } from "../generateFromDefaultEndpoint";
-
-export async function summarizeWeb(content: string, query: string, model: BackendModel) {
-	// if HF_ACCESS_TOKEN is set, we use a HF dedicated endpoint for summarization
-	try {
-		if (HF_ACCESS_TOKEN) {
-			const summary = (
-				await new HfInference(HF_ACCESS_TOKEN).summarization({
-					model: "facebook/bart-large-cnn",
-					inputs: content,
-					parameters: {
-						max_length: 512,
-					},
-				})
-			).summary_text;
-			return summary;
-		}
-	} catch (e) {
-		console.log(e);
-	}
-
-	// else we use the LLM to generate a summary
-	const summaryPrompt = defaultModel.webSearchSummaryPromptRender({
-		answer: content
-			.split(" ")
-			.slice(0, model.parameters?.truncate ?? 0)
-			.join(" "),
-		query: query,
-	});
-	const summary = await generateFromDefaultEndpoint(summaryPrompt).then((txt: string) =>
-		txt.trim()
-	);
-
-	return summary;
-}

src/lib/types/Template.ts

@@ -13,11 +13,8 @@ export type ChatTemplateInput = {
 	preprompt?: string;
 };
 
-export type WebSearchSummaryTemplateInput = {
-	answer: string;
-	query: string;
-};
-
 export type WebSearchQueryTemplateInput = {
-	messages: Pick<Message, "from" | "content">[];
+	message: Pick<Message, "from" | "content">;
+	previousMessages: string;
+	currentDate: string;
 };

src/lib/types/WebSearch.ts

@@ -2,6 +2,12 @@ import type { ObjectId } from "mongodb";
 import type { Conversation } from "./Conversation";
 import type { Timestamps } from "./Timestamps";
 
+export interface WebSearchSource {
+	title: string;
+	link: string;
+	hostname: string;
+}
+
 export interface WebSearch extends Timestamps {
 	_id: ObjectId;
 
@@ -10,10 +16,9 @@ export interface WebSearch extends Timestamps {
 	prompt: string;
 
 	searchQuery: string;
-	results: string[];
-	knowledgeGraph: string;
-	answerBox: string;
-	summary: string;
+	results: WebSearchSource[];
+	context: string;
+	contextSources: WebSearchSource[];
 
 	messages: WebSearchMessage[];
 }
@@ -35,7 +40,13 @@ export type WebSearchMessageResult = {
 	id: string;
 };
 
+export type WebSearchMessageSources = {
+	type: "sources";
+	sources: WebSearchSource[];
+};
+
 export type WebSearchMessage =
 	| WebSearchMessageUpdate
 	| WebSearchMessageResult
-	| WebSearchMessageError;
+	| WebSearchMessageError
+	| WebSearchMessageSources;

src/lib/utils/chunk.ts

@@ -0,0 +1,33 @@
+/**
+ * Chunk array into arrays of length at most `chunkSize`
+ *
+ * @param chunkSize must be greater than or equal to 1
+ */
+export function chunk<T extends unknown[] | string>(arr: T, chunkSize: number): T[] {
+	if (isNaN(chunkSize) || chunkSize < 1) {
+		throw new RangeError("Invalid chunk size: " + chunkSize);
+	}
+
+	if (!arr.length) {
+		return [];
+	}
+
+	/// Small optimization to not chunk buffers unless needed
+	if (arr.length <= chunkSize) {
+		return [arr];
+	}
+
+	return range(Math.ceil(arr.length / chunkSize)).map((i) => {
+		return arr.slice(i * chunkSize, (i + 1) * chunkSize);
+	}) as T[];
+}
+
+function range(n: number, b?: number): number[] {
+	return b
+		? Array(b - n)
+				.fill(0)
+				.map((_, i) => n + i)
+		: Array(n)
+				.fill(0)
+				.map((_, i) => i);
+}

src/routes/conversation/[id]/+page.server.ts

@@ -2,7 +2,7 @@ import { collections } from "$lib/server/database";
 import { ObjectId } from "mongodb";
 import { error } from "@sveltejs/kit";
 import { authCondition } from "$lib/server/auth";
-import type { WebSearchMessageResult } from "$lib/types/WebSearch";
+import type { WebSearchMessageResult, WebSearchMessageSources } from "$lib/types/WebSearch";
 import { UrlDependency } from "$lib/types/UrlDependency";
 
 export const load = async ({ params, depends, locals }) => {
@@ -39,7 +39,11 @@ export const load = async ({ params, depends, locals }) => {
 	const searches = Object.fromEntries(
 		results.map((x) => [
 			x._id.toString(),
-			[...x.messages, { type: "result", id: x._id.toString() } satisfies WebSearchMessageResult],
+			[
+				...x.messages,
+				{ type: "sources", sources: x.contextSources ?? [] } satisfies WebSearchMessageSources,
+				{ type: "result", id: x._id.toString() } satisfies WebSearchMessageResult,
+			],
 		])
 	);
 

src/routes/conversation/[id]/+page.svelte

@@ -184,6 +184,7 @@
 							if (lastSearchMessage.type === "result") {
 								searchResponseId = lastSearchMessage.id;
 								reader.cancel();
+								data.searches[searchResponseId] = [...webSearchMessages];
 								return;
 							}
 						})

src/routes/conversation/[id]/web-search/+server.ts

@@ -1,31 +1,22 @@
 import { authCondition } from "$lib/server/auth";
 import { collections } from "$lib/server/database";
-import { defaultModel } from "$lib/server/models";
 import { searchWeb } from "$lib/server/websearch/searchWeb";
 import type { Message } from "$lib/types/Message";
 import { error } from "@sveltejs/kit";
 import { ObjectId } from "mongodb";
 import { z } from "zod";
-import type { WebSearch } from "$lib/types/WebSearch";
+import type { WebSearch, WebSearchSource } from "$lib/types/WebSearch";
 import { generateQuery } from "$lib/server/websearch/generateQuery";
 import { parseWeb } from "$lib/server/websearch/parseWeb";
-import { summarizeWeb } from "$lib/server/websearch/summarizeWeb";
+import { chunk } from "$lib/utils/chunk";
+import { findSimilarSentences } from "$lib/server/websearch/sentenceSimilarity";
 import { RATE_LIMIT } from "$env/static/private";
 import { ERROR_MESSAGES } from "$lib/stores/errors.js";
 
-interface GenericObject {
-	[key: string]: GenericObject | unknown;
-}
+const MAX_N_PAGES_SCRAPE = 10 as const;
+const MAX_N_PAGES_EMBED = 5 as const;
 
-function removeLinks(obj: GenericObject) {
-	for (const prop in obj) {
-		if (prop.endsWith("link")) delete obj[prop];
-		else if (typeof obj[prop] === "object") removeLinks(obj[prop] as GenericObject);
-	}
-	return obj;
-}
 export async function GET({ params, locals, url, getClientAddress }) {
-	const model = defaultModel;
 	const convId = new ObjectId(params.id);
 	const searchId = new ObjectId();
 
@@ -68,10 +59,9 @@ export async function GET({ params, locals, url, getClientAddress }) {
 				convId: convId,
 				prompt: prompt,
 				searchQuery: "",
-				knowledgeGraph: "",
-				answerBox: "",
 				results: [],
-				summary: "",
+				context: "",
+				contextSources: [],
 				messages: [],
 				createdAt: new Date(),
 				updatedAt: new Date(),
@@ -92,45 +82,62 @@ export async function GET({ params, locals, url, getClientAddress }) {
 
 				appendUpdate("Searching Google", [webSearch.searchQuery]);
 				const results = await searchWeb(webSearch.searchQuery);
-
-				let text = "";
 				webSearch.results =
 					(results.organic_results &&
-						results.organic_results.map((el: { link: string }) => el.link)) ??
+						results.organic_results.map((el: { title: string; link: string }) => {
+							const { title, link } = el;
+							const { hostname } = new URL(link);
+							return { title, link, hostname };
+						})) ??
 					[];
-
-				if (results.answer_box) {
-					// if google returns an answer box, we use it
-					webSearch.answerBox = JSON.stringify(removeLinks(results.answer_box));
-					text = webSearch.answerBox;
-					appendUpdate("Found a Google answer box");
-				} else if (results.knowledge_graph) {
-					// if google returns a knowledge graph, we use it
-					webSearch.knowledgeGraph = JSON.stringify(removeLinks(results.knowledge_graph));
-					text = webSearch.knowledgeGraph;
-					appendUpdate("Found a Google knowledge page");
-				} else if (webSearch.results.length > 0) {
-					let tries = 0;
-
-					while (!text && tries < 3) {
-						const searchUrl = webSearch.results[tries];
-						appendUpdate("Browsing result", [JSON.stringify(searchUrl)]);
+				webSearch.results = webSearch.results
+					.filter(({ link }) => !link.includes("youtube.com")) // filter out youtube links
+					.slice(0, MAX_N_PAGES_SCRAPE); // limit to first 10 links only
+
+				let paragraphChunks: { source: WebSearchSource; text: string }[] = [];
+				if (webSearch.results.length > 0) {
+					appendUpdate("Browsing results");
+					const promises = webSearch.results.map(async (result) => {
+						const { link } = result;
+						let text = "";
 						try {
-							text = await parseWeb(searchUrl);
-							if (!text) throw new Error("text of the webpage is null");
+							text = await parseWeb(link);
+							appendUpdate("Browsing webpage", [link]);
 						} catch (e) {
-							appendUpdate("Error parsing webpage", [], "error");
-							tries++;
+							console.error(`Error parsing webpage "${link}"`, e);
 						}
+						const CHUNK_CAR_LEN = 512;
+						const MAX_N_CHUNKS = 100;
+						const texts = chunk(text, CHUNK_CAR_LEN).slice(0, MAX_N_CHUNKS);
+						return texts.map((t) => ({ source: result, text: t }));
+					});
+					const nestedParagraphChunks = (await Promise.all(promises)).slice(0, MAX_N_PAGES_EMBED);
+					paragraphChunks = nestedParagraphChunks.flat();
+					if (!paragraphChunks.length) {
+						throw new Error("No text found on the first 5 results");
 					}
-					if (!text) throw new Error("No text found on the first 3 results");
 				} else {
 					throw new Error("No results found for this search query");
 				}
 
-				appendUpdate("Creating summary");
-				webSearch.summary = await summarizeWeb(text, webSearch.searchQuery, model);
-				appendUpdate("Injecting summary", [JSON.stringify(webSearch.summary)]);
+				appendUpdate("Extracting relevant information");
+				const topKClosestParagraphs = 8;
+				const texts = paragraphChunks.map(({ text }) => text);
+				const indices = await findSimilarSentences(prompt, texts, {
+					topK: topKClosestParagraphs,
+				});
+				webSearch.context = indices.map((idx) => texts[idx]).join("");
+
+				const usedSources = new Set<string>();
+				for (const idx of indices) {
+					const { source } = paragraphChunks[idx];
+					if (!usedSources.has(source.link)) {
+						usedSources.add(source.link);
+						webSearch.contextSources.push(source);
+					}
+				}
+
+				appendUpdate("Injecting relevant information");
 			} catch (searchError) {
 				if (searchError instanceof Error) {
 					webSearch.messages.push({
@@ -142,6 +149,10 @@ export async function GET({ params, locals, url, getClientAddress }) {
 			}
 
 			const res = await collections.webSearches.insertOne(webSearch);
+			webSearch.messages.push({
+				type: "sources",
+				sources: webSearch.contextSources,
+			});
 			webSearch.messages.push({
 				type: "result",
 				id: res.insertedId.toString(),

src/routes/r/[id]/+page.server.ts

@@ -2,7 +2,7 @@ import type { PageServerLoad } from "./$types";
 import { collections } from "$lib/server/database";
 import { error } from "@sveltejs/kit";
 import { ObjectId } from "mongodb";
-import type { WebSearchMessageResult } from "$lib/types/WebSearch";
+import type { WebSearchMessageResult, WebSearchMessageSources } from "$lib/types/WebSearch";
 
 export const load: PageServerLoad = async ({ params }) => {
 	const conversation = await collections.sharedConversations.findOne({
@@ -22,7 +22,11 @@ export const load: PageServerLoad = async ({ params }) => {
 	const searches = Object.fromEntries(
 		results.map((x) => [
 			x._id.toString(),
-			[...x.messages, { type: "result", id: x._id.toString() } satisfies WebSearchMessageResult],
+			[
+				...x.messages,
+				{ type: "sources", sources: x.contextSources ?? [] } satisfies WebSearchMessageSources,
+				{ type: "result", id: x._id.toString() } satisfies WebSearchMessageResult,
+			],
 		])
 	);