langchain.chains.natbot.crawler.Crawler¶
class langchain.chains.natbot.crawler.Crawler[source]¶
A crawler for web pages.
Security Note: This is an implementation of a crawler that uses a browser viaPlaywright.
This crawler can be used to load arbitrary webpages INCLUDING content
from the local file system.
Control access to who can submit crawling requests and what network access
the crawler has.
Make sure to scope permissions to the minimal permissions necessary for
the application.
See https://python.langchain.com/docs/security for more information.
Methods
__init__()
click(id)
crawl()
enter()
go_to_page(url)
scroll(direction)
type(id, text)
__init__() → None[source]¶
click(id: Union[str, int]) → None[source]¶
crawl() → List[str][source]¶
enter() → None[source]¶
go_to_page(url: str) → None[source]¶
scroll(direction: str) → None[source]¶
type(id: Union[str, int], text: str) → None[source]¶