text: custom scraping rulesâ€‹The SitemapLoader uses beautifulsoup4 for the scraping process, and it scrapes every element on the page by default. The SitemapLoader constructor accepts a custom scraping function. This feature can be helpful to tailor the scraping process to your specific needs; for example, you might want to avoid scraping headers or navigation elements. The following example shows how to develop and use a custom function to avoid navigation and header elements.Import the beautifulsoup4 library and define the custom function.pip install beautifulsoup4from bs4 import BeautifulSoupdef remove_nav_and_header_elements(content: BeautifulSoup) -> str:    # Find all 'nav' and 'header' elements in the BeautifulSoup object    nav_elements = content.find_all("nav")    header_elements = content.find_all("header")    # Remove each 'nav' and 'header' element from the BeautifulSoup object    for element in nav_elements + header_elements:        element.decompose()    return str(content.get_text())Add your custom function to the SitemapLoader object.loader = SitemapLoader(    "https://langchain.readthedocs.io/sitemap.xml",    filter_urls=["https://python.langchain.com/en/latest/"],    parsing_function=remove_nav_and_header_elements,)Local Sitemapâ€‹The sitemap loader can also be used to load local files.sitemap_loader = SitemapLoader(web_path="example_data/sitemap.xml", is_local=True)docs = sitemap_loader.load()    Fetching pages: 100%|####################################################################################################################################| 3/3 [00:00<00:00,  3.91it/s]PreviousRSTNextSlackFiltering sitemap URLsAdd custom scraping rulesLocal SitemapCommunityDiscordTwitterGitHubPythonJS/TSMoreHomepageBlogCopyright Â© 2023 LangChain, Inc.