langchain.document_loaders.generic.GenericLoader¶
class langchain.document_loaders.generic.GenericLoader(blob_loader: BlobLoader, blob_parser: BaseBlobParser)[source]¶
A generic document loader.
A generic document loader that allows combining an arbitrary blob loader with
a blob parser.
Examples
from langchain.document_loaders import GenericLoader
from langchain.document_loaders.blob_loaders import FileSystemBlobLoader
loader = GenericLoader.from_filesystem(path=”path/to/directory”,
glob=”**/[!.]*”,
suffixes=[“.pdf”],
show_progress=True,
)
docs = loader.lazy_load()
next(docs)
Example instantiations to change which files are loaded:
… code-block:: python
# Recursively load all text files in a directory.
loader = GenericLoader.from_filesystem(“/path/to/dir”, glob=”**/*.txt”)
# Recursively load all non-hidden files in a directory.
loader = GenericLoader.from_filesystem(“/path/to/dir”, glob=”**/[!.]*”)
# Load all files in a directory without recursion.
loader = GenericLoader.from_filesystem(“/path/to/dir”, glob=”*”)
Example instantiations to change which parser is used:
… code-block:: python
from langchain.document_loaders.parsers.pdf import PyPDFParser
# Recursively load all text files in a directory.
loader = GenericLoader.from_filesystem(
“/path/to/dir”,
glob=”**/*.pdf”,
parser=PyPDFParser()
)
A generic document loader.
Parameters
blob_loader – A blob loader which knows how to yield blobs
blob_parser – A blob parser which knows how to parse blobs into documents
Methods
__init__(blob_loader, blob_parser)
A generic document loader.
from_filesystem(path, *[, glob, suffixes, ...])