metadata

license: mit
title: crawling rag
sdk: gradio
emoji: 👁
colorFrom: gray
colorTo: purple
short_description: a rag that can crawle website
sdk_version: 5.7.1

🔍 Intelligent Web Search Engine

An advanced AI-powered search engine that provides deep understanding of web content, code analysis, and intelligent insights.

🌟 Features

Multi-model AI analysis
Semantic search and caching
Automatic insights generation
Smart follow-up questions
Code-aware analysis
Related searches

🚀 Deployment to Hugging Face Spaces

Create a new Space:
- Go to huggingface.co/spaces
- Click "Create new Space"
- Choose "Gradio" as the SDK
- Select "CPU" as the hardware
- Name your space (e.g., "intelligent-web-search")
Upload Files:
- Upload all files from the aiws directory
- Make sure to include:
  - app.py
  - search_engine.py
  - requirements.txt
  - packages.txt
Space Settings:
- Go to the "Settings" tab
- Under "Repository secrets", add any required API keys
- Under "Variables", set:
```
PYTHON_PACKAGES_PATH=/home/user/.local/lib/python3.9/site-packages
```
The space will automatically build and deploy your app

📦 Local Development

Clone the repository:

git clone [your-repo-url]
cd aiws

Install dependencies:

pip install -r requirements.txt

Run the app:

python app.py

🔧 Configuration

The search engine uses several AI models:

Summarization: facebook/bart-base
Code Understanding: Salesforce/codet5-small
General QA: google/flan-t5-base
Embeddings: sentence-transformers/all-MiniLM-L6-v2

📝 Usage

Enter your search query
Adjust the maximum number of results (1-20)
Click "Search"
View results including:
- Key insights
- Follow-up questions
- Detailed analysis
- Related searches

🤝 Contributing

Contributions are welcome! Please feel free to submit a Pull Request.