first commit

README.md

@@ -0,0 +1,259 @@
+# PanSea University Search
+
+An AI-powered RAG (Retrieval-Augmented Generation) system for searching ASEAN university admission requirements, designed to help prospective students find accurate and up-to-date information about study opportunities across Southeast Asia.
+
+## 🎯 Problem & Solution
+
+**Problem:** Prospective students worldwide seeking to study abroad face difficulty finding accurate, up-to-date university admission requirements. Information is scattered across PDFs, brochures, and outdated agency websites. Many waste time applying to unsuitable programs due to missing criteria and pay high agent fees.
+
+**Solution:** An LLM-powered, RAG-based platform powered by **SEA-LION multilingual models** that ingests official admissions documents from ASEAN universities. Students can query in any ASEAN language and receive ranked program matches with fees, entry requirements, deadlines, application windows, and source citations.
+
+## 🌟 Features
+
+- 📄 **PDF Document Ingestion**: Upload official university admission documents
+- 🔍 **Intelligent Search**: Natural language queries in multiple ASEAN languages
+- 🎯 **Accurate Responses**: AI-powered answers with source citations
+- 🔗 **Shareable Results**: Generate links to share query results
+- 🌏 **Multi-language Support**: English, Chinese, Malay, Thai, Indonesian, Vietnamese, Filipino
+- 💰 **Advanced Filtering**: Budget range, study level, country preferences
+
+## 🚀 Quick Start
+
+### Prerequisites
+
+- Python 3.11+
+- SEA-LION API Key
+- OpenAI API Key (optional, for fallback embeddings)
+
+### Installation
+
+1. **Clone and navigate to the project:**
+   ```bash
+   cd pansea
+   ```
+
+2. **Activate virtual environment:**
+   ```bash
+   source .venv/bin/activate  # On Windows: .venv\Scripts\activate
+   ```
+
+3. **Install dependencies:**
+   ```bash
+   pip install -r requirements.txt
+   ```
+
+4. **Set up environment variables:**
+   ```bash
+   cp .env.example .env
+   # Edit .env and add your SEA-LION API key (OpenAI key optional for fallback)
+   ```
+
+5. **Run the application:**
+   ```bash
+   streamlit run app.py
+   ```
+
+6. **Open your browser to:** `http://localhost:8501`
+
+### Usage
+
+#### 1. Upload Documents
+- Go to the "Upload Documents" page
+- Enter university name and country
+- Select document type (admission requirements, tuition fees, etc.)
+- Upload PDF files containing university information
+- Click "Process Documents"
+
+#### 2. Search Universities
+- Go to the "Search Universities" page
+- Choose your response language
+- Enter questions like:
+  - "Show me universities in Malaysia for master's degrees with tuition under 40,000 RMB per year"
+  - "专科毕业，无雅思，想在马来西亚读硕士，学费不超过4万人民币/年"
+  - "What are the English proficiency requirements for Singapore universities?"
+- Apply optional filters (budget, study level, countries)
+- Get AI-powered responses with source citations
+
+#### 3. Share Results
+- Each query generates a unique shareable link
+- Share results with friends, family, or education consultants
+- Access shared results without needing to upload documents again
+
+## 📁 Project Structure
+
+```
+pansea/
+├── app.py                 # Main Streamlit application
+├── rag_system.py          # RAG system implementation
+├── requirements.txt       # Python dependencies
+├── .env                   # Environment variables
+├── .venv/                 # Virtual environment
+├── chroma_db/            # Vector database storage
+├── documents/            # Uploaded documents storage
+├── query_results/        # Shared query results
+└── README.md            # This file
+```
+
+## 🛠️ Core Components
+
+### DocumentIngestion Class
+- Handles PDF text extraction using PyPDF2
+- Creates document chunks with metadata
+- Builds and persists ChromaDB vector store
+- Manages document preprocessing and storage
+
+### RAGSystem Class
+- Implements retrieval-augmented generation
+- Uses BGE-small-en-v1.5 embeddings for semantic search (with OpenAI fallback)
+- Leverages SEA-LION models for response generation:
+  - **SEA-LION v3.5 Reasoning Model** for complex university queries
+  - **SEA-LION v3 Instruct Model** for translation and simple questions
+- Provides multilingual query support with automatic model selection
+
+### Streamlit UI
+- Clean, intuitive interface
+- Multi-page navigation
+- File upload with progress tracking
+- Advanced search filters
+- Shareable query results
+
+## 🌏 Supported Languages
+
+The system supports queries and responses in:
+- **English** - Primary language
+- **中文 (Chinese)** - For Chinese-speaking students
+- **Bahasa Malaysia** - For Malaysian context
+- **ไทย (Thai)** - For Thai students
+- **Bahasa Indonesia** - For Indonesian students
+- **Tiếng Việt (Vietnamese)** - For Vietnamese students
+- **Filipino** - For Philippines context
+
+## 🎯 Target ASEAN Countries
+
+- 🇸🇬 Singapore
+- 🇲🇾 Malaysia
+- 🇹🇭 Thailand
+- 🇮🇩 Indonesia
+- 🇵🇭 Philippines
+- 🇻🇳 Vietnam
+- 🇧🇳 Brunei
+- 🇰🇭 Cambodia
+- 🇱🇦 Laos
+- 🇲🇲 Myanmar
+
+## 🔧 Configuration
+
+### Environment Variables (.env)
+```bash
+# SEA-LION API Configuration
+SEA_LION_API_KEY=your_sea_lion_api_key_here
+SEA_LION_BASE_URL=https://api.sea-lion.ai/v1
+
+# OpenAI API Configuration (for embeddings)
+OPENAI_API_KEY=your_openai_api_key_here
+
+# Application Configuration
+APP_NAME=PanSea University Search
+APP_VERSION=1.0.0
+CHROMA_PERSIST_DIRECTORY=./chroma_db
+UPLOAD_FOLDER=./documents
+MAX_FILE_SIZE_MB=50
+```
+
+### Customization Options
+- **Chunk Size**: Adjust text splitting in `rag_system.py`
+- **Retrieval Count**: Modify number of retrieved documents (default: 5)
+- **Model Selection**: Configure SEA-LION model selection logic
+- **UI Themes**: Modify CSS in `app.py`
+- **Query Classification**: Adjust complex vs simple query detection
+
+## 📊 Example Queries
+
+Try these sample queries to test the system and see different model usage:
+
+### Complex Queries (Uses SEA-LION Reasoning Model)
+
+1. **Multi-criteria Search**: "Show me universities in Thailand and Malaysia for engineering master's programs with tuition under $15,000 per year"
+
+2. **Chinese Query**: "专科毕业，无雅思，想在马来西亚读硕士，学费不超过4万人民币/年"
+
+3. **Comparative Analysis**: "Compare MBA programs in Singapore and Indonesia with GMAT requirements and scholarship opportunities"
+
+### Simple Queries (Uses SEA-LION Instruct Model)
+
+4. **Translation**: "How do you say 'application deadline' in Thai and Indonesian?"
+
+5. **Definition**: "What is the difference between IELTS and TOEFL?"
+
+6. **Basic Information**: "What does GPA stand for and how is it calculated?"
+
+## 🔍 Technical Stack
+
+- **Backend**: Python 3.11, LangChain
+- **LLM Models**: 
+  - SEA-LION v3.5 8B Reasoning (complex queries)
+  - SEA-LION v3 9B Instruct (simple queries & translation)
+- **Embeddings**: BGE-small-en-v1.5 (with OpenAI ada-002 fallback)
+- **Vector Database**: ChromaDB with persistence
+- **Frontend**: Streamlit with custom CSS
+- **Document Processing**: PyPDF2, PyCryptodome (for encrypted PDFs), RecursiveCharacterTextSplitter
+
+## 📈 Roadmap
+
+- [ ] Support for additional document formats (Word, Excel)
+- [x] Integration with SEA-LION multilingual models
+- [ ] Real-time web scraping of university websites
+- [ ] Mobile-responsive design
+- [ ] User authentication and query history
+- [ ] Advanced analytics and insights
+- [ ] Integration with university application systems
+- [ ] Fine-tuning SEA-LION models on university-specific data
+
+## 🤝 Contributing
+
+1. Fork the repository
+2. Create a feature branch (`git checkout -b feature/amazing-feature`)
+3. Commit your changes (`git commit -m 'Add amazing feature'`)
+4. Push to the branch (`git push origin feature/amazing-feature`)
+5. Open a Pull Request
+
+## 📄 License
+
+This project is licensed under the MIT License - see the LICENSE file for details.
+
+## 💡 Tips for Best Results
+
+1. **Upload Quality Documents**: Use official admission guides and requirements documents
+2. **Be Specific**: Include specific criteria in your queries (budget, location, program type)
+3. **Use Natural Language**: Ask questions as you would to a human counselor
+4. **Try Multiple Languages**: The system works well with mixed-language queries
+5. **Check Sources**: Always review the source documents cited in responses
+
+## 🆘 Troubleshooting
+
+### Common Issues
+
+**"No documents found"**: Upload PDF documents first in the Upload Documents page
+
+**"API Key not found"**: Add your SEA-LION API key to the .env file
+
+**"No embeddings available"**: BGE embeddings are used by default. If issues occur, add your OpenAI API key for fallback embeddings
+
+**"Import errors"**: Install dependencies using `pip install -r requirements.txt`
+
+**"ChromaDB errors"**: Delete the `chroma_db` folder and restart the application
+
+**"PyCryptodome is required for AES algorithm"**: This error occurs with encrypted PDFs. PyCryptodome is now included in requirements.txt
+
+**"Could not extract text from PDF"**: This can happen with:
+- Password-protected PDFs (provide unprotected versions)  
+- Scanned PDFs or image-based documents (consider OCR tools)
+- Heavily encrypted or corrupted PDF files
+
+## 📞 Support
+
+For support, please create an issue on GitHub or contact the development team.
+
+---
+
+**Made with ❤️ for students seeking education opportunities in ASEAN** 🎓