Spaces:
Sleeping
Sleeping
DVampire
commited on
Commit
·
f7470ea
1
Parent(s):
78f6650
update readme
Browse files
README.md
CHANGED
|
@@ -11,15 +11,29 @@ pinned: false
|
|
| 11 |
|
| 12 |
# Paper Index - AI Paper Evaluation System
|
| 13 |
|
| 14 |
-
A comprehensive system for evaluating AI research papers using advanced language models.
|
| 15 |
|
| 16 |
## Features
|
| 17 |
|
| 18 |
- **Daily Paper Crawling**: Automatically fetches papers from Hugging Face daily
|
| 19 |
- **AI Evaluation**: Uses Claude Sonnet to evaluate papers across multiple dimensions
|
|
|
|
|
|
|
|
|
|
| 20 |
- **Interactive Dashboard**: Beautiful web interface for browsing and evaluating papers
|
| 21 |
-
- **Database
|
| 22 |
- **Smart Navigation**: Intelligent date navigation with fallback mechanisms
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 23 |
|
| 24 |
## Hugging Face Spaces Deployment
|
| 25 |
|
|
@@ -93,7 +107,9 @@ PORT=7860 # Optional, defaults to 7860
|
|
| 93 |
### Evaluation Endpoints
|
| 94 |
|
| 95 |
- `POST /api/papers/evaluate/{arxiv_id}` - Start paper evaluation
|
|
|
|
| 96 |
- `GET /api/papers/evaluate/{arxiv_id}/status` - Get evaluation status
|
|
|
|
| 97 |
|
| 98 |
### Cache Management
|
| 99 |
|
|
@@ -105,19 +121,23 @@ PORT=7860 # Optional, defaults to 7860
|
|
| 105 |
|
| 106 |
### Frontend
|
| 107 |
- **HTML/CSS/JavaScript**: Modern, responsive interface
|
| 108 |
-
- **Real-time Updates**: Dynamic content loading
|
| 109 |
- **Theme Support**: Light/dark mode toggle
|
|
|
|
|
|
|
| 110 |
|
| 111 |
### Backend
|
| 112 |
- **FastAPI**: High-performance web framework
|
| 113 |
-
- **SQLite**:
|
| 114 |
-
- **Async Processing**: Background evaluation tasks
|
|
|
|
| 115 |
- **Caching**: Intelligent caching system for performance
|
| 116 |
|
| 117 |
### AI Integration
|
| 118 |
-
- **
|
| 119 |
- **Multi-dimensional Analysis**: Comprehensive evaluation criteria
|
| 120 |
- **Structured Output**: JSON-based evaluation results
|
|
|
|
| 121 |
|
| 122 |
## Database Schema
|
| 123 |
|
|
@@ -142,6 +162,11 @@ CREATE TABLE papers (
|
|
| 142 |
);
|
| 143 |
```
|
| 144 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 145 |
## Evaluation Dimensions
|
| 146 |
|
| 147 |
The system evaluates papers across 12 key dimensions:
|
|
@@ -159,6 +184,52 @@ The system evaluates papers across 12 key dimensions:
|
|
| 159 |
11. **3-Year Feasibility** - Short-term potential
|
| 160 |
12. **Overall Automatability** - Comprehensive assessment
|
| 161 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 162 |
## Contributing
|
| 163 |
|
| 164 |
1. Fork the repository
|
|
|
|
| 11 |
|
| 12 |
# Paper Index - AI Paper Evaluation System
|
| 13 |
|
| 14 |
+
A comprehensive system for evaluating AI research papers using advanced language models with asynchronous processing and concurrent evaluation capabilities.
|
| 15 |
|
| 16 |
## Features
|
| 17 |
|
| 18 |
- **Daily Paper Crawling**: Automatically fetches papers from Hugging Face daily
|
| 19 |
- **AI Evaluation**: Uses Claude Sonnet to evaluate papers across multiple dimensions
|
| 20 |
+
- **Concurrent Processing**: True asynchronous evaluation with multiple papers processed simultaneously
|
| 21 |
+
- **Re-evaluation**: Ability to re-run evaluations for papers with updated results
|
| 22 |
+
- **Batch Evaluation**: "Evaluate All" feature to process multiple papers at once
|
| 23 |
- **Interactive Dashboard**: Beautiful web interface for browsing and evaluating papers
|
| 24 |
+
- **Asynchronous Database**: High-performance SQLite with WAL mode for concurrent operations
|
| 25 |
- **Smart Navigation**: Intelligent date navigation with fallback mechanisms
|
| 26 |
+
- **Real-time Status Updates**: Live progress tracking and notifications
|
| 27 |
+
|
| 28 |
+
## Recent Updates
|
| 29 |
+
|
| 30 |
+
### v0.1.0 - Asynchronous & Concurrent Features
|
| 31 |
+
- **Asynchronous Database**: Migrated from `sqlite3` to `aiosqlite` for better performance
|
| 32 |
+
- **Concurrent Evaluation**: Multiple papers can be evaluated simultaneously
|
| 33 |
+
- **Re-evaluation**: Added "Re-evaluate" button for papers to update evaluation results
|
| 34 |
+
- **Batch Processing**: "Evaluate All" button to process all un-evaluated papers
|
| 35 |
+
- **Enhanced UI**: Improved progress indicators and real-time notifications
|
| 36 |
+
- **Database Optimization**: WAL mode and performance pragmas for better concurrency
|
| 37 |
|
| 38 |
## Hugging Face Spaces Deployment
|
| 39 |
|
|
|
|
| 107 |
### Evaluation Endpoints
|
| 108 |
|
| 109 |
- `POST /api/papers/evaluate/{arxiv_id}` - Start paper evaluation
|
| 110 |
+
- `POST /api/papers/reevaluate/{arxiv_id}` - Re-evaluate a paper
|
| 111 |
- `GET /api/papers/evaluate/{arxiv_id}/status` - Get evaluation status
|
| 112 |
+
- `GET /api/papers/evaluate/active-tasks` - Get currently running evaluations
|
| 113 |
|
| 114 |
### Cache Management
|
| 115 |
|
|
|
|
| 121 |
|
| 122 |
### Frontend
|
| 123 |
- **HTML/CSS/JavaScript**: Modern, responsive interface
|
| 124 |
+
- **Real-time Updates**: Dynamic content loading with polling
|
| 125 |
- **Theme Support**: Light/dark mode toggle
|
| 126 |
+
- **Progress Indicators**: Visual feedback for evaluation status
|
| 127 |
+
- **Batch Operations**: "Evaluate All" functionality with sequential processing
|
| 128 |
|
| 129 |
### Backend
|
| 130 |
- **FastAPI**: High-performance web framework
|
| 131 |
+
- **Async SQLite**: `aiosqlite` with WAL mode for concurrent operations
|
| 132 |
+
- **Async Processing**: Background evaluation tasks with task tracking
|
| 133 |
+
- **Concurrent Evaluation**: Multiple papers evaluated simultaneously
|
| 134 |
- **Caching**: Intelligent caching system for performance
|
| 135 |
|
| 136 |
### AI Integration
|
| 137 |
+
- **Async Anthropic**: Non-blocking API calls with `AsyncAnthropic`
|
| 138 |
- **Multi-dimensional Analysis**: Comprehensive evaluation criteria
|
| 139 |
- **Structured Output**: JSON-based evaluation results
|
| 140 |
+
- **Error Handling**: Robust error handling and retry mechanisms
|
| 141 |
|
| 142 |
## Database Schema
|
| 143 |
|
|
|
|
| 162 |
);
|
| 163 |
```
|
| 164 |
|
| 165 |
+
### Database Optimizations
|
| 166 |
+
- **WAL Mode**: `PRAGMA journal_mode=WAL` for better concurrency
|
| 167 |
+
- **Performance Pragmas**: Optimized settings for concurrent access
|
| 168 |
+
- **Asynchronous Operations**: All database calls are async/await
|
| 169 |
+
|
| 170 |
## Evaluation Dimensions
|
| 171 |
|
| 172 |
The system evaluates papers across 12 key dimensions:
|
|
|
|
| 184 |
11. **3-Year Feasibility** - Short-term potential
|
| 185 |
12. **Overall Automatability** - Comprehensive assessment
|
| 186 |
|
| 187 |
+
## Key Features
|
| 188 |
+
|
| 189 |
+
### Concurrent Evaluation
|
| 190 |
+
- Multiple papers can be evaluated simultaneously
|
| 191 |
+
- Global task tracking prevents duplicate evaluations
|
| 192 |
+
- Real-time status updates via polling
|
| 193 |
+
- Automatic error handling and recovery
|
| 194 |
+
|
| 195 |
+
### Re-evaluation System
|
| 196 |
+
- "Re-evaluate" button appears after initial evaluation
|
| 197 |
+
- Updates existing evaluation results in database
|
| 198 |
+
- Maintains evaluation history and timestamps
|
| 199 |
+
- Same comprehensive evaluation criteria
|
| 200 |
+
|
| 201 |
+
### Batch Processing
|
| 202 |
+
- "Evaluate All" button processes all un-evaluated papers
|
| 203 |
+
- Sequential processing with delays to prevent API overload
|
| 204 |
+
- Progress tracking and real-time notifications
|
| 205 |
+
- Automatic button state management
|
| 206 |
+
|
| 207 |
+
### Enhanced UI/UX
|
| 208 |
+
- Progress circles with proper layering
|
| 209 |
+
- Bottom-right notification system
|
| 210 |
+
- Dynamic button states and text updates
|
| 211 |
+
- Responsive design with modern styling
|
| 212 |
+
|
| 213 |
+
## Performance Optimizations
|
| 214 |
+
|
| 215 |
+
### Database
|
| 216 |
+
- Asynchronous operations with `aiosqlite`
|
| 217 |
+
- WAL mode for better concurrency
|
| 218 |
+
- Optimized SQLite pragmas
|
| 219 |
+
- Connection pooling and management
|
| 220 |
+
|
| 221 |
+
### API Calls
|
| 222 |
+
- Non-blocking Anthropic API calls
|
| 223 |
+
- Concurrent evaluation processing
|
| 224 |
+
- Task tracking and management
|
| 225 |
+
- Error handling and retry logic
|
| 226 |
+
|
| 227 |
+
### Frontend
|
| 228 |
+
- Efficient DOM manipulation
|
| 229 |
+
- Polling with appropriate intervals
|
| 230 |
+
- Memory management for log entries
|
| 231 |
+
- Optimized event handling
|
| 232 |
+
|
| 233 |
## Contributing
|
| 234 |
|
| 235 |
1. Fork the repository
|