README.md

---
language:
- en
license: apache-2.0
library_name: pytorch
tags:
- text-classification
- fiction-detection
- byte-level
- cnn
datasets:
- HuggingFaceTB/cosmopedia
- BEE-spoke-data/gutenberg-en-v1-clean
- common-pile/arxiv_abstracts
- ccdv/cnn_dailymail
metrics:
- accuracy
- f1
- roc_auc
model-index:
- name: TinyByteCNN-Fiction-Classifier
  results:
  - task:
      type: text-classification
      name: Fiction vs Non-Fiction Classification
    dataset:
      name: Custom Fiction/Non-Fiction Dataset (85k samples)
      type: custom
      split: validation
    metrics:
    - type: accuracy
      value: 99.91
      name: Validation Accuracy
    - type: f1
      value: 99.91
      name: F1 Score
    - type: roc_auc
      value: 99.99
      name: ROC AUC
  - task:
      type: text-classification
      name: Curated Test Samples
    dataset:
      name: 18 Diverse Fiction/Non-Fiction Samples
      type: curated
      split: test
    metrics:
    - type: accuracy
      value: 100.0
      name: Test Accuracy
    - type: confidence_avg
      value: 96.3
      name: Average Confidence
---

# TinyByteCNN Fiction vs Non-Fiction Detector

A lightweight, byte-level CNN model for detecting fiction vs non-fiction text with 99.91% validation accuracy.

## Model Description

TinyByteCNN is a highly efficient byte-level convolutional neural network designed for binary classification of fiction vs non-fiction text. The model operates directly on UTF-8 byte sequences, eliminating the need for tokenization and making it robust to various text formats and languages.

### Architecture Highlights

- **Model Size**: 942,313 parameters (~3.6MB)
- **Input**: Raw UTF-8 bytes (max 4096 bytes ≈ 512 words)
- **Architecture**: Depthwise-separable 1D CNN with Squeeze-Excitation
- **Receptive Field**: ~2.8KB covering multi-paragraph context
- **Key Features**:
  - 4 stages with progressive downsampling (32x reduction)
  - Dilated convolutions for larger receptive field
  - SE attention modules for channel recalibration
  - Global average + max pooling head

## Intended Uses & Limitations

### Intended Uses
- Automated content categorization for libraries and archives
- Fiction/non-fiction filtering for content platforms
- Educational content classification
- Writing style analysis
- Content recommendation systems

### Limitations
- **Personal narratives**: May misclassify personal journal entries and memoirs as fiction (observed ~97% fiction confidence on journal entries)
- **Mixed content**: Struggles with creative non-fiction and narrative journalism
- **Length**: Optimized for 512-4096 byte inputs; longer texts should be chunked
- **Language**: Primarily trained on English text

## Training Data

The model was trained on a diverse dataset of 85,000 samples (60k train, 15k validation, 10k test) drawn from:

### Fiction Sources (50%)
1. **Cosmopedia Stories** (HuggingFaceTB/cosmopedia)
   - Synthetic fiction stories
   - License: Apache 2.0

2. **Project Gutenberg** (BEE-spoke-data/gutenberg-en-v1-clean)
   - Classic literature
   - License: Public Domain

3. **Reddit WritingPrompts**
   - Community-generated creative writing
   - Via synthetic alternatives

### Non-Fiction Sources (50%)
1. **Cosmopedia Educational** (HuggingFaceTB/cosmopedia)
   - Textbooks, WikiHow, educational blogs
   - License: Apache 2.0

2. **Scientific Papers** (common-pile/arxiv_abstracts)
   - Academic abstracts and introductions
   - License: Various (permissive)

3. **News Articles** (ccdv/cnn_dailymail)
   - CNN and Daily Mail articles
   - License: Apache 2.0

## Training Procedure

### Preprocessing
- Unicode NFC normalization
- Whitespace normalization (max 2 consecutive spaces)
- UTF-8 byte encoding
- Padding/truncation to 4096 bytes

### Training Hyperparameters
- **Optimizer**: AdamW (lr=3e-3, betas=(0.9, 0.98), weight_decay=0.01)
- **Schedule**: Cosine decay with 5% warmup
- **Batch Size**: 32
- **Epochs**: 10
- **Label Smoothing**: 0.05
- **Gradient Clipping**: 1.0
- **Device**: Apple M-series (MPS)

## Evaluation Results

### Validation Set (15,000 samples)
| Metric | Value |
|--------|-------|
| Accuracy | 99.91% |
| F1 Score | 0.9991 |
| ROC AUC | 0.9999 |
| Loss | 0.1194 |

### Detailed Test Results on 18 Curated Samples

The model achieved **100% accuracy** across all categories, but shows interesting confidence patterns:

| Category | Sample Title/Type | True Label | Predicted | Confidence | Analysis |
|----------|------------------|------------|-----------|------------|----------|
| **FICTION - General** | | | | | |
| Literary | Lighthouse Keeper Storm | Fiction | Fiction | **79.8%** | ⚠️ **Lowest confidence** - realistic setting |
| Sci-Fi | Time Travel Bedroom | Fiction | Fiction | 97.2% | ✅ Clear fantastical elements |
| Mystery | Detective Rose Case | Fiction | Fiction | 97.3% | ✅ Strong narrative structure |
| **FICTION - Children's** | | | | | |
| Animal Tale | Benny's Carrot Problem | Fiction | Fiction | 97.1% | ✅ Clear storytelling markers |
| Fantasy | Princess Luna's Paintings | Fiction | Fiction | 97.3% | ✅ Magical elements detected |
| Magical | Tommy's Dream Sprites | Fiction | Fiction | **96.0%** | ⚠️ Lower confidence - whimsical tone |
| **FICTION - Fantasy** | | | | | |
| Epic Fantasy | Shadowgate & Void Lords | Fiction | Fiction | 97.4% | ✅ High fantasy vocabulary |
| Magic System | Moonlight Weaver Elara | Fiction | Fiction | 96.8% | ✅ Complex world-building |
| Urban Fantasy | Dragon Memory Markets | Fiction | Fiction | 97.3% | ✅ Supernatural commerce |
| **NON-FICTION - Academic** | | | | | |
| Biology | Photosynthesis Process | Non-Fiction | Non-Fiction | 97.8% | ✅ Technical terminology |
| Mathematics | Calculus Theorem | Non-Fiction | Non-Fiction | 97.8% | ✅ Mathematical concepts |
| Economics | Market Equilibrium | Non-Fiction | Non-Fiction | 97.9% | ✅ Economic theory |
| **NON-FICTION - News** | | | | | |
| Financial | Federal Reserve Decision | Non-Fiction | Non-Fiction | 97.8% | ✅ Factual reporting style |
| Local Gov | Homeless Crisis Plan | Non-Fiction | Non-Fiction | 97.9% | ✅ Policy announcement format |
| Science | Exoplanet Discovery | Non-Fiction | Non-Fiction | 97.9% | ✅ Research reporting |
| **NON-FICTION - Journals** | | | | | |
| Financial | Wall Street Journal Market | Non-Fiction | Non-Fiction | 97.7% | ✅ Professional journalism |
| Scientific | Nature Research Report | Non-Fiction | Non-Fiction | 97.7% | ✅ Academic publication style |
| Personal | Kyoto Travel Log | Non-Fiction | Non-Fiction | **97.5%** | ⚠️ Slightly lower - personal narrative |

### Key Insights:
- **Weakest Performance**: Realistic literary fiction (79.8% confidence) - the lighthouse story lacks obvious fantastical elements
- **Strongest Performance**: Academic/news content (97.8-97.9% confidence) - clear technical/factual language
- **Edge Cases**: Personal narratives and whimsical children's stories show slightly lower confidence
- **Perfect Accuracy**: 18/18 samples correctly classified despite confidence variations

### Detailed Test Results

#### ✅ All 12 Samples Correctly Classified

**Fiction Samples (3/3):**
1. Lighthouse keeper narrative → Fiction (79.8% conf)
2. Time travel story → Fiction (97.2% conf)
3. Detective mystery → Fiction (97.3% conf)

**Textbook Samples (3/3):**
1. Photosynthesis (Biology) → Non-Fiction (97.8% conf)
2. Fundamental theorem (Calculus) → Non-Fiction (97.8% conf)
3. Market equilibrium (Economics) → Non-Fiction (97.9% conf)

**News Articles (3/3):**
1. Federal Reserve decision → Non-Fiction (97.8% conf)
2. City homeless initiative → Non-Fiction (97.9% conf)
3. Exoplanet discovery → Non-Fiction (97.9% conf)

**Journal Articles (3/3):**
1. Wall Street Journal (Financial) → Non-Fiction (97.7% conf)
2. Nature Scientific Reports → Non-Fiction (97.7% conf)
3. Personal Travel Journal → Non-Fiction (97.5% conf)

## How to Use

### PyTorch

```python
import torch
import numpy as np
from model import TinyByteCNN, preprocess_text

# Load model
model = TinyByteCNN.from_pretrained("username/tinybytecnn-fiction-detector")
model.eval()

# Prepare text
text = "Your text here..."
input_bytes = preprocess_text(text)  # Returns tensor of shape [1, 4096]

# Predict
with torch.no_grad():
    logits = model(input_bytes)
    probability = torch.sigmoid(logits).item()
    
    if probability > 0.5:
        print(f"Non-Fiction (confidence: {probability:.1%})")
    else:
        print(f"Fiction (confidence: {1-probability:.1%})")
```

### Batch Processing

```python
def classify_texts(texts, model, batch_size=32):
    results = []
    for i in range(0, len(texts), batch_size):
        batch = texts[i:i+batch_size]
        inputs = torch.stack([preprocess_text(t) for t in batch])
        
        with torch.no_grad():
            logits = model(inputs)
            probs = torch.sigmoid(logits)
            
        for text, prob in zip(batch, probs):
            results.append({
                'text': text[:100] + '...',
                'class': 'Non-Fiction' if prob > 0.5 else 'Fiction',
                'confidence': prob.item() if prob > 0.5 else 1-prob.item()
            })
    
    return results
```

## Training Infrastructure

- **Hardware**: Apple M-series with 8GB MPS memory limit
- **Training Time**: ~20 minutes
- **Framework**: PyTorch 2.0+

## Environmental Impact

- **Hardware Type**: Apple Silicon M-series
- **Hours used**: 0.33
- **Carbon Emitted**: Minimal (ARM-based efficiency, ~10W average)

## Citation

```bibtex
@model{tinybytecnn-fiction-2024,
  title={TinyByteCNN Fiction vs Non-Fiction Detector},
  author={Mitchell Currie},
  year={2024},
  publisher={HuggingFace},
  url={https://huggingface.co/username/tinybytecnn-fiction-detector}
}
```

## Acknowledgments

This model uses data from:
- HuggingFace Team (Cosmopedia dataset)
- Project Gutenberg
- Common Pile contributors
- CNN/Daily Mail dataset creators

## License

Apache 2.0

## Contact

For questions or issues, please open an issue on the [model repository](https://huggingface.co/username/tinybytecnn-fiction-detector).