Upload 528 files

.cursorrules

@@ -0,0 +1 @@
+# Project Overview This project, named GPT-Researcher, LLM based autonomous agent that conducts local and web research on any topic and generates a comprehensive report with citations, is built using Next.js and TypeScript. It integrates various libraries for their strenghts. Your primary goal is to help with Next.js app router patterns, TypeScript type safety, Tailwind CSS best practices, code quality standards, and Python/FastAPI backend optimizations. # Key URLs - Project Home Page: https://gptr.dev/ - GitHub Repository: https://github.com/assafelovic/gpt-researcher - Documentation: https://docs.gptr.dev/ # Project Structure - Frontend user interface built with Next.js, TypeScript, and Tailwind CSS in `/frontend` - Static FastAPI version for lightweight deployments - Next.js version for production use with enhanced features - Multi-agent research system using LangChain and LangGraph in `/backend/multi_agents` - Browser, Editor, Researcher, Reviewer, Revisor, Writer, and Publisher agents - Task configuration and agent coordination - Document processing using Unstructured and PyMuPDF in `/backend/document_processing` - PDF, DOCX, and web content parsing - Text extraction and preprocessing - Report generation using LangChain and Jinja2 templates in `/backend/report_generation` - Template-based report structuring - Dynamic content formatting - Multiple output formats in `/backend/output_formats` - PDF via md2pdf - Markdown via mistune - DOCX via python-docx - Format conversion utilities - Export functionality - GPT Researcher core functionality in `/gpt_researcher` - Web scraping and content aggregation - Research planning and execution - Source validation and tracking - Query processing and response generation - Testing infrastructure in `/tests` - Unit tests for individual components - Integration tests for agent interactions - End-to-end research workflow tests - Mock data and fixtures for testing # Language Model Configuration - Default model: gpt-4-turbo - Alternative models: gpt-3.5-turbo, claude-3-opus - Temperature settings for different tasks - Context window management - Token limit handling - Cost optimization strategies # Error Handling - Research failure recovery - API rate limiting - Network timeout handling - Invalid input management - Source validation errors - Report generation failures # Performance - Parallel processing strategies - Caching mechanisms - Memory management - Response streaming - Resource allocation - Query optimization # Development Workflow - Branch naming conventions - Commit message format - PR review process - Testing requirements - Documentation updates - Version control guidelines # API Documentation - REST endpoints - WebSocket events - Request/Response formats - Authentication methods - Rate limits - Error codes # Monitoring - Performance metrics - Error tracking - Usage statistics - Cost monitoring - Research quality metrics - User feedback tracking # Frontend Components - Static FastAPI version for lightweight deployments - Next.js version for production use with enhanced features # Backend Components - Multi-agent system architecture - Document processing pipeline - Report generation system - Output format handlers # Core Research Components - Web scraping and aggregation - Research planning and execution - Source validation - Query processing # Testing - Unit tests - Integration tests - End-to-end tests - Performance testing # Rule Violation Monitoring - Alert developer when changes conflict with project structure - Warn about deviations from coding standards - Flag unauthorized framework or library additions - Monitor for security and performance anti-patterns - Track API usage patterns that may violate guidelines - Report TypeScript strict mode violations - Identify accessibility compliance issues # Development Guidelines - Use TypeScript with strict mode enabled - Follow ESLint and Prettier configurations - Ensure components are responsive and accessible - Use Tailwind CSS for styling, following the project's design system - Minimize AI-generated comments, prefer self-documenting code - Follow React best practices and hooks guidelines - Validate all user inputs and API responses - Use existing components as reference implementations # Important Scripts - `npm run dev`: Start development server - `npm run build`: Build for production - `npm run test`: Run test suite - `python -m pytest`: Run Python tests - `docker-compose up`: Start all services - `docker-compose run gpt-researcher-tests`: Run test suite in container - `python -m uvicorn backend.server.server:app --host=0.0.0.0 --port=8000`: Start FastAPI server - `python -m uvicorn backend.server.server:app --reload`: Start FastAPI server with auto-reload for development - `python main.py`: Run the main application directly # AI Integration Guidelines - Prioritize type safety in all AI interactions - Follow LangChain and LangGraph best practices - Implement proper error handling for AI responses - Maintain context window limits - Handle rate limiting and API quotas - Validate AI outputs before processing - Log AI interactions for debugging # Lexicon - **GPT Researcher**: Autonomous research agent system - **Multi-Agent System**: Coordinated AI agents for research tasks - **Research Pipeline**: End-to-end research workflow - **Agent Roles**: Browser, Editor, Researcher, Reviewer, Revisor, Writer, Publisher - **Source Validation**: Verification of research sources - **Report Generation**: Process of creating final research output # Additional Resources - [Next.js Documentation](https://nextjs.org/docs) - [TypeScript Handbook](https://www.typescriptlang.org/docs/) - [Tailwind CSS Documentation](https://tailwindcss.com/docs) - [LangChain Documentation](https://python.langchain.com/docs/) - [FastAPI Documentation](https://fastapi.tiangolo.com/) - [Project Documentation](https://docs.gptr.dev/) End all your comments with a :-) symbol.

.dockerignore

@@ -0,0 +1,2 @@
+.git
+output/

.env

@@ -0,0 +1,8 @@
+GOOGLE_API_KEY=AIzaSyCISHY92IzU60M8Jf0qCWIRCyhGUAj_haU
+FAST_LLM="google_genai:gemini-1.5-flash"
+SMART_LLM="google_genai:gemini-1.5-pro"
+STRATEGIC_LLM="google_genai:gemini-1.5-pro"
+
+EMBEDDING="google_genai:models/text-embedding-004"
+
+TAVILY_API_KEY=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

.gitattributes

@@ -33,3 +33,24 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+docs/blog/2023-09-22-gpt-researcher/architecture.png filter=lfs diff=lfs merge=lfs -text
+docs/blog/2023-09-22-gpt-researcher/planner.jpeg filter=lfs diff=lfs merge=lfs -text
+docs/blog/2024-05-19-gptr-langgraph/blog-langgraph.jpeg filter=lfs diff=lfs merge=lfs -text
+docs/blog/2024-09-7-hybrid-research/gptr-hybrid.png filter=lfs diff=lfs merge=lfs -text
+docs/docs/gpt-researcher/context/gptr-hybrid.png filter=lfs diff=lfs merge=lfs -text
+docs/static/img/architecture.png filter=lfs diff=lfs merge=lfs -text
+docs/static/img/leaderboard.png filter=lfs diff=lfs merge=lfs -text
+frontend/nextjs/public/img/agents/academicResearchAgentAvatar.png filter=lfs diff=lfs merge=lfs -text
+frontend/nextjs/public/img/agents/businessAnalystAgentAvatar.png filter=lfs diff=lfs merge=lfs -text
+frontend/nextjs/public/img/agents/computerSecurityanalystAvatar.png filter=lfs diff=lfs merge=lfs -text
+frontend/nextjs/public/img/agents/financeAgentAvatar.png filter=lfs diff=lfs merge=lfs -text
+frontend/nextjs/public/img/agents/mathAgentAvatar.png filter=lfs diff=lfs merge=lfs -text
+frontend/nextjs/public/img/agents/travelAgentAvatar.png filter=lfs diff=lfs merge=lfs -text
+frontend/nextjs/public/img/gptr-logo.png filter=lfs diff=lfs merge=lfs -text
+frontend/static/academicResearchAgentAvatar.png filter=lfs diff=lfs merge=lfs -text
+frontend/static/businessAnalystAgentAvatar.png filter=lfs diff=lfs merge=lfs -text
+frontend/static/computerSecurityanalystAvatar.png filter=lfs diff=lfs merge=lfs -text
+frontend/static/financeAgentAvatar.png filter=lfs diff=lfs merge=lfs -text
+frontend/static/mathAgentAvatar.png filter=lfs diff=lfs merge=lfs -text
+frontend/static/travelAgentAvatar.png filter=lfs diff=lfs merge=lfs -text
+tests/docs/doc.pdf filter=lfs diff=lfs merge=lfs -text

.github/ISSUE_TEMPLATE/bug_report.md

@@ -0,0 +1,38 @@
+---
+name: Bug report
+about: Create a report to help us improve
+title: ''
+labels: ''
+assignees: ''
+
+---
+
+**Describe the bug**
+A clear and concise description of what the bug is.
+
+**To Reproduce**
+Steps to reproduce the behavior:
+1. Go to '...'
+2. Click on '....'
+3. Scroll down to '....'
+4. See error
+
+**Expected behavior**
+A clear and concise description of what you expected to happen.
+
+**Screenshots**
+If applicable, add screenshots to help explain your problem.
+
+**Desktop (please complete the following information):**
+ - OS: [e.g. iOS]
+ - Browser [e.g. chrome, safari]
+ - Version [e.g. 22]
+
+**Smartphone (please complete the following information):**
+ - Device: [e.g. iPhone6]
+ - OS: [e.g. iOS8.1]
+ - Browser [e.g. stock browser, safari]
+ - Version [e.g. 22]
+
+**Additional context**
+Add any other context about the problem here.

.github/ISSUE_TEMPLATE/feature_request.md

@@ -0,0 +1,20 @@
+---
+name: Feature request
+about: Suggest an idea for this project
+title: ''
+labels: ''
+assignees: ''
+
+---
+
+**Is your feature request related to a problem? Please describe.**
+A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]
+
+**Describe the solution you'd like**
+A clear and concise description of what you want to happen.
+
+**Describe alternatives you've considered**
+A clear and concise description of any alternative solutions or features you've considered.
+
+**Additional context**
+Add any other context or screenshots about the feature request here.

.github/dependabot.yml

@@ -0,0 +1,15 @@
+# To get started with Dependabot version updates, you'll need to specify which
+# package ecosystems to update and where the package manifests are located.
+# Please see the documentation for all configuration options:
+# https://docs.github.com/github/administering-a-repository/configuration-options-for-dependency-updates
+
+version: 2
+updates:
+  - package-ecosystem: "pip" # See documentation for possible values
+    directory: "/" # Location of package manifests
+    schedule:
+      interval: "weekly"
+  - package-ecosystem: "docker"
+    directory: "/"
+    schedule:
+      interval: "weekly"

.github/workflows/docker-build.yml

@@ -0,0 +1,45 @@
+name: GPTR tests
+run-name: ${{ github.actor }} ran the GPTR tests flow
+permissions:
+  contents: read
+  pull-requests: write
+on:
+  workflow_dispatch:  # Add this line to enable manual triggering
+  # pull_request:
+  #   types: [opened, synchronize]
+
+jobs:
+  docker:
+    runs-on: ubuntu-latest
+    environment: tests  # Specify the environment to use for this job
+    env:
+      # Ensure these environment variables are set for the entire job
+      OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
+      TAVILY_API_KEY: ${{ secrets.TAVILY_API_KEY }}
+      LANGCHAIN_API_KEY: ${{ secrets.LANGCHAIN_API_KEY }}
+    steps:
+      - name: Git checkout
+        uses: actions/checkout@v3
+
+      - name: Set up QEMU
+        uses: docker/setup-qemu-action@v2
+
+      - name: Set up Docker Buildx
+        uses: docker/setup-buildx-action@v2
+        with:
+          driver: docker
+
+      # - name: Build Docker images
+      #   uses: docker/build-push-action@v4
+      #   with:
+      #     push: false
+      #     tags: gptresearcher/gpt-researcher:latest
+      #     file: Dockerfile          
+
+      - name: Set up Docker Compose
+        run: |
+          sudo curl -L "https://github.com/docker/compose/releases/download/1.29.2/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose
+          sudo chmod +x /usr/local/bin/docker-compose
+      - name: Run tests with Docker Compose
+        run: |
+          docker-compose --profile test run --rm gpt-researcher-tests

.gitignore

@@ -0,0 +1,53 @@
+#Ignore env containing secrets
+.env
+.venv
+.envrc
+
+#Ignore Virtual Env
+env/
+venv/
+.venv/
+
+# Other Environments
+ENV/
+env.bak/
+venv.bak/
+
+#Ignore generated outputs
+outputs/
+*.lock
+dist/
+gpt_researcher.egg-info/
+
+#Ignore my local docs
+my-docs/
+
+#Ignore pycache
+**/__pycache__/
+
+#Ignore mypy cache
+.mypy_cache/
+node_modules
+.idea
+.DS_Store
+.docusaurus
+build
+docs/build
+
+.vscode/launch.json
+.langgraph-data/
+.next/
+package-lock.json
+
+#Vim swp files
+*.swp
+
+# Log files
+logs/
+*.orig
+*.log
+server_log.txt
+
+#Cursor Rules
+.cursorrules
+CURSOR_RULES.md

CODE_OF_CONDUCT.md

@@ -0,0 +1,123 @@
+# Contributor Covenant Code of Conduct
+
+## Our Pledge
+
+We, as members, contributors, and leaders, pledge to make participation in our
+community a harassment-free experience for everyone, regardless of age, body
+size, visible or invisible disability, ethnicity, sex characteristics, gender
+identity and expression, level of experience, education, socio-economic status,
+nationality, personal appearance, race, religion, sexual identity, or
+orientation.
+
+We commit to acting and interacting in ways that contribute to an open, welcoming,
+diverse, inclusive, and healthy community.
+
+## Our Standards
+
+Examples of behavior that contributes to a positive environment for our
+community include:
+
+- Demonstrating empathy and kindness toward others
+- Being respectful of differing opinions, viewpoints, and experiences
+- Giving and gracefully accepting constructive feedback
+- Accepting responsibility and apologizing to those affected by our mistakes, and learning from the experience
+- Focusing on what is best not just for us as individuals, but for the
+  overall community
+
+Examples of unacceptable behavior include:
+
+- The use of sexualized language or imagery, and sexual attention or
+  advances of any kind
+- Trolling, insulting or derogatory comments, and personal or political attacks
+- Public or private harassment
+- Publishing others' private information, such as a physical or email address, without their explicit permission
+- Other conduct that could reasonably be considered inappropriate in a professional setting
+
+## Enforcement Responsibilities
+
+Community leaders are responsible for clarifying and enforcing our standards of
+acceptable behavior and will take appropriate and fair corrective action in
+response to any behavior deemed inappropriate, threatening, offensive,
+or harmful.
+
+Community leaders have the right and responsibility to remove, edit, or reject
+comments, commits, code, wiki edits, issues, and other contributions that do not
+align with this Code of Conduct, and will communicate reasons for moderation
+decisions when appropriate.
+
+## Scope
+
+This Code of Conduct applies to all community spaces and also applies when
+an individual is officially representing the community in public spaces.
+Examples include using an official email address, posting via an official
+social media account, or acting as an appointed representative at an online or offline event.
+
+## Enforcement
+
+Instances of abusive, harassing, or otherwise unacceptable behavior may be
+reported to the community leaders responsible for enforcement at
+[[email protected]](mailto:[email protected]).
+All complaints will be reviewed and investigated promptly and fairly.
+
+All community leaders are obligated to respect the privacy and security of the
+reporter of any incident.
+
+## Enforcement Guidelines
+
+Community leaders will follow these Community Impact Guidelines in determining
+the consequences for any action they deem in violation of this Code of Conduct:
+
+### 1. Correction
+
+**Community Impact**: Use of inappropriate language or other behavior deemed
+unprofessional or unwelcome in the community.
+
+**Consequence**: A private, written warning from community leaders, providing
+clarity around the nature of the violation and an explanation of why the
+behavior was inappropriate. A public apology may be requested.
+
+### 2. Warning
+
+**Community Impact**: A violation through a single incident or series
+of actions.
+
+**Consequence**: A warning with consequences for continued behavior. No
+interaction with the people involved, including unsolicited interaction with
+those enforcing the Code of Conduct, for a specified period. This includes
+avoiding interactions in community spaces and external channels like social media.
+Violating these terms may lead to a temporary or permanent ban.
+
+### 3. Temporary Ban
+
+**Community Impact**: A serious violation of community standards, including
+sustained inappropriate behavior.
+
+**Consequence**: A temporary ban from any interaction or public
+communication with the community for a specified period. No public or
+private interaction with the people involved, including unsolicited interaction
+with those enforcing the Code of Conduct, is allowed during this period.
+Violating these terms may lead to a permanent ban.
+
+### 4. Permanent Ban
+
+**Community Impact**: Demonstrating a pattern of violation of community
+standards, including sustained inappropriate behavior, harassment of an
+individual, or aggression toward or disparagement of groups of individuals.
+
+**Consequence**: A permanent ban from any public interaction within
+the community.
+
+## Attribution
+
+This Code of Conduct is adapted from the [Contributor Covenant][homepage],
+version 2.0, available at
+https://www.contributor-covenant.org/version/2/0/code_of_conduct.html.
+
+Community Impact Guidelines were inspired by [Mozilla's code of conduct
+enforcement ladder](https://github.com/mozilla/diversity).
+
+[homepage]: https://www.contributor-covenant.org
+
+For answers to common questions about this code of conduct, see the FAQ at
+https://www.contributor-covenant.org/faq. Translations are available at
+https://www.contributor-covenant.org/translations.

CONTRIBUTING.md

@@ -0,0 +1,42 @@
+# Contributing to GPT Researcher
+
+First off, we'd like to welcome you and thank you for your interest and effort in contributing to our open-source project ❤️. Contributions of all forms are welcome—from new features and bug fixes to documentation and more.
+
+We are on a mission to build the #1 AI agent for comprehensive, unbiased, and factual research online, and we need your support to achieve this grand vision.
+
+Please take a moment to review this document to make the contribution process easy and effective for everyone involved.
+
+## Reporting Issues
+
+If you come across any issue or have an idea for an improvement, don't hesitate to create an issue on GitHub. Describe your problem in sufficient detail, providing as much relevant information as possible. This way, we can reproduce the issue before attempting to fix it or respond appropriately.
+
+## Contributing Code
+
+1. **Fork the repository and create your branch from `master`.**  
+   If it’s not an urgent bug fix, branch from `master` and work on the feature or fix there.
+
+2. **Make your changes.**  
+   Implement your changes following best practices for coding in the project's language.
+
+3. **Test your changes.**  
+   Ensure that your changes pass all tests if any exist. If the project doesn’t have automated tests, test your changes manually to confirm they behave as expected.
+
+4. **Follow the coding style.**  
+   Ensure your code adheres to the coding conventions used throughout the project, including indentation, accurate comments, etc.
+
+5. **Commit your changes.**  
+   Make your Git commits informative and concise. This is very helpful for others when they look at the Git log.
+
+6. **Push to your fork and submit a pull request.**  
+   When your work is ready and passes tests, push your branch to your fork of the repository and submit a pull request from there.
+
+7. **Pat yourself on the back and wait for review.**  
+   Your work is done, congratulations! Now sit tight. The project maintainers will review your submission as soon as possible. They might suggest changes or ask for improvements. Both constructive conversation and patience are key to the collaboration process.
+
+## Documentation
+
+If you would like to contribute to the project's documentation, please follow the same steps: fork the repository, make your changes, test them, and submit a pull request.
+
+Documentation is a vital part of any software. It's not just about having good code; ensuring that users and contributors understand what's going on, how to use the software, or how to contribute is crucial.
+
+We're grateful for all our contributors, and we look forward to building the world's leading AI research agent hand-in-hand with you. Let's harness the power of open source and AI to change the world together!

CURSOR_RULES.md

@@ -0,0 +1,181 @@
+> **Note**: This is a readable copy of the `.cursorrules` file maintained for legibility. The actual rules are implemented from the `.cursorrules` file in the root directory.
+
+# GPT-Researcher Cursor Rules
+
+## Project Overview
+This project, named GPT-Researcher, is an LLM-based autonomous agent that conducts local and web research on any topic and generates a comprehensive report with citations. It is built using Next.js and TypeScript, integrating various libraries for their strengths.
+
+Your primary goal is to help with:
+- Next.js app router patterns
+- TypeScript type safety
+- Tailwind CSS best practices
+- Code quality standards
+- Python/FastAPI backend optimizations
+
+## Key URLs
+- Project Home Page: https://gptr.dev/
+- GitHub Repository: https://github.com/assafelovic/gpt-researcher
+- Documentation: https://docs.gptr.dev/
+
+## Project Structure
+- Frontend user interface built with Next.js, TypeScript, and Tailwind CSS in `/frontend`
+  - Static FastAPI version for lightweight deployments
+  - Next.js version for production use with enhanced features
+
+- Multi-agent research system using LangChain and LangGraph in `/backend/multi_agents`
+  - Browser, Editor, Researcher, Reviewer, Revisor, Writer, and Publisher agents
+  - Task configuration and agent coordination
+
+- Document processing using Unstructured and PyMuPDF in `/backend/document_processing`
+  - PDF, DOCX, and web content parsing
+  - Text extraction and preprocessing
+
+- Report generation using LangChain and Jinja2 templates in `/backend/report_generation`
+  - Template-based report structuring
+  - Dynamic content formatting
+
+- Multiple output formats in `/backend/output_formats`
+  - PDF via md2pdf
+  - Markdown via mistune
+  - DOCX via python-docx
+  - Format conversion utilities
+  - Export functionality
+
+- GPT Researcher core functionality in `/gpt_researcher`
+  - Web scraping and content aggregation
+  - Research planning and execution
+  - Source validation and tracking
+  - Query processing and response generation
+
+- Testing infrastructure in `/tests`
+  - Unit tests for individual components
+  - Integration tests for agent interactions
+  - End-to-end research workflow tests
+  - Mock data and fixtures for testing
+
+## Language Model Configuration
+- Default model: gpt-4-turbo
+- Alternative models: gpt-3.5-turbo, claude-3-opus
+- Temperature settings for different tasks
+- Context window management
+- Token limit handling
+- Cost optimization strategies
+
+## Error Handling
+- Research failure recovery
+- API rate limiting
+- Network timeout handling
+- Invalid input management
+- Source validation errors
+- Report generation failures
+
+## Performance
+- Parallel processing strategies
+- Caching mechanisms
+- Memory management
+- Response streaming
+- Resource allocation
+- Query optimization
+
+## Development Workflow
+- Branch naming conventions
+- Commit message format
+- PR review process
+- Testing requirements
+- Documentation updates
+- Version control guidelines
+
+## API Documentation
+- REST endpoints
+- WebSocket events
+- Request/Response formats
+- Authentication methods
+- Rate limits
+- Error codes
+
+## Monitoring
+- Performance metrics
+- Error tracking
+- Usage statistics
+- Cost monitoring
+- Research quality metrics
+- User feedback tracking
+
+## Frontend Components
+- Static FastAPI version for lightweight deployments
+- Next.js version for production use with enhanced features
+
+## Backend Components
+- Multi-agent system architecture
+- Document processing pipeline
+- Report generation system
+- Output format handlers
+
+## Core Research Components
+- Web scraping and aggregation
+- Research planning and execution
+- Source validation
+- Query processing
+
+## Testing
+- Unit tests
+- Integration tests
+- End-to-end tests
+- Performance testing
+
+## Rule Violation Monitoring
+- Alert developer when changes conflict with project structure
+- Warn about deviations from coding standards
+- Flag unauthorized framework or library additions
+- Monitor for security and performance anti-patterns
+- Track API usage patterns that may violate guidelines
+- Report TypeScript strict mode violations
+- Identify accessibility compliance issues
+
+## Development Guidelines
+- Use TypeScript with strict mode enabled
+- Follow ESLint and Prettier configurations
+- Ensure components are responsive and accessible
+- Use Tailwind CSS for styling, following the project's design system
+- Minimize AI-generated comments, prefer self-documenting code
+- Follow React best practices and hooks guidelines
+- Validate all user inputs and API responses
+- Use existing components as reference implementations
+
+## Important Scripts
+- `npm run dev`: Start development server
+- `npm run build`: Build for production
+- `npm run test`: Run test suite
+- `python -m pytest`: Run Python tests
+- `python -m uvicorn backend.server.server:app --host=0.0.0.0 --port=8000`: Start FastAPI server
+- `python -m uvicorn backend.server.server:app --reload`: Start FastAPI server with auto-reload for development
+- `python main.py`: Run the main application directly
+- `docker-compose up`: Start all services
+- `docker-compose run gpt-researcher-tests`: Run test suite in container
+
+## AI Integration Guidelines
+- Prioritize type safety in all AI interactions
+- Follow LangChain and LangGraph best practices
+- Implement proper error handling for AI responses
+- Maintain context window limits
+- Handle rate limiting and API quotas
+- Validate AI outputs before processing
+- Log AI interactions for debugging
+
+## Lexicon
+- **GPT Researcher**: Autonomous research agent system
+- **Multi-Agent System**: Coordinated AI agents for research tasks
+- **Research Pipeline**: End-to-end research workflow
+- **Agent Roles**: Browser, Editor, Researcher, Reviewer, Revisor, Writer, Publisher
+- **Source Validation**: Verification of research sources
+- **Report Generation**: Process of creating final research output
+
+## Additional Resources
+- [Next.js Documentation](https://nextjs.org/docs)
+- [TypeScript Handbook](https://www.typescriptlang.org/docs/)
+- [Tailwind CSS Documentation](https://tailwindcss.com/docs)
+- [LangChain Documentation](https://python.langchain.com/docs/)
+- [FastAPI Documentation](https://fastapi.tiangolo.com/)
+- [Project Documentation](https://docs.gptr.dev/)
+
+_Note: End all your comments with a :-) symbol._ 

Dockerfile

@@ -0,0 +1,46 @@
+# Stage 1: Browser and build tools installation
+FROM python:3.11.4-slim-bullseye AS install-browser
+
+# Install Chromium, Chromedriver, Firefox, Geckodriver, and build tools in one layer
+RUN apt-get update && \
+    apt-get satisfy -y "chromium, chromium-driver (>= 115.0)" && \
+    apt-get install -y --no-install-recommends firefox-esr wget build-essential && \
+    wget https://github.com/mozilla/geckodriver/releases/download/v0.33.0/geckodriver-v0.33.0-linux64.tar.gz && \
+    tar -xvzf geckodriver-v0.33.0-linux64.tar.gz && \
+    chmod +x geckodriver && \
+    mv geckodriver /usr/local/bin/ && \
+    rm geckodriver-v0.33.0-linux64.tar.gz && \
+    chromium --version && chromedriver --version && \
+    rm -rf /var/lib/apt/lists/*  # Clean up apt lists to reduce image size
+
+# Stage 2: Python dependencies installation
+FROM install-browser AS gpt-researcher-install
+
+ENV PIP_ROOT_USER_ACTION=ignore
+WORKDIR /usr/src/app
+
+# Copy and install Python dependencies in a single layer to optimize cache usage
+COPY ./requirements.txt ./requirements.txt
+COPY ./multi_agents/requirements.txt ./multi_agents/requirements.txt
+
+RUN pip install --no-cache-dir -r requirements.txt && \
+    pip install --no-cache-dir -r multi_agents/requirements.txt
+
+# Stage 3: Final stage with non-root user and app
+FROM gpt-researcher-install AS gpt-researcher
+
+# Create a non-root user for security
+RUN useradd -ms /bin/bash gpt-researcher && \
+    chown -R gpt-researcher:gpt-researcher /usr/src/app
+
+USER gpt-researcher
+WORKDIR /usr/src/app
+
+# Copy the rest of the application files with proper ownership
+COPY --chown=gpt-researcher:gpt-researcher ./ ./
+
+# Expose the application's port
+EXPOSE 8000
+
+# Define the default command to run the application
+CMD ["uvicorn", "main:app", "--host", "0.0.0.0", "--port", "8000"]

LICENSE

@@ -0,0 +1,201 @@
+                                 Apache License
+                           Version 2.0, January 2004
+                        http://www.apache.org/licenses/
+
+   TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
+
+   1. Definitions.
+
+      "License" shall mean the terms and conditions for use, reproduction,
+      and distribution as defined by Sections 1 through 9 of this document.
+
+      "Licensor" shall mean the copyright owner or entity authorized by
+      the copyright owner that is granting the License.
+
+      "Legal Entity" shall mean the union of the acting entity and all
+      other entities that control, are controlled by, or are under common
+      control with that entity. For the purposes of this definition,
+      "control" means (i) the power, direct or indirect, to cause the
+      direction or management of such entity, whether by contract or
+      otherwise, or (ii) ownership of fifty percent (50%) or more of the
+      outstanding shares, or (iii) beneficial ownership of such entity.
+
+      "You" (or "Your") shall mean an individual or Legal Entity
+      exercising permissions granted by this License.
+
+      "Source" form shall mean the preferred form for making modifications,
+      including but not limited to software source code, documentation
+      source, and configuration files.
+
+      "Object" form shall mean any form resulting from mechanical
+      transformation or translation of a Source form, including but
+      not limited to compiled object code, generated documentation,
+      and conversions to other media types.
+
+      "Work" shall mean the work of authorship, whether in Source or
+      Object form, made available under the License, as indicated by a
+      copyright notice that is included in or attached to the work
+      (an example is provided in the Appendix below).
+
+      "Derivative Works" shall mean any work, whether in Source or Object
+      form, that is based on (or derived from) the Work and for which the
+      editorial revisions, annotations, elaborations, or other modifications
+      represent, as a whole, an original work of authorship. For the purposes
+      of this License, Derivative Works shall not include works that remain
+      separable from, or merely link (or bind by name) to the interfaces of,
+      the Work and Derivative Works thereof.
+
+      "Contribution" shall mean any work of authorship, including
+      the original version of the Work and any modifications or additions
+      to that Work or Derivative Works thereof, that is intentionally
+      submitted to Licensor for inclusion in the Work by the copyright owner
+      or by an individual or Legal Entity authorized to submit on behalf of
+      the copyright owner. For the purposes of this definition, "submitted"
+      means any form of electronic, verbal, or written communication sent
+      to the Licensor or its representatives, including but not limited to
+      communication on electronic mailing lists, source code control systems,
+      and issue tracking systems that are managed by, or on behalf of, the
+      Licensor for the purpose of discussing and improving the Work, but
+      excluding communication that is conspicuously marked or otherwise
+      designated in writing by the copyright owner as "Not a Contribution."
+
+      "Contributor" shall mean Licensor and any individual or Legal Entity
+      on behalf of whom a Contribution has been received by Licensor and
+      subsequently incorporated within the Work.
+
+   2. Grant of Copyright License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      copyright license to reproduce, prepare Derivative Works of,
+      publicly display, publicly perform, sublicense, and distribute the
+      Work and such Derivative Works in Source or Object form.
+
+   3. Grant of Patent License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      (except as stated in this section) patent license to make, have made,
+      use, offer to sell, sell, import, and otherwise transfer the Work,
+      where such license applies only to those patent claims licensable
+      by such Contributor that are necessarily infringed by their
+      Contribution(s) alone or by combination of their Contribution(s)
+      with the Work to which such Contribution(s) was submitted. If You
+      institute patent litigation against any entity (including a
+      cross-claim or counterclaim in a lawsuit) alleging that the Work
+      or a Contribution incorporated within the Work constitutes direct
+      or contributory patent infringement, then any patent licenses
+      granted to You under this License for that Work shall terminate
+      as of the date such litigation is filed.
+
+   4. Redistribution. You may reproduce and distribute copies of the
+      Work or Derivative Works thereof in any medium, with or without
+      modifications, and in Source or Object form, provided that You
+      meet the following conditions:
+
+      (a) You must give any other recipients of the Work or
+          Derivative Works a copy of this License; and
+
+      (b) You must cause any modified files to carry prominent notices
+          stating that You changed the files; and
+
+      (c) You must retain, in the Source form of any Derivative Works
+          that You distribute, all copyright, patent, trademark, and
+          attribution notices from the Source form of the Work,
+          excluding those notices that do not pertain to any part of
+          the Derivative Works; and
+
+      (d) If the Work includes a "NOTICE" text file as part of its
+          distribution, then any Derivative Works that You distribute must
+          include a readable copy of the attribution notices contained
+          within such NOTICE file, excluding those notices that do not
+          pertain to any part of the Derivative Works, in at least one
+          of the following places: within a NOTICE text file distributed
+          as part of the Derivative Works; within the Source form or
+          documentation, if provided along with the Derivative Works; or,
+          within a display generated by the Derivative Works, if and
+          wherever such third-party notices normally appear. The contents
+          of the NOTICE file are for informational purposes only and
+          do not modify the License. You may add Your own attribution
+          notices within Derivative Works that You distribute, alongside
+          or as an addendum to the NOTICE text from the Work, provided
+          that such additional attribution notices cannot be construed
+          as modifying the License.
+
+      You may add Your own copyright statement to Your modifications and
+      may provide additional or different license terms and conditions
+      for use, reproduction, or distribution of Your modifications, or
+      for any such Derivative Works as a whole, provided Your use,
+      reproduction, and distribution of the Work otherwise complies with
+      the conditions stated in this License.
+
+   5. Submission of Contributions. Unless You explicitly state otherwise,
+      any Contribution intentionally submitted for inclusion in the Work
+      by You to the Licensor shall be under the terms and conditions of
+      this License, without any additional terms or conditions.
+      Notwithstanding the above, nothing herein shall supersede or modify
+      the terms of any separate license agreement you may have executed
+      with Licensor regarding such Contributions.
+
+   6. Trademarks. This License does not grant permission to use the trade
+      names, trademarks, service marks, or product names of the Licensor,
+      except as required for reasonable and customary use in describing the
+      origin of the Work and reproducing the content of the NOTICE file.
+
+   7. Disclaimer of Warranty. Unless required by applicable law or
+      agreed to in writing, Licensor provides the Work (and each
+      Contributor provides its Contributions) on an "AS IS" BASIS,
+      WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+      implied, including, without limitation, any warranties or conditions
+      of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
+      PARTICULAR PURPOSE. You are solely responsible for determining the
+      appropriateness of using or redistributing the Work and assume any
+      risks associated with Your exercise of permissions under this License.
+
+   8. Limitation of Liability. In no event and under no legal theory,
+      whether in tort (including negligence), contract, or otherwise,
+      unless required by applicable law (such as deliberate and grossly
+      negligent acts) or agreed to in writing, shall any Contributor be
+      liable to You for damages, including any direct, indirect, special,
+      incidental, or consequential damages of any character arising as a
+      result of this License or out of the use or inability to use the
+      Work (including but not limited to damages for loss of goodwill,
+      work stoppage, computer failure or malfunction, or any and all
+      other commercial damages or losses), even if such Contributor
+      has been advised of the possibility of such damages.
+
+   9. Accepting Warranty or Additional Liability. While redistributing
+      the Work or Derivative Works thereof, You may choose to offer,
+      and charge a fee for, acceptance of support, warranty, indemnity,
+      or other liability obligations and/or rights consistent with this
+      License. However, in accepting such obligations, You may act only
+      on Your own behalf and on Your sole responsibility, not on behalf
+      of any other Contributor, and only if You agree to indemnify,
+      defend, and hold each Contributor harmless for any liability
+      incurred by, or claims asserted against, such Contributor by reason
+      of your accepting any such warranty or additional liability.
+
+   END OF TERMS AND CONDITIONS
+
+   APPENDIX: How to apply the Apache License to your work.
+
+      To apply the Apache License to your work, attach the following
+      boilerplate notice, with the fields enclosed by brackets "[]"
+      replaced with your own identifying information. (Don't include
+      the brackets!)  The text should be enclosed in the appropriate
+      comment syntax for the file format. We also recommend that a
+      file or class name and description of purpose be included on the
+      same "printed page" as the copyright notice for easier
+      identification within third-party archives.
+
+   Copyright [yyyy] [name of copyright owner]
+
+   Licensed under the Apache License, Version 2.0 (the "License");
+   you may not use this file except in compliance with the License.
+   You may obtain a copy of the License at
+
+       http://www.apache.org/licenses/LICENSE-2.0
+
+   Unless required by applicable law or agreed to in writing, software
+   distributed under the License is distributed on an "AS IS" BASIS,
+   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+   See the License for the specific language governing permissions and
+   limitations under the License.

Procfile

@@ -0,0 +1 @@
+web: python -m uvicorn backend.server.server:app --host=0.0.0.0 --port=${PORT}

README-ja_JP.md

@@ -0,0 +1,159 @@
+<div align="center">
+<!--<h1 style="display: flex; align-items: center; gap: 10px;">
+  <img src="https://github.com/assafelovic/gpt-researcher/assets/13554167/a45bac7c-092c-42e5-8eb6-69acbf20dde5" alt="Logo" width="25">
+  GPT Researcher
+</h1>-->
+<img src="https://github.com/assafelovic/gpt-researcher/assets/13554167/20af8286-b386-44a5-9a83-3be1365139c3" alt="Logo" width="80">
+
+
+####
+
+[![公式サイト](https://img.shields.io/badge/公式サイト-gptr.dev-blue?style=for-the-badge&logo=world&logoColor=white)](https://gptr.dev)
+[![Documentation](https://img.shields.io/badge/Documentation-DOCS-f472b6?logo=googledocs&logoColor=white&style=for-the-badge)](https://docs.gptr.dev)
+[![Discord Follow](https://img.shields.io/discord/1127851779011391548?style=for-the-badge&logo=discord&label=Chat%20on%20Discord)](https://discord.gg/QgZXvJAccX)
+
+[![PyPI version](https://img.shields.io/pypi/v/gpt-researcher?logo=pypi&logoColor=white&style=flat)](https://badge.fury.io/py/gpt-researcher)
+![GitHub Release](https://img.shields.io/github/v/release/assafelovic/gpt-researcher?style=flat&logo=github)
+[![Open In Colab](https://img.shields.io/static/v1?message=Open%20in%20Colab&logo=googlecolab&labelColor=grey&color=yellow&label=%20&style=flat&logoSize=40)](https://colab.research.google.com/github/assafelovic/gpt-researcher/blob/master/docs/docs/examples/pip-run.ipynb)
+[![Docker Image Version](https://img.shields.io/docker/v/elestio/gpt-researcher/latest?arch=amd64&style=flat&logo=docker&logoColor=white&color=1D63ED)](https://hub.docker.com/r/gptresearcher/gpt-researcher)
+[![Twitter Follow](https://img.shields.io/twitter/follow/assaf_elovic?style=social)](https://twitter.com/assaf_elovic)
+
+[English](README.md) |
+[中文](README-zh_CN.md) |
+[日本語](README-ja_JP.md) |
+[한국어](README-ko_KR.md)
+</div>
+
+# 🔎 GPT Researcher
+
+**GPT Researcher は、さまざまなタスクに対する包括的なオンラインリサーチのために設計された自律エージェントです。**
+
+このエージェントは、詳細で事実に基づいた偏りのない研究レポートを生成することができ、関連するリソース、アウトライン、およびレッスンに焦点を当てるためのカスタマイズオプションを提供します。最近の [Plan-and-Solve](https://arxiv.org/abs/2305.04091) および [RAG](https://arxiv.org/abs/2005.11401) 論文に触発され、GPT Researcher は速度、決定論、および信頼性の問題に対処し、同期操作ではなく並列化されたエージェント作業を通じてより安定したパフォーマンスと高速化を提供します。
+
+**私たちの使命は、AIの力を活用して、個人や組織に正確で偏りのない事実に基づいた情報を提供することです。**
+
+## なぜGPT Researcherなのか？
+
+- 手動の研究タスクで客観的な結論を形成するには時間がかかることがあり、適切なリソースと情報を見つけるのに数週間かかることもあります。
+- 現在のLLMは過去の情報に基づいて訓練されており、幻覚のリスクが高く、研究タスクにはほとんど役に立ちません。
+- 現在のLLMは短いトークン出力に制限されており、長く詳細な研究レポート（2,000語以上）には不十分です。
+- Web検索を可能にするサービス（ChatGPT + Webプラグインなど）は、限られたリソースとコンテンツのみを考慮し、場合によっては表面的で偏った回答をもたらします。
+- Webソースの選択のみを使用すると、研究タスクの正しい結論を導く際にバイアスが生じる可能性があります。
+
+## アーキテクチャ
+主なアイデアは、「プランナー」と「実行」エージェントを実行することであり、プランナーは研究する質問を生成し、実行エージェントは生成された各研究質問に基づいて最も関連性の高い情報を探します。最後に、プランナーはすべての関連情報をフィルタリングおよび集約し、研究レポートを作成します。<br /> <br /> 
+エージェントは、研究タスクを完了するために gpt-4o-mini と gpt-4o（128K コンテキスト）の両方を活用します。必要に応じてそれぞれを使用することでコストを最適化します。**平均的な研究タスクは完了するのに約3分かかり、コストは約0.1ドルです**。
+
+<div align="center">
+<img align="center" height="500" src="https://cowriter-images.s3.amazonaws.com/architecture.png">
+</div>
+
+
+詳細説明:
+* 研究クエリまたはタスクに基づいて特定のドメインエージェントを作成します。
+* 研究タスクに対する客観的な意見を形成する一連の研究質問を生成します。
+* 各研究質問に対して、与えられたタスクに関連する情報をオンラインリソースから収集��るクローラーエージェントをトリガーします。
+* 各収集されたリソースについて、関連情報に基づいて要約し、そのソースを追跡します。
+* 最後に、すべての要約されたソースをフィルタリングおよび集約し、最終的な研究レポートを生成します。
+
+## デモ
+https://github.com/assafelovic/gpt-researcher/assets/13554167/a00c89a6-a295-4dd0-b58d-098a31c40fda
+
+## チュートリアル
+ - [動作原理](https://docs.gptr.dev/blog/building-gpt-researcher)
+ - [インストール方法](https://www.loom.com/share/04ebffb6ed2a4520a27c3e3addcdde20?sid=da1848e8-b1f1-42d1-93c3-5b0b9c3b24ea)
+ - [ライブデモ](https://www.loom.com/share/6a3385db4e8747a1913dd85a7834846f?sid=a740fd5b-2aa3-457e-8fb7-86976f59f9b8)
+
+## 特徴
+- 📝 研究、アウトライン、リソース、レッスンレポートを生成
+- 🌐 各研究で20以上のWebソースを集約し、客観的で事実に基づいた結論を形成
+- 🖥️ 使いやすいWebインターフェース（HTML/CSS/JS）を含む
+- 🔍 JavaScriptサポート付きのWebソースをスクレイピング
+- 📂 訪問および使用されたWebソースのコンテキストを追跡
+- 📄 研究レポートをPDF、Wordなどにエクスポート
+
+## 📖 ドキュメント
+
+完全なドキュメントについては、[こちら](https://docs.gptr.dev/docs/gpt-researcher/getting-started/getting-started)を参照してください：
+
+- 入門（インストール、環境設定、簡単な例）
+- 操作例（デモ、統合、dockerサポート）
+- 参考資料（API完全ドキュメント）
+- Tavilyアプリケーションインターフェースの統合（コア概念の高度な説明）
+
+## クイックスタート
+> **ステップ 0** - Python 3.11 以降をインストールします。[こちら](https://www.tutorialsteacher.com/python/install-python)を参照して、ステップバイステップのガイドを確認してください。
+
+<br />
+
+> **ステップ 1** - プロジェクトをダウンロードします
+
+```bash
+$ git clone https://github.com/assafelovic/gpt-researcher.git
+$ cd gpt-researcher
+```
+
+<br />
+
+> **ステップ2** - 依存関係をインストールします
+```bash
+$ pip install -r requirements.txt
+```
+<br />
+
+> **ステップ 3** - OpenAI キーと Tavily API キーを使用して .env ファイルを作成するか、直接エクスポートします
+
+```bash
+$ export OPENAI_API_KEY={Your OpenAI API Key here}
+```
+```bash
+$ export TAVILY_API_KEY={Your Tavily API Key here}
+```
+
+- **LLMには、[OpenAI GPT](https://platform.openai.com/docs/guides/gpt) を使用することをお勧めします**が、[Langchain Adapter](https://python.langchain.com/docs/guides/adapters/openai) がサポートする他の LLM モデル（オープンソースを含む）を使用することもできます。llm モデルとプロバイダーを config/config.py で変更するだけです。[このガイド](https://python.langchain.com/docs/integrations/llms/) に従って、LLM を Langchain と統合する方法を学んでください。
+- **検索エンジンには、[Tavily Search API](https://app.tavily.com)（LLM 用に最適化されています）を使用することをお勧めします**が、他の検索エンジンを選択することもできます。config/config.py で検索プロバイダーを「duckduckgo」、「googleAPI」、「googleSerp」、「searchapi」、「searx」に変更するだけです。次に、config.py ファイルに対応する env API キーを追加します。
+- **最適なパフォーマンスを得るために、[OpenAI GPT](https://platform.openai.com/docs/guides/gpt) モデルと [Tavily Search API](https://app.tavily.com) を使用することを強くお勧めします。**
+<br />
+
+> **ステップ 4** - FastAPI を使用してエージェントを実行します
+
+```bash
+$ uvicorn main:app --reload
+```
+<br />
+
+> **ステップ 5** - 任意のブラウザで http://localhost:8000 にアクセスして、リサーチを楽しんでください！
+
+Docker の使い方や機能とサービスの詳細については、[ドキュメント](https://docs.gptr.dev) ページをご覧ください。
+
+## 🚀 貢献
+私たちは貢献を大歓迎します！興味がある場合は、[貢献](CONTRIBUTING.md) をご覧ください。
+
+私たちの[ロードマップ](https://trello.com/b/3O7KBePw/gpt-researcher-roadmap) ページを確認し、私たちの使命に参加することに興味がある場合は、[Discord コミュニティ](https://discord.gg/QgZXvJAccX) を通じてお問い合わせください。
+
+## ✉️ サポート / お問い合わせ
+- [コミュニティディスカッション](https://discord.gg/spBgZmm3Xe)
+- 私たちのメール: [email protected]
+
+## 🛡 免責事項
+
+このプロジェクト「GPT Researcher」は実験的なアプリケーションであり、明示または黙示のいかなる保証もなく「現状のまま」提供されます。私たちは学術目的のためにMITライセンスの下でコードを共有しています。ここに記載されている内容は学術的なアドバイスではなく、学術論文や研究論文での使用を推奨するものではありません。
+
+私たちの客観的な研究主張に対する見解：
+1. 私たちのスクレイピングシステムの主な目的は、不正確な事実を減らすことです。どうやって解決するのか？私たちがスクレイピングするサイトが多ければ多いほど、誤ったデータの可能性は低くなります。各研究で20の情報を収集し、それらがすべて間違っている可能性は非常に低いです。
+2. 私たちの目標はバイアスを排除することではなく、可能な限りバイアスを減らすことです。**私たちはここでコミュニティとして最も効果的な人間と機械の相互作用を探求しています**。
+3. 研究プロセスでは、人々も自分が研究しているトピックに対してすでに意見を持っているため、バイアスがかかりやすいです。このツールは多くの意見を収集し、偏った人が決して読まないであろう多様な見解を均等に説明します。
+
+**GPT-4 言語モデルの使用は、トークンの使用により高額な費用がかかる可能性があることに注意してください**。このプロジェクトを利用することで、トークンの使用状況と関連する費用を監視および管理する責任があることを認めたことになります。OpenAI API の使用状況を定期的に確認し、予期しない料金が発生しないように必要な制限やアラートを設定することを強くお勧めします。
+
+---
+
+<p align="center">
+<a href="https://star-history.com/#assafelovic/gpt-researcher">
+  <picture>
+    <source media="(prefers-color-scheme: dark)" srcset="https://api.star-history.com/svg?repos=assafelovic/gpt-researcher&type=Date&theme=dark" />
+    <source media="(prefers-color-scheme: light)" srcset="https://api.star-history.com/svg?repos=assafelovic/gpt-researcher&type=Date" />
+    <img alt="Star History Chart" src="https://api.star-history.com/svg?repos=assafelovic/gpt-researcher&type=Date" />
+  </picture>
+</a>
+</p>

README-ko_KR.md

@@ -0,0 +1,242 @@
+<div align="center">
+<!--<h1 style="display: flex; align-items: center; gap: 10px;">
+  <img src="https://github.com/assafelovic/gpt-researcher/assets/13554167/a45bac7c-092c-42e5-8eb6-69acbf20dde5" alt="Logo" width="25">
+  GPT Researcher
+</h1>-->
+<img src="https://github.com/assafelovic/gpt-researcher/assets/13554167/20af8286-b386-44a5-9a83-3be1365139c3" alt="Logo" width="80">
+
+
+####
+
+[![Website](https://img.shields.io/badge/Official%20Website-gptr.dev-teal?style=for-the-badge&logo=world&logoColor=white&color=0891b2)](https://gptr.dev)
+[![Documentation](https://img.shields.io/badge/Documentation-DOCS-f472b6?logo=googledocs&logoColor=white&style=for-the-badge)](https://docs.gptr.dev)
+[![Discord Follow](https://img.shields.io/discord/1127851779011391548?style=for-the-badge&logo=discord&label=Chat%20on%20Discord)](https://discord.gg/QgZXvJAccX)
+
+[![PyPI version](https://img.shields.io/pypi/v/gpt-researcher?logo=pypi&logoColor=white&style=flat)](https://badge.fury.io/py/gpt-researcher)
+![GitHub Release](https://img.shields.io/github/v/release/assafelovic/gpt-researcher?style=flat&logo=github)
+[![Open In Colab](https://img.shields.io/static/v1?message=Open%20in%20Colab&logo=googlecolab&labelColor=grey&color=yellow&label=%20&style=flat&logoSize=40)](https://colab.research.google.com/github/assafelovic/gpt-researcher/blob/master/docs/docs/examples/pip-run.ipynb)
+[![Docker Image Version](https://img.shields.io/docker/v/elestio/gpt-researcher/latest?arch=amd64&style=flat&logo=docker&logoColor=white&color=1D63ED)](https://hub.docker.com/r/gptresearcher/gpt-researcher)
+[![Twitter Follow](https://img.shields.io/twitter/follow/assaf_elovic?style=social)](https://twitter.com/assaf_elovic)
+
+[English](README.md) |
+[中文](README-zh_CN.md) |
+[日本語](README-ja_JP.md) |
+[한국어](README-ko_KR.md)
+</div>
+
+# 🔎 GPT Researcher
+
+**GPT Researcher는 다양한 작업을 대해 포괄적인 온라인 연구를 수행하도록 설계된 자율 에이전트입니다.**
+
+이 에이전트는 세부적이고 사실에 기반하며 편견 없는 연구 보고서를 생성할 수 있으며, 관련 리소스와 개요에 초점을 맞춘 맞춤형 옵션을 제공합니다.  최근 발표된 [Plan-and-Solve](https://arxiv.org/abs/2305.04091) 및 [RAG](https://arxiv.org/abs/2005.11401) 논문에서 영감을 받아 GPT Researcher는 잘못된 정보, 속도, 결정론적 접근 방식, 신뢰성 문제를 해결하고, 동기화 작업이 아닌 병렬 에이전트 작업을 통해 더 안정적이고 빠른 성능을 제공합니다.
+
+**우리의 목표는 AI의 힘을 활용하여 개인과 조직에게 정확하고 편향 없는 사실에 기반한 정보를 제공하는 것입니다.**
+
+## 왜 GPT Researcher인가?
+
+- 직접 수행하는 연구 과정은 객관적인 결론을 도출하는 데 시간이 오래 걸리며, 적절한 리소스와 정보를 찾는 데 몇 주가 걸릴 수 있습니다.
+- 현재의 대규모 언어 모델(LLM)은 과거 정보에 기반해 훈련되었으며, 환각 현상이 발생할 위험이 높아 연구 작업에는 적합하지 않습니다.
+- 현재 LLM은 짧은 토큰 출력으로 제한되며, 2,000단어 이상의 길고 자세한 연구 보고서를 작성하는 데는 충분하지 않습니다.
+- 웹 검색을 지원하는 서비스(예: ChatGPT 또는 Perplexity)는 제한된 리소스와 콘텐츠만을 고려하여 경우에 따라 피상적이고 편향된 답변을 제공합니다.
+- 웹 소스만을 사용하면 연구 작업에서 올바른 결론을 도출할 때 편향이 발생할 수 있습니다.
+
+## 데모
+https://github.com/user-attachments/assets/092e9e71-7e27-475d-8c4f-9dddd28934a3
+
+## 아키텍처
+주요 아이디어는 "플래너"와 "실행" 에이전트를 실행하는 것으로, 플래너는 연구할 질문을 생성하고, 실행 에이전트는 생성된 각 연구 질문에 따라 가장 관련성 높은 정보를 찾습니다. 마지막으로 플래너는 모든 관련 정보를 필터링하고 집계하여 연구 보고서를 작성합니다.
+<br /> <br /> 
+에이전트는 `gpt-4o-mini`와 `gpt-4o`(128K 컨텍스트)를 활용하여 연구 작업을 완료합니다. 필요에 따라 각각을 사용하여 비용을 최적화합니다. **평균 연구 작업은 약 2분이 소요되며, 비용은 약 $0.005입니다.**.
+
+<div align="center">
+<img align="center" height="600" src="https://github.com/assafelovic/gpt-researcher/assets/13554167/4ac896fd-63ab-4b77-9688-ff62aafcc527">
+</div>
+
+구체적으로:
+* 연구 쿼리 또는 작업을 기반으로 도메인별 에이전트를 생성합니다.
+* 주어진 작업에 대해 객관적인 의견을 형성할 수 있는 일련의 연구 질문을 생성합니다.
+* 각 연구 질문에 대해 크롤러 에이전트를 실행하여 작업과 관련된 정보를 온라인 리소스에서 수집합니다.
+* 수집된 각 리소스에서 관련 정보를 요약하고 출처를 기록합니다.
+* 마지��으로, 요약된 모든 정보를 필터링하고 집계하여 최종 연구 보고서를 생성합니다.
+
+## 튜토리얼
+ - [동작원리](https://docs.gptr.dev/blog/building-gpt-researcher)
+ - [설치방법](https://www.loom.com/share/04ebffb6ed2a4520a27c3e3addcdde20?sid=da1848e8-b1f1-42d1-93c3-5b0b9c3b24ea)
+ - [라이브 데모](https://www.loom.com/share/6a3385db4e8747a1913dd85a7834846f?sid=a740fd5b-2aa3-457e-8fb7-86976f59f9b8)
+
+
+## 기능
+- 📝 로컬 문서 및 웹 소스를 사용하여 연구, 개요, 리소스 및 학습 보고서 생성
+- 📜 2,000단어 이상의 길고 상세한 연구 보고서 생성 가능
+- 🌐 연구당 20개 이상의 웹 소스를 집계하여 객관적이고 사실에 기반한 결론 도출
+- 🖥️ 경량 HTML/CSS/JS와 프로덕션용 (NextJS + Tailwind) UX/UI 포함
+- 🔍 자바스크립트 지원 웹 소스 스크래핑 기능
+- 📂 연구 과정에서 맥락과 메모리 추적 및 유지
+- 📄 연구 보고서를 PDF, Word 등으로 내보내기 지원
+
+## 📖 문서
+
+전체 문서(설치, 환경 설정, 간단한 예시)를 보려면 [여기](https://docs.gptr.dev/docs/gpt-researcher/getting-started/getting-started)를 참조하세요.
+
+- 시작하기 (설치, 환경 설정, 간단한 예시)
+- 맞춤 설정 및 구성
+- 사용 방법 예시 (데모, 통합, 도커 지원)
+- 참고자료 (전체 API 문서)
+
+## ⚙️ 시작하기
+### 설치
+> **1단계** - Python 3.11 또는 그 이상의 버전을 설치하세요. [여기](https://www.tutorialsteacher.com/python/install-python)를 참조하여 단계별 가이드를 확인하세요.
+
+> **2단계** - 프로젝트를 다운로드하고 해당 디렉토리로 이동하세요.
+
+```bash
+git clone https://github.com/assafelovic/gpt-researcher.git
+cd gpt-researcher
+```
+
+> **3단계** - 두 가지 방법으로 API 키를 설정하세요: 직접 export하거나 `.env` 파일에 저장하세요.
+
+Linux/Windows에서 임시 설정을 하려면 export 방법을 사용하세요:
+
+```bash
+export OPENAI_API_KEY={OpenAI API 키 입력}
+export TAVILY_API_KEY={Tavily API 키 입력}
+```
+
+더 영구적인 설정을 원한다면, 현재의 `gpt-researcher` 디렉토리에 `.env` 파일을 생성하고 환경 변수를 입력하세요 (export 없이).
+
+- 기본 LLM은 [GPT](https://platform.openai.com/docs/guides/gpt)이지만, `claude`, `ollama3`, `gemini`, `mistral` 등 다른 LLM도 사용할 수 있습니다. LLM 제공자를 변경하는 방법은 [LLMs 문서](https://docs.gptr.dev/docs/gpt-researcher/llms/llms)를 참조하세요. 이 프로젝트는 OpenAI GPT 모델에 최적화되어 있습니다.
+- 기본 검색기는 [Tavily](https://app.tavily.com)이지만, `duckduckgo`, `google`, `bing`, `searchapi`, `serper`, `searx`, `arxiv`, `exa` 등의 검색기를 사용할 수 있습니다. 검색 제공자를 변경하는 방법은 [검색기 문서](https://docs.gptr.dev/docs/gpt-researcher/retrievers)를 참조하세요.
+
+### 빠른 시작
+
+> **1단계** - 필요한 종속성 설치
+
+```bash
+pip install -r requirements.txt
+```
+
+> **2단계** - FastAPI로 에이전트 실행
+
+```bash
+python -m uvicorn main:app --reload
+```
+
+> **3단계** - 브라우저에서 http://localhost:8000 으로 이동하여 연구를 시작하세요!
+
+<br />
+
+**[Poetry](https://docs.gptr.dev/docs/gpt-researcher/getting-started/getting-started#poetry) 또는 [가상 환경](https://docs.gptr.dev/docs/gpt-researcher/getting-started/getting-started#virtual-environment)에 대해 배우고 싶다면, [문서](https://docs.gptr.dev/docs/gpt-researcher/getting-started/getting-started)를 참조하세요.**
+
+### PIP 패키지로 실행하기
+```bash
+pip install gpt-researcher
+```
+
+```python
+...
+from gpt_researcher import GPTResearcher
+
+query = "왜 Nvidia 주식이 오르고 있나요?"
+researcher = GPTResearcher(query=query, report_type="research_report")
+# 주어진 질문에 대한 연구 수행
+research_result = await researcher.conduct_research()
+# 보고서 작성
+report = await researcher.write_report()
+...
+```
+
+**더 많은 예제와 구성 옵션은 [PIP 문서](https://docs.gptr.dev/docs/gpt-researcher/gptr/pip-package)를 참조하세요.**
+
+## Docker로 실행
+
+> **1단계** - [Docker 설치](https://docs.gptr.dev/docs/gpt-researcher/getting-started/getting-started-with-docker)
+
+> **2단계** - `.env.example` 파일을 복사하고 API 키를 추가한 후, 파일을 `.env`로 저장하세요.
+
+> **3단계** - docker-compose 파일에서 실행하고 싶지 않은 서비스를 주석 처리하세요.
+
+```bash
+$ docker-compose up --build
+```
+
+> **4단계** - docker-compose 파일에서 아무 것도 주석 처리하지 않았다면, 기본적으로 두 가지 프로세스가 시작됩니다:
+ - localhost:8000에서 실행 중인 Python 서버<br>
+ - localhost:3000에서 실행 중인 React 앱<br>
+
+브라우저에서 localhost:3000으로 이동하여 연구를 시작하세요!
+
+## 📄 로컬 문서로 연구하기
+
+GPT Researcher를 사용하여 로컬 문서를 기반으로 연구 작업을 수행할 수 있습니다. 현재 지원되는 파일 형식은 PDF, 일반 텍스트, CSV, Excel, Markdown, PowerPoint, Word 문서입니다.
+
+1단계: `DOC_PATH` 환경 변수를 설정하여 문서가 있는 폴더를 지정하세요.
+
+```bash
+export DOC_PATH="./my-docs"
+```
+
+2단계:
+ - 프론트엔드 앱을 localhost:8000에서 실행 중이라면, "Report Source" 드롭다운 옵션에서 "My Documents"를 선택하세요.
+ - GPT Researcher를 [PIP 패키지](https://docs.tavily.com/docs/gpt-researcher/pip-package)로 실행 중이라면, `report_source` 인수를 "local"로 설정하여 `GPTResearcher` 클래스를 인스턴스화하세요. [코드 예제](https://docs.gptr.dev/docs/gpt-researcher/context/tailored-research)를 참조하세요.
+
+## 👪 다중 에이전트 어시스턴트
+
+AI가 프롬프트 엔지니어링 및 RAG에서 다중 에이전트 시스템으로 발전함에 따라, 우리는 [LangGraph](https://python.langchain.com/v0.1/docs/langgraph/)로 구축된 새로운 다중 에이전트 어시스턴트를 소개합니다.
+
+LangGraph를 사용하면 여러 에이전트의 전문 기술을 활용하여 연구 과정의 깊이와 질을 크게 향상시킬 수 있습니다. 최근 [STORM](https://arxiv.org/abs/2402.14207) 논문에서 영감을 받아, 이 프로젝트는 AI 에이전트 팀이 주제에 대한 연구를 계획에서 출판까지 함께 수행하는 방법을 보여줍니다.
+
+평균 실행은 5-6 페이지 분량의 연구 보고서를 PDF, Docx, Markdown 형식으로 생성합니다.
+
+[여기](https://github.com/assafelovic/gpt-researcher/tree/master/multi_agents)에서 확인하거나 [문서](https://docs.gptr.dev/docs/gpt-researcher/multi_agents/langgraph)에서 자세한 내용을 참조하세요.
+
+## 🖥️ 프론트엔드 애플리케이션
+
+GPT-Researcher는 사용자 경험을 개선하고 연구 프로세스를 간소화하기 위해 향상된 프론트엔드를 제공합니다. 프론트엔드는 다음과 같은 기능을 제공합니다:
+
+- 연구 쿼리를 입력할 수 있는 직관적인 인터페이스
+- 연구 작업의 실시간 진행 상황 추적
+- 연구 결과의 대화형 디스플레이
+- 맞춤형 연구 경험을 위한 설정 가능
+
+두 가지 배포 옵션이 있습니다:
+1. FastAPI로 제공되는 경량 정적 프론트엔드
+2. 고급 기능을 제공하는 NextJS 애플리케이션
+
+프론트엔드 기능에 대한 자세한 설치 방법 및 정보를 원하시면 [문서 페이지](https://docs.gptr.dev/docs/gpt-researcher/frontend/frontend)를 참조하세요.
+
+## 🚀 기여하기
+우리는 기여를 적극 환영합니다! 관심이 있다면 [기여 가이드](https://github.com/assafelovic/gpt-researcher/blob/master/CONTRIBUTING.md)를 확인해 주세요.
+
+[로드맵](https://trello.com/b/3O7KBePw/gpt-researcher-roadmap) 페이지를 확인하고, 우리 [Discord 커뮤니티](https://discord.gg/QgZXvJAccX)에 가입하여 우리의 목표에 함께 참여해 주세요.
+<a href="https://github.com/assafelovic/gpt-researcher/graphs/contributors">
+  <img src="https://contrib.rocks/image?repo=assafelovic/gpt-researcher" />
+</a>
+
+## ✉️ 지원 / 문의
+- [커뮤니티 Discord](https://discord.gg/spBgZmm3Xe)
+- 저자 이메일: [email protected]
+
+## 🛡️ 면책 조항
+
+이 프로젝트인 GPT Researcher는 실험적인 응용 프로그램이며, 명시적이거나 묵시적인 보증 없이 "있는 그대로" 제공됩니다. 우리는 이 코드를 학술적 목적으로 Apache 2 라이선스 하에 공유하고 있습니다. 여기에 있는 것은 학술적 조언이 아니며, 학술 또는 연구 논문에 사용하는 것을 권장하지 않습니다.
+
+편향되지 않은 연구 주장에 대한 우리의 견해:
+1. GPT Researcher의 주요 목표는 잘못된 정보와 편향된 사실을 줄이는 것입니다. 그 방법은 무엇일까요? 우리는 더 많은 사이트를 스크래핑할수록 잘못된 데이터의 가능성이 줄어든다고 가정합니다. 여러 사이트에서 정보를 스크래핑하고 가장 빈번한 정보를 선택하면, 모든 정보가 틀릴 확률은 매우 낮습니다.
+2. 우리는 편향을 완전히 제거하려고 하지는 않지만, 가능한 한 줄이는 것을 목표로 합니다. **우리는 인간과 LLM의 가장 효과적인 상호작용을 찾기 위한 커뮤니티입니다.**
+3. 연구에서 사람들도 이미 자신이 연구하는 주제에 대해 의견을 가지고 있기 때문에 편향되는 경향이 있습니다. 이 도구는 많은 의견을 스크래핑하며, 편향된 사람이라면 결코 읽지 않았을 다양한 견해를 고르게 설명합니다.
+
+**GPT-4 모델을 사용할 경우, 토큰 사용량 때문에 비용이 많이 들 수 있습니다.** 이 프로젝트를 사���하는 경우, 자신의 토큰 사용량 및 관련 비용을 모니터링하고 관리하는 것은 본인의 책임입니다. OpenAI API 사용량을 정기적으로 확인하고, 예상치 못한 비용을 방지하기 위해 필요한 한도를 설정하거나 알림을 설정하는 것이 좋습니다.
+
+
+---
+
+<p align="center">
+<a href="https://star-history.com/#assafelovic/gpt-researcher">
+  <picture>
+    <source media="(prefers-color-scheme: dark)" srcset="https://api.star-history.com/svg?repos=assafelovic/gpt-researcher&type=Date&theme=dark" />
+    <source media="(prefers-color-scheme: light)" srcset="https://api.star-history.com/svg?repos=assafelovic/gpt-researcher&type=Date" />
+    <img alt="Star History Chart" src="https://api.star-history.com/svg?repos=assafelovic/gpt-researcher&type=Date" />
+  </picture>
+</a>
+</p>

README-zh_CN.md

@@ -0,0 +1,158 @@
+<div align="center">
+<!--<h1 style="display: flex; align-items: center; gap: 10px;">
+  <img src="https://github.com/assafelovic/gpt-researcher/assets/13554167/a45bac7c-092c-42e5-8eb6-69acbf20dde5" alt="Logo" width="25">
+  GPT Researcher
+</h1>-->
+<img src="https://github.com/assafelovic/gpt-researcher/assets/13554167/20af8286-b386-44a5-9a83-3be1365139c3" alt="Logo" width="80">
+
+
+####
+
+[![Website](https://img.shields.io/badge/Official%20Website-gptr.dev-teal?style=for-the-badge&logo=world&logoColor=white&color=0891b2)](https://gptr.dev)
+[![Documentation](https://img.shields.io/badge/Documentation-DOCS-f472b6?logo=googledocs&logoColor=white&style=for-the-badge)](https://docs.gptr.dev)
+[![Discord Follow](https://img.shields.io/discord/1127851779011391548?style=for-the-badge&logo=discord&label=Chat%20on%20Discord)](https://discord.gg/QgZXvJAccX)
+
+[![PyPI version](https://img.shields.io/pypi/v/gpt-researcher?logo=pypi&logoColor=white&style=flat)](https://badge.fury.io/py/gpt-researcher)
+![GitHub Release](https://img.shields.io/github/v/release/assafelovic/gpt-researcher?style=flat&logo=github)
+[![Open In Colab](https://img.shields.io/static/v1?message=Open%20in%20Colab&logo=googlecolab&labelColor=grey&color=yellow&label=%20&style=flat&logoSize=40)](https://colab.research.google.com/github/assafelovic/gpt-researcher/blob/master/docs/docs/examples/pip-run.ipynb)
+[![Docker Image Version](https://img.shields.io/docker/v/elestio/gpt-researcher/latest?arch=amd64&style=flat&logo=docker&logoColor=white&color=1D63ED)](https://hub.docker.com/r/gptresearcher/gpt-researcher)
+[![Twitter Follow](https://img.shields.io/twitter/follow/assaf_elovic?style=social)](https://twitter.com/assaf_elovic)
+
+[English](README.md) |
+[中文](README-zh_CN.md) |
+[日本語](README-ja_JP.md) |
+[한국어](README-ko_KR.md)
+</div>
+
+# 🔎 GPT Researcher
+
+**GPT Researcher 是一个智能体代理，专为各种任务的综合在线研究而设计。**
+
+代理可以生成详细、正式且客观的研究报告，并提供自定义选项，专注于相关资源、结构框架和经验报告。受最近发表的[Plan-and-Solve](https://arxiv.org/abs/2305.04091) 和[RAG](https://arxiv.org/abs/2005.11401) 论文的启发，GPT Researcher 解决了速度、确定性和可靠性等问题，通过并行化的代理运行，而不是同步操作，提供了更稳定的性能和更高的速度。
+
+**我们的使命是利用人工智能的力量，为个人和组织提供准确、客观和事实的信息。**
+
+## 为什么选择GPT Researcher?
+
+- 因为人工研究任务形成客观结论可能需要时间和经历，有时甚至需要数周才能找到正确的资源和信息。
+- 目前的LLM是根据历史和过时的信息进行训练的，存在严重的幻觉风险，因此几乎无法胜任研究任务。
+- 网络搜索的解决方案（例如 ChatGPT + Web 插件）仅考虑有限的资源和内容，在某些情况下会导致肤浅的结论或不客观的答案。
+- 只使用部分资源可能会在确定研究问题或任务的正确结论时产生偏差。
+
+## 架构
+主要思想是运行“**计划者**”和“**执行**”代理，而**计划者**生成问题进行研究，“**执行**”代理根据每个生成的研究问题寻找最相关的信息。最后，“**计划者**”过滤和聚合所有相关信息并创建研究报告。<br /> <br /> 
+代理同时利用 gpt-40-mini 和 gpt-4o（128K 上下文）来完成一项研究任务。我们仅在必要时使用这两种方法对成本进行优化。**研究任务平均耗时约 3 分钟，成本约为 ~0.1 美元**。
+
+<div align="center">
+<img align="center" height="500" src="https://cowriter-images.s3.amazonaws.com/architecture.png">
+</div>
+
+
+详细说明:
+* 根据研究搜索或任务创建特定领域的代理。
+* 生成一组研究问题，这些问题共同形成答案对任何给定任务的客观意见。
+* 针对每个研究问题，触发一个爬虫代理，从在线资源中搜索与给定任务相关的信息。
+* 对于每一个抓取的资源，根据相关信息进行汇总，并跟踪其来源。
+* 最后，对所有汇总的资料来源进行过滤和汇总，并生成最终研究报告。
+
+## 演示
+https://github.com/assafelovic/gpt-researcher/assets/13554167/a00c89a6-a295-4dd0-b58d-098a31c40fda
+
+## 教程
+ - [运行原理](https://docs.gptr.dev/blog/building-gpt-researcher)
+ - [如何安装](https://www.loom.com/share/04ebffb6ed2a4520a27c3e3addcdde20?sid=da1848e8-b1f1-42d1-93c3-5b0b9c3b24ea)
+ - [现场演示](https://www.loom.com/share/6a3385db4e8747a1913dd85a7834846f?sid=a740fd5b-2aa3-457e-8fb7-86976f59f9b8)
+
+## 特性
+- 📝 生成研究问题、大纲、资源和课题报告
+- 🌐 每项研究汇总超过20个网络资源，形成客观和真实的结论
+- 🖥️ 包括易于使用的web界面 (HTML/CSS/JS)
+- 🔍 支持JavaScript网络资源抓取功能
+- 📂 追踪访问过和使用过的网络资源和来源
+- 📄 ���研究报告导出为PDF或其他格式...
+
+## 📖 文档
+
+请参阅[此处](https://docs.gptr.dev/docs/gpt-researcher/getting-started/getting-started)，了解完整文档：
+
+- 入门（安装、设置环境、简单示例）
+- 操作示例（演示、集成、docker 支持）
+- 参考资料（API完整文档）
+- Tavily 应用程序接口集成（核心概念的高级解释）
+
+## 快速开始
+> **步骤 0** - 安装 Python 3.11 或更高版本。[参见此处](https://www.tutorialsteacher.com/python/install-python) 获取详细指南。
+
+<br />
+
+> **步骤 1** - 下载项目
+
+```bash
+$ git clone https://github.com/assafelovic/gpt-researcher.git
+$ cd gpt-researcher
+```
+
+<br />
+
+> **步骤2** -安装依赖项
+```bash
+$ pip install -r requirements.txt
+```
+<br />
+
+> **第 3 步** - 使用 OpenAI 密钥和 Tavily API 密钥创建 .env 文件，或直接导出该文件
+
+```bash
+$ export OPENAI_API_KEY={Your OpenAI API Key here}
+```
+```bash
+$ export TAVILY_API_KEY={Your Tavily API Key here}
+```
+
+- **LLM，我们推荐使用 [OpenAI GPT](https://platform.openai.com/docs/guides/gpt)**，但您也可以使用 [Langchain Adapter](https://python.langchain.com/docs/guides/adapters/openai) 支持的任何其他 LLM 模型（包括开源），只需在 config/config.py 中更改 llm 模型和提供者即可。请按照 [这份指南](https://python.langchain.com/docs/integrations/llms/) 学习如何将 LLM 与 Langchain 集成。
+- **对于搜索引擎，我们推荐使用 [Tavily Search API](https://app.tavily.com)（已针对 LLM 进行优化）**，但您也可以选择其他搜索引擎，只需将 config/config.py 中的搜索提供程序更改为 "duckduckgo"、"googleAPI"、"searchapi"、"googleSerp "或 "searx "即可。然后在 config.py 文件中添加相应的 env API 密钥。
+- **我们强烈建议使用 [OpenAI GPT](https://platform.openai.com/docs/guides/gpt) 模型和 [Tavily Search API](https://app.tavily.com) 以获得最佳性能。**
+<br />
+
+> **第 4 步** - 使用 FastAPI 运行代理
+
+```bash
+$ uvicorn main:app --reload
+```
+<br />
+
+> **第 5 步** - 在任何浏览器上访问 http://localhost:8000，享受研究乐趣！
+
+要了解如何开始使用 Docker 或了解有关功能和服务的更多信息，请访问 [documentation](https://docs.gptr.dev) 页面。
+
+## 🚀 贡献
+我们非常欢迎您的贡献！如果您感兴趣，请查看 [contributing](CONTRIBUTING.md)。
+
+如果您有兴趣加入我们的任务，请查看我们的 [路线图](https://trello.com/b/3O7KBePw/gpt-researcher-roadmap) 页面，并通过我们的 [Discord 社区](https://discord.gg/QgZXvJAccX) 联系我们。
+
+## ✉️ 支持 / 联系我们
+- [社区讨论区](https://discord.gg/spBgZmm3Xe)
+- 我们的邮箱: [email protected]
+
+## 🛡 免责声明
+
+本项目 "GPT Researcher "是一个实验性应用程序，按 "现状 "提供，不做任何明示或暗示的保证。我们根据 MIT 许可分享用于学术目的的代码。本文不提供任何学术建议，也不建议在学术或研究论文中使用。
+
+我们对客观研究主张的看法：
+1.  我们抓取系统的全部目的是减少不正确的事实。如何解决？我们抓取的网站越多，错误数据的可能性就越小。我们每项研究都会收集20条信息，它们全部错误的可能性极低。
+2. 我们的目标不是消除偏见，而是尽可能减少偏见。**作为一个社区，我们在这里探索最有效的人机互动**。
+3. 在研究过程中，人们也容易产生偏见，因为大多数人对自己研究的课题都有自己的看法。这个工具可以搜罗到许多观点，并均匀地解释各种不同的观点，而有偏见的人是绝对读不到这些观点的。
+
+**请注意，使用 GPT-4 语言模型可能会因使用令牌而产生高昂费用**。使用本项目即表示您承认有责任监控和管理自己的令牌使用情况及相关费用。强烈建议您定期检查 OpenAI API 的使用情况，并设置任何必要的限制或警报，以防止发生意外费用。
+
+---
+
+<p align="center">
+<a href="https://star-history.com/#assafelovic/gpt-researcher">
+  <picture>
+    <source media="(prefers-color-scheme: dark)" srcset="https://api.star-history.com/svg?repos=assafelovic/gpt-researcher&type=Date&theme=dark" />
+    <source media="(prefers-color-scheme: light)" srcset="https://api.star-history.com/svg?repos=assafelovic/gpt-researcher&type=Date" />
+    <img alt="Star History Chart" src="https://api.star-history.com/svg?repos=assafelovic/gpt-researcher&type=Date" />
+  </picture>
+</a>
+</p>

README.md

@@ -1,11 +1,231 @@
----
-title: GPT Researcher
-emoji: 👁
-colorFrom: indigo
-colorTo: red
-sdk: static
-pinned: false
-license: apache-2.0
----
-
-Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference
+<div align="center" id="top">
+
+<img src="https://github.com/assafelovic/gpt-researcher/assets/13554167/20af8286-b386-44a5-9a83-3be1365139c3" alt="Logo" width="80">
+
+####
+
+[![Website](https://img.shields.io/badge/Official%20Website-gptr.dev-teal?style=for-the-badge&logo=world&logoColor=white&color=0891b2)](https://gptr.dev)
+[![Documentation](https://img.shields.io/badge/Documentation-DOCS-f472b6?logo=googledocs&logoColor=white&style=for-the-badge)](https://docs.gptr.dev)
+[![Discord Follow](https://dcbadge.vercel.app/api/server/QgZXvJAccX?style=for-the-badge&theme=clean-inverted&?compact=true)](https://discord.gg/QgZXvJAccX)
+
+[![PyPI version](https://img.shields.io/pypi/v/gpt-researcher?logo=pypi&logoColor=white&style=flat)](https://badge.fury.io/py/gpt-researcher)
+![GitHub Release](https://img.shields.io/github/v/release/assafelovic/gpt-researcher?style=flat&logo=github)
+[![Open In Colab](https://img.shields.io/static/v1?message=Open%20in%20Colab&logo=googlecolab&labelColor=grey&color=yellow&label=%20&style=flat&logoSize=40)](https://colab.research.google.com/github/assafelovic/gpt-researcher/blob/master/docs/docs/examples/pip-run.ipynb)
+[![Docker Image Version](https://img.shields.io/docker/v/elestio/gpt-researcher/latest?arch=amd64&style=flat&logo=docker&logoColor=white&color=1D63ED)](https://hub.docker.com/r/gptresearcher/gpt-researcher)
+[![Twitter Follow](https://img.shields.io/twitter/follow/assaf_elovic?style=social)](https://twitter.com/assaf_elovic)
+
+[English](README.md) | [中文](README-zh_CN.md) | [日本語](README-ja_JP.md) | [한국어](README-ko_KR.md)
+
+</div>
+
+# 🔎 GPT Researcher
+
+**GPT Researcher is an autonomous agent designed for comprehensive web and local research on any given task.** 
+
+The agent produces detailed, factual, and unbiased research reports with citations. GPT Researcher provides a full suite of customization options to create tailor made and domain specific research agents. Inspired by the recent [Plan-and-Solve](https://arxiv.org/abs/2305.04091) and [RAG](https://arxiv.org/abs/2005.11401) papers, GPT Researcher addresses misinformation, speed, determinism, and reliability by offering stable performance and increased speed through parallelized agent work.
+
+**Our mission is to empower individuals and organizations with accurate, unbiased, and factual information through AI.**
+
+## Why GPT Researcher?
+
+- Objective conclusions for manual research can take weeks, requiring vast resources and time.
+- LLMs trained on outdated information can hallucinate, becoming irrelevant for current research tasks.
+- Current LLMs have token limitations, insufficient for generating long research reports.
+- Limited web sources in existing services lead to misinformation and shallow results.
+- Selective web sources can introduce bias into research tasks.
+
+## Demo
+https://github.com/user-attachments/assets/2cc38f6a-9f66-4644-9e69-a46c40e296d4
+
+## Architecture
+
+The core idea is to utilize 'planner' and 'execution' agents. The planner generates research questions, while the execution agents gather relevant information. The publisher then aggregates all findings into a comprehensive report.
+
+<div align="center">
+<img align="center" height="600" src="https://github.com/assafelovic/gpt-researcher/assets/13554167/4ac896fd-63ab-4b77-9688-ff62aafcc527">
+</div>
+
+Steps:
+* Create a task-specific agent based on a research query.
+* Generate questions that collectively form an objective opinion on the task.
+* Use a crawler agent for gathering information for each question.
+* Summarize and source-track each resource.
+* Filter and aggregate summaries into a final research report.
+
+## Tutorials
+ - [How it Works](https://docs.gptr.dev/blog/building-gpt-researcher)
+ - [How to Install](https://www.loom.com/share/04ebffb6ed2a4520a27c3e3addcdde20?sid=da1848e8-b1f1-42d1-93c3-5b0b9c3b24ea)
+ - [Live Demo](https://www.loom.com/share/6a3385db4e8747a1913dd85a7834846f?sid=a740fd5b-2aa3-457e-8fb7-86976f59f9b8)
+
+## Features
+
+- 📝 Generate detailed research reports using web and local documents.
+- 🖼️ Smart image scraping and filtering for reports.
+- 📜 Generate detailed reports exceeding 2,000 words.
+- 🌐 Aggregate over 20 sources for objective conclusions.
+- 🖥️ Frontend available in lightweight (HTML/CSS/JS) and production-ready (NextJS + Tailwind) versions.
+- 🔍 JavaScript-enabled web scraping.
+- 📂 Maintains memory and context throughout research.
+- 📄 Export reports to PDF, Word, and other formats.
+
+## 📖 Documentation
+
+See the [Documentation](https://docs.gptr.dev/docs/gpt-researcher/getting-started/getting-started) for:
+- Installation and setup guides
+- Configuration and customization options
+- How-To examples
+- Full API references
+
+## ⚙️ Getting Started
+
+### Installation
+
+1. Install Python 3.11 or later. [Guide](https://www.tutorialsteacher.com/python/install-python).
+2. Clone the project and navigate to the directory:
+
+    ```bash
+    git clone https://github.com/assafelovic/gpt-researcher.git
+    cd gpt-researcher
+    ```
+
+3. Set up API keys by exporting them or storing them in a `.env` file.
+
+    ```bash
+    export OPENAI_API_KEY={Your OpenAI API Key here}
+    export TAVILY_API_KEY={Your Tavily API Key here}
+    ```
+
+4. Install dependencies and start the server:
+
+    ```bash
+    pip install -r requirements.txt
+    python -m uvicorn main:app --reload
+    ```
+
+Visit [http://localhost:8000](http://localhost:8000) to start.
+
+For other setups (e.g., Poetry or virtual environments), check the [Getting Started page](https://docs.gptr.dev/docs/gpt-researcher/getting-started/getting-started).
+
+## Run as PIP package
+```bash
+pip install gpt-researcher
+
+```
+### Example Usage:
+```python
+...
+from gpt_researcher import GPTResearcher
+
+query = "why is Nvidia stock going up?"
+researcher = GPTResearcher(query=query, report_type="research_report")
+# Conduct research on the given query
+research_result = await researcher.conduct_research()
+# Write the report
+report = await researcher.write_report()
+...
+```
+
+**For more examples and configurations, please refer to the [PIP documentation](https://docs.gptr.dev/docs/gpt-researcher/gptr/pip-package) page.**
+
+
+## Run with Docker
+
+> **Step 1** - [Install Docker](https://docs.gptr.dev/docs/gpt-researcher/getting-started/getting-started-with-docker)
+
+> **Step 2** - Clone the '.env.example' file, add your API Keys to the cloned file and save the file as '.env'
+
+> **Step 3** - Within the docker-compose file comment out services that you don't want to run with Docker.
+
+```bash
+docker-compose up --build
+```
+
+If that doesn't work, try running it without the dash:
+```bash
+docker compose up --build
+```
+
+
+> **Step 4** - By default, if you haven't uncommented anything in your docker-compose file, this flow will start 2 processes:
+ - the Python server running on localhost:8000<br>
+ - the React app running on localhost:3000<br>
+
+Visit localhost:3000 on any browser and enjoy researching!
+
+
+
+## 📄 Research on Local Documents
+
+You can instruct the GPT Researcher to run research tasks based on your local documents. Currently supported file formats are: PDF, plain text, CSV, Excel, Markdown, PowerPoint, and Word documents.
+
+Step 1: Add the env variable `DOC_PATH` pointing to the folder where your documents are located.
+
+```bash
+export DOC_PATH="./my-docs"
+```
+
+Step 2: 
+ - If you're running the frontend app on localhost:8000, simply select "My Documents" from the "Report Source" Dropdown Options.
+ - If you're running GPT Researcher with the [PIP package](https://docs.tavily.com/docs/gpt-researcher/pip-package), pass the `report_source` argument as "local" when you instantiate the `GPTResearcher` class [code sample here](https://docs.gptr.dev/docs/gpt-researcher/context/tailored-research).
+
+
+## 👪 Multi-Agent Assistant
+As AI evolves from prompt engineering and RAG to multi-agent systems, we're excited to introduce our new multi-agent assistant built with [LangGraph](https://python.langchain.com/v0.1/docs/langgraph/).
+
+By using LangGraph, the research process can be significantly improved in depth and quality by leveraging multiple agents with specialized skills. Inspired by the recent [STORM](https://arxiv.org/abs/2402.14207) paper, this project showcases how a team of AI agents can work together to conduct research on a given topic, from planning to publication.
+
+An average run generates a 5-6 page research report in multiple formats such as PDF, Docx and Markdown.
+
+Check it out [here](https://github.com/assafelovic/gpt-researcher/tree/master/multi_agents) or head over to our [documentation](https://docs.gptr.dev/docs/gpt-researcher/multi_agents/langgraph) for more information.
+
+## 🖥️ Frontend Applications
+
+GPT-Researcher now features an enhanced frontend to improve the user experience and streamline the research process. The frontend offers:
+
+- An intuitive interface for inputting research queries
+- Real-time progress tracking of research tasks
+- Interactive display of research findings
+- Customizable settings for tailored research experiences
+
+Two deployment options are available:
+1. A lightweight static frontend served by FastAPI
+2. A feature-rich NextJS application for advanced functionality
+
+For detailed setup instructions and more information about the frontend features, please visit our [documentation page](https://docs.gptr.dev/docs/gpt-researcher/frontend/frontend).
+
+## 🚀 Contributing
+We highly welcome contributions! Please check out [contributing](https://github.com/assafelovic/gpt-researcher/blob/master/CONTRIBUTING.md) if you're interested.
+
+Please check out our [roadmap](https://trello.com/b/3O7KBePw/gpt-researcher-roadmap) page and reach out to us via our [Discord community](https://discord.gg/QgZXvJAccX) if you're interested in joining our mission.
+<a href="https://github.com/assafelovic/gpt-researcher/graphs/contributors">
+  <img src="https://contrib.rocks/image?repo=assafelovic/gpt-researcher" />
+</a>
+## ✉️ Support / Contact us
+- [Community Discord](https://discord.gg/spBgZmm3Xe)
+- Author Email: [email protected]
+
+## 🛡 Disclaimer
+
+This project, GPT Researcher, is an experimental application and is provided "as-is" without any warranty, express or implied. We are sharing codes for academic purposes under the Apache 2 license. Nothing herein is academic advice, and NOT a recommendation to use in academic or research papers.
+
+Our view on unbiased research claims:
+1. The main goal of GPT Researcher is to reduce incorrect and biased facts. How? We assume that the more sites we scrape the less chances of incorrect data. By scraping multiple sites per research, and choosing the most frequent information, the chances that they are all wrong is extremely low.
+2. We do not aim to eliminate biases; we aim to reduce it as much as possible. **We are here as a community to figure out the most effective human/llm interactions.**
+3. In research, people also tend towards biases as most have already opinions on the topics they research about. This tool scrapes many opinions and will evenly explain diverse views that a biased person would never have read.
+
+---
+
+<p align="center">
+<a href="https://star-history.com/#assafelovic/gpt-researcher">
+  <picture>
+    <source media="(prefers-color-scheme: dark)" srcset="https://api.star-history.com/svg?repos=assafelovic/gpt-researcher&type=Date&theme=dark" />
+    <source media="(prefers-color-scheme: light)" srcset="https://api.star-history.com/svg?repos=assafelovic/gpt-researcher&type=Date" />
+    <img alt="Star History Chart" src="https://api.star-history.com/svg?repos=assafelovic/gpt-researcher&type=Date" />
+  </picture>
+</a>
+</p>
+
+
+<p align="right">
+  <a href="#top">⬆️ Back to Top</a>
+</p>

backend/__init__.py

@@ -0,0 +1 @@
+from multi_agents import agents

backend/chat/__init__.py

@@ -0,0 +1 @@
+from .chat import ChatAgentWithMemory

backend/chat/chat.py

@@ -0,0 +1,106 @@
+from fastapi import WebSocket
+import uuid
+
+from gpt_researcher.utils.llm import get_llm
+from gpt_researcher.memory import Memory
+from gpt_researcher.config.config import Config
+
+from langgraph.prebuilt import create_react_agent
+from langgraph.checkpoint.memory import MemorySaver
+
+from langchain_community.vectorstores import InMemoryVectorStore
+from langchain.text_splitter import RecursiveCharacterTextSplitter
+from langchain.tools import Tool, tool
+
+class ChatAgentWithMemory:
+    def __init__(
+        self,
+        report: str,
+        config_path,
+        headers,
+        vector_store = None
+    ):
+        self.report = report
+        self.headers = headers
+        self.config = Config(config_path)
+        self.vector_store = vector_store
+        self.graph = self.create_agent()
+
+    def create_agent(self):
+        """Create React Agent Graph"""
+        cfg = Config()
+
+        # Retrieve LLM using get_llm with settings from config
+        provider = get_llm(
+            llm_provider=cfg.smart_llm_provider,
+            model=cfg.smart_llm_model,
+            temperature=0.35,
+            max_tokens=cfg.smart_token_limit,
+            **self.config.llm_kwargs
+        ).llm
+
+        # If vector_store is not initialized, process documents and add to vector_store
+        if not self.vector_store:
+            documents = self._process_document(self.report)
+            self.chat_config = {"configurable": {"thread_id": str(uuid.uuid4())}}
+            self.embedding = Memory(
+                cfg.embedding_provider,
+                cfg.embedding_model,
+                **cfg.embedding_kwargs
+            ).get_embeddings()
+            self.vector_store = InMemoryVectorStore(self.embedding)
+            self.vector_store.add_texts(documents)
+
+        # Create the React Agent Graph with the configured provider
+        graph = create_react_agent(
+            provider,
+            tools=[self.vector_store_tool(self.vector_store)],
+            checkpointer=MemorySaver()
+        )
+        
+        return graph
+    
+    def vector_store_tool(self, vector_store) -> Tool:
+        """Create Vector Store Tool"""
+        @tool 
+        def retrieve_info(query):
+            """
+            Consult the report for relevant contexts whenever you don't know something
+            """
+            retriever = vector_store.as_retriever(k = 4)
+            return retriever.invoke(query)
+        return retrieve_info
+        
+    def _process_document(self, report):
+        """Split Report into Chunks"""
+        text_splitter = RecursiveCharacterTextSplitter(
+            chunk_size=1024,
+            chunk_overlap=20,
+            length_function=len,
+            is_separator_regex=False,
+        )
+        documents = text_splitter.split_text(report)
+        return documents
+
+    async def chat(self, message, websocket):
+        """Chat with React Agent"""
+        message = f"""
+         You are GPT Researcher, a autonomous research agent created by an open source community at https://github.com/assafelovic/gpt-researcher, homepage: https://gptr.dev. 
+         To learn more about GPT Researcher you can suggest to check out: https://docs.gptr.dev.
+         
+         This is a chat message between the user and you: GPT Researcher. 
+         The chat is about a research reports that you created. Answer based on the given context and report.
+         You must include citations to your answer based on the report.
+         
+         Report: {self.report}
+         User Message: {message}
+        """
+        inputs = {"messages": [("user", message)]}
+        response = await self.graph.ainvoke(inputs, config=self.chat_config)
+        ai_message = response["messages"][-1].content
+        if websocket is not None:
+            await websocket.send_json({"type": "chat", "content": ai_message})
+
+    def get_context(self):
+        """return the current context of the chat"""
+        return self.report

backend/memory/draft.py

@@ -0,0 +1,10 @@
+from typing import TypedDict, List, Annotated
+import operator
+
+
+class DraftState(TypedDict):
+    task: dict
+    topic: str
+    draft: dict
+    review: str
+    revision_notes: str

backend/memory/research.py

@@ -0,0 +1,20 @@
+from typing import TypedDict, List, Annotated
+import operator
+
+
+class ResearchState(TypedDict):
+    task: dict
+    initial_research: str
+    sections: List[str]
+    research_data: List[dict]
+    # Report layout
+    title: str
+    headers: dict
+    date: str
+    table_of_contents: str
+    introduction: str
+    conclusion: str
+    sources: List[str]
+    report: str
+
+

backend/report_type/__init__.py

@@ -0,0 +1,7 @@
+from .basic_report.basic_report import BasicReport
+from .detailed_report.detailed_report import DetailedReport
+
+__all__ = [
+    "BasicReport",
+    "DetailedReport"
+]

backend/report_type/basic_report/basic_report.py

@@ -0,0 +1,46 @@
+from fastapi import WebSocket
+from typing import Any
+
+from gpt_researcher import GPTResearcher
+
+
+class BasicReport:
+    def __init__(
+        self,
+        query: str,
+        report_type: str,
+        report_source: str,
+        source_urls,
+        document_urls,
+        tone: Any,
+        config_path: str,
+        websocket: WebSocket,
+        headers=None
+    ):
+        self.query = query
+        self.report_type = report_type
+        self.report_source = report_source
+        self.source_urls = source_urls
+        self.document_urls = document_urls
+        self.tone = tone
+        self.config_path = config_path
+        self.websocket = websocket
+        self.headers = headers or {}
+
+    async def run(self):
+        # Initialize researcher
+        researcher = GPTResearcher(
+            query=self.query,
+            report_type=self.report_type,
+            report_source=self.report_source,
+            source_urls=self.source_urls,
+            document_urls=self.document_urls,
+            tone=self.tone,
+            config_path=self.config_path,
+            websocket=self.websocket,
+            headers=self.headers
+        )
+
+        await researcher.conduct_research()
+        report = await researcher.write_report()
+        return report

backend/report_type/detailed_report/README.md

@@ -0,0 +1,12 @@
+## Detailed Reports
+
+Introducing long and detailed reports, with a completely new architecture inspired by the latest [STORM](https://arxiv.org/abs/2402.14207) paper.
+
+In this method we do the following:
+
+1. Trigger Initial GPT Researcher report based on task
+2. Generate subtopics from research summary
+3. For each subtopic the headers of the subtopic report are extracted and accumulated
+4. For each subtopic a report is generated making sure that any information about the headers accumulated until now are not re-generated.
+5. An additional introduction section is written along with a table of contents constructed from the entire report.
+6. The final report is constructed by appending these : Intro + Table of contents + Subsection reports

backend/report_type/detailed_report/detailed_report.py

@@ -0,0 +1,139 @@
+import asyncio
+from typing import List, Dict, Set, Optional, Any
+from fastapi import WebSocket
+
+from gpt_researcher import GPTResearcher
+
+
+class DetailedReport:
+    def __init__(
+        self,
+        query: str,
+        report_type: str,
+        report_source: str,
+        source_urls: List[str] = [],
+        document_urls: List[str] = [],
+        config_path: str = None,
+        tone: Any = "",
+        websocket: WebSocket = None,
+        subtopics: List[Dict] = [],
+        headers: Optional[Dict] = None
+    ):
+        self.query = query
+        self.report_type = report_type
+        self.report_source = report_source
+        self.source_urls = source_urls
+        self.document_urls = document_urls
+        self.config_path = config_path
+        self.tone = tone
+        self.websocket = websocket
+        self.subtopics = subtopics
+        self.headers = headers or {}
+
+        self.gpt_researcher = GPTResearcher(
+            query=self.query,
+            report_type="research_report",
+            report_source=self.report_source,
+            source_urls=self.source_urls,
+            document_urls=self.document_urls,
+            config_path=self.config_path,
+            tone=self.tone,
+            websocket=self.websocket,
+            headers=self.headers
+        )
+        self.existing_headers: List[Dict] = []
+        self.global_context: List[str] = []
+        self.global_written_sections: List[str] = []
+        self.global_urls: Set[str] = set(
+            self.source_urls) if self.source_urls else set()
+
+    async def run(self) -> str:
+        await self._initial_research()
+        subtopics = await self._get_all_subtopics()
+        report_introduction = await self.gpt_researcher.write_introduction()
+        _, report_body = await self._generate_subtopic_reports(subtopics)
+        self.gpt_researcher.visited_urls.update(self.global_urls)
+        report = await self._construct_detailed_report(report_introduction, report_body)
+        return report
+
+    async def _initial_research(self) -> None:
+        await self.gpt_researcher.conduct_research()
+        self.global_context = self.gpt_researcher.context
+        self.global_urls = self.gpt_researcher.visited_urls
+
+    async def _get_all_subtopics(self) -> List[Dict]:
+        subtopics_data = await self.gpt_researcher.get_subtopics()
+
+        all_subtopics = []
+        if subtopics_data and subtopics_data.subtopics:
+            for subtopic in subtopics_data.subtopics:
+                all_subtopics.append({"task": subtopic.task})
+        else:
+            print(f"Unexpected subtopics data format: {subtopics_data}")
+
+        return all_subtopics
+
+    async def _generate_subtopic_reports(self, subtopics: List[Dict]) -> tuple:
+        subtopic_reports = []
+        subtopics_report_body = ""
+
+        for subtopic in subtopics:
+            result = await self._get_subtopic_report(subtopic)
+            if result["report"]:
+                subtopic_reports.append(result)
+                subtopics_report_body += f"\n\n\n{result['report']}"
+
+        return subtopic_reports, subtopics_report_body
+
+    async def _get_subtopic_report(self, subtopic: Dict) -> Dict[str, str]:
+        current_subtopic_task = subtopic.get("task")
+        subtopic_assistant = GPTResearcher(
+            query=current_subtopic_task,
+            report_type="subtopic_report",
+            report_source=self.report_source,
+            websocket=self.websocket,
+            headers=self.headers,
+            parent_query=self.query,
+            subtopics=self.subtopics,
+            visited_urls=self.global_urls,
+            agent=self.gpt_researcher.agent,
+            role=self.gpt_researcher.role,
+            tone=self.tone,
+        )
+
+        subtopic_assistant.context = list(set(self.global_context))
+        await subtopic_assistant.conduct_research()
+
+        draft_section_titles = await subtopic_assistant.get_draft_section_titles(current_subtopic_task)
+
+        if not isinstance(draft_section_titles, str):
+            draft_section_titles = str(draft_section_titles)
+
+        parse_draft_section_titles = self.gpt_researcher.extract_headers(draft_section_titles)
+        parse_draft_section_titles_text = [header.get(
+            "text", "") for header in parse_draft_section_titles]
+
+        relevant_contents = await subtopic_assistant.get_similar_written_contents_by_draft_section_titles(
+            current_subtopic_task, parse_draft_section_titles_text, self.global_written_sections
+        )
+
+        subtopic_report = await subtopic_assistant.write_report(self.existing_headers, relevant_contents)
+
+        self.global_written_sections.extend(self.gpt_researcher.extract_sections(subtopic_report))
+        self.global_context = list(set(subtopic_assistant.context))
+        self.global_urls.update(subtopic_assistant.visited_urls)
+
+        self.existing_headers.append({
+            "subtopic task": current_subtopic_task,
+            "headers": self.gpt_researcher.extract_headers(subtopic_report),
+        })
+
+        return {"topic": subtopic, "report": subtopic_report}
+
+    async def _construct_detailed_report(self, introduction: str, report_body: str) -> str:
+        toc = self.gpt_researcher.table_of_contents(report_body)
+        conclusion = await self.gpt_researcher.write_report_conclusion(report_body)
+        conclusion_with_references = self.gpt_researcher.add_references(
+            conclusion, self.gpt_researcher.visited_urls)
+        report = f"{introduction}\n\n{toc}\n\n{report_body}\n\n{conclusion_with_references}"
+        return report

backend/server/app.py

@@ -0,0 +1,16 @@
+from fastapi import FastAPI
+from fastapi.middleware.cors import CORSMiddleware
+import logging
+
+logger = logging.getLogger(__name__)
+
+app = FastAPI()
+
+# Add CORS middleware
+app.add_middleware(
+    CORSMiddleware,
+    allow_origins=["*"],  # In production, replace with your frontend domain
+    allow_credentials=True,
+    allow_methods=["*"],
+    allow_headers=["*"],
+)

backend/server/logging_config.py

@@ -0,0 +1,83 @@
+import logging
+import json
+import os
+from datetime import datetime
+from pathlib import Path
+
+class JSONResearchHandler:
+    def __init__(self, json_file):
+        self.json_file = json_file
+        self.research_data = {
+            "timestamp": datetime.now().isoformat(),
+            "events": [],
+            "content": {
+                "query": "",
+                "sources": [],
+                "context": [],
+                "report": "",
+                "costs": 0.0
+            }
+        }
+
+    def log_event(self, event_type: str, data: dict):
+        self.research_data["events"].append({
+            "timestamp": datetime.now().isoformat(),
+            "type": event_type,
+            "data": data
+        })
+        self._save_json()
+
+    def update_content(self, key: str, value):
+        self.research_data["content"][key] = value
+        self._save_json()
+
+    def _save_json(self):
+        with open(self.json_file, 'w') as f:
+            json.dump(self.research_data, f, indent=2)
+
+def setup_research_logging():
+    # Create logs directory if it doesn't exist
+    logs_dir = Path("logs")
+    logs_dir.mkdir(exist_ok=True)
+    
+    # Generate timestamp for log files
+    timestamp = datetime.now().strftime("%Y%m%d_%H%M%S")
+    
+    # Create log file paths
+    log_file = logs_dir / f"research_{timestamp}.log"
+    json_file = logs_dir / f"research_{timestamp}.json"
+    
+    # Configure file handler for research logs
+    file_handler = logging.FileHandler(log_file)
+    file_handler.setLevel(logging.INFO)
+    file_handler.setFormatter(logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s'))
+    
+    # Get research logger and configure it
+    research_logger = logging.getLogger('research')
+    research_logger.setLevel(logging.INFO)
+    
+    # Remove any existing handlers to avoid duplicates
+    research_logger.handlers.clear()
+    
+    # Add file handler
+    research_logger.addHandler(file_handler)
+    
+    # Add stream handler for console output
+    console_handler = logging.StreamHandler()
+    console_handler.setFormatter(logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s'))
+    research_logger.addHandler(console_handler)
+    
+    # Prevent propagation to root logger to avoid duplicate logs
+    research_logger.propagate = False
+    
+    # Create JSON handler
+    json_handler = JSONResearchHandler(json_file)
+    
+    return str(log_file), str(json_file), research_logger, json_handler
+
+# Create a function to get the logger and JSON handler
+def get_research_logger():
+    return logging.getLogger('research')
+
+def get_json_handler():
+    return getattr(logging.getLogger('research'), 'json_handler', None)

backend/server/server.py

@@ -0,0 +1,134 @@
+import json
+import os
+from typing import Dict, List
+
+from fastapi import FastAPI, Request, WebSocket, WebSocketDisconnect, File, UploadFile, Header
+from fastapi.middleware.cors import CORSMiddleware
+from fastapi.staticfiles import StaticFiles
+from fastapi.templating import Jinja2Templates
+from pydantic import BaseModel
+
+from backend.server.websocket_manager import WebSocketManager
+from backend.server.server_utils import (
+    get_config_dict,
+    update_environment_variables, handle_file_upload, handle_file_deletion,
+    execute_multi_agents, handle_websocket_communication
+)
+
+
+from gpt_researcher.utils.logging_config import setup_research_logging
+
+import logging
+
+# Get logger instance
+logger = logging.getLogger(__name__)
+
+# Don't override parent logger settings
+logger.propagate = True
+
+logging.basicConfig(
+    level=logging.INFO,
+    format="%(asctime)s - %(levelname)s - %(message)s",
+    handlers=[
+        logging.StreamHandler()  # Only log to console
+    ]
+)
+
+# Models
+
+
+class ResearchRequest(BaseModel):
+    task: str
+    report_type: str
+    agent: str
+
+
+class ConfigRequest(BaseModel):
+    ANTHROPIC_API_KEY: str
+    TAVILY_API_KEY: str
+    LANGCHAIN_TRACING_V2: str
+    LANGCHAIN_API_KEY: str
+    OPENAI_API_KEY: str
+    DOC_PATH: str
+    RETRIEVER: str
+    GOOGLE_API_KEY: str = ''
+    GOOGLE_CX_KEY: str = ''
+    BING_API_KEY: str = ''
+    SEARCHAPI_API_KEY: str = ''
+    SERPAPI_API_KEY: str = ''
+    SERPER_API_KEY: str = ''
+    SEARX_URL: str = ''
+    XAI_API_KEY: str
+    DEEPSEEK_API_KEY: str
+
+
+# App initialization
+app = FastAPI()
+
+# Static files and templates
+app.mount("/site", StaticFiles(directory="./frontend"), name="site")
+app.mount("/static", StaticFiles(directory="./frontend/static"), name="static")
+templates = Jinja2Templates(directory="./frontend")
+
+# WebSocket manager
+manager = WebSocketManager()
+
+# Middleware
+app.add_middleware(
+    CORSMiddleware,
+    allow_origins=["http://localhost:3000"],
+    allow_credentials=True,
+    allow_methods=["*"],
+    allow_headers=["*"],
+)
+
+# Constants
+DOC_PATH = os.getenv("DOC_PATH", "./my-docs")
+
+# Startup event
+
+
+@app.on_event("startup")
+def startup_event():
+    os.makedirs("outputs", exist_ok=True)
+    app.mount("/outputs", StaticFiles(directory="outputs"), name="outputs")
+    os.makedirs(DOC_PATH, exist_ok=True)
+    
+
+# Routes
+
+
+@app.get("/")
+async def read_root(request: Request):
+    return templates.TemplateResponse("index.html", {"request": request, "report": None})
+
+
+@app.get("/files/")
+async def list_files():
+    files = os.listdir(DOC_PATH)
+    print(f"Files in {DOC_PATH}: {files}")
+    return {"files": files}
+
+
+@app.post("/api/multi_agents")
+async def run_multi_agents():
+    return await execute_multi_agents(manager)
+
+
+@app.post("/upload/")
+async def upload_file(file: UploadFile = File(...)):
+    return await handle_file_upload(file, DOC_PATH)
+
+
+@app.delete("/files/{filename}")
+async def delete_file(filename: str):
+    return await handle_file_deletion(filename, DOC_PATH)
+
+
+@app.websocket("/ws")
+async def websocket_endpoint(websocket: WebSocket):
+    await manager.connect(websocket)
+    try:
+        await handle_websocket_communication(websocket, manager)
+    except WebSocketDisconnect:
+        await manager.disconnect(websocket)

backend/server/server_utils.py

@@ -0,0 +1,259 @@
+import json
+import os
+import re
+import time
+import shutil
+from typing import Dict, List, Any
+from fastapi.responses import JSONResponse, FileResponse
+from gpt_researcher.document.document import DocumentLoader
+from backend.utils import write_md_to_pdf, write_md_to_word, write_text_to_md
+from pathlib import Path
+from datetime import datetime
+from fastapi import HTTPException
+import logging
+
+logging.basicConfig(level=logging.DEBUG)
+logger = logging.getLogger(__name__)
+
+class CustomLogsHandler:
+    """Custom handler to capture streaming logs from the research process"""
+    def __init__(self, websocket, task: str):
+        self.logs = []
+        self.websocket = websocket
+        sanitized_filename = sanitize_filename(f"task_{int(time.time())}_{task}")
+        self.log_file = os.path.join("outputs", f"{sanitized_filename}.json")
+        self.timestamp = datetime.now().isoformat()
+        # Initialize log file with metadata
+        os.makedirs("outputs", exist_ok=True)
+        with open(self.log_file, 'w') as f:
+            json.dump({
+                "timestamp": self.timestamp,
+                "events": [],
+                "content": {
+                    "query": "",
+                    "sources": [],
+                    "context": [],
+                    "report": "",
+                    "costs": 0.0
+                }
+            }, f, indent=2)
+
+    async def send_json(self, data: Dict[str, Any]) -> None:
+        """Store log data and send to websocket"""
+        # Send to websocket for real-time display
+        if self.websocket:
+            await self.websocket.send_json(data)
+            
+        # Read current log file
+        with open(self.log_file, 'r') as f:
+            log_data = json.load(f)
+            
+        # Update appropriate section based on data type
+        if data.get('type') == 'logs':
+            log_data['events'].append({
+                "timestamp": datetime.now().isoformat(),
+                "type": "event",
+                "data": data
+            })
+        else:
+            # Update content section for other types of data
+            log_data['content'].update(data)
+            
+        # Save updated log file
+        with open(self.log_file, 'w') as f:
+            json.dump(log_data, f, indent=2)
+        logger.debug(f"Log entry written to: {self.log_file}")
+
+
+class Researcher:
+    def __init__(self, query: str, report_type: str = "research_report"):
+        self.query = query
+        self.report_type = report_type
+        # Generate unique ID for this research task
+        self.research_id = f"{datetime.now().strftime('%Y%m%d_%H%M%S')}_{hash(query)}"
+        # Initialize logs handler with research ID
+        self.logs_handler = CustomLogsHandler(self.research_id)
+        self.researcher = GPTResearcher(
+            query=query,
+            report_type=report_type,
+            websocket=self.logs_handler
+        )
+
+    async def research(self) -> dict:
+        """Conduct research and return paths to generated files"""
+        await self.researcher.conduct_research()
+        report = await self.researcher.write_report()
+        
+        # Generate the files
+        sanitized_filename = sanitize_filename(f"task_{int(time.time())}_{self.query}")
+        file_paths = await generate_report_files(report, sanitized_filename)
+        
+        # Get the JSON log path that was created by CustomLogsHandler
+        json_relative_path = os.path.relpath(self.logs_handler.log_file)
+        
+        return {
+            "output": {
+                **file_paths,  # Include PDF, DOCX, and MD paths
+                "json": json_relative_path
+            }
+        }
+
+def sanitize_filename(filename: str) -> str:
+    # Split into components
+    prefix, timestamp, *task_parts = filename.split('_')
+    task = '_'.join(task_parts)
+    
+    # Calculate max length for task portion
+    # 255 - len("outputs/") - len("task_") - len(timestamp) - len("_.json") - safety_margin
+    max_task_length = 255 - 8 - 5 - 10 - 6 - 10  # ~216 chars for task
+    
+    # Truncate task if needed
+    truncated_task = task[:max_task_length] if len(task) > max_task_length else task
+    
+    # Reassemble and clean the filename
+    sanitized = f"{prefix}_{timestamp}_{truncated_task}"
+    return re.sub(r"[^\w\s-]", "", sanitized).strip()
+
+
+async def handle_start_command(websocket, data: str, manager):
+    json_data = json.loads(data[6:])
+    task, report_type, source_urls, document_urls, tone, headers, report_source = extract_command_data(
+        json_data)
+
+    if not task or not report_type:
+        print("Error: Missing task or report_type")
+        return
+
+    # Create logs handler with websocket and task
+    logs_handler = CustomLogsHandler(websocket, task)
+    # Initialize log content with query
+    await logs_handler.send_json({
+        "query": task,
+        "sources": [],
+        "context": [],
+        "report": ""
+    })
+
+    sanitized_filename = sanitize_filename(f"task_{int(time.time())}_{task}")
+
+    report = await manager.start_streaming(
+        task, 
+        report_type, 
+        report_source, 
+        source_urls, 
+        document_urls,
+        tone, 
+        websocket,
+        headers
+    )
+    report = str(report)
+    file_paths = await generate_report_files(report, sanitized_filename)
+    # Add JSON log path to file_paths
+    file_paths["json"] = os.path.relpath(logs_handler.log_file)
+    await send_file_paths(websocket, file_paths)
+
+
+async def handle_human_feedback(data: str):
+    feedback_data = json.loads(data[14:])  # Remove "human_feedback" prefix
+    print(f"Received human feedback: {feedback_data}")
+    # TODO: Add logic to forward the feedback to the appropriate agent or update the research state
+
+async def handle_chat(websocket, data: str, manager):
+    json_data = json.loads(data[4:])
+    print(f"Received chat message: {json_data.get('message')}")
+    await manager.chat(json_data.get("message"), websocket)
+
+async def generate_report_files(report: str, filename: str) -> Dict[str, str]:
+    pdf_path = await write_md_to_pdf(report, filename)
+    docx_path = await write_md_to_word(report, filename)
+    md_path = await write_text_to_md(report, filename)
+    return {"pdf": pdf_path, "docx": docx_path, "md": md_path}
+
+
+async def send_file_paths(websocket, file_paths: Dict[str, str]):
+    await websocket.send_json({"type": "path", "output": file_paths})
+
+
+def get_config_dict(
+    langchain_api_key: str, openai_api_key: str, tavily_api_key: str,
+    google_api_key: str, google_cx_key: str, bing_api_key: str,
+    searchapi_api_key: str, serpapi_api_key: str, serper_api_key: str, searx_url: str
+) -> Dict[str, str]:
+    return {
+        "LANGCHAIN_API_KEY": langchain_api_key or os.getenv("LANGCHAIN_API_KEY", ""),
+        "OPENAI_API_KEY": openai_api_key or os.getenv("OPENAI_API_KEY", ""),
+        "TAVILY_API_KEY": tavily_api_key or os.getenv("TAVILY_API_KEY", ""),
+        "GOOGLE_API_KEY": google_api_key or os.getenv("GOOGLE_API_KEY", ""),
+        "GOOGLE_CX_KEY": google_cx_key or os.getenv("GOOGLE_CX_KEY", ""),
+        "BING_API_KEY": bing_api_key or os.getenv("BING_API_KEY", ""),
+        "SEARCHAPI_API_KEY": searchapi_api_key or os.getenv("SEARCHAPI_API_KEY", ""),
+        "SERPAPI_API_KEY": serpapi_api_key or os.getenv("SERPAPI_API_KEY", ""),
+        "SERPER_API_KEY": serper_api_key or os.getenv("SERPER_API_KEY", ""),
+        "SEARX_URL": searx_url or os.getenv("SEARX_URL", ""),
+        "LANGCHAIN_TRACING_V2": os.getenv("LANGCHAIN_TRACING_V2", "true"),
+        "DOC_PATH": os.getenv("DOC_PATH", "./my-docs"),
+        "RETRIEVER": os.getenv("RETRIEVER", ""),
+        "EMBEDDING_MODEL": os.getenv("OPENAI_EMBEDDING_MODEL", "")
+    }
+
+
+def update_environment_variables(config: Dict[str, str]):
+    for key, value in config.items():
+        os.environ[key] = value
+
+
+async def handle_file_upload(file, DOC_PATH: str) -> Dict[str, str]:
+    file_path = os.path.join(DOC_PATH, os.path.basename(file.filename))
+    with open(file_path, "wb") as buffer:
+        shutil.copyfileobj(file.file, buffer)
+    print(f"File uploaded to {file_path}")
+
+    document_loader = DocumentLoader(DOC_PATH)
+    await document_loader.load()
+
+    return {"filename": file.filename, "path": file_path}
+
+
+async def handle_file_deletion(filename: str, DOC_PATH: str) -> JSONResponse:
+    file_path = os.path.join(DOC_PATH, os.path.basename(filename))
+    if os.path.exists(file_path):
+        os.remove(file_path)
+        print(f"File deleted: {file_path}")
+        return JSONResponse(content={"message": "File deleted successfully"})
+    else:
+        print(f"File not found: {file_path}")
+        return JSONResponse(status_code=404, content={"message": "File not found"})
+
+
+async def execute_multi_agents(manager) -> Any:
+    websocket = manager.active_connections[0] if manager.active_connections else None
+    if websocket:
+        report = await run_research_task("Is AI in a hype cycle?", websocket, stream_output)
+        return {"report": report}
+    else:
+        return JSONResponse(status_code=400, content={"message": "No active WebSocket connection"})
+
+
+async def handle_websocket_communication(websocket, manager):
+    while True:
+        data = await websocket.receive_text()
+        if data.startswith("start"):
+            await handle_start_command(websocket, data, manager)
+        elif data.startswith("human_feedback"):
+            await handle_human_feedback(data)
+        elif data.startswith("chat"):
+            await handle_chat(websocket, data, manager)
+        else:
+            print("Error: Unknown command or not enough parameters provided.")
+
+
+def extract_command_data(json_data: Dict) -> tuple:
+    return (
+        json_data.get("task"),
+        json_data.get("report_type"),
+        json_data.get("source_urls"),
+        json_data.get("document_urls"),
+        json_data.get("tone"),
+        json_data.get("headers", {}),
+        json_data.get("report_source")
+    )