# DAG Visualization for QA_LLM_Module ## Overview This enhancement adds DAG (Directed Acyclic Graph) visualization capability to the QA_LLM_Module, allowing users to see visual representations of robot task dependencies generated by the LLM. ## Features - **Automatic DAG Generation**: Visualizations are generated automatically when the LLM outputs valid task JSON - **IEEE-Style Formatting**: Professional, publication-ready graphs - **Hierarchical Layout**: Tasks are organized by dependency levels - **Color-Coded Nodes**: - 🔴 Red: Start tasks (no dependencies) - 🟣 Purple: End tasks (no dependents) - 🟠 Orange: Intermediate tasks - **Detailed Information**: Each task shows function name, assigned robots, and object keywords - **Two Visualization Modes**: Detailed and simplified versions - **🔒 Safety Confirmation Workflow**: - Task plans require operator approval before deployment - "Validate & Deploy Task Plan" button for safety compliance - Visual status updates (Pending → Approved & Deployed) - Prevents accidental execution of unverified plans ## Installation The required dependencies are already added to `requirements.txt`: ```bash pip install matplotlib==3.9.4 pip install networkx==3.4 ``` ## Usage ### Automatic Integration with Safety Workflow When you run the main application and input a query that generates task JSON: 1. **DAG Generation**: The visualization appears automatically with "Pending Approval" status 2. **Safety Review**: Operator reviews the task dependency graph 3. **Confirmation**: Click "🔒 Validate & Deploy Task Plan" to send to construction site 4. **Deployment**: Graph updates to "APPROVED & DEPLOYED" status ```bash python3 main.py ``` ### Enhanced Safety Workflow Steps 1. 🤖 **LLM generates task plan** → DAG shows "Pending Approval" 2. 👨‍💼 **Operator reviews visualization** → Confirms task safety and correctness 3. 📝 **[Optional] Edit Task Plan** → Manual JSON editing if modifications needed 4. 🔄 **[Optional] Update DAG Visualization** → Regenerate graph with edits 5. 🔒 **Validate & Deploy Task Plan** → Final approval and deployment to robots 6. ✅ **Confirmation displayed** → DAG shows "APPROVED & DEPLOYED" ### Three-Button IEEE-Style Workflow - **📝 Edit Task Plan**: Opens JSON editor for manual modifications - **🔄 Update DAG Visualization**: Regenerates graph from edited JSON - **🔒 Validate & Deploy Task Plan**: Final safety approval and deployment ### Manual Testing You can test the DAG visualization independently: ```bash python3 test_dag_visualization.py python3 dag_demo.py ``` ## API Reference ### DAGVisualizer Class ```python from dag_visualizer import DAGVisualizer visualizer = DAGVisualizer() ``` #### Methods ##### `create_dag_visualization(task_data, title="Robot Task Dependency Graph")` Creates a detailed DAG visualization with full task information. **Parameters:** - `task_data` (dict): Task data in the expected JSON format - `title` (str): Title for the graph **Returns:** - `str`: Path to the generated PNG image, or `None` if failed ##### `create_simplified_dag_visualization(task_data, title="Robot Task Graph")` Creates a simplified DAG visualization suitable for smaller displays. **Parameters:** - `task_data` (dict): Task data in the expected JSON format - `title` (str): Title for the graph **Returns:** - `str`: Path to the generated PNG image, or `None` if failed ## Expected JSON Format The visualizer expects task data in the following format: ```json { "tasks": [ { "task": "task_name", "instruction_function": { "name": "function_name", "robot_ids": ["robot_id_1", "robot_id_2"], "dependencies": ["dependency_task_name"], "object_keywords": ["object1", "object2"] } } ] } ``` ## Examples ### Single Task ```json { "tasks": [ { "task": "target_area_for_specific_robots_1", "instruction_function": { "name": "target_area_for_specific_robots", "robot_ids": ["robot_dump_truck_01"], "dependencies": [], "object_keywords": ["puddle1"] } } ] } ``` ### Multiple Tasks with Dependencies ```json { "tasks": [ { "task": "excavate_soil", "instruction_function": { "name": "excavate_soil_from_pile", "robot_ids": ["robot_excavator_01"], "dependencies": [], "object_keywords": ["soil_pile"] } }, { "task": "transport_soil", "instruction_function": { "name": "transport_soil_to_pit", "robot_ids": ["robot_dump_truck_01"], "dependencies": ["excavate_soil"], "object_keywords": ["pit"] } } ] } ``` ## File Structure ``` QA_LLM_Module/ ├── dag_visualizer.py # Main visualization module ├── test_dag_visualization.py # Test suite ├── dag_demo.py # Demo script ├── gradio_llm_interface.py # Updated with DAG integration ├── main.py # Updated Gradio interface └── requirements.txt # Updated dependencies ``` ## Integration Details ### Modified Files 1. **gradio_llm_interface.py**: - Added `DAGVisualizer` import - Modified `predict()` method to generate visualizations - Returns additional DAG image output 2. **main.py**: - Added `gr.Image` component for DAG display - Updated input/output mappings for DAG visualization 3. **requirements.txt**: - Added `matplotlib==3.9.4` - Added `networkx==3.4` ## Troubleshooting ### Common Issues 1. **"No module named 'matplotlib'"** - Solution: `pip install matplotlib networkx` 2. **"No tasks found or invalid graph structure"** - Solution: Ensure your JSON follows the expected format 3. **Images not displaying in Gradio** - Solution: Check that the image paths are accessible and files exist ### Debug Mode Enable detailed logging by setting the log level: ```python from loguru import logger logger.add("dag_debug.log", level="DEBUG") ``` ## Performance Notes - Visualizations are generated in temporary directories - Images are automatically cleaned up by the system - Large graphs (>20 nodes) may take longer to render - Memory usage scales with graph complexity ## Future Enhancements Potential improvements for future versions: 1. **Interactive Graphs**: Clickable nodes with detailed information 2. **Real-time Updates**: Live visualization updates as tasks execute 3. **Export Options**: Support for PDF, SVG, and other formats 4. **Custom Styling**: User-configurable colors and layouts 5. **Animation**: Animated execution flow visualization