feat: add Inventory Scanner for extracting item icons and stats
- New module: modules/inventory_scanner.py - Detects inventory and item details windows - Extracts item icons from inventory grid - Reads item stats from details panel - Parses weapon, armor, and common stats - Handles scrolling (planned for future) - New dialog: ui/inventory_scanner_dialog.py - Visual interface for scanning - Shows extracted icons in grid view - Displays item stats in table - Saves results to JSON - Background worker for non-blocking scans - Updated main_window.py: - Added Tools → Computer Vision → Inventory Scanner (Ctrl+I) - Integrated with existing GameVisionAI This allows users to extract item data from Entropia Universe for gear management and loadout configuration.
This commit is contained in:
LemonNexus
parent
27b3bd0fe1
commit
1a0c0d4231
|
|
@ -0,0 +1,290 @@
|
|||
# Lemontropia Suite - OCR System Implementation Summary
|
||||
|
||||
## Overview
|
||||
Implemented a **robust multi-backend OCR system** that handles PyTorch DLL errors on Windows Store Python and provides graceful fallbacks to working backends.
|
||||
|
||||
## Problem Solved
|
||||
- **PyTorch fails to load c10.dll on Windows Store Python 3.13**
|
||||
- PaddleOCR requires PyTorch which causes DLL errors
|
||||
- Need working OCR for game text detection without breaking dependencies
|
||||
|
||||
## Solution Architecture
|
||||
|
||||
### 1. OCR Backends (Priority Order)
|
||||
|
||||
| Backend | File | Speed | Accuracy | Dependencies | Windows Store Python |
|
||||
|---------|------|-------|----------|--------------|---------------------|
|
||||
| **OpenCV EAST** | `opencv_east_backend.py` | ⚡ Fastest | Detection only | None | ✅ Works |
|
||||
| **EasyOCR** | `easyocr_backend.py` | 🚀 Fast | ⭐⭐⭐ Good | PyTorch | ❌ May fail |
|
||||
| **Tesseract** | `tesseract_backend.py` | 🐢 Slow | ⭐⭐ Medium | Tesseract binary | ✅ Works |
|
||||
| **PaddleOCR** | `paddleocr_backend.py` | 🚀 Fast | ⭐⭐⭐⭐⭐ Best | PaddlePaddle | ❌ May fail |
|
||||
|
||||
### 2. Hardware Detection
|
||||
|
||||
**File**: `modules/hardware_detection.py`
|
||||
|
||||
- Detects GPU availability (CUDA, MPS, DirectML)
|
||||
- Detects PyTorch with **safe error handling for DLL errors**
|
||||
- Detects Windows Store Python
|
||||
- Recommends best OCR backend based on hardware
|
||||
|
||||
### 3. Unified OCR Interface
|
||||
|
||||
**File**: `modules/game_vision_ai.py` (updated)
|
||||
|
||||
- `UnifiedOCRProcessor` - Main OCR interface
|
||||
- Auto-selects best available backend
|
||||
- Graceful fallback chain
|
||||
- Backend switching at runtime
|
||||
|
||||
## Files Created/Modified
|
||||
|
||||
### New Files
|
||||
|
||||
```
|
||||
modules/
|
||||
├── __init__.py # Module exports
|
||||
├── hardware_detection.py # GPU/ML framework detection
|
||||
└── ocr_backends/
|
||||
├── __init__.py # Backend factory and base classes
|
||||
├── opencv_east_backend.py # OpenCV EAST text detector
|
||||
├── easyocr_backend.py # EasyOCR backend
|
||||
├── tesseract_backend.py # Tesseract OCR backend
|
||||
└── paddleocr_backend.py # PaddleOCR backend with DLL handling
|
||||
|
||||
test_ocr_system.py # Comprehensive test suite
|
||||
demo_ocr.py # Interactive demo
|
||||
requirements-ocr.txt # OCR dependencies
|
||||
OCR_SETUP.md # Setup guide
|
||||
```
|
||||
|
||||
### Modified Files
|
||||
|
||||
```
|
||||
modules/
|
||||
└── game_vision_ai.py # Updated to use unified OCR interface
|
||||
|
||||
vision_example.py # Updated examples
|
||||
```
|
||||
|
||||
## Key Features
|
||||
|
||||
### 1. PyTorch DLL Error Handling
|
||||
|
||||
```python
|
||||
# The system detects and handles PyTorch DLL errors gracefully
|
||||
try:
|
||||
import torch
|
||||
# If this fails with DLL error on Windows Store Python...
|
||||
except OSError as e:
|
||||
if 'dll' in str(e).lower() or 'c10' in str(e).lower():
|
||||
# Automatically use fallback backends
|
||||
logger.warning("PyTorch DLL error - using fallback OCR")
|
||||
```
|
||||
|
||||
### 2. Auto-Selection Logic
|
||||
|
||||
```python
|
||||
# Priority order (skips PyTorch-based if DLL error detected)
|
||||
DEFAULT_PRIORITY = [
|
||||
'paddleocr', # Best accuracy (if PyTorch works)
|
||||
'easyocr', # Good balance (if PyTorch works)
|
||||
'tesseract', # Stable fallback
|
||||
'opencv_east', # Always works
|
||||
]
|
||||
```
|
||||
|
||||
### 3. Simple Usage
|
||||
|
||||
```python
|
||||
from modules.game_vision_ai import GameVisionAI
|
||||
|
||||
# Initialize (auto-selects best backend)
|
||||
vision = GameVisionAI()
|
||||
|
||||
# Process screenshot
|
||||
result = vision.process_screenshot("game_screenshot.png")
|
||||
|
||||
print(f"Backend: {result.ocr_backend}")
|
||||
print(f"Text regions: {len(result.text_regions)}")
|
||||
```
|
||||
|
||||
### 4. Backend Diagnostics
|
||||
|
||||
```python
|
||||
from modules.game_vision_ai import GameVisionAI
|
||||
|
||||
# Run diagnostics
|
||||
diag = GameVisionAI.diagnose()
|
||||
|
||||
# Check available backends
|
||||
for backend in diag['ocr_backends']:
|
||||
print(f"{backend['name']}: {'Available' if backend['available'] else 'Not available'}")
|
||||
```
|
||||
|
||||
## Testing
|
||||
|
||||
### Run Test Suite
|
||||
```bash
|
||||
python test_ocr_system.py
|
||||
```
|
||||
|
||||
### Run Demo
|
||||
```bash
|
||||
python demo_ocr.py
|
||||
```
|
||||
|
||||
### Run Examples
|
||||
```bash
|
||||
# Hardware detection
|
||||
python vision_example.py --hardware
|
||||
|
||||
# List OCR backends
|
||||
python vision_example.py --backends
|
||||
|
||||
# Full diagnostics
|
||||
python vision_example.py --diagnostics
|
||||
|
||||
# Test with image
|
||||
python vision_example.py --full path/to/screenshot.png
|
||||
```
|
||||
|
||||
## Installation
|
||||
|
||||
### Option 1: Minimal (OpenCV EAST Only)
|
||||
```bash
|
||||
pip install opencv-python numpy pillow
|
||||
```
|
||||
|
||||
### Option 2: With EasyOCR
|
||||
```bash
|
||||
pip install torch torchvision # May fail on Windows Store Python
|
||||
pip install easyocr
|
||||
pip install opencv-python numpy pillow
|
||||
```
|
||||
|
||||
### Option 3: With Tesseract
|
||||
```bash
|
||||
# Install Tesseract binary first
|
||||
choco install tesseract # Windows
|
||||
# or download from https://github.com/UB-Mannheim/tesseract/wiki
|
||||
|
||||
pip install pytesseract opencv-python numpy pillow
|
||||
```
|
||||
|
||||
## Windows Store Python Compatibility
|
||||
|
||||
### The Problem
|
||||
```
|
||||
OSError: [WinError 126] The specified module could not be found
|
||||
File "torch\__init__.py", line xxx, in <module>
|
||||
from torch._C import * # DLL load failed
|
||||
```
|
||||
|
||||
### The Solution
|
||||
The system automatically:
|
||||
1. Detects Windows Store Python
|
||||
2. Detects PyTorch DLL errors on import
|
||||
3. Excludes PyTorch-based backends from selection
|
||||
4. Falls back to OpenCV EAST or Tesseract
|
||||
|
||||
### Workarounds for Full PyTorch Support
|
||||
1. **Use Python from python.org** instead of Windows Store
|
||||
2. **Use Anaconda/Miniconda** for better compatibility
|
||||
3. **Use WSL2** (Windows Subsystem for Linux)
|
||||
|
||||
## API Reference
|
||||
|
||||
### Hardware Detection
|
||||
|
||||
```python
|
||||
from modules.hardware_detection import (
|
||||
HardwareDetector,
|
||||
print_hardware_summary,
|
||||
recommend_ocr_backend
|
||||
)
|
||||
|
||||
# Get hardware info
|
||||
info = HardwareDetector.detect_all()
|
||||
print(f"PyTorch available: {info.pytorch_available}")
|
||||
print(f"PyTorch DLL error: {info.pytorch_dll_error}")
|
||||
|
||||
# Get recommendation
|
||||
recommended = recommend_ocr_backend() # Returns: 'opencv_east', 'easyocr', etc.
|
||||
```
|
||||
|
||||
### OCR Backends
|
||||
|
||||
```python
|
||||
from modules.ocr_backends import OCRBackendFactory
|
||||
|
||||
# Check all backends
|
||||
backends = OCRBackendFactory.check_all_backends()
|
||||
|
||||
# Create specific backend
|
||||
backend = OCRBackendFactory.create_backend('opencv_east')
|
||||
|
||||
# Get best available
|
||||
backend = OCRBackendFactory.get_best_backend()
|
||||
```
|
||||
|
||||
### Unified OCR
|
||||
|
||||
```python
|
||||
from modules.game_vision_ai import UnifiedOCRProcessor
|
||||
|
||||
# Auto-select best backend
|
||||
ocr = UnifiedOCRProcessor()
|
||||
|
||||
# Force specific backend
|
||||
ocr = UnifiedOCRProcessor(backend_priority=['tesseract', 'opencv_east'])
|
||||
|
||||
# Extract text
|
||||
regions = ocr.extract_text("image.png")
|
||||
|
||||
# Switch backend
|
||||
ocr.set_backend('tesseract')
|
||||
```
|
||||
|
||||
### Game Vision AI
|
||||
|
||||
```python
|
||||
from modules.game_vision_ai import GameVisionAI
|
||||
|
||||
# Initialize
|
||||
vision = GameVisionAI()
|
||||
|
||||
# Or with specific backend
|
||||
vision = GameVisionAI(ocr_backend='tesseract')
|
||||
|
||||
# Process screenshot
|
||||
result = vision.process_screenshot("screenshot.png")
|
||||
|
||||
# Switch backend at runtime
|
||||
vision.switch_ocr_backend('opencv_east')
|
||||
```
|
||||
|
||||
## Performance Notes
|
||||
|
||||
- **OpenCV EAST**: ~97 FPS on GPU, ~23 FPS on CPU
|
||||
- **EasyOCR**: ~10 FPS on CPU, faster on GPU
|
||||
- **Tesseract**: Slower but very stable
|
||||
- **PaddleOCR**: Fastest with GPU, best accuracy
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
| Issue | Solution |
|
||||
|-------|----------|
|
||||
| "No OCR backend available" | Install opencv-python |
|
||||
| "PyTorch DLL error" | Use OpenCV EAST or Tesseract |
|
||||
| "Tesseract not found" | Install Tesseract binary |
|
||||
| Low accuracy | Use EasyOCR or PaddleOCR |
|
||||
| Slow performance | Enable GPU or use OpenCV EAST |
|
||||
|
||||
## Future Enhancements
|
||||
|
||||
- [ ] ONNX Runtime backend (lighter than PyTorch)
|
||||
- [ ] TensorFlow Lite backend
|
||||
- [ ] Custom trained models for game UI
|
||||
- [ ] YOLO-based UI element detection
|
||||
- [ ] Online learning for icon recognition
|
||||
|
|
@ -0,0 +1,241 @@
|
|||
"""
|
||||
Lemontropia Suite - OCR Demo
|
||||
Demonstrates the multi-backend OCR system.
|
||||
"""
|
||||
|
||||
import sys
|
||||
import cv2
|
||||
import numpy as np
|
||||
from pathlib import Path
|
||||
|
||||
# Add parent directory to path
|
||||
sys.path.insert(0, str(Path(__file__).parent))
|
||||
|
||||
|
||||
def create_demo_image():
|
||||
"""Create a demo image with text-like regions."""
|
||||
# Create white image
|
||||
img = np.ones((600, 800, 3), dtype=np.uint8) * 255
|
||||
|
||||
# Draw some "UI elements" that look like game text
|
||||
# Header bar
|
||||
cv2.rectangle(img, (50, 30), (750, 70), (50, 50, 50), -1)
|
||||
|
||||
# Loot window
|
||||
cv2.rectangle(img, (450, 100), (780, 400), (200, 200, 200), -1)
|
||||
cv2.rectangle(img, (450, 100), (780, 130), (100, 100, 100), -1)
|
||||
|
||||
# Item slots (grid)
|
||||
slot_size = 50
|
||||
gap = 10
|
||||
start_x, start_y = 470, 150
|
||||
for row in range(4):
|
||||
for col in range(5):
|
||||
x = start_x + col * (slot_size + gap)
|
||||
y = start_y + row * (slot_size + gap)
|
||||
cv2.rectangle(img, (x, y), (x + slot_size, y + slot_size), (150, 150, 150), -1)
|
||||
# Add small colored squares (items)
|
||||
if (row + col) % 3 == 0:
|
||||
color = (0, 100, 200) if row % 2 == 0 else (200, 100, 0)
|
||||
cv2.rectangle(img, (x+5, y+5), (x+slot_size-5, y+slot_size-5), color, -1)
|
||||
|
||||
# Text-like regions (rectangles that could contain text)
|
||||
# Chat/log area
|
||||
cv2.rectangle(img, (20, 450), (400, 580), (240, 240, 240), -1)
|
||||
cv2.rectangle(img, (20, 450), (400, 475), (180, 180, 180), -1)
|
||||
|
||||
# Status bars
|
||||
cv2.rectangle(img, (20, 150), (200, 170), (200, 200, 200), -1)
|
||||
cv2.rectangle(img, (20, 150), (150, 170), (0, 150, 0), -1) # Health bar
|
||||
|
||||
cv2.rectangle(img, (20, 180), (200, 200), (200, 200, 200), -1)
|
||||
cv2.rectangle(img, (20, 180), (120, 200), (0, 100, 200), -1) # Energy bar
|
||||
|
||||
return img
|
||||
|
||||
|
||||
def demo_hardware_detection():
|
||||
"""Demo hardware detection."""
|
||||
print("\n" + "=" * 60)
|
||||
print("HARDWARE DETECTION DEMO")
|
||||
print("=" * 60)
|
||||
|
||||
from modules.hardware_detection import (
|
||||
print_hardware_summary,
|
||||
recommend_ocr_backend,
|
||||
HardwareDetector
|
||||
)
|
||||
|
||||
# Print summary
|
||||
print_hardware_summary()
|
||||
|
||||
# Get recommendation
|
||||
recommended = recommend_ocr_backend()
|
||||
print(f"\n📋 Recommended OCR backend: {recommended}")
|
||||
|
||||
# Check for Windows Store Python issues
|
||||
info = HardwareDetector.detect_all()
|
||||
if info.is_windows_store_python:
|
||||
print("\n⚠️ Note: Windows Store Python detected")
|
||||
print(" If you see DLL errors, use 'opencv_east' or 'tesseract' backend")
|
||||
|
||||
|
||||
def demo_ocr_backends():
|
||||
"""Demo OCR backends."""
|
||||
print("\n" + "=" * 60)
|
||||
print("OCR BACKEND DEMO")
|
||||
print("=" * 60)
|
||||
|
||||
from modules.ocr_backends import OCRBackendFactory
|
||||
|
||||
# Create demo image
|
||||
demo_img = create_demo_image()
|
||||
|
||||
# Save for reference
|
||||
demo_path = Path.home() / ".lemontropia" / "demo_screenshot.png"
|
||||
demo_path.parent.mkdir(parents=True, exist_ok=True)
|
||||
cv2.imwrite(str(demo_path), demo_img)
|
||||
print(f"\nDemo image saved to: {demo_path}")
|
||||
|
||||
# Check available backends
|
||||
print("\nChecking OCR backends...")
|
||||
backends = OCRBackendFactory.check_all_backends(use_gpu=True)
|
||||
|
||||
for backend_info in backends:
|
||||
status = "✅" if backend_info.available else "❌"
|
||||
gpu = "🚀 GPU" if backend_info.gpu_accelerated else "💻 CPU"
|
||||
print(f"\n{status} {backend_info.name.upper()}")
|
||||
print(f" Available: {backend_info.available}")
|
||||
print(f" GPU: {gpu}")
|
||||
|
||||
if backend_info.error_message:
|
||||
print(f" Error: {backend_info.error_message}")
|
||||
|
||||
# Test if available
|
||||
if backend_info.available:
|
||||
print(f" Testing...", end=" ")
|
||||
try:
|
||||
backend = OCRBackendFactory.create_backend(
|
||||
backend_info.name, use_gpu=True
|
||||
)
|
||||
if backend:
|
||||
regions = backend.extract_text(demo_img)
|
||||
print(f"Detected {len(regions)} text regions ✓")
|
||||
except Exception as e:
|
||||
print(f"Error: {e}")
|
||||
|
||||
|
||||
def demo_game_vision():
|
||||
"""Demo GameVisionAI."""
|
||||
print("\n" + "=" * 60)
|
||||
print("GAME VISION AI DEMO")
|
||||
print("=" * 60)
|
||||
|
||||
from modules.game_vision_ai import GameVisionAI
|
||||
|
||||
# Create demo image
|
||||
demo_img = create_demo_image()
|
||||
demo_path = Path.home() / ".lemontropia" / "demo_screenshot.png"
|
||||
cv2.imwrite(str(demo_path), demo_img)
|
||||
|
||||
# Initialize
|
||||
print("\nInitializing GameVisionAI...")
|
||||
vision = GameVisionAI(use_gpu=True)
|
||||
|
||||
print(f"✅ Initialized!")
|
||||
print(f" OCR Backend: {vision.ocr.get_current_backend()}")
|
||||
print(f" GPU: {vision.backend.value}")
|
||||
|
||||
# Process screenshot
|
||||
print(f"\nProcessing demo screenshot...")
|
||||
result = vision.process_screenshot(demo_path)
|
||||
|
||||
print(f"✅ Processing complete!")
|
||||
print(f" Processing time: {result.processing_time_ms:.1f}ms")
|
||||
print(f" Text regions: {len(result.text_regions)}")
|
||||
print(f" Icons detected: {len(result.icon_regions)}")
|
||||
|
||||
# Show detected regions
|
||||
if result.text_regions:
|
||||
print("\n Detected text regions:")
|
||||
for i, region in enumerate(result.text_regions[:5]):
|
||||
x, y, w, h = region.bbox
|
||||
print(f" {i+1}. bbox=({x},{y},{w},{h}), conf={region.confidence:.2f}")
|
||||
|
||||
if result.icon_regions:
|
||||
print("\n Detected icons:")
|
||||
for i, icon in enumerate(result.icon_regions[:5]):
|
||||
x, y, w, h = icon.bbox
|
||||
print(f" {i+1}. bbox=({x},{y},{w},{h}), hash={icon.icon_hash[:16]}...")
|
||||
|
||||
|
||||
def demo_backend_switching():
|
||||
"""Demo switching between backends."""
|
||||
print("\n" + "=" * 60)
|
||||
print("BACKEND SWITCHING DEMO")
|
||||
print("=" * 60)
|
||||
|
||||
from modules.game_vision_ai import UnifiedOCRProcessor
|
||||
|
||||
# Create processor
|
||||
processor = UnifiedOCRProcessor()
|
||||
print(f"\nDefault backend: {processor.get_current_backend()}")
|
||||
|
||||
# List all available backends
|
||||
print("\nAvailable backends:")
|
||||
for info in processor.get_available_backends():
|
||||
status = "✅" if info.available else "❌"
|
||||
print(f" {status} {info.name}")
|
||||
|
||||
# Try switching backends
|
||||
demo_img = create_demo_image()
|
||||
|
||||
for backend_name in ['opencv_east', 'tesseract', 'easyocr', 'paddleocr']:
|
||||
print(f"\nTrying to switch to '{backend_name}'...", end=" ")
|
||||
success = processor.set_backend(backend_name)
|
||||
|
||||
if success:
|
||||
print(f"✅ Success!")
|
||||
regions = processor.extract_text(demo_img)
|
||||
print(f" Detected {len(regions)} regions")
|
||||
else:
|
||||
print(f"❌ Not available")
|
||||
|
||||
|
||||
def main():
|
||||
"""Run demos."""
|
||||
print("\n" + "=" * 60)
|
||||
print("LEMONTROPIA SUITE - OCR SYSTEM DEMO")
|
||||
print("=" * 60)
|
||||
print("\nThis demo shows the multi-backend OCR system.")
|
||||
print("The system automatically handles PyTorch DLL errors")
|
||||
print("and falls back to working backends.")
|
||||
|
||||
try:
|
||||
demo_hardware_detection()
|
||||
except Exception as e:
|
||||
print(f"\n❌ Hardware detection failed: {e}")
|
||||
|
||||
try:
|
||||
demo_ocr_backends()
|
||||
except Exception as e:
|
||||
print(f"\n❌ OCR backend demo failed: {e}")
|
||||
|
||||
try:
|
||||
demo_game_vision()
|
||||
except Exception as e:
|
||||
print(f"\n❌ Game Vision demo failed: {e}")
|
||||
|
||||
try:
|
||||
demo_backend_switching()
|
||||
except Exception as e:
|
||||
print(f"\n❌ Backend switching demo failed: {e}")
|
||||
|
||||
print("\n" + "=" * 60)
|
||||
print("DEMO COMPLETE")
|
||||
print("=" * 60)
|
||||
print("\nFor more information, see OCR_SETUP.md")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
|
|
@ -0,0 +1,510 @@
|
|||
"""
|
||||
Lemontropia Suite - Inventory Scanner
|
||||
Specialized computer vision for extracting item data from Entropia Universe inventory.
|
||||
|
||||
Features:
|
||||
- Extract item icons from inventory grid
|
||||
- Read item stats from details panel
|
||||
- Handle scrolling for long stat lists
|
||||
- Auto-detect inventory window position
|
||||
"""
|
||||
|
||||
import cv2
|
||||
import numpy as np
|
||||
import logging
|
||||
from pathlib import Path
|
||||
from typing import List, Dict, Optional, Tuple, Any
|
||||
from dataclasses import dataclass, field
|
||||
from datetime import datetime
|
||||
import json
|
||||
|
||||
from modules.game_vision_ai import GameVisionAI, TextRegion, IconRegion
|
||||
from modules.ocr_backends import create_ocr_backend, AVAILABLE_BACKENDS
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
@dataclass
|
||||
class InventoryItem:
|
||||
"""Represents an item in inventory."""
|
||||
name: str = ""
|
||||
icon_path: Optional[str] = None
|
||||
icon_hash: str = ""
|
||||
slot_position: Tuple[int, int] = (0, 0) # Grid position (row, col)
|
||||
quantity: int = 1
|
||||
|
||||
def to_dict(self) -> Dict[str, Any]:
|
||||
return {
|
||||
'name': self.name,
|
||||
'icon_hash': self.icon_hash,
|
||||
'slot_position': self.slot_position,
|
||||
'quantity': self.quantity
|
||||
}
|
||||
|
||||
|
||||
@dataclass
|
||||
class ItemStats:
|
||||
"""Stats extracted from item details panel."""
|
||||
item_name: str = ""
|
||||
item_class: str = "" # Weapon, Armor, etc.
|
||||
|
||||
# Weapon stats
|
||||
damage: Optional[float] = None
|
||||
range: Optional[float] = None
|
||||
attacks_per_min: Optional[int] = None
|
||||
decay: Optional[float] = None # PEC
|
||||
ammo_burn: Optional[float] = None # PEC
|
||||
damage_per_pec: Optional[float] = None
|
||||
|
||||
# Armor stats
|
||||
protection_stab: Optional[float] = None
|
||||
protection_impact: Optional[float] = None
|
||||
protection_cut: Optional[float] = None
|
||||
protection_penetration: Optional[float] = None
|
||||
protection_shrapnel: Optional[float] = None
|
||||
protection_burn: Optional[float] = None
|
||||
protection_cold: Optional[float] = None
|
||||
protection_acid: Optional[float] = None
|
||||
protection_electric: Optional[float] = None
|
||||
|
||||
# Common stats
|
||||
weight: Optional[float] = None
|
||||
level: Optional[int] = None
|
||||
durability: Optional[float] = None # %
|
||||
markup: Optional[float] = None
|
||||
|
||||
# Raw text for manual parsing
|
||||
raw_text: str = ""
|
||||
|
||||
def to_dict(self) -> Dict[str, Any]:
|
||||
return {
|
||||
'item_name': self.item_name,
|
||||
'item_class': self.item_class,
|
||||
'damage': self.damage,
|
||||
'range': self.range,
|
||||
'attacks_per_min': self.attacks_per_min,
|
||||
'decay': self.decay,
|
||||
'ammo_burn': self.ammo_burn,
|
||||
'damage_per_pec': self.damage_per_pec,
|
||||
'protection_stab': self.protection_stab,
|
||||
'protection_impact': self.protection_impact,
|
||||
'weight': self.weight,
|
||||
'level': self.level,
|
||||
'durability': self.durability,
|
||||
'raw_text': self.raw_text
|
||||
}
|
||||
|
||||
|
||||
@dataclass
|
||||
class InventoryScanResult:
|
||||
"""Result of scanning inventory."""
|
||||
timestamp: datetime = field(default_factory=datetime.now)
|
||||
items: List[InventoryItem] = field(default_factory=list)
|
||||
details_item: Optional[ItemStats] = None
|
||||
inventory_region: Optional[Tuple[int, int, int, int]] = None # x, y, w, h
|
||||
details_region: Optional[Tuple[int, int, int, int]] = None
|
||||
|
||||
def to_dict(self) -> Dict[str, Any]:
|
||||
return {
|
||||
'timestamp': self.timestamp.isoformat(),
|
||||
'items': [item.to_dict() for item in self.items],
|
||||
'details': self.details_item.to_dict() if self.details_item else None,
|
||||
'inventory_region': self.inventory_region,
|
||||
'details_region': self.details_region
|
||||
}
|
||||
|
||||
def save(self, filepath: str):
|
||||
"""Save scan result to JSON."""
|
||||
with open(filepath, 'w') as f:
|
||||
json.dump(self.to_dict(), f, indent=2)
|
||||
|
||||
|
||||
class InventoryScanner:
|
||||
"""
|
||||
Scanner for Entropia Universe inventory.
|
||||
|
||||
Usage:
|
||||
scanner = InventoryScanner()
|
||||
|
||||
# Scan inventory for item icons
|
||||
result = scanner.scan_inventory()
|
||||
|
||||
# Read item details
|
||||
stats = scanner.read_item_details()
|
||||
"""
|
||||
|
||||
def __init__(self, vision_ai: Optional[GameVisionAI] = None):
|
||||
self.vision = vision_ai or GameVisionAI()
|
||||
|
||||
# Inventory window detection
|
||||
self.inventory_title_patterns = ["INVENTORY", "Inventory"]
|
||||
self.item_slot_size = (40, 40) # Typical inventory slot size
|
||||
self.item_slot_gap = 4 # Gap between slots
|
||||
|
||||
# Icon extraction settings
|
||||
self.icon_output_dir = Path.home() / ".lemontropia" / "extracted_icons"
|
||||
self.icon_output_dir.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
# Detection results cache
|
||||
self._last_screenshot: Optional[np.ndarray] = None
|
||||
self._last_inventory_region: Optional[Tuple[int, int, int, int]] = None
|
||||
self._last_details_region: Optional[Tuple[int, int, int, int]] = None
|
||||
|
||||
def capture_screen(self) -> np.ndarray:
|
||||
"""Capture current screen."""
|
||||
try:
|
||||
import mss
|
||||
with mss.mss() as sct:
|
||||
monitor = sct.monitors[1] # Primary monitor
|
||||
screenshot = sct.grab(monitor)
|
||||
img = np.array(screenshot)
|
||||
img = cv2.cvtColor(img, cv2.COLOR_BGRA2BGR)
|
||||
self._last_screenshot = img
|
||||
return img
|
||||
except Exception as e:
|
||||
logger.error(f"Failed to capture screen: {e}")
|
||||
return None
|
||||
|
||||
def detect_inventory_window(self, image: Optional[np.ndarray] = None) -> Optional[Tuple[int, int, int, int]]:
|
||||
"""
|
||||
Detect inventory window position.
|
||||
|
||||
Returns:
|
||||
(x, y, w, h) of inventory window or None
|
||||
"""
|
||||
if image is None:
|
||||
image = self.capture_screen()
|
||||
|
||||
if image is None:
|
||||
return None
|
||||
|
||||
# Look for "INVENTORY" text in the image
|
||||
texts = self.vision.ocr.extract_text(image)
|
||||
|
||||
for text_region in texts:
|
||||
text_upper = text_region.text.upper()
|
||||
if "INVENTORY" in text_upper:
|
||||
# Found inventory title, estimate window bounds
|
||||
# Inventory window typically extends down and to the right from title
|
||||
x, y, w, h = text_region.bbox
|
||||
|
||||
# Estimate full window (typical size ~300x400)
|
||||
window_x = x - 20 # Slight offset for border
|
||||
window_y = y
|
||||
window_w = 350
|
||||
window_h = 450
|
||||
|
||||
# Ensure within image bounds
|
||||
img_h, img_w = image.shape[:2]
|
||||
window_x = max(0, window_x)
|
||||
window_y = max(0, window_y)
|
||||
window_w = min(window_w, img_w - window_x)
|
||||
window_h = min(window_h, img_h - window_y)
|
||||
|
||||
region = (window_x, window_y, window_w, window_h)
|
||||
self._last_inventory_region = region
|
||||
logger.info(f"Detected inventory window: {region}")
|
||||
return region
|
||||
|
||||
logger.warning("Could not detect inventory window")
|
||||
return None
|
||||
|
||||
def detect_item_details_window(self, image: Optional[np.ndarray] = None) -> Optional[Tuple[int, int, int, int]]:
|
||||
"""
|
||||
Detect item details window position.
|
||||
|
||||
Returns:
|
||||
(x, y, w, h) of details window or None
|
||||
"""
|
||||
if image is None:
|
||||
image = self.capture_screen()
|
||||
|
||||
if image is None:
|
||||
return None
|
||||
|
||||
# Look for details panel indicators
|
||||
texts = self.vision.ocr.extract_text(image)
|
||||
|
||||
for text_region in texts:
|
||||
text = text_region.text.upper()
|
||||
# Look for common details panel headers
|
||||
if any(keyword in text for keyword in ["OVERVIEW", "DETAILS", "DESCRIPTION", "STATS"]):
|
||||
x, y, w, h = text_region.bbox
|
||||
|
||||
# Estimate full details window (typically ~250x350)
|
||||
window_x = x - 10
|
||||
window_y = y - 10
|
||||
window_w = 280
|
||||
window_h = 400
|
||||
|
||||
# Ensure within bounds
|
||||
img_h, img_w = image.shape[:2]
|
||||
window_x = max(0, window_x)
|
||||
window_y = max(0, window_y)
|
||||
window_w = min(window_w, img_w - window_x)
|
||||
window_h = min(window_h, img_h - window_y)
|
||||
|
||||
region = (window_x, window_y, window_w, window_h)
|
||||
self._last_details_region = region
|
||||
logger.info(f"Detected details window: {region}")
|
||||
return region
|
||||
|
||||
logger.warning("Could not detect item details window")
|
||||
return None
|
||||
|
||||
def extract_inventory_icons(self, inventory_region: Optional[Tuple[int, int, int, int]] = None,
|
||||
image: Optional[np.ndarray] = None) -> List[InventoryItem]:
|
||||
"""
|
||||
Extract item icons from inventory grid.
|
||||
|
||||
Args:
|
||||
inventory_region: Region of inventory window (auto-detect if None)
|
||||
image: Screenshot (capture new if None)
|
||||
|
||||
Returns:
|
||||
List of InventoryItem with icon paths
|
||||
"""
|
||||
if image is None:
|
||||
image = self.capture_screen()
|
||||
|
||||
if image is None:
|
||||
return []
|
||||
|
||||
if inventory_region is None:
|
||||
inventory_region = self.detect_inventory_window(image)
|
||||
|
||||
if inventory_region is None:
|
||||
logger.error("Cannot extract icons: inventory window not detected")
|
||||
return []
|
||||
|
||||
items = []
|
||||
|
||||
try:
|
||||
x, y, w, h = inventory_region
|
||||
inventory_img = image[y:y+h, x:x+w]
|
||||
|
||||
# Use GameVisionAI to detect icons
|
||||
icon_regions = self.vision.detect_icons(inventory_img)
|
||||
|
||||
for i, icon_region in enumerate(icon_regions):
|
||||
# Calculate grid position (approximate)
|
||||
slot_x = icon_region.bbox[0] // (self.item_slot_size[0] + self.item_slot_gap)
|
||||
slot_y = icon_region.bbox[1] // (self.item_slot_size[1] + self.item_slot_gap)
|
||||
|
||||
# Save icon
|
||||
timestamp = datetime.now().strftime("%Y%m%d_%H%M%S")
|
||||
icon_filename = f"inv_icon_{timestamp}_{i}_{icon_region.icon_hash[:8]}.png"
|
||||
icon_path = self.icon_output_dir / icon_filename
|
||||
|
||||
cv2.imwrite(str(icon_path), icon_region.image)
|
||||
|
||||
item = InventoryItem(
|
||||
icon_path=str(icon_path),
|
||||
icon_hash=icon_region.icon_hash,
|
||||
slot_position=(int(slot_y), int(slot_x))
|
||||
)
|
||||
items.append(item)
|
||||
|
||||
logger.debug(f"Extracted icon {i}: {icon_path}")
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"Failed to extract inventory icons: {e}")
|
||||
|
||||
return items
|
||||
|
||||
def read_item_details(self, details_region: Optional[Tuple[int, int, int, int]] = None,
|
||||
image: Optional[np.ndarray] = None) -> Optional[ItemStats]:
|
||||
"""
|
||||
Read item stats from details panel.
|
||||
|
||||
Args:
|
||||
details_region: Region of details window (auto-detect if None)
|
||||
image: Screenshot (capture new if None)
|
||||
|
||||
Returns:
|
||||
ItemStats with extracted data
|
||||
"""
|
||||
if image is None:
|
||||
image = self.capture_screen()
|
||||
|
||||
if image is None:
|
||||
return None
|
||||
|
||||
if details_region is None:
|
||||
details_region = self.detect_item_details_window(image)
|
||||
|
||||
if details_region is None:
|
||||
logger.error("Cannot read details: details window not detected")
|
||||
return None
|
||||
|
||||
try:
|
||||
x, y, w, h = details_region
|
||||
details_img = image[y:y+h, x:x+w]
|
||||
|
||||
# Extract all text from details panel
|
||||
texts = self.vision.ocr.extract_text(details_img)
|
||||
|
||||
# Combine all text
|
||||
full_text = "\n".join([t.text for t in texts])
|
||||
|
||||
# Parse stats
|
||||
stats = self._parse_item_stats(full_text)
|
||||
stats.raw_text = full_text
|
||||
|
||||
logger.info(f"Read item details: {stats.item_name}")
|
||||
return stats
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"Failed to read item details: {e}")
|
||||
return None
|
||||
|
||||
def _parse_item_stats(self, text: str) -> ItemStats:
|
||||
"""
|
||||
Parse item stats from extracted text.
|
||||
|
||||
This handles various item types (weapons, armor, etc.)
|
||||
"""
|
||||
stats = ItemStats()
|
||||
lines = text.split('\n')
|
||||
|
||||
# Try to find item name (usually first non-empty line)
|
||||
for line in lines:
|
||||
line = line.strip()
|
||||
if line and len(line) > 2:
|
||||
# Skip common headers
|
||||
if line.upper() not in ["OVERVIEW", "DETAILS", "DESCRIPTION", "BASIC", "STATS"]:
|
||||
stats.item_name = line
|
||||
break
|
||||
|
||||
# Parse numeric values
|
||||
for line in lines:
|
||||
line = line.strip()
|
||||
|
||||
# Weapon stats
|
||||
if "Damage" in line or "damage" in line:
|
||||
val = self._extract_number(line)
|
||||
if val:
|
||||
stats.damage = val
|
||||
|
||||
if "Range" in line or "range" in line:
|
||||
val = self._extract_number(line)
|
||||
if val:
|
||||
stats.range = val
|
||||
|
||||
if "Attacks" in line or "attacks" in line or "Att. Per" in line:
|
||||
val = self._extract_int(line)
|
||||
if val:
|
||||
stats.attacks_per_min = val
|
||||
|
||||
if "Decay" in line or "decay" in line:
|
||||
val = self._extract_number(line)
|
||||
if val:
|
||||
stats.decay = val
|
||||
|
||||
if "Ammo" in line or "ammo" in line or "Ammo Burn" in line:
|
||||
val = self._extract_number(line)
|
||||
if val:
|
||||
stats.ammo_burn = val
|
||||
|
||||
# Armor stats
|
||||
if "Stab" in line or "stab" in line:
|
||||
val = self._extract_number(line)
|
||||
if val:
|
||||
stats.protection_stab = val
|
||||
|
||||
if "Impact" in line or "impact" in line:
|
||||
val = self._extract_number(line)
|
||||
if val:
|
||||
stats.protection_impact = val
|
||||
|
||||
if "Cut" in line or "cut" in line:
|
||||
val = self._extract_number(line)
|
||||
if val:
|
||||
stats.protection_cut = val
|
||||
|
||||
# Common stats
|
||||
if "Weight" in line or "weight" in line:
|
||||
val = self._extract_number(line)
|
||||
if val:
|
||||
stats.weight = val
|
||||
|
||||
if "Level" in line or "level" in line or "Req. Level" in line:
|
||||
val = self._extract_int(line)
|
||||
if val:
|
||||
stats.level = val
|
||||
|
||||
# Detect item class from text content
|
||||
if stats.damage:
|
||||
stats.item_class = "Weapon"
|
||||
elif stats.protection_impact:
|
||||
stats.item_class = "Armor"
|
||||
|
||||
return stats
|
||||
|
||||
def _extract_number(self, text: str) -> Optional[float]:
|
||||
"""Extract first float number from text."""
|
||||
import re
|
||||
matches = re.findall(r'[\d.]+', text)
|
||||
if matches:
|
||||
try:
|
||||
return float(matches[0])
|
||||
except:
|
||||
return None
|
||||
return None
|
||||
|
||||
def _extract_int(self, text: str) -> Optional[int]:
|
||||
"""Extract first integer from text."""
|
||||
val = self._extract_number(text)
|
||||
if val:
|
||||
return int(val)
|
||||
return None
|
||||
|
||||
def scan_inventory(self, extract_icons: bool = True, read_details: bool = True) -> InventoryScanResult:
|
||||
"""
|
||||
Full inventory scan - icons + details.
|
||||
|
||||
Returns:
|
||||
InventoryScanResult with all extracted data
|
||||
"""
|
||||
result = InventoryScanResult()
|
||||
|
||||
# Capture screen
|
||||
image = self.capture_screen()
|
||||
if image is None:
|
||||
logger.error("Failed to capture screen for inventory scan")
|
||||
return result
|
||||
|
||||
# Detect windows
|
||||
inventory_region = self.detect_inventory_window(image)
|
||||
details_region = self.detect_item_details_window(image)
|
||||
|
||||
result.inventory_region = inventory_region
|
||||
result.details_region = details_region
|
||||
|
||||
# Extract icons
|
||||
if extract_icons and inventory_region:
|
||||
result.items = self.extract_inventory_icons(inventory_region, image)
|
||||
|
||||
# Read details
|
||||
if read_details and details_region:
|
||||
result.details_item = self.read_item_details(details_region, image)
|
||||
|
||||
return result
|
||||
|
||||
def save_item_to_database(self, item: InventoryItem, stats: ItemStats):
|
||||
"""Save extracted item to database for future reference."""
|
||||
# TODO: Implement database storage
|
||||
logger.info(f"Would save item to DB: {item.icon_hash}")
|
||||
|
||||
|
||||
# Convenience function
|
||||
def scan_inventory() -> InventoryScanResult:
|
||||
"""Quick inventory scan."""
|
||||
scanner = InventoryScanner()
|
||||
return scanner.scan_inventory()
|
||||
|
||||
|
||||
def scan_item_details() -> Optional[ItemStats]:
|
||||
"""Quick item details scan."""
|
||||
scanner = InventoryScanner()
|
||||
return scanner.read_item_details()
|
||||
|
|
@ -0,0 +1,453 @@
|
|||
"""
|
||||
Lemontropia Suite - Inventory Scanner Dialog
|
||||
UI for scanning and extracting item data from Entropia Universe inventory.
|
||||
"""
|
||||
|
||||
import logging
|
||||
from pathlib import Path
|
||||
from typing import Optional
|
||||
|
||||
from PyQt6.QtWidgets import (
|
||||
QDialog, QVBoxLayout, QHBoxLayout, QGridLayout,
|
||||
QLabel, QPushButton, QListWidget, QListWidgetItem,
|
||||
QGroupBox, QSplitter, QTableWidget, QTableWidgetItem,
|
||||
QHeaderView, QTextEdit, QMessageBox, QFileDialog,
|
||||
QProgressBar, QCheckBox, QSpinBox, QTabWidget,
|
||||
QWidget, QScrollArea, QFrame
|
||||
)
|
||||
from PyQt6.QtCore import Qt, QThread, pyqtSignal, QTimer
|
||||
from PyQt6.QtGui import QPixmap, QImage
|
||||
|
||||
from modules.inventory_scanner import InventoryScanner, InventoryScanResult, ItemStats
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
class InventoryScanWorker(QThread):
|
||||
"""Background worker for inventory scanning."""
|
||||
|
||||
scan_complete = pyqtSignal(object) # InventoryScanResult
|
||||
scan_error = pyqtSignal(str)
|
||||
progress_update = pyqtSignal(str)
|
||||
|
||||
def __init__(self, scanner: InventoryScanner, extract_icons: bool = True,
|
||||
read_details: bool = True):
|
||||
super().__init__()
|
||||
self.scanner = scanner
|
||||
self.extract_icons = extract_icons
|
||||
self.read_details = read_details
|
||||
self._is_running = True
|
||||
|
||||
def run(self):
|
||||
"""Run the scan."""
|
||||
try:
|
||||
self.progress_update.emit("Capturing screen...")
|
||||
|
||||
result = self.scanner.scan_inventory(
|
||||
extract_icons=self.extract_icons,
|
||||
read_details=self.read_details
|
||||
)
|
||||
|
||||
if self._is_running:
|
||||
self.scan_complete.emit(result)
|
||||
|
||||
except Exception as e:
|
||||
if self._is_running:
|
||||
self.scan_error.emit(str(e))
|
||||
|
||||
def stop(self):
|
||||
"""Stop the scan."""
|
||||
self._is_running = False
|
||||
|
||||
|
||||
class InventoryScannerDialog(QDialog):
|
||||
"""
|
||||
Dialog for scanning Entropia Universe inventory.
|
||||
|
||||
Features:
|
||||
- Auto-detect inventory window
|
||||
- Extract item icons
|
||||
- Read item stats from details panel
|
||||
- Save results
|
||||
"""
|
||||
|
||||
def __init__(self, parent=None):
|
||||
super().__init__(parent)
|
||||
self.setWindowTitle("🔍 Inventory Scanner")
|
||||
self.setMinimumSize(900, 700)
|
||||
self.resize(1100, 800)
|
||||
|
||||
self.scanner = InventoryScanner()
|
||||
self.scan_worker: Optional[InventoryScanWorker] = None
|
||||
self.last_result: Optional[InventoryScanResult] = None
|
||||
|
||||
self._setup_ui()
|
||||
self._apply_dark_theme()
|
||||
|
||||
def _setup_ui(self):
|
||||
"""Setup the dialog UI."""
|
||||
layout = QVBoxLayout(self)
|
||||
layout.setContentsMargins(15, 15, 15, 15)
|
||||
layout.setSpacing(10)
|
||||
|
||||
# Header
|
||||
header = QLabel("Inventory Scanner - Extract items and stats from Entropia Universe")
|
||||
header.setStyleSheet("font-size: 16px; font-weight: bold; color: #4caf50;")
|
||||
layout.addWidget(header)
|
||||
|
||||
# Controls
|
||||
controls_layout = QHBoxLayout()
|
||||
|
||||
# Scan buttons
|
||||
self.scan_btn = QPushButton("🔍 Scan Inventory")
|
||||
self.scan_btn.setMinimumHeight(40)
|
||||
self.scan_btn.setStyleSheet("""
|
||||
QPushButton {
|
||||
background-color: #0d47a1;
|
||||
font-weight: bold;
|
||||
font-size: 12px;
|
||||
}
|
||||
QPushButton:hover { background-color: #1565c0; }
|
||||
""")
|
||||
self.scan_btn.clicked.connect(self.on_scan)
|
||||
controls_layout.addWidget(self.scan_btn)
|
||||
|
||||
self.scan_details_btn = QPushButton("📋 Scan Item Details Only")
|
||||
self.scan_details_btn.setMinimumHeight(40)
|
||||
self.scan_details_btn.clicked.connect(self.on_scan_details_only)
|
||||
controls_layout.addWidget(self.scan_details_btn)
|
||||
|
||||
# Options
|
||||
options_group = QGroupBox("Options")
|
||||
options_layout = QHBoxLayout(options_group)
|
||||
|
||||
self.extract_icons_check = QCheckBox("Extract Icons")
|
||||
self.extract_icons_check.setChecked(True)
|
||||
options_layout.addWidget(self.extract_icons_check)
|
||||
|
||||
self.read_details_check = QCheckBox("Read Item Details")
|
||||
self.read_details_check.setChecked(True)
|
||||
options_layout.addWidget(self.read_details_check)
|
||||
|
||||
controls_layout.addWidget(options_group)
|
||||
controls_layout.addStretch()
|
||||
|
||||
# Save button
|
||||
self.save_btn = QPushButton("💾 Save Results")
|
||||
self.save_btn.clicked.connect(self.on_save_results)
|
||||
self.save_btn.setEnabled(False)
|
||||
controls_layout.addWidget(self.save_btn)
|
||||
|
||||
layout.addLayout(controls_layout)
|
||||
|
||||
# Progress bar
|
||||
self.progress = QProgressBar()
|
||||
self.progress.setRange(0, 0) # Indeterminate
|
||||
self.progress.setVisible(False)
|
||||
layout.addWidget(self.progress)
|
||||
|
||||
# Status label
|
||||
self.status_label = QLabel("Ready to scan")
|
||||
self.status_label.setStyleSheet("color: #888; padding: 5px;")
|
||||
layout.addWidget(self.status_label)
|
||||
|
||||
# Results tabs
|
||||
self.results_tabs = QTabWidget()
|
||||
layout.addWidget(self.results_tabs)
|
||||
|
||||
# Tab 1: Item Icons
|
||||
self.icons_tab = self._create_icons_tab()
|
||||
self.results_tabs.addTab(self.icons_tab, "🖼️ Item Icons")
|
||||
|
||||
# Tab 2: Item Details
|
||||
self.details_tab = self._create_details_tab()
|
||||
self.results_tabs.addTab(self.details_tab, "📋 Item Details")
|
||||
|
||||
# Tab 3: Raw Data
|
||||
self.raw_tab = self._create_raw_tab()
|
||||
self.results_tabs.addTab(self.raw_tab, "📝 Raw Data")
|
||||
|
||||
# Help text
|
||||
help = QLabel(
|
||||
"Instructions:\n"
|
||||
"1. Open Entropia Universe\n"
|
||||
"2. Open your inventory window\n"
|
||||
"3. Click an item to show its details panel\n"
|
||||
"4. Click 'Scan Inventory' above"
|
||||
)
|
||||
help.setStyleSheet("color: #888; padding: 10px; background-color: #252525; border-radius: 4px;")
|
||||
help.setWordWrap(True)
|
||||
layout.addWidget(help)
|
||||
|
||||
# Close button
|
||||
close_btn = QPushButton("Close")
|
||||
close_btn.clicked.connect(self.accept)
|
||||
layout.addWidget(close_btn)
|
||||
|
||||
def _create_icons_tab(self) -> QWidget:
|
||||
"""Create the item icons tab."""
|
||||
tab = QWidget()
|
||||
layout = QVBoxLayout(tab)
|
||||
|
||||
# Icons count
|
||||
self.icons_count_label = QLabel("No icons extracted yet")
|
||||
layout.addWidget(self.icons_count_label)
|
||||
|
||||
# Icons grid
|
||||
scroll = QScrollArea()
|
||||
scroll.setWidgetResizable(True)
|
||||
scroll.setStyleSheet("background-color: #1a1a1a;")
|
||||
|
||||
self.icons_container = QWidget()
|
||||
self.icons_grid = QGridLayout(self.icons_container)
|
||||
self.icons_grid.setSpacing(10)
|
||||
scroll.setWidget(self.icons_container)
|
||||
|
||||
layout.addWidget(scroll)
|
||||
return tab
|
||||
|
||||
def _create_details_tab(self) -> QWidget:
|
||||
"""Create the item details tab."""
|
||||
tab = QWidget()
|
||||
layout = QVBoxLayout(tab)
|
||||
|
||||
# Details table
|
||||
self.details_table = QTableWidget()
|
||||
self.details_table.setColumnCount(2)
|
||||
self.details_table.setHorizontalHeaderLabels(["Property", "Value"])
|
||||
self.details_table.horizontalHeader().setStretchLastSection(True)
|
||||
self.details_table.setSelectionBehavior(QTableWidget.SelectionBehavior.SelectRows)
|
||||
layout.addWidget(self.details_table)
|
||||
|
||||
return tab
|
||||
|
||||
def _create_raw_tab(self) -> QWidget:
|
||||
"""Create the raw data tab."""
|
||||
tab = QWidget()
|
||||
layout = QVBoxLayout(tab)
|
||||
|
||||
self.raw_text = QTextEdit()
|
||||
self.raw_text.setReadOnly(True)
|
||||
self.raw_text.setFontFamily("Consolas")
|
||||
layout.addWidget(self.raw_text)
|
||||
|
||||
return tab
|
||||
|
||||
def on_scan(self):
|
||||
"""Start full inventory scan."""
|
||||
self._start_scan(
|
||||
extract_icons=self.extract_icons_check.isChecked(),
|
||||
read_details=self.read_details_check.isChecked()
|
||||
)
|
||||
|
||||
def on_scan_details_only(self):
|
||||
"""Scan only item details."""
|
||||
self._start_scan(extract_icons=False, read_details=True)
|
||||
|
||||
def _start_scan(self, extract_icons: bool, read_details: bool):
|
||||
"""Start scan worker."""
|
||||
if self.scan_worker and self.scan_worker.isRunning():
|
||||
QMessageBox.warning(self, "Scan in Progress", "A scan is already running.")
|
||||
return
|
||||
|
||||
# Update UI
|
||||
self.scan_btn.setEnabled(False)
|
||||
self.scan_details_btn.setEnabled(False)
|
||||
self.progress.setVisible(True)
|
||||
self.status_label.setText("Scanning...")
|
||||
self.save_btn.setEnabled(False)
|
||||
|
||||
# Clear previous results
|
||||
self._clear_icons_grid()
|
||||
self.details_table.setRowCount(0)
|
||||
self.raw_text.clear()
|
||||
|
||||
# Start worker
|
||||
self.scan_worker = InventoryScanWorker(
|
||||
self.scanner,
|
||||
extract_icons=extract_icons,
|
||||
read_details=read_details
|
||||
)
|
||||
self.scan_worker.scan_complete.connect(self._on_scan_complete)
|
||||
self.scan_worker.scan_error.connect(self._on_scan_error)
|
||||
self.scan_worker.progress_update.connect(self._on_progress_update)
|
||||
self.scan_worker.start()
|
||||
|
||||
def _on_scan_complete(self, result: InventoryScanResult):
|
||||
"""Handle scan completion."""
|
||||
self.last_result = result
|
||||
|
||||
# Update UI
|
||||
self.progress.setVisible(False)
|
||||
self.scan_btn.setEnabled(True)
|
||||
self.scan_details_btn.setEnabled(True)
|
||||
self.save_btn.setEnabled(True)
|
||||
|
||||
self.status_label.setText(
|
||||
f"Scan complete: {len(result.items)} icons, "
|
||||
f"details: {'Yes' if result.details_item else 'No'}"
|
||||
)
|
||||
|
||||
# Update tabs
|
||||
self._update_icons_tab(result)
|
||||
self._update_details_tab(result)
|
||||
self._update_raw_tab(result)
|
||||
|
||||
QMessageBox.information(
|
||||
self,
|
||||
"Scan Complete",
|
||||
f"Found {len(result.items)} items\n"
|
||||
f"Item details: {'Yes' if result.details_item else 'No'}"
|
||||
)
|
||||
|
||||
def _on_scan_error(self, error: str):
|
||||
"""Handle scan error."""
|
||||
self.progress.setVisible(False)
|
||||
self.scan_btn.setEnabled(True)
|
||||
self.scan_details_btn.setEnabled(True)
|
||||
self.status_label.setText(f"Error: {error}")
|
||||
|
||||
QMessageBox.critical(self, "Scan Error", error)
|
||||
|
||||
def _on_progress_update(self, message: str):
|
||||
"""Handle progress update."""
|
||||
self.status_label.setText(message)
|
||||
|
||||
def _clear_icons_grid(self):
|
||||
"""Clear icons grid."""
|
||||
while self.icons_grid.count():
|
||||
item = self.icons_grid.takeAt(0)
|
||||
if item.widget():
|
||||
item.widget().deleteLater()
|
||||
|
||||
def _update_icons_tab(self, result: InventoryScanResult):
|
||||
"""Update icons tab with results."""
|
||||
self.icons_count_label.setText(f"Extracted {len(result.items)} item icons")
|
||||
|
||||
self._clear_icons_grid()
|
||||
|
||||
for i, item in enumerate(result.items):
|
||||
if item.icon_path and Path(item.icon_path).exists():
|
||||
# Create icon widget
|
||||
icon_widget = QFrame()
|
||||
icon_widget.setStyleSheet("""
|
||||
QFrame {
|
||||
background-color: #2a2a2a;
|
||||
border: 1px solid #444;
|
||||
border-radius: 4px;
|
||||
padding: 5px;
|
||||
}
|
||||
""")
|
||||
icon_layout = QVBoxLayout(icon_widget)
|
||||
|
||||
# Image
|
||||
pixmap = QPixmap(item.icon_path)
|
||||
if not pixmap.isNull():
|
||||
scaled = pixmap.scaled(64, 64, Qt.AspectRatioMode.KeepAspectRatio)
|
||||
img_label = QLabel()
|
||||
img_label.setPixmap(scaled)
|
||||
img_label.setAlignment(Qt.AlignmentFlag.AlignCenter)
|
||||
icon_layout.addWidget(img_label)
|
||||
|
||||
# Info
|
||||
info = QLabel(f"Slot: {item.slot_position}\nHash: {item.icon_hash[:8]}...")
|
||||
info.setStyleSheet("font-size: 10px; color: #888;")
|
||||
icon_layout.addWidget(info)
|
||||
|
||||
# Add to grid (5 columns)
|
||||
row = i // 5
|
||||
col = i % 5
|
||||
self.icons_grid.addWidget(icon_widget, row, col)
|
||||
|
||||
def _update_details_tab(self, result: InventoryScanResult):
|
||||
"""Update details tab with results."""
|
||||
if not result.details_item:
|
||||
self.details_table.setRowCount(1)
|
||||
self.details_table.setItem(0, 0, QTableWidgetItem("Status"))
|
||||
self.details_table.setItem(0, 1, QTableWidgetItem("No item details detected"))
|
||||
return
|
||||
|
||||
stats = result.details_item
|
||||
properties = [
|
||||
("Item Name", stats.item_name),
|
||||
("Class", stats.item_class),
|
||||
("Damage", f"{stats.damage:.2f}" if stats.damage else "N/A"),
|
||||
("Range", f"{stats.range:.2f}" if stats.range else "N/A"),
|
||||
("Attacks/Min", str(stats.attacks_per_min) if stats.attacks_per_min else "N/A"),
|
||||
("Decay (PEC)", f"{stats.decay:.4f}" if stats.decay else "N/A"),
|
||||
("Ammo Burn (PEC)", f"{stats.ammo_burn:.4f}" if stats.ammo_burn else "N/A"),
|
||||
("DPP", f"{stats.damage_per_pec:.4f}" if stats.damage_per_pec else "N/A"),
|
||||
("Weight", f"{stats.weight:.2f}" if stats.weight else "N/A"),
|
||||
("Level", str(stats.level) if stats.level else "N/A"),
|
||||
("Durability", f"{stats.durability:.1f}%" if stats.durability else "N/A"),
|
||||
("Protection Impact", f"{stats.protection_impact:.1f}" if stats.protection_impact else "N/A"),
|
||||
("Protection Stab", f"{stats.protection_stab:.1f}" if stats.protection_stab else "N/A"),
|
||||
]
|
||||
|
||||
self.details_table.setRowCount(len(properties))
|
||||
for row, (prop, value) in enumerate(properties):
|
||||
self.details_table.setItem(row, 0, QTableWidgetItem(prop))
|
||||
self.details_table.setItem(row, 1, QTableWidgetItem(value))
|
||||
|
||||
def _update_raw_tab(self, result: InventoryScanResult):
|
||||
"""Update raw data tab."""
|
||||
import json
|
||||
text = json.dumps(result.to_dict(), indent=2)
|
||||
self.raw_text.setText(text)
|
||||
|
||||
def on_save_results(self):
|
||||
"""Save scan results to file."""
|
||||
if not self.last_result:
|
||||
return
|
||||
|
||||
filepath, _ = QFileDialog.getSaveFileName(
|
||||
self,
|
||||
"Save Inventory Scan",
|
||||
str(Path.home() / f"inventory_scan_{datetime.now():%Y%m%d_%H%M%S}.json"),
|
||||
"JSON Files (*.json)"
|
||||
)
|
||||
|
||||
if filepath:
|
||||
try:
|
||||
self.last_result.save(filepath)
|
||||
QMessageBox.information(self, "Saved", f"Results saved to:\n{filepath}")
|
||||
except Exception as e:
|
||||
QMessageBox.critical(self, "Error", f"Failed to save:\n{e}")
|
||||
|
||||
def closeEvent(self, event):
|
||||
"""Handle dialog close."""
|
||||
if self.scan_worker and self.scan_worker.isRunning():
|
||||
self.scan_worker.stop()
|
||||
self.scan_worker.wait(1000)
|
||||
event.accept()
|
||||
|
||||
def _apply_dark_theme(self):
|
||||
"""Apply dark theme."""
|
||||
self.setStyleSheet("""
|
||||
QDialog {
|
||||
background-color: #1e1e1e;
|
||||
color: #e0e0e0;
|
||||
}
|
||||
QGroupBox {
|
||||
font-weight: bold;
|
||||
border: 1px solid #444;
|
||||
border-radius: 6px;
|
||||
margin-top: 10px;
|
||||
padding-top: 10px;
|
||||
}
|
||||
QTableWidget {
|
||||
background-color: #252525;
|
||||
border: 1px solid #444;
|
||||
}
|
||||
QHeaderView::section {
|
||||
background-color: #2d2d2d;
|
||||
padding: 6px;
|
||||
border: none;
|
||||
border-right: 1px solid #444;
|
||||
}
|
||||
QTextEdit {
|
||||
background-color: #1a1a1a;
|
||||
border: 1px solid #444;
|
||||
color: #d0d0d0;
|
||||
}
|
||||
""")
|
||||
|
|
@ -120,6 +120,7 @@ from ui.hud_overlay_clean import HUDOverlay
|
|||
from ui.session_history import SessionHistoryDialog
|
||||
from ui.gallery_dialog import GalleryDialog, ScreenshotCapture
|
||||
from ui.settings_dialog import SettingsDialog
|
||||
from ui.inventory_scanner_dialog import InventoryScannerDialog
|
||||
|
||||
# ============================================================================
|
||||
# Screenshot Hotkey Integration
|
||||
|
|
@ -752,6 +753,14 @@ class MainWindow(QMainWindow):
|
|||
vision_test_action.triggered.connect(self.on_vision_test)
|
||||
vision_menu.addAction(vision_test_action)
|
||||
|
||||
vision_menu.addSeparator()
|
||||
|
||||
# Inventory Scanner
|
||||
inventory_scan_action = QAction("🔍 &Inventory Scanner", self)
|
||||
inventory_scan_action.setShortcut("Ctrl+I")
|
||||
inventory_scan_action.triggered.connect(self.on_inventory_scan)
|
||||
vision_menu.addAction(inventory_scan_action)
|
||||
|
||||
# View menu
|
||||
view_menu = menubar.addMenu("&View")
|
||||
|
||||
|
|
@ -1895,6 +1904,15 @@ class MainWindow(QMainWindow):
|
|||
self.log_error("Vision", f"Failed to open test dialog: {e}")
|
||||
QMessageBox.warning(self, "Error", f"Could not open Vision Test: {e}")
|
||||
|
||||
def on_inventory_scan(self):
|
||||
"""Open Inventory Scanner dialog."""
|
||||
try:
|
||||
dialog = InventoryScannerDialog(self)
|
||||
dialog.exec()
|
||||
except Exception as e:
|
||||
self.log_error("Vision", f"Failed to open inventory scanner: {e}")
|
||||
QMessageBox.warning(self, "Error", f"Could not open Inventory Scanner: {e}")
|
||||
|
||||
# ========================================================================
|
||||
# Settings Management
|
||||
# ========================================================================
|
||||
|
|
|
|||
|
|
@ -1,6 +1,8 @@
|
|||
"""
|
||||
Lemontropia Suite - Game Vision AI Example
|
||||
Demonstrates usage of the Game Vision AI module.
|
||||
|
||||
Updated for multi-backend OCR system with Windows Store Python support.
|
||||
"""
|
||||
|
||||
import sys
|
||||
|
|
@ -12,32 +14,57 @@ logging.basicConfig(level=logging.INFO, format='%(levelname)s: %(message)s')
|
|||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
def demo_gpu_detection():
|
||||
"""Demonstrate GPU detection."""
|
||||
def demo_hardware_detection():
|
||||
"""Demonstrate hardware detection including PyTorch DLL error handling."""
|
||||
print("\n" + "="*60)
|
||||
print("GPU DETECTION DEMO")
|
||||
print("HARDWARE DETECTION DEMO")
|
||||
print("="*60)
|
||||
|
||||
from modules.game_vision_ai import GPUDetector, GPUBackend
|
||||
from modules.hardware_detection import (
|
||||
HardwareDetector,
|
||||
print_hardware_summary,
|
||||
recommend_ocr_backend
|
||||
)
|
||||
|
||||
# Detect GPU
|
||||
backend = GPUDetector.detect_backend()
|
||||
print(f"\nDetected GPU Backend: {backend.value}")
|
||||
# Print full summary
|
||||
print_hardware_summary()
|
||||
|
||||
# Get detailed info
|
||||
info = GPUDetector.get_gpu_info()
|
||||
print(f"\nGPU Details:")
|
||||
print(f" Backend: {info['backend']}")
|
||||
print(f" CUDA Available: {info['cuda_available']}")
|
||||
print(f" MPS Available: {info['mps_available']}")
|
||||
# Get recommendation
|
||||
recommended = recommend_ocr_backend()
|
||||
print(f"\n📋 Recommended OCR backend: {recommended}")
|
||||
|
||||
if info.get('devices'):
|
||||
print(f"\n Devices:")
|
||||
for dev in info['devices']:
|
||||
mem_gb = dev.get('memory_total', 0) / (1024**3)
|
||||
print(f" [{dev['id']}] {dev['name']} ({mem_gb:.1f} GB)")
|
||||
# Check for Windows Store Python
|
||||
info = HardwareDetector.detect_all()
|
||||
if info.is_windows_store_python:
|
||||
print("\n⚠️ Windows Store Python detected!")
|
||||
if info.pytorch_dll_error:
|
||||
print(" PyTorch DLL errors expected - using fallback OCR backends")
|
||||
|
||||
print(f"\n PyTorch Device String: {GPUDetector.get_device_string(backend)}")
|
||||
|
||||
def demo_ocr_backends():
|
||||
"""Demonstrate OCR backend selection."""
|
||||
print("\n" + "="*60)
|
||||
print("OCR BACKEND DEMO")
|
||||
print("="*60)
|
||||
|
||||
from modules.ocr_backends import OCRBackendFactory
|
||||
|
||||
print("\nChecking all OCR backends...")
|
||||
backends = OCRBackendFactory.check_all_backends(use_gpu=True)
|
||||
|
||||
for info in backends:
|
||||
status = "✅ Available" if info.available else "❌ Not Available"
|
||||
gpu_status = "🚀 GPU" if info.gpu_accelerated else "💻 CPU"
|
||||
|
||||
print(f"\n{info.name.upper()}:")
|
||||
print(f" Status: {status}")
|
||||
print(f" GPU: {gpu_status}")
|
||||
|
||||
if info.error_message:
|
||||
print(f" Note: {info.error_message}")
|
||||
|
||||
available = [b.name for b in backends if b.available]
|
||||
print(f"\n📊 Available backends: {', '.join(available) if available else 'None'}")
|
||||
|
||||
|
||||
def demo_ocr(image_path: str = None):
|
||||
|
|
@ -46,11 +73,15 @@ def demo_ocr(image_path: str = None):
|
|||
print("OCR TEXT EXTRACTION DEMO")
|
||||
print("="*60)
|
||||
|
||||
from modules.game_vision_ai import OCRProcessor
|
||||
from modules.game_vision_ai import UnifiedOCRProcessor
|
||||
import cv2
|
||||
import numpy as np
|
||||
|
||||
# Initialize OCR
|
||||
print("\nInitializing OCR (this may take a moment on first run)...")
|
||||
ocr = OCRProcessor(use_gpu=True, lang='en')
|
||||
# Initialize OCR with auto-selection
|
||||
print("\nInitializing OCR (auto-selecting best backend)...")
|
||||
ocr = UnifiedOCRProcessor(use_gpu=True, lang='en', auto_select=True)
|
||||
|
||||
print(f"Selected backend: {ocr.get_current_backend()}")
|
||||
|
||||
if image_path and Path(image_path).exists():
|
||||
print(f"\nProcessing: {image_path}")
|
||||
|
|
@ -58,11 +89,21 @@ def demo_ocr(image_path: str = None):
|
|||
|
||||
print(f"\nDetected {len(regions)} text regions:")
|
||||
for i, region in enumerate(regions, 1):
|
||||
print(f" {i}. '{region.text}' (confidence: {region.confidence:.2%})")
|
||||
text_preview = region.text[:50] + "..." if len(region.text) > 50 else region.text
|
||||
print(f" {i}. '{text_preview}' (confidence: {region.confidence:.2%})")
|
||||
print(f" Backend: {region.backend}")
|
||||
print(f" Position: ({region.bbox[0]}, {region.bbox[1]}) {region.bbox[2]}x{region.bbox[3]}")
|
||||
else:
|
||||
print(f"\nNo image provided or file not found: {image_path}")
|
||||
print("Usage: python vision_example.py --ocr path/to/screenshot.png")
|
||||
# Create demo image
|
||||
print("\nNo image provided, creating demo image...")
|
||||
demo_img = np.ones((400, 600, 3), dtype=np.uint8) * 255
|
||||
cv2.rectangle(demo_img, (50, 50), (200, 80), (0, 0, 0), -1)
|
||||
cv2.rectangle(demo_img, (50, 100), (250, 130), (0, 0, 0), -1)
|
||||
|
||||
regions = ocr.extract_text(demo_img)
|
||||
print(f"\nDetected {len(regions)} text regions in demo image")
|
||||
|
||||
print(f"\nUsage: python vision_example.py --ocr path/to/screenshot.png")
|
||||
|
||||
|
||||
def demo_icon_detection(image_path: str = None):
|
||||
|
|
@ -73,6 +114,7 @@ def demo_icon_detection(image_path: str = None):
|
|||
|
||||
from modules.game_vision_ai import IconDetector
|
||||
import cv2
|
||||
import numpy as np
|
||||
|
||||
detector = IconDetector()
|
||||
|
||||
|
|
@ -98,7 +140,23 @@ def demo_icon_detection(image_path: str = None):
|
|||
icons = detector.extract_icons_from_region(image, (0, 0, w, h))
|
||||
print(f"Found {len(icons)} potential icons in full image")
|
||||
else:
|
||||
print(f"\nNo image provided or file not found: {image_path}")
|
||||
print("\nNo image provided, creating demo image...")
|
||||
# Create demo image with icon-like regions
|
||||
demo_img = np.ones((600, 800, 3), dtype=np.uint8) * 200
|
||||
|
||||
# Draw some icon-sized squares
|
||||
for i in range(3):
|
||||
for j in range(4):
|
||||
x = 100 + j * 60
|
||||
y = 100 + i * 60
|
||||
color = (100 + i*50, 150, 200 - j*30)
|
||||
cv2.rectangle(demo_img, (x, y), (x+48, y+48), color, -1)
|
||||
|
||||
h, w = demo_img.shape[:2]
|
||||
icons = detector.extract_icons_from_region(demo_img, (0, 0, w, h))
|
||||
print(f"Found {len(icons)} potential icons in demo image")
|
||||
|
||||
print(f"\nUsage: python vision_example.py --icons path/to/screenshot.png")
|
||||
|
||||
|
||||
def demo_full_vision(image_path: str = None):
|
||||
|
|
@ -108,13 +166,17 @@ def demo_full_vision(image_path: str = None):
|
|||
print("="*60)
|
||||
|
||||
from modules.game_vision_ai import GameVisionAI
|
||||
import cv2
|
||||
import numpy as np
|
||||
|
||||
# Initialize vision AI
|
||||
print("\nInitializing Game Vision AI...")
|
||||
vision = GameVisionAI(use_gpu=True, ocr_lang='en')
|
||||
|
||||
print(f"✅ Initialized!")
|
||||
print(f" OCR Backend: {vision.ocr.get_current_backend()}")
|
||||
print(f" GPU: {vision.backend.value}")
|
||||
print(f" GPU Available: {vision.is_gpu_available()}")
|
||||
print(f"Backend: {vision.backend.value}")
|
||||
|
||||
if image_path and Path(image_path).exists():
|
||||
print(f"\nProcessing: {image_path}")
|
||||
|
|
@ -124,11 +186,13 @@ def demo_full_vision(image_path: str = None):
|
|||
|
||||
print(f"\n--- Results ---")
|
||||
print(f"Processing Time: {result.processing_time_ms:.1f}ms")
|
||||
print(f"OCR Backend: {result.ocr_backend}")
|
||||
print(f"GPU Backend: {result.gpu_backend}")
|
||||
|
||||
print(f"\nText Regions ({len(result.text_regions)}):")
|
||||
for region in result.text_regions:
|
||||
print(f" • '{region.text}' ({region.confidence:.2%})")
|
||||
text_preview = region.text[:40] + "..." if len(region.text) > 40 else region.text
|
||||
print(f" • '{text_preview}' ({region.confidence:.2%}) [{region.backend}]")
|
||||
|
||||
print(f"\nIcon Regions ({len(result.icon_regions)}):")
|
||||
for region in result.icon_regions:
|
||||
|
|
@ -136,8 +200,71 @@ def demo_full_vision(image_path: str = None):
|
|||
|
||||
print(f"\nExtracted icons saved to: {vision.extracted_icons_dir}")
|
||||
else:
|
||||
print(f"\nNo image provided or file not found: {image_path}")
|
||||
print("Usage: python vision_example.py --full path/to/screenshot.png")
|
||||
print("\nNo image provided, creating demo image...")
|
||||
# Create demo image
|
||||
demo_img = np.ones((600, 800, 3), dtype=np.uint8) * 240
|
||||
|
||||
# Draw loot window
|
||||
cv2.rectangle(demo_img, (400, 50), (750, 400), (220, 220, 220), -1)
|
||||
|
||||
# Draw some icon slots
|
||||
for row in range(4):
|
||||
for col in range(5):
|
||||
x = 420 + col * 55
|
||||
y = 80 + row * 55
|
||||
cv2.rectangle(demo_img, (x, y), (x+48, y+48), (180, 180, 180), -1)
|
||||
if (row + col) % 2 == 0:
|
||||
color = (100, 150, 200) if row % 2 == 0 else (200, 150, 100)
|
||||
cv2.rectangle(demo_img, (x+5, y+5), (x+43, y+43), color, -1)
|
||||
|
||||
# Save and process
|
||||
demo_path = Path.home() / ".lemontropia" / "demo_screenshot.png"
|
||||
demo_path.parent.mkdir(parents=True, exist_ok=True)
|
||||
cv2.imwrite(str(demo_path), demo_img)
|
||||
|
||||
result = vision.process_screenshot(demo_path)
|
||||
|
||||
print(f"\n--- Demo Results ---")
|
||||
print(f"Processing Time: {result.processing_time_ms:.1f}ms")
|
||||
print(f"Text Regions: {len(result.text_regions)}")
|
||||
print(f"Icon Regions: {len(result.icon_regions)}")
|
||||
|
||||
print(f"\nUsage: python vision_example.py --full path/to/screenshot.png")
|
||||
|
||||
|
||||
def demo_backend_switching():
|
||||
"""Demonstrate switching between OCR backends."""
|
||||
print("\n" + "="*60)
|
||||
print("OCR BACKEND SWITCHING DEMO")
|
||||
print("="*60)
|
||||
|
||||
from modules.game_vision_ai import UnifiedOCRProcessor
|
||||
import cv2
|
||||
import numpy as np
|
||||
|
||||
# Create demo image
|
||||
demo_img = np.ones((400, 600, 3), dtype=np.uint8) * 255
|
||||
cv2.rectangle(demo_img, (50, 50), (200, 80), (0, 0, 0), -1)
|
||||
cv2.rectangle(demo_img, (50, 100), (250, 130), (0, 0, 0), -1)
|
||||
|
||||
# Initialize with auto-selection
|
||||
print("\nInitializing with auto-selected backend...")
|
||||
ocr = UnifiedOCRProcessor()
|
||||
print(f"Default backend: {ocr.get_current_backend()}")
|
||||
|
||||
# Try switching to each backend
|
||||
backends = ['opencv_east', 'tesseract', 'easyocr', 'paddleocr']
|
||||
|
||||
for backend_name in backends:
|
||||
print(f"\nTrying '{backend_name}'...", end=" ")
|
||||
success = ocr.set_backend(backend_name)
|
||||
|
||||
if success:
|
||||
print(f"✅ Success!")
|
||||
regions = ocr.extract_text(demo_img)
|
||||
print(f" Detected {len(regions)} text regions")
|
||||
else:
|
||||
print(f"❌ Not available (see error above)")
|
||||
|
||||
|
||||
def demo_icon_matching():
|
||||
|
|
@ -180,23 +307,41 @@ def demo_icon_matching():
|
|||
print(f" Similarity between two images: {similarity:.2%}")
|
||||
|
||||
|
||||
def demo_calibration():
|
||||
"""Demonstrate calibration."""
|
||||
def demo_diagnostics():
|
||||
"""Demonstrate system diagnostics."""
|
||||
print("\n" + "="*60)
|
||||
print("CALIBRATION DEMO")
|
||||
print("SYSTEM DIAGNOSTICS DEMO")
|
||||
print("="*60)
|
||||
|
||||
from modules.game_vision_ai import GameVisionAI
|
||||
|
||||
vision = GameVisionAI(use_gpu=True)
|
||||
print("\nRunning full system diagnostics...")
|
||||
diag = GameVisionAI.diagnose()
|
||||
|
||||
print("\nTo calibrate, provide sample screenshots:")
|
||||
print(" vision.calibrate_for_game([path1, path2, ...])")
|
||||
print("\nThis will:")
|
||||
print(" 1. Process each screenshot")
|
||||
print(" 2. Measure detection accuracy")
|
||||
print(" 3. Calculate average processing time")
|
||||
print(" 4. Provide recommendations")
|
||||
print("\n--- Hardware ---")
|
||||
hw = diag['hardware']
|
||||
print(f" Platform: {hw['system']['platform']}")
|
||||
print(f" Windows Store Python: {hw['system']['windows_store']}")
|
||||
print(f" GPU Backend: {hw['gpu']['backend']}")
|
||||
print(f" CUDA Available: {hw['gpu']['cuda_available']}")
|
||||
print(f" OpenCV CUDA: {hw['gpu']['opencv_cuda']}")
|
||||
|
||||
print("\n--- ML Frameworks ---")
|
||||
ml = hw['ml_frameworks']
|
||||
print(f" PyTorch: {'Available' if ml['pytorch']['available'] else 'Not Available'}")
|
||||
if ml['pytorch']['available']:
|
||||
print(f" Version: {ml['pytorch']['version']}")
|
||||
if ml['pytorch']['dll_error']:
|
||||
print(f" ⚠️ DLL Error detected!")
|
||||
|
||||
print("\n--- OCR Backends ---")
|
||||
for backend in diag['ocr_backends']:
|
||||
status = "✅" if backend['available'] else "❌"
|
||||
print(f" {status} {backend['name']}")
|
||||
|
||||
print("\n--- Recommendations ---")
|
||||
print(f" OCR Backend: {diag['recommendations']['ocr_backend']}")
|
||||
print(f" GPU: {diag['recommendations']['gpu']}")
|
||||
|
||||
|
||||
def main():
|
||||
|
|
@ -208,53 +353,68 @@ def main():
|
|||
formatter_class=argparse.RawDescriptionHelpFormatter,
|
||||
epilog="""
|
||||
Examples:
|
||||
python vision_example.py --gpu # GPU detection demo
|
||||
python vision_example.py --hardware # Hardware detection
|
||||
python vision_example.py --backends # List OCR backends
|
||||
python vision_example.py --ocr image.png # OCR demo
|
||||
python vision_example.py --icons image.png # Icon detection demo
|
||||
python vision_example.py --full image.png # Full vision demo
|
||||
python vision_example.py --switch # Backend switching demo
|
||||
python vision_example.py --matching # Icon matching demo
|
||||
python vision_example.py --diagnostics # System diagnostics
|
||||
python vision_example.py --all # Run all demos
|
||||
"""
|
||||
)
|
||||
|
||||
parser.add_argument('--gpu', action='store_true', help='GPU detection demo')
|
||||
parser.add_argument('--ocr', type=str, metavar='IMAGE', help='OCR demo with image')
|
||||
parser.add_argument('--icons', type=str, metavar='IMAGE', help='Icon detection demo')
|
||||
parser.add_argument('--full', type=str, metavar='IMAGE', help='Full vision demo')
|
||||
parser.add_argument('--hardware', action='store_true', help='Hardware detection demo')
|
||||
parser.add_argument('--backends', action='store_true', help='List OCR backends')
|
||||
parser.add_argument('--ocr', type=str, metavar='IMAGE', nargs='?', const='', help='OCR demo')
|
||||
parser.add_argument('--icons', type=str, metavar='IMAGE', nargs='?', const='', help='Icon detection demo')
|
||||
parser.add_argument('--full', type=str, metavar='IMAGE', nargs='?', const='', help='Full vision demo')
|
||||
parser.add_argument('--switch', action='store_true', help='Backend switching demo')
|
||||
parser.add_argument('--matching', action='store_true', help='Icon matching demo')
|
||||
parser.add_argument('--calibration', action='store_true', help='Calibration demo')
|
||||
parser.add_argument('--diagnostics', action='store_true', help='System diagnostics')
|
||||
parser.add_argument('--all', action='store_true', help='Run all demos')
|
||||
|
||||
args = parser.parse_args()
|
||||
|
||||
# If no args, show help
|
||||
if not any([args.gpu, args.ocr, args.icons, args.full, args.matching, args.calibration, args.all]):
|
||||
if not any([args.hardware, args.backends, args.ocr is not None,
|
||||
args.icons is not None, args.full is not None,
|
||||
args.switch, args.matching, args.diagnostics, args.all]):
|
||||
parser.print_help()
|
||||
return
|
||||
|
||||
try:
|
||||
if args.all or args.gpu:
|
||||
demo_gpu_detection()
|
||||
if args.all or args.hardware:
|
||||
demo_hardware_detection()
|
||||
|
||||
if args.all or args.ocr:
|
||||
demo_ocr(args.ocr)
|
||||
if args.all or args.backends:
|
||||
demo_ocr_backends()
|
||||
|
||||
if args.all or args.icons:
|
||||
demo_icon_detection(args.icons)
|
||||
if args.all or args.ocr is not None:
|
||||
demo_ocr(args.ocr if args.ocr else None)
|
||||
|
||||
if args.all or args.full:
|
||||
demo_full_vision(args.full)
|
||||
if args.all or args.icons is not None:
|
||||
demo_icon_detection(args.icons if args.icons else None)
|
||||
|
||||
if args.all or args.full is not None:
|
||||
demo_full_vision(args.full if args.full else None)
|
||||
|
||||
if args.all or args.switch:
|
||||
demo_backend_switching()
|
||||
|
||||
if args.all or args.matching:
|
||||
demo_icon_matching()
|
||||
|
||||
if args.all or args.calibration:
|
||||
demo_calibration()
|
||||
if args.all or args.diagnostics:
|
||||
demo_diagnostics()
|
||||
|
||||
except ImportError as e:
|
||||
print(f"\n❌ Import Error: {e}")
|
||||
print("\nMake sure all dependencies are installed:")
|
||||
print(" pip install -r requirements.txt")
|
||||
print(" pip install opencv-python numpy pillow")
|
||||
print("\nFor full OCR support:")
|
||||
print(" pip install easyocr pytesseract paddleocr")
|
||||
except Exception as e:
|
||||
print(f"\n❌ Error: {e}")
|
||||
import traceback
|
||||
|
|
|
|||
Loading…
Reference in New Issue