231 lines
6.3 KiB
Markdown
231 lines
6.3 KiB
Markdown
# EU-Utility Test Suite
|
|
|
|
Comprehensive test plugins for validating the Three-Tier API (PluginAPI, WidgetAPI, ExternalAPI).
|
|
|
|
## Test Plugins Overview
|
|
|
|
### 1. API Comprehensive Test (`api_comprehensive_test/`)
|
|
**Purpose:** Tests every method across all three API tiers
|
|
|
|
**Coverage:**
|
|
- **PluginAPI (12 services, 30+ tests):**
|
|
- Log Reader: `read_log_lines()`, `read_log_since()`
|
|
- Window Manager: `get_eu_window()`, `is_eu_focused()`, `is_eu_visible()`, `bring_eu_to_front()`
|
|
- OCR: `recognize_text()`, `ocr_available()`
|
|
- Screenshot: `capture_screen()`, `screenshot_available()`
|
|
- Nexus: `search_items()`, `get_item_details()`
|
|
- HTTP: `http_get()`, `http_post()`
|
|
- Audio: `play_sound()`, `beep()`
|
|
- Notifications: `show_notification()`
|
|
- Clipboard: `copy_to_clipboard()`, `paste_from_clipboard()`
|
|
- EventBus: `subscribe()`, `unsubscribe()`, `publish()`
|
|
- DataStore: `get_data()`, `set_data()`, `delete_data()`
|
|
- Tasks: `run_task()`, `cancel_task()`
|
|
|
|
- **WidgetAPI (25+ tests):**
|
|
- Creation: `create_widget()`, `create_from_preset()`
|
|
- Access: `get_widget()`, `get_all_widgets()`, `get_visible_widgets()`, `widget_exists()`
|
|
- Management: `show_widget()`, `hide_widget()`, `close_widget()`, `show_all_widgets()`, etc.
|
|
- Instance methods: `show()`, `hide()`, `move()`, `resize()`, `set_opacity()`, etc.
|
|
- Layout: `arrange_widgets()`, `snap_to_grid()`
|
|
- Persistence: `save_all_states()`, `load_all_states()`, `save_state()`, `load_state()`
|
|
|
|
- **ExternalAPI (15+ tests):**
|
|
- Server: `start_server()`, `stop_server()`, `get_status()`
|
|
- Endpoints: `register_endpoint()`, `unregister_endpoint()`, `@endpoint` decorator
|
|
- Webhooks: `register_webhook()`, `unregister_webhook()`, `post_webhook()`
|
|
- Auth: `create_api_key()`, `revoke_api_key()`
|
|
- IPC: `register_ipc_handler()`, `send_ipc()`
|
|
|
|
**Features:**
|
|
- Visual HTML-based results display
|
|
- Pass/fail tracking for each test
|
|
- Error reporting with details
|
|
- Results export to JSON
|
|
|
|
---
|
|
|
|
### 2. Widget Stress Test (`widget_stress_test/`)
|
|
**Purpose:** Tests widget system under load
|
|
|
|
**Test Scenarios:**
|
|
- **Bulk Creation:** Creates 10-50 widgets rapidly
|
|
- **Layout Stress:** Tests grid, horizontal, vertical, cascade arrangements
|
|
- **Visibility Cycles:** Rapid show/hide operations
|
|
- **Property Modifications:** Rapid opacity, position, size changes
|
|
- **Concurrent Operations:** Create while modifying
|
|
|
|
**Metrics:**
|
|
- Operation count
|
|
- Duration (ms)
|
|
- Success rate
|
|
- Error tracking
|
|
|
|
---
|
|
|
|
### 3. External Integration Test (`external_integration_test/`)
|
|
**Purpose:** Tests third-party integration features
|
|
|
|
**Test Categories:**
|
|
- **REST Server:** Start/stop lifecycle, endpoint registration, CORS
|
|
- **Webhooks:** Incoming/outgoing, HMAC signature verification
|
|
- **Authentication:** API key creation/revocation
|
|
- **IPC:** Handler registration, message sending
|
|
- **Utilities:** Status endpoint, URL generation, webhook history
|
|
|
|
---
|
|
|
|
### 4. Event Bus Test (`event_bus_test/`)
|
|
**Purpose:** Tests pub/sub messaging system
|
|
|
|
**Test Coverage:**
|
|
- Basic subscribe/publish
|
|
- Unsubscribe functionality
|
|
- Multiple subscribers to single event
|
|
- Various data types (string, int, dict, list, nested)
|
|
- Wildcard/pattern subscriptions
|
|
- High-volume publishing (100 events)
|
|
- Rapid subscribe/unsubscribe cycles
|
|
- Empty/null event data
|
|
- Large payloads
|
|
- Special characters in event types
|
|
|
|
**Visualization:**
|
|
- Real-time event log
|
|
- Delivery statistics
|
|
- Performance metrics
|
|
|
|
---
|
|
|
|
### 5. Performance Benchmark (`performance_benchmark/`)
|
|
**Purpose:** Measures API performance metrics
|
|
|
|
**Benchmarks:**
|
|
- **DataStore:** Read/write operations (1000+ iterations)
|
|
- **EventBus:** Publish/subscribe throughput
|
|
- **HTTP:** Network request latency
|
|
- **WidgetAPI:** Creation and operation speed
|
|
- **ExternalAPI:** Key and endpoint registration
|
|
|
|
**Metrics:**
|
|
- Average latency (ms)
|
|
- Min/max latency
|
|
- Throughput (ops/sec)
|
|
- Total operations
|
|
|
|
**Output:**
|
|
- Interactive results table
|
|
- Performance grades (Good/Slow)
|
|
- Export to JSON
|
|
|
|
---
|
|
|
|
### 6. Error Handling Test (`error_handling_test/`)
|
|
**Purpose:** Tests error conditions and exception handling
|
|
|
|
**Error Types Tested:**
|
|
- Invalid input
|
|
- Service unavailable
|
|
- Resource not found
|
|
- Type errors
|
|
- Timeout
|
|
- Boundary conditions
|
|
|
|
**Test Counts:**
|
|
- PluginAPI: 15+ error scenarios
|
|
- WidgetAPI: 10+ error scenarios
|
|
- ExternalAPI: 10+ error scenarios
|
|
|
|
**Verification:**
|
|
- Graceful error handling (no crashes)
|
|
- Correct exception types
|
|
- Meaningful error messages
|
|
|
|
---
|
|
|
|
## Running the Tests
|
|
|
|
### Automatic Execution
|
|
All test plugins run automatically on initialization and display results in their widget.
|
|
|
|
### Manual Execution
|
|
Each plugin provides control buttons to:
|
|
- Run all tests
|
|
- Run specific test categories
|
|
- Clear results
|
|
- Export data
|
|
|
|
### Expected Results
|
|
- **Comprehensive Test:** 60+ tests
|
|
- **Stress Test:** 6+ stress scenarios
|
|
- **Integration Test:** 12+ integration tests
|
|
- **Event Bus Test:** 12+ messaging tests
|
|
- **Performance Benchmark:** 8+ benchmarks
|
|
- **Error Handling Test:** 35+ error scenarios
|
|
|
|
---
|
|
|
|
## Test Metadata
|
|
|
|
Each plugin includes `manifest.json` with:
|
|
```json
|
|
{
|
|
"test_metadata": {
|
|
"test_type": "comprehensive|stress|integration|messaging|performance|error_handling",
|
|
"apis_tested": ["PluginAPI", "WidgetAPI", "ExternalAPI"],
|
|
"automated": true
|
|
}
|
|
}
|
|
```
|
|
|
|
---
|
|
|
|
## File Structure
|
|
```
|
|
plugins/test_suite/
|
|
├── api_comprehensive_test/
|
|
│ ├── manifest.json
|
|
│ └── plugin.py
|
|
├── widget_stress_test/
|
|
│ ├── manifest.json
|
|
│ └── plugin.py
|
|
├── external_integration_test/
|
|
│ ├── manifest.json
|
|
│ └── plugin.py
|
|
├── event_bus_test/
|
|
│ ├── manifest.json
|
|
│ └── plugin.py
|
|
├── performance_benchmark/
|
|
│ ├── manifest.json
|
|
│ └── plugin.py
|
|
├── error_handling_test/
|
|
│ ├── manifest.json
|
|
│ └── plugin.py
|
|
└── README.md (this file)
|
|
```
|
|
|
|
---
|
|
|
|
## Continuous Integration
|
|
|
|
These plugins can be used in CI pipelines:
|
|
1. Load plugin
|
|
2. Wait for initialization
|
|
3. Parse results from exported JSON
|
|
4. Fail build on critical test failures
|
|
|
|
---
|
|
|
|
## Maintenance
|
|
|
|
When adding new API features:
|
|
1. Add corresponding tests to `api_comprehensive_test`
|
|
2. Add performance benchmark if applicable
|
|
3. Add error handling tests for edge cases
|
|
4. Update this README
|
|
|
|
---
|
|
|
|
**Version:** 1.0.0
|
|
**Last Updated:** 2026-02-15
|
|
**Compatible with:** EU-Utility API v2.2.0+ |