JARVIS AI Assistant 🤖

Just A Rather Very Intelligent System - An advanced AI-powered desktop automation platform with voice capabilities, vision analysis, and comprehensive system control.

• Voice Commands: Natural language speech recognition with Google Cloud Speech-to-Text
• Voice Responses: High-quality Google TTS with adjustable speed (1.6x optimized)
• Voice Modes: Chat (text), Voice (input only), or Always-Speak (full conversation)
• Fast & Clear: Australian accent TTS at 1.6x speed for efficient communication

• Screen Analysis: Explain what's on your screen
• Visual Q&A: Ask questions about images
• OCR: Extract and analyze text from screenshots
• Debug Assistance: Find and explain errors visually
• UI/UX Analysis: Evaluate interface design
• Code Reading: Understand code from screenshots
• Object Detection: Identify elements in images
• Accessibility Check: WCAG compliance analysis

• WebSocket Integration: Control browser via WebSocket server
• Tab Management: List, switch, create, close tabs programmatically
• URL Navigation: Open websites via voice or text commands
• Real-time Communication: Bidirectional browser-desktop communication

• Complete DateTime Info: Current time, date, day, month, year
• Day of Week: Current weekday with weekend detection
• Timezone: Timezone information and UTC offset
• System Uptime: Boot time and system runtime statistics
• Formatted Output: Multiple output formats for integration

• System Monitoring: CPU, RAM, disk, battery, network, GPU, temperature
• Volume Control: Set, increase, decrease, mute/unmute
• Brightness Control: Adjust screen brightness
• Process Management: List and kill processes
• Power Management: Lock, sleep, restart, shutdown
• Date/Time Info: 9 different date/time operations

• Smart App Launcher: Launch applications with fuzzy name matching
• File Operations: Read, write, create, delete, search, and open files
• Terminal Execution: Execute shell commands with safety validation
• Window Management: Control application windows (minimize, maximize, focus)

• Web Search: Search Google/DuckDuckGo with intelligent parsing
• Web Scraping: Extract content from websites
• URL Opening: Open links in browser
• Email Sending: Send emails via SMTP
• News & Weather: Get latest news and weather updates

• Calculator: Math calculations with natural language processing
• Unit Conversions: Temperature, length, weight, currency conversions
• Date Calculations: Calculate days between dates, date arithmetic
• Clipboard Operations: Programmatic clipboard access (copy, paste, get, clear)
• Screenshots: Automated screen capture (full screen or regions)
• Reminders: Task reminder system with notifications
• Notes: Create and search text notes

• Media Control: Play, pause, next, previous, volume
• Network Monitor: Check connection status and speed

• Lock Screen: Instantly lock your computer
• Empty Recycle Bin: Clear recycle bin and free up space
• Clear Cache: Clean temporary files and browser cache
• Focus Mode: Enable distraction-free mode
• Do Not Disturb: Silence notifications
• Break Reminders: Take break notifications
• Restart Explorer: Refresh Windows Explorer

• Auto-Organize Downloads: Sort files by type automatically
• Find Duplicates: Detect duplicate files by hash
• Batch Rename: Rename multiple files with patterns
• Compress Files: Create ZIP archives
• Archive Old Files: Move old files by age threshold
• Clean Folders: Remove empty folders

• Content Search: Search files by content (text search)
• Recent Files: Find recently modified files
• Large Files: Locate files over specified size
• Date Range Search: Find files within date range
• Fuzzy Filename Search: Smart filename matching

• Schedule Tasks: One-time scheduled tasks
• Recurring Tasks: Daily, weekly, monthly automation
• List Tasks: View all scheduled tasks
• Delete Tasks: Remove scheduled tasks
• Startup Tasks: Run programs at system boot

• Clean All: Run all cleanup operations
• Clear Cache: System cache cleanup
• Clean Logs: Remove old log files
• Thumbnail Cleanup: Clear Windows thumbnail cache
• Windows Update Cleanup: Clean update files
• Disk Cleanup: Launch Windows Disk Cleanup utility

• Save Workspace: Save current window layout
• Load Workspace: Restore saved layout
• List Workspaces: View saved workspaces
• Auto-Arrange: Smart window tiling (tile/cascade/grid)
• Multi-Monitor Setup: Configure monitor layouts

• Clipboard History: View last 100 items
• Save Clips: Permanently save important clips
• Search History: Search clipboard history
• Clear History: Clear clipboard history
• Format Text: Uppercase, lowercase, title case, etc.

• Total Tools: 25 (18 original + 7 new)
• Total Operations: 100+ unique operations
• Multi-Tool Execution: 2-5x faster with parallel execution
• Voice Recognition: Google Cloud Speech-to-Text
• Vision AI: Free NVIDIA Nemotron vision model
• Browser Control: WebSocket-based real-time control

• Python: 3.9 or higher
• Operating System: Windows (primary), macOS, Linux
• APIs:
  • OpenRouter API key (free tier available) - Get it here
  • Or Google Gemini API key - Get it here
• Optional: Microphone for voice commands

git clone <repository-url>
cd NewJ

# Create virtual environment (recommended)
python -m venv venv

# Activate virtual environment
# Windows:
venv\Scripts\activate
# Linux/Mac:
source venv/bin/activate

# Install all requirements
pip install -r requirements.txt

Create a .env file (copy from .env.example):

# AI Provider Selection
AI_PROVIDER=openrouter

# OpenRouter API (Free tier available)
OPENROUTER_API_KEY=your_key_here
OPENROUTER_MODEL=nvidia/nemotron-3-nano-30b-a3b:free
OPENROUTER_VISION_MODEL=nvidia/nemotron-nano-12b-v2-vl:free

# User Preferences
USER_NAME=Sir
PREFERRED_BROWSER=chrome
PREFERRED_EDITOR=vscode

# Voice Settings
VOICE_ENABLED=true
VOICE_MODE=chat                  # chat, voice, or always_speak
TTS_METHOD=gtts                  # Google TTS (high quality)
TTS_FREQUENCY=70560              # 1.6x faster speed

# Safety Settings
CONFIRM_DESTRUCTIVE=true
RESTRICT_TO_HOME_DIR=true

For browser control features:

1. Open Chrome/Brave
2. Go to chrome://extensions/
3. Enable "Developer mode"
4. Click "Load unpacked"
5. Select browser_extension folder
6. Extension will connect automatically when JARVIS starts

python main.py

# Start with voice mode enabled (VOICE_MODE=voice in .env)
python main.py

You: [Speak] "What time is it?"
JARVIS: The current time is 11:08 AM.

You: [Speak] "What's today's date?"
JARVIS: Today is Wednesday, July 8th, 2026.

You: [Speak] "Open YouTube"
JARVIS: Opening YouTube.

You: analyze my screen
JARVIS: *takes screenshot and analyzes*
I can see a code editor with a Python file containing a class definition...

You: what errors do you see on my screen?
JARVIS: *analyzes screenshot*
There is a syntax error on line 42: missing closing parenthesis...

You: read the code on my screen
JARVIS: *performs OCR and code analysis*
This code defines a User class with authentication methods...

You: list all open tabs
JARVIS: You have 3 tabs open:
1. YouTube
2. GitHub  
3. Gmail

You: switch to youtube
JARVIS: Switched to YouTube tab.

You: open reddit
JARVIS: Opening Reddit in new tab.

You: what time is it?
JARVIS: The current time is 11:08 AM.

You: what day is it?
JARVIS: Today is Wednesday.

You: what's the full date?
JARVIS: Today is Wednesday, July 8th, 2026.

You: how long has the computer been running?
JARVIS: The system has been running for 3 hours and 8 minutes.

You: lock my screen
JARVIS: Screen locked.

You: empty recycle bin
JARVIS: Recycle bin emptied successfully. Freed 2.3 GB of space.

You: clear temporary files
JARVIS: Cleared 450 temporary files, freed 1.8 GB.

You: clear chrome cache
JARVIS: Chrome browser cache cleared.

You: enable focus mode
JARVIS: Focus mode enabled. Notifications disabled.

You: remind me to take a break
JARVIS: Break reminder notification sent.

You: organize my downloads
JARVIS: Downloads organized:
- 23 images moved to Images folder
- 12 documents moved to Documents folder
- 8 archives moved to Archives folder
- 4 videos moved to Videos folder

You: find duplicate files
JARVIS: Found 15 duplicate files totaling 456 MB.

You: compress my documents folder
JARVIS: Created Documents.zip (23.5 MB) with 47 files.

You: archive files older than 30 days
JARVIS: Archived 28 files to Archive folder.

You: search for files containing "project report"
JARVIS: Found 8 files containing "project report":
1. project_report_final.docx (Documents)
2. report_notes.txt (Desktop)
...

You: find files modified in last 7 days
JARVIS: Found 34 files modified in the last 7 days.

You: find large files over 100 MB
JARVIS: Found 12 files over 100 MB, totaling 3.2 GB.

You: fuzzy search for "rapport"
JARVIS: Found similar filenames:
- project_report.docx
- annual_report_2026.pdf
- status_report.txt

You: schedule a task to backup files at 2 PM tomorrow
JARVIS: Task "BackupFiles" scheduled for tomorrow at 2:00 PM.

You: create a daily reminder at 9 AM
JARVIS: Daily task created to run at 9:00 AM.

You: list all scheduled tasks
JARVIS: You have 5 scheduled tasks:
1. BackupFiles - Tomorrow at 2:00 PM
2. DailyReminder - Daily at 9:00 AM
...

You: add notepad to startup
JARVIS: Notepad added to startup tasks.

You: clean all temporary files
JARVIS: Cleanup complete:
- Cache: 890 MB freed
- Logs: 234 MB freed
- Thumbnails: 156 MB freed
Total: 1.28 GB freed

You: clear system cache
JARVIS: System cache cleared, freed 890 MB.

You: run disk cleanup
JARVIS: Windows Disk Cleanup utility launched.

You: save current workspace as coding
JARVIS: Workspace "coding" saved with 6 windows.

You: load workspace coding
JARVIS: Restored 6 windows from workspace "coding".

You: list all workspaces
JARVIS: Saved workspaces:
1. coding (6 windows)
2. browsing (3 windows)
3. design (4 windows)

You: arrange windows in tile layout
JARVIS: Arranged 5 windows in tile layout.

You: show clipboard history
JARVIS: Last 10 clipboard items:
1. "Project deadline: July 15"
2. "https://github.com/..."
3. "Meeting notes from..."
...

You: save this clip as important
JARVIS: Clipboard saved with label "important".

You: search clipboard for "project"
JARVIS: Found 3 items matching "project".

You: format clipboard text as uppercase
JARVIS: Text formatted as uppercase and copied to clipboard.

You: show system info
JARVIS: System Status:
CPU: 25.3% | Memory: 60.2% | Disk: 45.8%
Battery: 85% (Charging)

You: set volume to 50
JARVIS: Volume set to 50%.

You: mute audio
JARVIS: Audio muted.

You: read example.txt
JARVIS: Here's the content of example.txt:
[file contents]

You: create a file called notes.txt with "Hello World"
JARVIS: Created notes.txt successfully.

You: open Downloads folder
JARVIS: Opened Downloads folder in File Explorer.

You: search for Python tutorials
JARVIS: Here are the search results:
1. Python Tutorial - Official Documentation
2. Learn Python - Codecademy
...

VOICE_MODE=chat

• Text input/output only
• No voice, fastest mode
• Use when you want silent operation

VOICE_MODE=voice

• Voice input (speech-to-text)
• Text output (on screen)
• 10-second listening timeout
• Say "stop listening" to exit

VOICE_MODE=always_speak

• Voice input (speech-to-text)
• Voice output (Google TTS at 1.6x speed)
• Full voice conversation experience
• Say "stop listening" to exit

Adjust TTS_FREQUENCY in .env:

JARVIS includes 8 vision analysis operations:

1. explain_screen: Describe what's on screen
2. visual_qa: Answer questions about images
3. ocr_analyze: Extract and analyze text
4. debug_screen: Find and explain errors
5. analyze_ui: UI/UX evaluation
6. read_code: Understand code from screenshots
7. identify_objects: List visible elements
8. accessibility_check: WCAG compliance

Usage:

You: analyze my screen
You: what errors do you see?
You: read the code on my screen
You: check accessibility of this UI

# AI Provider (choose one)
AI_PROVIDER=openrouter    # Recommended (free tier available)
# AI_PROVIDER=gemini      # Alternative

# OpenRouter (Free Models Available!)
OPENROUTER_API_KEY=your_key_here
OPENROUTER_MODEL=nvidia/nemotron-3-nano-30b-a3b:free
OPENROUTER_VISION_MODEL=nvidia/nemotron-nano-12b-v2-vl:free

# Voice Settings
VOICE_MODE=voice          # chat, voice, or always_speak
TTS_METHOD=gtts           # Google TTS (best quality)
TTS_FREQUENCY=70560       # 1.6x faster (44100=normal)

# User Preferences
USER_NAME=Sir
PREFERRED_BROWSER=chrome

# Voice Modes
VOICE_MODE=chat           # Text only (silent)
VOICE_MODE=voice          # Voice input, text output
VOICE_MODE=always_speak   # Full voice conversation

# TTS Method
TTS_METHOD=gtts           # Google TTS (high quality, requires internet)
TTS_METHOD=pyttsx3        # Offline TTS (robotic voice)

# TTS Speed (for gtts only)
TTS_FREQUENCY=70560       # 1.6x faster (recommended)

NewJ/
├── browser_extension/           # Chrome/Brave extension for browser control
│   ├── manifest.json
│   ├── background.js
│   ├── popup.html
│   └── icons/
├── core/
│   ├── agent.py                # AI agent with tool orchestration
│   ├── memory.py               # SQLite-based conversation memory
│   ├── model_manager.py        # Multi-provider AI management
│   ├── browser_server.py       # WebSocket server for browser
│   └── voice_manager.py        # Voice recognition and TTS
├── tools/
│   ├── base.py                 # Tool base class
│   ├── app_launcher.py         # Application launcher
│   ├── file_system.py          # File operations
│   ├── system_control.py       # System & datetime info
│   ├── terminal.py             # Command execution
│   ├── web_search.py           # Web search
│   ├── browser_automation.py   # Browser control
│   ├── vision_analysis.py      # Vision AI tools
│   ├── quick_actions.py        # Quick system actions (NEW)
│   ├── file_organizer.py       # File organization (NEW)
│   ├── smart_search.py         # Advanced file search (NEW)
│   ├── scheduler.py            # Task scheduling (NEW)
│   ├── system_cleaner.py       # Deep system cleanup (NEW)
│   ├── workspace_manager.py    # Window layouts (NEW)
│   ├── clipboard_manager.py    # Clipboard history (NEW)
│   └── ... (25 tools total)
├── config.py                   # Configuration management
├── main.py                     # Main entry point
├── requirements.txt            # All dependencies
├── .env.example                # Configuration template
└── README.md                   # This file

"Speech recognition not available"

pip install SpeechRecognition pyaudio

Voice is too slow

• Increase TTS_FREQUENCY in .env (try 79380 for 1.8x)
• Make sure TTS_METHOD=gtts (not pyttsx3)

Microphone not working

• Check Windows Sound settings
• Make sure microphone is set as default
• Grant microphone permissions to Python

"Vision model not responding"

• Make sure you're using free vision model: nvidia/nemotron-nano-12b-v2-vl:free
• Vision analysis takes 30-60 seconds (normal for free models)
• Check your OpenRouter API key is valid

"Browser extension not connecting"

• Make sure extension is loaded in Chrome/Brave
• Check extension is enabled
• Restart JARVIS (WebSocket server on port 9876)
• Check firewall isn't blocking localhost:9876

"Too many requests / Rate limit"

• Voice mode uses Google Speech API (has rate limits)
• If you get rate limit errors, wait 60 seconds
• Consider reducing voice usage
• Use free OpenRouter models (no rate limits)

"OpenRouter API key invalid"

• Get free API key from https://openrouter.ai/keys
• Make sure key starts with sk-or-v1-
• Check .env file has correct key

"Module not found" errors

pip install -r requirements.txt

Permission errors

• Run as Administrator on Windows for system operations
• Check RESTRICT_TO_HOME_DIR setting in .env

High CPU usage

• Voice recognition uses CPU when listening
• Vision analysis is intensive (30-60s per operation)
• This is normal for AI operations

In JARVIS chat:

• help - Show available commands
• status - Display system status
• clear - Clear conversation history
• exit or quit - Exit JARVIS
• stop listening - Exit voice mode (when in voice mode)

• ✅ Voice recognition and TTS (Google TTS at 1.6x speed)
• ✅ Vision analysis (8 operations using NVIDIA free model)
• ✅ Browser extension for tab/URL control
• ✅ Date/Time system tools (9 operations)
• ✅ Updated to new Google Genai package
• ✅ Fixed voice speed (clean sample rate manipulation)
• ✅ Fixed unicode logging errors
• ✅ 18 total tools registered

• Google TTS with adjustable speed (1.6x recommended)
• Speech recognition with 10-second timeout
• Three voice modes (chat, voice, always_speak)
• Clean audio with no echo or artifacts

• 8 vision analysis operations
• Uses free NVIDIA vision model
• Screen capture and analysis
• OCR, debugging, UI analysis

• Command Validation: Dangerous commands require confirmation
• Path Restrictions: File access can be limited to home directory
• Timeout Enforcement: Commands have maximum execution time
• Process Protection: Critical system processes cannot be killed
• Audit Trail: All operations logged to jarvis.log

This project is provided as-is for educational and personal use.

• Powered by OpenRouter (free models available)
• Google Gemini AI support
• NVIDIA free vision models
• Google Cloud Speech-to-Text and TTS
• Built with Python and modern AI technologies

JARVIS has direct system-level access. Use responsibly:

• Review commands before execution
• Be careful with file deletion and system commands
• Voice commands are executed immediately - speak clearly
• Vision analysis captures screenshots (consider privacy)
• Browser extension accesses tab information (consider privacy)
• Always test in safe environments first

Additional documentation:

• VOICE_MODE_SETTINGS.md - Complete voice configuration guide
• VOICE_SPEED_FIX.md - Voice speed technical details
• VISION_TOOLS_GUIDE.md - Vision analysis guide
• BROWSER_EXTENSION_GUIDE.md - Browser extension setup
• CONTINUOUS_LISTENING_GUIDE.md - Voice mode details
• FIXES_SUMMARY.md - All fixes and updates

For issues:

1. Check troubleshooting section
2. Review jarvis.log file
3. Check documentation files
4. Open an issue on repository

Built with ❤️ for developers who want an AI assistant that actually gets work done.

"Just A Rather Very Intelligent System" - Ready to serve! 🚀