The AI that lives on your desktop — sees, thinks, and acts.

(Just kidding, don't try that!!)

Glitch breaks the fourth wall of the operating system.

AI Desktop Companion (Glitch) is a fully multimodal, autonomous desktop agent that:

• 👁️ Sees your screen
• 🎤 Talks with you
• 🤖 Controls your system
• 🖥️ Lives directly on your desktop as a playful character
• …and a lot more (I want you to explore 😄)

This isn’t just an assistant you use.
It’s one you work with.

▶️ Full Demo (Vimeo)
https://vimeo.com/1150677379

The Vimeo demo shows Glitch executing real tasks end-to-end.

(Also check out our landing page here!!)

Glitch runs as a transparent, click-through desktop overlay.
He shares your workspace instead of hiding in a window or sidebar.

• Interactive pixel-style characters
• Drag, click, and interact
• Customizable appearance and behavior

Inspired by classic desktop pets, powered by modern multimodal AI.

Everything is built in:

• 🎤 Voice Mode
• 👁️ Vision Mode
• 🤖 Agent Mode
• ⚙️ Settings (character & voice customization)

No switching apps. No broken context.

This is not just another chatbot.

Agent Mode lets Glitch:

• Control mouse & keyboard
• Open applications
• Execute multi-step workflows
• Do real things on your system

There’s always a stop button. Safety matters.

Glitch is especially useful while building.

Here, it creates a complete Next.js project structure from a single voice command — turning ideas into runnable code instantly.

Glitch can:

• Summarize information
• Extract key points
• Save them directly to Notepad or files

Your AI remembers for you.

Ask once — Glitch searches Google, parses results, and gives you the useful bits.

Hands-free.

Glitch isn’t robotic.

He has personality.
He reacts.
He feels present.

Working with AI finally feels alive, not transactional.

Glitch uses a hybrid multimodal agent architecture:

• 🧠 Brain — Google Gemini 2.0 Flash (chat + vision)
• 👁️ Vision — Screen understanding via screenshots
• 🎤 Voice — ElevenLabs (low-latency TTS)
• 🤖 Automation — nut.js (mouse, keyboard, OS control)
• 🖥️ UI Soul — Electron + PixiJS (desktop overlay)

• Node.js (v16 or higher)
• Python (v3.8 or higher) — Required for mouse/keyboard automation
• Google Gemini API Key (Get it here)
• ElevenLabs API Key (Get it here)

1. Clone the repository

   git clone https://github.com/KirthanNB/AI-Companion.git
   cd AI-Companion

2. Install Node.js dependencies

   npm install

3. Install Python dependencies (Required for Agent Mode automation)

   pip install -r requirements.txt

4. Configure Environment Create a .env file in the root directory (copy .env.example):

   GOOGLE_API_KEY=your_gemini_key
   ELEVEN_API_KEY=your_elevenlabs_key
   ELEVEN_VOICE_ID=your_voice_id

5. Run the application

   npm start

• "Create a portfolio website" -> Generates a project folder and opens VS Code.
• "What is on my screen?" -> Analyzes the current window content.
• "Open YouTube and search for lofi beats" -> Automates the browser.
• "Type a python script to calculate fibonacci" -> Tyupes code into your active editor.

ai-companion/
├── src/
│   ├── ai/                 # AI logic & GameAgent
│   ├── services/           # Automation & helper services
│   ├── renderer.js         # Frontend logic (PixiJS)
│   └── main.js             # Electron main process
├── assets/                 # Images & sounds
└── package.json            # Dependencies & scripts

To create an installer for your OS:

# Windows
npm run build:win

We welcome contributions! Please see CONTRIBUTING.md for details on how to get started.

This project is licensed under the MIT License.

Made with ❤️ by Kirthan NB & Rohith M