Fun AI Lab

You finish a two-hour architectural review meeting, minimize six browser windows, and immediately forget the exact API parameter your lead architect suggested for the authentication pipeline. You search your quick notes, check Slack, and scan terminal history—nothing.

Human working memory is notoriously volatile. We spend hours filtering information through screens, meetings, and voice calls, expecting our brains to act as high-throughput database indexers. They do not.

This is the exact operational void that Omi targets. Built by BasedHardware, Omi is an open-source AI second brain designed to record, index, and organize everything you hear and see across your desktop, phone, and wearable hardware.

---

The Problem with Context Fragmentation

Knowledge workers suffer from severe context switching. Every conversation, meeting, Zoom call, terminal session, and browser tab carries operational context. When you close those tools, that context evaporates or ends up scattered across disparate apps like Notion, Slack, Apple Notes, and Jira.

Traditional note-taking demands manual effort:
1. Stop thinking about the problem to write down notes.
2. Summarize key takeaways after the call ends.
3. Organize notes into folder hierarchies that you will likely never open again.

When you need that specific detail three weeks later, keyword search fails because you cannot remember whether it was typed in a document, spoken in a call, or rendered on your screen.

Omi automates this entire pipeline by converting passive environmental input—ambient microphone audio, screen pixels, and platform events—into structured vector databases that an AI chat model can query instantly.

---

What is Omi?

At its core, Omi is a full-stack, open-source context retrieval ecosystem. It runs silently in the background across macOS, Windows, iOS, Android, and specialized BLE wearable devices.

┌─────────────────────────────────────────────────────────────┐
│                       INPUT CAPTURE                         │
│   [ Microphones ]      [ Screen Pixels ]     [ Wearables ]  │
└──────────────────────────────┬──────────────────────────────┘
                               │
                               ▼
┌─────────────────────────────────────────────────────────────┐
│                      OMI ENGINE STACK                       │
│  - Real-time Audio Transcription                            │
│  - Optical Screen Processing & Context Tracking             │
│  - Summarization & Action Item Extraction Engine            │
└──────────────────────────────┬──────────────────────────────┘
                               │
                               ▼
┌─────────────────────────────────────────────────────────────┐
│                        USER ACTIONS                         │
│   [ Natural Language Chat ]    [ Automated Task Lists ]     │
└─────────────────────────────────────────────────────────────┘

Instead of requiring you to open a prompt box and manually feed information to an LLM, Omi sits at the operating system level:
* Audio Layer: Continuously transcribes spoken audio into clean text.
* Visual Layer: Captures display activity to correlate what you were looking at when a conversation happened.
* Intelligence Layer: Passes raw streams into language models to generate real-time summaries, action items, and conversational recall capabilities.

With over 300,000 users and 13,000+ GitHub stars, Omi offers a fully open, transparent alternative to closed-source options like Rewind AI or Limitless.

---

System Architecture: How Omi Processes Your Day

To understand how Omi handles dense audio and visual input without draining systems, examine its data ingestion pipeline:

graph TD
    A[Wearable Device / BLE Audio] -->|Raw Audio Stream| D[Omi Backend Service]
    B[macOS / Windows App] -->|Screen Frames & System Audio| D
    C[Mobile App Flutter] -->|Voice Notes & Mobile Input| D

    D --> E[Speech-to-Text Pipeline]
    D --> F[Frame Indexing Engine]

    E --> G[Context Aggregator & Storage]
    F --> G

    G --> H[LLM Summarization & Action Extraction]
    G --> I[Vector Memory Index]

    H --> J[Structured Workspace / Dashboard]
    I --> K[Interactive AI Chat Interface]

1. Ingestion Clients

• Desktop (macOS / Windows): Native client apps built in Swift (macOS) and Node.js/Electron (Windows) gather desktop screen captures and system audio pipelines.
• Mobile (Flutter): Provides cross-platform access on iOS and Android to manage settings, read summaries, and record conversations on the go.
• BLE Wearables: Connects via Bluetooth Low Energy (BLE) to hardware devices (like the Omi necklace/wearable kit), feeding ambient room voice streams directly into the pipeline.

2. Processing & Memory Engine

The Python backend manages heavy audio processing through real-time Speech-to-Text (STT) models. Screen frame metadata is aligned with timestamped transcriptions, allowing the engine to construct a cohesive timeline of your activities.

3. Query & Extraction Interface

Once stored, an interactive chat interface exposes your historical memory. You can ask queries like: “What were the three edge cases Sarah mentioned during our design call this morning?” or “What terminal command did I run to fix the Docker build error yesterday?”

---

Core Features Breakdown

Real-Time Audio Transcription

Omi records microphone input and processes it through speech-to-text models. It handles multi-speaker dialogue, separates background noise, and indexes transcripts automatically.

Automated Action Item Generation

Instead of forcing you to review long transcripts, Omi isolates actionable tasks. When someone says, “I’ll update the database schema by Thursday,” Omi extracts this phrase, categorizes it as an action item, and makes it available in your dashboard.

Context-Aware Screen Capture

By monitoring your active screen, Omi links spoken words to visual elements. If you are reviewing a Figma file while discussing UI changes, Omi ties the transcription timestamps directly to the visual work session.

Cross-Platform Hardware Support

Whether you prefer running software quietly on your MacBook Pro, capturing voice notes on your phone, or using a dedicated wearable hardware pendant, Omi syncs data across all endpoints seamlessly.

---

Omi vs. Proprietary AI Second Brains

The AI memory assistant space is growing rapidly. Here is how Omi compares to traditional note-taking applications and proprietary desktop memory recording software:

Feature	Omi (BasedHardware)	Limitless / Rewind	Notion / Apple Notes
License Model	Open Source (MIT)	Proprietary Closed Source	Proprietary Closed Source
Hardware Wearable Support	Yes (Native BLE Hardware)	Limited (Pendant only)	None
Passive Ingestion	Audio + Screen + Mobile	Audio + Screen	Manual Entry Only
Custom Backend / Extensibility	Full Source Code Access	Locked API / Closed Cloud	Standard REST APIs
Data Sovereignty	Self-Hostable Option	Vendor Cloud Controlled	Vendor Cloud Controlled

---

Hands-On Setup Guide

Omi is straightforward to build locally, whether you want to test the macOS native client, launch the Windows Electron app, or compile the Flutter mobile app.

Setting Up Omi on macOS

Requirements: macOS 14+, Xcode (for Swift compilation and local signing), and Node.js.

Run the following quickstart command directly in your terminal:

git clone https://github.com/BasedHardware/omi.git && cd omi/desktop/macos && ./run.sh --yolo

This single-line execution clones the repository, navigates into the desktop build directory, compiles the Swift components, binds dependencies, connects to the cloud backend, and launches the app interface.

Running Omi on Windows

Requirements: Node.js installed.

Open PowerShell and execute:

# Clone the repository
git clone https://github.com/BasedHardware/omi.git

# Move into the Windows client directory
cd omi\desktop\windows

# Install dependencies and launch configuration
npm install
copy .env.example .env
npm run dev

Full Local Development Setup (Backend Stack)

If you want to modify backend services or host your own Python orchestration engine, run the root baseline setup:

# Verify prerequisites
xcode-select --install
uv --version

# Clone and configure environment
git clone https://github.com/BasedHardware/omi.git
cd omi
make setup

# Configure local backend environment variables
cd omi/desktop/macos
cp ../../backend/.env.example ../../backend/.env
./run.sh

Mobile App Deployment (iOS / Android)

For developer testing on mobile platforms using the Flutter client:

cd app
# For iOS Target
bash setup.sh ios

# For Android Target
bash setup.sh android

---

Real-World Use Cases

1. The Software Architect

• Scenario: You participate in multiple architecture calls across the day, discussing system designs, API payloads, and database migrations.
• How Omi Helps: You don’t need to manually transcribe whiteboard sessions or stop screen sharing to take notes. Omi transcribes the conversation, indexes your IDE and browser windows, and builds a searchable reference map. Later, you can ask Omi for the exact database migration steps discussed in the morning session.

2. Product Managers & Engineering Leads

• Scenario: You run back-to-back sprint planning meetings and need to log action items across multiple teams.
• How Omi Helps: Omi continuously parses spoken dialogue, separates speaker inputs, and generates concise lists of assigned tasks. After each call, you can review, edit, and push those tasks directly into your task management workflow.

3. Freelancers & Consultants

• Scenario: Managing client deliverables, billing requirements, and verbal change requests without clear audit trails often leads to scope creep.
• How Omi Helps: Omi preserves an index of client calls, providing a searchable record of agreed-upon changes and deadlines so project scopes stay on track.

---

Common Myths & Misconceptions

Myth 1: “Always-on audio recording destroys battery life.”

Fact: Omi offloads processing tasks efficiently. The desktop clients use native low-overhead hooks (like Apple’s Swift native frameworks on macOS), while audio processing is handled cleanly using optimized local streaming or backend API offloading.

Myth 2: “You must buy specialized hardware to use Omi.”

Fact: Hardware is optional. While BasedHardware designs custom wearable hardware (such as BLE voice necklaces), the entire platform functions smoothly as a software-only suite on macOS, Windows, iOS, and Android.

Myth 3: “My screen and microphone data are sold or used for public training.”

Fact: Omi is published under the open-source MIT License. You can inspect the source code to see how data is handled, audit API calls, or self-host backend components to maintain absolute control over your private information.

---

Actionable Plan to Maximize Your Omi Setup

To get the best performance from Omi, follow this setup approach:

1. Start with the Desktop Client: Install Omi on your main workstation first. Run it during standard meetings to adjust microphone sensitivities and transcription preferences.
2. Set Up Keyword Filtering: Customize how Omi captures data by configuring workspace rules to exclude sensitive applications (like password managers or personal banking sites).
3. Establish an Action Item Routine: At the end of your workday, ask Omi: “What open action items were assigned to me today?” Use this generated summary to organize your tasks for the next morning.
4. Connect Mobile Endpoints: Install the mobile app on iOS or Android to keep your context memory in sync when you step away from your desk.

---

The Bottom Line

The limits of human working memory shouldn’t restrict how effectively you manage information. Omi turns messy, scattered daily interactions into an organized, searchable memory layer.

Because it is fully open-source, flexible, and supported by a vibrant community, Omi lets you build an AI second brain that fits your specific needs—keeping you in complete control of your data.

📂 Explore the open-source repository on GitHub: https://github.com/BasedHardware/omi