Skip to main content

What is NeoMind?

NeoMind is an edge-deployed AI platform that brings intelligence to IoT. It runs LLM-powered agents directly on your hardware, connects to devices via MQTT / BLE / Webhook, automates responses through a rule engine, and visualizes everything on real-time dashboards — all without relying on cloud services.

Key idea: Talk to your devices in natural language. The AI understands your intent, queries device states, creates automation rules, and takes action autonomously.

Product at a Glance​

Three core surfaces of NeoMind — manage your devices, visualize your data, and drive everything through natural language.

Device management — unified MQTT/BLE/Webhook devicesReal-time dashboard — drag-and-drop builder, WebSocket live updatesAI Chat — query devices and create automations in natural language

Product Architecture​

NeoMind uses a single-process, multi-layer architecture — all core capabilities are packaged in one process, with no external database or message broker required. It's ready the moment you start.

Three Design Philosophies

  1. Single-Process, Self-Contained — API, MQTT broker, storage, and rule engine all live in one process. A single cargo run and everything is ready. No Docker compose, no external dependencies.

  2. Edge-First, Cloud-Optional — Defaults to local LLMs (Ollama) for 100% offline operation — data never leaves your LAN. When you need more power, switch to cloud models (OpenAI / Claude / GLM) with one click.

  3. Crash-Isolated Extensions — Extensions run in separate processes and communicate via FFI. A YOLO extension crashes? The main service and other extensions are completely unaffected.

Want to go deeper into each layer's design? Read Core Concepts — full breakdown of the data lifecycle, extension model, and agent execution loop.

Why NeoMind?​

FeatureDescription
Fully self-containedEmbedded MQTT broker, redb storage, no external database or broker to install
Type-safe end-to-endRust backend with compile-time guarantees; agent CLI commands dispatch in-process with structured data, no fragile string parsing
Crash-proof extensionsExtensions run in isolated processes with capability-based permissions; a misbehaving extension never takes down the server
Cloud-optionalWorks 100% offline with local LLMs (Ollama), or connect cloud models when you need more power

Core Capabilities​

AI Intelligence​

  • Built-in local AI, zero setup — one-click curated model download in the wizard (Qwen 3.5 / Gemma 4 etc.), no API key, no Ollama; GGUF import supported
  • Natural Language Chat — Conversational interface to query and control all connected devices
  • Autonomous Agents — Scheduled or event-driven AI agents that monitor, analyze, and act on device data independently
  • 10+ LLM Backends — Ollama, llama.cpp (with the platform's built-in local models, one-click download and ready to use), OpenAI, Anthropic, Google, xAI, Qwen, DeepSeek, GLM, MiniMax, and any OpenAI-compatible endpoint
  • Memory System — Cross-session memory (user profile / knowledge base + execution journals), auto-extracted and persisted after each conversation
  • Skill System — YAML + Markdown skill files that guide agent behavior for specific scenarios
  • Multimodal — Image upload and visual analysis support

Device Management​

  • MQTT Protocol — Primary device integration with embedded broker, mTLS, and CA certificate support
  • BLE Provisioning — Zero-touch device setup via Bluetooth (Tauri native + Web Bluetooth)
  • HTTP / Webhook — Flexible REST-based device adapter
  • Auto-Discovery — Automatic device detection, type registration, and AI-assisted onboarding
  • Command Queue — Send control commands to devices with parameter validation and tracking
  • Custom Device Types — Define device metrics and commands via JSON type definitions

Automation​

  • JSON Rule Engine — Structured rule definitions: {"condition": {"source": "device:sensor:temperature", "operator": "greater_than", "threshold": 30}}
  • Data Transforms — JavaScript-based data transformation for creating virtual metrics
  • Scheduled Agents — Time-based or event-driven AI agent execution
  • Event Bus — Pub/sub architecture for decoupled component communication

Dashboards & Visualization​

  • Drag-and-Drop Builder — Visual dashboard editor with responsive grid layout
  • Rich Widgets — Value cards, charts, gauges, tables, VLM vision components
  • Real-time Updates — WebSocket / SSE for live data streaming to dashboards
  • Dashboard Sharing — Public links with expiration
  • Custom Components — Build and publish your own dashboard widgets

Voice & Multimodal​

  • Real-time voice assistant — mic → VAD → ASR → LLM → TTS, fully local end-to-end
  • ASR / TTS — SenseVoice 5-language ASR; three interchangeable TTS engines incl. voice cloning
  • Vision AI extensions — object detection, OCR, face recognition, visual grounding, video VLM

Industry & Protocol Integrations​

  • Industrial protocols — Modbus / OPC-UA / BACnet bridges for PLCs, meters, and BACnet controllers
  • LoRaWAN — ChirpStack / TTN sensor auto-discovery with downlink control
  • Home Assistant — two-way sync with 3000+ smart-home entities

Notification & Data Push​

  • 7 Notification Channels — Webhook, Email, Telegram, WeCom, DingTalk, Slack, Feishu, plus the in-app notification center
  • IM Bridges — Chat with AI agents directly in Telegram / Feishu (independent from the notification channels)
  • Data Push — Forward telemetry data to external systems via Webhook or MQTT
  • Delivery Tracking (Data Push) — Exponential backoff retry, delivery history, and log management
  • Message Deduplication — Prevent notification storms from high-frequency triggers

Platform​

  • Multi-Instance — Connect to and manage multiple NeoMind backends from a single interface
  • Extension System — Native & WASM extensions with process isolation and capability-based permissions
  • Cross-Platform Desktop — macOS, Windows, Linux native apps via Tauri
  • Mobile-Friendly Web — Responsive web UI optimized for phone and tablet
  • i18n — English and Chinese
  • Dark Mode — System-aware dark/light theme
  • API Key Auth — Alternative to JWT for programmatic access
  • CLI Tools — Full-featured command-line interface

Ecosystem​

NeoMind is a modular ecosystem with specialized repositories for each concern:

RepositoryPurpose
NeoMindCore platform — backend, frontend, desktop app
NeoMind-ExtensionsOfficial extension marketplace — weather, YOLO detection, OCR, face recognition, streaming
NeoMind-DeviceTypesDevice type definitions — standardized metrics and commands for IoT hardware
NeoMind-Dashboard-ComponentsDashboard widget marketplace — community-contributed React components

Who It's For​

  • IoT integrators / solution engineers — Need to rapidly build edge intelligent solutions connecting cameras, sensors, and controllers with automation
  • Industrial / campus / retail operators — Want to manage devices, configure alerts, and visualize data in natural language
  • Secondary developers — Extend the platform via the Extension SDK, custom device types, or dashboard components
  • AI application engineers — Run multimodal LLM agents at the edge, connected to real physical devices

Next Steps​

Applies to: v0.9.x (Sep 2026)

Last updated: 2026-09-08