Start with the Problem, Not the AI
The most important lesson from building GreatChat, GreatStudios, and GreatRouter is this: AI is a means, not an end. Each product started with a clear user problem — teams struggling with fragmented communication, creatives juggling a dozen separate tools, developers wasting time evaluating AI models. AI was the technology that made elegant solutions possible, but it wasn't the product itself. When building AI-first products, resist the temptation to add AI features because they're trendy. Every AI integration should trace directly back to a well-understood user need.
Architecture: Edge-First, Model-Agnostic
The GreatChat architecture reflects a core principle: stay close to users, and never marry a single AI provider. Stateful agents handle long-running conversations, a relational data layer stores workspace records, and the app tier serves requests globally with low latency. Critically, GreatRouter abstracts away the AI provider layer entirely — the application sends requests with model: "auto" and never needs to know which provider is handling the request. This architecture has proven resilient through provider outages, pricing changes, and rapid model evolution.
UX Patterns for AI Products
Designing UX for AI products requires different patterns than traditional software. Key principles we learned: (1) Show your work — AI responses should include reasoning steps, confidence levels, or source citations so users can evaluate quality. (2) Design for fallback — every AI feature needs a graceful degradation path when the model is unavailable or uncertain. (3) Make AI invisible until it's needed — the interface should work perfectly without AI; AI features should enhance, not define, the experience. (4) Respect user control — AI suggestions should be easily accepted, edited, or dismissed; never force AI output on users. These patterns are visible throughout GreatChat's interface, from the Assistant chat mode to the Editor's inline AI toolbar.
Testing AI: Beyond Unit Tests
Testing AI-powered features is fundamentally different from testing deterministic code. Traditional unit tests — "given input X, expect output Y" — don't work when outputs are probabilistic and model-dependent. Our approach combines three strategies: (1) Golden dataset evaluation — a curated set of inputs with expected outputs, run periodically across all available models to detect regressions. (2) Human-in-the-loop review — for critical features, AI outputs are sampled and reviewed by domain experts on a regular cadence. (3) Production monitoring — GreatRouter provides quality metrics, latency tracking, and cost analytics that feed into continuous improvement cycles. The key insight: AI testing is an ongoing process, not a one-time gate.