When people talk about "AI changing the world," our brain might be tempted to think there's one mega-entity doing everything. But the reality of our experience at Formant is there's a huge variety of LLMs that are appropriate for many facets of automation, each unique to their own environment and constraints. We're living in an era of specialized artificial minds—each designed for specific tasks, operating at different scales, and optimized for particular use cases. At Formant, we've embraced this diversity out of necessity, building a rich ecosystem of AI tools that work together to push the boundaries of what's possible in robotics.

The Orchestra of Intelligence

We thought it might be interesting to give you all an inner look into all the types of partner technologies we've come to encounter, and help you think about the AI landscape in a different way. We come away seeing the rich need for a team of partner companies working to make LLMs for all the special needs.

Realtime AIs: The First Responders

Our realtime AI systems are the smooth operators—optimized for human interaction (particularly voice) with minimal latency. These models prioritize speed and natural conversation flow over deep reasoning, making them perfect for customer support, initial problem triage, and interactive demonstrations. They're the friendly face of AI that users encounter first, designed to understand context quickly and respond in milliseconds rather than minutes.

When a technician asks "Why did robot #47 stop moving?" during a live demo, these systems instantly parse the question, and start the tool to check recent telemetry, and respond with "Battery dropped to 12% at 2:15 PM" before the awkward silence can set in.

Small but Mighty

Think of these as our "Application-Specific Integrated Circuit" AIs—small, fast, and incredibly efficient at their designated tasks. With lower costs and larger context windows but compact reasoning capabilities, they excel at rapid, iterative execution. Need to make a quick succession of tool calls that use a lot of context and not hit limits? These are your workhorses. They're affordable to run at scale and perfect for the kind of repetitive, high-volume tasks that would overwhelm larger models. Specifically there’s models that run on ASICs that have interesting trade offs in this realm.

Picture processing 10,000 temperature readings from a fleet of delivery robots every minute, automatically flagging the three units running hot and generating maintenance alerts—all for pennies in compute costs.

The Heavy Hitters: Maximum Reasoning Power

When we need to bring out the big guns, we turn to our most powerful AI systems. These models sacrifice speed and cost for depth, offering massive context analysis and sophisticated reasoning capabilities. They're the ones we call when facing complex multi-step problems, strategic planning, or situations requiring deep domain expertise. Think of them as the senior engineers of our AI team—they take their time, but their insights are invaluable. Importantly they can do this while we are on lunch break!

When a warehouse robot fleet starts showing coordinated inefficiencies across multiple sites, these systems can analyze months of operational data, identify subtle patterns in traffic flows, and recommend a complete reconfiguration of patrol routes that increases throughput by 23%.

On-Robot Intelligence: Beyond Traditional Boundaries

Perhaps most exciting are the AI systems running directly on our robots. These aren't just following pre-programmed instructions; they're bringing broader conceptual awareness to robotic systems. They face a hard constraint that they have to not steal resources from important on board AI, but are exceptional for signaling to higher level AIs something important needs to be done that might not have come standard issue with the robots on board AIs. 

A cleaning robot encounters an unexpected spill during a board meeting. Instead of blindly following its schedule, it recognizes the social context according to the organization's context, quietly routes around the room, and indicates up to an organization AI that reschedules that area for cleaning after the meeting ends.

Offline Warriors: Independence from Connectivity

Not every environment has reliable internet connectivity. Our offline AI systems ensure that intelligence doesn't stop when the connection drops. Leveraging open-source models optimized for high-end desktop hardware, these systems maintain critical AI capabilities even in remote locations, harsh environments, or secure facilities where cloud connectivity isn't an option.

Deep in an oil rig operation where cell towers are a distant memory, robots continue analyzing wall gauge samples and making autonomous navigation decisions using locally-running models, ensuring operations never pause for a connection timeout.

Data Archaeologists: Mapping the Information Landscape

Some of our most specialized AIs are designed purely for data exploration and analysis. These systems excel at plowing through vast amounts of information, identifying patterns, and mapping the conceptual space of complex datasets. They're the scouts that go ahead of other AI systems, preparing the terrain and highlighting the most promising areas for deeper investigation. These generally include embedding specific models.

We have 1000 documents that might contain useful context for another AI to do it’s job, let’s chunk these all out and have those chunks of context searchable with their LLM embedding vector.

Visual Specialists: Precision in Perception

Our hyper-specific visual AI systems represent some of the most sophisticated applications of multimodal learning. These models don't just recognize objects—they understand spatial relationships, extract precise x-y coordinates, determine joint states, and interpret complex visual scenes with remarkable accuracy. They're the result of specialized training regimens designed to excel in the visual-heavy world of robotics.

When a robotic arm needs to grasp a wrench from a cluttered toolbox, these systems don't just identify "tool"—they provide exact coordinates in a camera, then robot estimates estimate the grip angle needed, and detect that the wrench is partially obscured by cables, adjusting the approach trajectory accordingly.

The Power of Orchestration

The future of AI isn't about building one perfect system—it's about orchestrating many specialized systems that complement each other's strengths and compensate for each other's limitations. This is because humans have a variety of values at any number of situations you might be in through a day. Sometimes you need speed over accuracy, sometimes you need depth over breadth, sometimes you need privacy over connectivity. A single AI system optimized for one set of values will inevitably fall short in scenarios where different priorities take precedence. At Formant, we're not just users of this technology; we're active participants in shaping how these diverse AI capabilities can work together to solve real-world problems in robotics and beyond.

To all our partners in this endeavor—the model providers, the open-source communities, the hardware manufacturers, and the researchers pushing the boundaries of what's possible—we extend our heartfelt gratitude. You're helping pave the way for our industry to build something remarkable.

We're excited by the early conversations with our partners and their challenges in this new era of AI-powered robotics. If you're building LLMs in this space  or want to deploy machines powered by AI, or integrate new LLMs to complement the symphony of AI you already have going in your machines, we'd love to talk. Come join us and help define the new way we work together with AIs!

The AI revolution isn't coming—it's here, and it's beautifully diverse