Generative and agentic AI are mainstream these days. Many solutions are available for free but with limits, while paid access typically provides additional functionality, more performance or newer, more sophisticated models. Adding third-party services to the mix via Model Context Protocol (MCP) support may also incur usage costs, so figuring out what you're spending in a month can be a challenge.

One way companies charge for AI consumption is by the number of tokens used. Tokens can be whole or partial words, punctuation, and even spaces. Pricing tends to be different for input (the prompt) and output tokens; it's based on the model employed. Costs are often specified in millions of tokens. This is assuming text in and text out, which is just one way to work with chatbots. Adding images, videos, and other data just increases the complexity.

Part of the problem these days is that many are using the free versions and haven’t grasped the concept of the actual costs. Or they haven’t thought about what will happen when the free versions disappear or become so limited that they become unusable. The long-term goal of those providing these AI services is to make money — lots of it. The underlying hardware and software in the massive data centers being built need to pay for themselves or someone is going to lose a ton of money in the long run.

Adding to the confusion is the plethora of models and services available. Some systems target software development while others are more general. There are also tools and research designed to reduce the token overhead while still providing the same level of functionality.

The challenge for users and developers is finding a consistent and affordable AI solution. One option is to run large language models (LLMs) and chatbots locally instead of paying for cloud-based solutions.

Local Large Language Models

Systems like OpenClaw bring LLMs and MCPs to local systems. Such possibilities have been around for a long time, but the setup and support weren’t for the novice or uninitiated AI user. This is changing as well, with better user interfaces to manage AI chores locally.

I recently switched over to an ASUS NUC 16 Pro (Fig. 1). The compact PC runs an Intel Core Ultra X7 368H AI PC processor with an integrated GPU and neural processing unit (NPU). The NPU has more power than some of the embedded AI chips designed for edge computing, but it’s less powerful than dedicated GPU/NPU PCI Express cards. Still, the NPU has enough horsepower to handle many LLMs locally.

The ASUS NUC 16 Pro can run ASUS’s AI SuperBuild (Fig. 2). It’s designed to let non-AI experts utilize LLMs locally. This includes built-in Retrieval-Augmented Generation (RAG) support. Having used OpenClaw, I find the ASUS support to be much easier to utilize at this point.

Typically, running models locally doesn’t incur any additional cost. However, various keys are usually required to use models downloaded from the cloud. It’s also possible to have MCPs local as well as accessing them in the cloud. Both open users to security and privacy issues, though that’s beyond this discussion.

SuperBuild is tuned for the ASUS product like that of AI PCs. I found it to be useful for my research with AI, and it can provide useful AI-assisted functionality. It even supports vision language models (VLMs) for local image analysis.

Moving agentic AI is one reason to go local, although even here you’ll find links to the cloud. For example, OpenClaw has plugins and skills that work with home audio tied to the cloud via support for Sonos and Spotify. If you want to see where things might be headed with OpenClaw, take a look at Molty, an AI assistant.

It’s still early for local AI for those who want the performance but without the need to configure applications. A few applications like Adobe Photoshop and Microsoft’s Copilot don’t require any configuration.