More people are running AI chatbots on their own computers for privacy

In short: A growing number of AI users are choosing to run large language models on their own computers instead of using cloud services like ChatGPT.

What's going on

Large language models, or LLMs (the text making “brain” inside chatbots like ChatGPT), usually run on company servers in the cloud. That means your prompts and files are sent over the internet to be processed.

Wired reports that many of these models can also be downloaded and run locally, meaning on your own Windows PC, Mac, or Linux computer. People do this mainly for offline use and for privacy, since your questions and documents do not have to be sent to someone else’s servers.

Cost is another reason. When you run a model locally, you can avoid monthly subscriptions and usage limits. The tradeoff is that local models are often slower and less capable than the best paid cloud options, and you may need to handle updates yourself.

Hardware matters. Wired says 8 GB of RAM is a bare minimum, 16 GB is better, and 32 GB or more helps if you want to run larger models. A dedicated graphics card, especially on Windows, can also help because it gives the model more room to “think” (like a bigger workbench for a complex project).

On the software side, people use apps that act like the chat window and manager for models. Wired highlights LM Studio Bionic as a free option for Windows and macOS, and also mentions tools like Ollama and GPT4All. Models can be downloaded from places like Hugging Face, which hosts millions of options.

What to watch

Local AI is becoming easier to set up, but it still requires some tinkering and enough computer power. Expect more tools that simplify downloads, updates, and choosing the right model, especially for people who want private, offline assistants.

Source: Wired