This week's political scrimmage has been all about data centers. Or maybe I should write that DATA CENTERS. As seems to often happen, very little of the scrimmage on either side is well informed. So, let's get our facts in order.
First, we need some definitions.
The first one, of course, is to ask "what is a data center?" One of the great confusions is that there are lots of different kinds of data centers. Generically, a data center is just a big building full of computers. Most of them either look like big office buildings or big warehouses.
Once upon a time, I toured Rackspace's big master data center in San Antonio, in a building that an even longer time ago had been the AT&T Central Office for the whole San Antonio area. A "central office" was really just another kind of data center, a big building full of the switching equipment that let you dial Aunt Bertha and have your call automatically connected.
When telephone switching went digital in the early 80s, they no longer needed the big buildings, and the folks at Rackspace had the idea to fill the building with acres of chrome wire shelves filled with what were pretty much normal desktop computers. Data centers such as these exist to provide the conventional web-facing business applications — such as the one that provides PJ Media to you now.
The controlling factor for any data center is how many jobs per second it must service, and what kind of jobs they are. The technical term for this is the workload. These old-fashioned conventional data centers service a workload with potentially thousands or even tens of thousands of web requests a second; those web requests arrive relatively randomly throughout the day with peak loads in the afternoon and evening, and quieter times later in the evening and nighttime hours.
But there are other workloads, and that's where the confusion starts. Netflix, HBO, Amazon Prime all have a very different workload, with web requests coming in like any web-facing application, but then needing to support immense amounts of data to provide streaming video without interruptions that make customers unhappy.
Google has another kind of workload. Oh, it has web requests like the others, but under the covers, there are thousands of computers that are running through the internet — crawling — to look at (in theory) all the web pages, downloading them, and doing an amazingly big computation to build the index that is then used to service your search requests.
This means that Netflix, Google, and so on are really servicing two workloads, and that means they need to configure two kinds of data centers.
In the last few years, AI has become a major new workload. We'll talk about it in more detail below, but from a very high level, AI data centers are like conventional data centers. But the AI workload is radically different, and that means the computers are different.
Related: Data Centers and Flock Cameras and AI, Oh My!
You can think of the normal web-facing data center conceptually as a whole bunch of clerks in a massive retail operation. Each clerk gets a request from a customer, and fulfills that request. What the clerks are doing doesn't really have much interaction with the other clerks — they're all just answering requests, more or less independently.
A conventional data center is designed to be scalable, which just means that if you need more capacity, you can easily add more by buying some more computers and wiring them in.
This works up to a point: Eventually you need to be ready to provide more power and more network connections, and more power inevitably means more cooling.
A data center that's designed to handle nearly unlimited scaling is called hyperscale, which is a technical term for "really big and really scalable." But it sounds much more impressive with a Greek-root name.
Hyperscale data centers have been around for years in places such as Google, Amazon AWS, Microsoft Azure, and a few others. But hyperscale data centers aren't really new, and they are all over the world.
When we go inside an AI data center, it is handling a very different workload, so it has to be built differently.
First of all, an AI data center such as for Grok or Google's AI actually handles three different workloads. The one you interact with is the front end, the conventional web-server part. The second workload is called training. The last is called inference, which takes the results of training and uses it to generate answers.
To understand what makes training different, we need to talk a little about how AI models work. I wrote a detailed explanation back in 2023; if you want details, go look at that. The real issue is that what the AI does is radically different from the conventional data center.
The data on which the model is trained — the corpus — is immense, many petabytes (that is, thousands of trillions of characters). This data is preprocessed, first by cleaning up duplication and redundancies, then converted into tokens by splitting the input characters up and assigning them an integer number, for example 13225. Those numbers are the tokens. Think of these as locker numbers: 13225 might match something as simple as "the."
The tokens are then fed into the actual AI model. The old-fashioned name is a neural net, which is nicely described at Ars Technica; if you want to look more deeply, Grant Sanderson's 3Blue1Brown has a nice video. (3blue1brown is excellent overall, highly recommended.)
Training is an immense job. The tokenized input corpus is then fed to the neural net (it's called a "large language model" or "LLM" now) that uses those tokens to adjust the values — it's "turning the knobs" to tune the predictor that does the inference. Training takes input, makes a guess, checks it, and fiddles with the knobs until it gets good at making those guesses.
That means the training is more like a factory. The training machines run for a long time, weeks or months, and run full time, 24/7. It also means the computers doing the training are built differently, with specialized hardware. Those are built around GPUs, what used to be called "graphics processors," or purpose-built specialized processors. It's all doing the same job: a huge number of multiplications. The result is that the neural net is tuned to be good at one job: predicting the next word in an answer to a prompt.
This is the part that spooks people. Too many people imagine there's some kind of an elf in there, a conscious entity that is talking to you.
All it's really doing is predicting what the right next word should be in the answer. So when you use an AI, such as OpenAI's ChatGPT, Anthropic's Claude, or xAI's Grok, what happens is you connect to a web application or an app on your phone. That's the front end, which captures your question (the prompt) and passes it back to the inference engine, that trained and tuned predictor. The prompt is processed to find the most likely tokens to your answer, and then goes to the lockers to turn the tokens back into words.
The point is just that there is no elf anywhere. There's a factory that runs for months turning knobs, and a clerk using the training to predict the next token.
Related: AI Music — Not the Composer, the Orchestra
Related: Using Grok As A Second Brain
I've called Grok my "research partner" in my Sky Candy columns for a while, and have used it to do research and copyediting for this article. Some people ask "what is the use case for these big data centers?"
That is the use case. That's what the data centers are for.
Part Two: Cooling, Water, Power, Land. Answering the objections that make up so much of the controversy, coming soon.
