OpenAI just shipped its first custom inference chip. Cool, but I'm not sold.
Here’s the problem with hardcoding AI into silicon: the models are still improving too fast. You bake a chip around today’s architecture and it’s dated before it ships. Inference chips will get more common, sure. But chasing a moving target with hardware is a tough bet.
You know what I’d build instead? A translation chip. Fast, local, on-device.
I was just in Japan, leaning on Google Translate’s camera mode to read menus and signs in real time. It kept crashing. The one moment I needed it most, standing in a restaurant pointing my phone at a menu, and it choked. That’s not a model problem. That’s a “doing too much, too far away, too slowly” problem.
Language doesn’t change quickly. English is English. Japanese is Japanese. That’s a stable target, which is exactly what you want when you’re committing something to hardware. Put it on a dedicated chip in the phone. No cloud round trip, no latency, no privacy tradeoff. Just instant translation in your ear or through your camera, every time.
🌶️ Hot take: the best use of an AI chip isn’t running the biggest model. It’s nailing the one job that doesn’t change. Translation is sitting right there, and my crashing camera app says nobody’s nailed it yet.