‘Team Red,’ ‘Team Blue,’ and ‘Team Green’ — United in Objective
Generative AI took the spotlight at CES 2024
During the Consumer Electronics Show of Las Vegas, I write several columns describing my favorite technologies of the event, and this year is no exception. Today’s column is all about the three computer processor companies — Intel, Advanced Micro Devices, and Nvidia — and how they all are enabling new on-device generative artificial intelligence features for consumers with faster tensor processing units, neural processing units, and software enhancements.
To power generative AI features in consumer-grade desktop and laptop personal computers, AMD, Intel, and Nvidia — the three major computer processor manufacturing companies — focused heavily on NPU horsepower in their latest processors. While AMD and Nvidia held their CES press conferences on Monday, Intel is saving its address with Pat Gelsinger, its chief executive, for Tuesday. Still, the company announced new mobile gaming processors that still bear the “i” names — i5, i7, and i9 — for laptops. Meanwhile, AMD announced new 8000-series desktop processors with integrated graphics and improved “XDNA” NPUs to power “Ryzen AI,” AMD’s brand name for its suite of AI features. Nvidia announced new “Super” versions of its popular “RTX” line of graphics cards, namely the RTX 4080 Super, replacing the current RTX 4080; the RTX 4070 Super, which sits alongside the existing RTX 4070; and the RTX 4070 Ti Super, which replaces the now-outgoing RTX 4070 Ti. The most exciting additions to these graphics cards — aside from price reductions — are new generative AI features and improved video encoding for video game streamers.
Advanced Micro Devices
AMD began the day’s CES announcements with two new series’s of desktop processors: the flagship 8000-series, which is built for AI, and lower-end 5000-series desktop processors built on the company’s older “Zen 3” processor architecture — and a new lower-end graphics card called the Radeon RX 7600 XT, aimed at competing with Nvidia’s RTX 4060.
The 8000-series processors come with a maximum of eight cores built on AMD’s new “Gen 4” architecture, 16 threads, and a 5.1-gigahertz boost clock speed on the highest-end Ryzen 7 8700G. The highest-end offering also comes equipped with a Radeon 780M integrated graphics processor. However, the notable news is the addition of AMD Ryzen AI, powered by the XDNA NPUs, which AMD says is meant to power Windows AI features such as Microsoft Copilot and image generators. AMD did not name all of these features, however, as it is expected that Microsoft will announce a newer version of Windows — dubbed “Windows 12” — with a heavier focus on AI capabilities and integration with the operating system. These new processors will be ready to handle the demands of those software features when they inevitably launch. In the meantime, AMD announced support for Zoom, Blender, and Adobe AI features.
In addition to the highest-end 8700G, AMD announced three lower-end offerings, all suited for “entry-level gaming and productivity systems,” AMD said during its press conference, held virtually. The Ryzen 5 8600G also comes equipped with Ryzen AI akin to its more expensive counterpart, but only ships with six cores, 12 threads, and Radeon 760M graphics with a 5-gigahertz boost clock speed — a measure of how many times a processor generates electric pulses to power the computer. The Ryzen 5 8500G does not come with Ryzen AI but is otherwise the same as the 8600G, aside from the graphics component, which has been downgraded to a Radeon 740M. Finally, the lowest-end Ryzen 3 8300G comes with two fewer cores, eight threads, a 4.9-gigahertz clock speed, and the same Radeon 740M graphics, though is only available in partner systems coming later this year.
AMD touted the highest-end of the 8000-series processors as having “the fastest desktop PC processor graphics in the world,” though its claims could not be immediately verified as the press was not granted access to hardware in time for testing. However, the company did show some benchmarks during its press conference, where it highlighted four times higher frame rates in certain video games than rival Intel’s Core i7–14700K processor — a mid-range offering.
AMD’s 8000-series processors cost $330 at their most — in the case of the 8700G — and all ship on January 31 worldwide. The lowest-end Ryzen 3 processor will be available in pre-built machines from AMD partners later this quarter.
The 5000-series processors, meanwhile, were less interesting. These processors — built on an older silicon architecture called “Zen 3” — do not come equipped with very powerful integrated graphics and have slower clock speeds and cores. They begin at $250 with the Ryzen 7 5700X3D — which lacks integrated graphics — and go down to $125 in the case of the low-end Ryzen 5 5500GT. Since these processors are built on Zen 3 and use the “AM4” motherboard socket, they also do not come equipped with Ryzen AI — but that is to be expected at such low prices. These refreshed processors will also be available starting January 31.
Intel
Intel announced its 14th Generation mobile processors, as well, made for gaming-focused systems. Currently, Intel’s processor lineup is a bit convoluted: These new offerings are lower-end than the previously introduced “Core Ultra” lineup of processors, which are more expensive and draw more power, though the models announced today still offer impressive performance — especially for gaming laptops, as those typically focus more heavily on graphics performance. These new mobile units bear the “i-series” monikers, such as the i5, i7, and i9, unlike the Core Ultra models, which use a separate naming scheme.
The new processors presented today are strictly for gaming, not AI — meaning that they do not have NPUs. But the Core Ultra processors announced last month — and that have been put in new computers announced at CES on Monday — do. The NPUs are supposed to make it more efficient to run AI models, such as Stable Diffusion, on-device, while not impacting standard computing performance. In other words, running a large language model or text-to-image model on-device will not affect other applications and tasks on the computer since the NPU is a separate component of the processor. Not many people are running Stable Diffusion models on their laptops very often, even in 2024, but the new NPUs are evidently to assist Windows AI tools rumored to be arriving later this year — just like in the case of AMD.
Monday’s processor announcements, however, are for high-performance gaming. The new models are a refresh of last year’s “Raptor Lake” 13th Generation mobile processors and come equipped with higher core counts, an increased focus on efficiency, and higher clock speeds. The flagship Intel Core i9–14900HX ships in high-end gaming laptops and has 24 cores — using Intel’s big-little architecture, with eight “performance” cores and 16 “efficiency” cores — and up to a 5.8-gigahertz turbo clock speed. All new models also support up to 192 gigabytes of DDR5 memory, now.
On the Core i7 front, the Core i7–14700HX now comes equipped with 20 cores — eight performance, and 12 efficiency — which is four more efficiency cores than last year’s model. These processors are also more power-efficient, Intel says. Intel compared the i9–14900HX to rival AMD’s flagship Ryzen 9 7945HX, and according to the company’s claims, the processor beats AMD’s offering in most scenarios. However, Intel was shy to compare the new 2024 models to last year’s 13th Generation processors, possibly indicating no real performance gains. And the company did not compare the i7 models to any other processors.
These new processors are shipping in new laptops that also debuted simultaneously at CES 2024. Those laptops are not particularly newsworthy, so I won’t cover them here.
Nvidia
Nvidia announced three new graphics cards, all for desktop computers, during its press conference. The new cards all have a focus on AI development, including a new “Chat with RTX” software feature in which an LLM runs on the graphics cards. At the beginning of its keynote address, held virtually, the company touted how its AI-focused graphics cards are being used to support the new frontier of AI development, and company presenters said that Nvidia wants to bring AI to consumers next.
The new “Super” graphics cards announced Monday offer better performance at similar prices. The RTX 4080 Super, the highest-end card of Monday’s showcase, replaces the existing RTX 4080 model and adds more Compute Unified Device Architecture, more commonly known as CUDA, cores. The video memory bandwidth has also been expanded to 736 gigabytes per second, and the card ships with 16 GB of video memory by default. The entire package draws 320 watts of power, Nvidia says, and comes equipped with version three of Nvidia’s deep learning super sampling technology, which renders graphics at a lower resolution, then upscales the lower-quality graphics with AI as if the image had been originally rendered in high quality. This allows for higher-quality textures and graphical settings, and higher frame rates in games that support the technology. The latest version, called DLSS 3, is the best version of the technology. The most noteworthy — and welcome — announcement, however, is a $200 price reduction; the RTX 4080 Super now costs $1,000 for a “Founders Edition” card, which comes in a matte black color and launches on January 31.
The RTX 4070 Ti Super replaces the RTX 4070 Ti and begins at the same $800 price. Nvidia has upgraded the video memory to 16 GB, similar to the RTX 4080 Super, which should provide better performance. Nvidia claims that more memory, combined with the upgraded 256-bit memory bus as found on the RTX 4080 and 4090 cards, will provide a roughly 10 percent performance improvement, which is remarkable for the same price. Nvidia also says that the RTX 4070 Ti Super draws the same amount of power as previous models — 285 watts — but that there will not be a Founders Edition version of the card.
Lastly, Nvidia announced the RTX 4070 Super, which sits alongside the existing RTX 4070 in Nvidia’s graphics card lineup. Nvidia claims that the new RTX 4070 Super will be more performant than the three-year-old RTX 3090 — the top-of-the-line graphics card in 2020 — while using less power: 220 watts. The RTX 4070 Super retains the 12 GB of video memory and 192-bit memory bus from the RTX 4070, but adds 20 percent more CUDA cores, according to Nvidia. Nvidia also touted the RTX 4070 Super as having “568 AI TOPS,” referring to terra operations per second, or how many computing operations an AI chip can perform each second.
All three of the new RTX cards enable all-new AI-powered features, including the new “Chat with RTX” feature. Because of the more powerful tensor cores — units used to assist in AI processing — Nvidia highlighted the new graphics cards’ abilities to run LLMs and text-to-image generators, like Stable Diffusion, on the device itself, without having to send queries to other Nvidia AI chips via the internet. (Nvidia really does have a hold over this market.) Chat with RTX uses a custom Nvidia-designed language model to process queries on RTX-series graphics cards. Nvidia calls Chat with RTX a “playground” for enthusiasts to “connect an RTX-accelerated LLM to their own data, from locally stored documents to YouTube videos.” This means that users will be able to use their own files on their computer as context for their queries — all while maintaining privacy since none of the data is sent to other servers. The company also said that it is using a technique called retrieval-augmented generation, which increases the accuracy of LLMs to prevent hallucinations, or misinformation, from being generated by the program.
Alongside its own generative AI technologies, coming “later this month” according to the company, it also announced TensorRT acceleration for Stable Diffusion, the leading text-to-image model. This new technology, running on the graphics units themselves, provides for up to a 60 percent boost in performance, the company said. And a new technology called “Avatar Cloud Engine,” in the company’s words, “brings digital avatars to life with generative AI.” According to a blog post published by the company, A.C.E. gathers audio input from the user, passes it to a speech recognition model made by Nvidia, then passes it to an animation model to create “a realistic lip sync.” The new character is then rendered into the game. Game developers will have to add support for A.C.E., but the technology is fascinating, and more or less feels like a proof of concept. A.C.E. can run on RTX graphics cards and in the cloud, Nvidia said.
Nvidia also announced an unusual partnership with Getty Images, the stock image website, during its Monday press conference. The two companies are announcing a service called “Generative AI by iStock,” which is meant to generate stock images. The service uses “Nvidia Picasso,” an AI text-to-image model, and the model is specially trained on other stock images, ensuring no trademark violations. The generative AI-powered service is available for users to try now, and users will be able to buy 100 credits for $15.
Finally, Nvidia announced a partnership with the video game live-streaming website Twitch to enable 4K video streaming for the first time, powered by the AV1 video codec. 4K video streaming is currently being “experimented with,” the companies say and might be available on RTX 40-series cards later in the year. The two companies also announced a beta feature where streamers will be able to broadcast up to three concurrent Twitch streams simultaneously using RTX graphics cards all at different resolutions and qualities so “each viewer gets the optimal experience.”
Many of the new features — DLSS, A.C.E., Generative AI by iStock, and Chat with RTX, in particular — are powered by generative AI for the first time, and many of these new technologies also run on-device. Nvidia clearly is betting on generative AI being the future of computing.
Chip refreshes — that is, updates to core counts and clock speeds — are nothing new, especially at CES. However, the focus on hardware-accelerated AI is indicative of a broader theme at this year’s conference. Nearly every laptop and desktop manufacturer that debuted these processors in their new offerings also announced at CES touted the new computers’ AI capabilities, including the newfound ability to run large language models on-device, rather than sending queries to a server farm run by OpenAI or Microsoft. Generally, whenever a trend like this sweeps the CES show floor, it is safe to assume that it will not stick, and so I express caution when assuming that any of these AI features will be popular or well-received amongst the general public.
Some of the new generative AI programs are extremely fascinating proofs of concept and appeal to nerdy people who are interested in experiencing the cutting edge of technology. However, broadly, I am unsure if any of these products will go mainstream as these technology giants would like them to be. Perhaps Windows 12 will change my mind, as hardware always compliments software, but currently, there just is not a tangible enough benefit to the end-user customer experience brought by these new hardware-accelerated AI engines.
This article is part of my CES 2024 coverage. More is coming soon: stay tuned.
