Computer experts at Microsoft Research have done a great job of rolling out a new AI model that runs across regular CPUs, instead of the usual GPU.
The news is major because it’s something rare in today's time. The latest efforts are published in a new paper that outlines how exactly the masterpiece came into play, what features are on offer, and performance stats during the testing phase.
Over the past couple of years, LLMs have been the rage, and we’ve seen models from ChatGPT to more get fame for all the right reasons. The idea of chatbots is incredible, and each day, there’s something new on offer.
One thing that they all have in common is the use and training of GPU chips. This has to do with the huge computing power needed for training on such big data collections. Recently, concerns were on the rise about the huge energy consumption used by data centers and how they were more harmful and costly than anything else.
Therefore, the Microsoft team seems to have found the best solution for data processing, and now there’s a model that really proves their claims true. One of the most energy-draining parts on offer here is how weights are stored and how they’re used. There’s so much memory involved, and the same goes for the processing of the CPU.
The alternative approach involved floating numbers and using what they call a 1-bit architecture.
In the latest innovation, the weights get stored as three single values, and this only includes simple math of adding or subtracting for the processing. All of these operations are done through the CPU.
The latest model’s test showed that it was a fierce competition against other GPU models in the same class size. They could outperform a bunch of those, and it would use less memory and less energy.
To run these models, the team produced runtime environments. It’s dubbed Bitnet.cpp, which is designed to use a 1-bit architecture. Now the question if these claims will hold up until the end.
If the answer is yes, then you can say hello to a game-changing opportunity. You don’t need to rely too heavily on the likes of big data companies. Users might run a single chatbot on a PC or perhaps a phone.
Such LLM processing that’s more localized can also better protect privacy and enable performance without any internet connection. How brilliant would that be?
Image: DIW-Aigen
Read next: Apple Accused of Misleading Users about AI Features on iPhone 16
The news is major because it’s something rare in today's time. The latest efforts are published in a new paper that outlines how exactly the masterpiece came into play, what features are on offer, and performance stats during the testing phase.
Over the past couple of years, LLMs have been the rage, and we’ve seen models from ChatGPT to more get fame for all the right reasons. The idea of chatbots is incredible, and each day, there’s something new on offer.
One thing that they all have in common is the use and training of GPU chips. This has to do with the huge computing power needed for training on such big data collections. Recently, concerns were on the rise about the huge energy consumption used by data centers and how they were more harmful and costly than anything else.
Therefore, the Microsoft team seems to have found the best solution for data processing, and now there’s a model that really proves their claims true. One of the most energy-draining parts on offer here is how weights are stored and how they’re used. There’s so much memory involved, and the same goes for the processing of the CPU.
The alternative approach involved floating numbers and using what they call a 1-bit architecture.
In the latest innovation, the weights get stored as three single values, and this only includes simple math of adding or subtracting for the processing. All of these operations are done through the CPU.
The latest model’s test showed that it was a fierce competition against other GPU models in the same class size. They could outperform a bunch of those, and it would use less memory and less energy.
To run these models, the team produced runtime environments. It’s dubbed Bitnet.cpp, which is designed to use a 1-bit architecture. Now the question if these claims will hold up until the end.
If the answer is yes, then you can say hello to a game-changing opportunity. You don’t need to rely too heavily on the likes of big data companies. Users might run a single chatbot on a PC or perhaps a phone.
Such LLM processing that’s more localized can also better protect privacy and enable performance without any internet connection. How brilliant would that be?
Image: DIW-Aigen
Read next: Apple Accused of Misleading Users about AI Features on iPhone 16
