Agile Development
The global GPU market was valued at approximately USD 52.1 billion in 2023 and is projected to grow at a CAGR of over 27% through 2032. Yesterday Microsoft released the Bitnet cpp, an open source framework to run inference of 1 bit quantised models in CPU. The framework achieves energy savings of up to 82%, speed improvements ranging from 1.37x to over 6x compared to traditional methods and can support hypothetical 100 Billion parameter models. This makes LLMs more accessible without the need for GPUs to do heavy computations.
The development comes at a time when the sustainability of AI products has been a big concern with the computation heavy, expensive GPUs, heavy energy consumption and fresh water shortages needed in huge quantities for cooling data centers. Companies like Google and Microsoft are already building nuclear reactors to meet their projected power requirements. The ability to rethink the entire hardware architecture gives startups a great opportunity to deliver AI solutions with limited resources and power consumption. At the same time the big tech companies like AWS have already made heavy infrastructural investment in GPUs and cannot afford to have same agility to offer light weight solutions based on ongoing innovations.
Continuous addressal of technical debt form an integral part of ensuring the best quality service or product. The agility to redesign and redevelop parts of your system as you come across better solutions or newer innovations at a steady pace results in solutions that are lean, easier to maintain, more power efficient, performant, adaptable to newer feature requirements and ultimately better user experience. Maintaining legacy codes in big organisations is a big drain on resources and developer motivation. Even though frameworks like React provide a more responsive and modern user friendly experience, 79.2% of all websites are still found in PHP which is even difficult to scale.
Evolving a company’s systems is a stategic decision based on the existing infrastructure, team expertise, market demand of current offerings and the projected value proposition for the time and resources spent on upgrading. This results in different versions of solution being available in the market not all optimized to latest technological advancements.
With AI models running on CPUs, AI powered malicious bots are also expected to cause widespread attack in large numbers. This calls for an urgent need to upgrade every system present online to be resilient to rapidly evolving ecosystem and be a leader in industry standards.
