Edge artificial intelligence：Distributed computing

As the application and complexity of artificial intelligence (AI) continue to increase, people often question the sustainability of the traditional model of using large centralized data centers. Today, hyperscale data centers handle the majority of AI workloads, but they have high energy demands and environmental costs.

Sam Altman of OpenAI pointed out in a recent blog post that on average, a ChatGPT query consumes "approximately one-fifteenth of a teaspoon" of water and "about 0.34 watt-hours" of electricity. Multiply this number by billions of queries and you will start to see the scale of the problem.

But what if there are other methods? Edge AI does not need to send data over long distances to the cloud for processing. Instead, it runs tasks closer to the source of the data, such as in your mobile phone, car or factory workshop. This means a faster response speed, lower energy consumption and higher efficiency. But given that investment in centralized artificial intelligence infrastructure is expected to reach 7 trillion US dollars by 2030, the key question is: Can edge artificial intelligence really take over everything?

The environmental cost of centralized artificial intelligence

The energy demand for artificial intelligence is astonishing and growing rapidly. According to data from Lawrence Berkeley National Laboratory, the electricity consumption of data centers in the United States in 2023 was 176 terawatt-hours, accounting for 4.4% of the country's total electricity consumption. The International Energy Agency (IEA) predicts that by 2030, the energy demand of global data centers will more than double to approximately 945 terawatt-hours, slightly higher than the current electricity consumption in Japan.

The problem lies not only in the use of electricity. The carbon dioxide (CO2) emissions from data centers account for approximately 2% of the global total, which is almost equivalent to the carbon footprint of the entire aviation industry. The cooling system alone accounts for approximately 40% of the total energy consumption, which exerts tremendous pressure on operating costs and environmental goals.

The prospects of edge AI

Edge computing does not need to send data to remote servers but processes it locally. This is particularly valuable for time-sensitive applications, such as self-driving cars or intelligent industrial systems.

Edge nodes can reduce latency, energy consumption and network usage fees. They allow AI models to run in real time without the need for continuous connection to the cloud.

Take self-driving cars like Waymo as an example. They rely on edge AI to process data from sensors such as radar and lidar in real time for navigation and respond promptly to potential safety hazards. Relying on remote servers and always-on Internet connections is too slow and risky.

Small Language Model (SLM) : Edge AI Enabler

The rise of small Language Models (SLM) is one of the main driving forces promoting the development of edge computing. They are designed to be streamlined, efficient and purpotic-specific, and can run on local hardware without an Internet connection, which is different from large models like ChatGPT or Gemini that require powerful computing capabilities.

Due to the lightweight structure of SLM, it can run on smaller chips and is applicable to various technologies, ranging from mobile phones and smartwatches to built-in systems of machines. SLM has lower operating costs, is easier to fine-tune, and has significantly reduced power consumption. Another huge advantage is privacy protection, as the data does not need to leave the device. These SLMS have opened up new possibilities for fields such as the Internet of Things, smart home, logistics, and healthcare.

Energy efficiency of edge data centers

Hyperscale data centers require large cooling systems and backup infrastructure, while edge data centers are often smaller in scale and more flexible. They usually benefit from natural cooling (especially in colder climates), local energy management, and the function of automatic power outages when idle, which are rarely available in hyperscale data centers.

For instance, the dynamic "sleep mode" enables edge infrastructure to shut down power-consuming systems when idle, thereby reducing energy costs and carbon emissions. In addition, edge artificial intelligence deployments typically employ dedicated chips, such as neural processing units (Npus) or application-specific integrated circuits (ASics), which are more energy-efficient than general-purpose cpus or Gpus.

The practical application of edge artificial intelligence

In the field of transportation, truck platooning is one of the most obvious examples of edge artificial intelligence applications. This enables a group of trucks to form a coordinated fleet. By leveraging local sensors and artificial intelligence for real-time communication, trucks keep a distance from each other, thereby reducing wind resistance and increasing fuel efficiency by up to 10%. Without vehicle-to-vehicle communication supported by Edge AI, such automated real-time analysis and decision-making would not be achievable. Due to the need for an Internet connection, traditional cloud processing is too slow and unreliable.

You can see similar benefits in smart grids, retail and manufacturing. From shelf scanning robots in grocery stores to factory machines that can predict their own maintenance needs, edge artificial intelligence is bringing about changes in smarter, more economical and more environmentally friendly ways.

Obstacles to edge AI applications

Although edge AI has many advantages, it still faces many challenges:

Power consumption limitation: Edge devices typically operate in power-constrained environments. Even with optimized chips, intensive models may deplete battery power or overwhelm local infrastructure.

Security vulnerability: Although edge AI enhances privacy protection, it also brings new security risks. Terminal nodes are more vulnerable to physical and network attacks.

Lack of production models and professional knowledge: R&D and engineering have always focused on cloud-based LLMS. Due to the fact that edge AI requires a more specialized skill set, there is a lack of experts and production models.

The hybrid future of AI infrastructure

The future of AI infrastructure may not be either-or. On the contrary, we are moving towards a hybrid model, where training is carried out in large data centers, while reasoning (the actual "thinking") is conducted at the edge. Training AI models requires a large amount of data and computing power. The centralized environment is most suitable for this situation. However, after the training is completed, these models can be deployed to edge positions in a smaller and more compressed form for real-time use.

This balanced model reduces the reliance on the central server, lowers costs, and enhances resilience. It can also ensure that while we pursue a more environmentally friendly and efficient system, we do not sacrifice performance or scalability.

Conclusion: Disruption or diversification?

So, will edge computing disrupt the prosperity of data centers? No, but it will significantly reshape data centers, making them a more diversified, specialized and resilient global infrastructure. Hyperscale infrastructure remains crucial for artificial intelligence training and global-scale services. But edge artificial intelligence will turn what was once science fiction into reality.

Real-time language translation has been implemented in devices such as Google Pixel Buds, which brings us closer to the universal translator that appeared in Star Trek. Advanced home automation systems and robot vacuum cleaners are approaching the vision of "The Jason Family".

As we witness an increasing number of edge AI applications, this transformation will provide a key path for the sustainable expansion of AI. It unleashes the transformative advantages of artificial intelligence without incurring the exponential energy costs of pure cloud-based artificial intelligence.