Etched Unveils High-Speed Sohu Chip for Transformer Models

Etched Unveils High-Speed Sohu Chip for Transformer Models

In 2022, startup Etched showcased their innovative move with a substantial investment in transformers, the powerhouse behind AI models like ChatGPT. This endeavor led to the creation of Sohu, an ASIC chip tailored specifically for transformer models. Unlike conventional GPUs, Sohu's specialization lies solely in transformer tasks, quickly outpacing Nvidia's B200 GPU by almost tenfold.

Sohu's niche design fosters unparalleled efficiency, with its FLOPS capacity garnering over 90% utilization, a stark contrast to the 30% achieved by general-purpose GPUs. Consequently, Sohu exhibits remarkable prowess in transformer-based responsibilities, processing over 500,000 tokens per second in Llama 70B throughput.

Etched propounds the belief that specialized chips akin to Sohu epitomize the future, especially as major AI conglomerates, such as Google and Microsoft, continue to expand transformer technology. The company is fervently intensifying Sohu's production, asserting their confidence in the forthcoming reign of transformers in AI computation markets. They even hint at potential future ASICs for emerging technologies, expressing their readiness to fabricate ASICs for any technology that might challenge transformers.

Key Takeaways

• Etched introduces Sohu, an ASIC chip fine-tuned for transformer models, surpassing Nvidia's B200 GPU in speed by almost tenfold.
• Sohu achieves over 90% FLOPS utilization, significantly surpassing general-purpose GPUs, making it an efficient choice for transformer tasks.
• The chip's design exclusively favors transformers, a strategic approach centered on mathematical operations and the elimination of superfluous control flow logic.
• Etched sees specialized chips like Sohu shaping the future, with substantial production orders already in place.
• The company endeavors to remain at the forefront of the hardware market, ready to evolve with the emergence of new AI models and technologies.

Analysis

Etched's strategic focus on transformer-specific ASICs, such as Sohu, positions them to exploit the escalating dominion of transformer models in AI. This move not only challenges Nvidia's GPU dominance but also harmonizes with the demands of major AI laboratories like Google and Microsoft. Immediate consequences could include a surge in Etched's market share and revenue, while long-term effects may overhaul the AI hardware landscape, favoring specialized ASICs over general GPUs. This development could potentially cause fluctuations in financial instruments tied to Nvidia, and prompt other chipmakers to develop comparable specialized solutions.

Did You Know?

• ASIC (Application-Specific Integrated Circuit):
  • Designed for a specific task or application, ASICs diverge from general-purpose integrated circuits like CPUs or GPUs. Sohu, tailored for transformer computations in AI models, outshines general-purpose chips in efficiency and performance.
• Transformer Models in AI:
  • These dominant neural network architectures, including GPT and BERT, have revolutionized natural language processing due to their adeptness in handling sequential data without the need for recurrent layers.
• FLOPS Utilization:
  • This measure of computer performance, especially in intricate mathematical calculations, underscores Sohu’s exceptional efficiency, distinct from general-purpose GPUs. Achieving over 90% FLOPS utilization underscores the chip's specialization and efficiency in executing designated computations.