Today, Cerebras Systems, the pioneer in high performance AI compute, smashed its previous industry record for inference, delivering 2,100 tokens/second performance on Llama 3.2 70B. This is 16x faster...
Cerebras Inference delivers 2,100 tokens/second for Llama 3.2B 70B -- 16X performance of the fastest GPUs and 68x faster than hyperscale clouds
SUNNYVALE, Calif.: Today, Cerebras Systems, the pioneer in high performance AI compute, smashed its previous industry record for inference, delivering 2,100 tokens/second performance on Llama 3.2 70B. This is 16x faster than any known GPU solution and 68x faster than hyperscale clouds as measured by Artificial Analysis, a third-party benchmarking organization. Moreover, Cerebras Inference serves Llama 70B more than 8x faster than GPUs serve Llama 3B, delivering an aggregate 184x advantage (8x faster on models 23 x larger). By providing Instant Inference for large models, Cerebras is unlocking new AI use cases powered by real-time, higher quality responses, chain of thought reasoning, more interactions and higher user engagement.
“The world’s fastest AI inference just got faster. It takes graphics processing units an entirely new hardware generation -- two to three years- - to triple their performance. We just did it in a single software release,” said Andrew Feldman, CEO and co-founder, Cerebras. “Early adopters and AI developers are creating powerful AI use cases that were impossible to build on GPU-based solutions. Cerebras Inference is providing a new compute foundation for the next era of AI innovation.”
From global pharmaceutical giants like GlaxoSmithKline (GSK), to pioneering startups like Audivi, Tavus, Vellum and LiveKit, Cerebras is eliminating AI application latency with 60x speed-ups:
Cerebras is gathering the llama community in llamapalooza NYC, a developer event that will feature talks from meta, Hugging Face, LiveKit, Vellum, LaunchDarkly, Val.town, Haize Labs, Crew AI, Cloudflare, South Park Commons, and Slingshot.
Cerebras Inference is powered by the Cerebras CS-3 system and its industry-leading AI processor, the Wafer Scale Engine 3 (WSE-3). Unlike graphic processing units that force customers to make trade-offs between speed and capacity, the CS-3 delivers best in class per-user performance while delivering high throughput. The massive size of the WSE-3 enables many concurrent users to benefit from blistering speed. With 7,000x more memory bandwidth than the Nvidia H100, the WSE-3 solves Generative AI’s fundamental technical challenge: memory bandwidth. Developers can easily access the Cerebras Inference API, which is fully compatible with the OpenAI Chat Completions API, making migration seamless with just a few lines of code.
Cerebras Inference is available now, at a fraction of the cost of hyperscale and GPU clouds. Try Cerebras Inference today: www.cerebras.ai.
About Cerebras Systems
Cerebras Systems is a team of pioneering computer architects, computer scientists, deep learning researchers, and engineers of all types. We have come together to accelerate generative AI by building from the ground up a new class of AI supercomputer. Our flagship product, the CS-3 system, is powered by the world’s largest and fastest AI processor, our Wafer-Scale Engine-3. CS-3s are quickly and easily clustered together to make the largest AI supercomputers in the world, and make placing models on the supercomputers dead simple by avoiding the complexity of distributed computing. Cerebras Inference, powered by Wafer-Scale Engine 3, delivers breakthrough inference speeds, empowering customers to create cutting-edge AI applications. Leading corporations, research institutions, and governments use Cerebras solutions for the development of pathbreaking proprietary models, and to train open-source models with millions of downloads. Cerebras solutions are available through the Cerebras Cloud and on premise. For further information, visit www.cerebras.ai or follow us on LinkedIn or X.
Fonte: Business Wire
Successfully completing a Proof of Concept implementation in Athens, the two Italian companies prove that QKD can be easily implemented also in pre-existing…
Eni's VC company invest in the Italian drone company to develop new solutions for industrial plants monitoring
Oracle recognizes Technology Reply’s ability to develop and deliver pioneering solutions through partnering with Oracle
Scheduled for October, the world's largest startup event will bring together more than 2,000 exhibitors in Dubai, UAE
AuditBoard, the leading cloud-based platform transforming audit, risk, compliance, and ESG management, today announced it has been recognized in the 8th…
Princess Juliana International Airport (PJIAE), in collaboration with the Ministry of Justice and Ministry of Tourism, Economic Affairs, Traffic and Telecommunication…
ZenBusiness®, a platform helping small business owners start, run, and grow successful businesses, is advancing its mission to empower entrepreneurs by…
$III #ContinuousProductivityServices--Enterprises in the U.S. are cautiously optimistic about workplace management amid macroeconomic uncertainty and…