Sizing up #MI300A’s #GPU
It’s well ahead of #Nvidia’s #H100 PCIe for just about every major category of 32- or 64-bit operations. MI300A can achieve 113.2 TFLOPS of #FP32 throughput, with each FMA counting as two floating point operations. For comparison, H100 PCIe achieved 49.3 TFLOPS in same test.
#AMD cut down #MI300X’s GPU to create MI300A. 24 #Zen4 cores is a lot of #CPU power, and occupies one quadrant on the MI300 chip. But MI300’s main attraction is still the GPU.
https://chipsandcheese.com/p/sizing-up-mi300as-gpu

The 4x #Nvidia #H100 SXM5 server in the new Festus cluster at Uni Bayreuth is the fastest system I've ever tested in #FluidX3D #CFD, achieving 78 GLUPs/s #LBM performance at ~1650W #GPU power draw.
https://github.com/ProjectPhysX/FluidX3D?tab=readme-ov-file#multi-gpu-benchmarks
https://www.hpc.uni-bayreuth.de/clusters/festus/#__tabbed_1_3

#Huawei #HiSilicon #Ascend 910C is a version of the company's Ascend 910 processor for #AI training introduced in 2019. By now, the performance of the Ascend 910 is barely sufficient for the cost-efficient training of large AI models. Still, when it comes to inference, it delivers 60% of #Nvidia #H100 performance, according to researchers from #DeepSeek While the Ascend 910C is not a performance champion, it can succeed in reducing China's reliance on Nvidia #GPU's https://www.tomshardware.com/tech-industry/artificial-intelligence/deepseek-research-suggests-huaweis-ascend-910c-delivers-60-percent-nvidia-h100-inference-performance

Okay, loosing my mind here a bit. I just tested #OpenGL rendering under Linux on an #NVIDIA #H100 GPU, through #VirtualGL's #EGL backend.

And it worked... Renderer "NVIDIA H100/PCIe/SSE2", driver 555.42.06

I always understood the H100s to be incapable of OpenGL. But it seems I missed a crucial part in the H100 architecture doc (https://resources.nvidia.com/en-us-tensor-core), shown in the image.

Except, I'm sure I tested OpenGL at some point under X, but it didn't work. So, did anything change (e.g. driver)?

DeepSeek R1 reproduced for $30: Berkeley researchers replicate DeepSeek R1 for $30—casting doubt on H100 claims and controversy

https://techstartups.com/2025/01/31/deepseek-r1-reproduced-for-30-berkeley-researchers-replicate-deepseek-r1-for-30-casting-doubt-on-h100-claims-and-controversy/

»#DeepSeek Debates: Chinese Leadership On #Cost, True #TrainingCost, Closed Model Margin Impacts #H100 Pricing Soaring, Subsidized Inference Pricing, #ExportControls, MLA.« https://semianalysis.com/2025/01/31/deepseek-debates/?eicker.news #tech #media

NVIDIA pide explicaciones a Super Micro por vender GPU H100 en China a pesar de las sanciones https://blog.elhacker.net/2025/01/nvidia-pide-explicaciones-super-micro-vender-gpu-h100-china.html #estadosunidos #restricciones #supermicro #nvidia #china #h100 #gpu

Global AI Giants GPU Resources Revealed: Over 12.4 Million H100 Equivalents Projected by 2025!

https://www.lesswrong.com/posts/bdQhzQsHjNrQp7cNS/estimates-of-gpu-or-equivalent-resources-of-large-ai-players

Breakthrough in AI-Powered #Audio Generation and Transformation

#Fugatto, developed by #NVIDIA researchers, introduces universal sound manipulation through text prompts, handling music, voice & sound effects simultaneously

Advanced capabilities include accent modification, emotion control, and creation of never-before-heard sounds using #AI technology

Technical specs: 2.5B parameters, trained on #DGX systems with 32 #H100 GPUs, featuring ComposableART for instruction combination

Applications span #music production, game development, advertising & language learning - enables real-time audio asset generation & modification

Developed by international team from India, Brazil, China, Jordan & South Korea, enhancing multi-accent & multilingual capabilities

#ai #ML

https://blogs.nvidia.com/blog/fugatto-gen-ai-sound-model/

https://youtu.be/qj1Sp8He6e4

How To Install One Click, Pre-configured Hugging Face (HUGS) AI Models On DigitalOcean GPU Droplets https://youtu.be/-jwA9FrDLgc #Websplaining #HuggingFace #Hugs #HF #DigitalOcean #Droplet #GpuDroplets #GPU #OneClickAiModels #AI #AiModels #ML #LLM #LLMs #NVIDIA #TGI #Inference #H100

How To Create A NVIDIA H100 GPU Cloud Server To Run And Train AI, ML, And LLMs Apps On DigitalOcean https://youtu.be/aDPUOzk443E #Websplaining #GPU #NVIDIA #DigitalOcean #GpuDroplet #Droplet #AI #ML #LLM #H100 #NvidiaH100 #H100GPU #CloudServer #VPS #Server #GpuServer #Ubuntu #Linux

»#Meta is using more than 100,000 #Nvidia #H100 AI GPUs to train #Llama4: Mark Zuckerberg says that Llama 4 is being trained on a cluster “bigger than anything that I’ve seen”.« https://www.tomshardware.com/tech-industry/artificial-intelligence/meta-is-using-more-than-100-000-nvidia-h100-ai-gpus-to-train-llama-4-mark-zuckerberg-says-that-llama-4-is-being-trained-on-a-cluster-bigger-than-anything-that-ive-seen?eicker.news #tech #media

The converged memory architecture of #GH200 can boost TTFT (time to first token) in multiturn user interactions by up to 2x on Llama 3 70B, compared to x86-based #H100 servers

https://developer.nvidia.com/blog/nvidia-gh200-superchip-accelerates-inference-by-2x-in-multiturn-interactions-with-llama-models/

Elon Musk announced that he wants to build a training cluster for #AI with 200,000 graphics cards from #Nvidia #H100 GPUs. Let's assume 500 watts for a single #GPU. That means 500 watts * 24 hours * 365 days per year * 200,000 (GPUs) = 876 GWh or 876,000,000 kWh.

#Musk #Grok #ChatGPT #o1 #energyConsumption #climateCrisis
https://www.tomshardware.com/tech-industry/nvidias-h100-gpus-will-consume-more-power-than-some-countries-each-gpu-consumes-700w-of-power-35-million-are-expected-to-be-sold-in-the-coming-year

Inside the World's Largest #AI Supercluster: xAI Colossus with 100,000 Nvidia #H100 GPUs in Memphis, Tennessee

The cluster was built in just 122 days!

https://www.servethehome.com/inside-100000-nvidia-gpu-xai-colossus-cluster-supermicro-helped-build-for-elon-musk/

@Patrick1Kennedy @ServeTheHome
on making and sharing this video

https://youtube.com/watch?v=Jf8EPSBZU7Y

Gefion: NVIDIA CEO Jensen Huang joined the King of Denmark to launch the country’s largest sovereign #AI #supercomputer

NVIDIA DGX SuperPOD driven by 1,528 NVIDIA #H100 Tensor Core GPUs and interconnected using NVIDIA Quantum-2 InfiniBand

https://blogs.nvidia.com/blog/denmark-sovereign-ai-supercomputer/

German IT magazine #ct tested the usual suspect LLMs, focusing on how they handle German language content and their energy footprint. The results speak for themselves.
The French LLM #Mistral delivered impressive results, almost matching the quality of the market leader chatGPT while consuming far less energy.
You can download this model and run it on-premises with just four #H100 cards (although they still cost 30,000 EUR each). ... https://byzzyy.micro.blog/2024/10/20/german-it-magazine.html

"TLDR: Don’t buy H100s. The market has flipped from shortage ($8/hr) to oversupplied ($2/hr), because of reserved compute resales, open model finetuning, and decline in new foundation model co’s. Rent instead."

https://www.latent.space/p/gpu-bubble

Really interesting long-read about the economics of companies that sell GPU compute. It also has a list (Bill of Materials) of all components you'd need to start such a company from scratch!

https://www.semianalysis.com/p/ai-neocloud-playbook-and-anatomy