In a recent article, we highlighted the NVIDIA L40S as a noteworthy alternative to the NVIDIA A100 and H100 GPUs, offering distinct advantages in terms of price, performance, and capabilities. To further explore this topic, we have produced a video showcasing the L40S in action during our visit to Supermicro.
The L40S represents a significant improvement over its predecessor, the L40, particularly in AI training and inferencing. Despite their common heritage, it is clear that the L40S brings unique enhancements to the table.
While the L40 and L40S may not compete with the A100 and H100 in terms of absolute memory capacity, bandwidth, or FP64 performance, it is worth noting that AI workloads are increasingly prioritized over traditional FP64 compute. For most users, this trade-off should be more than satisfactory.
Although the L40S appears to have less memory compared to the A100, it supports the NVIDIA Transformer Engine and FP8. The use of FP8 significantly reduces data size, allowing for less memory consumption and bandwidth requirements, while maintaining performance. NVIDIA’s promotion of the Transformer Engine aims to optimize cost and improve AI performance in their parts, a feature shared with the H100.
In terms of video encoding and decoding, the L40S offers a more visualization-centric approach, while the H100 prioritizes decoding capabilities. This distinction gives users flexibility based on their specific needs.
Despite the H100 being faster, it comes with a significantly higher price tag. Currently, the H100 is approximately 2.6 times more expensive than the L40S according to public prices listed on CDW.
Another advantage of the L40S is its availability. These GPUs are more easily obtainable compared to the in-demand NVIDIA H100, which often involves waiting in line for a purchase.
Feedback received since the initial publication has shed light on diverse usage scenarios beyond AI clusters. Users have reported leveraging the L40S for visualization and virtual GPU (vGPU) clusters. With the inclusion of video pipelines and RT cores, these cards can seamlessly transition from vGPU workloads during the day to AI tasks in the evening when vGPU demands are lower.
One example of a system suited to these use cases is the Supermicro SYS-521GE-TNRT.
In conclusion, the NVIDIA L40S is an intriguing GPU, readily available and offering features that the H100 and A100 lack. While it may not cater to those requiring FP64 computing, it serves as an excellent alternative for users who do not require such precision.
The source of the article is from the blog shakirabrasil.info