NVIDIA released the Nemotron-4 340B models under the NVIDIA Open Model License, a permissive licence that allows commercial use without attribution requirements.
The most distinctive feature of these models is that they are trained mainly on high-quality synthetic data. NVIDIA believes that this will lead to further advancement in AI development as it will eliminate many data-related issues such as rights, quality and preparation efforts.
NVIDIA plans to publish the datasets and to open source the synthetic data-generation pipeline it used to train the Nemotron-4 models, hoping to provide the AI community with more high-quality tools.
Compared to other open-source models such as Llama 3 70b, Mistral 8x22 and Qwen 2 72b Base, the Nemotro-4 family topped many performance benchmarks.
References
- NVIDIA Technical Blog. (2024). Leverage the Latest Open Models for Synthetic Data Generation with NVIDIA Nemotron-4 340B. [online] Available at: https://developer.nvidia.com/blog/leverage-our-latest-open-models-for-synthetic-data-generation-with-nvidia-nemotron-4-340b/ [Accessed 20 Jul. 2024].