GitHub

Granite 3.0 Language Models Project Description

What is the project about?

The project introduces Granite 3.0, a set of open, lightweight, state-of-the-art language models developed by IBM. These models are designed for enterprise use and are capable of handling multiple languages, coding, reasoning, and tool usage. They are available in various sizes to accommodate different computational resources.

What problem does it solve?

The project addresses the need for powerful, yet efficient, language models that can be used in enterprise settings. It offers a range of model sizes, allowing users to balance performance with computational cost. The models are specifically designed with governance, risk, and compliance (GRC) in mind, making them suitable for sensitive business applications. They also bridge the gap between very large, resource-intensive models and smaller, less capable ones.

What are the features of the project?

  • Multilingual Support: Natively supports multiple languages.
  • Coding Capabilities: Trained on code data, enabling code generation and understanding.
  • Reasoning Abilities: Performs well on reasoning tasks.
  • Tool Usage: Designed to integrate with and utilize external tools.
  • Model Variety: Includes both dense (2B and 8B parameters) and Mixture-of-Expert (MoE) models (1B and 3B, with 400M and 800M activated parameters, respectively).
  • Base and Instruct Versions: Offers both base (pre-trained) and instruct (fine-tuned for dialogue and instruction following) versions of each model.
  • Data Governance: Trained on data rigorously vetted for GRC, ownership, licensing, and sensitive information.
  • Open Source: Released under the Apache 2.0 license for both research and commercial use.
  • Post-training Techniques: Instruct models are created using techniques like supervised finetuning, PPO, best-of-N sampling, BRAIn, and model merging.
  • Comprehensive Evaluation: Thoroughly evaluated on a wide range of benchmarks, demonstrating strong performance.
  • Hugging Face Integration: Easily accessible and usable through Hugging Face.

What are the technologies used in the project?

  • Transformer Architecture: The models are based on the transformer architecture.
  • Byte Pair Encoding (BPE): Uses BPE for tokenization, similar to StarCoder.
  • Mixture-of-Experts (MoE): Some models utilize the MoE architecture for improved efficiency.
  • 3D Parallelism: Employs Tensor, Pipeline, and Data Parallelism for training.
  • Maximum Update Parameterization and Power Scheduler: Used for hyperparameter optimization.
  • Proximal Policy Optimization (PPO): Used for model alignment in post-training.
  • Python: The provided example code is in Python.
  • PyTorch: The example uses the PyTorch library.
  • Transformers Library (Hugging Face): The models are integrated with the Hugging Face Transformers library.
  • Git: Used for version control and model downloading.

What are the benefits of the project?

  • Enterprise-Ready: Designed for enterprise use with a focus on GRC.
  • Cost-Effective: Offers a range of model sizes to balance performance and computational cost.
  • Open and Accessible: Open-source license and availability on Hugging Face promote accessibility and collaboration.
  • High Performance: Outperforms models of similar sizes on various benchmarks.
  • Versatile: Suitable for a wide range of tasks, including multilingual applications, coding, and reasoning.
  • Customizable: The models' data curation and training procedure were designed for enterprise usage and customization.

What are the use cases of the project?

  • Natural Language Processing (NLP) tasks: Text generation, summarization, translation, question answering.
  • Code Generation and Understanding: Software development, code completion, bug detection.
  • Dialogue Systems: Chatbots, virtual assistants.
  • Reasoning and Problem Solving: Applications requiring logical inference.
  • Enterprise Applications: Cybersecurity, retrieval augmented generation (RAG), and other business-specific tasks.
  • Multilingual Applications: Tasks requiring support for multiple languages.
  • Research: A foundation for further research in language modeling.
  • Function Calling: Integration with external tools and APIs.
granite-3.0-language-models screenshot