IBM Granite 4.0 Tiny's Open Source Preview Yields Promising Efficiency and Performance Results

IBM is unveiling IBM Granite 4.0 Tiny Preview—a preliminary version of the smallest model within the upcoming Granite 4.0 model family—to the open source community. Now available on Hugging Face, this preview showcases Granite 4.0 Tiny’s performance advancements, rivalling IBM Granite 3.3 2B Instruct—despite fewer active parameters and about a 72% reduction in memory requirements.

Granite 4.0 Tiny, while only partially trained, is already demonstrating significant improvements compared to its predecessors. IBM expects the model’s performance to match Granite 3.3 8B Instruct once its training and post-training have been completed.

These tech boons are achieved despite Granite 4.0 Tiny—as the name would suggest—being among the Granite 4.0 model family’s smallest offerings. Additionally, through the Granite 4.0 architecture’s use of no positional encoding (NoPE), Granite 4.0 Tiny does add any additional computational burden with long-context performance, enabling the model to easily run on a modest consumer GPU. Its compact, highly efficient nature not only offers significant gains in terms of performance but places no constraint on context length.

At the core of the Granite 4.0 model family’s memory efficiency and low latency is an all-new hybrid mixture of experts (MoE) model, which combines the speed and efficiency of Mamba with the precision of transformers. Mamba—a type of state space model (SSM)—enables selectivity mechanisms that efficiently capture global context, while transformers enable a more nuanced parsing of local context, according to IBM.

As a result, this model family makes significant strides in balancing a reduction in memory without impacting performance—which Granite 4.0 Tiny doubles down  even further with 7B total parameters and 64 experts, yielding 1B active parameters at inference time.

Granite 4.0 Tiny will be officially released this summer, alongside Granite 4.0 Small and Granite 4.0 Medium. At its preliminary release, it is not recommended for enterprise implementation.

To learn more about Granite 4.0 Tiny and the Granite 4.0 model family, please visit https://www.ibm.com/us-en.