Everything you need to know about gemma 4 models use training trick.
[ad_1]TL;DR
- Gemma 4 models are now available for download with quantization-aware training (QAT), which reduces the size and memory footprint of the models.
- These open-source models retain quality better thanks to QAT compared to those that use post-training quantization (PTQ).
- The Gemma 4 models optimized with QAT are available in five sizes: Gemma 4 E2B, Gemma 4 E4B, Gemma 4 12B, Gemma 4 26B A4B, and Gemma 4 31B.
Following Google’s launch of the laptop-grade Gemma 4 12B model earlier this week, the company is releasing new Gemma 4 model checkpoints with quantization-aware training. Quantization is necessary to reduce the amount of memory required to run lightweight models. The standard method is post-training quantization (PTQ), which quantizes the model after training, but could result in weaker performance. The latest Gemma 4 versions use quantization-aware training (QAT) instead to reduce model quality loss and accelerate decode speed, according to Google’s blog post.
Google says that incorporating quantization into the training process results in checkpoints with better performance than models refined with PTQ. The compressed models run on phones and laptops well thanks to a custom mobile-quantization schema. This involves using pre-calculated settings, 2-bit compression in certain parts of the model, and vocabulary list and short-term memory compression. For the user, this results in a smaller model that consumes less system memory.
Don’t want to miss the best from Android Authority?
There are multiple model sizes available with QAT optimization, include Gemma 4 E2B, Gemma 4 E4B, Gemma 4 12B, Gemma 4 26B A4B, and Gemma 4 31B. The smallest versions, like the text-only Gemma 4 E2B model, require less than a gigabyte of memory to run. These small Gemma 4 checkpoints without intensive resource requirements are ideal for running on phones.
Google shared the approximate memory requirements to load the new Gemma 4 models with QAT in various sizes:
There are four different formats of Gemma 4 QAT models available for download: unquantized QAT checkpoints, GPT-Generated Unified Format (GGUF), mobile-optimized, and Compressed Tensors. These models preserve “similar quality to bfloat16 while dramatically reducing the memory requirements to load the model,” according to Google.
After downloading the Gemma 4 QAT model weights, users can run the checkpoints on their phones, laptops, or desktops. You can find the mobile and desktop models on Hugging Face, as well as in LM Studio.
Thank you for being part of our community. Read our Comment Policy before posting.
[ad_2]
Source link
More on gemma 4 models use training trick
Stay tuned for updates on gemma 4 models use training trick as the story develops. Related: LW — I Gave an AI a Civilization to Run. It Built a Nuke..
Gemma 4 models use training trick: Key Details Explained
Gemma 4 models use a training trick to slash their memory footprint — let’s break down what actually matters here. This one stands out because it touches the things buyers care about most: performance, features, and value for money. If you’re comparing options right now, it’s worth keeping on your shortlist.
Looking at the bigger picture, this fits into a wider trend where brands compete harder with every release cycle. Expect more details to surface as things develop, including confirmed specifications and pricing from official sources.
Should You Keep Following This?
If you follow mobile tech closely, the answer is yes. Developments like this could influence your next purchase decision, so it’s worth tracking. Bookmark this page — we’ll keep the coverage fresh as the story evolves.
Frequently Asked Questions
New checkpoints with quantization-aware training, which reduces model size and memory footprint.
QAT incorporates quantization into the training process, so checkpoints retain quality better than with post-training quantization.
Gemma 4 E2B, E4B, 12B, 26B A4B, and 31B.
On phones and laptops, thanks to a custom mobile-quantization schema with pre-calculated settings and 2-bit compression in parts of the model.
