How to Fine-Tune Gemma 4: Complete Step by Step Guide

Google has introduced Gemma 4, describing it as its most intelligent open model family so far, built for strong reasoning and agentic workflows. Gemma models are designed to be flexible across environments, with official support and tooling for local development, cloud deployment, and model customization, which makes them a strong choice for fine-tuning projects.

In this complete tutorial, we will fine-tune Gemma 4 E4B-it on a human emotion classification dataset from Hugging Face. We will set up a 3090 GPU environment, load and inspect the dataset, prepare and format the data for supervised fine-tuning, load the base model, run baseline evaluation before training, fine-tune the model, and then evaluate its performance again after training.

1. Setting Up the Environment

Start by launching a new RunPod instance, and make sure your account has at least $5 in credit before you begin. For this tutorial, choose a 3090 GPU pod and select the latest PyTorch template.

Before deploying, open the template settings and make a few updates. Increase both the container disk and volume disk to 40 GB so you have enough space for the model, dataset, cached files, and training checkpoints.

You should also add your Hugging Face token as an environment variable. You can generate this token from Settings > Access Tokens in your Hugging Face account.

Once these settings are in place, go ahead and deploy the pod. It may take a minute or two for the instance to start. After it is ready, open the JupyterLab interface so you can begin working inside the environment.

The first thing to do in JupyterLab is launch the new Python notebook and install all the required Python packages. Run the following command in a notebook cell:

These packages will cover the full workflow, including loading the dataset, preparing the model, fine-tuning, and evaluation.

The last step is to sign in to the Hugging Face Hub using your saved token. This gives you access to the gated model and also makes it easier to upload files, create repositories, and push your fine-tuned model later.

2. Load and Prepare the Emotion Dataset

Now that the environment is ready, the next step is to load the emotion dataset from Hugging Face and prepare smaller splits for training and evaluation.

For this tutorial, we are not using the full dataset. Instead, we create limited train, validation, and test splits so the fine-tuning process stays faster and easier to run on a single GPU.

The final dataset contains 4,000 training examples, 400 validation examples, and 400 test examples.

Next, we look at the label names stored in the dataset. These are the emotion classes the model will learn to predict.

This shows that the task has six emotion categories: sadness, joy, love, anger, fear, and surprise.

3. Formatting Data for Gemma 4 Fine-Tuning

Before we can fine-tune the model, we need to convert the dataset into the format Gemma 4 will use during training.

Instead of passing only raw text and labels, we structure each example as a short chat interaction with a system message, a user message, and the expected assistant response.

The system prompt tells the model exactly what task it should perform. In this case, we want the model to act as an emotion classification assistant and return only one of the six allowed labels.

Now we create a function to format the data into the prompt-completion format required for supervised fine-tuning:

4. Load Gemma E4B-it With 4-Bit Quantization

Now we can load Gemma 4 E4B-it and prepare it for fine-tuning. Since this is a relatively large model, we load it with 4-bit quantization to reduce memory usage and make it easier to run on a 3090 GPU. We also use bfloat16 as the compute type, which helps keep the setup efficient.

We start by importing the required libraries and defining the main model settings:

Next, we enable CUDA optimizations and load the tokenizer:

Now we prepare the quantization settings and model loading arguments:

Finally, we load the model and align its configuration with the tokenizer:

5. Evaluate the Base Model

Before fine-tuning, it is useful to evaluate the base model first so we have a clear baseline to compare against later.

In this section, we define a few helper functions that generate predictions, extract valid emotion labels, and run evaluation on the test split.

Now we run the baseline evaluation on the test split:

These baseline results show that the untuned model already performs reasonably well, but there is still room for improvement. The accuracy is around 58.25%, the macro F1 score is around 0.42, and the model produced 33 invalid predictions.

6. Fine-Tune Gemma 4 With LoRA

Now that we have the baseline results, we can fine-tune Gemma 4 using LoRA. LoRA is a parameter-efficient fine-tuning method, which means we do not update the full model. Instead, we attach a small number of trainable adapter weights on top of the base model. This makes training much lighter and more practical on a single GPU.

We start by defining the LoRA configuration:

Next, we define the training configuration and set up the trainer:

Now we initialize the trainer with the LoRA configuration:

Now we can start training. The training typically takes about 9 minutes on a 3090 GPU:

Once training is complete, we can save the adapter and tokenizer locally:

Finally, we can push the model to the Hugging Face Hub:

7. Evaluate the Fine-Tuned Model

Now that training is complete, the final step is to evaluate the fine-tuned model on the same test split and compare the results with the base model. This helps us see whether LoRA fine-tuning improved the model's ability to classify emotions more accurately.

These results are clearly stronger than the baseline. After fine-tuning, the model reaches 77.25% accuracy and a macro F1 score of 0.698. The number of invalid predictions also drops from 33 to 20.

Final Thoughts

Fine-tuning Gemma 4 is very sensitive to setup, especially the prompt structure and training arguments. If the prompt format is wrong, or you do not use the proper template consistently, the model may go through training without actually learning the task well.

Another important lesson is max_length. If you reduce it too much, especially below around 125, the model may not learn the pattern properly at all. Most issues come back to the same two areas: prompt formatting and training configuration.

To improve the results further, a good next step would be to fine-tune on the full dataset and train for at least 3 epochs instead of just one. That would give the model more examples to learn from and more time to adapt, which should lead to stronger accuracy and F1 scores.