# Unsloth: A Fine-Tuning Guide for Developers

Mia Gouffray

March 21, 2025

## Introduction to Fine-Tuning

Fine-tuning is the process of adjusting a pre-trained model to fit a specific task or use case. It is an essential part of developing a successful model. Fine-tuning is great for developers who want to use an existing model but may need to make some adjustments to better suit their needs. It is much simpler and faster than creating a new model.

## Fine-Tuning with Unsloth

In this article, we’ll take a look at how we can fine-tune the **Meta [Llama 3](https://www.beam.cloud/blog/fine-tuning-llama3).1 8B** LLM using Beam with Unsloth. **.**

In the following demo, you will see how to use the [alpaca-cleaned](https://huggingface.co/datasets/yahma/alpaca-cleaned) dataset from Hugging Face to fine-tune the Meta Llama LLM with Unsloth; thus improving its instruction-following capabilities.

Unsloth is a Python package that allows developers to quickly and efficiently fine-tune models like Llama. The advantages of using Unsloth for fine-turning include memory optimization, speed and efficiency, compatibility with different hardware, and its open-source nature.

## Getting Started with Unsloth

First, let’s import the necessary packages and set up our environment in `finetune.py` with our `Image` container where we define our model and dependencies:

```python
from beam import endpoint, Image, Volume, env

if env.is_remote():
    import torch
    from unsloth import FastLanguageModel
    from transformers import TrainingArguments
    from trl import SFTTrainer
    from datasets import load_dataset
    import os

MODEL_NAME = "unsloth/Meta-Llama-3.1-8B-bnb-4bit"
MAX_SEQ_LENGTH = 2048
VOLUME_PATH = "./model_storage"
TRAIN_CONFIG = {
    "batch_size": 2,
    "grad_accumulation": 4,
    "max_steps": 60,
    "learning_rate": 2e-4,
    "seed": 3407,
}

image = (
    Image(python_version="python3.11")
    .add_python_packages(
        [\
            "ninja",\
            "packaging",\
            "wheel",\
            "torch",\
            "xformers",\
            "trl",\
            "peft",\
            "accelerate",\
            "bitsandbytes",\
        ]
    )
    .add_commands(
        [\
            "pip uninstall unsloth -y",\
            'pip install "unsloth[colab-new] @ git+https://github.com/unslothai/unsloth.git"',\
        ]
    )
)
```

Let’s start our fine-tuning function by defining our model and tokenizer:

```python
def fine_tune_model():
    output_dir = os.path.join(VOLUME_PATH, "fine_tuned_model")
    os.makedirs(output_dir, exist_ok=True)

model, tokenizer = FastLanguageModel.from_pretrained(
        model_name=MODEL_NAME, max_seq_length=MAX_SEQ_LENGTH, load_in_4bit=True
    )
```

Here we have defined our model as a `FastLanguageModel` from the Unsloth package.

Now we’ll create some helper functions within the `fine_tune_model` function to format the alpaca-cleaned dataset for our model, then we’ll load the data and perform the transformation:

```python
def fine_tune_model():

#model and token definition
    def format_alpaca_prompt(instruction, input_text, output):
        template = (
            "Below is an instruction that describes a task, paired with an input that "
            "provides further context. Write a response that appropriately completes the request.\n"
            "### Instruction:\n{}\n### Input:\n{}\n### Response:\n{}"
        )
        return template.format(instruction, input_text, output) + tokenizer.eos_token

def format_dataset(examples):
        texts = [\
            format_alpaca_prompt(instruction, input_text, output)\\
            for instruction, input_text, output in zip(\
                examples["instruction"], examples["input"], examples["output"]\
            )\
        ]
        return {"text": texts}

dataset = load_dataset("yahma/alpaca-cleaned", split="train")
    dataset = dataset.map(format_dataset, batched=True)
```

We loaded our model earlier, but now we’ll use `FastLanguageModel.get_peft_model` to attach adapters in order to perform the fine-tuning:

```python
def fine_tune_model():
  ### previous code
  model = FastLanguageModel.get_peft_model(
    model,
    r=16,
    target_modules=[\
      "q_proj",\
      "k_proj",\
      "v_proj",\
      "o_proj",\
      "gate_proj",\
      "up_proj",\
      "down_proj",\
    ],
    lora_alpha=16,
    lora_dropout=0,
    use_gradient_checkpointing="unsloth",
    random_state=TRAIN_CONFIG["seed"],
  )
```

We’ll take a deeper look into some of the parameters used to fine-tune the model in a later section.

Now we’ll use the supervised fine-tuning training ( [SFT Trainer](https://huggingface.co/docs/trl/en/sft_trainer)) which is a specialized tool within Hugging Face’s ecosystem designed to streamline the fine-tuning process for language models like the one we are using. We’ll pass our model, tokenizer, and training data to the trainer along with some more parameters to complete the model training:

```python
def fine_tune_model():
  ### previous code

trainer = SFTTrainer(
    model=model,
    tokenizer=tokenizer,
    train_dataset=dataset,
    dataset_text_field="text",
    max_seq_length=MAX_SEQ_LENGTH,
    dataset_num_proc=2,
    packing=False,
    args=TrainingArguments(
      per_device_train_batch_size=TRAIN_CONFIG["batch_size"],
      gradient_accumulation_steps=TRAIN_CONFIG["grad_accumulation"],
      max_steps=TRAIN_CONFIG["max_steps"],
      learning_rate=TRAIN_CONFIG["learning_rate"],
      fp16=False,
      bf16=True,
      logging_steps=1,
      output_dir=output_dir,
      seed=TRAIN_CONFIG["seed"],
    ),
  )
```

Lastly, we will train the model and save the results in our `output_dir`:

```python
def fine_tune_model():
  ### previous code
  with torch.autograd.set_detect_anomaly(True):
    trainer.train()

model.save_pretrained(output_dir)
  tokenizer.save_pretrained(output_dir)

return {
    "status": "success",
    "message": "Fine-tuning complete",
    "model_path": output_dir,
  }
```

You can check out the full code repository [here](https://github.com/beam-cloud/examples/tree/main/unsloth).

## Deploying the Fine-Tuned Model

After executing the fine-tuning script, you should verify that the files are saved in your Beam Volume:

```bash
beam ls model-storage/fine_tuned_model
```

The expected output with the fine-tuned files should look like:

```text
Name                           Size   Modified Time   IsDir
 ─────────────────────────────────────────────────────────────
  fine_tuned_model/README.md                  4.99 KiB   1 hour ago      No
  fine_tuned_model/adapter_config.json        805.00 B   1 hour ago      No
  fine_tuned_model/adapter_model.safeten...   160.06 MiB 1 hour ago      No
  fine_tuned_model/checkpoint-60/                        1 hour ago      Yes
  fine_tuned_model/special_tokens_map.js...     459.00 B 1 hour ago      No
  fine_tuned_model/tokenizer.json            16.41 MiB   1 hour ago      No
  fine_tuned_model/tokenizer_config.json     49.46 KiB   1 hour ago      No
  ...
```

You then need to run the interference script which you can find [here](https://github.com/beam-cloud/examples/blob/main/unsloth/inference.py). The script (inference.py) loads the fine-tuned model and exposes an endpoint for generating responses.

Once you deploy the endpoint `beam deploy inference.py:generate`, you will get back a URL with the endpoint and can now utilize your fine-tuned LLM.

## Fine-Tuning with Hugging Face

The dataset used to fine-tune LLama 3.1B is the alpaca-cleaned dataset which you can view [here](https://huggingface.co/datasets/yahma/alpaca-cleaned) **.** This dataset is a cleaned version of the Alpaca dataset, which contains over 52,00 instructions and demonstrations generated using OpenAI’s text-davinci-003 engine. The data can be used to fine-tune models, improving their ability to follow instructions. Some minor changes are needed before training the model on the data. After loading the dataset, format_dataset is called which then calls format_alpaca_prompt. Format_alpaca_prompt alters the data to be formatted like:

```text
{"Below is an instruction that describes a task, paired with an input that provides further context. Write a response that appropriately completes the request

Instructions:
Evaluate the sentence for spelling and grammar mistakes

Input:
 He finnished his meal and left the resturant

Response:
There are two spelling errors in the sentence. The corrected ...."}
```

This data transformation helps fine-tune the model because it standardizes the format of instruction-based prompts, ensuring a consistent input-output structure for training. This improves the model's ability to learn task-specific responses.

## Optimizing Fine-Tuning for Fast Inference

The goal of Unsloth is to optimize the training and fine-tuning of LLMs. In our case, we want to fine-tune LLama to be more capable of responding to instructions. There are tons of different model parameters that can be changed to improve model performance and choosing the right parameters is a crucial piece of fine-tuning. We want a highly accurate model but don’t want to over or underfit. Let's take a look at some of the model parameters we can modify:

- **Learning Rate:** How much the model weights adjust per training step
- **Epochs:** The number of times the model sees the full training data
- **Gradient Accumulation:** The number of steps over which gradients are accumulated before performing a backward update or optimizer step

In our code we defined the parameters in the `TRAIN_CONFIG` object:

```python
TRAIN_CONFIG = {
    "batch_size": 2,
    "grad_accumulation": 4,
    "max_steps": 60,
    "learning_rate": 2e-4,
    "seed": 3407,
}
```

While most models have default parameters that they recommend following, developers often hyper-tune parameters to optimize performance. You can find additional information about using Unsloth for fine-tuning [here](https://docs.unsloth.ai/get-started/beginner-start-here/lora-parameters-encyclopedia).

## Conclusion

In this article, we looked at fine-tuning Meta LLaMA 3.1B LLM using Unsloth by adding task-specific data from Hugging Face. This method allows the model to better adapt to the context and nuances of instruction following while maintaining the strength of the pre-trained model. The fine-tuning process is efficient and effective, resulting in an improved model without requiring full retraining.

Mia Gouffray

Published March 21, 2025