Fine-Tuning LLMs with LoRA

Customizing open-weights models on consumer and budget GPUs with PEFT and QLoRA.

One-time purchase

£29.00

Instant Access

I consent to immediate access to this digital content and acknowledge that under the UK Consumer Contracts Regulations 2013, my 14-day statutory right of cancellation is waived upon purchase.

Encrypted payment processing via Stripe Checkout

DRM-free PDF e-book delivered immediately after payment

VAT invoice and purchase receipt emailed automatically

About this Book

Learn when fine-tuning outperforms RAG and how to do it efficiently. Step-by-step instructions for preparing clean instruction datasets, selecting rank and alpha hyper-parameters, computing loss curves, and merging Low-Rank Adaptation (LoRA) adapters into base models.

Target Audience: Engineers who want to build domain-specialized language models on accessible hardware.

What You Will Learn & Build

Decide between Prompt Engineering, RAG, and Parameter-Efficient Fine-Tuning (PEFT)

Prepare synthetic and human-curated datasets with strict schema validation

Train 7B-70B models on a single GPU using 4-bit quantization (QLoRA)

Evaluate loss curves, avoid catastrophic forgetting, and export GGUF models

1. When to Fine-Tune vs When to RetrieveSection 1

2. Fundamentals of Low-Rank Adaptation (LoRA)Section 2

3. QLoRA: 4-Bit NormalFloat Quantization in ActionSection 3

4. Dataset Formatting: Alpaca, ShareGPT, and ChatMLSection 4

5. Training Hyperparameters: Learning Rate, Rank, Alpha & Target ModulesSection 5

6. Evaluation, Model Merging & Quantization for InferenceSection 6