rdf-syntax-grammar
mods:
  schema location: http://www.loc.gov/mods/v3 http://www.loc.gov/standards/mods/v3/mods-3-2.xsd
  title info:
    title: Building large language models from scratch
    sub title: design, train, and deploy LLMs with PyTorch
  name:
    name part: Grigorov, Dilyan
    role:
      role term: creator
    role:
      role term: author.
  type of resource: text
  origin info:
    place:
      place term: cau
    date issued: 2026
    issuance: monographic
  language:
    language term: eng
  physical description:
    form: remote
    extent: 1 online resource (xxv, 530 pages) : illustrations
  abstract: This book is a complete, hands-on guide to designing, training, and deploying your own Large Language Models (LLMs)-from the foundations of tokenization to the advanced stages of fine-tuning and reinforcement learning. Written for developers, data scientists, and AI practitioners, it bridges core principles and state-of-the-art techniques, offering a rare, transparent look at how modern transformers truly work beneath the surface. Starting from the essentials, you'll learn how to set up your environment with Python and PyTorch, manage datasets, and implement critical fundamentals such as tensors, embeddings, and gradient descent. You'll then progress through the architectural heart of modern models, covering RMS normalization, rotary positional embeddings (RoPE), scaled dot-product attention, Grouped Query Attention (GQA), Mixture of Experts (MoE), and SwiGLU activations, each explored in depth and built step by step in code. As you advance, the book introduces custom CUDA kernel integration, teaching you how to optimize key components for speed and memory efficiency at the GPU level-an essential skill for scaling real-world LLMs. You'll also gain mastery over the phases of training that define today's leading models: Pretraining - Building general linguistic and semantic understanding. Midtraining - Expanding domain-specific capabilities and adaptability. Supervised Fine-Tuning (SFT) - Aligning behavior with curated, task-driven data. Reinforcement Learning from Human Feedback (RLHF) - Refining responses through reward-based optimization for human alignment. The final chapters guide you through dataset preparation, filtering, deduplication, and training optimization, culminating in model evaluation and real-world prompting with a custom TokenGenerator for text generation and inference. By the end of this book, you'll have the knowledge and confidence to architect, train, and deploy your own transformer-based models, equipped with both the theoretical depth and prac…
  note: Dilyan Grigorov.
  note: Includes index.
  subject:
    topic: Deep learning (Machine learning)
  subject:
    topic: Python (Computer program language)
  subject:
    topic: Neural networks (Computer science)
  subject:
    topic: Apprentissage profond
  subject:
    topic: Python (Langage de programmation)
  subject:
    topic: Réseaux neuronaux (Informatique)
  classification: Q325.5 .G75 2026
  classification: 006.3/1
  related item:
    title info:
      title: Building Large Language Models from Scratch
    name:
      name part: Grigorov, Dilyan
    origin info:
      publisher: Berkeley, CA : Apress L. P.,c2026
  related item:
    identifier: (OCoLC)1548245012
  related item:
    title info:
      title: Professional and Applied Computing Series
  identifier: 9798868822971 (electronic bk.)
  identifier: 9798868822964
  identifier: 9798868822988
  identifier: 9798868822971 O'Reilly Media
  location:
    physical location: POD
  record info:
    record content source: GW5XE
    record creation date: 260504
    record change date: 20260726034402.6
    record identifier: in00005789140
    language of cataloging:
      language term: eng