Building LLM-Powered Applications: Understanding the Foundations

Welcome to my learning log! Today, we are diving into Chapter 1 of Building LLM Powered Applications by Valentina Alto. This chapter lays down the architectural and mathematical foundations of Large Language Models (LLMs) and discusses what it takes to transition from a software engineer using API wrappers to a true AI Systems Architect.

Here is a synthesized guide to the core concepts, complete with practical math, architectural analogies and my own learning reflections.

---

What is a Large Language Model (LLM)?

At their core, LLMs are the engine of Generative AI. Unlike traditional software, Generative AI can produce human-like content from natural language prompts.

But LLMs are more than just text generators; they act as reasoning engines that serve as the brains of intelligent applications.

       [ User Input (Natural Language) ]
                       │
                       ▼
       ┌───────────────────────────────┐
       │      LLM (Reasoning Engine)   │◄─── Processed by parameters
       └───────────────────────────────┘
                       │
                       ▼
         [ Human-like Text / Output ]

Key Terminology

• Large: Refers to the massive scale of the training data (unstructured text) and the hundreds of millions (or billions) of parameters.
• Language: Indicates the nature of the primary training and generation data.
• Unsupervised Learning: Because LLMs learn from vast amounts of unlabelled text to predict the next token, they are primarily classified under unsupervised machine learning.
• Foundation Models (FMs): These are broad-knowledge models trained on diverse data formats (not just text), enabling them to grasp complex cross-domain patterns. FMs act as the base for tools like ChatGPT, DALL-E and ElevenLabs.

Generative AI vs. Natural Language Understanding (NLU)

A common point of confusion is how Generative AI differs from standard NLU. Here is the distinction:

Feature	Natural Language Understanding (NLU)	Generative AI (LLMs)
Goal	Understand existing text and map it to intent.	Generate completely new, coherent content.
Logic	Relies heavily on a predetermined set of responses/rules.	Infers patterns to dynamically predict the next best words.
Paradigm Shift	Traditional Computer Programming / Classic ML.	Modern Deep Learning / Transfer Learning.

---

The Architecture: Scale and Adaptability

Foundation models leverage transfer learning and inference to apply acquired knowledge to brand-new tasks.

To understand why scale matters, consider this architectural analogy:

The Building Analogy
Think of a neural network like a building. A small, single-story house has simple utility systems. But as you scale up to a massive skyscraper (representing millions/billions of parameters), its internal routing systems—circulation, HVAC, plumbing, electrical—become exponentially more complex.

In neural networks, with complexity comes adaptability. A massive architecture allows the model to capture incredibly complex, non-linear relationships and generalize across multiple domains.

---

Under the Hood: Artificial Neural Networks (ANNs)

An Artificial Neural Network (ANN) is a computational structure inspired by the human brain. It is composed of layers of artificial neurons (nodes) connected by weights.

Input Layer         Hidden Layer        Output Layer
   ( ) ──────────────► ( ) ──────────────► ( )
   ( ) ──────────────► ( ) ──────────────► ( )
                [Weights (w) & Biases (b)]

• Weights ($w$): Represent the strength of the connection between neurons. These are the parameters adjusted during training.
• Backpropagation: The core learning algorithm. During a backward pass, errors are calculated and propagated back through the network, updating the weights via gradient descent to improve prediction accuracy:

\[\theta \leftarrow \theta - \eta \nabla_{\theta} L(\theta)\]

From Text to Vectors: Preprocessing

Because ANNs are purely mathematical models, they cannot read text directly. Data must go through a preprocessing pipeline:

1. Tokenization: Breaking down continuous text into smaller units (tokens).
2. Embedding: Mapping tokens into dense numerical vectors within a high-dimensional vector space.

This process relies heavily on linear algebra. While we often visualize embeddings in a simple 2-dimensional space, state-of-the-art LLMs utilize embedding spaces with thousands of dimensions (e.g., 1536 dimensions in OpenAI’s text-embedding-3-small), capturing incredibly nuanced semantic relationships.

---

Connecting the Dots: Mathematics and Machine Learning

My study journey has repeatedly shown me how critical mathematics and statistics are to artificial intelligence. For instance, Bayes’ Theorem is a fundamental building block of classification tasks:

\[P(A|B) = \frac{P(B|A)P(A)}{P(B)}\]

While studying Python Machine Learning by Example by Yuxi (Hayden) Liu, I applied the Naive Bayes classifier to build a movie recommendation engine. You can check out my ongoing implementation attempt here:

👉 GitHub Repository: Naive Bayes Movie Recommendation Engine

---

Moving Beyond “Wrappers”: The Role of an AI Engineer

There is a major overlap between a Machine Learning Engineer and an AI Engineer. However, to build production-ready LLM infrastructure, Agentic workflows and Retrieval-Augmented Generation (RAG) systems, a superficial understanding of APIs is not enough.

Warning
If you don’t understand the foundational ML principles beneath LLMs, your application architecture could be built on a shaky foundation, ultimately causing the entire system to collapse under production constraints.

Key Evaluation Metrics

To assess LLM performance beyond simple outputs, AI Architects rely on established benchmarks:

• GLUE / SuperGLUE: General Language Understanding Evaluation.
• MMLU: Massive Multitask Language Understanding (tests world knowledge and problem-solving).
• HellaSwag: Tests common-sense reasoning.

Customization Techniques

When standard out-of-the-box LLMs fall short, we can adapt them using:

1. Few-Shot Learning: Providing input-output examples in the context prompt.
2. Extended Non-Parametric Knowledge (RAG): Connecting the model to an external vector database.
3. Fine-Tuning: Updating the actual weights of the model on a specialized dataset.

---

Summary: The LLM Processing Pipeline

To summarize Chapter 1, the workflow of passing data through an LLM can be simplified into three key phases:

\[\text{Raw Text} \xrightarrow{\text{Tokenizer}} \text{Tokens} \xrightarrow{\text{Encoder}} \text{Vector Representations} \xrightarrow{\text{Softmax}} \text{Probability Values (Output)}\]

1. Tokenizer: Breaks down the input text.
2. Encoder: Converts text into vector representations.
3. Softmax Function: Converts the vector outputs of the network into probability distributions to select the most likely next word:

\[\sigma(\mathbf{z})_i = \frac{e^{z_i}}{\sum_{j=1}^K e^{z_j}}\]

Understanding this foundation is our first step toward building resilient, intelligent and scalable LLM applications. Stay tuned for the next chapter!