If you follow artificial intelligence news, you have certainly heard of LLMs (Large Language Models), the technology behind tools like ChatGPT and Google Gemini. However, the AI universe has expanded dramatically. Today, engineers and researchers are developing highly specialized architectures to solve problems that pure text simply cannot address. Below, we explore the main new … Read more
The Self-Attention mechanism is the true heart of the Transformer architecture, the foundation behind every modern Large Language Model (LLM). Before Self-Attention was introduced in the landmark paper “Attention Is All You Need” (Vaswani et al., 2017), language models processed text sequentially, word by word (or token by token), using Recurrent Neural Networks (RNNs). This … Read more
To understand Multi-Head Attention (MHA), we first need to take a step back and examine the limitation of applying the attention equation (Scaled Dot-Product Attention) only once. The Problem with a Single “Head” Human language is complex and possesses multiple layers of meaning that occur simultaneously. Consider the sentence: “The student delivered the book to … Read more
The rapid evolution of AI agents (agentic systems) has brought a surge of new terms to software architecture vocabulary: Prompt Engineering, Context Engineering, Harness Engineering, and, more recently, Loop Engineering. With so many neologisms, it is easy to dismiss these names as mere buzzwords. However, the confusion between Harness Engineering and Loop Engineering is the … Read more
The main difference between a Foundation Model and a traditional (non-foundation) model lies in their scale, versatility, and how they are trained. Think of a Foundation Model as the “foundation” of a skyscraper: a solid base that can be used to build different types of apartments, offices, or shops on top. A traditional model, on … Read more
If you follow the evolution of LLMs even slightly, you’ve likely noticed the constant shifts and developed your own personal usage preferences. Looking at the chart below (from LMArena – Goldman Sachs Global Investment Research), the changes are undeniable. The Goldman Sachs data is brutal: what was once a cutthroat “knife fight” between startups has … Read more