Benjamin

With a degree in Computer Science and an graduate in Electrical Engineering, he is a Professor at the Federal University of Santa Catarina (UFSC) - Joinville. His teaching and research focus on programming, intelligent systems, data science, and systems modeling.

Beyond LLMs: Discover the New Types of AI Models (VLM, SLM, LAM, and More)

by

If you follow artificial intelligence news, you have certainly heard of LLMs (Large Language Models), the technology behind tools like ChatGPT and Google Gemini. However, the AI universe has expanded dramatically. Today, engineers and researchers are developing highly specialized architectures to solve problems that pure text simply cannot address. Below, we explore the main new … Read more

Explaining the Self-Attention Mechanism in LLMs

by

The Self-Attention mechanism is the true heart of the Transformer architecture, the foundation behind every modern Large Language Model (LLM). Before Self-Attention was introduced in the landmark paper “Attention Is All You Need” (Vaswani et al., 2017), language models processed text sequentially, word by word (or token by token), using Recurrent Neural Networks (RNNs). This … Read more

Explaining Multi-Head Attention (MHA)

by

To understand Multi-Head Attention (MHA), we first need to take a step back and examine the limitation of applying the attention equation (Scaled Dot-Product Attention) only once. The Problem with a Single “Head” Human language is complex and possesses multiple layers of meaning that occur simultaneously. Consider the sentence: “The student delivered the book to … Read more

Explaining Quantization

by

Quantization is one of the fundamental optimization techniques in the field of Machine Learning. It allows for the compression of Large Language Models (LLMs), dramatically reducing memory consumption (VRAM/RAM) and speeding up response times, with minimal loss in reasoning capability. In simple terms, quantizing is reducing the numerical precision of a neural network’s weights. What … Read more

What are Harness Engineering and Loop Engineering, and How Do They Differ?

by

The rapid evolution of AI agents (agentic systems) has brought a surge of new terms to software architecture vocabulary: Prompt Engineering, Context Engineering, Harness Engineering, and, more recently, Loop Engineering. With so many neologisms, it is easy to dismiss these names as mere buzzwords. However, the confusion between Harness Engineering and Loop Engineering is the … Read more

What is a Foundation Model?

by

The main difference between a Foundation Model and a traditional (non-foundation) model lies in their scale, versatility, and how they are trained. Think of a Foundation Model as the “foundation” of a skyscraper: a solid base that can be used to build different types of apartments, offices, or shops on top. A traditional model, on … Read more