Qlaudia

← All courses

Non LLM AI Survey

  • Artificial Intelligence
  • Technology & Computing

Explore eight major AI problem families—computer vision, tabular data, recommendations, time series, audio, image generation, reinforcement learning, and domain-specific models—each with purpose-built tools that outperform LLMs. Learn to recognize problem structures and match them to specialized architectures rather than defaulting to language models.

Sign in to enroll$1033 lessons

What you'll learn

  • Model Deployment Trade-offsDeploying an AI model in production requires balancing latency, computational cost, infrastructure complexity, and maintainability — constraints that often override academic accuracy when making architectural decisions.
  • Cloud AI APIs for Non-LLM TasksManaged, pre-trained models exposed as HTTP services by cloud providers for computer vision, audio analysis, document processing, and forecasting without infrastructure management.
  • LLM Limitations and Trade-offsLLMs are sophisticated pattern-matchers optimized for language understanding that sacrifice task-specific accuracy for breadth across many problems.
  • AI Architectural Decision MakingThe practice of evaluating multiple AI approaches before committing to one based on problem structure, data characteristics, and operational constraints.
  • Embedding and Latent Space ConceptsA dense numeric vector that encodes the meaning or characteristics of data as a fixed-length list of numbers existing in an abstract mathematical space.
  • Scientific and Specialized AI ModelsPurpose-built AI systems designed for specific scientific domains that handle structured, non-textual data like physical grids, molecular graphs, and genomic sequences more effectively than general-purpose language models.
  • Problem-Tool Fit RecognitionThe skill of matching a specific problem to the most appropriate AI technique based on requirements for accuracy, latency, cost, and structure rather than defaulting to general-purpose solutions.
  • Anomaly DetectionThe task of identifying data points, events, or sequences that differ significantly from established patterns of normal data.
  • Recommendation System Problem TypeA matching problem that identifies items from a large catalog a user is most likely to engage with, given sparse interaction data across thousands to millions of users.
  • Computer Vision Task TypesA categorization of visual problems including image classification, object detection, facial recognition, and medical imaging, each with distinct outputs and model requirements.
  • Tabular Data CharacteristicsData organized into rows representing observations and columns representing attributes, where each row is a record and columns can be numerical or categorical values.
  • Image Classification ModelsNeural networks that assign category labels to images by learning hierarchical features through convolutional filters that detect increasingly complex spatial patterns.
  • Graph Neural NetworksNeural network architectures designed to operate directly on graph-structured data by aggregating information from local neighborhoods rather than requiring fixed-size inputs.
  • Domain-Tuned TransformersTransformers pre-trained or fine-tuned on large bodies of domain-specific data that learn the native data format and linguistic patterns of a particular field.
  • Gradient Boosted TreesAn ensemble machine learning method that sequentially builds decision trees where each tree corrects the errors made by all previous trees in the sequence.
  • Time Series Problem IdentificationA time series problem involves data ordered by time where that order carries meaning and the goal depends on understanding temporal structure like forecasting or anomaly detection.
  • LSTMs for Time SeriesLSTMs are recurrent neural networks designed to learn long-range dependencies in sequential data by solving the vanishing gradient problem that limits standard RNNs.
  • When to Use Tree-Based ModelsTree-based models like XGBoost and LightGBM are the appropriate choice when working with structured tabular data organized into rows and columns with well-defined features.
  • Dedicated Audio ArchitecturesPurpose-built neural network models optimized for audio-native tasks like wake word detection and sound classification by exploiting the sequential and frequency-based structure of audio signals.
  • Reinforcement Learning FrameworkA machine learning paradigm where an agent learns to make sequences of decisions by interacting with an environment and receiving rewards for good outcomes and penalties for bad ones.
  • Diffusion Models for Image GenerationNeural networks that generate images by learning to reverse a process of noise corruption, starting from random noise and iteratively denoising toward a coherent image.
  • Collaborative FilteringA recommendation technique that predicts user preferences by identifying similar users and their interactions, without requiring knowledge of item content.
  • Temporal Fusion TransformersA neural architecture designed specifically for time series forecasting that handles heterogeneous inputs including past observations, known future values, and static metadata.
  • RL Application DomainsProblem contexts where reinforcement learning is appropriate because they involve sequential decisions, clear reward signals, and learning through environmental interaction rather than labeled datasets.
  • Protein Structure PredictionAn AI-solved computational problem that determines the three-dimensional shape a protein folds into based on its amino acid sequence, enabling drug discovery and biological understanding.
  • Object Detection ModelsComputer vision systems that identify what objects are present in an image and localize their positions using bounding boxes paired with class labels and confidence scores.
  • XGBoost and LightGBMXGBoost and LightGBM are the two dominant libraries that implement gradient boosting efficiently and at scale for tabular machine learning problems.
  • Audio AI Task TypesA family of distinct audio machine learning problems—including wake word detection, speech recognition, and music classification—each with specific architectures, data requirements, and evaluation metrics.
  • Classical Forecasting MethodsMathematically grounded time series methods like ARIMA and Prophet that remain effective for production forecasting without requiring deep learning.
  • Two-Tower NetworksA neural network architecture for recommendations consisting of two independent networks that map users and items into a shared embedding space where relevance is measured by vector similarity.
  • LLMs vs Diffusion Models DistinctionLLMs generate discrete token sequences one at a time while diffusion models generate continuous-valued arrays like images all at once, making them fundamentally different tools optimized for different outputs.
  • Matrix FactorizationMatrix factorization decomposes sparse user-item interaction matrices into compact learned representations that enable scalable recommendations by discovering latent factors.
  • Computer Vision APIs and LibrariesCloud-hosted and pre-trained tools that solve common visual tasks by accepting images and returning structured predictions without requiring model deployment or infrastructure management.

About this course

Non-LLM AI: A Map of the Territory

Large language models have become the default hammer for many developers building AI-powered systems. They are capable, accessible, and increasingly easy to integrate. But a hammer is not always the right tool — and in AI, reaching for an LLM when the problem calls for something else carries real costs: lower accuracy, higher latency, unnecessary complexity, and avoidable expense.

This course is organized around a single practical skill: recognizing the shape of a problem and knowing that a purpose-built tool exists for it. You do not need to become an expert in every AI subfield covered here. You need enough awareness to make better architectural decisions — and to know when to bring in someone who is a specialist.

Why This Matters for LLM Developers

If you have spent time building with LLMs, you have probably run into their edges. They are slow compared to lightweight classifiers. They are expensive at scale. They hallucinate on structured data problems. They are overkill for detecting a wake word or flagging an anomaly in a time series. The wider AI ecosystem has spent decades developing specialized tools for exactly these problems, and those tools routinely outperform general-purpose language models on the tasks they were designed for.

The goal here is not to replace your LLM skills — it is to surround them with context, so you know when not to use them.

The Problem-First Mental Model

This course is organized by problem type rather than by technology family, because that is how these decisions actually arise in practice. You are not sitting down to choose between XGBoost and a transformer in the abstract. You are building a feature that needs to flag suspicious transactions, or recommend products, or detect a sound event, and you need to know what kind of problem you are actually holding before you can choose the right tool.

The central organizing question is: What is the structure of my data, and what kind of output do I need?

What is my problem? Structure × Output type Images / Video → Computer Vision Rows & Columns → Gradient Boosting Sequences over time → Time Series Models Users & Items → Recommendation Systems Audio / Sound → Audio Architectures Strategy / Control → Reinforcement Learning Image Generation → Diffusion Models

The central question — what is the structure of my data and what kind of output do I need — branches into distinct AI problem families, each with its own purpose-built tooling.

The Major Problem Families

Computer Vision

When your data is images or video, the computer vision ecosystem offers purpose-built architectures — convolutional networks, detection models, segmentation networks — that are faster, cheaper, and more accurate than asking a vision-capable LLM to do the same work. Tasks like image classification, object detection, facial recognition, and medical imaging each have well-established model families with pre-trained weights you can fine-tune or deploy directly.

Structured / Tabular Data

Rows and columns of records — user transactions, sensor readings, clinical observations — are the natural habitat of gradient boosted trees. Models like XGBoost and LightGBM consistently outperform deep learning on tabular data and are far simpler to train, explain, and deploy. If your problem involves predicting a numerical outcome or classifying records with structured features, this is usually where to look first.

Recommendation Systems

Matching users to items at scale is a domain with its own well-developed toolkit: collaborative filtering, matrix factorization, two-tower networks. These approaches are built for the structural properties of recommendation problems — sparse interaction matrices, millions of items — in ways that prompt-based LLM approaches are not.

Time Series

Forecasting, anomaly detection, and trend analysis all share a defining structure: observations ordered in time, where the sequence matters. Specialized models — from classical ARIMA to LSTMs to Temporal Fusion Transformers — exploit this structure directly. LLMs treat text as a sequence, not sensor readings; the tools designed for time series understand the domain.

Audio

Wake word detection, speaker verification, and sound classification are handled by dedicated audio architectures trained on audio features, not text tokens. These models are leaner and more accurate for their specific tasks than general-purpose language models.

Image Generation

Diffusion models — not LLMs — are the technology behind modern image generation systems like Stable Diffusion and DALL-E. Understanding this distinction helps you integrate the right generative tools for visual applications rather than misattributing the capability to language models.

Reinforcement Learning

When a system must discover a strategy through interaction with an environment rather than by pattern-matching on static data — game-playing agents, robotics, optimization — reinforcement learning is the appropriate framework. LLMs have no native mechanism for learning from trial and error.

Scientific and Domain-Specific Models

Protein structure prediction (AlphaFold), weather forecasting, and drug discovery use architectures purpose-built for their domains — graph neural networks, domain-tuned transformers, physics-informed models. Landmark results in these fields have come not from scaling general-purpose language models, but from encoding domain structure into the architecture itself.

How to Use This Course

For each domain, you will encounter the same sequence of questions:

  1. What does this problem look like in practice? How do you recognize it when you are building something real?
  2. Why is an LLM often the wrong choice here? What specific costs or failure modes does it introduce?
  3. What technology family handles this well? What is the core idea behind it?
  4. Where do you go to use it? Which libraries, APIs, or cloud services give you access?

You are not expected to implement these methods from scratch. You are expected to recognize the shape of a problem, know that a better tool exists, and know where to reach for it — or when to bring in someone who knows that tool deeply.

The Broader Takeaway

LLMs are remarkable general-purpose tools, but general-purpose is not the same as best-in-class. Every problem family covered in this course has specialist methods that have been refined over years or decades for exactly that problem. The developer who knows when to use them — and when to step away from the LLM — builds systems that are faster, cheaper, more accurate, and easier to maintain.

Non LLM AI Survey — Qlaudia