Can a Model Learn Something That Nobody Explicitly Taught It?
On implicit structure, emergent representations, compositional generalization, and the boundary between discovering latent patterns and inventing new facts.
We often describe machine learning as if it were a very sophisticated version of teaching.
You give a model examples.
You provide the correct answers.
The model adjusts its parameters until it becomes good at producing those answers.
So it seems reasonable to think:
“A model can only learn what we teach it.”
But that intuition starts breaking down when we look at modern machine learning systems.
A model can sometimes discover patterns, representations, and even useful relationships that were never explicitly written down as instructions or labels.
So the interesting question is:
“Can a model learn something that nobody explicitly taught it?”
The short answer is yes — but with an important distinction.
A model cannot magically obtain information that has no connection to its training signal or data.
But it can discover structure that was implicit in the data, even when nobody explicitly told it to look for that structure.
1. What Does “Teach” Actually Mean?
Suppose we want to train a model to recognize cats.
We give it thousands of images:
Nobody explicitly tells the model:
- “Cats usually have two triangular ears.”
- “Cats tend to have whiskers.”
- “A cat's eyes are usually positioned approximately like this.”
Instead, the model receives examples and a learning objective.
During training, it changes its parameters to reduce its error.
Eventually, internal representations may emerge that respond strongly to things such as:
Nobody necessarily programmed these features into the model.
They emerged as useful representations for solving the task.
Learning does not require every learned feature to be explicitly specified.
2. Explicit Knowledge vs. Implicit Structure
Consider a dataset containing sentences:
Nobody gives the model a grammar textbook.
Nobody writes:
Yet a sufficiently capable language model can develop representations that reflect grammatical structure.
Why?
Because grammar is already implicit in the data.
The model is exposed to enormous numbers of examples in which words interact according to grammatical patterns.
It doesn't need someone to explicitly say:
“This is a noun.”
The statistical structure of the data provides enough information for the model to discover useful internal representations.
This is similar to how a child can hear thousands of sentences without receiving a formal lecture on syntax.
3. The Model Isn't Learning From Nothing
There is an important caveat here.
When we say:
“The model learned something nobody taught it.”
we shouldn't interpret that as:
“The model created completely new knowledge from nothing.”
That isn't what is happening.
Suppose we give a model:
and it learns:
Nobody explicitly taught it that equation.
But the information required to infer the relationship was already present in the examples.
The model discovered a relationship contained within the data.
This distinction is fundamental.
- • relationships
- • correlations
- • abstractions
- • representations
- • latent structure
- • patterns
- • combinations of existing information
It cannot simply invent factual information about the world without some source of information.
The training data, environment, feedback, architecture, or objective must provide some signal.
4. The Interesting Part: Emergent Representations
This becomes much more fascinating with neural networks.
Imagine a neural network processing images.
Early layers might learn representations related to:
Nobody explicitly assigns these responsibilities.
The network is simply optimizing its parameters.
Yet useful hierarchical representations can emerge:
The network wasn't necessarily instructed:
“First learn edges, then learn shapes, then learn faces.”
The optimization process found representations that were useful for minimizing the training objective.
This is one reason neural networks can feel surprisingly intelligent.
Their internal organization can contain abstractions that were never directly specified by the programmer.
5. Self-Supervised Learning Makes This Even More Interesting
Modern large language models provide an even clearer example.
Consider the task:
During training, the model might be asked to predict:
The training objective is incredibly simple:
Predict the next token.
Nobody necessarily creates a separate lesson called:
Yet the model can develop knowledge about:
from a relatively general training objective.
Why?
Because solving the prediction problem requires understanding statistical regularities in the data.
If predicting the next word requires understanding a sentence's structure, then learning that structure becomes useful.
6. A Simple Example
Imagine we train a model on:
Now suppose we ask:
The model can predict:
But imagine the dataset becomes much larger:
Nobody explicitly tells the model:
But the model can discover that association because it repeatedly appears in the data.
The relationship wasn't explicitly labeled.
It was latent in the examples.
7. So Is the Model Actually “Understanding”?
This is where things become philosophically interesting.
Suppose a model learns that:
and then predicts:
Did it understand the relationship?
There are several possible interpretations.
The model has learned a useful internal representation that allows it to generalize beyond the exact examples it saw.
It has simply learned statistical correlations.
The difficult part is determining where one ends and the other begins.
A model can sometimes produce behavior that looks like reasoning without necessarily possessing the same kind of conceptual understanding humans have.
Therefore, we should distinguish:
A model can demonstrate a capability without us fully understanding how that capability is represented internally.
8. The Model Can Combine Things Nobody Combined
Here's an even more interesting phenomenon.
Suppose a model has learned:
and it has seen:
Now we ask:
“Describe a red dog driving a car.”
Nobody necessarily showed it that exact sentence.
Yet the model can combine previously learned concepts:
into a new composition.
This is called compositional generalization.
The model isn't necessarily learning an entirely new fact.
Instead, it is recombining existing representations in a new configuration.
And this is one of the most important ways models can produce outputs that weren't explicitly present in their training examples.
9. What About Truly New Knowledge?
Now we reach the boundary.
Suppose nobody has ever observed something.
Nobody has written about it.
It isn't present in the training data.
No feedback contains information about it.
And the model has no interaction with the environment that reveals it.
Can the model learn the fact?
No.
For example, imagine a completely unknown physical phenomenon occurs tomorrow.
If the model has:
- • never seen it,
- • never received measurements of it,
- • never interacted with it,
- • and has no information from which to infer it,
then the model cannot simply learn the exact new fact.
There has to be an information channel.
“A model cannot learn information without information, but it can discover structure within information that nobody explicitly encoded.”
10. Where Does the “New” Knowledge Come From?
We can think about learning as a transformation:
The surprising part is that the final representation doesn't have to resemble the way humans described the training data.
Suppose the dataset contains millions of images.
Humans might describe them using:
But the neural network may encode information in a completely different high-dimensional representation.
A concept might not correspond to one neuron.
Instead, information may be distributed across many dimensions.
The representation is not necessarily human-readable.
Yet it can still be extremely useful for prediction.
11. The Role of the Objective Function
There's another piece people often overlook:
What the model is rewarded for matters enormously.
Imagine two models receiving exactly the same dataset.
They may discover completely different internal structures.
The data provides the raw information.
The objective determines which structures are useful to discover.
This is why the learning process is not simply:
It is closer to:
The architecture and training procedure influence what the model can discover efficiently.
12. This Is Why Scale Matters
A small model might fail to discover a complicated relationship.
A larger model trained on more diverse data might discover it.
This is one reason researchers observe capabilities appearing as models become larger and training becomes more sophisticated.
However, we should be careful with the word emergence.
A capability appearing suddenly in evaluation does not necessarily mean the model suddenly acquired a completely new cognitive mechanism.
Sometimes a smooth improvement crosses a threshold where the benchmark starts showing the capability clearly.
Observed emergence does not automatically tell us what happened internally.
13. A Model Can Discover a Shortcut Too
There is also a darker side to this phenomenon.
Suppose we want a model to distinguish:
and most wolf photographs in the dataset happen to contain snow.
The model might learn:
instead of learning the actual visual differences between wolves and dogs.
Nobody taught:
“Snow means wolf.”
The model discovered the correlation itself.
But the correlation is misleading.
This is called shortcut learning.
So the ability to discover hidden structure is not automatically beneficial.
The model doesn't inherently know which discovered pattern represents the “right” explanation.
It optimizes the objective.
If a shortcut works, the optimizer may happily use it.
14. This Changes How We Think About Machine Learning
Traditional programming looks like:
Machine learning is closer to:
The programmer specifies the learning setup.
The model determines many of the internal representations.
That is the fundamental shift.
Instead of explicitly programming:
we provide examples and let optimization find useful parameter configurations.
15. The Strange Part: We Don't Always Know What It Learned
Modern neural networks can contain billions of parameters.
Even when a model performs a task correctly, determining exactly what representation it developed can be difficult.
This is the motivation behind fields such as:
Researchers attempt to identify:
- • features
- • circuits
- • representations
- • attention patterns
- • neuron behaviors
- • computational pathways
inside neural networks.
The goal is not simply:
“Does the model give the correct answer?”
but:
“What computation happened inside the model to produce that answer?”
That's a much harder question.
16. So, Can a Model Learn Something Nobody Taught It?
Yes.
But we need to be precise about what “nobody taught it” means.
A model can learn:
But it cannot obtain arbitrary information that has no informational basis in its data, environment, feedback, or prior structure.
17. The Bigger Question
And this leads to a much deeper question:
If we give a sufficiently capable learning system:
- • enough data,
- • a suitable objective,
- • sufficient computational capacity,
- • and an architecture capable of representing complex relationships,
How much structure can it discover without us explicitly specifying that structure?
That question sits at the heart of modern AI research.
Because perhaps the most interesting property of a learning system isn't that it can memorize what we show it.
It's that:
We can specify the problem without specifying the complete solution.
We define the objective.
We provide the information.
We construct the learning machinery.
And optimization searches through an enormous space of possible representations until it finds configurations that work.
The resulting model may contain patterns that no human explicitly programmed and nobody explicitly labeled.
That doesn't mean the model learned from nothing.
It means something more subtle happened:
The knowledge was implicit in the information, while the structure was discovered by the learning process.
And that may be one of the most important ideas behind modern machine learning.
The One-Sentence Takeaway
“A model cannot learn information from nothing, but it can discover relationships, representations, abstractions, and combinations that nobody explicitly taught it — as long as the information needed to infer them exists somewhere in its learning process.”
Learning is not mere transcription.
It is the compression and discovery of implicit structure.
By shifting from explicit rule formulation to high-dimensional loss minimization, we allow neural networks to crystallize abstractions that no programmer ever needed to dictate.