What is LLM?

Tensor Owl started using ChatGPT two years ago. Initially, he hesitated to rely on it because sometimes it hallucinated and gave incorrect answers, while at other times its responses felt incomplete or frustrating.

Slowly, the tools improved a lot over time.

Back then, he did not really understand why systems like ChatGPT were called Large Language Models.

The word “Large” referred to the enormous amount of data the system was trained on — books, websites, articles, and conversations from across the internet.

“Language” meant the model mainly worked with human language: reading it, finding patterns in it, and generating responses from it.

Once Tensor Owl understood the meaning behind the model’s name, he became curious about something else:

How does a Large Language Model actually work?

He had seen AI systems before, such as image recognition models, but ChatGPT felt different. It could answer questions, write code, summarize documents, and hold conversations.

So how was it able to do all of this?

Let’s look at the process step by step.

How LLM Works?

An LLM is an AI model that takes human language as input, finds patterns in it, and predicts what comes next.

However, an LLM does not understand language the same way humans do.

Stage 1: Input

The process starts when a user gives a prompt to the model. Since the model does not understand language the way humans do, it first converts the input into smaller pieces called tokens.

What is the color of the sky?

Tokens = 'What' | 'is' | 'the' | 'color' | 'of' | 'the' | 'sky' | '?'

Breaking language into tokens helps the machine represent language mathematically, which makes it easier for the model to process patterns and predictions.

Stage 2: Embedding

After tokenization, the model converts each token into numerical representations called embeddings. These embeddings help the model understand relationships and patterns between different words.

Sample Sky Embedding

'sky' → [0.24, -0.81, 0.56, 0.12]

Machines process these numerical representations using mathematical calculations inside neural networks.

These mathematical calculations are designed in such a way that words used in similar contexts often end up having similar numerical representations.

For example, the words “sky” and “cloud” often appear together in sentences such as “Clouds move across the sky.”

Because of this, their numerical representations inside the model may end up closer to each other.

Stage 3: Contextual Attention

In human language, the exact same word can mean different things depending on its neighbors.

The Context Problem

Sentence A: 'The bank of the river.'
Sentence B: 'The money is in the bank.'

The word “bank” looks identical in both sentences. However, the meaning changes completely based on the surrounding text.

So the model needs to see how neighbor words affect each other. It does this using a tool called Attention.

Attention acts like a set of spotlights. When the model reads the word “bank” in Sentence A, it shines a spotlight on the word “river.” In Sentence B, it shines the spotlight on “money.”

By looking at these connected words, the model figures out the correct meaning of the word before moving forward.

Stage 4: Pattern Recognition

The model takes these numerical representations (embeddings) and starts detecting relationships and repeated structures between them. This process is called pattern recognition.

Simple Pattern

'I am eating', 'I am reading', 'I am coding' → model notices the repeating structure: 'I am + action'

So the model passes them through multiple neural network layers. At each layer, the model checks patterns in the sequence — what usually comes before and what is likely to come next.

As the model sees billions of examples during training, it slowly learns:

• sentence structures
• grammar patterns
• relationships between words
• common sequences in language

For example, if a user asks: "Will it rain today?"

the model activates patterns related to:

• weather
• clouds
• temperature
• forecasts

The model is not consciously thinking like humans. It is mathematically detecting patterns that are most relevant to the given input.

Stage 5: Prediction

After detecting patterns, the model predicts the next most likely token in the sequence.

For example, if the user asks: “What is the color of the sky?”

The model does not generate the complete answer at once. Instead, it predicts one token at a time.

Prediction Flow

'The' → 'sky' → 'is' → 'blue' → '.'

The complete response becomes: The sky is blue.

At every step, the model looks at:

• the original input
• previously generated tokens
• learned language patterns

and mathematically predicts what is most likely to come next.

That is why you may sometimes observe systems like ChatGPT or Gemini generating responses one word at a time.

Even though LLMs became extremely powerful at generating language, they mainly learn from patterns in text rather than directly understanding the physical world.

Humans do not learn only from books and conversations. We learn by watching people, observing actions, and experiencing how events unfold around us.

This idea leads us to a different type of AI model called V-JEPA.

What is V-JEPA?

A child learns in multiple ways. One important way is through language, which closely resembles how LLMs learn from text and conversations.

But children also learn by observing the world around them — watching people walk, seeing objects fall, and understanding how actions unfold over time.

V-JEPA is closer to this observation-based style of learning.

Unlike LLMs, which mainly learn from text, V-JEPA learns by watching videos and detecting patterns in movement, actions, and relationships between objects.

By observing how events unfold over time, V-JEPA tries to understand what is likely to happen next in a scene.

As we grow, this process feels simple. We observe what is happening now and often predict what is likely to happen next.

But how does a model learn to do this from videos?

Let’s look at the process step by step.

How V-JEPA Works?

V-JEPA learns by observing videos and detecting patterns between different moments in time.
Imagine the model watching many videos of a child throwing a ball:

In these videos, it sees a sequence:

• the child holds the ball
• pulls the arm back
• swings the arm forward
• releases the ball
• the ball moves through the air

As the model watches many such videos, it gradually learns how actions and movements usually happen in the real world.

It learns that releasing the ball is usually followed by the ball moving through the air, not by the ball suddenly appearing somewhere else.

Unlike a normal image, a video contains actions unfolding over time.

V-JEPA tries to understand that actions have a sequence — a beginning, a middle, and an outcome.

However, unlike some older video models, V-JEPA does not try to memorize every single pixel in the video. Instead, it focuses on understanding important patterns within a scene:

• movement
• actions
• relationships between objects
• changes over time

Rather than predicting the next word like an LLM, V-JEPA tries to predict what is likely to happen next in a scene.

This helps V-JEPA build an understanding of how the physical world works by observing events as they unfold.

V-JEPA also follows a step-by-step learning process.

Stage 1: Video Input

V-JEPA takes video frames as input instead of text.
For example, it may observe many videos of children playing with balls. In these videos, it sees actions such as holding a ball, throwing it, catching it, and watching it move through the air.

Stage 2: Visual Embeddings

The model converts important visual information from the video into numerical representations called embeddings.

These embeddings capture information such as:

• the child
• the ball
• their positions
• their movement over time

Stage 3: Pattern Recognition

The model processes these embeddings through neural network layers and begins detecting patterns.

For example, it learns that:

• a child pulling an arm back is often followed by a throwing motion
• releasing a ball is often followed by the ball moving through the air
• actions tend to occur in a particular sequence

Stage 4: Future Prediction

V-JEPA then predicts what is likely to happen next in the scene.

For example, after seeing a child release a ball, it may predict that the ball will continue moving through the air rather than suddenly changing position.

LLM vs V-JEPA

Both LLMs and V-JEPA are AI models, but they learn differently.

LLMs mainly learn from language and text. V-JEPA mainly learns from videos and observation.

An LLM predicts: what word is likely to come next.

V-JEPA predicts: what is likely to happen next inside a scene.

A quick note on their differences:

LLM	V-JEPA
Learns patterns from text and language	Learns patterns from videos and visual observations
Predicts the next word	Predicts what may happen next in a scene
Works with tokens	Works with visual embeddings
Focused on language patterns	Focused on visual patterns and events
Strong in conversations and coding	Strong in understanding actions and movement

🧾 Summary

    LLMs learn from language and text, while V-JEPA learns from videos and observation.
    Before processing language, LLMs convert text into tokens and embeddings.
    LLMs detect patterns in language and predict one token at a time to generate responses.
    V-JEPA learns by observing how actions and events unfold over time inside videos.
    V-JEPA converts visual information into embeddings and learns patterns in movement, actions, and object relationships.
    Instead of predicting the next word, V-JEPA predicts what is likely to happen next in a scene.
    LLMs help AI understand language, while V-JEPA helps AI understand the physical world through observation.

Summary – Voice Recording

Conclusion

LLMs and V-JEPA represent two different approaches to learning:

• LLMs learn from language and become skilled at generating text, answering questions, writing code, and reasoning through words.
• V-JEPA learns through observation, detecting patterns in actions, movement, and how events unfold over time.

Tensor Owl feels that as AI continues to evolve, future systems may combine language understanding with observation-based learning, bringing them closer to how humans learn from both conversation and experience.

🔎 Recap & Reflection

An LLM has seen millions of sentences containing the pattern: 'Birds can fly.' Later, it is asked whether a penguin can fly. Why might this be a challenging situation for the model?

Embeddings cannot represent birds

The tokenizer removes uncommon animals

The model may rely on common language patterns and overlook exceptions

The model can only answer questions about mammals

A student asks the same question to two different LLMs. Both answers are reasonable, but the wording and examples are completely different. Which explanation best describes this behavior?

LLMs retrieve answers from a database, and the two models may have different databases

LLMs generate responses by predicting likely token sequences, and multiple valid sequences may exist

The tokenizer randomly changes the meaning of the question

The models made mistakes while converting the question into tokens

A researcher trains an LLM using billions of text documents. Later, the model confidently generates an incorrect explanation for a scientific question. Which statement best explains how this can happen?

The model stops using embeddings when answering science questions

The model forgets its training data after deployment

The tokenizer changes scientific terms into random words

The model predicts likely language patterns, not guaranteed facts

An LLM reads two sentences: 'She ate an apple.' 'Apple released a new iPhone.' How can the model understand that the word 'Apple' refers to different things?

The tokenizer creates a different word each time

The model uses surrounding words to determine the intended meaning

Embeddings give every word a fixed meaning

The model ignores context and guesses randomly

A researcher wants to build an AI model that can understand the physics of motion and predict how objects are likely to move in the future. He has collected hundreds of videos showing objects being thrown, falling, rolling, and colliding with each other. Which type of model is most aligned with this goal?

LLM, because it specializes in predicting the next word in a sequence

V-JEPA, because it learns patterns from observations and predicts future events in a scene

Both models are equally focused on understanding physical motion

Neither model can learn from examples

A robot is standing near a table. A cup has started sliding toward the edge, and the robot must decide whether it has enough time to catch it before it falls. Which capability becomes most important for the AI controlling the robot?

Generating detailed explanations about cups

Predicting the next word in a sentence

Understanding how actions and physical events are likely to unfold

Detecting grammar mistakes in instructions

An AI assistant suggests adding glue to pizza to help cheese stick better. The answer sounds confident, but the advice is clearly incorrect. What does this best demonstrate?

The model predicted a plausible-looking response based on patterns rather than verifying the fact

The tokenizer incorrectly processed the word 'pizza'

The embeddings stopped working

The model forgot its training data