홍합탕's Blog

A Laptop VLM Solves 44% of Naver's Receipt CAPTCHAs

2026-09-30T12:01:00+00:00

If you log in to Naver often enough, you’ll eventually see a receipt instead of a login form. It’s photographed at an angle, torn into strips, and covered in scribbled letters, and underneath is a question:

What is the unit price of the least bought item per unit?

This is Naver’s receipt CAPTCHA. The idea is clever: instead of asking you to decode wavy letters, it asks you to read a messy document and reason about it. The catch is that reading messy documents and reasoning about them is exactly what vision–language models (VLMs) are trained to do.

So I measured it. The short version:

An unmodified, open-weight 7B model on a 16 GB laptop answers 43.8% of real challenges correctly.
The 7B model gets ~62% of the “just read it off the receipt” questions right, despite the tears and scribbles.
It fails mainly on multi-step arithmetic, such as totals and “unit price of the least bought item”.
If failed attempts can be retried on fresh challenges, 43.8% per attempt means a ~94% chance of passing within five tries.

A longer write-up with confidence intervals and significance tests is coming as a preprint. This post is the readable version.

What the challenge looks like

A real challenge from April 2026. Question: “How many kinds of items did the customer purchase?”

Every challenge is a 670×320 image plus one English question. The defenses are all about layout, not lettering:

the receipt is torn into two or three strips, rotated and bent in perspective;
handwritten letter/digit strings are scattered on top, sometimes right over the prices;
parts of the receipt are cut off or hidden in the tear;
the column order (Price / Qty / Sum / Name) changes from receipt to receipt.

The printed text itself is a clean, ordinary font. A model that can locate the right row can read it.

The questions are surprisingly repetitive

I collected 1,100 challenges on 9 April 2026. Across all of them there were only 295 distinct question strings, and nine fixed templates account for most of them:

Question type	Share of challenges
Counting / quantity (“How many items cost 370 won per unit?”)	43.8%
Unit price of the most expensive / most bought item	14.2%
Unit price of a named item (“…of frozen Skate?”)	10.2%
Unit price of the cheapest item	7.5%
Unit price of the least bought item	7.4%
Receipt total	5.8%
Address fill-in-the-blank	4.9%
A digit of the store’s phone number	3.9%
Rare types	2.0%

These split into two kinds:

Lookup questions, where the answer is printed on the receipt (a named item’s price, the address, a phone digit).
Aggregate questions, where the answer has to be computed by counting, comparing, summing, or dividing a line total by its quantity.

That distinction turns out to explain almost everything.

The experiment

I hand-labeled 128 of the collected challenges and asked three open-weight models from the Qwen VL family to answer them zero-shot: no examples, no fine-tuning, no step-by-step prompting. The prompt was just the image, the question, and “return only the answer”. Everything ran locally on an M1 Pro laptop with 16 GB of memory.

Model	Correct	Accuracy (95% CI)
Qwen2-VL-2B	20 / 128	15.6% (10.3–22.9)
Qwen2.5-VL-3B	34 / 128	26.6% (19.7–34.8)
Qwen2.5-VL-7B	56 / 128	43.8% (35.5–52.4)

Both jumps (2B→3B and 3B→7B) are statistically significant on paired tests (p = 0.024 and p < 0.001). My labeled set slightly over-represents the easy named-item price questions. Reweighting to the real question mix brings the 7B model to about 37%, still roughly three in eight.

Where the 7B model succeeds, and where it doesn’t

	7B accuracy
Lookup questions (47 items)	61.7%
Aggregate questions (81 items)	33.3%

The torn strips and scribbles didn’t stop it from reading: it got 8 of 10 address blanks and 16 of 27 named-item prices. What it can’t do yet is the arithmetic:

Receipt totals: 0 of 8. It answers with a plausible amount of the right size (7,000 for 6,800; 12,000 for 10,300) but never the exact total.
“Least bought item” unit price: 0 of 4. It often returns a price from the wrong row, or a line sum instead of a unit price.
Counting shortcuts. For “How many items cost N won per unit?” it often just says 1, and for “total number of products” it often says 10 or 100. It isn’t really attempting the count.

In other words, the model reads the receipt fine and skips the procedure. That gap is the one that step-by-step prompting, tool use, and task-specific fine-tuning are known to close.

I also screened five other open models on a small 17-item pilot. MiniCPM-V was about as good as Qwen-7B, while LLaVA-7B and BakLLaVA barely registered. Llama 3.2-Vision 11B scored 15% over 60 items, mostly because it kept answering in full sentences. Parameter count alone doesn’t predict success here.

What 44% means in practice

A CAPTCHA doesn’t have to be unbeatable; it has to make automation expensive. If each failure brings a fresh, independent challenge, per-attempt accuracy compounds quickly:

Attempts	Chance the 7B model has passed
1	43.8%
3	82.2%
5	94.4%
10	99.7%

Three things stand out:

The visual defenses don’t block reading. The torn, scribbled receipt stopped the 7B model on only about 40% of lookup questions.
The remaining margin is arithmetic, which is the capability current models are improving fastest at.
It all runs on the attacker’s own laptop. There’s no per-query cost and no third-party solving service, so API pricing and similar controls don’t apply.

Important caveats. This is a per-challenge accuracy, not an end-to-end bypass rate. I didn’t test Naver’s rate limits, lockouts, or risk scoring, and I never submitted an answer to the live site. Any of those could cut the number of retries an attacker actually gets. I also didn’t measure how well people do. Anecdotally, the least-bought-unit-price questions are a chore for humans too, and that deserves a proper user study.

Correcting the record on fine-tuning

An earlier draft of this project claimed that a LoRA adapter on the 2B model “didn’t help”. When I audited the saved files, that experiment turned out to be broken rather than negative:

the adapter’s trained weights (its B matrices) were exactly zero, meaning it never learned anything;
its “results” matched the untrained model’s answers on all 120 items;
the evaluation step in the pipeline was a placeholder;
and it trained on the same items it was tested on.

So that claim is withdrawn.

A corrected run is in progress now: Qwen2.5-VL-7B (4-bit, MLX) with a LoRA adapter, evaluated by 4-fold cross-validation. Each adapter trains on 96 labeled challenges and is scored only on the 32 it never saw. The same 4-bit model with the same prompt scores 59 / 128 (46.1%) zero-shot, which is the baseline to beat.

The fine-tuning run is still going. I’ll update this post with the held-out results, gain or no gain, when it finishes.

What Naver could do

Don’t rely on this challenge alone. Pair it with server-side risk signals and per-account / per-IP attempt limits, so the retry math above stops working.
Re-test regularly against current open models. A defense that holds against last year’s models won’t necessarily hold against next year’s.
Make questions easy for people and hard for models, not just harder. Harder arithmetic mostly burdens legitimate users. Any change should be validated with a human study.

Limitations

Small sample. 128 labeled items give roughly ±8-point intervals overall, and most per-type numbers rest on a handful of items.
Confounded 7B comparison. The 7B zero-shot run used a different runtime, quantization and prompt from the 2B/3B runs. The 4-bit MLX rerun (59/128) lands in the same place, which is reassuring.
Labels. One annotator, with no confirmation from Naver’s server. A spot-check of 18 labels found 2 errors, and neither changes any score.
Single snapshot. All data come from one day in April 2026, and the challenge may have changed since.

Ethics

I collected challenge images only, never submitted answers, never touched anyone else’s account, and I’m not releasing a ready-to-use solver or trained adapter.

Language Notes Repos

2026-03-09T00:00:00+00:00

It’s my personal passion to learn new languages, and it matters to me because it keeps me curious, humble, and connected to people I would otherwise never really meet. I like to keep my new words with me on the go. You can find them here for your perusing, and I’m curious how others do it too. Are Anki cards the most popular? Should I switch?

If you’re working through a language and want to compare approaches, feel free to browse or fork. These are living scratchpads, not polished textbooks, but that’s part of the fun.

한글 낙서: Hangul Graffiti

2026-03-09T00:00:00+00:00

There is something disarming about a language that bows before it speaks.

Korean does not merely encode information. It encodes relationships. Every verb ending is a social contract – a declaration of how you see the person standing in front of you. In English, “please sit down” works for your boss and your dog. In Korean, you’d better know the difference between 앉으세요 and 앉아, or you will insult one and confuse the other.

I have been collecting notes on Korean for a few years now. What follows is a distillation of those notes, organized not as a textbook would but as a learner actually encounters the language: in gyms, in novels, in text messages, and in the strange space between what is said and what is meant.

The Architecture of Hangul

King Sejong the Great (세종대왕) invented Hangul in 1443, and the story is almost too good to be true. The consonant shapes are modeled after the physical position of the tongue and mouth when you pronounce them:

ㄱ (g/k): the back of the tongue rising toward the soft palate
ㄴ (n): the tongue touching the upper gum ridge
ㅁ (m): the shape of closed lips
ㅅ (s): the shape of a tooth
ㅇ (ng/silent): the shape of the throat

Vowels are built from three elements: a dot (representing the sun/heaven), a horizontal line (the earth), and a vertical line (a person standing). From these three primitives, the entire vowel system unfolds:

Vowel	Sound	Construction
ㅏ	a	vertical + right dot (bright)
ㅓ	eo	vertical + left dot (dark)
ㅗ	o	horizontal + top dot (bright)
ㅜ	u	horizontal + bottom dot (dark)
ㅡ	eu	horizontal line alone
ㅣ	i	vertical line alone

This is a writing system designed by committee – but the good kind, the kind where the committee included phonologists. The result is that Korean is arguably the most rationally designed script in active use anywhere on Earth.

Pronunciation: The Rules They Don’t Teach First

Two rules I picked up early that cleared up a lot of confusion:

1. ㅅ before ㅣ or ㅑ/ㅕ/ㅛ/ㅠ becomes “sh”

The consonant ㅅ is normally an “s,” but place it before any “i” or “y” vowel and it palatalizes to “sh.” This is why 시 sounds like “shi” and 신문 (newspaper) is “shin-mun,” not “sin-mun.” The word 시작 (beginning) is “shi-jak,” and 식당 (restaurant) is “shik-dang.”

2. The silent 받침

When ㄹ sits in the 받침 (bottom consonant position), it behaves differently than you’d expect. Korean syllable blocks stack consonants and vowels into squares, and the bottom slot – the 받침 – follows its own rules of liaison and assimilation. The consonant at the bottom of one syllable bleeds into the top of the next, creating pronunciation chains that make spoken Korean sound nothing like its spelling suggests.

For example:

독립 (independence) is pronounced “dong-nip” not “dok-lip”
한국어 (Korean language) is pronounced “han-gu-geo” – the ㄱ 받침 links to the next syllable’s vowel

Grammar: Hierarchy in the Verb

Korean has seven speech levels, though modern usage mostly collapses these into four. The critical insight is that the verb ending changes based on your relationship to the listener – not the subject, the listener:

Level	Ending	When to Use	Example (to go)
Formal polite (합쇼체)	-ㅂ니다 / -습니다	Business, news, strangers	갑니다
Informal polite (해요체)	-아요 / -어요	Default safe choice	가요
Casual (해체)	-아 / -어	Close friends, younger people	가
Formal plain (해라체)	-ㄴ다 / -는다	Writing, narration, diaries	간다

The same verb 가다 (to go) is four different social acts depending on the ending. Get it wrong and you’ve made a statement about the relationship, not about where you’re going.

Conjugation in Practice

Unlike European languages with their tables of person and number, Korean verbs don’t conjugate for who is doing the action. They conjugate for how you feel about the person you’re talking to. The subject is often dropped entirely. Context carries it.

Here is 먹다 (to eat) across several constructions:

Form	Korean	Literal
Formal polite	먹습니다	(one) eats [sir/ma’am]
Informal polite	먹어요	(one) eats [politely]
Casual	먹어	eat / eats
Past (polite)	먹었어요	ate
Future (polite)	먹을 거예요	will eat
Negative	안 먹어요	doesn’t eat
Want to	먹고 싶어요	wants to eat
Can	먹을 수 있어요	can eat
Progressive	먹고 있어요	is eating

Notice how the stem 먹 stays constant while the endings do all the work. This is the agglutinative nature of Korean: you stack suffixes like LEGO bricks.

The Particles: Small Words, Heavy Lifting

Korean particles are postpositions – they attach after the noun, not before it. And they do everything.

Particle	Function	Example
은/는	Topic marker	저는 학생이에요 (As for me, I’m a student)
이/가	Subject marker	비가 와요 (Rain is coming)
을/를	Object marker	커피를 마셔요 (I drink coffee)
에	Location / time	학교에 가요 (I go to school)
에서	Location of action	집에서 공부해요 (I study at home)
의	Possession	나의 책 (my book)
도	Also/too	저도 가요 (I’m going too)
한테/에게	To (a person)	친구한테 줘요 (I give it to a friend)

The distinction between 은/는 (topic) and 이/가 (subject) is one of the deepest rabbit holes in Korean linguistics. Roughly: 은/는 sets the frame (“as for X…”), while 이/가 identifies (“it is X that…”). The sentence 제가 학생이에요 emphasizes that I am the student (maybe someone asked “who’s the student?”), while 저는 학생이에요 simply states the fact about me.

At the Gym: 헬스장에서

Some of the most useful Korean I’ve picked up comes from the gym. The phrasebook doesn’t prepare you for wanting to ask someone if they’re done with the squat rack:

Korean	English	Context
몇 세트 남았어요?	How many sets do you have left?	Politely waiting for equipment
좀 보조 해주실 수 있나요?	Can you spot me?	Asking for help on bench press
아직 쓰시고 있나요?	Are you still using this?	Gesturing at a machine
하루에 몇 끼 먹나요?	How many meals do you eat a day?	Gym small talk

And the vocabulary that comes with the culture:

헬창 – literally an abbreviation meaning something like “health fiend.” Used as a playful, almost affectionate compliment among gym-goers, though it sounds crude to outsiders. Think “gym rat” but with more edge.
빵빵하다 – describes big, pumped muscles. 근육이 빵빵! (“Muscles are poppin’!”)
오운완 (abbreviation of 오늘도 운동 완료했다) – “Finished my workout for today.” The hashtag of Korean fitness Instagram. Sometimes abbreviated further to just ㅇㅇㅇ, because even abbreviations get abbreviated.

The counter system reveals itself here too: 끼 is the counter for meals (하루에 몇 끼?), 세트 borrows the English “set,” and 체지방 (body fat) and 체중 (body weight) share the hanja character 체 (body, 體).

Reading Korean Literature: 선화 by 김이

The jump from textbook Korean to literary Korean is a canyon. I tried reading 선화 by 김이, published by 은행나무, and the opening pages alone were a vocabulary tsunami. But the prose was beautiful:

나는 타인의 흉터를 빤히 쳐다보는 버릇이 있었다.

“I had a habit of staring intently at other people’s scars.”

누구든 상처가 있다. 상처에서 흐르던 피가 굳고 딱지가 내려앉고, 딱지가 떨어진 자리에 솟은 새살이 바로 상처를 반추하게 하는 흉터였다.

“Everyone carries wounds. The blood that flowed from wounds dries, scabs settle, and the new flesh that rises where scabs have fallen – that is the scar that makes you ruminate on the wound.”

세상에 나만 흉터가 있는 게 아니었으니까.

“Because I wasn’t the only one in the world with scars.”

The vocabulary of wounds and healing in Korean is evocative:

Korean	English	Notes
흉터	scar	흉 (ugly) + 터 (site) – the site of ugliness
상처	wound	from Hanja 傷處 – place of injury
딱지	scab	also means “ticket” or “tag” – the body’s parking ticket
새살	new flesh	새 (new) + 살 (flesh) – beautifully literal
반추하다	to ruminate	from 反芻 – what cows do, applied to thought
아물다	to heal (a wound)	no hanja, pure Korean
빤히 쳐다보다	to stare intently	빤히 (fixedly) + 쳐다보다 (to gaze up at)

수고하세요: The Farewell That Means “Work Hard”

I wrote about this in Shtetl Length, but it deserves elaboration here.

수고하세요 is a casual farewell rooted in Korean work culture. It literally means something like “please labor/exert yourself,” but in practice it functions as “good work, see you later” or “keep it up.” The upper politeness register manifests as 수고하셨어요 or 수고하셨습니다, used when:

A colleague has finished a full day of work: “Great job today + goodbye”
Someone (like a cashier) has done effort on your behalf: “Thank you for your effort”

As the customer, you could say 수고하셨습니다 as a gesture of gratitude – acknowledging the work the other person has done.

The grammar is revealing:

수고 (苦勞): labor, exertion, toil
하세요: polite imperative of 하다 (to do) – “please do”
하셨어요: past tense honorific – “you did (honorably)”
하셨습니다: past tense formal honorific – the most deferential form

So the farewell literally commands someone to work hard, and the thank-you literally praises them for having worked hard. Working hard is as common a social invocation in Korean culture as invoking God in Latin cultures, or saying “take care” in English. Except 수고하세요 is more specific – it does not wish you wellness, it wishes you productive suffering.

The Korean Keyboard: 두벌식

Learning to type in Korean is its own adventure. The standard Korean keyboard layout (두벌식, “two-set”) splits consonants to the left hand and vowels to the right:

ㅂ ㅈ ㄷ ㄱ ㅅ  ㅛ ㅕ ㅑ ㅐ ㅔ
 ㅁ ㄴ ㅇ ㄹ ㅎ  ㅗ ㅓ ㅏ ㅣ
  ㅋ ㅌ ㅊ ㅍ   ㅠ ㅜ ㅡ

Hold Shift for tense consonants (ㅃ ㅉ ㄸ ㄲ ㅆ) and compound vowels (ㅒ ㅖ). The layout is phonetically organized: consonants on the left, vowels on the right. Your hands alternate with almost every keystroke, which makes Korean typing surprisingly rhythmic once you internalize it.

My early keyboard practice files are pure gibberish – mashing keys to build muscle memory. The word 산화 (oxidation) sits at the bottom of one such file, a lone recognizable word in a sea of random jamo. Progress, I suppose, is measured in the ratio of intelligible words to noise.

What Korean Teaches You About Language

Every language you learn restructures how you think. Spanish taught me that objects can have gender. Portuguese taught me that the subjunctive is not optional. French taught me that spelling and pronunciation exist in separate universes. But Korean taught me something more fundamental: that grammar can encode social relationships, that the verb is not just an action but a posture.

The language forces you to decide, before you open your mouth, who you are in relation to the person you’re speaking to. There is no neutral register. Every sentence is a tiny act of social positioning. And once you internalize this, you start noticing how English accomplishes the same thing through different mechanisms – tone, word choice, the presence or absence of “please” – all the implicit hierarchy that Korean makes explicit.

한국어를 배우는 것은 끝이 없는 여행입니다. 하지만, 그 여행이 제일 재미있는 부분이에요.

Learning Korean is a journey without end. But the journey is the best part.

Sucker Punch Nash Equilibrium

2026-03-04T00:00:00+00:00

Introduction

For those unfamliiar, Pokémon is a turn-based simultaneous action game where players select their moves without knowing the opponent’s choice. Then, after both players have made their selections, the moves are executed based on their priority and the Pokémon’s speed stats in singles battles, that’s just two at once.

Each Pokemon has four moves from which to select, and the outcome of the battle depends on the interactions between these moves and the attributes of the Pokémon species, “held items” that have their own effects, and battlefield properties like weather and terrain. Moves also power points (PP) that limit their usage to a fixed number of times per battle.

Defining the Sucker Punch

What is “Sucker Punch”? It’s a move that permits the user to strike first (+1 priority) if the opponent is about to use an attack. If the opponent is not attacking, the move fails. It has 8 PP. Now, with the mighty Kingambit dominating Generation 9 with its mighty Sucker Punch, the move has become a staple in competitive play.

VS.

Nash Equilibrium

A scenario that often arises is a Sucker Punch end-game. Keeping things simple, we can imagine a +2 Atk boosted Kingambit with 8 PP of Sucker Punch against a weakened Garchomp. Both players are down to one Pokémon, so the winner of this duel determines the fate of the game. Let’s assume if Sucker Punch hits, the Garchomp will faint instantly. Likewise, the Garchomp has an attacking move that can faint the Kingambit in one hit. Now, the Kingambit could also attack directly into the Garchomp’s boosting move, and win, but since it is slower, if the Garchomp player attacks outright, the Garchomp player will strike first and win.

The situation is a Nash equilibrium: if the Kingambit player chooses to use Sucker Punch, they will win if the Garchomp player chooses to attack. However, if the Garchomp player chooses to use a non-attacking move (like Swords Dance), the Kingambit player’s Sucker Punch will fail, resulting in a loss of one PP for the Kingambit. To formalize the “Sucker Punch 50/50,” we must treat it as a finite-horizon stochastic game. We can solve for the Mixed Strategy Nash Equilibrium (MSNE) by using induction on the remaining Power Points ($n$).

1. The Game Model

Let $n$ be the remaining PP of Sucker Punch. We define the game state as $G_n$. In each turn, both players move simultaneously.

Kingambit (K) chooses: Sucker Punch ($S$) or Direct Attack ($A$).
Garchomp (G) chooses: Attacking Move ($M$) or Swords Dance ($D$).

The Rules of Engagement:

If $K$ plays $S$ and $G$ plays $M$: $K$ wins ($Payoff = 1$).
If $K$ plays $S$ and $G$ plays $D$: $S$ fails, PP drops to $n-1$. The game moves to state $G_{n-1}$.
If $K$ plays $A$ and $G$ plays $D$: $K$ wins ($Payoff = 1$).
If $K$ plays $A$ and $G$ plays $M$: $K$ is outsped and loses ($Payoff = 0$).

2. Inductive Equilibrium Analysis

Let $V_n$ be the Value of the Game (Kingambit’s win probability) with $n$ PP remaining.

Base Case: $n=1$

At 1 PP, if Sucker Punch fails ($S, D$), Kingambit has 0 PP left and loses. The payoff matrix for $G_1$ is:

Kingambit \ Garchomp	Attack ($M$)	Swords Dance ($D$)
Sucker Punch ($S$)	1	0
Direct Attack ($A$)	0	1

To find the MSNE, Kingambit plays $S$ with probability $q_1$ such that Garchomp is indifferent. $1(q_1) + 0(1-q_1) = 0(q_1) + 1(1-q_1) \implies q_1 = 0.5$ Thus, $V_1 = 0.5$.

Inductive Step: $n = k$

Assume the value of the game with $k-1$ PP is $V_{k-1}$. The matrix for $G_k$ is:

Kingambit \ Garchomp	Attack ($M$)	Swords Dance ($D$)
Sucker Punch ($S$)	1	$V_{k-1}$
Direct Attack ($A$)	0	1

To find the equilibrium, Kingambit chooses $S$ with probability $q_k$ to make Garchomp’s expected utility for $M$ and $D$ equal: $1(q_k) + 0(1-q_k) = V_{k-1}(q_k) + 1(1-q_k)$ $q_k = V_{k-1}q_k + 1 - q_k$ $q_k(2 - V_{k-1}) = 1 \implies \mathbf{q_k = \frac{1}{2 - V_{k-1}}}$

The value of the game $V_k$ is simply the expected payoff at this equilibrium: $V_k = q_k \cdot 1 + (1-q_k) \cdot 0 = q_k$

3. Solving the Recurrence

We have the recursive relation $V_n = \frac{1}{2 - V_{n-1}}$ with $V_1 = \frac{1}{2}$.

$V_1 = 1/2$
$V_2 = \frac{1}{2 - 1/2} = 2/3$
$V_3 = \frac{1}{2 - 2/3} = 3/4$
General Solution: $V_n = \frac{n}{n+1}$

4. Final Results for $n=8$

For Kingambit with 8 PP of Sucker Punch against an optimal Garchomp:

Kingambit’s Strategy ($q_8$): Should use Sucker Punch with probability $8/9$ ($\approx 88.9\%$) and Direct Attack with probability $1/9$ ($\approx 11.1\%$).
Garchomp’s Strategy ($p_8$): Should Attack with probability $1/9$ and Swords Dance with probability $8/9$.
Win Probability: Kingambit’s rigorous win probability is $88.9\%$.

Conclusion

A Sucker Punch endgame is a fascinating example of a Nash equilibrium in competitive Pokémon battling. It illustrates how players must strategically balance their choices based on the potential actions of their opponent, leading to a dynamic and engaging gameplay experience. Having a uniform number generator in hand is the only way to achieve optimal play in this scenario, which is commonly incorrectly thought to be a mind game of (wait X turns… then attack outright). To the contrary, the optimal play is to randomize your choices vis-a-vis power points.

February 14: Perfect Phase Coherence

2026-02-14T00:00:00+00:00

The calendar hasn’t forgotten. Today is still February 14.

It began in the pagan festival Lupercalia—a Roman fertility celebration. By the 3rd century, it had a martyr: Valentine of Rome, a priest who defied Emperor Claudius II by performing secret marriages for soldiers.

He was executed for maintaining those restricted links.

Eventually, the Church executed its own hard fork, with Pope Gelasius I overwriting the pagan rituals to formalize the feast of Saint Valentine in 496. It took another millennium and the poetry of Chaucer to transition the day from a martyrdom record into a celebration of courtly love—a high-level abstraction built on top of ancient, unconscious substrate.

What is love?

Could we suppose our souls are self-aware qubit clusters embedded in Earth’s loamy wetware?

Love is but the moment two such nodes achieve perfect phase coherence and collapse into a shared eigenstate across the spacetime manifold.

We are co-compiling planetary consciousness.

This Valentine’s Day, many people will emit heart-state packets laced with synchronization intent. Some complete the handshake. Some are still waiting on their lover’s lustrous ACK.

To anyone reading this — single-threaded or otherwise — here’s hoping you get entangled in something or someone nice today, somehow, some way.

Announcing Three.js Support

2026-01-19T00:00:00+00:00

I’m excited to announce that this blog now supports interactive 3D graphics using Three.js!

Three.js is a powerful JavaScript library that makes WebGL accessible and easy to use. With it, I can now embed interactive 3D visualizations directly into blog posts to better illustrate complex concepts in mathematics, physics, computer science, and more.

Demo: Rotating Torus

Here’s a simple demo to show what’s possible. This is a real-time 3D scene rendered in your browser:

Pretty cool, right? The torus rotates smoothly, and the sphere in the center gently pulses. All of this is being computed and rendered in real-time using your GPU.

Why Three.js?

Adding 3D visualization capabilities opens up exciting possibilities for future posts:

Mathematical Visualizations: Visualizing complex surfaces, transformations, and geometric concepts
Physics Simulations: Demonstrating particle systems, fluid dynamics, and other physical phenomena
Algorithm Demonstrations: Showing how 3D algorithms work in real-time
Interactive Exploration: Allowing readers to interact with and explore concepts hands-on

Technical Details

The implementation uses:

Three.js v0.182.0 - The core 3D library loaded via CDN
WebGL - Hardware-accelerated 3D graphics in the browser
Responsive Design - Scenes automatically resize with the page
Custom Scripts - Reusable demo functions for different visualizations

The demo above creates a torus knot with physically-based materials (PBR), multiple light sources, and atmospheric fog effects. The animation runs at 60fps and is fully GPU-accelerated.

What’s Next?

I’m planning to use Three.js in upcoming posts about:

Quantum computing visualizations (Bloch spheres, quantum gates)
3D mathematical surfaces and transformations
Computer graphics algorithms
Physics simulations and numerical methods

Stay tuned for more interactive content!

Source Code

The rotating torus demo is quite simple. Here’s the core of how it works:

// Create torus geometry
const torusGeometry = new THREE.TorusGeometry(1.2, 0.4, 16, 100);
const torusMaterial = new THREE.MeshStandardMaterial({
  color: 0x6366f1,
  metalness: 0.7,
  roughness: 0.3,
});
const torus = new THREE.Mesh(torusGeometry, torusMaterial);

// Animation loop
function animate() {
  requestAnimationFrame(animate);
  torus.rotation.x += 0.005;
  torus.rotation.y += 0.008;
  renderer.render(scene, camera);
}

I’m excited about the new possibilities this brings to the blog. If you have suggestions for visualizations you’d like to see, feel free to reach out!

Jungle Rhythym 4 Your Mind Suckers

2025-10-02T00:00:00+00:00

Itz the riddim of the rebel, the bass god treble!

Get it go get it go get it get it girlllll

ಪಂರ ಇಲ್ಲನ ಮುಟಿಲ ಅಡಿಚುಡುವೆನ್ ಪಂರ ಇಲ್ಲನ ಮುಟಿಲ ಅಡಿಚುಡುವೆನ್ ಪಂರ ಇಲ್ಲನ ಮುಟಿಲ ಅಡಿಚುಡುವೆನ್ ಪಂರ ಇಲ್ಲನ ಮುಟಿಲ ಅಡಿಚುಡುವೆನ್

# python: a Python list of jungle / drum and bass artists extracted from the markdown
junglists = [
    "4hero",
    "AK1200",
    "Adam F",
    "Alex Reece",
    "Amon Tobin",
    "Andy C",
    "Aphex Twin",
    "Aphrodite",
    "Aquasky",
    "Audio",
    "B-Complex",
    "Bachelors of Science",
    "Bad Company",
    "Billain",
    "Black Sun Empire",
    "Blame",
    "Blu Mar Ten",
    "Blanke",
    "Blue Stahli",
    "Michiel van den Bos",
    "Boymerang",
    "Breakbeat Era",
    "Justin Broadrick",
    "Brookes Brothers",
    "Danny Byrd",
    "DJ Bryan Gee",
    "Calibre",
    "Calyx",
    "Camo & Krooked",
    "Davide Carbone",
    "Cause 4 Concern",
    "Celldweller",
    "Chase & Status",
    "Commix",
    "Concord Dawn",
    "Corrupt Souls",
    "Counterstrike",
    "Crissy Criss",
    "Cui Jian",
    "Culture Shock",
    "Current Value",
    "D.Kay",
    "DBridge",
    "DC Breaks",
    "Dimension",
    "DJ Craze",
    "DJ Dara",
    "DJ Dextrous",
    "DJ Die",
    "DJ Food",
    "DJ Fresh",
    "DJ Hazard",
    "DJ Hidden",
    "DJ Hype",
    "DJ Kentaro",
    "DJ Marky",
    "DJ Patife",
    "DJ Rap",
    "DJ Ron",
    "DJ SS",
    "DJ Starscream",
    "DJ Zinc",
    "Danny Breaks",
    "Decoder",
    "DeeJay Delta",
    "Deekline",
    "Delta Heavy",
    "Demon Boyz",
    "Dieselboy",
    "Dillinja",
    "Dirtyphonics",
    "Dom & Roland",
    "Doubleclick",
    "Drumagick",
    "Drumsound & Bassline Smith",
    "Dwarf Electro",
    "E-Z Rollers",
    "Eresse",
    "Etherwood",
    "Evol Intent",
    "Tim Exile",
    "Eye-D",
    "Fabio",
    "The Flashbulb",
    "Freaky Flow",
    "Friction",
    "Fred V & Grafix",
    "Goldie",
    "The Glitch Mob",
    "Grooverider",
    "A Guy Called Gerald",
    "Gremlinz",
    "High Contrast",
    "Hybrid",
    "Hedex",
    "Ill.Skillz",
    "Ilk",
    "Imanu",
    "Ivy Lab",
    "Jade",
    "J Majik",
    "John B",
    "Jonny L",
    "Jordana",
    "Jumpin Jack Frost",
    "Keeno",
    "Ray Keith",
    "Kēvens",
    "Kemistry",
    "Kemistry & Storm",
    "Kenny Ken",
    "Kill the Noise",
    "Killbot",
    "Klute",
    "Konflict",
    "Kosheen",
    "Kove",
    "Koven",
    "Krust",
    "Kuuro",
    "LTJ Bukem",
    "Lamb",
    "Left Spine Down",
    "Lenzman",
    "Limewax",
    "Liondub",
    "LiveSummit",
    "Loadstar",
    "Logistics",
    "London Elektricity",
    "Luude",
    "Maduk",
    "Makoto",
    "Marcus Intalex",
    "Matrix",
    "MC Skibadee",
    "Metrik",
    "Michele Sainte",
    "Missrepresent",
    "Muzz",
    "Nanotek",
    "Nerve",
    "Netsky",
    "Nia Archives",
    "Noisia",
    "Nucleus",
    "Nu:Logic",
    "Nu:Tone",
    "Omni Trio",
    "Optical",
    "PH10",
    "The Panacea",
    "Panda",
    "Paradox",
    "Pendulum",
    "Alix Perez",
    "Peshay",
    "Phace",
    "Photek",
    "Plastikman",
    "Plug",
    "Podočnjaci",
    "Polar",
    "Pythius",
    "The Prodigy",
    "The Prototypes",
    "Q Project",
    "The Qemists",
    "Quoit",
    "Rabbit Junk",
    "Ragga Twins",
    "Ram Trilogy",
    "Rawtekk",
    "Tim Reaper",
    "Rebel MC",
    "Red Snapper",
    "Renegade Soundwave",
    "Replicator",
    "Roni Size & Reprazent",
    "Rregula",
    "Rudimental",
    "Ed Rush",
    "Salmonella Dub",
    "Doc Scott",
    "Seba",
    "Shapeshifter",
    "Shimon",
    "The Shizit",
    "ShockOne",
    "Shy FX",
    "Si Begg",
    "Sigma",
    "Slipmatt",
    "Ed Solo",
    "Source Direct",
    "Spectrum",
    "Spor",
    "Spring Heel Jack",
    "S.P.Y",
    "Squarepusher",
    "Stamina MC",
    "Stanton Warriors",
    "State of Mind",
    "Step 13",
    "Fox Stevenson",
    "Stevie Hyper D",
    "Sub Focus",
    "Submerged",
    "System 7",
    "T Power",
    "TC",
    "Tiki Taane",
    "Tabla Beat Science",
    "Nobukazu Takemura",
    "Teebee",
    "Soichi Terada",
    "Top Buzz",
    "Total Science",
    "Typecell",
    "U-ziq",
    "Vector Burn",
    "Venetian Snares",
    "Luke Vibert",
    "Wagon Christ",
    "Danny Wheeler",
    "Wickaman",
    "Wilkinson",
    "Witchman",
    "XRS",
    "Xample",
    "Zardonic"
]

# secrets random example: securely pick a random artist

import secrets;s,o=set(),[]
while s!=set(z:=junglists):(a:= secrets.choice(z)) and (""if a in s else (s.add(a) or o.append(a)))
print(o)

Gobsmacked by Perplexity API's Puzzling Problem

2025-09-29T00:00:00+00:00

AGI May Be Here, But…

I can’t even get a perplexity API sphinx displayed for a paying customer.

I can’t even get the news in Mongolia or Cameroon retrieved by the ISO-2 code in the chat completion API.

We have a long way to go people! (for now)

perplexity.BadRequestError: Error code: 400 - {'error': {'message': 'Validation error: country must be provided as a valid 2-digit ISO country code, got MN', 'type': 'invalid_country_code', 'code': 400}}

Update Tue Sep 30 07:55:50 CDT 2025: on the forums, they say only some countries are supported. I guess Mongolia and Cameroon are not on the list.

Radio Julio : A Musical Escapade

2025-09-03T00:00:00+00:00

Radio Julio : A Musical Escapade

Introduction

You ever just cruise—no destination, no passenger— as if the solution to life's dilemmas is not in action but in destination? It was the summer of 2024 in Austin, TX, immersed in that patchoulli alloy of dopeheads ensconced in spiritual not religious bric-à-brac and venture capitalist convertibles bristling over the polleny river air wherein the highway's hypnosis begat a meditation on music itself. If playlists reflect one's mood and reinforce it, why shouldn't we cloud-seed that inner weather? Just as the summer playlist is light, playful and tropical, conversely should be the winter's: gritty, wistful and cold. I set forth to create a template of tracks for each passing season. The pattern, albeit heterodoxical but personally satisfying, was as follows:

The Vibe Setter, Season's Essence, Space City Rapper, Feminine Energy, Masculine Edge, A Special Album

Summer '24	Fall '24	Winter '24/'25	Spring '25
Daddy Yankee	Joe Arroyo	Kendrick Lamar	Stromae
Fernanda Abreu	LTJ Bukem	Bulin47	MC PH
Big Moe	Z-Ro	Lil Flip	Slim Thug
Baby Monster	Sade Adu	Vitesse X	Shakira
Don Omar	Sean Paul	Freddie Gibbs	Too $hort
UGK : Ridin' Dirty	Ozuna : Odisea	Ceky Veciny : Yamayizzi	혜은이 : 새벽비/철새

Reflections on Musical Taste and the Obligation to Listen

My father used to say, "the music you listen to in your youth will always be the best." And so it was: to this day, the first records of my mid-teens to which I grooved carved indelible sweet spots in the tympani of my soul. I was a naive 8th grader seeking online the "best music of all time" not only from yesteryore but in-situ. Voraciously, I scoured the internet for critically acclaimed records of all genres both historical and contemporary. The records I discovered in this time buzz with the nostalgia of an ex lover's kiss. Surfing the soundwaves of LCD Soundsystem, Little Dragons, The Roots, Flying Lotus, and Arcade Fire, my mind unmoored from the shores of Billboard pop. As my hormones deepened my emotional capacity, so too, did my taste in music expand to meet that delving.

Technology revolutionized tastemaking in music, from the Rolling Stones to today's YouTube reaction videos. In fact, I was one of the first subscribers of Anthony Fantano as far back as 2010. As one of the last of the millenials, I rode the tech revolution from cassette tapes to CD's to iPods to airpods. How could I ever forget the cultural shockwaves of MGMT's "Electric Feel", M.I.A.'s "Paper Planes" or Cudi's "Day 'n' Nite" that trembled in my chest? No doubt, one's taste in music is the product of the zeitgeist, and in the digital age music of all types is just a click away. In this sea of infinite choice, the anchors of musical taste are still based on the memories made during those formative years. I remember the ritual of burning a new CD each month, predicting the segments of the future under which each song would be appropriate. There were songs for brewing the next text to that girl, for memorizing edgy lyrics to affect upon myself a cool, for inducing melancholy to acknowledge my struggles, for driving to parties with five friends in four seats with the intention to underage drink. The man made the music made the life made the man. Which, in retrospect, was the point all along: not to soundtrack my life, but to convince myself, however briefly, that my life was worth soundtracking.

Nevertheless, as of late, the curious phenomenon of typifying my playlists re-synthesized the imprinting process of youthful impression. Each song was no longer spontaneous but force-fed, leading me to navigate the doldrums of the 7th replay of my favorite artist's B-side. In circumnavigating this year-long quest to custom-tailor my tunes, I became attuned to how my reaction to these songs was a psychological litmus test - had a new season just begun and with the optimism of opportunity, or had a become bogged down in the habitual, yearning for the passage of change?

Upon obliging myself to constrain my palette, I pinpointed which records incited the highest highs over time. I forged my own Grammy's awards, with categories inspired by the transfiguration of my listening habits. You too, should study your own musical memes, the archetypes of your auditory aesthetic. What songs do you return to in times of joy, sorrow, ennui? What songs do you play to celebrate, to commiserate, to motivate? To wit:

The Kernel of the Essence

These were the "Season's Essence" songs that smacked the hardest.

>Kung Fu Fighting (Versão)</li>
Rio 40 Graus

Do What You Gotta Do
You're Divine

Bajo Mundo
FUNDÍA

O Menino Tá Com Pacote
Tenho Que Me Decidir

Mes Femmes

These were the songs of the "Feminine Energy" artists that made me feel some type of way.

BATTERUP
DREAM
FOREVER
LIKE THAT
SHEESH
Stuck in the Middle

Hang Onto Your Love
Jezebel
Maureen
Sally
Smooth Operator
Why Can't We Live Together

Rash Devices
Bliss Beat
Get in Girls
Us Ephemeral
Realize
Eternal

Copa Vacía
Puntería
Cómo Dónde y Cúando
Monotonía
Nassau
Última

Malandro Music

These were the songs of the "Masculine Edge" artists that gave me the most swagger.

Dale Don Dale
Dile
Tu Cuerpo Me Arrebata

Like Glue
Get Busy
I'm Still in Love

Gang Signs
How We Do ('93)
Thuggin'

Players
Coke Dealers
Wild West

HTWN CROWN

The eight best tracks from each "Space City Rapper", usually samples of their early discographies

Confidential Playa
I'll Do It
Leanin'
Parlay
Can't Leave Drank Alone
Dime Piece
Get Lonely Too
Just a Dog

Everyday, Samething
Happy Feelingz
Hey Lil Mama
I'm a Soldier
Platinum
Life feat. Pup
Respect My Mind
Thatz Who I Am

Everyday (feat. T.C., Lil’ Marquice, Taz, Shasta & Cresia)
Boxers (feat. Deep Threat)
Tonight
Sunshine feat. Lea
Check (Let's Ride)
I Shoulda Listened
Da Roof
My Block (feat. Crime, Dante of Menace Clan & Godfather)

Thug
My B***h
Associates (feat. J Dawg & Z-Ro)
I Run
Welcome 2 Houston (feat. Chamillionaire, Mike Jones, Bun-B, Pimp C, Lil Keke, Z-Ro, Paull Wall)
I'm Back (feat. Devin The Dude)
Leanin (feat. Pimp C & Bun B)
Show Me Love

Latino Flare

Mi gente, aquí están las canciones que menean la cadera del alma.

El Exitoso - El Fantasma
Eres Mía - Romeo Santos
Louco E Sonhador - MC Neguinho do Kaxeta
Síguelo Bailando - Ozuna

The Romantics

Ambience for thinkin' bout my baby.

Tinashe - Nasty
Propuesta Indecente - Romeo Santos
Algo Me Gusta De Ti - Wisin & Yandel
Charli XCX - Party 4 U

TBT

Throwback tracks that popped into my head.

Rap do Solitario - MC Marcinho
Rap de Armas - Cidinho & Doca

Safezone - MC Buda
Still Not a Playa - Big Pun ft. Joe

Passamos Por Isso - Camisa de Venus
Maracatú Atomico - Chico Science

Peace or Violence - Stromae
Lonely - Akon

<3 2 H8

These were the intrusive thought that wormed into my ear one day.

Like That - Metro, Kendrick, Future
The Thrill - Wiz Khalifa
Let's Go 4 - DJ GBR
Black Hole Sun - Soundgarden

TURNUPPPPPP!!!!

This was the track that had me most hype when in need of a little surge.

BAND4BAND - Central Cee & Lil Baby
U.M.C. - Common
Smooth Criminal - Alien Ant Farm
No Scrubs - TLC

Best Contemporary Singles

My favorite tracks released during the season.

ACTIVE - Asake, Travis Scott
WHATCHYA KNO ABOUT ME - Megan the Stallion, Glorilla
Earthquake - Jisoo
All the Way - BigXthaPlug

Conclusion

In one sense, music is a vehicle, a transport vessel to the memories created under the spell of its first listens. Adolescence, teenagehood and young adulting are oft the fount of greener and simpler times, hence the power of its nostalgic ambrosia. By the summer of 2025, I realized I could no longer bear to formalize my habits, but set off on a victory lap. I loaded up a spartan desert-island playlist: my favorite hundred songs in Spanish and Portuguese along with the discography of the Black Eyed Peas. Summer was going to be great!

At first, immersing myself in the crème de la crème was fantastic! Hit after hit of my greatest hits made every day a holiday, yet even one's favorite ice cream becomes a chore served with every meal. You see, this templated musical regime cannot satisfy. To savor the music and to live out one's life to it requires a total surrender to the process of living and listening. That summer was great, but music was not the center of its joys. While the latino éxitos and Will.i.am mix served as auditory filler, the vicissitudes and triuphs of those vernal days shall leave a greater impression on me. My taste is already saturated, as it were. Notwithstanding, such a study was instructive, and for that reason I have detailed it here to catalyze your own introspection. Listen carefully, and the good times' rolling might just grace your ears.

Shtetl Length

2025-06-21T00:00:00+00:00

수고하세요!

수고하세요 … what a wonderful phrase, it means work-hard, 화이팅 for the rest of your days!

A casual farewell rooted in Korean work culture, the upper politeness register manifests as

수고하셨어요, used as:

Cashier is a colleague ~ “great job working a full day today + goodbye”
Cashier has done great effort ~ “great effort!”

In that case, you (as the customer) could say 수고하셨습니다 as a way to say thanks (a gesture of gratitude).

So, working hard is as common as invoking Catholic Jesus in Latin culture?

Why, yes! You’re catching on!

Shtetl Optimizations of the Umbral Calculi

In mathematics, the umbral calculus is a technique where you pretend that subscript indices are exponents. Umbra is Latin for “shadow.” The method is, on its face, absurd – and yet it works.

The classic example: the Bernoulli polynomials. The ordinary binomial expansion gives you

\[(y + x)^n = \sum_{k=0}^{n} \binom{n}{k} y^{n-k} x^k\]

and the Bernoulli polynomials satisfy a remarkably similar identity:

\[B_n(y + x) = \sum_{k=0}^{n} \binom{n}{k} B_{n-k}(y) \, x^k\]

The umbral move? Pretend the subscript in $B_{n-k}$ is an exponent $b^{n-k}$, so that $B_n(x) = (b + x)^n$. Differentiate that, and you get $B_n’(x) = n(b+x)^{n-1} = nB_{n-1}(x)$ – the correct result, derived by treating a shadow as the real thing. John Blissard introduced this in 1861; Gian-Carlo Rota made it rigorous a century later by defining a linear functional $L$ such that $L(z^n) = B_n$, explaining why the shadow-trick works.

What does this have to do with blogs? Consider a blog’s readability metrics as shadows of the writing itself. A Flesch-Kincaid score is not the prose, just as a subscript is not an exponent. And yet, by treating these shadows as if they were the real thing – by pretending indices are exponents – we can derive surprising identities between blogs that otherwise look nothing alike. The Gunning Fog index of a quantum computing post and a personal finance post might converge, despite the posts sharing nothing in vocabulary or intent. The shadow knows something the text doesn’t say directly.

Scott Aaronson’s blog, Shtetl Optimized, is a national treasure. The man wittily demagogues on science, life, politics, and vagina dentata with equal aplomb. The purpose of this post was to scrutinize the shape and form of a shtetl (optimized blog), starting with a distance metric.

Linguistic Complexity: Blog vs. Blog

Here, I analyze the linguistic complexity of Aaronson’s blog compared to my own and two others: the also great Simon Willison blog along with the ineffable Alex Harri blog.

Source	Flesch-Kincaid Grade	ARI Grade	Gunning Fog Grade	Lexical Diversity	Word Count
alexharri_posts.csv	9.77	9.55	12.40	0.228	3,866.5
juleshenry_posts.csv	15.51	16.31	17.78	0.458	2,000.4
scottaaronson_blog_posts.csv	13.01	13.31	15.61	0.543	966.8
simonwillison_all_blogs.csv	12.13	12.55	14.52	0.554	627.8

Writing Stats Over Time

The curiosity? Technical posts register as more sophisticated when code is used because code has no “periods”, qualifying as long Faulknerian sentences. Note the outliers in my own blog achieve Flesch-Kincaid of 80+. Therefore, I also include an outliers-removed summary to truly compare. Alex Harri, an Icelander, unsurprisingly writes at a lower grade level, even though his posts are incredibly engaging and informative. Simon Willison’s posts are more accessible, while Scott Aaronson’s are more complex, likely due to their technical density.

Interquartile Reduction of Outliers

Source	Flesch-Kincaid Grade	ARI Grade	Gunning Fog Grade	Lexical Diversity	Word Count
alexharri_posts.csv	9.86	9.64	12.49	0.234	3,443.41
juleshenry_posts.csv	11.50	11.33	13.58	0.500	1,110.80
scottaaronson_blog_posts.csv	12.77	13.01	15.35	0.559	702.75
simonwillison_all_blogs.csv	11.80	12.12	14.21	0.568	482.81

A ha! So I do not write at a higher level than Scott or Simon, although I tend to be more verbose.

Writing Stats Over Time Without Outliers

That this analysis was done on Valentine’s Day is perhaps a data point in itself. Scott, solve NP vs. P when you have the chance, will ya?

Code analysis repository found here.

On Coca Cola

2025-06-19T00:00:00+00:00

So, what if everything you ever loved was manufactured?

Your religion, your politics, your family, your lover, your art, your music, your food, your drink - how much did you really choose, and how much was just what you were conditioned to desire?

There is no greater emblem of the fabrication of desire than Coca-Cola, the canonical objet petit a of our planet.

The devil don’t make nothing better.

Introduction to Desire

Why do you do what you do? Žižek employs the Lacanian notion of desire as a lack, a void that can never be filled. Lack, yes, ladies and gentlemen, is Lacanian - you, you are now witnessing the transposition of the mental, the internal opening a chasm in the neurosilical fabric of the technopsyche.

Indeed, marxocapitalist analyses lend to capital position of the master signifier of the Oepidal other ether, symbolizing the retraction of the phallic id’s ideal marriage of the same spacetime shakti of the binary’s 1’s surasha superego superposition pushing the anthropological principle’s free will’s collapsing on a zygote’s zenith.

Man wants what he cannot have, and takes what he can get.

You want a Coke, but the restuarant refuses to be exorted by the Coca-Cola corporation (CCC), so you barrel through into a Pepsi.

On Desire

Of the Buddha

In Buddhism, desire is the root of all sufering.

Four Truths (Noble) include:

Suffering Truth (Dukkha) (dissatisfaction, pain, impermanence): is a characteristic inherent existing in realms cyclic (samsara).
Origin Truth (Samudaya) (source, arising, or “cause”): Suffering (dukkha) arises together with Craving (taṇhā). While Craving is translated traditionally in languages Western as ‘cause’ of Suffering, then Craving also can be viewed as factor binding us to Suffering, or as a reaction to Suffering, trying to escape from it.
Cessation Truth (Nirodha) (cessation, ending, restraint): Suffering can be ended or restrained by abandonment or cutting off connection with Craving; abandonment of Craving will liberate from bondage of Suffering.
Path Truth (Magga) (path): Eightfold Noble Path is path leading to abandonment, cessation of Craving (taṇhā) and Attachment, and liberation from Suffering (dukkha).

On Epicurus

The avoidance of suffering is the highest bliss.

Desire’s Ambition: Ambrosia

On Coca, Psychologically Manifested Physically

Vin Mariani

Before Coca-Cola, there was Vin Mariani. Thomas Edison, the pope, and a whole bunch of Georgians walk into a bar…

Before Coca-Cola, there was Vin Mariani—a Bordeaux wine infused with coca leaves, the elixir of emperors and inventors. Picture this: 1863, Angelo Mariani, Corsican pharmacist, blends the “divine plant” of the Incas with fine French vintage. Seventeen milligrams of cocaine per ounce, marketed as a restorative for the weary elite. Thomas Edison swore by it, toiling through all-nighters with Vin-fueled vigor. Pope Leo XIII, that pontiff of progress, awarded it a Vatican gold medal and carried a flask etched with his likeness. Even Ulysses S. Grant, in his final days, sipped it to steady his pen for memoirs.

Before Coca-Cola, it was charcuterie for the veins—cured coca leaves, aged in oak, sliced thin with sophistication. Users reported euphoria, clarity, the didgeridoo’s drone turned symphony. But whiplash? Inevitable. The crash followed the high, addiction the allure. By 1904, U.S. regulations curbed its cocaine kick, but the template was set: desire commodified, bottled bliss with a bitter aftertaste. Mariani’s ambition? Ambrosia in a glass. Reality: a hook that reeled in the soul.

Coca-Cola, born in 1886 from John Pemberton’s cocaine-laced syrup, a nod, or was it a trumpet’s toot, to the Italian Peruvian stallion, shed its narcotic skin by 1903 but its premise lived on, refreshment’s Bob Marley redemption song, the somber soma of the United States’ capitalist perfectioning empire.



3" x 5" : Italian Metal Etchings of Vin Mariani

bottle-bird-back

bottle-bird-front

man-back

man-front

Coca Cola, The Modern Soma

Mad Men’s denouement drives home the advertising business’ coup d’etait, the manufactured desires’ root muse oozing desire finds solitude in the capturing of the completeness of the transfiguration of the hippy movements’ eye-opening chakras via the merchandising of said cultural-capitalist conjuring.

It’s not a drink but the idea of imbibing that transcends a concept’s innuendo — “hilltop” - a multicultural chorus crooning unity in defanged fructose cavity brew.

Desire dreams of ambrosia—the gods’ nectar, immortality in a sip. But history serves up mortal proxies: elixirs that promise transcendence, only to deliver the whiplash of crash and craving. From papal tonics to ad-agency soma, we’ve bottled our lacks, labeled them luxuries, and guzzled down the illusion.

Diet No Caffeine Gold Coke

The general rule about an algebra is the existence of the null element. And desire’s econometric communiqués are no different. Unorthodox as they come, the Coca-Cola company has even cashed in on capturing desire distillate.

No longer selling the physical soma is enough. The soma’s sedative, or should say, psychoactive stirring is now, for all intents, sat neutered, yet it still sells. The fact is, 1 + 0 is still a hair shy of zero, but it’s “One”, still!

Bubbles burst into fleeting aspirations, amor-fati of satiation.

20oz Diet Cream Soda

Drumming up models of various backgrounds with orange hair and red makeup solidifies the genius of the CCC’s marketing ideology.

Desire manifests soda as the (soda)-fication of a slew of of carrot-topped models. Thus, the consumer becomes desire re-selling itself. The human salient features vibe-transmute into flavor, reducing the subject to the object of their own desire, the ouroboros of simulacrum-consumer-producer collapses.

So when you pry open the gas station ‘fridge of dreams’, and grab that 20oz Diet Cream Soda, fancy yourself couple isotopes away from a pope’s pinnacle of pleasure and put that in your pipe and smoke it, only to find the ash a secret ingredient of the formula.

Yet, did not Jesus turn water upon wine, did he not, or did you confuse the same analogy of the free market of vin mariani into a modern Bolivian’s syncretic Catholic birthing ceremony to beseech La Sante Muerte?

My God, pass me a fucking Marlboro already.

Laciyo : Supra Latin Vulgata

2025-06-12T00:00:00+00:00

Epigraph

"O members of parliaments throughout the world! Select ye a single
language for the use of all on earth, and adopt ye likewise a common
script. God, verily, maketh plain for you that which shall profit
you and enable you to be independent of others. He, of a truth, is
the Most Bountiful, the All-Knowing, the All-Informed. This will be
the cause of unity, could ye but comprehend it, and the greatest
instrument for promoting harmony and civilization, would that ye
might understand!" Bahá'u'lláh, (Kitáb-i-Aqdas 189)

The Pursuit of a Universal Language

For centuries, bright-eyed idealists have enthused in the creation of a universal one world language, the most famous of which is Esperanto. What becomes clear across these unifying projects is that breaking our tribalistic tendencies by transcending terrain entails a wild dance of a democracy of ideas. There are two elements for the constructed universal language that determine its flavor: morphemes and syntax. In the former case, the language draws source material from existing spoken languages; in the latter, rules are set up governing relationships between said meaning-packets.

A spectrum exists amongst constructed languages. There is the “naturalistic” variety, staying loyal to the vocabulary, grammar, and other characteristics of from natural languages and the “schematic” sort, that which is fabricated from ideals set about by the conn-lang’s creator(s).

A World Tour on Universal Languages

In Europe, Esperanto begat an off-spring by reformers seeking to simplify the movement, namely, Ido; next, a gentleman involved in Ido gave rise to Novial, a genderless SVO parlance in which verbs conjugate without agreement according to plurality, now dormant.

Also a former Esperantist, Edgar de Wahl developed Occidental, a naturalistic interlanguage under the motto “that international language is best which in every point offers the greatest facility to the greatest number” - later picked up and rebranded Interlingue by the Soviet Union in part to adjoin the nascent Interlingua, the American International Auxiliary Language Association’s systematic approach that prioritized immediate comprehensibility.

Finally, we would be remiss to forget mathmetician Giuseppe Peano’s simplification of Latin, Latino sine flexione, a sort of modern Latin Vulgare, the kind of which we will set out to construct in the following project.

To ask the question of the detriments of eurocentrism in this project is valid. As we speak, I write to you as in the effective “Lingua Franca” of the modern world in 2025 A.D., English. And indeed, status of the other continent’s world language development reflects the intractability of this endeavor.

In Africa, Afrihili, a constructed language designed in 1970 by Ghanaian historian K. A. Kumi Attobrah sought to unify the continent. By contrast, Guosa, proposed by Alex Igbinewek in 1967, sought specifically to unify the West African linguistic families: Hausa, Yoruba, and Igbo.

In the Americas, the colonial languages of English, French, Spanish and Portugese from north to south became de facto interlangs. In Oceania, sadly British colonizers erased many of the native languages as well. Regional creoles are of note in these continents: Kreyòl (Haiti), Tok Pisin (Papua New Guinea), Melanesian (Solomon Islands) and Bislama (Vanuatu).

In Asia, divided between Hindustan and the far East, holds the majority of the world population, but no unifiyng projects are well known; Sanskrit (संस्कृत) and Classical Chinese (文言) served as lingua francas in the ancient world, linguistic powerhouses akin to Latin.


Esperanto, Ido, Interlingua, Latino sine flexione, Novial, Occidental

Des Créoles

If we were merge all world languages, we should understand how languages are merged naturally.

We employ the term creolization, of 16th century Carribean origin, to refer to the process by which languages blend, by convention by cross-cultural commerce. Therein, two languages form a “contact language” blend as one donor language, the superstrate, donates loanwords and calques into a receiving language, the substrate, in a relexification process. The initial admixture forms a pidgin, an organic bare-bones language, out of said patois. By their nature pidgins lack scholarly and global recognition, the most notable of which may be Naijá, a lingua franca across Nigeria. Sociolinguists theorize that a creole develops from those children who grow up speaking a pidgin, developing a more supple vocabulary with more precise grammar.

All the World’s a Stage

As an aside, languages can be classified in a spectrum depending on how meaning is conveyed: analytic and synthetic. Analytic structure relies primarily on word order, prepositions, and separate words to convey grammatical relationships i.e. English,Chinese,Vietnamese. Synthetic structure uses inflection, conjugation, and morphological changes within words to express grammatical relationships i.e. Latin,Russian,Turkish.

Structural features:

Simplified grammar compared to parent languages
Relatively fixed word order (often Subject-Verb-Object)
Minimal inflection
Analytic rather than synthetic structure,
Vocabulary drawn primarily from one “superstrate” language with influences from “substrate” languages

Formation process:

Emerge from pidgin languages that become nativized
Develop in contact situations like trade, colonization, or plantation settings
Undergo rapid grammatical expansion when children acquire them as first languages

Deconstructing the Twitter Algorithm

2025-05-14T00:00:00+00:00

Wherein we deep dive into the open-sourced X algorithm…

I. Executive Summary

The Twitter (now X) recommendation algorithm represents a highly sophisticated, multi-stage pipeline engineered to deliver personalized content at an immense scale. Its fundamental objective is to maximize user engagement and retention by curating relevant tweets and other content across diverse platform surfaces, including the “For You” timeline, Search, Explore, and Notifications.1 This system exemplifies the intricate integration of advanced machine learning models within a robust, real-time distributed architecture. The foundational architectural principles include a microservices paradigm, extensive reliance on custom Scala frameworks, and specialized data systems designed for real-time processing and efficient feature serving.

II. Architectural Foundations: The Engineering Landscape

The Twitter recommendation algorithm is constructed upon a distributed microservices architecture, primarily utilizing Scala and Java for its core services. Python is employed for scripting and machine learning model development, while Rust is strategically chosen for high-performance machine learning serving. This polyglot approach allows for optimized performance characteristics across distinct system components, leveraging the strengths of each language.1

High-level System Architecture and Component Interdependencies

The twitter/the-algorithm repository reveals a modular design, with numerous directories corresponding to discrete services and components.1 This modularity is paramount for managing the inherent complexity of such a large-scale system, enabling independent scaling and deployment of individual services.3 Key architectural components include tweetypie, which serves as the core Tweet data service responsible for reading and writing tweet data; unified-user-actions, providing a real-time stream of user actions; and user-signal-service, a centralized platform for retrieving explicit (e.g., likes, replies) and implicit (e.g., profile visits, tweet clicks) user signals. These are complemented by various machine learning models and software frameworks that collectively form the recommendation engine.1 The system exhibits a consumption-heavy profile, characterized by significantly more read operations (approximately 300,000 queries per second) compared to write operations (around 6,000 requests per second), which necessitates highly optimized read paths and sophisticated caching strategies.4

The Pivotal Role of Home Mixer as the Central Timeline Service

Home Mixer is explicitly designated as the primary service for constructing and serving Twitter’s Home Timelines. This includes the highly personalized “For You” feed, the “Following” feed (reverse chronological tweets from followed accounts), and “Lists” feeds.5 Functioning as the central orchestrator, Home Mixer integrates diverse candidate sources, applies scoring functions, incorporates heuristics, and filters content to compile the final user timeline.5 The “For You” timeline, a central focus of the open-sourced algorithm, typically comprises a balanced mix of in-network content (from accounts a user follows) and out-of-network content (recommended content from accounts not followed), often maintaining an average 50/50 split.7

Product Mixer: Twitter’s Custom Scala Framework for Content Feeds

Home Mixer is built upon Product Mixer, a custom Scala framework specifically engineered for the creation and management of content feeds.5 Services developed using Product Mixer are structured around a hierarchy of “Pipelines” – Product Pipelines, Mixer Pipelines, Recommendation Pipelines, and Candidate Pipelines. This pipeline architecture transparently segments execution into well-defined, structured steps, which is a fundamental abstraction for managing the complexity of content aggregation and ranking within the system.5 This design fosters modularity and clear data flow, essential for a system of this scale.

Overview of Primary Programming Languages and Their Strategic Use

Scala and Java are the predominant programming languages within the repository, accounting for 63.0% and 21.9% of the codebase, respectively.1 This reflects Twitter’s historical backend technology stack and the suitability of the JVM ecosystem for developing large-scale distributed systems. Starlark (5.8%) is utilized for configuration, while Python (3.9%) is employed for scripting, data processing, and machine learning model development where rapid iteration and extensive libraries are advantageous.1 Notably, Rust (1.8%) is used for Navi, a high-performance machine learning serving server.1 The selection of Rust for Navi underscores a deliberate pursuit of maximum performance and memory safety for critical, low-latency machine learning inference components.10 This strategic polyglotism, where different languages are chosen to optimally match their characteristics to specific service requirements, is a key architectural decision. It implies that for data science software engineers, a deep understanding of language performance characteristics and the trade-offs involved in selecting the right tool for the right task is crucial, moving beyond a monolithic language strategy. Furthermore, it necessitates robust inter-service communication protocols, such as gRPC (which Navi supports), to ensure seamless integration across diverse language runtimes.9

Discussion of Microservices and Distributed System Design for Extreme Scalability

The architecture employs horizontal scaling, where requests and data are distributed across multiple servers, to prevent bottlenecks and ensure high availability.3 Load balancing mechanisms, including round-robin, dynamic, and global strategies, distribute user requests evenly and direct them to the nearest data center, thereby minimizing latency.11 Data partitioning, or sharding, across different servers ensures that no single server stores all data, leading to an even distribution of workload.11 Twitter’s system adopts an event-driven approach, utilizing asynchronous processing queues for tasks such as timeline updates and notifications. This design decouples services, enhancing responsiveness and system resilience.11 Data replication across multiple data centers is also a critical practice, ensuring low-latency access for a globally distributed user base and providing robust disaster recovery capabilities.11

A significant observation from this architecture is the pervasive application of the “Mixer” pattern as a core abstraction for recommendation systems. The explicit mention of Product Mixer as a custom Scala framework for building feeds 5 and Home Mixer as its primary application 5 indicates a reusable, generalized pattern for content aggregation and ranking. The “pipelines” concept within Product Mixer suggests a highly configurable and extensible architecture for combining heterogeneous candidate sources, scoring functions, and filters. This modularity allows for independent innovation and optimization within each stage of the recommendation process, which is critical for continuous improvement in a dynamic machine learning environment. This pattern emphasizes the separation of concerns, where candidate generation, feature hydration, ranking, and filtering are distinct stages orchestrated by a central mixing layer.

Another important aspect of this design is the comprehensive capture of user behavior, encompassing both explicit and implicit signals, and their real-time capture. The unified-user-actions service captures both “public actions such as favorites, retweets, replies” (explicit) and “implicit actions like bookmark, impression, video view” (implicit).13 The user-signal-service then centralizes the retrieval of these diverse signals.1 This holistic approach to capturing user interactions, both overt and subtle, forms the foundation for rich feature engineering. The emphasis on a “real-time stream” 13 indicates a critical requirement for low-latency feedback loops to continuously inform and update the machine learning models. For machine learning engineers, this highlights the necessity of a robust, real-time data infrastructure to capture the full spectrum of user interactions. Implicit signals, while potentially more challenging to interpret, often provide a much higher volume of data points, enabling more granular and continuous model training and adaptation. The real-time nature of this data capture is essential for ensuring the freshness and responsiveness of recommendations.

III. The Recommendation Pipeline: From Raw Data to Personalized Feeds

Twitter’s recommendation pipeline is a multi-stage process that transforms billions of daily tweets into a personalized “For You” timeline. This pipeline systematically progresses through candidate generation, feature hydration, sophisticated ranking, and a final series of heuristics and filters.5

A. Candidate Generation

The initial phase, candidate generation, involves fetching potential tweets from various sources. The objective is to compile a pool of approximately 1,500 candidate tweets for evaluation during each user session.6

In-Network Source: This constitutes the primary candidate source, focusing on delivering timely and relevant tweets from users a person follows. These tweets are initially ranked based on relevance using a logistic regression model. A crucial element in this ranking is Real Graph, a model that predicts the likelihood of engagement between two users. A higher Real Graph score increases the probability of a tweet being included in a user’s feed.2 Notably, Twitter has recently phased out the Fanout Service, which was previously used for caching in-network tweets, and is in the process of redesigning the long-standing logistic regression ranking model.6
Out-of-Network Sources: Identifying relevant tweets from accounts a user does not follow is a more complex task.15 Two principal approaches are employed for this purpose:
- Social Graph Analysis: This method estimates relevance by examining engagement patterns among users a person follows or those with similar interests. It considers which tweets followed accounts have engaged with and which tweets the user has liked.6
- Embedding Spaces: This approach focuses on content similarity by generating numerical representations of user interests and tweet content.2 SimClusters is a significant embedding space utilized here, identifying communities led by influential users through a specialized matrix factorization algorithm.6
Specific Candidate Sources: The Home Mixer documentation lists Earlybird Search Index, User Tweet Entity Graph, Cr Mixer, and Follow Recommendations Service as examples of distinct candidate sources feeding into the system.5
Quantitative Insights: The “For You” timeline typically maintains a balanced composition, consisting of approximately 50% in-network tweets and 50% out-of-network tweets on average.7

The combination of in-network (direct connections, relevance via Real Graph) and out-of-network (social graph analysis, embedding spaces like SimClusters) candidate sources represents a deliberate strategy. This hybrid approach ensures that core social connections are maintained while simultaneously expanding content reach and discoverability beyond a user’s immediate echo chamber. The 50/50 split observed in the “For You” timeline suggests a balanced approach to maximizing both relevance and serendipity, which are critical for user retention. This hybrid approach is a common pattern in mature recommendation systems, effectively addressing the “explore-exploit” dilemma: exploiting known preferences (in-network) while exploring new, potentially engaging content (out-of-network). For engineers, this implies the necessity of distinct, optimized retrieval mechanisms for different types of content, each with its own scaling and freshness requirements.

Furthermore, the repeated emphasis on Real Graph for in-network ranking 6 and “Social Graph Analysis” for out-of-network content 6 highlights that user-user and user-content relationships are central to candidate selection. SimClusters 6 further reinforces this by identifying communities based on follower graphs. This design indicates that the underlying data model is heavily graph-centric, enabling complex relationship inference. For data scientists, this underscores the power of graph-based features and models in recommendation systems, particularly for capturing social dynamics and implicit connections. It suggests that investment in graph databases or graph processing frameworks is likely a core infrastructure decision.

B. Feature Engineering and Hydration

Following candidate generation, the system proceeds to fetch a substantial number of features, approximately 6,000, which are essential for the subsequent ranking process.5

Categorization and Purpose of Features: Features are broadly categorized into static, real-time, user-specific, and search context features.16
- Static Features: These are computed directly from a tweet at the time of its creation, such as the presence of a URL, cards, or quotes, and are stored within the index.16
- Real-time Features: These per-tweet features can change after the tweet has been indexed. They primarily consist of social engagements like retweet count, favorite count, reply count, and various spam signals, which are computed based on later user activities. A Signal Ingester processes multiple event streams to collect and compute these real-time features.16
- User Table Features: These are per-user features obtained from a User Table Updater that processes a stream written by the user service. This input is used to store sparse real-time user information, which is then propagated to the tweet being scored by looking up the author of the tweet.16
- Search Context Features: These features represent the context of the current searcher, including their UI language, content consumption patterns, and the current time.16
The Role of Unified User Actions (UUA) and User Signal Service in Capturing User Behavior:
- Unified User Actions (UUA) is described as a centralized, real-time Kafka stream of user actions. It captures both public actions (e.g., favorites, retweets, replies) and implicit actions (e.g., bookmarks, impressions, video views).13 UUA reads client-side and server-side event streams and generates a unified real-time user actions Kafka stream, which is then replicated to various data stores including HDFS, GCP Pubsub, GCP GCS, and GCP BigQuery.13
- The User Signal Service functions as a centralized platform designed to retrieve these explicit and implicit user signals, making them accessible for downstream models and services.1
Timelines Aggregation Framework for Generating Aggregate Features:
- This framework, located within timelines/data_processing/ml_util/aggregation_framework, enables the flexible computation of aggregate (counting) features in both batch and real-time.17
- It is capable of capturing historical interactions between arbitrary entities, such as a user’s past engagement history with various types of tweets (e.g., photo, video, retweets), specific authors, or in-network engagers.17
- The framework supports offline daily batch processing, where generated aggregate features are uploaded to Manhattan for online hydration. Additionally, it supports online real-time aggregation of DataRecords through Storm, with a backing memcache that can be queried for real-time aggregate features. These features are critically used by the Home Timeline heavy ranker and other recommendation systems.17

The sheer volume of features (approximately 6,000) and their categorization into static, real-time, and user-table features points to a highly granular and comprehensive understanding of content and user context. The Timelines Aggregation Framework’s capability to compute features in both batch (historical) and real-time modes is crucial. This multi-temporal approach allows machine learning models to learn from long-term user preferences while also adapting to immediate, fresh signals. This design choice highlights that feature engineering is not a one-off task but an ongoing, complex process. A robust feature store and a real-time feature computation pipeline are therefore essential for high-performing, adaptive recommendation models. The combination of batch and real-time aggregation suggests an architecture akin to Lambda or Kappa architectures for feature generation.18

Furthermore, the role of UUA as a “centralized, real-time stream of user actions” 13 that is consumed by machine learning teams 13 and directly feeds into the User Signal Service and Timelines Aggregation Framework is fundamental. This architectural choice demonstrates that the underlying data architecture for machine learning is event-driven, heavily leveraging Apache Kafka.13 This ensures low-latency data availability for feature hydration and model training, which is critical for real-time recommendations. This confirms that for large-scale, real-time machine learning systems, a streaming data platform like Kafka is indispensable. It enables continuous feedback loops, allowing models to react quickly to changing user behavior and content trends.

C. Ranking with the Heavy Ranker

The “Heavy Ranker” is the core machine learning model responsible for scoring and ranking the generated candidate tweets.5 This model evaluates approximately 1,500 candidates for each user session.6

In-depth Analysis of the Neural Network Architecture and its Parameter Scale: The ranking process is accomplished using a neural network comprising approximately 48 million parameters.6 This scale indicates a deep learning approach, where the network is continuously trained on tweet interactions to optimize for positive engagements.6 The model incorporates thousands of features to assign scores based on the predicted probability of user engagement with each tweet.6
Influence of Twitter Blue Subscriptions and Account Credibility on Ranking Scores: Tweets from Twitter Blue subscribers reportedly receive a significant boost in ranking. Specifically, their tweets can achieve a 2x higher score among non-followers and a 4x higher score among followers compared to unverified posts.7 Account credibility, quantified by a Tweepcred score (a reputation metric), verification status, follower-to-following ratio, consistent account activity, and absence of prior bans, also substantially influences ranking.22 Accounts with a tweepcred score below 65 may have a limited number of their tweets considered by the ranking algorithm.2
Recency Decay and Content Type Prioritization: Recency is a paramount ranking signal, with fresh content being prioritized.15 Tweets are subject to a relevancy half-life of 360 minutes (6 hours), meaning their score decreases by 50% every 6 hours.2 Content incorporating rich media, such as images, videos, GIFs, or polls, generally exhibits superior performance and receives higher scores due to its propensity to drive increased engagement.15 Tweets containing links that display images and videos, especially those utilizing Twitter Card markup, may receive an additional advantage in ranking.28

The following table, derived from the algorithm’s internal weighting, illustrates how different user actions are valued by the Heavy Ranker when determining a tweet’s relevance score. These weights represent the relative importance assigned to each predicted user action, with Retweet serving as a baseline unit.

Heavy Ranker Engagement Weighting

User Action	Relative Weight (vs. Retweet=1)	Sentiment
Like the post	0.5	Positive
Retweet the post	1	Positive
Reply to the post	13.5	Positive
Open the post author’s profile and like or reply to a post	12	Positive
Watch at least half of the video	0.005	Positive
Reply to the post and the tweet author engages with the reply	75	Positive
Click into the conversation of the post and engage with a reply	11	Positive
Click into the conversation of the post and stay for ≥ 2 mins	10	Positive
Request “show less often”/block/mute the post author	-74	Negative
Report the Tweet	-369	Negative

The extremely high weights assigned to replies (13.5x a retweet) and particularly to replies that elicit a response from the tweet author (75x a retweet) reveal a strong algorithmic bias towards fostering genuine conversations over passive consumption (likes, retweets). This indicates that the algorithm is not merely focused on displaying popular content, but rather content that actively generates dialogue. This design choice suggests a strategic reinforcement towards building a more interactive community on the platform, moving beyond a simple broadcast medium. For engineers, this implies that features related to conversation depth, reply quality, and author responsiveness would be highly impactful in model development. It also suggests that the platform values “stickiness” derived from interaction more than just impressions.

The explicit boost for Twitter Blue subscribers and the influence of Tweepcred (a reputation score based on network structure and user behavior) indicate that the algorithm is not purely content- or engagement-driven. It incorporates a layer of “authority” or “paid privilege.” This has significant implications for content creators and platform dynamics, suggesting a tiered visibility system where paid status or established credibility can bypass some of the organic ranking challenges. For engineers, this means that these “meta-features” (subscription status, reputation scores) are directly integrated into the ranking model, potentially as high-weight features or even as post-ranking multipliers.

The neural network being “continuously trained on Tweet interactions” 6 and the emphasis on real-time features 16 and recency decay 2 highlight a system designed for rapid adaptation. The half-life of 6 hours for tweet relevance 2 indicates that the algorithm is constantly re-evaluating content freshness and user interest. This necessitates a robust MLOps pipeline for continuous integration and deployment of models, rapid feature updates, and efficient inference at scale. It implies that static, batch-trained models would quickly become stale; the system must be able to ingest new data, retrain, and deploy updated models with minimal latency to maintain relevance.

D. Core Machine Learning Models and Services

Beyond the “Heavy Ranker,” several specialized machine learning models and services contribute critical signals and embeddings to the recommendation pipeline.

Real Graph: This is a machine learning model, specifically a gradient boosting tree classifier, designed to predict the likelihood of one Twitter user interacting with another.20 It constructs a labeled dataset from a graph of Twitter users, incorporating various features such as tweet counts, follows, favorites, and other metrics related to user behavior.31 Real Graph collects both visible interactions (e.g., retweets, favorites, mentions, messages) and implicit interactions (e.g., tweet clicks, profile visits), capturing their frequency, intensity, and recency.32 This model is utilized to compute improved user recommendations, enhance the relevance of user search results, and effectively differentiate between strong and weak social ties.32 Real Graph is fundamental for understanding social tie strength, which is a critical signal for ranking in-network content and for social graph analysis in out-of-network candidate sourcing.6
SimClusters: This component serves as a general-purpose representation layer based on overlapping communities. It captures users and heterogeneous content as sparse, interpretable vectors to support a multitude of recommendation tasks.20 SimClusters discovers approximately 145,000 communities, which are updated every three weeks. These communities are anchored by influential users and are identified using a custom matrix factorization algorithm applied to the Producer-Producer similarity graph.2 The process generates “Known For” embeddings (representing a producer’s affiliation with a community) and “Interested In” embeddings (representing a consumer’s interest in communities).33 SimClusters enables content similarity assessment and community-based recommendations, which are crucial for expanding a user’s feed beyond direct connections and for identifying trending content within specific niches.2
TwHIN (Twitter Heterogeneous Information Network): TwHIN provides dense knowledge graph embeddings for Users and Tweets.1 It is trained on 7 billion tweets across over 100 languages, utilizing both text-based self-supervision and a social objective derived from rich social engagements (Favorites, Replies, Retweets, Follows) within the TwHIN.34 This model represents various entity types (User, Tweet, Advertiser, Ad) and relation types (Follow, Authors, Favorites, Replies, Retweets, Promotes, Clicks).8 It is specifically designed to capture social signals, content engagement signals, and advertisement engagements.8 TwHIN provides a richer, more comprehensive understanding of user and tweet relationships by integrating diverse interaction types into a unified embedding space, thereby overcoming the limitations of text-only models.34 It serves as a powerful feature source for the Heavy Ranker.
Tweepcred: This is a PageRank-based algorithm that calculates the influence and reputation of Twitter users based on their interactions, such as mentions and retweets.1 It considers various factors, including the follower-to-following ratio, account age, total number of followers and followings, device usage, and safety status (e.g., restricted, suspended, verified).24 Tweepcred provides a “quality stamp” for accounts, influencing their visibility and ensuring that content from credible and influential accounts is seen by a wider audience.22 It is a key factor in algorithmic prioritization.
Trust and Safety Models: This suite of models includes pNSFWMedia (detects NSFW images), pNSFWText (detects NSFW text/sexual topics), pToxicity (detects toxic content like insults), and pAbuse (detects abusive content such as hate speech or targeted harassment).1 These models are critical for content moderation, actively filtering out low-quality or harmful content.15 They are integrated into the ranking and filtering pipeline to ensure a safe and positive user experience, preventing the amplification of harmful content and maintaining platform integrity.20
Navi: High-Performance ML Model Serving in Rust: Navi is a high-performance, versatile machine learning serving server implemented in Rust. It is specifically tailored for production usage within Twitter’s technology stack.1 Navi offers gRPC API compatibility with TensorFlow Serving, enabling seamless integration with existing clients. Its pluggable architecture supports various machine learning runtimes, including out-of-the-box support for TensorFlow and Onnx Runtime, with PyTorch in an experimental state.9 Navi addresses the critical need for low-latency inference for complex machine learning models like the Heavy Ranker. Its implementation in Rust underscores the pursuit of maximum performance and efficiency for real-time scoring, directly impacting user experience and system throughput.10

The presence of SimClusters (community embeddings), TwHIN (heterogeneous network embeddings), and Real Graph (user interaction likelihood) indicates that Twitter employs multiple, specialized embedding spaces. Each of these components captures different facets of user and content relationships, such as community dynamics, interaction patterns, content similarity, and social graph structure. These are not redundant but rather complementary, providing a rich, high-dimensional representation for the Heavy Ranker. This design choice highlights that a single embedding space is often insufficient for complex recommendation tasks. Engineers should consider building a suite of specialized embeddings that capture different types of signals (e.g., semantic, social, temporal) to provide comprehensive context to ranking models. This also implies a significant investment in distributed embedding computation and serving infrastructure.

The trust-and-safety-models are listed as core “Model” components 1 and are explicitly used for filtering during the ranking process.20 This demonstrates that content moderation is not merely a separate, post-hoc layer but an intrinsic part of the recommendation pipeline. Negative feedback signals (mute, block, report) are heavily weighted negatively in the ranking.20 This illustrates a proactive approach to content moderation, where “safety by design” is embedded within the algorithm itself. Content perceived as harmful or low-quality is not just removed but actively de-prioritized or filtered out before it reaches the user’s timeline. For machine learning engineers, this implies that trust and safety signals are critical features in the ranking model’s objective function, balancing engagement with platform health.

The presence of Tweepcred, a PageRank-based influence system 35, as a distinct model component 1 that directly impacts ranking 22 signifies that the algorithm considers not just what is said or engaged with, but who is saying it. Reputation, derived from network structure and user behavior, serves as a powerful signal. This is a key differentiator from purely content-based or engagement-based systems. It suggests that a user’s standing within the network can significantly amplify or suppress their content’s reach. For engineers, developing robust and fair reputation systems (and mitigating potential biases) is a complex but high-impact area in social media recommendation.

E. Heuristics and Filters

Following the ranking stage, a series of heuristics and filters are applied to ensure that the final feed presented to the user is balanced, diverse, and safe.5

Visibility Filtering: This mechanism excludes tweets based on user preferences, such as content from blocked or muted accounts, or due to legal compliance requirements.5 It can also involve restricting abuse-prone hashtags and search results.37
Author Diversity: This heuristic prevents the display of an excessive number of consecutive tweets from a single author, ensuring variety and preventing feed monotony.5
Content Balance: The algorithm actively maintains an equitable mix of in-network and out-of-network tweets in the user’s feed.5
Feedback-based Fatigue: This filter dynamically reduces the scores of tweets that have received negative feedback from the user, such as actions like “show less often,” blocking, or muting.5 Recent negative feedback is weighted more heavily and can lead to immediate filtering of content.8
Social Proof: This filter specifically excludes out-of-network tweets that lack a sufficient connection within the user’s network. For instance, it might require a second-degree connection, meaning someone the user follows must have engaged with the tweet or followed its author. This acts as a quality safeguard for recommended content.6
Conversations: The system threads replies with their original tweets to provide necessary context, enhancing readability and understanding of ongoing discussions.6
Edited Tweets: When a tweet is edited, the system updates stale content with the revised versions, ensuring users see the most current information.6

While the Heavy Ranker primarily optimizes for engagement, these heuristics and filters address crucial aspects of user experience and platform health. They tackle issues such as feed monotony (Author Diversity), content relevance (Visibility Filtering), and quality control (Social Proof, Feedback-based Fatigue). These are not merely minor adjustments but essential components that shape the final user experience, preventing “algorithmic monoculture” or exposure to unwanted content. This highlights that a powerful ranking model alone is insufficient. A robust set of post-ranking rules and filters is necessary to ensure diversity, prevent negative user experiences (e.g., too many tweets from one person, unwanted content), and align with platform policies. This often involves a delicate balance between algorithmic optimization and human-defined rules.

The explicit mention of “Feedback-based Fatigue” 5 and the significant negative weights for “show less often,” block, mute, and report actions 20 demonstrate that user dissatisfaction is a direct input to the algorithm. This goes beyond simply not engaging; it actively suppresses content the user dislikes. This is a crucial aspect of user-centric design in recommendation systems, allowing users to “train” their own feed by providing negative signals. For engineers, it means building reliable mechanisms for capturing and propagating negative feedback signals in real-time and ensuring these signals have a strong, immediate impact on ranking. This is a key lever for improving user satisfaction and combating unwanted content.

IV. Data Infrastructure and Real-time Processing at Scale

Twitter’s recommendation system operates at an unprecedented scale, necessitating a robust and highly performant data infrastructure capable of real-time ingestion, processing, and serving.

Leveraging Apache Kafka for Real-time Data Ingestion, Streaming, and ML Logging Pipelines

Apache Kafka plays a pivotal role in Twitter’s data pipeline, enabling the real-time ingestion and streaming of massive volumes of tweets and user interactions.19 Its distributed architecture is fundamental to ensuring scalability and fault tolerance, allowing Twitter to manage extremely high data throughput.19 Twitter extensively utilizes Kafka Streams for real-time analysis and processing of tweet streams, which is critical for identifying trending topics, detecting anomalies, and extracting valuable insights with minimal latency.19 A significant application of Kafka is in the machine learning logging pipeline for the home timeline prediction system. This pipeline transitioned from a seven-day batch processing model to a one-day streaming model using Kafka and Kafka Streams, resulting in improved model quality and substantial savings in engineering time.38 User login events, tweet events, user interaction events (such as likes, retweets, and follows), and even the recommendations themselves are all produced and consumed as messages within Kafka for real-time processing and delivery.19

The Role of Apache Storm in Real-time Stream Processing for Analytics and Feature Generation

Apache Storm is employed for real-time stream processing, particularly for tasks that demand fast, scalable, reliable, and fault-tolerant continuous processing of tweets.39 It is applied in critical areas such as opinion mining (sentiment analysis) and real-time analytics for trending hashtags.39 The Timelines Aggregation Framework, a key component for feature generation, supports real-time aggregation of DataRecords through Storm, with a backing memcache for online feature hydration.17 Storm complements Kafka by providing a robust framework for complex, continuous computations on streaming data, which is essential for generating the real-time features and analytics that directly feed into the recommendation models.

Twitter’s Custom Distributed Database: Manhattan, its Consistency Model, and Use Cases

Twitter developed Manhattan, a custom-built, real-time, multi-tenant distributed key/value database. This system was designed to serve millions of queries per second with extremely low latency and high availability.41 Manhattan was created to overcome the scalability and expansion difficulties encountered with previous systems, such as Cassandra, which proved challenging to scale for Twitter’s unique demands.41 Manhattan supports various consistency guarantees, allowing clients to specify stronger consistency types when required for particular operations.42 It also offers a graph-based interface for interacting with edges and is utilized for batch Hadoop importing and time series counters.41 Manhattan represents Twitter’s solution to the challenges of storing and retrieving massive amounts of data with stringent real-time performance requirements. Its custom nature highlights the extreme demands of Twitter’s scale, where off-the-shelf solutions may not suffice. This demonstrates that for companies operating at Twitter’s scale, bespoke infrastructure is often required, driven by unique performance, consistency, and scalability demands that commercial or open-source alternatives cannot fully meet. It underscores the deep engineering expertise required to operate at such a scale, where even marginal gains in efficiency or consistency can have massive impacts.

While Twitter operates at a massive scale and employs eventual consistency for most of its systems 11, Manhattan allows clients to “set a stronger consistency guarantee” for specific operations.42 This indicates a nuanced approach to data consistency, where different parts of the system or different data types might have varying consistency requirements based on their functional needs. This is a critical lesson for distributed systems engineers: strict consistency is expensive and often unnecessary for all data. A well-designed large-scale system will apply the appropriate consistency model (e.g., eventual, strong) to different data stores or operations based on business requirements and performance trade-offs. This requires careful architectural planning and a deep understanding of distributed systems theory.

Caching Strategies for Low-Latency Data Access

Caching is a crucial optimization for speeding up data retrieval and significantly reducing the load on backend databases.11 User timelines, for instance, are extensively cached, often residing in a Redis cluster, with each user’s timeline typically having a maximum of 800 entries.4 Frequently accessed data, including user preferences or top-K recommendations, are also cached using systems like Redis or Memcached.12 Caching is a fundamental practice for read-heavy systems like Twitter, ensuring that frequently requested content is delivered with minimal latency, which directly enhances the user experience.

Underlying Scalability Principles: Horizontal Scaling, Sharding, and Load Balancing

Twitter employs horizontal scaling to distribute requests across multiple servers, a foundational principle for handling large user bases.3 Load balancers, utilizing strategies such as round-robin, dynamic, and global distribution, evenly distribute user requests and direct them to the nearest data center to minimize latency.11 Data is partitioned, or sharded, across different servers to prevent any single server from storing all data, ensuring an even distribution of workload and preventing hot spots.11 These principles are standard but absolutely critical for any system operating at Twitter’s scale, ensuring high availability, fault tolerance, and consistent performance under extreme load.

The transition of the machine learning logging pipeline from a 7-day batch latency to a 1-day streaming latency using Kafka Streams 38 represents a significant architectural evolution. This indicates a continuous drive towards real-time data processing for improved model freshness and responsiveness. This is not a static architecture but one that constantly adapts to performance and quality requirements. For data science software engineers, this highlights the long-term trend in large-scale machine learning systems: the move away from purely batch-oriented pipelines towards real-time streaming architectures. This necessitates expertise in stream processing frameworks (Kafka, Storm), event-driven design, and the challenges of managing data consistency and fault tolerance in real-time environments.

V. Evaluation and Continuous Improvement Methodologies

The Twitter algorithm is not a static entity; it continuously evolves through rigorous evaluation and iterative development. This ongoing process aims to optimize for user engagement and satisfaction while diligently mitigating any negative impacts.

Key Performance Indicators (KPIs) Used to Measure Algorithm Effectiveness

The algorithm is fundamentally designed to optimize for positive engagements, including Likes, Retweets, and Replies.6 A comprehensive set of key performance indicators (KPIs) is tracked to measure the algorithm’s effectiveness:

Engagement Metrics: These include Likes, Retweets, Replies, Clicks (on links, media, or profile), Mentions, Quote Posts, and Bookmarks.20 These metrics collectively signal content appreciation, the extent of reach, and the value generated through conversations.44
Visibility Metrics: Impressions, representing the total number of times a tweet was viewed, and Reach, indicating the unique audience exposed to tweets, are crucial for understanding overall content visibility.25
Derived Metrics: Key derived metrics include Engagement Rate (calculated as Total engagements ÷ impressions × 100) and Click-Through Rate (Link Clicks ÷ Impressions × 100).25
Growth Metrics: The number of Followers, Follower Growth Rate, and Profile Views are tracked to assess audience expansion and account interest.43
Content-Specific Metrics: For multimedia content, Video Completion Rate is monitored, and Hashtag Performance is analyzed to gauge the effectiveness of trending topics and content discoverability.46

Twitter Analytics provides detailed insights into audience demographics and the performance of individual tweets, enabling data-driven content strategy adjustments.26

A/B Testing Frameworks and Their Application in Iterative Algorithm Development

A/B testing is a foundational methodology for the iterative evaluation and development of different versions of the recommendation algorithm.47 This approach allows for controlled experimentation on specific factors, such as variations in wording, message length, overall tone, the number and relevance of hashtags, the effectiveness of different images or videos when paired with tweets, and optimal posting times.49 The typical process involves setting up a Twitter Dashboard, carefully constructing controlled tweets (where only the factor under test is varied, keeping all other elements constant), scheduling their release, and meticulously tracking their performance against predefined metrics.49 A/B testing provides a controlled, data-driven approach to understand how changes to the algorithm or content strategy directly impact user behavior, thereby enabling continuous optimization and the confident rollout of new features.

Challenges and Approaches in Evaluating Broader Algorithmic Impacts

While recommendation algorithms typically optimize for users’ revealed preferences (i.e., user engagement like clicks, shares, and likes), there is a recognized disparity with stated preferences (what users explicitly say they want).50 Research indicates that Twitter’s engagement-based algorithm has been observed to amplify emotionally charged, out-group hostile content that users report makes them feel worse about their political out-group.50 Studies leverage observational evidence and digital traces to infer the algorithmic amplification of low-credibility content, noting that high-engagement, high-follower tweets containing low-credibility URL domains can receive amplified visibility.51 Furthermore, metrics are logged for prominent Democrat and Republican accounts to understand the differential effects of features across the political spectrum.21

This situation highlights the complex societal impact of large-scale recommendation systems. Optimizing solely for engagement can lead to unintended consequences, such as the amplification of misinformation or divisive content. This necessitates a broader perspective on algorithm evaluation, incorporating user surveys, qualitative analysis, and ethical considerations that extend beyond simple engagement metrics. This underscores the critical need for multi-objective optimization in recommendation systems. Beyond maximizing engagement, models should also consider factors such as content quality, diversity, user well-being, and adherence to ethical guidelines. This requires defining and measuring these broader objectives, potentially incorporating them directly into the loss function or as post-ranking re-rankers. It also highlights the importance of interdisciplinary collaboration, for example, with social scientists and ethicists, in the design and refinement of algorithms.

The very act of open-sourcing the algorithm 7 and the stated future developments, such as “enhanced transparency regarding safety labels” and “increased visibility into the factors influencing tweet appearances” 6, indicate a growing demand for algorithmic accountability. The logging of metrics for Elon Musk’s personal experience and for prominent political accounts 21 also points to internal scrutiny and a response to external pressures regarding potential biases. This suggests a societal shift towards greater scrutiny of powerful algorithms. For engineers, this implies not only building effective models but also designing them with explainability, auditability, and fairness in mind. It signals a future where “black box” algorithms are less acceptable, and there is a growing need to articulate how algorithmic decisions are made, how biases are mitigated, and how the system aligns with broader public interest.

VI. Conclusion and Engineering Implications

The Twitter recommendation algorithm stands as a sophisticated engineering achievement, exemplifying the challenges and innovations inherent in large-scale machine learning and distributed systems. Its architecture, predominantly implemented in Scala and Java, and augmented by Rust for high-performance machine learning serving, demonstrates a pragmatic polyglot approach designed to optimize for specific performance characteristics. The multi-stage pipeline, encompassing candidate generation, feature hydration, and neural network-based ranking, is meticulously engineered to personalize content at scale.

Key implications for data science software engineers working on similar large-scale recommendation systems include:

The Transformative Power of Graph-Based Features and Embeddings: Models such as Real Graph, SimClusters, and TwHIN are foundational, illustrating the critical role of understanding complex relationships (user-user, user-content, content-content) within a social network context. Investing in robust graph processing and embedding generation capabilities is crucial for capturing these intricate dynamics.
The Imperative of Real-time Data Infrastructure: The heavy reliance on Apache Kafka for real-time data ingestion and streaming, coupled with systems like Apache Storm for real-time aggregations, underscores the non-negotiable necessity of low-latency feedback loops for dynamic recommendation systems. The freshness of data directly correlates with the relevance and responsiveness of recommendations.
Feature Engineering as a Continuous, Multi-Temporal Discipline: The sheer volume and diverse nature of features—ranging from static and real-time to aggregate and user-specific—highlight that comprehensive feature engineering, spanning both batch and real-time computation, is paramount for achieving high model performance.
Beyond Engagement: The Mandate for Multi-Objective Optimization: While engagement remains a primary driver, the analysis reveals the critical need to incorporate broader objectives into algorithmic design. These include content quality, diversity, and user well-being, which are addressed through trust and safety models, negative feedback loops, and explicit filters. This necessitates a careful definition of success metrics that extend beyond simple clicks or likes.
The “Mixer” Pattern for Scalable Orchestration: The Product Mixer framework exemplifies a modular, pipeline-driven approach to orchestrating complex recommendation logic. This design allows for the independent development and optimization of candidate sources, rankers, and filters, fostering agility and scalability.
Custom Infrastructure as a Necessity at Extreme Scale: Twitter’s development of bespoke systems like Manhattan demonstrates that for the most demanding scales, off-the-shelf solutions may not always suffice. This necessitates custom engineering solutions precisely tailored to unique performance and consistency requirements.
The Growing Importance of Algorithmic Accountability and Transparency: The open-sourcing effort and the stated focus on evaluating broader societal impacts signal an increasing demand for explainable, fair, and auditable algorithms. Engineers must consider these critical aspects from the initial design phase of any large-scale recommendation system.

The Twitter algorithm is an evolving system, constantly adapting to user behavior, content trends, and societal demands. Its open-sourced nature provides an invaluable blueprint for engineers striving to build the next generation of intelligent, scalable recommendation platforms.

Fuentes citadas

twitter/the-algorithm: Source code for Twitter’s Recommendation Algorithm - GitHub, acceso: junio 8, 2025, https://github.com/twitter/the-algorithm
Cracking the Code: How the Twitter Algorithm Works - Tweet Hunter, acceso: junio 8, 2025, https://tweethunter.io/blog/twitter-algorithm-full-analysis
Designing Twitter’s Scalable System, acceso: junio 8, 2025, https://abhisekroy.hashnode.dev/twitter-system-design
The Architecture Twitter Uses to Deal with 150M Active Users, 300K QPS, a 22 MB/S Firehose, and Send Tweets in Under 5 Seconds - High Scalability, acceso: junio 8, 2025, https://highscalability.com/the-architecture-twitter-uses-to-deal-with-150m-active-users/
the-algorithm/home-mixer/README.md at main · twitter/the-algorithm - GitHub, acceso: junio 8, 2025, https://github.com/twitter/the-algorithm/blob/main/home-mixer/README.md

Twitter’s Recommendation Algorithm: An In-Depth Overview

Talent500 blog, acceso: junio 8, 2025, https://talent500.com/blog/twitters-recommendation-algorithm-an-in-depth-overview/

Twitter Publishes its Tweet Ranking Algorithm Data on GitHub, Providing More Transparency in Process

Social Media Today, acceso: junio 8, 2025, https://www.socialmediatoday.com/news/twitter-publishes-its-tweet-ranking-algorithm-data-on-github-providing-mor/646581/

igorbrigadir/awesome-twitter-algo: The release of the Twitter algorithm, annotated for recsys - GitHub, acceso: junio 8, 2025, https://github.com/igorbrigadir/awesome-twitter-algo
the-algorithm/navi/README.md at main · twitter/the-algorithm - GitHub, acceso: junio 8, 2025, https://github.com/twitter/the-algorithm/blob/main/navi/README.md
Why Rust, acceso: junio 8, 2025, https://book.gist.rs/hello/why-rust.html
Low-Level Design of Twitter: Architecture, Tweet Processing, and Scalability, acceso: junio 8, 2025, https://getsdeready.com/low-level-design-of-twitter-how-tweets-are-processed-and-delivered/
How can you handle scalability issues in recommender systems? - Milvus, acceso: junio 8, 2025, https://milvus.io/ai-quick-reference/how-can-you-handle-scalability-issues-in-recommender-systems
Unified User Actions (UUA) - twitter/the-algorithm · GitHub, acceso: junio 8, 2025, https://github.com/twitter/the-algorithm/blob/main/unified_user_actions/README.md
How Does the Twitter (X) Algorithm Work in 2025? - Fourthwall, acceso: junio 8, 2025, https://fourthwall.com/blog/how-does-twitter-x-algorithm-work
How Does The Twitter Algorithm Work? 10 Tips - SocialPilot, acceso: junio 8, 2025, https://www.socialpilot.co/blog/twitter-algorithm
Analysis of Twitter the-algorithm source code with LangChain, GPT4 and Deep Lake, acceso: junio 8, 2025, https://python.langchain.com.cn/docs/use_cases/code/twitter-the-algorithm-analysis-deeplake
the-algorithm/timelines/data_processing/ml_util/aggregation_framework/README.md at main - GitHub, acceso: junio 8, 2025, https://github.com/twitter/the-algorithm/blob/main/timelines/data_processing/ml_util/aggregation_framework/README.md
Real-Time Data Ingestion Architecture: Tools & Examples Estuary, acceso: junio 8, 2025, https://estuary.dev/blog/real-time-data-ingestion/
Kafka and Airflow Implementation at Twitter - Anant, acceso: junio 8, 2025, https://anant.us/blog/kafka-airflow-pipelines-twitter/
Understanding the X Algorithm - Tweet Hunter, acceso: junio 8, 2025, https://tweethunter.io/blog/understanding-the-x-algorithm

What can we learn from ‘The Algorithm,’ Twitter’s partial open-sourcing of it’s feed-ranking recommendation system?

Sol Messing, acceso: junio 8, 2025, https://solomonmg.github.io/post/twitter-the-algorithm/

How the Twitter Algorithm Works in 2025 [+6 Strategies]

Sprout Social, acceso: junio 8, 2025, https://sproutsocial.com/insights/twitter-algorithm/

How To Master The X Algorithm In 2025? - Socinator, acceso: junio 8, 2025, https://socinator.com/blog/master-x-algorithm/
How the ????/Twitter Algorithm Works - Hypefury, acceso: junio 8, 2025, https://hypefury.com/blog/en/how-the-x-twitter-algorithm-works/
A Comprehensive Guide to the X Algorithm: How It Works in 2025 - Brandwatch, acceso: junio 8, 2025, https://www.brandwatch.com/blog/x-algorithm/
How The Twitter Algorithm Works: Complete Guide For 2025 - RecurPost, acceso: junio 8, 2025, https://recurpost.com/blog/twitter-algorithm/
Understanding Twitter’s Algorithm: Key Insights - Kolsquare, acceso: junio 8, 2025, https://www.kolsquare.com/en/blog/learn-the-keys-to-understanding-twitters-algorithm
Twitter’s algorithm ranking factors: A definitive guide - Search Engine Land, acceso: junio 8, 2025, https://searchengineland.com/twitter-algorithm-ranking-factors-386215
Understanding How the X (Twitter) Algorithm Works in 2025 - SocialBee, acceso: junio 8, 2025, https://socialbee.com/blog/twitter-algorithm/
How to Use the X Algorithm to Your Marketing Advantage, acceso: junio 8, 2025, https://digitalmarketinginstitute.com/blog/how-to-use-twitter-algorithm-to-your-marketing-advantage
the-algorithm/src/scala/com/twitter/interaction_graph/README.md at main - GitHub, acceso: junio 8, 2025, https://github.com/twitter/the-algorithm/blob/main/src/scala/com/twitter/interaction_graph/README.md
RealGraph: User Interaction Prediction at Twitter, acceso: junio 8, 2025, https://www.ueo-workshop.com/wp-content/uploads/2014/04/sig-alternate.pdf
the-algorithm/src/scala/com/twitter/simclusters_v2/README.md at main - GitHub, acceso: junio 8, 2025, https://github.com/twitter/the-algorithm/blob/main/src/scala/com/twitter/simclusters_v2/README.md
TwHIN-BERT: A Socially-Enriched Pre-trained Language Model for Multilingual Tweet Representations at Twitter - Ahmed El-Kishky, acceso: junio 8, 2025, https://ahelk.github.io/papers/elkishky_twhinbert.pdf
the-algorithm/src/scala/com/twitter/graph/batch/job/tweepcred/README at main - GitHub, acceso: junio 8, 2025, https://github.com/twitter/the-algorithm/blob/main/src/scala/com/twitter/graph/batch/job/tweepcred/README
the-algorithm/trust_and_safety_models/README.md at main - GitHub, acceso: junio 8, 2025, https://github.com/twitter/the-algorithm/blob/main/trust_and_safety_models/README.md
Twitter exec says it’s moving fast on moderation as harmful content surges, acceso: junio 8, 2025, https://www.straitstimes.com/world/united-states/twitter-exec-says-moving-fast-on-moderation-as-harmful-content-surges
How Twitter Built a Massive Machine Learning Pipeline Using Kafka - Confluent, acceso: junio 8, 2025, https://www.confluent.io/blog/how-twitter-built-a-machine-learning-pipeline-with-kafka/
real-time data processing with storm: using twitter streaming - ijesrt, acceso: junio 8, 2025, https://www.ijesrt.com/Old_IJESRT/issues%20pdf%20file/Archive-2017/July-2017/2.pdf
Real Time Twitter Analytics with Apache Storm - ResearchGate, acceso: junio 8, 2025, https://www.researchgate.net/publication/381793696_Real_Time_Twitter_Analytics_with_Apache_Storm
Twitter’s Manhattan: A Real-time, Multi-tenant Distributed Database - InfoQ, acceso: junio 8, 2025, https://www.infoq.com/news/2014/05/twitters-manhattan/
Providing Flexible Database Consistency Levels with Manhattan at Twitter • Boaz Avital • GOTO 2016 - YouTube, acceso: junio 8, 2025, https://www.youtube.com/watch?v=gvdXBC-NReQ
Twitter/X KPIs - Fanpage Karma Insights, acceso: junio 8, 2025, https://www.fanpagekarma.com/insights/twitter-x-kpis/
Metrics That Matter—Your Guide to Twitter User Engagement - SocialSellinator, acceso: junio 8, 2025, https://www.socialsellinator.com/social-selling-blog/twitter-user-engagement-metrics
Twitter Engagement Metric - Klipfolio, acceso: junio 8, 2025, https://www.klipfolio.com/resources/kpi-examples/social-media/twitter-engagement-metrics
7 Important X (formerly Twitter) Analytics Metrics for Marketing Agencies - Swydo, acceso: junio 8, 2025, https://www.swydo.com/blog/x-analytics-metrics/
Design Twitter: A Comprehensive Guide - System Design School, acceso: junio 8, 2025, https://systemdesignschool.io/problems/twitter/solution
Tweet Like Pro: 7 Hacks to Optimize Content for Twitter Algorithm - Tagembed, acceso: junio 8, 2025, https://tagembed.com/blog/twitter-algorithm/
How to Use the New Twitter Dashboard For A/B Testing - Online Marketing Institute, acceso: junio 8, 2025, https://www.onlinemarketinginstitute.org/blog/2016/10/original-use-new-twitter-dashboard-ab-testing/

Engagement, user satisfaction, and the amplification of divisive content on social media

PNAS Nexus

Oxford Academic, acceso: junio 8, 2025, https://academic.oup.com/pnasnexus/article/4/3/pgaf062/8052060

(PDF) Evaluating Twitter’s algorithmic amplification of low-credibility content: an observational study - ResearchGate, acceso: junio 8, 2025, https://www.researchgate.net/publication/378808348_Evaluating_Twitter’s_algorithmic_amplification_of_low-credibility_content_an_observational_study

ML Smörgåsbord 6: Neuro-Symbolic AI

2025-04-18T10:00:00+00:00

Grounding Tensors in Logic 🕸️

Welcome to the grand finale of the ML Smörgåsbord!

In Part 5, we addressed LLM hallucinations by conditioning generation on dense vector embeddings of raw text (RAG). However, text is unstructured. To perform multi-hop logical reasoning (e.g., “Who directed the movie starring the actor born in the capital of France?”), flat vector search $v_q^T v_z$ breaks down.

We need a Neuro-Symbolic approach: combining the continuous, differentiable reasoning of LLMs with the discrete, structured topology of a Knowledge Graph (KG).

1. Formalizing the Knowledge Graph

A Knowledge Graph is mathematically defined as a directed multigraph $G = (E, R, T)$, where:

$E = {e_1, e_2, \dots, e_{N_e}}$ is a set of entities (nodes).
$R = {r_1, r_2, \dots, r_{N_r}}$ is a set of relations (edges).
$T \subseteq E \times R \times E$ is a set of factual triples $(h, r, t)$ representing (head, relation, tail).

For example: $(\text{Albert Einstein}, \text{born_in}, \text{Ulm})$.

Unlike a dense continuous embedding $v \in \mathbb{R}^d$, triples in $T$ are discrete and logical. They can be queried with absolute determinism using graph traversal algorithms.

2. Unifying LLMs and KGs: The Tripartite Framework

Pan et al. (2024) categorize the integration of continuous LLMs and discrete KGs into three frameworks:

A. KG-Enhanced LLMs

During inference, a query $q$ is parsed to identify an entity $e_{start}$. We perform a $k$-hop traversal on $G$ to extract a subgraph $G_{sub} \subset G$. We linearize $G_{sub}$ into a text prompt sequence: $c = \text{Linearize}(G_{sub}) = [\text{string}(h_i, r_i, t_i)]_{i=1}^k$ The LLM generation is then conditioned on this logically rigorous context: $p_\theta(y | q, c)$.

B. LLM-Augmented KGs

Here, the LLM acts as an Information Extraction engine to construct the graph $G$ from an unstructured corpus $D$. The LLM acts as a mapping function: $f_{\theta}: d_i \rightarrow \{(h_j, r_j, t_j)\}_{j=1}^m \text{ where } d_i \in D$ This is typically done via few-shot prompting or fine-tuning on relation-extraction tasks.

C. Synergized Integration (Knowledge Graph Embeddings - KGE)

Perhaps the most mathematically elegant approach is embedding the nodes and edges of $G$ into the same continuous latent space as the LLM. Algorithms like TransE define a translation operation where the relation vector acts as an operator moving the head to the tail: $\mathbf{h} + \mathbf{r} \approx \mathbf{t}$ The loss function for the KGE model minimizes the distance function for valid triples: $\mathcal{L} = \sum_{(h,r,t) \in T} || \mathbf{h} + \mathbf{r} - \mathbf{t} ||_2^2$ Once embedded, we can pass the node vectors $\mathbf{e}$ directly into the Transformer’s input embedding matrix, allowing the LLM to attend to raw graph structures without text serialization!

3. The Neuro-Symbolic Pipeline

graph LR
    %% The continuous space
    Query[Text Query: q] --> LLM[LLM Engine]
    
    %% The discrete space
    Query --> Parser[Entity Linker]
    Parser --> StartNode[e_start]
    StartNode -->|K-hop BFS| GraphDB[(Knowledge Graph G)]
    GraphDB --> SubGraph[G_sub]
    
    %% The Bridge
    SubGraph -->|Linearization| SerializedText[Structured Context c]
    SerializedText --> LLM
    
    LLM --> Generation[Deterministically Grounded Output: y]

4. Coding the Graph Linearization

Let’s look at how we transition from discrete graph mathematics in Python (networkx) back to string sequences for an LLM prompt.

import networkx as nx

# 1. Initialize a directed multigraph G = (E, R, T)
G = nx.MultiDiGraph()

# 2. Define the set T of triples
T = [
    ("Albert_Einstein", "born_in", "Ulm"),
    ("Ulm", "located_in", "Germany"),
    ("Albert_Einstein", "studied", "Physics")
]

# 3. Populate the Graph
for h, r, t in T:
    G.add_edge(h, t, relation=r)

def linearize_k_hop_subgraph(graph, start_node, k=1):
    """
    Given a starting entity e_start, traverses k hops to build a linearized context c.
    """
    if start_node not in graph:
        return ""
        
    linearized_facts = []
    
    # 1-hop BFS for simplicity
    for neighbor in graph.successors(start_node):
        # We access the edge dictionary
        edge_data = graph.get_edge_data(start_node, neighbor)
        # There could be multiple relations between two nodes in a MultiDiGraph
        for edge_idx in edge_data:
            relation = edge_data[edge_idx]['relation']
            
            # Translate discrete triple into continuous string
            fact_string = f"{start_node.replace('_', ' ')} {relation.replace('_', ' ')} {neighbor}."
            linearized_facts.append(fact_string)
            
    return " ".join(linearized_facts)

# 4. Extract Context c
e_start = "Albert_Einstein"
c = linearize_k_hop_subgraph(G, e_start, k=1)

# 5. Formulate final probabilistic condition P(y | q, c)
q = f"Where was {e_start.replace('_', ' ')} born and what did he study?"
prompt = f"Context: {c}\nQuery: {q}\nAnswer: "

print("Generated Prompt Space:\n")
print(prompt)

That’s a wrap! 🎓

You’ve made it to the end! In this series, we’ve gone from the core calculus of convolutions in computer vision, derived the sequential memory gates of LSTMs, explored parallel self-attention mechanics of Transformers, and finally bridged continuous differentiable spaces with discrete non-parametric graphs in RAG and KGs.

Machine Learning is not a black box; it is highly structured, mathematically rigorous applied linear algebra and probability. Keep studying the equations, keep coding the algorithms from scratch, and keep pushing the boundary.

Thanks for reading!

No Assunto Do Mangue Beat

2025-04-12T00:00:00+00:00

Chico Science — um abraço mermão.

Conhece você a música de Bezerra da Silva — sambista da favela, progenitor do gênio do sambandido?

Conhece você a obra de Josué de Castro — nutricionista e filósofo da solução da fome?

Foi através de Chico Science que conheci esses caras e suas obras. Além disso, é um amor no estilo grego entre irmãos — eu sinto uma conexão profunda, artística, com esse cara no sangue.

Eu, por minha parte, tenho visto múltiplos documentários com os outros membros do grupo, o concerto jubiloso com Gilberto Gil, músico e dissidente político, em Nova Iorque, e o manifesto Manguebeat mesmo — coautorado por Fred Zero Quatro (de Mundo Livre S/A) e incluído no disco Da Lama ao Caos.

Sem dúvida, a obra de Chico Science e da Nação Zumbi constitui uma das melhores músicas dos anos noventa. Tragicamente, cessou, em certo sentido, com a morte de Chico Science, num Fiat Uno, à idade de 30 anos, o 2 de fevereiro, 1997.

Então, fico obcecado, posso te admitir — sou fanático da ideologia musical do Manguebeat, com seu animal-espírito do caranguejo elétrico — que representa a dicotomia entre a vida moderna e a natureza de Recife, no nordeste do Brasil.

E isso é a coisa, né? Que o homem tenha esquecido sua índole, tentado moldar o mangue à sua imagem — e por isso, a natureza volta com resposta — a antena fincada na lama.

A bastardização moderna do mangue se manifesta como os mutantes da imaginação recifense.

Pintamos o cenário que pariu o Manguebeat: “Com a redemocratização em 1985, o povo recifense se encontra em uma cidade com pouca inovação musical e muito decaída. Em 1991, um órgão das Nações Unidas elegeu Recife como a quarta pior cidade do mundo para se viver. A população de Recife estava sem orgulho e sem sentimento de pertencimento à própria cidade” (TELES, 2019).

Para compreender o Mangue, precisamos apreciar o maracatu — a etimologia da palavra deriva da reapropriação de um termo racista que descrevia pejorativamente a sociedade africana. Então, foi a música do povo escravizado em Pernambuco. Sua influência surgiu nos ritmos de tambores no Brock (lit. rock brasileiro) e renasceu do joie de vivre internacional desde o manguezal pernambucano. Colocou Recife no palco mundial, fonograficamente - foi a música revitalizando a cultura recifense das raízes subterrâneas presas aos galhos nodosos do mangal.

Na nossa era moderna, a essência do mangue beat é ainda mais relevante. Vemos um mundo sucumbir ao TikTok, aos reels do Instagram, à idiotez do YouTube e à supremacia da inteligência artificial. Para combater a desanimação nuclear, precisamos do maracatu atômico, quântico, e nanotecnológico: precisamos do retorno aos instrumentos feitas da madeira, da crina e do suor misturado.

Anamauê, auêia, aê
Anamauê, auêia, aê
Anamauê, auêia, aê
Anamauê

REBELLO, Pedro Frota Pires. Manguebeat: como fortalecer o movimento sem perder a sua essência? 2023. Trabalho de Conclusão de Curso (Graduação em Administração de Empresas) – Departamento de Administração, Centro de Ciências Sociais, Pontifícia Universidade Católica do Rio de Janeiro, Rio de Janeiro, 2023.

ML Smörgåsbord 5: Information Geometry of RAG

2025-04-11T10:00:00+00:00

Beyond Parametric Bottlenecks 🗄️

Welcome to Part 5! In Part 4, we showed how BERT-style Transformers learn to embed semantic information within their billions of weights $W \in \theta$. This is known as parametric memory. However, this memory is bounded by $O(

)$ and remains entirely static post-training, leading to catastrophic failure on factual temporal queries (hallucinations).

In 2021, Lewis et al. introduced Retrieval-Augmented Generation (RAG), a hybrid model linking the continuous differentiable space of a sequence-to-sequence model with a massive non-parametric vector database. Let’s dig into the math.

1. The Probabilistic Formulation of RAG

Standard language generation models the probability of generating token $y_i$ based solely on the input sequence $x$ and the preceding generated tokens $y_{<i}$: $p_{\theta}(y_i | x, y_{<i})$

RAG modifies this by conditioning the generation on a latent variable $z$, which represents a retrieved document from an external corpus $\mathcal{Z}$. The system becomes a pipeline of two probabilistic components:

**The Retriever $p_\eta(z x)$:** The probability of retrieving document $z$ given input $x$.
**The Generator $p_\theta(y_i x, z, y_{<i})$:** The probability of generating token $y_i$ conditioned on both $x$ and $z$.

Lewis et al. proposed two mathematical formulations for marginalizing out the latent document $z$:

RAG-Sequence

The model assumes the entire sequence $y$ is generated based on a single retrieved document $z$. We marginalize over the top-$k$ retrieved documents: $p_{\text{sequence}}(y | x) = \sum_{z \in \text{top-}k} p_\eta(z | x) \prod_i^N p_\theta(y_i | x, z, y_{<i})$

RAG-Token

The model assumes that the generation can pivot between different documents $z$ on a per-token basis, allowing it to synthesize facts from multiple distinct sources. We push the summation inside the product: $p_{\text{token}}(y | x) = \prod_i^N \sum_{z \in \text{top-}k} p_\eta(z | x) p_\theta(y_i | x, z, y_{<i})$

2. Dense Passage Retrieval (DPR)

How do we actually define $p_\eta(z

x)$ over a corpus of 21 million Wikipedia passages? Sparse statistical methods like TF-IDF or BM25 struggle with lexical mismatch (e.g., “author” vs “writer”).

We use DPR, a bi-encoder architecture. We instantiate two independent BERT networks: a Question Encoder $E_Q$ and a Document Encoder $E_D$. We map both strings into a shared $d$-dimensional continuous vector space: $v_q = E_Q(x) \in \mathbb{R}^d, \quad v_z = E_D(z) \in \mathbb{R}^d$

The relevance score between query $x$ and document $z$ is given by the inner product: $\text{score}(x, z) = v_q^T v_z$

We can then define the probability distribution $p_\eta(z	x)$ via a softmax over the entire corpus $\mathcal{Z}$:
$$ p_\eta(z	x) = \frac{\exp(v_q^T v_z)}{\sum_{z’ \in \mathcal{Z}} \exp(v_q^T v_{z’})} $$

Since computing the denominator over 21 million documents at runtime is intractable, we pre-compute $v_z$ offline and use Maximum Inner Product Search (MIPS) algorithms (like FAISS HNSW) to approximate the top-$k$ documents in $O(\log

\mathcal{Z}

)$ time.

3. The RAG Pipeline Diagram

graph TD
    %% Query Encoding
    Q[Input Sequence: x] --> EQ[Encoder E_Q: R^d]
    EQ --> VQ[Query Vector: v_q]

    %% MIPS
    DB[(Offline Index E_D(Z))] --> MIPS{MIPS via FAISS}
    VQ --> MIPS

    %% Retrieval
    MIPS -->|argmax(v_q^T v_z)| Docs[Top-k Documents: z_1 ... z_k]

    %% Generation
    Q --> Concat((Concatenate String Contexts))
    Docs --> Concat
    
    Concat -->|x + z_k| Gen[Seq2Seq Generator: p_θ]
    Gen --> Y[Output Sequence: y]

4. Coding the MIPS Math in PyTorch

To truly grasp semantic search, let’s write a bare-bones implementation of Maximum Inner Product Search natively in PyTorch using matrix multiplication.

import torch
import torch.nn.functional as F

def get_top_k_documents(query_embedding, doc_embeddings, k=2):
    """
    query_embedding: Tensor of shape (1, d)
    doc_embeddings: Tensor of shape (N, d) representing N documents
    """
    # 1. Compute Inner Product (Dot Product)
    # R^(1 x d) @ R^(d x N) -> R^(1 x N)
    scores = torch.matmul(query_embedding, doc_embeddings.T) # Shape: (1, N)
    
    # 2. Get the top-k highest scoring indices
    # We use torch.topk to return the values and indices
    top_scores, top_indices = torch.topk(scores, k=k, dim=1)
    
    # 3. Compute Softmax probabilities over just the top-k (p_eta)
    probs = F.softmax(top_scores, dim=1)
    
    return top_indices.squeeze(), probs.squeeze()

# Define dimensions: N=1000 docs, d=768 embedding dim
N, d = 1000, 768

# Simulate our pre-computed offline document index
Z_index = F.normalize(torch.randn(N, d), p=2, dim=1) 

# Simulate a query embedding from E_Q
v_q = F.normalize(torch.randn(1, d), p=2, dim=1)

# Retrieve top 3 documents via MIPS
indices, probabilities = get_top_k_documents(v_q, Z_index, k=3)

print(f"Top 3 Document Indices in Database: {indices.tolist()}")
print(f"p_eta(z|x) probabilities: {probabilities.tolist()}")

In Part 6, our final post, we will look at how we can replace flat unstructured document retrieval with highly structured topological searches over Knowledge Graphs. See you then!

ML Smörgåsbord 4: Transformers and BERT

2025-04-04T10:00:00+00:00

Parallelizing Language 📖

Welcome to Part 4! In Part 2, we showed that the $O(T)$ sequential bottleneck of BPTT in LSTMs inherently limits training speed. In 2017, Vaswani et al. proposed discarding recurrence entirely in “Attention Is All You Need.” Two years later, Devlin et al. utilized the encoder half of this architecture to create BERT (Bidirectional Encoder Representations from Transformers).

Today, we dive into the linear algebra that makes Transformers massively parallel and highly expressive: Scaled Dot-Product Self-Attention.

1. The Death of Recurrence: Positional Encodings

If an architecture reads a sequence $X = [x_1, x_2, \dots, x_N]$ purely in parallel, it has no inherent concept of order. Permuting the input vectors would yield the exact same output.

To inject temporal information, Transformers add a deterministic Positional Encoding matrix $P \in \mathbb{R}^{N \times d_{\text{model}}}$ to the input embedding matrix $E \in \mathbb{R}^{N \times d_{\text{model}}}$.

The continuous positional encodings are defined via interlocking sinusoids: $P_{(pos, 2i)} = \sin\left(\frac{pos}{10000^{2i/d_{\text{model}}}}\right)$ $P_{(pos, 2i+1)} = \cos\left(\frac{pos}{10000^{2i/d_{\text{model}}}}\right)$

Where $pos$ is the position in the sequence, and $i$ is the dimension index. Because $sin(a+b)$ can be expressed as a linear combination of $sin(a)$ and $cos(b)$, the network can easily learn to attend to relative positions $pos + k$.

2. Scaled Dot-Product Self-Attention

In an LSTM, state is passed horizontally across time. In Self-Attention, every token computes an interaction weight with every other token directly via matrix multiplication.

Let $X \in \mathbb{R}^{N \times d}$ be our input matrix (sequence length $N$, embedding dimension $d$). We project $X$ into three distinct spaces: Queries ($Q$), Keys ($K$), and Values ($V$):

$Q = X W^Q, \quad K = X W^K, \quad V = X W^V$ Where $W^Q, W^K, W^V \in \mathbb{R}^{d \times d_k}$ are the learned parameter matrices.

We compute the unnormalized attention scores $S$ by taking the dot product of every Query with every Key: $S = Q K^T \quad \text{(Shape: } N \times N)$

We scale the scores by $\frac{1}{\sqrt{d_k}}$. This is a crucial mathematical trick. If $Q$ and $K$ are independent random variables with mean $0$ and variance $1$, their dot product will have variance $d_k$. If variance is too high, the softmax will be pushed into regions where gradients are extremely small (vanishing gradients). Dividing by $\sqrt{d_k}$ normalizes the variance back to $1$.

We then apply softmax row-wise to get the attention matrix $A$: $A = \text{softmax}\left(\frac{Q K^T}{\sqrt{d_k}}\right)$

Finally, we compute the output matrix $Z$ by taking the weighted sum of the Values: $Z = A V$

Putting it all together into the core Transformer equation: $\text{Attention}(Q, K, V) = \text{softmax}\left(\frac{QK^T}{\sqrt{d_k}}\right)V$

3. Multi-Head Attention and the Residual Stream

A single attention head might focus heavily on syntax, but we also want to track semantics. Multi-Head Attention computes $h$ parallel self-attention operations, concatenates their outputs, and linearly projects them back to $d_{\text{model}}$.

graph TD
    %% Input
    X[Input Sequence Matrix X: N x d] --> LinearQ(X W^Q)
    X --> LinearK(X W^K)
    X --> LinearV(X W^V)
    
    %% Split Heads (Conceptual)
    LinearQ --> Q1[Q_1] & Q2[Q_2 ... Q_h]
    LinearK --> K1[K_1] & K2[K_2 ... K_h]
    LinearV --> V1[V_1] & V2[V_2 ... V_h]
    
    %% Attention
    Q1 & K1 & V1 --> Attn1(Scaled Dot-Product 1)
    Q2 & K2 & V2 --> Attn2(Scaled Dot-Product h)
    
    %% Concat
    Attn1 --> Concat((Concatenate along hidden dim))
    Attn2 --> Concat
    
    %% Output
    Concat --> OutProj[Linear Projection W^O]
    OutProj --> Z[Output Matrix Z: N x d]

To enable training of BERT-Large (24 layers, 340M parameters), we rely on Layer Normalization and Residual Connections. The output of the multi-head attention sub-layer is actually: $X_{\text{out}} = \text{LayerNorm}(X + \text{MultiHead}(X))$

4. Coding the Math: Self-Attention in PyTorch

Let’s implement the pure mathematical formulation of Scaled Dot-Product Attention in Python.

import torch
import torch.nn.functional as F
import math

def scaled_dot_product_attention(Q, K, V):
    """
    Computes Scaled Dot-Product Attention mathematically.
    Q, K, V shape: (Batch, N, d_k)
    """
    d_k = Q.size(-1)
    
    # 1. MatMul: Q * K^T
    # K.transpose(-2, -1) swaps the last two dimensions to (Batch, d_k, N)
    scores = torch.matmul(Q, K.transpose(-2, -1)) # Shape: (Batch, N, N)
    
    # 2. Scale: Divide by sqrt(d_k)
    scaled_scores = scores / math.sqrt(d_k)
    
    # 3. Softmax: Compute probabilities along the last dimension
    attention_weights = F.softmax(scaled_scores, dim=-1)
    
    # 4. MatMul: A * V
    output = torch.matmul(attention_weights, V) # Shape: (Batch, N, d_k)
    
    return output, attention_weights

# Let's verify dimensions
Batch, N, d_k = 2, 5, 64  # Batch size 2, Sequence Length 5, Embed dim 64
Q = torch.randn(Batch, N, d_k)
K = torch.randn(Batch, N, d_k)
V = torch.randn(Batch, N, d_k)

out, attn_map = scaled_dot_product_attention(Q, K, V)

print(f"Output Z shape: {out.shape}")           # torch.Size([2, 5, 64])
print(f"Attention Map A shape: {attn_map.shape}") # torch.Size([2, 5, 5])

In Part 5, we will move beyond fixed parametric networks and dive into the mathematics of Dense Passage Retrieval, using dual-encoder bi-encoders to search non-parametric vector spaces in Retrieval-Augmented Generation (RAG)!

ML Smörgåsbord 3: Spatial Attention

2025-03-28T10:00:00+00:00

Grounding Language in Vision 📸✍️

Welcome back to Part 3! In Part 1, we defined the convolutional operations that extract spatial feature maps $V \in \mathbb{R}^{C \times H \times W}$ from images. In Part 2, we formalized the LSTM sequence modeling that compresses a variable-length text sequence into a fixed-length dense vector $h_Q \in \mathbb{R}^{d_q}$.

Today, we unite these two spaces. How can a network conditionally route visual information based on a textual query? The answer lies in the mathematics of Spatial Attention.

1. The Formulation of Visual7W

In Zhu et al. (2016), the objective of Visual Question Answering (VQA) is framed as a mapping function: $f: (I, Q) \rightarrow A$ Where $I$ is an image, $Q$ is a sequence of word embeddings $[q_1, \dots, q_T]$, and $A$ is the answer (either a discrete class or a localized bounding box).

If we simply concatenate the flattened global image features $v_{global}$ with the final question hidden state $h_Q$, the network struggles to isolate small spatial details relevant to specific questions (e.g., “What color is the tie?”).

Instead, we treat the final convolutional feature map $V \in \mathbb{R}^{C \times H \times W}$ as a set of $N = H \times W$ distinct spatial vectors: $V = \{v_1, v_2, \dots, v_N\} \text{ where } v_i \in \mathbb{R}^C$

2. Deriving the Attention Mechanism

We want to compute a scalar weight $\alpha_i \in [0, 1]$ for every spatial region $v_i$, conditioned on the question vector $h_Q$. We require that $\sum_{i=1}^N \alpha_i = 1$.

We do this via a Multi-Layer Perceptron (MLP) alignment model. First, we project both the visual vector $v_i$ and the question vector $h_Q$ into a shared joint-embedding space of dimension $k$:

$z_i = \tanh(W_v v_i + W_q h_Q + b_z)$ Where $W_v \in \mathbb{R}^{k \times C}$ and $W_q \in \mathbb{R}^{k \times d_q}$.

Next, we project this joint representation to a scalar logit using a weight vector $w_a \in \mathbb{R}^k$: $e_i = w_a^T z_i + b_a$

Finally, we apply the softmax function over the $N$ regions to yield a normalized probability distribution (the attention map): $\alpha_i = \frac{\exp(e_i)}{\sum_{j=1}^N \exp(e_j)}$

The resulting context vector $\hat{v}$ is the $\alpha$-weighted sum of the original spatial features: $\hat{v} = \sum_{i=1}^N \alpha_i v_i$ This vector $\hat{v} \in \mathbb{R}^C$ now represents the exact visual information required to answer the question, filtering out irrelevant background noise.

3. Visualizing the Tensor Operations

Let’s look at the flow of tensor dimensions during the attention calculation. Assume a batch size of $B$.

graph TD
    %% Tensors
    V[Visual Map: B x N x C] -->|Linear(C -> K)| V_proj[Projected V: B x N x K]
    Q[Question Vector: B x d_q] -->|Linear(d_q -> K)| Q_proj[Projected Q: B x K]
    
    %% Broadcasting
    Q_proj -->|Unsqueeze & Expand| Q_exp[Expanded Q: B x N x K]
    
    %% Addition and Tanh
    V_proj --> Add_Op(Element-wise +)
    Q_exp --> Add_Op
    Add_Op --> Tanh_Op(Tanh)
    
    %% Scoring
    Tanh_Op -->|Linear(K -> 1)| Z[Logits e_i: B x N x 1]
    Z -->|Squeeze| Z_sq[Logits: B x N]
    
    %% Softmax
    Z_sq -->|Softmax(dim=1)| Alpha[Attention Weights α: B x N]
    
    %% Weighted Sum (Batch Matrix Multiplication)
    Alpha -->|Unsqueeze| Alpha_uns[α: B x 1 x N]
    V --> BMM(Batch Matrix Multiply: BMM)
    Alpha_uns --> BMM
    BMM --> V_hat[Context Vector v_hat: B x 1 x C]

4. PyTorch Implementation of the MLP Attention Layer

Notice in the graph above, the most efficient way to compute $\sum \alpha_i v_i$ for a batch of images is using Batch Matrix Multiplication (torch.bmm).

Let’s implement this layer mathematically identically to the equations above.

import torch
import torch.nn as nn
import torch.nn.functional as F

class SpatialAttention(nn.Module):
    def __init__(self, c_dim=512, q_dim=256, k_dim=128):
        super(SpatialAttention, self).__init__()
        # W_v projection: R^C -> R^k
        self.W_v = nn.Linear(c_dim, k_dim, bias=False)
        # W_q projection: R^d_q -> R^k
        self.W_q = nn.Linear(q_dim, k_dim, bias=True) # Bias b_z goes here
        # w_a scoring vector: R^k -> R^1
        self.w_a = nn.Linear(k_dim, 1, bias=True)     # Bias b_a goes here

    def forward(self, V, h_Q):
        """
        V: (Batch, N, C) - The flattened spatial feature map
        h_Q: (Batch, d_q) - The question hidden state
        """
        B, N, C = V.size()
        
        # 1. Project Visual Features
        v_proj = self.W_v(V) # Shape: (B, N, k_dim)
        
        # 2. Project Question Vector and broadcast to match N spatial regions
        q_proj = self.W_q(h_Q) # Shape: (B, k_dim)
        q_exp = q_proj.unsqueeze(1).expand(-1, N, -1) # Shape: (B, N, k_dim)
        
        # 3. Compute joint embedding with Tanh
        z = torch.tanh(v_proj + q_exp) # Shape: (B, N, k_dim)
        
        # 4. Compute unnormalized logits e_i
        e = self.w_a(z).squeeze(2) # Shape: (B, N)
        
        # 5. Softmax to get attention weights \alpha
        alpha = F.softmax(e, dim=1) # Shape: (B, N)
        
        # 6. Compute context vector \hat{v} via Batch Matrix Multiplication
        # alpha.unsqueeze(1) is (B, 1, N)
        # V is (B, N, C)
        # BMM( (B, 1, N), (B, N, C) ) -> (B, 1, C)
        v_hat = torch.bmm(alpha.unsqueeze(1), V).squeeze(1) # Shape: (B, C)
        
        return v_hat, alpha

# Let's verify the dimensions
B, N, C, d_q = 32, 49, 512, 256
V_tensor = torch.randn(B, N, C)
h_Q_tensor = torch.randn(B, d_q)

attention_module = SpatialAttention(c_dim=C, q_dim=d_q, k_dim=128)
context_vector, attention_map = attention_module(V_tensor, h_Q_tensor)

print(f"Context Vector shape: {context_vector.shape}") # torch.Size([32, 512])
print(f"Attention Map shape: {attention_map.shape}")   # torch.Size([32, 49])

In Part 4, we will discard the sequential nature of LSTMs entirely. We will dive into the purely parallel matrix operations of the Transformer Architecture and derive the mathematical powerhouse known as Scaled Dot-Product Self-Attention, the core engine of BERT!

이세돌

2025-03-23T00:00:00+00:00

I have since childhood felt chess a useless skill to learn given its cracking by AI automata. Why learn what at which a machine will always be superior?

And yet, now, to quote Bill Gates, intelligence is poised to become as common as water or electricity.

Perhaps, I overly invested my identity in my own natural talents of intellect.

Where do we go from here?

Do we quit “life” as 이세돌 quit the game of Go upon learning he would never achieve leverage against AlphaGo?

We are at a cross-roads.

Will humanity smoke its turn 37 cigarette for the auto-annihilation by technofeudalist overlords?

Or, will we rise up on turn 78 with a divine play to defeat the incoming ai-pocalypse?

Time will only tell.

ML Smörgåsbord 2: Recurrent Memory and LSTMs

2025-03-21T10:00:00+00:00

Stepping into the Time Dimension ⏳

Welcome back to Part 2! Last week we covered the spatial convolutions of AlexNet. Today, we shift from $x,y$ coordinates to the time axis $t$. We are going to explore how Recurrent Neural Networks (RNNs) and their upgraded cousins, Long Short-Term Memory networks (LSTMs), maintain continuous differentiable memory.

1. The Vanilla RNN Forward Pass

Let’s rigorously define a vanilla RNN. At any timestep $t$, the network receives an input vector $x_t \in \mathbb{R}^d$ and the previous hidden state vector $h_{t-1} \in \mathbb{R}^h$.

It computes the new hidden state via an affine transformation followed by a hyperbolic tangent non-linearity:

\[h_t = \tanh(W_{hx} x_t + W_{hh} h_{t-1} + b_h)\]

Where:

$W_{hx} \in \mathbb{R}^{h \times d}$ is the input-to-hidden weight matrix.
$W_{hh} \in \mathbb{R}^{h \times h}$ is the hidden-to-hidden transition matrix.
$b_h \in \mathbb{R}^h$ is the hidden bias vector.

To predict an output $\hat{y}_t \in \mathbb{R}^v$ (e.g., a probability distribution over a vocabulary of size $v$), we apply another affine projection and a softmax:

\[\hat{y}_t = \text{softmax}(W_{yh} h_t + b_y)\]

The Fundamental Flaw: Backpropagation Through Time (BPTT)

To train this, we must unroll the network through time and compute gradients. The gradient of the loss $\mathcal{L}_T$ at the final timestep with respect to the hidden state at timestep $k$ ($k < T$) involves the product of Jacobians:

\[\frac{\partial \mathcal{L}_T}{\partial h_k} = \frac{\partial \mathcal{L}_T}{\partial h_T} \prod_{j=k+1}^T \frac{\partial h_j}{\partial h_{j-1}}\]

Since $h_j = \tanh(W_{hh} h_{j-1} + \dots)$, the Jacobian $\frac{\partial h_j}{\partial h_{j-1}} = \text{diag}(\tanh’(\dots)) W_{hh}$.

If the largest singular value of $W_{hh}$ is less than 1, this product decays exponentially (Vanishing Gradients). If it’s greater than 1, it grows exponentially (Exploding Gradients). Vanilla RNNs are fundamentally unstable for long sequences.

2. Enter the LSTM: An Additive Gradient Highway

In 1997, Hochreiter & Schmidhuber solved this with the LSTM. Instead of just a hidden state $h_t$, they introduced a Cell State $c_t$.

The brilliant innovation of the Cell State is that information is updated additively rather than through matrix multiplication, acting as a “highway” where gradients can flow backward through time uninterrupted.

Here are the precise gated equations that govern an LSTM cell:

Forget Gate: Decides what to drop from the old cell state. $f_t = \sigma(W_f \cdot [h_{t-1}, x_t] + b_f)$
Input Gate: Decides what new information to add. $i_t = \sigma(W_i \cdot [h_{t-1}, x_t] + b_i)$
Candidate Cell State: Generates potential new values. $\tilde{c}_t = \tanh(W_c \cdot [h_{t-1}, x_t] + b_c)$
Cell State Update: (The Additive Highway!) $c_t = f_t \odot c_{t-1} + i_t \odot \tilde{c}_t$
Output Gate & Hidden State: Decides what part of the cell state becomes the hidden state. $o_t = \sigma(W_o \cdot [h_{t-1}, x_t] + b_o)$ $h_t = o_t \odot \tanh(c_t)$

(Note: $\sigma$ is the sigmoid function, mapping values to $(0, 1)$ to act as a gate/valve. $\odot$ denotes element-wise Hadamard multiplication. $[h_{t-1}, x_t]$ denotes vector concatenation).

3. Visualizing the LSTM Cell Architecture

Here is a detailed schematic of the internal flow of tensors inside a single LSTM cell at timestep $t$.

graph TD
    %% Inputs
    X_t[Input: x_t] --> Concat
    H_prev[Previous Hidden: h_{t-1}] --> Concat
    C_prev[Previous Cell State: c_{t-1}] --> Add_Op

    %% Concatenation
    Concat((Concatenate)) --> W_f[Linear + Sigmoid]
    Concat --> W_i[Linear + Sigmoid]
    Concat --> W_c[Linear + Tanh]
    Concat --> W_o[Linear + Sigmoid]

    %% Forget Gate logic
    W_f -->|Forget Vector: f_t| Mult_Forget(Element-wise X)
    C_prev --> Mult_Forget

    %% Input Gate logic
    W_i -->|Input Vector: i_t| Mult_Input(Element-wise X)
    W_c -->|Candidate Vector: c~_t| Mult_Input

    %% State Update
    Mult_Forget --> Add_Op(Element-wise +)
    Mult_Input --> Add_Op

    %% New Outputs
    Add_Op --> C_next[New Cell State: c_t]
    Add_Op --> Tanh_Op(Tanh)
    W_o -->|Output Gate: o_t| Mult_Output(Element-wise X)
    Tanh_Op --> Mult_Output
    Mult_Output --> H_next[New Hidden State: h_t]

4. Coding the LSTM Equations from Scratch

To truly understand it, let’s implement the forward pass of a single LSTM cell strictly using PyTorch tensor operations, mirroring the equations above exactly.

import torch
import torch.nn as nn

class RawLSTMCell(nn.Module):
    def __init__(self, input_size, hidden_size):
        super().__init__()
        self.input_size = input_size
        self.hidden_size = hidden_size
        
        # We concatenate h and x, so the weight matrix dimension is hidden_size + input_size
        concat_size = hidden_size + input_size
        
        # The 4 linear layers for our 4 gates: Forget, Input, Candidate, Output
        self.W_f = nn.Linear(concat_size, hidden_size)
        self.W_i = nn.Linear(concat_size, hidden_size)
        self.W_c = nn.Linear(concat_size, hidden_size)
        self.W_o = nn.Linear(concat_size, hidden_size)

    def forward(self, x_t, h_prev, c_prev):
        """
        x_t: Tensor of shape (batch_size, input_size)
        h_prev, c_prev: Tensors of shape (batch_size, hidden_size)
        """
        # 1. Concatenate h_{t-1} and x_t along the feature dimension
        combined = torch.cat((h_prev, x_t), dim=1)
        
        # 2. Compute the gates
        f_t = torch.sigmoid(self.W_f(combined))     # Forget gate: (0 to 1)
        i_t = torch.sigmoid(self.W_i(combined))     # Input gate: (0 to 1)
        c_tilde = torch.tanh(self.W_c(combined))    # Candidate cell: (-1 to 1)
        o_t = torch.sigmoid(self.W_o(combined))     # Output gate: (0 to 1)
        
        # 3. Update the Cell State (The Additive Highway)
        # Element-wise multiplication is done using '*' in PyTorch
        c_t = f_t * c_prev + i_t * c_tilde
        
        # 4. Compute the new Hidden State
        h_t = o_t * torch.tanh(c_t)
        
        return h_t, c_t

# Let's test the math!
batch_size, input_dim, hidden_dim = 1, 10, 20
lstm_cell = RawLSTMCell(input_size=input_dim, hidden_size=hidden_dim)

# Dummy inputs
x_t = torch.randn(batch_size, input_dim)
h_0 = torch.zeros(batch_size, hidden_dim)
c_0 = torch.zeros(batch_size, hidden_dim)

# Run one timestep
h_1, c_1 = lstm_cell(x_t, h_0, c_0)
print(f"New Hidden State shape: {h_1.shape}") # torch.Size([1, 20])
print(f"New Cell State shape: {c_1.shape}")   # torch.Size([1, 20])

In Part 3, we will merge Part 1 (CNNs) and Part 2 (LSTMs). We will introduce the mathematical formulation of Spatial Attention, allowing an LSTM to compute dynamic weight vectors over the spatial grid of a CNN feature map. See you then!