The Marvel Web

Week 06 · From counts to meaning

Names or story?

When TF-IDF says two Marvel pages are alike, how much of that comes from the characters' names, and how much from what happens to them? A cosine similarity is a sum over words, so we can answer exactly, for every one of the 45,753 pairs.

Albert, Viktor & Mariona Representation: TF-IDF + cosine, split word by word 303 pages · week-1 link network 12 min read
98%
of pages have a lookalike that matches them mostly on names
29→87%
names' share of a page's words vs. of its TF-IDF vector (medians)
76%
of lookalikes change once only story words count
71→46%
of lookalikes are pages that link to each other, before vs. after

01 · The questionStorm is not the Human Torch

In the course explorable, TF-IDF puts Storm right next to the Human Torch. They aren't on the same team or in the same story, and they have different powers. The Human Torch is Johnny Storm, and the word storm shows up all over both pages. One shared surname was enough to make them the closest pair.

That made us suspicious of every "most similar character" list we had built this week. So we asked one question about the whole corpus:

When two Marvel pages look alike to TF-IDF, how much of the match is names, and how much is story? And once the names are handled, who resembles whom?

It matters for the network too. A hyperlink from page A to page B means A mentions B by name, so a similarity built on names may just be the link network read back from the text. A similarity built on story could tell us something the links don't.

02 · The trickA cosine is a receipt

Each page is a TF-IDF vector over 14,402 words, scaled to length 1. The cosine between two unit vectors is their dot product, a plain sum over words:

cos(a, b) = Σt at · bt = Σnames + Σpaperwork + Σstory

Each word adds its own product, and nothing else contributes. So any cosine can be split into exact parts, with no approximation or second model. We sort every word into one of three classes, using two rules we can defend:

Names

A word is a name if more than half of its uses in the corpus are capitalized. This is the course's rule. It catches storm, frost, kane, peter, avengers, claremont, so teams, places and writers count as names too.

Paperwork

A non-name word is paperwork if more than 70% of its uses sit in the real-world sections (Publication history, Creation, In other media, Reception, Collected editions). Those sections hold 34% of all words, so 70% means at least twice that.

Story

Everything else: what the characters are, do and suffer. Symbiote, vampire, armor, telekinetic, mutant, resurrection. These are 9,314 of the 14,402 words.

Figure 1 · Anatomy of a lookalike

Pick a pair and see which words make up its cosine

The bar is the whole cosine. Each segment is one word's product at·bt, sorted by class. The 14 largest words are shown and the rest are merged into "other". Below the bar, pick a word to read its concordance lines: the sentences where it occurs on each page. That's how we checked every claim in this post. Storm ↔ Human Torch: 0.37 of the 0.40 comes from the one word storm.

03 · The answerAlmost every lookalike is a name

Now do it for every page. For each of the 303 pages, take its nearest neighbor, the page with the highest cosine, and split that cosine into names, paperwork and story. Then line the 303 bars up.

Figure 2 · The nearest neighbor of every page

303 lookalikes, split by what they share

One thin bar per page, sorted by how much of its match is names. Hover a bar for the pair and its top words. Click it to inspect it below.

Bar = the cosine between a page and its single most similar page, split into the three word classes (with Share of cosine, each bar is scaled to 100%). The dashed line is 50%. A gold tick under a bar means a one-name match: a single name does almost all of the name work (fewer than 1.5 "effective" names; see section 4).

The orange takes up almost the whole chart. For 297 of 303 pages (98%) names make up most of the nearest-neighbor cosine, and the median is 93%. Story words contribute a median of 6%, and paperwork barely registers.

Why names win

TF-IDF rewards a word that is frequent on this page and rare everywhere else, and a character's name fits that exactly. Names are only 29% of a typical page's words (stopwords removed), but they carry 87% of the squared length of its TF-IDF vector. So a vector is mostly the character's name, and two vectors line up when they share a name. Across all 45,753 pairs the split is milder (names 54%, story 42%, paperwork 5%). The closer the match, the more of it is names.

04 · A closer lookTwo kinds of name match

Names aren't all noise, though. Iron Fist's nearest neighbor is Lei Kung, his teacher, through k'un-lun, rand, fist, kung, lei: that is shared story, written in proper nouns. Storm ↔ Human Torch is a pun. To tell them apart, we measured how concentrated the name part of the cosine is. If qt is each name word's share of the name part, the effective number of names is 1 / Σqt2: 1 when one name does everything, larger when many names share the work.

One-name matches · Neff < 1.5
91 pages · 45% linked
    Shared-cast matches · Neff ≥ 1.5
    212 pages · 83% linked

      The one-name matches are a gallery of homonyms. Emma Frost ↔ Jack Frost, Solomon Kane ↔ Garrison Kane, Arabian Knight ↔ Moon Knight, and Breeze Barton ↔ Hawkeye (Clint Barton). Peggy Carter matches Dogpool because Dogpool was played on screen by a dog actress named Peggy. Fewer than half of these pairs are linked. The shared-cast matches are linked 83% of the time, much closer to what you'd expect if the text were just repeating the hyperlinks: if A links to B, A's page contains B's name.

      05 · Names handledTake the names away and the lookalikes change

      Now drop the name and paperwork words, rescale each page to length 1 over story words only, and find the neighbors again. 76% of pages get a different nearest neighbor. Storm now matches Jean Grey (mutants, relationship, powerful, and also female, which comes back in section 7) instead of the Human Torch. Morbius matches Blade (vampire, blood, undead, bloodlust), and Garrison Kane matches Yo-Yo Rodriguez, because both have bionic arms.

      Figure 3 · One page, two representations

      Whom does a character resemble, with names and without?

      Left: the 8 closest pages by full TF-IDF. Right: the 8 closest by story words only. A line joins a page that appears in both lists. The tag shows how far apart the two pages are in the link network.

      Bar length = cosine (both columns on the same scale). Network distance: linked = one page links to the other; 2 hops = they share a neighbor; 3+ or none = far apart or in different components. Click any name to make it the focus.

      To see all the pairs at once, plot each one by both similarities. The diagonal is "the names didn't matter". Pairs far below it are name lookalikes that don't survive, and pairs above it are story twins that the names were hiding.

      Figure 4 · Every candidate pair, both ways

      Name lookalikes fall below the diagonal; story twins sit above it

      Each dot is a pair of pages. Across: full TF-IDF cosine. Up: story-only cosine. Hover to identify, click to inspect.

      — pairs: taken from each page's top 5 neighbors under both representations plus the 300 highest story-only pairs, keeping those where either cosine is at least 0.08. Both axes use a square-root scale so the crowded low end spreads out (the diagonal is still y = x). Most of the other 45,753 pairs sit near zero in the bottom-left corner. Gold dots are pairs that are 3+ hops apart or disconnected.

      The two sides of the diagonal sit differently in the network. Below it, where the names did the work, 49% of pairs are linked and 18% are 3+ hops apart. Above it, where story words did, only 24% are linked and 31% are far apart: that's the gold cloud in the upper left, pairs with different names (low full cosine) but a story cosine of 0.1–0.4.

      06 · Reading the textAre the story twins real?

      A high cosine doesn't prove two characters have something in common, so we read the pages. We took the 20 most similar unlinked pairs by story-only cosine and checked the concordance lines of their top words (the same view as Figure 1):

      #Pair · click to openstory costop shared story wordswhat we found in the text

      Twelve of the twenty share real story: the symbiotes Hybrid, Mania and She-Venom; Morbius and Spitfire, who was drained of blood by her vampire uncle Baron Blood; Blackthorn and Scaredycat, both volunteers for the suicidal Morituri process. The misses fall into three types. One is a name leak: brain in Bird-Brain ↔ Brain Drain is usually lowercase, so the capitalization rule misses it. Four are generic words (great, work, costume). Three are a stock description we'll come back to in section 7.

      Nineteen of the 20 are exactly two hops apart: no link between them, but a neighbor in common (the twentieth is the Bird-Brain name leak). To find story twins that are really far apart we have to look further down the list, and there a pattern shows up. The twins at 3+ hops are power types the link network never put together. Turbo sits in a scattered group of people in built suits: Detroit Steel (4 hops away), Guardian, Force, Frog-Man, Stingray. Blue Eagle and Red Raven both fly on artificial wings. Makkari and Speed are speedsters, and Black Knight and Brian Braddock share a cursed sword.

      Figure 5 · Do the links already know?

      How often a page's lookalike is a page it is linked to

      Share of the 303 nearest-neighbor pairs that are directly linked, under each representation, against the 3.1% of all page pairs that are linked. As a ranking, names-only cosine separates linked from unlinked pairs with AUC 0.93 and story-only with 0.86, so story still says a lot about links, just less.

      The links record who you know. The story words record what you can do.

      Wikipedia editors link a page to teammates, relatives, mentors and enemies. TF-IDF on raw text mostly finds those same people again, by their names. Story words find characters with the same kind of existence: the same powers, origins and technology. Those pages often aren't linked, because the characters rarely appear in each other's stories.

      07 · What surprised usThe women find each other

      Three of the misses in section 6 had the same cause: the sentence "described as one of Marvel's most notable and powerful female heroes", which appears word for word on She-Hulk's and Spider-Woman's pages, with a close variant on Jean Grey's. That made us check gender, which we had not planned to. We labelled each page by its pronouns (she/her vs. he/his, a 2:1 majority) and got 57 women, 227 men and 19 unclear. Pronouns are stopwords, so they never enter the vectors.

      Figure 6 · Same-gender lookalikes

      How often a woman's lookalike is another woman

      What the women's pages share: share of pages that mention…

      Top: for the 57 women, the share whose nearest neighbor is also a woman. Bottom: share of women's and men's pages whose raw text matches each pattern at least once (e.g. telekine*). Sorted by the gap between them.

      With full TF-IDF, a woman's lookalike is a woman 28% of the time, not much above the 19% you'd get by chance. With story words it's 53%. That isn't just the word female: removing 30 explicitly gendered words (female, woman, girlfriend, husband, mother…) still leaves 51%. The words that carry the women-to-women matches are mutants, telepathy, telekinetic, psychic. Marvel gives its women mind powers (telekinesis on 16% of women's pages vs. 5% of men's) and luck or probability powers (12% vs. 2%). Armor and guns lean male. There is also how the pages are written: relationship appears on 49% of women's pages and 27% of men's. Some of this is in the comics, and some is in how Wikipedia writes about women characters, and this method can't separate the two.

      08 · Is it robust?Changing the rules

      Both class rules and the TF weighting are choices, so we re-ran everything with each one changed:

      variantname share of lookalike (median)pages matched mostly on nameslookalike changes without namesstory lookalike linked

      The class rules barely matter: thresholds of 30% or 70%, or adding every word of the 303 page titles to the names, all give 91–94%. The TF weighting matters a lot. Sublinear TF (1 + log count) caps how much a page's 200th mention of its own name counts, and the name share falls to 58%. Even so, names are still most of the match for 63% of pages. So "names dominate" holds for both weightings. "Names are 93% of it" only holds for the standard one.

      09 · LimitationsWhat this can't tell you

      • A name is a spelling, not a meaning. The capitalization rule calls Avengers, Claremont and Wakanda names, and they carry real information. It also misses names written as ordinary words (brain, steel, wolf when lowercase). "Names" here means proper nouns, not "noise".
      • Story words still include style. The "powerful female heroes" phrase passes both filters, because it is neither capitalized nor confined to the real-world sections. A third rule for stock phrases (week 5's shared 8-grams) would catch it, and we didn't add one.
      • One representation, decomposed. We split TF-IDF itself rather than comparing it with topic models or Word2Vec. The decomposition is exact because the cosine is linear in words, and a dense embedding can't be split this way.
      • Pronoun gender is crude. It mislabels team pages and characters with several incarnations, and drops 19 pages. The section 7 numbers are a pattern worth checking, not a measurement.

      Reproduce: cd analysis && python week6_names.py (about 20 s; writes assets/week6_names.json and assets/week6_conc.json). TF-IDF: scikit-learn defaults (smooth idf, l2 norm), English stopwords, min_df=2, max_df=0.5, prose before the References heading. Data: the course's frozen Marvel snapshot (marvel_pages.zip + week-1 network, 2026-08-26).