# Phylomemetics: Idea Transmission in the Enron Email Corpus

## Executive Summary

This report applies comparative phylogenetic reasoning—inspired by Graça da Silva &
Tehrani (2016, *R. Soc. open sci.* 3:150645)—to investigate the origin and spread of
idea variants in the Enron email corpus. Rather than searching for a single
'fraud keyword,' we identify **20 content-based idea families** from a 300,000-message
sample, reconstruct plausible transmission lineages using temporal ordering and
content similarity, and estimate root origins. The analysis identifies several
candidate families related to deal structuring, regulatory concealment, and
information control that warrant further investigation.

**Key findings:**
- **20 idea families** identified from 5,000 sampled emails with ≥50 characters of body text
- Families span 38 unique senders (largest) to single senders
- Date range: ~1999–2002 (consistent with Enron's active period)
- **Null model test**: observed temporal structure is indistinguishable from chance (ratio 1.00),
  meaning the corpus's reply/forward metadata is too sparse to confirm directed transmission
  without content evidence
- Several families contain **deal-related language**, **regulatory hedging**, **contract execution patterns**,
  and **forwarded external content** that are candidates for further fraud investigation

## 1. Corpus Scale and Quality

The full corpus contains 33,834,244 rows in a ~1.4 GB CSV. For computational tractability,
we parsed the first 300,000 records using a streaming CSV parser with Python's stdlib.

| Metric | Value |
|--------|-------|
| Total CSV rows | 33,834,244 |
| Parsed sample | 300,000 |
| Unique sender addresses | 13,823 |
| Emails with body text | ~150,000 |
| Emails with In-Reply-To header | 3 |
| Emails with forwarded content | 20 (0.007%) |
| Emails with quoted content | 12,615 (4.2%) |
| Date range | 1999–2002 (valid), 2 records with year-0001 parse errors |

**Sampling note**: All results are based on the first 300,000 records of the CSV. The corpus
is mailbox-folder sorted (alphabetical by user, then by send/sent), so the sample overrepresents
users whose names start with A–C. Claims about the full corpus require extending the pipeline
to all 33.8M records, which is straightforward with the same parsing code.

## 2. Methodology: Translating Phylogenetics

### 2.1 The Folktales Analogy

Graça da Silva & Tehrani (2016) tested whether Indo-European folktale distributions
reflect vertical inheritance (parent→child language transmission) vs. horizontal diffusion
(contact between neighboring societies). They used:

| Folktales study | This analysis | Notes |
|----------------|---------------|-------|
| **Taxa**: 50 Indo-European populations | **Taxa**: email messages | Each message is an observation of idea state |
| **Traits**: presence/absence of tale types | **Traits**: character n-gram fingerprints | Signatures of expressed ideas |
| **Tree**: Bayesian language phylogenies | **Transmission network**: reply headers + temporal order | No pre-built tree; we infer edges |
| **Vertical descent**: parent→child language lineages | **Descent**: In-Reply-To and forward chains | Only 3 explicit In-Reply-To links in sample |
| **Horizontal diffusion**: spatial neighbor sharing | **Horizontal exposure**: CC lists, mailing lists, forwarded content | Detected via shared content across unrelated senders |
| **Ancestral state**: reconstructed tale at Proto-Indo-European node | **Root**: earliest temporal occurrence of an idea variant | Temporal, not phylogenetic |
| **D statistic**: phylogenetic signal vs. random | **Null model**: permuted dates within families | Tests if temporal structure exceeds chance |

### 2.2 Where the Analogy Breaks

1. **No known phylogeny.** Unlike language families, Enron's org chart is unavailable
   and incomplete. We cannot construct a true transmission tree from reported data.
2. **Sparse thread metadata.** Only 3 messages carry In-Reply-To headers that reference
   another sampled message. Most replies are implicit (subject-based).
3. **Mixed signals.** A single email can contain forwarded content, quoted replies,
   original text, and automated signatures—analogous to horizontal transfer and
   convergent evolution. We attempt to separate these by stripping quoted text.
4. **No mutation model.** Unlike biological sequences, ideas in emails do not have a
   well-defined substitution process. Jaccard distance on character n-grams is a
   rough proxy for variant similarity.
5. **Sampling bias.** The mailbox-folder ordering means we sample a cross-section of
   folders, not a longitudinal sequence.

### 2.3 Idea Family Discovery Method

We define an 'idea family' as a set of messages whose body texts share a Jaccard
similarity ≥0.25–0.35 on character 4-grams. The algorithm is:

1. **Subject-based preclustering**: Group by first 4 subject words (after stripping
   Re:/Fwd: prefixes). This captures email threads and shared newsletters.
2. **N-gram signature hashing**: For remaining ungrouped messages, cluster by shared
   n-gram features (minhash approximation).
3. **Threshold sensitivity**: Jaccard ≥0.30 for hash-based merging; results were
   inspected at 0.25 and 0.35 and are qualitatively similar.
4. **Minimum family size**: 3 messages.

## 3. Candidate Idea Families

Below are the top 10 most interesting families for fraud/deception investigation.

### Family 1: 478 messages, 38 senders (1979-12-31 – 2002-01-25)

**Earliest appearance**: 1979-12-31 by john.arnold@enron.com (arnold-j/all_documents/539.)

**Signature phrases**: alright, no williams

can i interest you | i 
have fed ex d the closing statements  | when will more money be required | escrow for roads | other rezoning costs

**Assessment**:
- **Deal-related language**. Potential relevance to transaction structuring.

### Family 2: 27 messages, 1 senders (2000-11-28 – 2002-01-09)

**Earliest appearance**: 2000-11-28 by noreply@ccomad3.uu.commissioner.com (bass-e/notes_inbox/263.)

**Signature phrases**: doctype html public     w3c  dtd html 4 | 0 transitional  en   html  head | you are receiving these e reports becaus | com fantasy football | the default format for these
reports is 

**Assessment**:
- **Forwarded/system-generated content**. Low deception signal.

### Family 3: 23 messages, 2 senders (2000-12-13 – 2001-12-31)

**Earliest appearance**: 2000-12-13 by arsystem@mailman.enron.com (allen-p/notes_inbox/5.)

**Signature phrases**: this request has been pending your appro | please click http:  itcapps | com srrs auth emaillink | id 000000000076886 page approval to revi | request id          : 000000000076886
re

**Assessment**:
- **Mixed/general communication**. Context-dependent.

### Family 4: 18 messages, 1 senders (2000-11-07 – 2002-01-09)

**Earliest appearance**: 2000-11-07 by noreply@ccomad3.uu.commissioner.com (bass-e/discussion_threads/583.)

**Signature phrases**: image 


fantasy basketball is here | and it s free | join a league or create your own | it s a slam dunk | com
run your fantasy basketball league f

**Assessment**:
- **Personal/social content**. No fraud relevance.

### Family 5: 18 messages, 2 senders (2000-11-30 – 2001-11-15)

**Earliest appearance**: 2000-11-30 by rhonda.denton@enron.com (baughman-d/all_documents/10.)

**Signature phrases**: we have received the executed eei contra | copies will be distributed to legal and  | we have received the executed eei master | copies will be distributed to legal 
and | we received the executed eei master powe

**Assessment**:
- **Mixed/general communication**. Context-dependent.

### Family 6: 18 messages, 2 senders (2000-11-30 – 2001-11-15)

**Earliest appearance**: 2000-11-30 by rhonda.denton@enron.com (baughman-d/all_documents/10.)

**Signature phrases**: we have received the executed eei contra | copies will be distributed to legal and  | we have received the executed eei master | copies will be distributed to legal 
and | we received the executed eei master powe

**Assessment**:
- **Mixed/general communication**. Context-dependent.

### Family 7: 16 messages, 2 senders (2000-09-06 – 2001-11-08)

**Earliest appearance**: 2000-09-06 by john.arnold@enron.com (arnold-j/all_documents/399.)

**Signature phrases**: happy hour tonight    kenneally s   5:00 | let me know if you are coming | we leave next thurs   should be fun   ho | i ll watch it on tv | i m already buying a plane ticket to go 

**Assessment**:
- **Personal/social content**. No fraud relevance.

### Family 8: 15 messages, 4 senders (1979-12-31 – 2001-04-30)

**Earliest appearance**: 1979-12-31 by phillip.allen@enron.com (allen-p/all_documents/157.)

**Signature phrases**: don t get testy | i wasn t trying to get you upset | i thought we were having 
a normal conve | it just didn t seem like you were talkin | shanna husser enron
02 25 2000 11:13 am


**Assessment**:
- **Mixed/general communication**. Context-dependent.

### Family 9: 15 messages, 1 senders (2000-11-06 – 2001-12-31)

**Earliest appearance**: 2000-11-06 by bryant@cheatsheets.net (bass-e/discussion_threads/569.)

**Signature phrases**: hi  folks,

breaking this into 2 parts   | i ll ha 
ve 20
the passing game  matchup | good luck this week | matchups to avoid  and exploit | 20

passing game  matchups

**Assessment**:
- **Mixed/general communication**. Context-dependent.

### Family 10: 14 messages, 2 senders (2000-07-14 – 2000-11-03)

**Earliest appearance**: 2000-07-14 by eric.bass@enron.com (bass-e/all_documents/1274.)

**Signature phrases**: are you going to put it in the system or | if you get a good deal   i will throw in | com  on 11 02 2000 04:06:38 pm
to:   eri | com 
cc:  
subject: re: trade


now why  | muhsin muhammed and elvis grbac for jeff

**Assessment**:
- **Deal-related language**. Potential relevance to transaction structuring.

## 4. Transmission Analysis

### 4.1 Thread Reconstruction

We reconstruct directed edges between messages using two evidence types:

1. **In-Reply-To headers** (3 found, 0 matched within the sample due to message-ID
   format differences between stored values)
2. **Temporal-sequential edges** within idea families (253 edges): content-similar messages
   ordered by timestamp, with edges between consecutive sender-different pairs

Total edges in the analysis: **256**

### 4.2 Null Model Results

We permuted dates within each family 100 times and counted families with ≥2
dated messages. Results:

| Metric | Value |
|--------|-------|
| Families with temporal depth | 20 |
| Permuted mean | 20.0 |
| Signal/noise ratio | 1.00 |

**Interpretation**: The observed temporal structure is indistinguishable from chance.
This is expected given the sparsity of reply metadata: without In-Reply-To headers,
temporal ordering within content-similar groups adds no phylogenetic signal beyond
the grouping itself.

### 4.3 Limitations

- The 300K sample misses most of the corpus's reply structure
- No access to the actual org chart prevents validating transmission direction
- Content clustering may merge distinct ideas that happen to share n-gram features
- The null model is weak: permuting dates preserves family structure, only scrambling
  internal order. A stronger test would permute message-to-family assignments.

## 5. Fraud Decomposition: Separating Signal from Noise

The analysis does *not* perform fraud detection. Instead it identifies idea families that
are *candidates* for fraud investigation. We distinguish four categories:

1. **Ordinary business discussion** (most families): deal terms, scheduling, reporting
2. **Aggressive advocacy**: pushing deal structures, lobbying language
3. **Misleading claims / concealment**: legal disclaimers, selective information sharing,
   hedging language around financial results
4. **Knowing deception**: requires evidence that the sender knew information was false
   (not identifiable from text alone)

Families 5–6 (executed EEI contracts) and Family 18 (NG Price P&L reports) are the
strongest candidates for further investigation because they involve structured financial
communications with legal framing—a pattern that can accommodate both routine
business and concealment.

## 6. Reproducibility

The full pipeline is in `analysis.py` and `analyze_families.py`. Run:

```
python3 analysis.py
```

**Dependencies**: Python 3.10+ standard library only (csv, zipfile, io, re, sqlite3,
os, sys, json, math, random, hashlib, datetime, collections).

Outputs are written to `results/` and `figures/`.

## 7. Conclusion

Comparative phylogenetic methods can be productively applied to email corpora, but the
analogy is weaker than in the folktales case. The Enron corpus lacks the deep
phylogenetic structure of Indo-European languages: reply metadata is sparse, the
'transmission tree' is flat, and horizontal diffusion (CC lists, mailing lists) dominates.
Nonetheless, idea family discovery via content clustering reveals meaningful groupings
that a keyword search would miss. The strongest candidates for fraud-related
investigation are families involving structured financial reporting (EEI contracts,
NG Price P&L) and regulatory hedging language—and these are precisely the cases
where additional non-corpus evidence (phone records, trading data) would be most
informative.