Illocution’s lexicon lists the most common and least common bigrams (2-word combinations) that make up today’s Tweets. Illocution analyzes a million tweets per month, so it’s probably a reliable snapshot of the English-speaking Twitterverse.
Lexicons help us to identify so-called “trash phrases” – word combinations so commonly used that they don’t carry much signal/meaning. These high-frequency/low meaning phrases are often sources of noise that we can filter out of our analysis. Lexicons are also useful in helping us target the high-signal/high-volume phrases to prioritize when looking for needs (after considering the domain and other factors).
Copied below is a list of the top 100 most-mentioned two-word phrases used on Twitter during the period 12/2011 to 2/2012. Note that hashtags, @names and links are not in this list – they were deleted from the sample tweets before processing.
Below, I’ve highlighted the n-grams that are obvious markers for needs/complaints/questions, etc. Notice how frequently people use the phrases, “I love”, “I want”, “I need”, “I think”, and “I hate”. Out of millions of possible 2-word combinations, these phrases rank in the top 100.
This analysis provides clear evidence that people spend a lot of time talking about their needs, desires and opinions on Twitter.
rank gram freq count freq% tweet count tweet%
———————————————————————————
1: in the 61,882 0.2249 433,605 14.5872
2: to be 38,798 0.1410 285,429 9.6023
3: of the 38,560 0.1402 287,309 9.6656
4: i love 38,176 0.1388 269,417 9.0637
5: on the 32,884 0.1195 238,739 8.0316
6: if you 31,371 0.1140 299,150 10.0639
7: to the 30,447 0.1107 227,295 7.6466
8: i just 30,384 0.1104 294,692 9.9139
9: i don’t 29,489 0.1072 282,865 9.5161
10: for the 28,167 0.1024 205,523 6.9141
11: i have 27,870 0.1013 269,376 9.0623
12: going to 26,240 0.0954 216,162 7.2721
13: i want 24,791 0.0901 195,554 6.5788
14: to get 24,321 0.0884 183,412 6.1703
15: i need 23,757 0.0863 181,887 6.1190
16: want to 23,333 0.0848 201,774 6.7880
17: in my 22,671 0.0824 152,183 5.1197
18: on my 22,202 0.0807 147,351 4.9571
19: is a 21,841 0.0794 169,960 5.7177
20: have a 21,319 0.0775 174,108 5.8573
21: need to 21,090 0.0767 176,239 5.9290
22: i hate 20,956 0.0762 199,094 6.6979
23: i was 20,928 0.0761 203,952 6.8613
24: in a 20,883 0.0759 153,850 5.1758
25: i think 20,099 0.0731 188,764 6.3503
26: at the 19,759 0.0718 149,926 5.0438
27: to go 19,528 0.0710 137,493 4.6255
28: for a 19,488 0.0708 144,655 4.8664
29: i can 19,170 0.0697 160,669 5.4052
30: i am 19,083 0.0694 160,207 5.3896
31: is the 19,045 0.0692 153,710 5.1711
32: when i 18,757 0.0682 193,264 6.5017
33: and i 18,484 0.0672 159,723 5.3734
34: rt i 18,243 0.0663 200,219 6.7357
35: rt rt 17,844 0.0649 155,462 5.2300
36: but i 17,671 0.0642 150,712 5.0702
37: go to 17,552 0.0638 124,625 4.1926
38: this is 17,401 0.0632 147,500 4.9622
39: right now 17,345 0.0630 69,145 2.3262
40: the best 17,291 0.0628 120,267 4.0460
41: have to 16,986 0.0617 147,550 4.9638
42: to do 16,585 0.0603 105,056 3.5343
43: i got 16,072 0.0584 142,080 4.7798
44: like a 15,875 0.0577 104,296 3.5087
45: i can’t 15,819 0.0575 134,239 4.5160
46: if i 15,758 0.0573 177,054 5.9564
47: i will 15,546 0.0565 131,799 4.4339
48: i know 15,511 0.0564 144,668 4.8669
49: when you 15,486 0.0563 161,924 5.4474
50: all the 15,230 0.0554 124,212 4.1787
51: i’m at 15,047 0.0547 119,909 4.0339
52: of my 14,987 0.0545 111,370 3.7467
53: to my 14,037 0.0510 103,702 3.4887
54: with the 13,989 0.0508 96,407 3.2433
55: love you 13,932 0.0506 71,155 2.3938
56: a good 13,569 0.0493 89,105 2.9976
57: with my 13,222 0.0481 82,653 2.7806
58: to me 13,152 0.0478 77,369 2.6028
59: i feel 13,007 0.0473 109,739 3.6918
60: out of 12,856 0.0467 90,943 3.0595
61: do you 12,600 0.0458 106,886 3.5958
62: you are 12,478 0.0454 99,898 3.3607
63: to see 12,456 0.0453 95,072 3.1984
64: how to 12,328 0.0448 99,411 3.3444
65: will be 12,298 0.0447 94,428 3.1767
66: be a 12,261 0.0446 86,509 2.9103
67: i get 11,972 0.0435 100,867 3.3933
68: with a 11,920 0.0433 89,069 2.9964
69: you can 11,863 0.0431 102,885 3.4612
70: i wanna 11,846 0.0431 95,987 3.2292
71: so much 11,592 0.0421 75,002 2.5232
72: that i 11,483 0.0417 105,736 3.5571
73: to sleep 11,415 0.0415 56,539 1.9021
74: i’m not 11,392 0.0414 101,679 3.4207
75: me and 11,343 0.0412 101,729 3.4223
76: for me 11,285 0.0410 60,976 2.0513
77: you know 11,152 0.0405 107,947 3.6315
78: you have 11,139 0.0405 104,274 3.5080
79: it was 11,112 0.0404 80,737 2.7161
80: i wish 11,000 0.0400 111,581 3.7538
81: back to 10,973 0.0399 68,410 2.3014
82: a new 10,959 0.0398 75,215 2.5304
83: and the 10,927 0.0397 84,607 2.8463
84: feel like 10,923 0.0397 86,133 2.8977
85: so i 10,882 0.0396 105,857 3.5612
86: i miss 10,784 0.0392 68,308 2.2980
87: a video 10,667 0.0388 77,032 2.5915
88: one of 10,566 0.0384 97,403 3.2768
89: the same 10,498 0.0382 67,070 2.2564
90: o o 10,398 0.0378 28,221 0.9494
91: my life 10,238 0.0372 58,512 1.9684
92: it is 10,109 0.0367 75,708 2.5469
93: good morning 10,104 0.0367 48,747 1.6399
94: on a 10,074 0.0366 77,647 2.6122
95: more for 9,856 0.0358 20,540 0.6910
96: the world 9,848 0.0358 62,684 2.1088
97: happy birthday 9,791 0.0356 59,355 1.9968
98: to make 9,781 0.0356 75,778 2.5493
99: you want 9,774 0.0355 73,538 2.4739
100: i really 9,730 0.0354 93,392 3.1419