{"id":232,"date":"2018-03-26T10:54:11","date_gmt":"2018-03-26T16:54:11","guid":{"rendered":"http:\/\/briancroxall.net\/w18dh\/?page_id=232"},"modified":"2018-03-29T09:57:31","modified_gmt":"2018-03-29T15:57:31","slug":"topic-modeling-step-by-step","status":"publish","type":"page","link":"https:\/\/briancroxall.net\/w18dh\/topic-modeling-step-by-step\/","title":{"rendered":"Topic Modeling Step by Step"},"content":{"rendered":"<p>So you want to be a <del>hero<\/del> topic modeler? It turns out it&#8217;s not that difficult, thanks to the work those who created and maintain MALLET and the team behind\u00a0<em>The Programming Historian<\/em>&#8216;s <a href=\"https:\/\/programminghistorian.org\/lessons\/topic-modeling-and-mallet\">lesson on topic modeling<\/a>\u00a0(see more in credits below).<\/p>\n<h2>The 7 Steps of Highly Effective Topic Modelers<\/h2>\n<h3>1. Get the JDK<\/h3>\n<p>Download and install the <a href=\"http:\/\/www.oracle.com\/technetwork\/java\/javase\/downloads\/index.html\">Java Development Kit<\/a> (JDK).<\/p>\n<p><a href=\"http:\/\/briancroxall.net\/w18dh\/wp-content\/uploads\/2018\/03\/jdk.png\"><img loading=\"lazy\" decoding=\"async\" class=\"alignnone wp-image-237 size-large\" src=\"http:\/\/briancroxall.net\/w18dh\/wp-content\/uploads\/2018\/03\/jdk-700x597.png\" alt=\"\" width=\"640\" height=\"546\" srcset=\"https:\/\/briancroxall.net\/w18dh\/wp-content\/uploads\/2018\/03\/jdk-700x597.png 700w, https:\/\/briancroxall.net\/w18dh\/wp-content\/uploads\/2018\/03\/jdk-300x256.png 300w, https:\/\/briancroxall.net\/w18dh\/wp-content\/uploads\/2018\/03\/jdk-768x655.png 768w\" sizes=\"auto, (max-width: 640px) 100vw, 640px\" \/><\/a><\/p>\n<h3>2. Get MALLET<\/h3>\n<p><a href=\"http:\/\/mallet.cs.umass.edu\">MALLET<\/a> is the\u00a0MAchine Learning for LanguagE Toolkit, which was developed at U Mass Amherst by Andrew McCallum and his grad students. <a href=\"http:\/\/mallet.cs.umass.edu\/download.php\">Download<\/a> the right zip file for your computer. What you do next depends on your operating system.<\/p>\n<h4>Windows PC<\/h4>\n<ul>\n<li>Unzip MALLET into your C: directory. This is important: it cannot be anywhere else. You will then have a directory called\u00a0<code>c:\\mallet-2.0.8<\/code>\u00a0or similar.<\/li>\n<li>MALLET uses an environment variable to tell the computer where to find all the various components of its processes when it is running. It\u2019s rather like a shortcut for the program. A programmer cannot know exactly where every user will install a program, so the programmer creates a variable in the code that will always stand in for that location. We tell the computer, once, where that location is by setting the environment variable. If you moved the program to a new location, you\u2019d have to change the variable.<\/li>\n<li>To create an environment variable in Windows 7, click on your\u00a0<code>Start Menu -&gt; Control Panel -&gt; System -&gt; Advanced System Settings<\/code>. Click new and type\u00a0<code>MALLET_HOME<\/code>\u00a0in the variable name box. It must be like this \u2013 all caps, with an underscore \u2013 since that is the shortcut that the programmer built into the program and all of its subroutines. Then type the exact path (location) of where you unzipped MALLET in the variable value, e.g.,\u00a0<code>c:\\mallet<\/code>.<\/li>\n<\/ul>\n<h4>Mac<\/h4>\n<ul>\n<li>Unzip the directory somewhere on your computer. For the sake of demonstration, let\u2019s go to your Documents folder. In that folder we\u2019ll create a new folder called \u201ctm\u201d for topic modeling (<code>mkdir<\/code>\u00a0on the command line). In\u00a0<em>that<\/em>\u00a0directory, we\u2019ll create one called \u201cmallet.\u201d<\/li>\n<li>Unzip the file in that folder.<\/li>\n<li>From the command line, navigate to that folder.<\/li>\n<\/ul>\n<h4>Check commands<\/h4>\n<p>Type\u00a0<code>.\/bin\/mallet<\/code>\u00a0(Mac) or\u00a0<code>bin\\mallet<\/code> (PC) to see the commands that you can run.<\/p>\n<h3>3. Increase MALLET memory (optional)<\/h3>\n<p>MALLET will work better if you give it more memory than it is allowed by default.<\/p>\n<ul>\n<li>Go to the\u00a0<code>bin<\/code>\u00a0directory in the MALLET folder.<\/li>\n<li>Open the\u00a0<code>mallet<\/code>\u00a0file in a text editor.<\/li>\n<li>Change the\u00a0<code>MEMORY=1g<\/code>\u00a0to a higher number to make more use of your RAM. Naturally, you can&#8217;t use more memory than you have. For example, I have 16GB of RAM, and I have set MALLET to use 8GB.<\/li>\n<\/ul>\n<h3>4. Create a MALLET corpus<\/h3>\n<p>Now you&#8217;re almost ready to start topic modeling. This is a two-step process. The first step is assembling all of your documents and then putting them into a single file for MALLET to read. It will have the extension <code>.mallet<\/code>. I&#8217;ll refer to this file as a MALLET corpus.<\/p>\n<p>First, bring all of your texts into one directory somewhere on your computer. Second, from the command line:<\/p>\n<ul>\n<li>make sure you\u2019re in the MALLET directory<\/li>\n<li>run one of the following commands based on your operating system:<\/li>\n<li>on a Mac:\u00a0<code>.\/bin\/mallet import-dir --input \u201cdirectory\u201d --output \u201cname\u201d.mallet --keep-sequence<\/code><\/li>\n<li>on a PC: <code>bin\\mallet import-dir --input \"directory\" --output \"name\".mallet --keep-sequence<\/code>\n<ul>\n<li><code>.\/bin\/mallet<\/code>\u00a0invokes MALLET<\/li>\n<li><code>import-dir<\/code>\u00a0tells it that you\u2019re importing a directory<\/li>\n<li><code>--input<\/code>\u00a0plus the directory location tells it what text files you want to pull from<\/li>\n<li><code>--output<\/code>\u00a0tells it what the MALLET corpus should be named<\/li>\n<li><code>--keep-sequence<\/code>: \u201cThis option preserves the document as a sequence of word features, rather than a vector of word feature counts. Use this option for sequence labeling tasks. The MALLET topic modeling toolkit also requires feature sequences rather than feature vectors.\u201d<\/li>\n<\/ul>\n<\/li>\n<li>Check that you have a new\u00a0<code>.mallet<\/code>\u00a0file in your directory with the\u00a0<code>ls<\/code>\u00a0command.<\/li>\n<\/ul>\n<h4>Optional Commands for Stopwords<\/h4>\n<p>MALLET comes with a stopword list that you can call when creating the MALLET corpus. To do this, add the command <code>--remove-stopwords<\/code> to the import command. To see what terms are included in the default MALLET stopwords list, go to the MALLET directory &gt; stoplists &gt; en.txt.<\/p>\n<p>You\u00a0<em>cannot<\/em> add words to the MALLET stopwords file. (Well, you <em>can<\/em>, but it won&#8217;t affect what MALLET does as the list is just there as a reference for an internal function.) Instead, you can create another text file (in TextEdit on Mac or Notepad on Windows) with one stopword per line. You should name and save this file to your MALLET directory (e.g., extra-stopwords.txt). When creating your MALLET corpus, you would use both <code>--remove-stopwords<\/code> and a new flag, <code>--extra-stopwords [filename]<\/code> (e.g., <code>--remove-stopwords --extra-stopwords extra-stopwords.txt<\/code>)<\/p>\n<p>Remember, of course, that using stop words will radically change your output. Doing so can be good <em>or<\/em> bad; the one thing for certain is that it is never simply neutral.<\/p>\n<h3>5. Train Your Topics<\/h3>\n<p>Now that we have a MALLET corpus, we are ready to start\u00a0the unsupervised machine learning. This is referred to as &#8220;training&#8221; your topics.<\/p>\n<p>From the command line, make sure that you&#8217;re (still) in your MALLET directory.<\/p>\n<p>Run the following command on a Mac:\u00a0<code>.\/bin\/mallet train-topics --input \u201cname\u201d.mallet --num-topics \"X\" --output-state \u201cname\u201d-topic-state.gz --output-topic-keys \u201cname\u201d-keys.txt --output-doc-topics \u201cname\u201d-composition.txt<\/code>. For a PC, use the same command but start it with <code>bin\\mallet train-topics...<\/code>.<\/p>\n<ul>\n<li><code>.\/bin\/mallet<\/code>\u00a0tells it what file will run the command<\/li>\n<li><code>train-topics<\/code>\u00a0is the command that you\u2019re running<\/li>\n<li><code>--input<\/code>\u00a0plus the file name that follows it is the MALLET file you\u2019ll perform your train your models on<\/li>\n<li><code>--num-topics<\/code>\u00a0plus the integer you put after it is the number of topics you have the model find. Default is 10.\n<ul>\n<li>The number of topics that you should use is perhaps the most vexing question for topic modeling. Remember that Micki Kaufman suggested she used 40 topics simply because it was how many Rob Nelson used for\u00a0<em>Mining the Dispatch<\/em>. Here&#8217;s a useful passage from Shawn Graham, Scott Weingart, and Ian Milligan&#8217;s,\u00a0<a href=\"https:\/\/programminghistorian.org\/lessons\/topic-modeling-and-mallet#typing-in-mallet-commands\"><em>Programming Historian<\/em> tutorial<\/a>:\u00a0 &#8220;How do you know the number of topics to search for? Is there a natural number of topics? What we have found is that one has to run the train-topics with varying numbers of topics to see how the composition file breaks down. If we end up with the majority of our original texts all in a very limited number of topics, then we take that as a signal that we need to increase the number of topics; the settings were too coarse.&#8221;<\/li>\n<\/ul>\n<\/li>\n<li><code>--output<\/code>\u00a0Three different files that will contain our results from the training process. See below for more details.<\/li>\n<\/ul>\n<h4>Optional Commands<\/h4>\n<p>Here is another command that you can use while training your topics:\u00a0<code>--num-iterations \"X\" --optimize-interval \u201cX\u201d<\/code>.<\/p>\n<ul>\n<li><code>--num-iterations<\/code>\u00a0plus the integer that follows tells your computer how many times it should run through the process before it yields the final models.<\/li>\n<li><code>--optimize-interval<\/code>\u00a0plus the integer you put after allows the model to learn from itself as it goes along. I find that it gets better results to include this. You need to set this to an integer. Optimization should take place every 10 intervals, <a href=\"http:\/\/mallet.cs.umass.edu\/topics.php\">according to the MALLET website<\/a>.<\/li>\n<\/ul>\n<h3>6. Record Your Work<\/h3>\n<p>Like regular close reading, there is definitely some art to the process of distant reading. One of the advantages that the latter has over the former, however, is that it can be a lot simpler to record your process so your work could be replicated. That said, once you do a series of topic models, it can be really easy to lose track of what you&#8217;ve done. Therefore it&#8217;s important to keep a record as you work.<\/p>\n<ul>\n<li>Move your three output files\u00a0to another directory. You might want to name the folder with something helpful like &#8220;duffy-nostop-10topics.&#8221;<\/li>\n<li>You should also create a note of what you did to create these files. Create a text file (either from your GUI or from the command line using <code>touch \"name\".txt<\/code>.\u00a0Go to the terminal and copy the commands that you used to create the file and then paste them into the text file.<\/li>\n<\/ul>\n<h3>7. Interpret Your Results<\/h3>\n<p>We created three files from our process:<\/p>\n<ul>\n<li>topic-state.gz: every word in your corpus of materials and the topic it belongs to. You probably don&#8217;t need to look at this.<\/li>\n<li>topic-keys.txt: a text document showing you what the top \/ &#8220;key&#8221; words are for each topic.<\/li>\n<li>composition.txt: outputs a text file indicating the breakdown, by percentage, of each topic within each original text file you imported. This is normally best viewed in Excel.<\/li>\n<\/ul>\n<p>Take a look at the topic-keys. See if there are multiple topics that seem to use the same terms. If so, you might want to re-run the model with less topics. If there doesn&#8217;t seem to be coherence among the topics, you might need to re-run the model with more topics. Consider whether there are terms that seem to be throwing off or clogging the model, including stopwords or character names. You might want to re-compose the MALLET corpus to deal with such terms.<\/p>\n<p>Take a look at the composition in a spreadsheet like Excel. Find a topic that you&#8217;re interested in, and look for the number of the topic in the column at the top of the page. Sort the spreadsheet by that column and look at which documents have the highest percentage of being composed from that topic. Are the results expected? Do the documents seem to belong together?<\/p>\n<p>Don&#8217;t forget to consider the suggestions Miriam Posner provides in her &#8220;<a href=\"http:\/\/miriamposner.com\/blog\/very-basic-strategies-for-interpreting-results-from-the-topic-modeling-tool\/\">Very basic strategies for interpreting results from the Topic Modeling Tool<\/a>.&#8221; Even though we aren&#8217;t using\u00a0the same tool that she talks about, they are still useful tips.<\/p>\n<h2>Other Resources<\/h2>\n<p>David Blei, who is the lead author on the original paper about topic modeling, wrote a\u00a0<a href=\"http:\/\/www.cs.columbia.edu\/~blei\/papers\/Blei2012.pdf\">general introduction to topic modeling<\/a>. Scott Weingart has a really good post on &#8220;<a href=\"http:\/\/www.scottbot.net\/HIAL\/index.html@p=19113.html\">Topic Modeling for Humanists<\/a>,&#8221; which I think is a useful overview for anyone, regardless of the field they are in.<\/p>\n<h2>Credits<\/h2>\n<p>This guide is a <em>very<\/em> stripped-down and adapted version of &#8220;<a href=\"https:\/\/programminghistorian.org\/lessons\/topic-modeling-and-mallet\">Getting Started with Topic Modeling and MALLET<\/a>&#8221; by Shawn Graham, Scott Weingart, and Ian Milligan, which is just one of many fabulous lessons at\u00a0<em><a href=\"https:\/\/programminghistorian.org\/\">The Programming Historian<\/a>. <\/em>This lesson and the others at\u00a0<em>Programming Historian<\/em> are released under a CC BY license, which I&#8217;m grateful for.<\/p>\n<p>I prepared a version of this guide at first for my own use and\u00a0then deployed it in a number of workshops on topic modeling for Brown University Library&#8217;s <a href=\"https:\/\/library.brown.edu\/cds\/\">Center for Digital Scholarship<\/a>. This is an updated version for our class this year with a number of improvments.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>So you want to be a hero topic modeler? It turns out it&#8217;s not that difficult, thanks to the work those who created and maintain MALLET and the team behind\u00a0The Programming Historian&#8216;s lesson on topic modeling\u00a0(see more in credits below). The 7 Steps of Highly Effective Topic Modelers 1. Get the JDK Download and install [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":0,"parent":0,"menu_order":0,"comment_status":"closed","ping_status":"closed","template":"","meta":{"footnotes":"","_links_to":"","_links_to_target":""},"class_list":["post-232","page","type-page","status-publish","hentry"],"_links":{"self":[{"href":"https:\/\/briancroxall.net\/w18dh\/wp-json\/wp\/v2\/pages\/232","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/briancroxall.net\/w18dh\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/briancroxall.net\/w18dh\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/briancroxall.net\/w18dh\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/briancroxall.net\/w18dh\/wp-json\/wp\/v2\/comments?post=232"}],"version-history":[{"count":11,"href":"https:\/\/briancroxall.net\/w18dh\/wp-json\/wp\/v2\/pages\/232\/revisions"}],"predecessor-version":[{"id":391,"href":"https:\/\/briancroxall.net\/w18dh\/wp-json\/wp\/v2\/pages\/232\/revisions\/391"}],"wp:attachment":[{"href":"https:\/\/briancroxall.net\/w18dh\/wp-json\/wp\/v2\/media?parent=232"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}