Topic Modeling Duffy

Rationale

We have completed our reading of Carol Ann Duffy’s first five books, and we’ve gotten a closer look at various poems as we’ve marked them up with TEI. I’m guessing that most of us could now begin to put together a pretty cogent argument about whether or not The World’s Wife differs from the volumes that preceded it. This argument would no doubt be based on particular poems that stuck in our memories, themes that we traced across the volumes, the sequencing of the poems, and any of the other things we discussed in our close readings. We would certainly be interested in her use of language since, after all, that is what poetry is all about (if we are being incredibly reductive).

But in our digital humanities class, we have more tools at our disposal for considering language, and one of them is topic modeling. From our reading, we know that the topics created from LDA are, as Andrew Goldstone and Ted Underwood put it, “neither more nor less than a pattern of co-occuring words.” And from our other reading, we know that topic models are, first, not necessarily designed for literary language, and, second, a radical simplification of poetry (which is not all about language). Nevertheless, in our quest to understand potential differences between The World’s Wife and Duffy’s other books of poetry, topic modeling provides us with a chance to consider patterns of linguistic affinity and difference that are not especially visible to the human eye. In short, it’s worth a shot!

Nitty Gritty

For this assignment, you will produce several topic models of Duffy’s poetry and begin the process of analyzing them. The assignment has one outcome: a blog post. The blog post is due at the end of class on Monday, 1 April. 

Step One: Get the Data

Download the Duffy data from our repository in the cleaned-data folder. (If you sync your GitHub repo that you will have the files on your local hard drive.) I have removed the XML from our TEI editions and then combined our 60 marked-up poems with those marked-up by my 2018 class to get all 204 poems in a single data set.

  • 2019-cleaned-simple.zip takes this content and
    • lowercases the text
    • fixes curly apostrophes and single-character ellipsis to be standardized as straight apostrophes and ‘ . . . ‘, respectively
    • tokenizes the text (which, put very inaccurately is a fancy way of saying that it breaks the words apart from the punctuation so we don’t get grams like ‘cannot,’)
    • strips the punctuation from the text

Step  Two: Model the Data

Using MALLET and the Step-by-Step Guide to Topic Modeling, make at least 5 different topic models of Duffy’s poetry. Experiment with some or all of the following:

  • the number of topics
  • using and not using stopwords; consider whether you need to supplement MALLET’s default stopword list (viewable in the MALLET directory > stoplists > en.txt) with an additional stopwords file.
  • the number of iterations you run
  • whether or not you use the --optimize-interval flag

Make sure that you keep careful records of the different commands you use to create each of the different models.

Step Three: Analyze the Data

After making each model, look at the topic keys to see if there seem to be too few topics or too many. Too many topics tend to be easier to spot insofar as you will have two or more topics that look relatively similar to one another. For example, if you have three “body words” topics, you should probably run the model with fewer topics. Too few topics can be spotted by there not being especially clear patterns to the topic keys. (That said, since we are working with poetry rather than academic articles, the clarity of such “clear patterns” is not guaranteed.)

When you have a model with topic keys that seems workable, open the topic composition file in Excel. Insert a row at the top of the spreadsheet and name the columns to make them easy to work with. See which documents (AKA poems) rank high for the various topics. Do the poems that are grouped together surprise you? Do you see any patterns based on volume the poems are published in? Is there one topic that seems to be high for a lot more poems than you would expect? (This often happens and represents what I sometimes call the “English language” topic. This becomes easier to spot if you’ve used the --optimize-interval command.)

As always, please be especially attuned to the differences that may or may not exist between Duffy’s first four volumes and The World’s Wife.

Step Four: Visualize the Data

Pick three favorite poems from Duffy’s work, using at least two of her books. Do one or more of the following:

  • Make a pie chart for each of the poems, showing which topics comprise the poem using Excel and the guidelines in Miriam Posner’s “Very basic strategies for interpreting results from the Topic Modeling Tool“
  • Identify the three most prominent topics for each poem, according to your model. Using the topic keys, color-code the terms in the poem that come from the particular topic, similar to what Lisa M. Rhody does for Anne Sexton’s “The Starry Night” in “Some Assembly Required.” N.B. When taking this option, make sure you remove the stop words from the poem before you color-code it so that you can post it to the blog (in Step Five) and stay on the good side of copyright law.
  • Ask me to prep your data for network visualization in Gephi (or run mallet2gephi.py yourself from our repo).

Step Five: Blog about the Data

Write about your modeling and your data using Google Docs. When you are done, make the Doc public and post a link to it on our blog. The post is due by the end of class on Monday, 1 April. Your write-up should contain the following information:

  • A list (bullet points is fine) of the different models you tried. This might look something like, “20 topics, no stopwords, 1000 iterations, no optimization”
  • Identify which of the abovementioned models you think is best for Duffy’s poetry and discuss why you made this decision.
  • In your best model, try to identify at least two each for what Rhody, in “Some Assembly Required,” calls “semantically evident topics” and “semantically opaque topics.” For each of these four topics, write out the topic keys and explain what leads you to identify them as “evident” or “opaque” topics.
  • Write 2-3 paragraphs to report on what you observed in Step Three.
  • Include the visualizations that you produced in Step Four.
  • Write a paragraph that discusses what you learned from this approach that you didn’t learn through reading the poems or marking the poems up.

Grading

This assignment is worth 50 points. You will be graded on whether your post has all the required items; the quality of the work that you did in trying and evaluating different models; and your analysis of the models that you create.

Credits

This assignment was designed in 2018 and owes debts to Miriam Posner and Lisa Rhody for their ideas for visualizing the output of a topic model. In 2019, I simplified portions of the assignment based on what I learned the previous year.