Topic Modeling Duffy

Rationale

We have completed our reading of Carol Ann Duffy’s first five books, and we’ve each gotten a closer look at one of the volumes as we created TEI editions. I’m guessing that most of us could now begin to put together a pretty cogent argument about whether or not The World’s Wife differs from the volumes that preceded it. This argument would no doubt be based on particular poems that stuck in our memories, themes that we traced across the volumes, the sequencing of the poems, and any of the other things we discussed in our close readings. We would certainly be interested in her use of language since, after all, that is what poetry is all about (if we are being incredibly reductive).

But in our digital humanities class, we have more tools at our disposal for considering language, and one of them is topic modeling. From our reading, we know that the topics created from LDA are, as Andrew Goldstone and Ted Underwood put it, “neither more nor less than a pattern of co-occuring  words.” And from our discussions with guests, we know that topic models are, first, not necessarily designed for literary language, and, second, a radical simplification of poetry (which is not all about language). Nevertheless, in our quest to understand potential differences between The World’s Wife and Duffy’s other books of poetry, topic modeling provides us with a chance to consider patterns of linguistic affinity and difference that are not especially visible to the human eye. In short, it’s worth a shot!

Nitty Gritty

For this assignment, you will produce several topic models of Duffy’s poetry and begin the process of analyzing them. This assignment may be completed in groups or by yourself.

The assignment has two concrete outcomes: a blog post and a short reflection paper. The blog post is due at the end of the class period (1:20pm) on Wednesday, 11 April. The reflection paper is due at 5pm on Friday, 13 April.

Step One: Get the Data

Download the Duffy data from our repository in the cleaned-data folder. (If you sync your GitHub repo that you will have the files on your local hard drive.) I have removed the XML from our TEI editions and produced two different versions of the data.

  • 2018-xml-stripped.zip provides the title and text of each poem with the XML removed from it. Nothing else has been done to the data. This is, in short, plain text.
  • 2018-cleaned-simple.zip takes the content from 2018-xml-stripped and
    • lowercases the text
    • fixes curly apostrophes and single-character ellipsis to be standardized as straight apostrophes and ‘ . . . ‘, respectively
    • tokenizes the text (which, put very inaccurately is a fancy way of saying that it breaks the words apart from the punctuation so we don’t get grams like ‘cannot,’)
    • strips the punctuation from the text

Step  Two: Model the Data

Using MALLET and the Step-by-Step Guide to Topic Modeling, make at least 7 different topic models of Duffy’s poetry. Experiment with some or all of the following:

  • the two versions of the data that you use to create the MALLET corpus
  • the number of topics
  • using and not using stopwords; consider whether you need to supplement MALLET’s default stopword list (viewable in the MALLET directory > stoplists > en.txt) with an additional stopwords file.
  • the number of iterations you run
  • whether or not you use the --optimize-interval flag

Make sure that you keep careful records of the different commands you use to create each of the different models.

Step Three: Analyze the Data

After making each model, look at the topic keys to see if there seem to be too few topics or too many. Too many topics tend to be easier to spot insofar as you will have two or more topics that look relatively similar to one another. For example, if you have three “body words” topics, you should probably run the model with fewer topics. Too few topics can be spotted by there not being especially clear patterns to the topic keys.

When you have a model with topic keys that seems workable, open the topic composition file in Excel. Insert a row at the top of the spreadsheet and name the columns to make them easy to work with. See which documents (AKA poems) rank high for the various topics. Do the poems that are grouped together surprise you? Do you see any patterns based on volume the poems are published in? Is there one topic that seems to be high for a lot more poems than you would expect? (This often happens and represents what I sometimes call the “English language” topic. This becomes easier to spot if you’ve used the --optimize-interval command.)

As always, please be especially attuned to the differences that may or may not exist between Duffy’s first four volumes and The World’s Wife.

Step Four: Visualize the Data

Pick three favorite poems from Duffy’s work, using at least two of her books. Do one of the following:

  • Make a pie chart for each of the poems, showing which topics comprise the poem using Excel and the guidelines in Miriam Posner’s “Very basic strategies….”
  • Identify the three most prominent topics for each poem, according to your model. Using the topic keys, color-code the terms in the poem that come from the particular topic, similar to what Lisa M. Rhody does for Anne Sexton’s “The Starry Night” in “Some Assembly Required.” N.B. When taking this option, make sure you remove the stop words from the poem before you color-code it so that you can post it to the blog (in Step Five) and stay on the good side of copyright law.

Step Five: Blog about the Data

Write a blog post about your modeling and your data. The post is due by the beginning end of class on Wednesday, 11 April.The post should contain the following information:

  • A list (bullet points is fine) of the different models you tried. This might look something like, “cleaned-simple data, 20 topics, no stopwords, 1000 iterations, no optimization”
  • Identify which of the abovementioned models you think is best for Duffy’s poetry and discuss why you made this decision.
  • In your best model, try to identify at least two each for what Rhody, in “Some Assembly Required,” calls “semantically evident topics” and “semantically opaque topics.” For each of these four topics, write out the topic keys and explain what leads you to identify them as “evident” or “opaque” topics.
  • Write 2-3 paragraphs to report on what you observed in Step Three.
  • Include the visualizations that you produced in Step Four.

N.B. If you are working in a group, you will write this blog post together. Make sure that you put all of the group members’ names at the bottom of the post so I know who worked on it.

Step Six: Reflect on the Data

On your own (AKA not in your group), write a two- to three-page reflection paper (12pt, Times New Roman font) about what you’ve learned from topic modeling Duffy’s work. You should email the reflection to me by the end of class on Wednesday, 11 April 5pm on Friday, 13 April with the following format: lastname-topicmodel-reflection.pdf (e.g., croxall-topicmodel-reflection.pdf).

You might consider the following questions:

  • What did I learn about research from this assignment?
  • What did I learn about digital humanities from this assignment?
  • What did I learn about Duffy’s poetry from this approach (if anything) that I didn’t learn through reading the poems or making a TEI edition?

Grading

This assignment is worth 75 points.  You will be graded on both your blog post (50 points) and your reflection paper (25 points).

For the blog post, you will be graded on whether your post has all the required items; the quality of the work that you did in trying and evaluating different models; and your analysis of the models that you create.

For the reflection paper, you will be graded on your discussion of the assignment and what you learned, as well as, as always, your spelling, grammar, and punctuation.

Credits

This assignment is new this year and owes debts to Miriam Posner and Lisa Rhody for their ideas for visualizing the output of a topic model.

In the future, I look forward to thinking about how the assignment could incorporate aspects of Joanna Swafford‘s Spring 2015 topic modeling assignment, which Natalie Houston included in the “Text Analysis” keyword for Digital Pedagogy in the Humanities: Concepts, Models, and Experiments