{"id":111700,"date":"2018-07-26T16:00:00","date_gmt":"2018-07-26T21:00:00","guid":{"rendered":"https:\/\/beta.pewresearch.org\/pewresearch-org\/decoded\/\/\/detecting-subjectivity-and-tone-with-automated-text-analysis-tools\/"},"modified":"2024-04-14T04:10:45","modified_gmt":"2024-04-14T09:10:45","slug":"detecting-subjectivity-and-tone-with-automated-text-analysis-tools","status":"publish","type":"decoded","link":"https:\/\/beta.pewresearch.org\/pewresearch-org\/decoded\/2018\/07\/26\/detecting-subjectivity-and-tone-with-automated-text-analysis-tools\/","title":{"rendered":"Detecting subjectivity and tone with automated text analysis tools"},"content":{"rendered":"\n
\"\"<\/a>
(Photo illustration by Pew Research Center and Petr Vaclavek\/\/Shutterstock.com)<\/figcaption><\/figure>\n\n\n\n

There are a variety of tools that can help researchers analyze large volumes of written material. In this post, I\u2019ll examine two of these tools: part-of-speech tagging and tone analysis. I\u2019ll also show how to use these methods to find patterns in a large set of Facebook posts created by members of Congress.<\/p>\n\n\n\n

Part-of-speech (POS) tagging is a process that labels each word in a sentence with an algorithm\u2019s best guess for the word\u2019s part of speech (for example, noun, adjective or verb). This is based on both the definition of each word and the context in which it appears. There are several libraries in Python that perform POS tagging, such as NLTK<\/a>, SpaCy<\/a> and TextBlob<\/a>. These tagging systems are pre-trained using supervised machine learning models, based on a training dataset with labeled text. As a result, the POS tags returned by these models are estimates based on the data that each particular model was trained with.<\/p>\n\n\n\n

One reason researchers might care about parts of speech is that they can help reveal attributes of written text. For example, research has shown the presence of adjectives and adverbs is usually a good indicator of text subjectivity<\/a>. In other words, statements that use adjectives like \u201cinteresting,\u201d \u201cproblematic\u201d and \u201cawesome\u201d might be more likely to convey a subjective point of view than statements that do not include those adjectives. Adjectives and adverbs might also be especially likely to convey tone: Some indicate more positive qualities, while others indicate negative qualities.<\/p>\n\n\n\n

To identify adjectives using a computational model, I\u2019ll use a set of predefined part of speech tags. The set of possible tags comes from the Penn Treebank tagset<\/a>. If the returned tag is either JJ (adjective), JJR (adjective, comparative) or JJS (adjective, superlative), I\u2019ll treat the word as an adjective. Similarly, if a word has the POS tag of RB (adverb), RBR (adverb, comparative) or RBS (adverb, superlative), I\u2019ll consider it an adverb.<\/p>\n\n\n\n

The code below is a simple function in Python that returns a list of all adjectives and adverbs, and the total number of adjectives and adverbs for a given text using the TextBlob<\/a> library:<\/p>\n\n\n\n

from textblob import TextBlob\nfrom collections import Counterdef textblob_adj(text):\n    blobed = TextBlob(text)\n    counts = Counter(tag for word,tag in blobed.tags)\n    adj_list = []\n    adv_list = []\n    adj_tag_list = [\u2018JJ\u2019,\u2019JJR\u2019,\u2019JJS\u2019]\n    adv_tag_list = [\u2018RB\u2019,\u2019RBR\u2019,\u2019RBS\u2019]\n    for (a, b) in blobed.tags:\n        if b in adj_tag_list:\n           adj_list.append(a)\n        elif b in adv_tag_list:\n           adv_list.append(a)\n        else:\n            pass\n    return adj_list, adv_list, counts[\u2018JJ\u2019]+counts[\u2018JJR\u2019]+counts[\u2018JJS\u2019], counts[\u2018RB\u2019]+counts[\u2018RBR\u2019]+counts[\u2018RBS\u2019]<\/code><\/pre>\n\n\n\n
To show how part-of-speech recognition works in practice, I analyzed over 700,000 Facebook posts created between Jan. 1, 2015, and Dec. 30, 2017, by members of the U.S. Congress, collected using Facebook\u2019s public API. (This is part of a larger body of research<\/a> that Pew Research Center has conducted about lawmakers\u2019 Facebook posts.) My goal was to understand how adjective and adverb use in partisan speech changed across this period of time. The analysis shows that after President Donald Trump took office, Democrats in Congress started using more adjectives than Republicans in their Facebook posts. Democrats\u2019 ratio of positive to negative adjectives also decreased after Trump took office.<\/p>\n\n\n\n
I applied the function above to each Facebook post. Some of the adjectives that were used mostly by Democrats in Congress include \u201cwealthiest\u201d, \u201crichest\u201d and \u201creproductive,\u201d whereas \u201cunborn,\u201d \u201clibertarian\u201d and \u201cIslamist\u201d are among the adjectives used mostly by Republicans. The adjective ratio \u2014 defined as the number of adjectives divided by the total number of words in the post \u2014 was slightly higher for Republicans than Democrats in the early part of 2015. It was very similar for the two parties across most of 2016. But starting with Trump\u2019s 2016 election victory, Democrats became much more likely to use adjectives in their posts. This finding aligns with a previous Pew Research Center analysis<\/a> that shows that congressional Democrats expressed political opposition more frequently under Trump. It is also consistent with the hypothesis that adjective use is linked with stronger emotions.<\/p>\n\n\n\n
\"\"<\/a><\/figure>\n\n\n\n
I also examined the tone of particular adjectives by comparing the words that appeared in the congressional posts with a list of adjectives that usually have a positive or negative association. This association is based on a polarity dictionary: a list of positive and negative words. If an adjective appears in the list of positive or negative words in the dictionary, it is classified as such. The plot produced here uses the Harvard IV-4 dictionary<\/a>, which contains 1,915 positive and 2,291 negative words (the list is also used in the academic community<\/a>).<\/p>\n\n\n\n
Using that measure, the ratio of positive to negative adjectives declined in congressional Democrats\u2019 Facebook posts after Trump was inaugurated, but it increased slightly in Republicans\u2019 posts (I added 1 to the denominator to avoid dividing by 0). For much of the first half of 2015, Republicans had a lower positive-to-negative adjective ratio, but the partisan difference was absent across most of 2016.<\/p>\n\n\n\n
Adjective analysis using off-the-shelf packages and dictionaries can provide an accessible and inexpensive way to explore sentiment and subjectivity in texts, without the need to obtain a large labeled training set. However, researchers should be cautious when drawing inferences from the tools described here. There\u2019s no guarantee that adjectives provide a valid measure of important concepts like subjectivity across diverse contexts, and the link between particular adjectives and sentiment may not generalize across different domains. It\u2019s always necessary to validate a given model \u2014 either by using human coders, or by supplementing the analysis with other measures of subjectivity \u2014 before drawing strong conclusions about what text tells us.<\/p>\n","protected":false},"excerpt":{"rendered":"
There are a variety of tools that can help researchers analyze large volumes of written material. In this post, I\u2019ll examine two of these\u2026<\/p>\n","protected":false},"author":655,"featured_media":0,"comment_status":"closed","ping_status":"closed","template":"","meta":{"sub_headline":"","sub_title":"","_prc_public_revisions":[],"_ppp_expiration_hours":0,"_ppp_enabled":false,"ai_generated_summary":"","relatedPosts":[],"datacite_doi":"","datacite_doi_citation":"","_prc_seo_qr_attachment_id":0,"spoken_article_player_enabled":true,"displayBylines":true,"footnotes":"","prc_watchers":[],"_prc_fork_parent":0,"_prc_fork_status":"","_prc_active_fork":0},"categories":[353],"bylines":[819],"collection":[],"_post_visibility":[],"decoded-category":[530],"formats":[],"_fund_pool":[],"languages":[],"regions-countries":[],"research-teams":[524],"workflow-status":[],"class_list":["post-111700","decoded","type-decoded","status-publish","hentry","category-data-science","bylines-onyi-lam","decoded-category-data-science","research-teams-decoded"],"label":"Decoded","post_parent":0,"word_count":969,"canonical_url":"https:\/\/beta.pewresearch.org\/pewresearch-org\/decoded\/2018\/07\/26\/detecting-subjectivity-and-tone-with-automated-text-analysis-tools\/","art_direction":{"A1":{"id":126053,"rawUrl":"https:\/\/beta.pewresearch.org\/pewresearch-org\/wp-content\/uploads\/sites\/20\/2018\/07\/07.28.2016_featured.png","url":"https:\/\/beta.pewresearch.org\/pewresearch-org\/wp-content\/uploads\/sites\/20\/2018\/07\/07.28.2016_featured.png?w=564&h=317&crop=1","width":564,"height":317,"caption":"","chartArt":false},"A2":{"id":126053,"rawUrl":"https:\/\/beta.pewresearch.org\/pewresearch-org\/wp-content\/uploads\/sites\/20\/2018\/07\/07.28.2016_featured.png","url":"https:\/\/beta.pewresearch.org\/pewresearch-org\/wp-content\/uploads\/sites\/20\/2018\/07\/07.28.2016_featured.png?w=268&h=151&crop=1","width":268,"height":151,"caption":"","chartArt":false},"A3":{"id":126053,"rawUrl":"https:\/\/beta.pewresearch.org\/pewresearch-org\/wp-content\/uploads\/sites\/20\/2018\/07\/07.28.2016_featured.png","url":"https:\/\/beta.pewresearch.org\/pewresearch-org\/wp-content\/uploads\/sites\/20\/2018\/07\/07.28.2016_featured.png?w=194&h=110&crop=1","width":194,"height":110,"caption":"","chartArt":false},"A4":{"id":126053,"rawUrl":"https:\/\/beta.pewresearch.org\/pewresearch-org\/wp-content\/uploads\/sites\/20\/2018\/07\/07.28.2016_featured.png","url":"https:\/\/beta.pewresearch.org\/pewresearch-org\/wp-content\/uploads\/sites\/20\/2018\/07\/07.28.2016_featured.png?w=268&h=151&crop=1","width":268,"height":151,"caption":"","chartArt":false},"XL":{"id":126053,"rawUrl":"https:\/\/beta.pewresearch.org\/pewresearch-org\/wp-content\/uploads\/sites\/20\/2018\/07\/07.28.2016_featured.png","url":"https:\/\/beta.pewresearch.org\/pewresearch-org\/wp-content\/uploads\/sites\/20\/2018\/07\/07.28.2016_featured.png?w=640&h=360&crop=1","width":640,"height":360,"caption":"","chartArt":false},"social":{"id":126053,"rawUrl":"https:\/\/beta.pewresearch.org\/pewresearch-org\/wp-content\/uploads\/sites\/20\/2018\/07\/07.28.2016_featured.png","url":"https:\/\/beta.pewresearch.org\/pewresearch-org\/wp-content\/uploads\/sites\/20\/2018\/07\/07.28.2016_featured.png?w=640&h=360&crop=1","width":640,"height":360,"caption":"","chartArt":false}},"_embeds":[],"watchers":[],"table_of_contents":[],"datacite_doi":"","prc_seo_data":{"title":"Detecting subjectivity and tone with automated text analysis tools","description":"There are a variety of tools that can help researchers analyze large volumes of written material. In this post, I\u2019ll examine two of these\u2026","og_title":"Detecting subjectivity and tone with automated text analysis tools","og_description":"There are a variety of tools that can help researchers analyze large volumes of written material. In this post, I\u2019ll examine two of these\u2026","schema_type":"Article","noindex":false,"canonical_url":"","primary_terms":{"category":1},"custom_schema":[],"og_image":126053,"indexnow_submitted_at":null,"gsc_index_status":null},"prepublish_checks":{},"jetpack_sharing_enabled":true,"relatedPostsOrdered":[],"bylinesOrdered":[{"key":"_y6g2h66de","termId":819}],"acknowledgementsOrdered":[],"_links":{"self":[{"href":"https:\/\/beta.pewresearch.org\/pewresearch-org\/wp-json\/wp\/v2\/decoded\/111700","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/beta.pewresearch.org\/pewresearch-org\/wp-json\/wp\/v2\/decoded"}],"about":[{"href":"https:\/\/beta.pewresearch.org\/pewresearch-org\/wp-json\/wp\/v2\/types\/decoded"}],"author":[{"embeddable":true,"href":"https:\/\/beta.pewresearch.org\/pewresearch-org\/wp-json\/wp\/v2\/users\/655"}],"replies":[{"embeddable":true,"href":"https:\/\/beta.pewresearch.org\/pewresearch-org\/wp-json\/wp\/v2\/comments?post=111700"}],"version-history":[{"count":2,"href":"https:\/\/beta.pewresearch.org\/pewresearch-org\/wp-json\/wp\/v2\/decoded\/111700\/revisions"}],"predecessor-version":[{"id":138552,"href":"https:\/\/beta.pewresearch.org\/pewresearch-org\/wp-json\/wp\/v2\/decoded\/111700\/revisions\/138552"}],"wp:attachment":[{"href":"https:\/\/beta.pewresearch.org\/pewresearch-org\/wp-json\/wp\/v2\/media?parent=111700"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/beta.pewresearch.org\/pewresearch-org\/wp-json\/wp\/v2\/categories?post=111700"},{"taxonomy":"bylines","embeddable":true,"href":"https:\/\/beta.pewresearch.org\/pewresearch-org\/wp-json\/wp\/v2\/bylines?post=111700"},{"taxonomy":"collection","embeddable":true,"href":"https:\/\/beta.pewresearch.org\/pewresearch-org\/wp-json\/wp\/v2\/collection?post=111700"},{"taxonomy":"_post_visibility","embeddable":true,"href":"https:\/\/beta.pewresearch.org\/pewresearch-org\/wp-json\/wp\/v2\/_post_visibility?post=111700"},{"taxonomy":"decoded-category","embeddable":true,"href":"https:\/\/beta.pewresearch.org\/pewresearch-org\/wp-json\/wp\/v2\/decoded-category?post=111700"},{"taxonomy":"formats","embeddable":true,"href":"https:\/\/beta.pewresearch.org\/pewresearch-org\/wp-json\/wp\/v2\/formats?post=111700"},{"taxonomy":"_fund_pool","embeddable":true,"href":"https:\/\/beta.pewresearch.org\/pewresearch-org\/wp-json\/wp\/v2\/_fund_pool?post=111700"},{"taxonomy":"languages","embeddable":true,"href":"https:\/\/beta.pewresearch.org\/pewresearch-org\/wp-json\/wp\/v2\/languages?post=111700"},{"taxonomy":"regions-countries","embeddable":true,"href":"https:\/\/beta.pewresearch.org\/pewresearch-org\/wp-json\/wp\/v2\/regions-countries?post=111700"},{"taxonomy":"research-teams","embeddable":true,"href":"https:\/\/beta.pewresearch.org\/pewresearch-org\/wp-json\/wp\/v2\/research-teams?post=111700"},{"taxonomy":"workflow-status","embeddable":true,"href":"https:\/\/beta.pewresearch.org\/pewresearch-org\/wp-json\/wp\/v2\/workflow-status?post=111700"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}