{"id":88440,"date":"2015-10-07T14:55:19","date_gmt":"2015-10-07T19:55:19","guid":{"rendered":"https:\/\/beta.pewresearch.org\/pewresearch-org\/2015\/10\/07\/news-coverage-of-the-papal-visit-methods\/"},"modified":"2024-04-14T04:15:19","modified_gmt":"2024-04-14T09:15:19","slug":"news-coverage-of-the-papal-visit-methods","status":"publish","type":"post","link":"https:\/\/beta.pewresearch.org\/pewresearch-org\/journalism\/2015\/10\/07\/news-coverage-of-the-papal-visit-methods\/","title":{"rendered":"Methodology"},"content":{"rendered":"
\n
\n
\n
\n
\n
\n

This report examined the use of sources and social media in online news coverage of Pope Francis\u2019 visit to the United States. The study focused on coverage from 9 a.m. Eastern Standard Time on Sept. 22, 2015, (the day the pope arrived in the United States), until 9 a.m. on Sept. 26, 2015 (the day he left New York City), a time period that allowed researchers to study the majority of the visit coverage. Coding and data analysis were conducted by researchers Mike Barthel, Jeffrey Gottfried, Katerina Eva Matsa, Kristine Lu, Elisa Shearer, Galen Stocking and Nancy Vogt.<\/p>\n

Sample Design<\/h4>\n

Twelve news websites were studied. As of August 2015, these sites are all included in the top 25 entities of comScore\u2019s news and information category. Researchers pulled the initial list of the top 25 digital properties and then removed those that are entirely devoted to weather (for example, weather.com), those which do not regularly produce original reporting around daily news (for example, about.com), and those that are non-U.S. (such as BBC and the Guardian). For listings that appear at a company level but have several distinct news entities such as Gannett, we \u201cunrolled\u201d the data to traffic levels of each individual news site (i.e. USA Today). From those, the 12 sites with the highest number of unique visitors for the month of August were selected for this study, ranging from newspapers to online-only outlets to broadcast TV.<\/p>\n

The following are the news outlets included in this analysis:<\/p>\n

Yahoo News\nABC News\nThe Huffington Post\nCNN\nUSA Today\nNBC News\nBuzzfeed\nCBS News\nThe New York Times\nFox News\nThe Washington Post\nThe Los Angeles Times<\/em><\/p>\n

Note: The Washington Post, with both national and local elements in their coverage, produced substantially more stories than other outlets: 148 in total. The Huffington Post also produced more stories than most other outlets: 74 in total. The other outlets produced an average of 29 stories.<\/p>\n

Story Selection<\/h4>\n

The unit of analysis in this study was the story. Stories were included in the study only if 50% or more of their content was about the pope\u2019s visit to the United States (a small number of stories also addressed the pope\u2019s visit to Cuba, which preceded his arrival to the United States). To find the relevant stories in all websites, researchers used a specific set of keyword searches to collect potential stories.<\/p>\n

Researchers searched Google News for each news outlet\u2019s main domain with the following terms to find potential stories:<\/p>\n

Pope OR Francis OR Bergoglio OR \u201cHoly Father\u201d OR Papal<\/p>\n

Every effort was made to collect the entirety of the content from each provider in its primary form.<\/p>\n

Researchers manually captured and saved digital versions of the websites each morning between 9 a.m. and 12 p.m. EST. In some cases where sites produced a large number of stories, content capture for a given day occurred over a longer period of time. Duplicate stories were removed, as well as stories that were not in English. Also excluded were stories that appeared in a different domain or URL. For example, researchers coded all related stories found in foxnews.com, but did not code those that appeared under foxbusiness.com.<\/p>\n

Researchers then read each story to see if 50% or more of the story\u2019s content was about the pope\u2019s visit. The study did not include articles that were entirely standalone video, audio, slideshow, straight transcripts or quizzes that did not offer any content by the author of the piece. (Video, audio, transcript or slideshows embedded in a story were<\/em> included in the study.)<\/p>\n

Human Coding of Stories<\/h4>\n

The data in this study were created by a team of six experienced coders who were trained specifically for this project.<\/p>\n

The two central variables in this study were source type and social media reference. Source type refers to a person, group, organization or publication cited in a story. Citations included direct or indirect quotes, interviews or labels, or references accompanying factual information. The goal was to identify the different types of sources that were present in the stories and not to provide a count of the total number of sources. With this in mind, researchers coded for the presence of 17 different source types. Within each story, a source type was coded only once, even if it was cited repeatedly. A source type could be present both as a social media reference and a non-social media reference and therefore both cases were captured as one source type from two different platforms. The 17 different source types were:<\/p>\n

    \n
  • The Vatican\/Pope Francis<\/li>\n
  • Clergy\/Religious group (non-Vatican)<\/li>\n
  • Issue-based and\/or interest groups (non-religion)<\/li>\n
  • Quotes from religious texts<\/li>\n
  • Experts, non-clergy<\/li>\n
  • Polls<\/li>\n
  • State\/local official<\/li>\n
  • Obama\/Obama administration<\/li>\n
  • Presidential candidate<\/li>\n
  • Congress as institution or member of the Congress (D & R)<\/li>\n
  • Political & government figure not in office (GOP, Democratic or other partisan)<\/li>\n
  • Member of the public<\/li>\n
  • Other media (includes social media companies, as distinguished from users)<\/li>\n
  • Celebrity<\/li>\n
  • International political & government figures<\/li>\n
  • Business<\/li>\n
  • Other<\/li>\n<\/ul>\n

    The social media reference variable indicates whether the source was taken from a social networking site. The social networking sites tracked in this study were: Twitter, Facebook, Instagram and YouTube, with all others coded as \u201cother social networking site.\u201d<\/p>\n

    Coders were given two sets of news stories to evaluate during the training period. Once internal agreement on how to code the variables was established, a first round of five stories were coded by all members of the coding team. The percent of agreement on each variable was:<\/p>\n

    Presence of sources cited \u2014\u00a089% or higher<\/p>\n

    Social Networking Site \u2014 92% or higher<\/p>\n

    Finally, a second round of 25 stories\u00a0were coded\u00a0at the end of the study\u00a0by all members of the coding team.\u00a0The percent of agreement on each variable was:<\/p>\n

    Presence of sources cited \u2014 82% or higher<\/p>\n

    Social Networking Site \u2014 96% or higher<\/p>\n

    Throughout the coding process, staff discussed questions as they arose and arrived at decisions under supervision of the content analysis team leader.<\/p>\n

    \u00a0<\/p>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>","protected":false},"excerpt":{"rendered":"

    This report examined the use of sources and social media in online news coverage of Pope Francis\u2019 visit to the United States. The study focused on coverage from 9 a.m. Eastern Standard Time on Sept. 22, 2015, (the day the pope arrived in the United States), until 9 a.m. on Sept. 26, 2015 (the day […]<\/p>\n","protected":false},"author":43,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"sub_headline":"","sub_title":"","_prc_public_revisions":[],"_ppp_expiration_hours":0,"_ppp_enabled":false,"ai_generated_summary":"","_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"apple_news_api_id":"","apple_news_api_revision":"","apple_news_api_created_at":"","apple_news_api_modified_at":"","apple_news_api_share_url":"","apple_news_api_pending":"","apple_news_is_preview":false,"apple_news_is_hidden":false,"relatedPosts":[],"reportMaterials":[],"multiSectionReport":[],"package_parts__enabled":false,"package_parts":[],"datacite_doi":"","datacite_doi_citation":"","_prc_seo_qr_attachment_id":0,"spoken_article_player_enabled":true,"displayBylines":true,"footnotes":"","prc_watchers":[],"_prc_fork_parent":0,"_prc_fork_status":"","_prc_active_fork":0,"jetpack_post_was_ever_published":false},"categories":[],"tags":[],"bylines":[987,855,983],"collection":[],"datasets":[],"level_of_effort":[],"primary_audience":[],"information_type":[],"_post_visibility":[],"formats":[458],"_fund_pool":[],"languages":[],"regions-countries":[],"research-teams":[527],"workflow-status":[],"class_list":["post-88440","post","type-post","status-publish","format-standard","hentry","bylines-amy-mitchell","bylines-jesse-holcomb","bylines-katerina-eva-matsa","formats-report","research-teams-journalism"],"label":false,"post_parent":88453,"word_count":985,"canonical_url":"https:\/\/beta.pewresearch.org\/pewresearch-org\/journalism\/2015\/10\/07\/news-coverage-of-the-papal-visit-methods\/","art_direction":{"A1":{"id":110526,"rawUrl":"https:\/\/beta.pewresearch.org\/pewresearch-org\/wp-content\/uploads\/sites\/20\/2015\/10\/PJ_2015-10-07_pope_featured.png","url":"https:\/\/beta.pewresearch.org\/pewresearch-org\/wp-content\/uploads\/sites\/20\/2015\/10\/PJ_2015-10-07_pope_featured.png?w=564&h=317&crop=1","width":564,"height":317,"chartArt":false},"A2":{"id":110526,"rawUrl":"https:\/\/beta.pewresearch.org\/pewresearch-org\/wp-content\/uploads\/sites\/20\/2015\/10\/PJ_2015-10-07_pope_featured.png","url":"https:\/\/beta.pewresearch.org\/pewresearch-org\/wp-content\/uploads\/sites\/20\/2015\/10\/PJ_2015-10-07_pope_featured.png?w=268&h=151&crop=1","width":268,"height":151,"chartArt":false},"A3":{"id":110526,"rawUrl":"https:\/\/beta.pewresearch.org\/pewresearch-org\/wp-content\/uploads\/sites\/20\/2015\/10\/PJ_2015-10-07_pope_featured.png","url":"https:\/\/beta.pewresearch.org\/pewresearch-org\/wp-content\/uploads\/sites\/20\/2015\/10\/PJ_2015-10-07_pope_featured.png?w=194&h=110&crop=1","width":194,"height":110,"chartArt":false},"A4":{"id":110526,"rawUrl":"https:\/\/beta.pewresearch.org\/pewresearch-org\/wp-content\/uploads\/sites\/20\/2015\/10\/PJ_2015-10-07_pope_featured.png","url":"https:\/\/beta.pewresearch.org\/pewresearch-org\/wp-content\/uploads\/sites\/20\/2015\/10\/PJ_2015-10-07_pope_featured.png?w=268&h=151&crop=1","width":268,"height":151,"chartArt":false},"XL":{"id":110526,"rawUrl":"https:\/\/beta.pewresearch.org\/pewresearch-org\/wp-content\/uploads\/sites\/20\/2015\/10\/PJ_2015-10-07_pope_featured.png","url":"https:\/\/beta.pewresearch.org\/pewresearch-org\/wp-content\/uploads\/sites\/20\/2015\/10\/PJ_2015-10-07_pope_featured.png?w=640&h=320&crop=1","width":640,"height":320,"chartArt":false},"social":{"id":110526,"rawUrl":"https:\/\/beta.pewresearch.org\/pewresearch-org\/wp-content\/uploads\/sites\/20\/2015\/10\/PJ_2015-10-07_pope_featured.png","url":"https:\/\/beta.pewresearch.org\/pewresearch-org\/wp-content\/uploads\/sites\/20\/2015\/10\/PJ_2015-10-07_pope_featured.png?w=640&h=320&crop=1","width":640,"height":320,"chartArt":false}},"_embeds":[],"watchers":[],"table_of_contents":[{"id":88453,"title":"News Coverage of the Papal Visit: Narrative Made Room for the Personal, Not Just the Political","slug":"news-coverage-of-the-papal-visit","link":"https:\/\/beta.pewresearch.org\/pewresearch-org\/journalism\/2015\/10\/07\/news-coverage-of-the-papal-visit\/","is_active":false},{"id":88440,"title":"Methodology","slug":"news-coverage-of-the-papal-visit-methods","link":"https:\/\/beta.pewresearch.org\/pewresearch-org\/journalism\/2015\/10\/07\/news-coverage-of-the-papal-visit-methods\/","is_active":true}],"report_materials":[{"key":"345c6eb5-2a56-41d6-85fa-ef9a8cc19a29","type":"report","url":"https:\/\/beta.pewresearch.org\/pewresearch-org\/wp-content\/uploads\/sites\/8\/2015\/10\/PJ_2015-10-07_pope-visit_FINAL.pdf","label":"","icon":"","attachmentId":""}],"report_pagination":{"current_post":{"id":88440,"title":"Methodology","slug":"news-coverage-of-the-papal-visit-methods","link":"https:\/\/beta.pewresearch.org\/pewresearch-org\/journalism\/2015\/10\/07\/news-coverage-of-the-papal-visit-methods\/","is_active":true,"page_num":2},"next_post":null,"previous_post":{"id":88453,"title":"News Coverage of the Papal Visit: Narrative Made Room for the Personal, Not Just the Political","slug":"news-coverage-of-the-papal-visit","link":"https:\/\/beta.pewresearch.org\/pewresearch-org\/journalism\/2015\/10\/07\/news-coverage-of-the-papal-visit\/","is_active":false,"page_num":1},"pagination_items":[{"id":88453,"title":"News Coverage of the Papal Visit: Narrative Made Room for the Personal, Not Just the Political","slug":"news-coverage-of-the-papal-visit","link":"https:\/\/beta.pewresearch.org\/pewresearch-org\/journalism\/2015\/10\/07\/news-coverage-of-the-papal-visit\/","is_active":false,"page_num":1},{"id":88440,"title":"Methodology","slug":"news-coverage-of-the-papal-visit-methods","link":"https:\/\/beta.pewresearch.org\/pewresearch-org\/journalism\/2015\/10\/07\/news-coverage-of-the-papal-visit-methods\/","is_active":true,"page_num":2}]},"parent_info":{"parent_title":"News Coverage of the Papal Visit: Narrative Made Room for the Personal, Not Just the Political","parent_id":88453},"materialsOrdered":[],"chaptersOrdered":[],"partsOrdered":[],"partsEnabled":false,"datacite_doi":"","prc_seo_data":{"title":"Methodology","description":"This report examined the use of sources and social media in online news coverage of Pope Francis\u2019 visit to the United States. The study focused on coverage from 9 a.m.…","og_title":"Methodology","og_description":"","schema_type":"Article","noindex":false,"canonical_url":"","primary_terms":[],"custom_schema":[],"og_image":110526,"indexnow_submitted_at":null,"gsc_index_status":null},"prepublish_checks":{"prc-image-alt-text":{"status":"complete","message":"No image blocks in content.","data":null},"prc-about-this-research":{"status":"incomplete","message":"Add an \"About this research\" details block.","data":null},"prc-paragraph-count":{"status":"incomplete","message":"Add at least 2 paragraphs (currently 0).","data":{"count":0}},"prc-internal-link":{"status":"incomplete","message":"Add at least one internal link.","data":{"count":0}}},"jetpack_featured_media_url":"","jetpack_sharing_enabled":true,"relatedPostsOrdered":[],"bylinesOrdered":[{"key":"2d643eda-052b-4905-892d-72838604e1c0","termId":983},{"key":"4eab68da-34c5-426f-82ed-f0f863f2f8d9","termId":855},{"key":"2c5c3040-dcb4-4458-97b5-e6df173aa8a5","termId":987}],"acknowledgementsOrdered":[],"_links":{"self":[{"href":"https:\/\/beta.pewresearch.org\/pewresearch-org\/wp-json\/wp\/v2\/posts\/88440","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/beta.pewresearch.org\/pewresearch-org\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/beta.pewresearch.org\/pewresearch-org\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/beta.pewresearch.org\/pewresearch-org\/wp-json\/wp\/v2\/users\/43"}],"replies":[{"embeddable":true,"href":"https:\/\/beta.pewresearch.org\/pewresearch-org\/wp-json\/wp\/v2\/comments?post=88440"}],"version-history":[{"count":2,"href":"https:\/\/beta.pewresearch.org\/pewresearch-org\/wp-json\/wp\/v2\/posts\/88440\/revisions"}],"predecessor-version":[{"id":135061,"href":"https:\/\/beta.pewresearch.org\/pewresearch-org\/wp-json\/wp\/v2\/posts\/88440\/revisions\/135061"}],"wp:attachment":[{"href":"https:\/\/beta.pewresearch.org\/pewresearch-org\/wp-json\/wp\/v2\/media?parent=88440"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/beta.pewresearch.org\/pewresearch-org\/wp-json\/wp\/v2\/categories?post=88440"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/beta.pewresearch.org\/pewresearch-org\/wp-json\/wp\/v2\/tags?post=88440"},{"taxonomy":"bylines","embeddable":true,"href":"https:\/\/beta.pewresearch.org\/pewresearch-org\/wp-json\/wp\/v2\/bylines?post=88440"},{"taxonomy":"collection","embeddable":true,"href":"https:\/\/beta.pewresearch.org\/pewresearch-org\/wp-json\/wp\/v2\/collection?post=88440"},{"taxonomy":"datasets","embeddable":true,"href":"https:\/\/beta.pewresearch.org\/pewresearch-org\/wp-json\/wp\/v2\/datasets?post=88440"},{"taxonomy":"level_of_effort","embeddable":true,"href":"https:\/\/beta.pewresearch.org\/pewresearch-org\/wp-json\/wp\/v2\/level_of_effort?post=88440"},{"taxonomy":"primary_audience","embeddable":true,"href":"https:\/\/beta.pewresearch.org\/pewresearch-org\/wp-json\/wp\/v2\/primary_audience?post=88440"},{"taxonomy":"information_type","embeddable":true,"href":"https:\/\/beta.pewresearch.org\/pewresearch-org\/wp-json\/wp\/v2\/information_type?post=88440"},{"taxonomy":"_post_visibility","embeddable":true,"href":"https:\/\/beta.pewresearch.org\/pewresearch-org\/wp-json\/wp\/v2\/_post_visibility?post=88440"},{"taxonomy":"formats","embeddable":true,"href":"https:\/\/beta.pewresearch.org\/pewresearch-org\/wp-json\/wp\/v2\/formats?post=88440"},{"taxonomy":"_fund_pool","embeddable":true,"href":"https:\/\/beta.pewresearch.org\/pewresearch-org\/wp-json\/wp\/v2\/_fund_pool?post=88440"},{"taxonomy":"languages","embeddable":true,"href":"https:\/\/beta.pewresearch.org\/pewresearch-org\/wp-json\/wp\/v2\/languages?post=88440"},{"taxonomy":"regions-countries","embeddable":true,"href":"https:\/\/beta.pewresearch.org\/pewresearch-org\/wp-json\/wp\/v2\/regions-countries?post=88440"},{"taxonomy":"research-teams","embeddable":true,"href":"https:\/\/beta.pewresearch.org\/pewresearch-org\/wp-json\/wp\/v2\/research-teams?post=88440"},{"taxonomy":"workflow-status","embeddable":true,"href":"https:\/\/beta.pewresearch.org\/pewresearch-org\/wp-json\/wp\/v2\/workflow-status?post=88440"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}