{"id":5425,"date":"2019-09-05T09:33:06","date_gmt":"2019-09-05T14:33:06","guid":{"rendered":"https:\/\/beta.pewresearch.org\/pewresearch-org\/?p=5425"},"modified":"2025-09-02T17:03:56","modified_gmt":"2025-09-02T21:03:56","slug":"how-does-a-computer-see-gender","status":"publish","type":"feature","link":"https:\/\/beta.pewresearch.org\/pewresearch-org\/data-labs\/feature\/how-does-a-computer-see-gender\/","title":{"rendered":"How does a computer \u2018see\u2019 gender?"},"content":{"rendered":"\n<div class=\"wp-block-cover alignfull is-light\" style=\"min-height:483px;aspect-ratio:unset;\"><img decoding=\"async\" class=\"wp-block-cover__image-background wp-image-370374\" alt=\"\" src=\"https:\/\/beta.pewresearch.org\/pewresearch-org\/wp-content\/uploads\/2019\/09\/PL_19.08_machine_vision_interactive_featured_640-16.png\" data-object-fit=\"cover\"\/><span aria-hidden=\"true\" class=\"wp-block-cover__background has-gray-light-background-color has-background-dim-10 has-background-dim\"><\/span><div class=\"wp-block-cover__inner-container is-layout-flow wp-block-cover-is-layout-flow\"><h1 aria-level=\"1\" data-post-type=\"feature\" style=\"font-size:clamp(27.894px, 1.743rem + ((1vw - 3.2px) * 2.285), 48px);line-height:1.1\" class=\"has-text-align-center is-style-essay-title wp-block-post-title has-text-color has-white-color\">How does a computer \u2018see\u2019 gender?<\/h1>\n\n\n<div class=\"wp-block-group is-style-640-wide has-large-font-size is-layout-flow wp-block-group-is-layout-flow\">\n<h5 class=\"wp-block-heading has-text-align-center is-style-default has-white-color has-text-color has-h-one-font-size\" id=\"a-computer-can-be-trained-to-predict-whether-an-image-shows-a-man-or-a-woman-can-you-identify-which-parts-of-the-face-are-most-essential-to-the-computer-s-decision\">A computer can be trained to predict whether an image shows a man or a woman. Can you identify which parts of the face are most essential to the computer\u2019s decision?<\/h5>\n<\/div>\n<\/div><\/div>\n\n\n\n<div class=\"wp-block-group alignwide has-sans-serif-font-family has-small-font-size has-global-padding is-layout-constrained wp-container-core-group-is-layout-61d584a3 wp-block-group-is-layout-constrained\"><\/div>\n\n\n<div style=\"font-weight:600;letter-spacing:0px\" class=\"wp-block-prc-block-bylines-display has-small-label-font-size is-layout-flex wp-container-prc-block-bylines-display-is-layout-7b1574cb wp-block-prc-block-bylines-display-is-layout-flex\" class=\"wp-block-prc-block-bylines-display__bylines\"><span class=\"wp-block-prc-block-bylines-display__prefix\">By<\/span> <a rel=\"author\" href=\"https:\/\/beta.pewresearch.org\/pewresearch-org\/staff\/stefan-wojcik\/\" aria-label=\"View author archive for Stefan Wojcik\">Stefan Wojcik<\/a> <span class=\"prc-platform-staff-bylines__and-separator\">and<\/span> <a rel=\"author\" href=\"https:\/\/beta.pewresearch.org\/pewresearch-org\/staff\/chris-baronavski\/\" aria-label=\"View author archive for Chris Baronavski\">Chris Baronavski<\/a><\/div>\n\n\n<div class=\"wp-block-group essay--toolbar has-white-background-color has-ui-white-background-color has-background is-layout-flow wp-block-group-is-layout-flow\"><div style=\"letter-spacing:1px\" class=\"wp-block-post-date has-text-color has-gray-medium-color has-small-label-font-size\"><time datetime=\"2019-09-05T09:33:06-04:00\">September 5, 2019<\/time><\/div>\n\n\n<ul class=\"wp-block-social-links has-small-icon-size has-icon-color is-style-logos-only is-layout-flex wp-block-social-links-is-layout-flex\">\n\n\n\n<\/ul>\n<\/div>\n\n\n\n<p class=\"wp-block-paragraph\"><strong class=\"sans-serif\">Machine vision tools like facial recognition<\/strong> are increasingly being used for law enforcement, advertising, and other purposes. Pew Research Center itself recently used a machine vision system to measure the prevalence of men and women in online <a href=\"https:\/\/beta.pewresearch.org\/pewresearch-org\/social-trends\/2018\/12\/17\/gender-and-jobs-in-online-image-searches\/\" target=\"_blank\" rel=\"noopener noreferrer\">image search results<\/a>. This kind of system develops its own rules for identifying men and women after seeing thousands of example images, but these rules can be hard for to humans to discern. To better understand how this works, we showed images of the Center&#8217;s staff members to a trained machine vision system similar to the one we used to classify image searches. We then systematically obscured sections of each image to see which parts of the face caused the system to change its decision about the gender of the person pictured. Some of the results seemed intuitive, others baffling. In this interactive challenge, see if you can guess what makes the system change its decision.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Here\u2019s how it works:<\/p>\n\n\n<div class=\"wp-block-prc-platform-feature-loader\" id=\"js-interactive\"><\/div>\n\n\n<h4 class=\"wp-block-heading\" id=\"what-these-results-can-tell-us-about-machine-vision\">What these results can tell us about machine vision<\/h4>\n\n\n\n<figure class=\"wp-block-image aligncenter size-640-wide\"><a href=\"https:\/\/beta.pewresearch.org\/pewresearch-org\/data-labs\/feature\/how-does-a-computer-see-gender\/conclusion-8\/\"><img decoding=\"async\" src=\"https:\/\/beta.pewresearch.org\/pewresearch-org\/wp-content\/uploads\/sites\/20\/2019\/09\/conclusion.png\" alt=\"\"\/><\/a><\/figure>\n\n\n\n<p class=\"wp-block-paragraph\">Here are some of the images from the interactive challenge. As a reminder, if any yellow or purple area is covered, the model changes its initial decision of whether the image is showing a man or a woman.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">What do you see when you look at these images? Perhaps you noticed that sometimes the parts of a person\u2019s face that cause the model to change its mind about the gender of the person are not what you might have expected. Sometimes covering the middle of a person\u2019s face causes the system to change its decision \u2013 but sometimes the same thing happens if we cover up a part of their face that seems trivial to a human in determining gender, like their ear or a part of their forehead.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">You might also notice that there aren\u2019t obvious or consistent patterns to the parts of people\u2019s faces that cause the model to change its mind. Covering up a certain part of one person\u2019s face might change its decision for that image, but covering the same part of another person\u2019s face might not cause that change.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Machine learning tools can bring substantial efficiency gains to analyzing large quantities of data, which is why we used this type of system to examine thousands of image search results in our own studies. But unlike traditional computer programs \u2013 which follow a highly prescribed set of steps to reach their conclusions \u2013 these systems make their decisions in ways that are largely hidden from public view, and highly dependent on the data used to train them. As such, they can be prone to systematic biases and can fail in ways that are difficult to understand and hard to predict in advance.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">The technique used in this interactive is one way to help understand the features these algorithms rely on to make their decisions. Similarly, our <a href=\"https:\/\/beta.pewresearch.org\/pewresearch-org\/internet\/essay\/the-challenges-of-using-machine-learning-to-identify-gender-in-images\/\">data essay on machine vision<\/a> takes an in-depth look at how the data used to train these systems can introduce hidden biases and unexpected sources of error into their results.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Our machine learning tool is relatively simple and designed for the very specific purpose of identifying gender in digital images, so more advanced systems or those trained to do different tasks might achieve different results. But this exercise can provide important insights into the ways these systems make their decisions. More importantly, it can provide insight into how these processes are not always clear or intuitive. As algorithmic decisions are increasingly brought to bear on decisions with profound real-world impacts for human beings, it is important to understand their limitations \u2013 as well as the ways in which their results may be biased or simply inaccurate.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><em>For more about this analysis, read our <a href=\"https:\/\/beta.pewresearch.org\/pewresearch-org\/internet\/2019\/09\/05\/methodology-184\">methodology<\/a>. To learn more about machine vision, read our <a href=\"https:\/\/beta.pewresearch.org\/pewresearch-org\/internet\/essay\/the-challenges-of-using-machine-learning-to-identify-gender-in-images\/\">data essay<\/a> about the impact of training data on the accuracy of machine vision models. To read how Pew Research Center has used machine vision systems for research, see \u201c<a href=\"https:\/\/beta.pewresearch.org\/pewresearch-org\/journalism\/2019\/05\/23\/men-appear-twice-as-often-as-women-in-news-photos-on-facebook\/\" target=\"_blank\" rel=\"noopener noreferrer\">Men Appear Twice as Often as Women in News Photos on Facebook<\/a>\u201d and \u201c<a href=\"https:\/\/beta.pewresearch.org\/pewresearch-org\/social-trends\/2018\/12\/17\/gender-and-jobs-in-online-image-searches\/\" target=\"_blank\" rel=\"noopener noreferrer\">Gender and Jobs in Online Image Searches<\/a>.\u201d<\/em><\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><em>Photographs by Bill Webster<\/em><\/p>\n","protected":false},"excerpt":{"rendered":"<p>A computer can be trained to predict whether an image shows a man or a woman. Can you identify which parts of the face are most essential to the computer\u2019s decision?<\/p>\n","protected":false},"featured_media":5722,"parent":0,"comment_status":"closed","ping_status":"closed","template":"layout-essay-no-breadcrumbs","meta":{"sub_headline":"A computer can be trained to predict whether an image shows a man or a woman. Can you identify which parts of the face are most essential to the computer\u2019s decision?","sub_title":"A computer can be trained to predict whether an image shows a man or a woman. Can you identify which parts of the face are most essential to the computer\u2019s decision?","_prc_public_revisions":[],"_ppp_expiration_hours":0,"_ppp_enabled":false,"ai_generated_summary":"","_feature_data_attachment_id":0,"_feature_rewrites":[],"_feature_embed_referenced_in":0,"datacite_doi":"","datacite_doi_citation":"","_prc_seo_qr_attachment_id":0,"displayBylines":true,"footnotes":"","_prc_fork_parent":0,"_prc_fork_status":"","_prc_active_fork":0},"categories":[300,353,255,30,34,279,38,110],"bylines":[974,931],"datasets":[],"level_of_effort":[],"primary_audience":[],"information_type":[],"_post_visibility":[],"formats":[463,460,468],"_fund_pool":[],"regions-countries":[515],"research-teams":[521],"class_list":["post-5425","feature","type-feature","status-publish","has-post-thumbnail","hentry","category-algorithms","category-data-science","category-gender-demographics","category-gender-lgbtq","category-internet-technology","category-online-privacy-security","category-political-issues","category-privacy-rights","bylines-chris-baronavski","bylines-stefan-wojcik","formats-data-essay","formats-feature","formats-quiz","regions-countries-united-states","research-teams-data-labs"],"label":false,"post_parent":0,"word_count":730,"canonical_url":"https:\/\/beta.pewresearch.org\/pewresearch-org\/data-labs\/feature\/how-does-a-computer-see-gender\/","art_direction":{"A1":{"id":5722,"rawUrl":"https:\/\/beta.pewresearch.org\/pewresearch-org\/wp-content\/uploads\/sites\/20\/2019\/09\/PL_19.08_machine_vision_interactive_featured_640.png","url":"https:\/\/beta.pewresearch.org\/pewresearch-org\/wp-content\/uploads\/sites\/20\/2019\/09\/PL_19.08_machine_vision_interactive_featured_640.png?w=564&h=317&crop=1","width":564,"height":317,"chartArt":false},"A2":{"id":5722,"rawUrl":"https:\/\/beta.pewresearch.org\/pewresearch-org\/wp-content\/uploads\/sites\/20\/2019\/09\/PL_19.08_machine_vision_interactive_featured_640.png","url":"https:\/\/beta.pewresearch.org\/pewresearch-org\/wp-content\/uploads\/sites\/20\/2019\/09\/PL_19.08_machine_vision_interactive_featured_640.png?w=268&h=151&crop=1","width":268,"height":151,"chartArt":false},"A3":{"id":5722,"rawUrl":"https:\/\/beta.pewresearch.org\/pewresearch-org\/wp-content\/uploads\/sites\/20\/2019\/09\/PL_19.08_machine_vision_interactive_featured_640.png","url":"https:\/\/beta.pewresearch.org\/pewresearch-org\/wp-content\/uploads\/sites\/20\/2019\/09\/PL_19.08_machine_vision_interactive_featured_640.png?w=194&h=110&crop=1","width":194,"height":110,"chartArt":false},"A4":{"id":5722,"rawUrl":"https:\/\/beta.pewresearch.org\/pewresearch-org\/wp-content\/uploads\/sites\/20\/2019\/09\/PL_19.08_machine_vision_interactive_featured_640.png","url":"https:\/\/beta.pewresearch.org\/pewresearch-org\/wp-content\/uploads\/sites\/20\/2019\/09\/PL_19.08_machine_vision_interactive_featured_640.png?w=268&h=151&crop=1","width":268,"height":151,"chartArt":false},"XL":{"id":5722,"rawUrl":"https:\/\/beta.pewresearch.org\/pewresearch-org\/wp-content\/uploads\/sites\/20\/2019\/09\/PL_19.08_machine_vision_interactive_featured_640.png","url":"https:\/\/beta.pewresearch.org\/pewresearch-org\/wp-content\/uploads\/sites\/20\/2019\/09\/PL_19.08_machine_vision_interactive_featured_640.png?w=720&h=405&crop=1","width":720,"height":405,"chartArt":false},"social":{"id":5722,"rawUrl":"https:\/\/beta.pewresearch.org\/pewresearch-org\/wp-content\/uploads\/sites\/20\/2019\/09\/PL_19.08_machine_vision_interactive_featured_640.png","url":"https:\/\/beta.pewresearch.org\/pewresearch-org\/wp-content\/uploads\/sites\/20\/2019\/09\/PL_19.08_machine_vision_interactive_featured_640.png?w=1200&h=628&crop=1","width":1200,"height":628,"chartArt":false}},"_embeds":[],"table_of_contents":[],"datacite_doi":"","prc_seo_data":{"title":"How does a computer \u2018see\u2019 gender?","description":"A computer can be trained to predict whether an image shows a man or a woman. Can you identify which parts of the face are most essential to the computer\u2019s decision?","og_title":"How does a computer \u2018see\u2019 gender?","og_description":"A computer can be trained to predict whether an image shows a man or a woman. Can you identify which parts of the face are most essential to the computer\u2019s decision?","schema_type":"WebPage","noindex":false,"canonical_url":"","primary_terms":[],"custom_schema":[],"og_image":5722,"indexnow_submitted_at":null,"gsc_index_status":null},"prepublish_checks":{},"jetpack_sharing_enabled":true,"bylinesOrdered":[{"key":"9009abed-63b2-41e5-ad69-5d2b839ecf68","termId":931},{"key":"10c405a6-98ea-4f95-a94f-bccc6a1a0f5d","termId":974}],"acknowledgementsOrdered":[],"_links":{"self":[{"href":"https:\/\/beta.pewresearch.org\/pewresearch-org\/wp-json\/wp\/v2\/feature\/5425","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/beta.pewresearch.org\/pewresearch-org\/wp-json\/wp\/v2\/feature"}],"about":[{"href":"https:\/\/beta.pewresearch.org\/pewresearch-org\/wp-json\/wp\/v2\/types\/feature"}],"replies":[{"embeddable":true,"href":"https:\/\/beta.pewresearch.org\/pewresearch-org\/wp-json\/wp\/v2\/comments?post=5425"}],"version-history":[{"count":11,"href":"https:\/\/beta.pewresearch.org\/pewresearch-org\/wp-json\/wp\/v2\/feature\/5425\/revisions"}],"predecessor-version":[{"id":273389,"href":"https:\/\/beta.pewresearch.org\/pewresearch-org\/wp-json\/wp\/v2\/feature\/5425\/revisions\/273389"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/beta.pewresearch.org\/pewresearch-org\/wp-json\/wp\/v2\/media\/5722"}],"wp:attachment":[{"href":"https:\/\/beta.pewresearch.org\/pewresearch-org\/wp-json\/wp\/v2\/media?parent=5425"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/beta.pewresearch.org\/pewresearch-org\/wp-json\/wp\/v2\/categories?post=5425"},{"taxonomy":"bylines","embeddable":true,"href":"https:\/\/beta.pewresearch.org\/pewresearch-org\/wp-json\/wp\/v2\/bylines?post=5425"},{"taxonomy":"datasets","embeddable":true,"href":"https:\/\/beta.pewresearch.org\/pewresearch-org\/wp-json\/wp\/v2\/datasets?post=5425"},{"taxonomy":"level_of_effort","embeddable":true,"href":"https:\/\/beta.pewresearch.org\/pewresearch-org\/wp-json\/wp\/v2\/level_of_effort?post=5425"},{"taxonomy":"primary_audience","embeddable":true,"href":"https:\/\/beta.pewresearch.org\/pewresearch-org\/wp-json\/wp\/v2\/primary_audience?post=5425"},{"taxonomy":"information_type","embeddable":true,"href":"https:\/\/beta.pewresearch.org\/pewresearch-org\/wp-json\/wp\/v2\/information_type?post=5425"},{"taxonomy":"_post_visibility","embeddable":true,"href":"https:\/\/beta.pewresearch.org\/pewresearch-org\/wp-json\/wp\/v2\/_post_visibility?post=5425"},{"taxonomy":"formats","embeddable":true,"href":"https:\/\/beta.pewresearch.org\/pewresearch-org\/wp-json\/wp\/v2\/formats?post=5425"},{"taxonomy":"_fund_pool","embeddable":true,"href":"https:\/\/beta.pewresearch.org\/pewresearch-org\/wp-json\/wp\/v2\/_fund_pool?post=5425"},{"taxonomy":"regions-countries","embeddable":true,"href":"https:\/\/beta.pewresearch.org\/pewresearch-org\/wp-json\/wp\/v2\/regions-countries?post=5425"},{"taxonomy":"research-teams","embeddable":true,"href":"https:\/\/beta.pewresearch.org\/pewresearch-org\/wp-json\/wp\/v2\/research-teams?post=5425"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}