Review of Communication Research

MEDIA LITERACY                                                                                                

Title & Authors

Reviewing Media Literacy Intervention Studies for Validity

W. James Potter (University of California at Santa Barbara, USA)

Chan L. Thai (Santa Clara University, USA)

Highlights

 

• The designs of published studies claiming to test media literacy interventions were analyzed to determine how the conceptual foundation was constructed in each study and whether the study was designed based on those conceptual foundations to determine content and face validity.

• A total of 88 studies were selected after a series of electronic searches of studies that used the term “media literacy intervention” in their keyword lists, titles, and abstracts.

A meaning analysis found that 22 studies (25.0%) provided no conceptual foundation for media literacy, and 21 (23.9%) used an existing definition of media literacy. Despite there being hundreds of definitions for media literacy in the literature, the authors of the remaining 45 studies (51.1%) presented their own definition for media literacy.

• The assessment of validity found that none of the studies presented a test of media literacy that completely captured the elements in their definitions of media literacy, so the content validity of this literature was judged as poor.

• The evaluation of face validity uncovered many problems in a lack of correspondence between what authors intended to measure and what they actually measured.

• The most prevalent discrepancy was with measures of skills where authors frequently measured beliefs about study participants’ levels of skills rather than taking measures of actual performance.

• We pose a series of three questions that illuminate the current nature of the media literacy intervention literature as well as serve to guide future designers of such studies.

Abstract

This study is an examination of validity in published articles that have provided tests of the effectiveness of media literacy interventions. We identified 88 published tests of media literacy interventions then analyzed their content using five coding variables that indicated the degree to which authors of those studies established basic validity. We first conducted a meaning analysis to identify the definitions that authors of those studies presented for media literacy. Then we used those definitions to determine the extent to which those authors provided a complete (content validity) and accurate (face validity) operationalization in the design of their measures.

Download

 

open access verde 

 

Reviewing Media Literacy Intervention Studies for Validity

W. James Potter and Chan Thai

Keywords: Media literacy interventions; validity; meaning analysis; explication; critical analysis;

 

  Filesize 398.95 KB Download 197

 

 

Content

 I. The Criterion of Validity in Social Science Research

   A. Content Validity

   B. Face Validity

II. Meaning Analysis of Media Literacy

III. Method
   A. Selecting Media Literacy Intervention Studies

   B. Coding Variables
      Media literacy definitions cited
      Focal definition of media literacy
      Definitional elements
      Content validity
      Face validity

   C. Reliability

IV. Results
   A. Presentation of Meaning

   B. Definitional Elements

   C. Content Validity

   D. Face Validity

V. Discussion
   A. Reasonable Standard
      1. Conceptual foundation
      2. Measures of media literacy

   B. Important Questions
      1. Basic research study or evaluation study?
      2. Persuasion or empowerment perspective?
      3. Skills as performances or as beliefs?

VI. Conclusion

 

 

Keywords

Media literacy interventions, validity, meaning analysis, explication, critical analysis.

 

About the Authors

Authors biographical information:

Personal website: W. James Potter

Personal website: Chan L. Thai

JoomShaper