Review of Communication Research

MASS COMMUNICATION                                                                                                                

Title & Author

A Review and Analysis of Patterns of Design Decisions in Recent Media Effects Research

W. James Potter (University of California at Santa Barbara, USA)

Highlights

• The findings of published content analyses of the media effects literature are critically analyzed to provide a foundation for an analysis of current methodological patterns.

• The design decisions made by authors of 211 articles reporting a test of media effects that were recently published (2010 to 2015) in six core communication journals are analyzed.

• The literature continues to display an atheoretical pattern evidenced by the majority of designers of media effects studies ignoring theories when constructing a foundation for their studies.

• There is a continuing pattern of many authors of media effects studies selecting weaker options over stronger ones when designing their samples, measures, and experiments.

• A critical analysis of the likely reasons for these patterns of design decisions reveals that many of the selected design options are supported by assumptions that have been found to be faulty.

• Recommendations are presented for an evolution in thinking about methods, theory, and paradigm.

Abstract

This essay presents a critical analysis of patterns of research design decisions exhibited by authors of recently published empirical tests of media effects. The content of 211 articles published from 2010 to 2015 in six core communication journals were analyzed to document the design decisions made by the authors concerning their use of theory, sampling, measurement, and experiments. We also recorded the amount of variance explained by their tests and use this indicator of strength of findings to explain the patterns of methodological design decisions. The findings indicate that authors of these studies commonly select weaker design options over stronger ones. The reasons for these patterns are explored then critiqued leading to a series of recommendations calling for an evolution in thinking in the areas of method, theory, and paradigm. The methods recommendations attempt to increase (a) awareness of the advantages and disadvantages of options available for each design decision, (b) an understanding that often assumptions made to justify the selection of an option are faulty, and (c) a commitment to meeting a higher degree of challenges. The theory recommendations focus on increasing an understanding about why designers of most tests of media effects ignore the many theories available when designing their studies. And the paradigm recommendations focus on examining more critically the assumptions we make about the nature of human beings, the purpose of our research as challenges evolve, and the defaults in practices we have established in an exploratory phase.

Download

 

open access verde 

A Review and Analysis of Patterns of Design Decisions in Recent Media Effects Research

W. James Potter

Keywords: Media effects; Design decisions; Methods; Development of knowledge; Critical analysis;

 

  Filesize 414.74 KB Download 79

 

 

Content

 I. General Criticism of the Development of the Field of Media Effects

 

II. Criticisms of the Design of Empirical Tests of Media Effects

A. Theory as Foundation

B. Sampling Procedures

C. Measurement Features

1. Mundane behaviors

2. Attribute variables as surrogates

3. Measuring change

4. Evidence for quality of measures

D. Design of Experiments

1. Random assignment

2. Balance check

3. Manipulation check

E. Reporting Effect Sizes

 

III. Current Patterns in Effects Research

A. Generating Data to Document Recent Patterns

1. Sample

2. Coding variables

3. Testing reliability

B. Findings

1. Reporting the use of theory

2. Sampling decisions

3. Measurement decisions

4. Design decisions in experiments

5. Reporting variance explained

 

IV. Big Picture of Patterns

A. Theory-Methods-Findings Nexus

Table 1. Uniformity of Patterns Across Theory Usage

Table 2. Strength of Findings by Design Decisions

B. Analyzing Practices for Supporting Assumptions

1. Assumptions attributable to research perspective

Table 3. Analyzing Assumptions Underlying Selections of Methodological Options

2. Faulty assumptions

3. Continuing value of exploratory research

 

V. Recommendations for Moving Forward

A. Methods Recommendations

1. Recommendations with higher change leverage

Table 4. Recommendations by Costs and Benefits

2. Recommendations with lower change leverage

B. Theory Recommendations 

C. Paradigm Recommendations

References

Copyrights and Repositories

 

Keywords

Media effects, design decisions, methods, development of knowledge, critical analysis.

 

About the Author

W. James Potter is professor at the Department of Communication of University of California at Santa Barbara, USA.

 

Author biographical information:

Personal website

Curriculum Vitae

JoomShaper