“My advice to new researchers who are critiquing literature…is that you practice, that every time that you go through and you read a research article for a course paper
for your dissertation, for your doctoral study, that you read it with a critical eye.” â€”Dr. Gary Burkholder
Critiquing an article is a skill that researchers continuously hone throughout their careers. By doing this, researchers can better understand existing literature and improve their own research skills. For this Application, you will critique an article as a first step toward completing a literature review.
To prepare for this Application:
- Select one of the articles from your literature review for this assignment.
- Review the Journal Article Critique handout as well as the media segment on article critique.
- Print out the Journal Article Critique handout and use the guidelines to help you read the article with a critical eye. Make notes on the handout to help you determine the overall strengths and weaknesses of the article.
- Craft a 5-page article critique based on the guidelines in the Journal Article Critique handout.
Journal Article Critique
Scholarly critique is an important aspect of your role as scholar-practitioner. Scholarly critique takes on a number of forms, of which peer review and critical reviews of the literature are two common examples. There are many examples of why it is important to develop good skills in scholarly critique.
1. In the peer-review process commonly used by editors of journals, scholars review research prior to publication to ensure quality, relevance, and contribution to the field.
2. The system of tenure that is common in traditional colleges and universities depends on a process of peer review by departmental colleagues. At some point in time, you may be on such a committee (at a university or in some other setting) where you have to critically evaluate a colleague for promotion.
3. Because there is no such thing as the perfect research study, scholarly critique is critical to understanding the limitations of research studies and how those limitations could be addressed in future studies.
4. The critical literature review article requires good critical evaluation skills to help other scientists understand the current state of knowledge in a particular field of study in terms of what we know and what still needs to be more fully understood.
5. The skills you will develop in the critical evaluation of articles will also help you become a better reviewer of your own drafts. In the short term, this better self-assessment will strengthen the work you submit to dissertation committee members. In general, this approach helps researchers become their own â€œskeptics/critics,â€ thereby developing more valid research designs and research write-ups that can better stand up to peer scrutiny.
6. Your dissertation requires a scholarly critique of the literature, similar to a critical literature review. Thus, you need to understand how to train yourself to be a critical reader of the research. If you want to become an expert on a particular topic, you also need to become an expert on critically dissecting and evaluating the research studies in that area of study.
7. The scholar-practitioner model is one that relies on evidence-based practice. You are being trained to be critical consumers of literature so that you can make key decisions regarding best practices in the field. Understanding best practices requires one to be able to critically evaluate study findings and determine whether those findings will remain valid in â€œreal-worldâ€ settings.
8. In the dissertation process, your committee is essentially a team of peer reviewers who provide input to help you achieve the best possible research project that demonstrates doctoral level scholarship.
Start the process of learning to be a critical reviewer of the scientific literature by dispelling a couple of common myths that you have.
Â· â€œI am not worthy of critiquing something that has already been published in a peer-reviewed journal.â€ In fact, you have a lot of knowledge now about research design and statistics that you can use to determine if a study is truly valid and where the limits to the findings exist. If you are not worthy of critiquing a published article, then can you possibly be worthy of attaining scholarship in a particular discipline?
Â· â€œIf a study has already been published in a peer-reviewed journal, then it must be a really good study.â€ There is no such thing as a perfect study. In fact, just because a study is published does not guarantee that it is necessarily sound in all aspects. Studies may be good, but all studies have areas for improvement. Typically, three reviewers provide input; sometimes, it may be only two reviewers. Thus, decisions are made based on a very small set of inputs, which means that there may be key things missed that may not have been noticed by the original reviewers.
Thus, the assignment to critique a peer-reviewed article should not be approached as something negative. Peer review is one of the key processes that sustains and furthers knowledge. Approach the critique from the following perspectives:
1. You are reviewing a study that was deemed to be publishable, but challenge yourself to use the knowledge you have gained in your research courses to see how the study could be made even better.
2. Review research using the criteria described below to understand better where the gaps are in our understanding of the field. Limitations in sample size, sampling strategy, and design, for example, can easily form the basis for another dissertation or scholarly research study. Therefore, you should use the critique as a means of generating ideas for your own research as well as approaches you can take in your own dissertation to improve the validity and overall soundness of your study.
A suggested resource for this course is an excellent, easy to read (and relatively inexpensive) text on how to approach the critical literature review. This would also be a great resource as you complete the literature review for the dissertation.
Pyrczak, F. (2009). Evaluating research in academic journals: A practical guide to realistic evaluation (4th ed.). Los Angeles, CA: Pyrczak.
Â· 5 pages maximum (not including the reference page).
Â· Times Roman or Arial 12-point font.
Â· One-inch margins at top and bottom and along left and right sides.
Â· Any editing marks (track changes or comments) are removed.
Â· Paper is appropriately spell checked and grammar checked.
Â· Must be a paper that is available as full text through the Walden library.
Â· At the end of the paper, provide to the Instructor the full hyperlink to the full text article.
Â· The article chosen must have been published within the previous 5 years.
Â· Introduction. Provide a brief and concise summary of the problem of the study and main findings. Cite the study using APA format and provide the full citation at the end of the paper in a reference section.
Â· Critique of the Literature Review. Some things to consider are:
o Was the problem clearly articulated, and was ample evidence provided to support the problem being addressed?
o Was the theoretical or conceptual framework present, was its relationship to the present study described, and was it appropriate to the problem being addressed?
o Was the literature cited appropriate to the topic?
o Was the literature primarily from current sources (within 5 years of the article publication date)?
o Did the author choose citations judiciously, or did it appear that quantity of citations was emphasized over quality?
o Does the literature review present a clear and non-biased approach to the topic?
o Were the research questions and/or hypotheses clearly stated? Do they logically derive from the literature review?
Â· Critique of the Methods/Research Design. Some things to consider are:
o Were the participants adequately described in terms of population, inclusion and exclusion criteria, and sampling strategy?
o Is the sample representative of the population?
o Is there support that the sample size ensures adequate statistical power?
o For qualitative, was the approach to sampling adequately described and justified? Is the number of participants in the study justified?
o Was there a statement indicating that IRB approval was obtained?
o Were procedures for protecting participant rights included?
o Were the procedures for executing the design carefully described in a way that you or other scientists could replicate the study?
o For qualitative, is the process for collecting, organizing, and analyzing the data appropriately transparent?
o Is the role and activity of the researcher in the data collection setting/sites described?
o Were reliability and validity measures of questionnaires, scales, or other measurement instruments presented? Do measures exhibit adequate reliability and validity?
o Were instruments used in populations for which they may not have been normed? Was there effort made to ensure reliability and validity in the study sample?
o Was the design appropriate to test the hypothesis(es) or address the research questions?
o Was random assignment used? If not, what are the potential flaws to internal and external validity?
Â· Critique of the Results section. Some things to consider are:
o Are the important characteristics of the sample described?
o Are participation rates (and attrition rates in longitudinal studies) described? For longitudinal studies, was differential attrition determined?
o Were key descriptive statistics provided for all variables?
o Do the results address the hypotheses under question?
o For qualitative, were the findings tied back to the research question?
o For qualitative, was it clear how findings arose? Were findings linked to the application of steps/methodology described in the methods section?
o For mixed methods, are findings described in a holistic, complementary way in order to address the research questions?
o For mixed methods, is the necessity of each type of data (qualitative and quantitative) clear for answering each research question?
o Are tables and figures used effectively? Were tables not used when they would have been very helpful to the reader? For qualitative (when needed) are models used effectively (and labeled effectively) in order to understand findings and interpretations? (Models are sometimes included in the discussion rather than the results section.)
o Are effect sizes and p-values reported for all inferential findings?
Â· Critique of the Discussion section. Some things to consider are:
o Are the results discussed in the context of the research presented in the literature review section?
o Are methodological limitations adequately addressed? Think in terms of sample representativeness, generalizability of results, and potential threats to internal and external validity.
o Are implications for further research described?
o Are implications for practitioners described?
o Is the contribution/significance to the field in relation to the continuum of inquiry clear?
o For a qualitative study, is there some discussion of the researcherâ€™s reflections, insights, challenges, biases, surprises, etc.?
Â· Overall Evaluation. Some things to consider are:
o Evidence to support strengths and limitations of the study.
o Evidence that supports that the study was both reliable and valid.
o Justification for why you would include this as an important piece of evidence in your study.
_____ ( 5) Introduction
_____(10) Critique of the literature review
_____(10) Critique of the Methods/Research Design
_____(10) Critique of the Results Section
_____(10) Critique of the Discussion
_____( 5) Overall Impression of the study