Rating Tips
Here are detailed explanations of each of the rating questions on the NewsTrust review form:
Recommendation
Q. Is it a good story?
Would you recommend this story to a friend? This required question simply records whether you like this story or not. It is comparable to the up and down arrows used by popularity-based social news sites like Digg or Reddit. This lets us measure the overall popularity of each story, and compare it to our in-depth quality ratings.
The next questions in our review form will help you reach beyond your initial impressions, by carefully checking journalistic qualities like fairness, evidence and sourcing, for a more reliable quality rating.
Trust
Q. Do you trust this publication?
This question measures how much you personally trust the publisher of this story. Based on your experience, is this publication credible? Can it be trusted to provide reliable information? We ask this subjective question upfront, because trust plays such an important role in how we perceive the news. Note that your rating for that question will be automatically filled-in next time you review a story from this publication.
The questions that follow will help you verify this source's journalistic integrity in more depth. If your initial trust level changes as a result of your story review, please adjust this rating accordingly.
Information
Q. Is this story informative?
Here we evaluate the depth, thoroughness and originality of this story's news coverage and/or interpretation. Is the journalist providing new information here, or merely repeating what others have written about this topic? To answer this question, you may want to compare this story to other stories on the same topic. To view related links, click on "See Reviews" in the top navbar of the review form.
Fairness
Q. Is this story fair?
Journalists are expected to present fairly all sides of a controversy. Note this doesn't necessarily mean equal space for all sides. The space allotted to each side should be based on the evidence for its claims and its willingness to respond. Each relevant side, however, should be afforded the opportunity to make its core argument, or decline comment.
Sources
Q. Is this story well sourced?
This question evaluates the level of care the authors take to find reliable sources for a story. Are these sources qualified to provide information based on special expertise, or unique knowledge of the facts? Expert sources whose view might not be influenced by a stake in the event or issue are particularly valuable; they help move reporting beyond "he said/she said" counterclaims that can leave readers confused. Additionally, how well does the author inform readers about where the information came from (clearly indicating and naming sources, explaining why a source may have been granted anonymity and providing some identifying information to help the reader/viewer assess credibility). As a rule, all sources should be named, including other news organizations. When a source cannot be named, readers are owed an explanation of why.
Context
Q. Does this story show the "big picture"?
This question asks how well the story provides context by seeking answers to questions the public most needs to know about this issue or event. "Big picture" reporting that asks about causes, effects, solutions and trends, rather than describing a particular episode, such as a shooting, fire or accident, is also more helpful for this key "sense-making" function of news.
Evidence
Q. Does this story provide factual evidence?
This question asks whether the authors of a story provide verifiable, factual evidence to support their assertions. Pay particular attention to whether the right type of evidence is provided (authoritative quotes, appropriate statistics, documents, etc.), especially in the headline and lead paragraphs (i.e., the principal generalizations the story makes). Documents and statistics often provide more compelling evidence for a generalization than statements by individuals. Multiple sources of evidence strengthen the story, as well as independent verification by the authors that this evidence is valid.
Importance
Q. Is this an important topic?
This question focuses on the importance of the topic or subject of the story, in order to help citizens participate in local, state and national government. Socially responsible journalism requires careful attention to what people in the community served need to know to make wise decisions, particularly collective (usually political) ones. Stories about celebrities, sports, fender-benders or lifestyle are generally are not as helpful for making sense of the world as stories on politics, government, education, crime, science, etc. Another way to judge the importance of a story's topic is to ask yourself how many people are affected by its topic: tens? thousands? millions? billions?
Balance
Q. Does this story present all key viewpoints?
This question probes whether one or more important sides to the event or issue are missing or given less space than they deserve. In general, the more perspectives a story includes, the fuller the picture of reality it provides. Note that most news stories only have room for the core arguments each side makes, rather than their complete point of view.
Style
Q. Is this story well presented?
The way a subject is treated also matters. Here we evaluate this story's overall style and the quality of its writing. Is the story clear and easy to understand? Does it present useful information in an interesting way? Does it make complex issues meaningful? Does it explain unfamiliar terms and processes to the average reader? Does it inspire as well as inform?
Your Knowledge
Q. Do you know about this topic?
This question measures how much you personally knew about this topic, before reading this story. How much information did you have on this subject already? This knowledge could be specific to the event or issue, or more general (e.g., familiarity with related topics).
Accuracy
Q. Is this story accurate?
Only answer this question if you have sufficient knowledge of this topic. In most cases, you may not be able to judge the accuracy of a story without additional research. This question examines how closely the description of events and issues in this story matches reality and verified facts. When assessing a story's accuracy, be sure to review both its sources and the journalistic statements. How well do they square with the reality you observed? what other media reported? or what you already know about this topic? If you don't have access to sufficient information, please do not answer this question. Don't rate the story merely on plausibility.
My Average Rating
As you answer rating questions in this review form, your average rating for this story is dynamically updated and shown at the bottom of the ratings section. (see below to find out how this average rating is calculated).
Note that you can change your story ratings at any time, by simply clicking different values for each questions, until you are satisfied that your average rating represents your overall evaluation of that story's journalistic quality.
Comment
Q. Is this good journalism?
Write a short comment about this story, to explain why you gave it these quality ratings. Please keep your comment to a maximum of 250 words.
Back to Top
Other Review Tips
Focus on the journalism
We aim to rate articles based on their journalistic quality, not whether we agree with the views presented in these articles. It's not always easy to separate the two, but it's worth a try: we often get the best information when we go past our pre-conceived ideas about an article, to look at how it was created.
Be civil
When writing your comments, be civil, thoughtful, concise, and respectful of others. From time to time, you'll see a review you disagree with; try to respond in a way that is not offensive to other members. Please treat people with the same respect you would like them to show you, and adhere to our terms of service.
Disclosure: Are you the author of a story? or personally related to it?
If you are the author, publisher or subject of a story (or otherwise personally related to that story), please use our comment field to disclose any conflict of interest you might have as reviewer for this story. For example, if you are the author or publisher of this story, please identify yourself as such at the start of your comments. If you are a friend of the author, or work for the story's publication, please disclose the nature of your relationship in your comments. If you are featured in the story, or work for an organization that is related to this story, also disclose it in your comments. If you fit any of these cases, we ask that you rate these stories fairly, and refrain from manipulating our ratings for personal, ideological or commercial purposes.
Deceptive, manipulative or offensive reviews may be removed at NewsTrust's discretion and can lead to termination of your membership. Thank you for your understanding.
Other Review Form Sections
Our review form includes more review sections for advanced reviewers (e.g. Tags, Research, Improvements). Simply click on the arrows next to each section name to open or close any of these sections. To find out more about these sections, check our Reviewer FAQ.
Rating Weights
NewsTrust story ratings are weighted in three successive steps, to insure their reliability.
We first weigh your answers based on the significance of each rating question for evaluating journalistic quality, based on findings from our research studies. At this time, we use these weights for all stories:
- Facts: 8 (News), 8 (Opinion)
- Fairness: 6 (News), 6 (Opinion)
- Information: 10 (News), 10 (Opinion)
- Insight: 5 (News), 5 (Opinion)
- Sourcing: 5 (News), 5 (Opinion)
- Style: 4 (News), 4 (Opinion)
- Accuracy: 6 (News), 6 (Opinion)
- Balance: 4 (News), 4 (Opinion)
- Context: 10 (News), 10 (Opinion)
- Depth: 4 (News), 4 (Opinion)
- Enterprise: 4 (News), 4 (Opinion)
- Expertise: 4 (News), 4 (Opinion)
- Originality: 3 (News), 3 (Opinion)
- Relevance: 3 (News), 3 (Opinion)
- Transparency: 3 (News), 3 (Opinion)
- Responsibility: 1 (News), 1 (Opinion)
That first weighted average is then weighted again, based on these individual reviewer criteria:
- Your Rating: 60%
- Number of Answers: 20%
- Topic Knowledge: 4%
- Member Ratings for your Review: 16%
In short, your ratings will count more if you answer more questions about this story, if you know more about its topic, and if your review received high ratings from other members.
