# Reliability Rubric A structured checklist for assessing the reliability of a source before committing to full reading. These criteria are warning signals, not guarantees. A source that passes all criteria can still have weak arguments or thin evidence. A source that fails one criterion may still be useful if its other qualities are strong. Use judgment. --- ## Core 5 Criteria (All Researchers) ### Criterion 1: Publication Venue **Question:** Is the source published or posted by a reputable press or organization? **What to look for:** - University presses (Oxford, Cambridge, Chicago, MIT, etc.) are reliable for most purposes - Established commercial publishers known in the field (Norton for literature, West for law, Ablex for sciences) are generally reliable - Peer-reviewed journals (any field) — reliability depends partly on the journal's reputation - Government agencies, major research institutions, established professional associations **Red flags:** - Self-published books or articles with no visible publisher - Sensationalist claims on the cover or in the abstract that are out of proportion to the evidence - Publisher or sponsor driven by ideology on contested social issues (gun control, climate, abortion) — check who funds the press or organization - No publisher listed at all **Note:** Libraries include materials from all kinds of publishers for the sake of coverage. Presence in a library catalog does not indicate reliability. --- ### Criterion 2: Peer Review **Question:** Was the work peer-reviewed before publication? **What to look for:** - "Peer reviewed" designation in the database or journal description - Journal articles in established academic journals (most are peer-reviewed) - Books from university presses (typically reviewed by field experts before acceptance) **What does not constitute peer review:** - Editorial review alone (most commercial magazines, newspapers) - Review only by the named editor of an essay collection (not the same as peer review) - Preprints without clear peer-review notice - Blogs and personal websites **If you cannot determine peer-review status:** Treat the source as unverified and increase scrutiny on other criteria. --- ### Criterion 3: Author Credentials **Question:** Is the author a reputable scholar or expert in this specific field? **What to look for:** - Academic affiliation listed (institution, department) - Track record of publication in the field (check citations, CV if accessible) - Cited by other reputable sources in the same field - No undisclosed financial ties to parties with a stake in the outcome **Complications:** - Established scholars can have axes to grind, especially on politically or commercially contested topics - Check acknowledgments and funding disclosures: if a pharmaceutical company funded the research, that is relevant context - On highly contested social issues, even credentialed scholars may be advocates first; require stronger evidence from these sources **If you are new to the field and cannot assess credentials:** Use citation count as a proxy — frequently cited authors are generally considered significant by the field. Use citation indexing (forward citation search) to check. --- ### Criterion 4: Currency **Question:** Is the source current enough for this field and this question? **Field norms:** | Field | Outer limit (findings) | Notes | |---|---|---| | Sciences and technology | ~2 years | Foundational theory may be cited regardless of age | | Social sciences | ~10 years | Check recent literature for updates or reversals | | Humanities (literary criticism, history, philosophy) | Decades | Foundational works cited regardless of date; recent editions matter | **Rules:** - Assume textbooks are not current — they synthesize existing knowledge but typically lag the cutting edge by years - Always use the most recent edition of a secondary or tertiary source; researchers revise (and sometimes reverse) views across editions - For primary sources (novels, letters, historical documents), standard editions are preferred — usually not the most recent reprint - **Rule of thumb for any field:** Look at the citations in 2-3 recent articles in your target area. The oldest sources cited regularly define the acceptable floor for age **If you are unsure:** Flag the source as possibly outdated and continue; read for content and let the argument determine whether the age matters. --- ### Criterion 5: Scholarly Apparatus **Question:** Does the source include notes, citations, and bibliography that allow you to verify its claims? **What to look for:** - Footnotes or endnotes with specific citations - A bibliography or works cited list - For websites: clear attribution (author name), date of publication or last update, named sponsor or organization **Red flags:** - A book-length argument with no bibliography or citations: you have no way to follow up on anything it claims - A website with no stated author, no date, and no organizational sponsor: you cannot assess the source of its claims - A website that has not been updated recently may be abandoned and no longer endorsed by its sponsor --- ## Advanced Criteria (Experienced Researchers or High-Stakes Projects) ### Criterion 6: Published Reviews **Question:** Has the source been reviewed in field-specific review journals? Indexes to published reviews exist for most fields. A source reviewed favorably by multiple field experts has an additional layer of vetting. A source that has attracted no reviews (for a book expected to be significant) or only negative reviews warrants extra scrutiny. --- ### Criterion 7: Citation Impact **Question:** Has the source been frequently cited by other researchers? High citation count = high impact factor. Use forward citation indexing (looking up who has cited a source) to assess influence. A source cited repeatedly by other scholars is considered significant by the field — even if it presents a contested view, you need to engage with it. **Caution:** Citation count is a measure of influence, not necessarily accuracy. A highly cited source may be influential because it made a major claim that many subsequent papers argue against. Read beyond the count. --- ### Criterion 8: Website Topic Approach **Question (for websites only):** Does the site approach its topic judiciously? A site that engages in heated advocacy, attacks those who disagree, makes wild or unsubstantiated claims, uses abusive language, or contains frequent spelling and grammatical errors is unlikely to be accepted by careful readers as reliable evidence. --- ## Reliability Verdict After assessing all applicable criteria, assign one of three verdicts: | Verdict | Criteria profile | |---|---| | **Reliable** | Passes all 5 core criteria, or minor issue on 1 criterion offset by strengths on the others | | **Use with Caution** | Issues on 2 criteria; use the source but flag the specific concerns in your notes and consider corroborating from a stronger source | | **Exclude** | Fails 3 or more criteria; or fails simultaneously on peer review AND author credentials — the two criteria most central to scholarly vetting | **Remember:** Even Reliable sources must be read critically. Reliability indicators lower the probability of errors; they do not eliminate it.