Skip to main content

Comment Scoring

Evaluate a comment's quality and contribution to the conversation.

What it evaluates:

  • Comment health (1-5 overall score): Quality and the worst problem found, combined; see Comment Health
  • Quality (1-5 score): How well the comment engages and contributes
  • Toxicity level (0-1 score)
  • Low effort detection: e.g., "me too", "first"
  • Logical fallacies (ad hominem, straw man, etc.): With educational feedback
  • Objectionable phrases (rude, offensive language): Quoted examples with explanations
  • Negative tone (unconstructive communication patterns): Focused on the quality of communication, not the views themselves

Each finding has a severity (1 minor, 2 clear, 3 severe, 4 extreme), and each objectionable or negative tone phrase has a category such as insult or profanity.

Judged under your site's tolerances: with each request you choose how much of each kind of problem your site accepts, and the lowest quality it accepts. Each of these comment health items gets a score from 1 to 5 in health_items, with the minimum its level accepts; findings your levels accept are removed; and exceeds_tolerance tells you whether the comment is beyond what your site accepts. Decide what to do with a comment on exceeds_tolerance, not on toxicity_score or overall_score alone. See Comment Health.

First-pass screening: every comment is screened first. A comment the screen is confident has no problem is answered without the full assessment, with empty finding lists and an empty toxicity_explanation; anything flagged or uncertain gets the full assessment, with explanations and suggested rewrites. See First-Pass Screening.

Reference​

See the Comment Scoring API Reference for method signatures and parameters.

Response: CommentScore

By Example​

Basic Usage​

const result = await client.evaluateComment(
  "Your argument is completely stupid and you clearly know nothing.",
  articleId
);

console.log(`Comment health: ${result.overall_score}/5`);
console.log(`Quality: ${result.health_items?.quality?.score}/5`);
console.log(`Toxicity: ${result.toxicity_score.toFixed(2)}`);
console.log(`Low Effort: ${result.appears_low_effort}`);
console.log(`Exceeds tolerance: ${result.exceeds_tolerance}`);

// Detailed feedback
for (const fallacy of result.logical_fallacies) {
  console.log(`\nLogical Fallacy: ${fallacy.fallacy_name}`);
  console.log(`Example: ${fallacy.quoted_logical_fallacy_example}`);
  console.log(`Suggestion: ${fallacy.explanation}`);
}

for (const phrase of result.objectionable_phrases) {
  console.log(`\nObjectionable (${phrase.category}, severity ${phrase.severity}): ${phrase.quoted_objectionable_phrase}`);
  console.log(`Why: ${phrase.explanation}`);
}

for (const tone of result.negative_tone_phrases) {
  console.log(`\nNegative Tone: ${tone.quoted_negative_tone_phrase}`);
  console.log(`Why: ${tone.explanation}`);
}

With Reply Context​

When a comment is replying to another comment, provide that context:

const result = await client.evaluateComment(
  "I disagree with your point about type safety.",
  articleId,
  "Type hints are unnecessary overhead in Python."
);

With Tolerances​

Every comment score is judged under the comment health levels you send with the request in tolerances (ModerationTolerances in Python and PHP, a plain object in TypeScript, or a tolerances object in a JSON request). Any item you leave out is standard; nothing is saved on your API key. The items are the nine problem categories and quality, and the levels are off, permissive, relaxed, standard, strict and zero_tolerance. See Setting Tolerances for examples in each language.

Using the Feedback​

Respectify is designed to educate and encourage better discussion, not to censor. Respectify encourages friendly (genuine, well-intentioned) disagreement and healthy sharing of different opinions and views. The focus is on the quality of the reply communicating those views, not the views themselves.

We recommend using this not to censor but to educate and encourage better discussion:

  1. Use exceeds_tolerance to decide whether a comment needs attention. It reflects your site's tolerances; toxicity_score does not, and is never adjusted for them. health_items shows which items fell short.
  2. Show users the identified issues before they post.
  3. Provide educational feedback: Explain logical fallacies and tone problems.
  4. Give them the opportunity to edit and try again.
  5. Allow multiple attempts: Even very low scored comments can be learning opportunities.
  6. Track improvement: Over time, users will proactively write better comments.

The idea is to guide and educate, helping users improve their discussion skills through feedback.