How we score vendors

Ask UpGuard why you're a B. Ask us why you're Elevated Risk - here are the sources.

Incumbent ratings tools often hide their inputs. Vendorisk.ai takes the opposite approach: each dossier links to the pages that were read, the model and prompt version that synthesised them, and the tier criteria applied. If you disagree with a verdict, you can inspect the same material we did.

See a live example on any vendor dossier under Evidence & provenance, and the Executive Strip link labeled How we score. Every published number is re-derivable from stored evaluation fields plus the latest outside-in scan snapshot.

A research run follows a fixed pipeline. Plain-language steps match the production code path:

1. Public-web search in buckets company, product, security, and news (plus enrichment and scan evidence URLs when present).
2. LLM synthesis from those excerpts only. Current prompt_version is 2026-08-13.1; inference uses temperature 0 and top_p 1. We do not guarantee a specific model vendor; the model id used for that run is stored on the evaluation.
3. Provenance gate: claims, events, and firmographics without a real URL and title are dropped. They are not shown with fabricated or placeholder provenance.
4. Persist an evaluation row. Re-research creates a new row; prior rows are kept.
5. Outside-in scan runs after persist and adjusts the displayed score on read (see section 06).

The synthesis model must assign exactly one of four tiers. Criteria below are copied from the production system prompt - they are the rules the model is instructed to follow, not post-hoc marketing copy.

• High Risk

  Active or recent unresolved breach (~12 months) with material impact; or sensitive data handled with essentially no public security posture plus concerning signals.

• Elevated Risk

  Breach or incident 12-36 months ago with incomplete remediation; material legal or regulatory issues; repeated outages; weak public security disclosures relative to risk profile.

• Moderate Risk

  Historical issues with credible remediation; incomplete public data but no active crisis; mixed typical SaaS signals.

• Low Risk

  No material recent incidents found; coherent public security posture or certifications with corroboration; stable news - still advisory, not an attestation.

When evidence is ambiguous, the model is instructed to prefer the more conservative adjacent tier only when negative signals exist - not to invent Low Risk from silence when a vendor likely handles sensitive data and posture is unknown.

The model assigns the tier; Vendorisk does not auto-rewrite it. If the model emits Low Risk with a dated breach in the last 12 months, the evaluation is forced to status partial (low_risk_with_recent_breach) rather than silently changing the tier.

The headline tier is the primary output. The 0-100 risk score shown on dossiers is a deterministic composite on top of a tier base. It is not a statistically calibrated loss model. Recomputing it from stored fields always yields the same number.

Computation order

1. Tier base from RISK_TIER_SCORE:
   • Low Risk → base 15
   • Moderate Risk → base 40
   • Elevated Risk → base 65
   • High Risk → base 90
2. Incidents: +5 per material event (breach | legal | outage; funding, leadership, and other types do not count), capped at +25. Count is from gated events on that run, not the historical events table pile-up.
3. Confidence delta from exported CONFIDENCE_ADJUST:
   • complete → -5
   • partial → +0
   • limited → +8
4. Optional questionnaire blend at persist (see section 05).
5. Outside-in scan ±10 on read (see section 06).
6. Round and clamp 0-100. Higher = riskier.

The score does notinclude an age or freshness term. Age is shown separately (for example "as of" / data-age label). Evaluation freshness window is FRESHNESS_DAYS (30) from lib/utils/freshness.ts.

Worked example

Moderate Risk, 2 material incidents, complete confidence, no questionnaire, no scan → 40 + 10 + (-5) = 45.

Buyers complete a SIG-lite questionnaire themselves; Vendorisk does not questionnaire the vendor.

• Blend only when a row is submitted, numeric, and ≤ QUESTIONNAIRE_SCORE_STALE_DAYS (90).
• Formula: score' = 0.75 * score + 0.25 * (100 - q) (higher posture lowers risk).
• Applied at persist into evaluations.risk_score; read paths do not re-blend.
• First research of a brand-new company has no company+org row yet, so no blend until a later run.

After an evaluation is persisted, an outside-in scan probes the vendor domain. Production runs all 7 checks from SCAN_CHECK_ORDER, in this order:

• tls: valid certificate with ≥14 days to expiry
• headers: at least two-thirds of six security headers present, including HSTS (CSP, X-Frame-Options, X-Content-Type-Options, Referrer-Policy, Permissions-Policy)
• email auth: SPF and DMARC both present
• dns: A or AAAA record
• subdomains: public certificate-transparency count below the warn threshold (25)
• breach db: public HIBP catalog count of 0 (not a paid breached-domain ownership lookup)
• status page: at least one common status URL returns 2xx/3xx

Adjustment: each check with a definite result contributes PASS_BONUS (-1) on pass, or FAIL_WEIGHT by severity (info/low 1, medium 2, high 3) on fail. Skipped (passed === null) contributes 0. Sum is rounded and clamped to ±10. Applied on read on top of the persisted score so dossier, export, and API share one displayed number. Scan is a shallow posture probe, not continuous attack-surface scoring.

Research Coverage assignment

Research Coverage (stored as confidence) is mechanical, not a qualitative evidence-quality grade. Evidence Quality is a separate label: Verified, Observed, Vendor Claim, Unknown, or Conflict.

• limited if status is missing or failed
• complete if status is complete and there is at least one surviving citation
• otherwise partial(including evaluations marked partial for insufficient sections, zero citations, "insufficient public data" prose, or the recent-breach review flag)

Recommendations

Mapping from TIER_RECOMMENDATION:

• Low Risk → Proceed
• Moderate Risk → Proceed with caution
• Elevated Risk → Conditional
• High Risk → Do not proceed
• failed or missing → Insufficient data

These are advisory labels, not approvals.

Each evaluation row stores everything needed to audit or reproduce the verdict later:

• source_set - every URL retrieved, grouped by search bucket (company, product, security, news) with titles as retrieved.
• generated_at and next_review_after (+30d) - when the run happened and when a refresh is due.
• model_id, prompt_version, and model_params - which model and instruction set produced the synthesis, and the inference settings used.
• citations and field_sources - claim-level and firmographic links back to specific URLs.
• Gated executive fields: tier, rationale, summaries, confidence, incident count, and risk score.

Changelog records tier, score, and evidence-URL movements. Changelog scores are persist-time (questionnaire in, scan usually not yet). The number on the dossier includes the latest scan adjustment.

• Advisory research, not certification, audit, pentest, or legal opinion.
• Public web plus optional enrichment plus shallow outside-in probes only.
• No vendor interviews; no completing questionnaires on the vendor's behalf.
• Absence of a finding is not absence in the world; an LLM can misread or conflate sources.
• Partial or failed dossiers still show what was attempted.
• The numeric score is a deterministic composite, not a statistically calibrated loss model.

Questions about a specific dossier? Use the contact page. For terms governing use of assessments, see Terms of Service.