Skip to content
Issue 142 · Since 2014
A Great Portal The Methodology Brief
Issue No. 142
Filed Austin, Texas · Q1 2025
Author Maya Okafor, Editor-in-Chief
Last verified 4h ago

The Standing Research Brief · Methodology

How We Vet, Rank, and Re-verify Every Listing.

A line-by-line account of the editorial machinery behind A Great Portal — the 47-point review, the 62% rejection rate, the 14-day re-verification SLA, and the Dreyfus Trust Score. This is the page we publish so that the rest of the site doesn't need to ask for your trust twice.

Get the Weekly Shortlist — Free or browse 52,417 vetted tools →
Reading time · 11 minutes Audited annually by BDO ISSN 2371-0428

The 60-Second Answer

In plain terms, here is how every listing earns (and keeps) its place.

01

Every submission passes a 47-point editorial review written by hand and updated quarterly. Six full-time analysts and two external auditors score each tool across security, pricing, support, roadmap, integration breadth, and operational maturity.

02

Only 38% of submissions are published — meaning a 62% rejection rate. The other 62% are sent back with a written explanation. We do not list on request; we list on evidence.

03

Every live listing is re-checked against the same 47 points every 14 days. If a vendor slips — pricing changes, support degrades, a CVE is disclosed — the listing is downgraded or paused within the same cycle. The audit trail is public.

04

Rankings are produced by the Dreyfus Trust Score, a 0–100 credibility index with published weights. Paid placements are badged, never influence scores, and the firewall is audited annually by BDO.

If you have ten seconds, the four numbers above are the brief. If you have eleven minutes, the rest of this page explains how each one works.

Section I · The Editorial Review

The 47-point editorial review, broken into the four buckets our editors actually use.

The review is not a checklist of marketing questions. It is a structured instrument with weighted scoring, calibration rounds every six weeks, and a documented appeals process. Below: the four editorial buckets, what they cover, and what the editors look for in each.

Bucket A

Trust & Security

14 criteria · weight 32%

  • SOC 2 Type II attestation status & audit date
  • GDPR, CCPA, and HIPAA data handling posture
  • Incident history (last 36 months) & disclosure quality
  • Encryption at rest and in transit, key management
  • Authentication: SSO/SAML, SCIM provisioning, MFA
  • Vulnerability disclosure program & patching SLA
  • Penetration test cadence and remediation closure rate
  • Sub-processor list transparency
  • Data residency and regional availability
  • Backup durability, RPO, and RTO commitments
  • Admin audit log completeness
  • Breach notification mechanism (timed)
  • Third-party security ratings (BitSight, SecurityScorecard)
  • Vendor risk questionnaire responsiveness

Bucket B

Product & Operations

13 criteria · weight 28%

  • Documented uptime history (last 12 months)
  • Public status page maturity and incident comms
  • Product roadmap cadence and shipped commitments
  • Documentation depth and accuracy review
  • Onboarding time-to-first-value (measured)
  • Time-to-resolution benchmarks from support data
  • Release notes quality and changelog transparency
  • API coverage, OpenAPI spec, and rate-limit policy
  • Mobile and offline capability (where applicable)
  • Accessibility conformance (WCAG 2.2 AA)
  • i18n / l10n coverage
  • Deprecation policy and customer migration support
  • Sandbox or trial environment availability

Bucket C

Commercial & Pricing

11 criteria · weight 22%

  • Published pricing or documented rationale for opacity
  • Hidden-cost audit: implementation, support tiers, overages
  • Contract flexibility (month-to-month vs. annual lock-in)
  • Multi-year discount transparency
  • Mid-term price-change notification terms
  • Refund / pro-rata policy on cancellation
  • Procurement documentation (SOC 2 report, MSA, DPA)
  • Procurement cycle time & friction
  • Vendor financial health signals
  • Customer concentration risk indicators
  • Pricing change history on file (re-verified every 14 days)

Bucket D

Ecosystem & Reputation

9 criteria · weight 18%

  • Integration breadth against our published taxonomy
  • Quality of native vs. Zapier-style bridges
  • Verified customer references (3 per listing minimum)
  • Independent review sentiment across G2, Capterra, TrustRadius
  • Negative-review response quality (not just volume)
  • Community health: forums, Slack/Discord, GitHub
  • Public roadmap participation
  • Analyst recognition (Forrester, Gartner, IDC)
  • Press posture: how the vendor talks when things break

Total: 47 criteria across four buckets, weighted to 100%. A listing must clear an 70-point composite to publish, and a 78-point composite to qualify for the Dreyfus Top Tier (≥85). Reviews are conducted by named analysts, peer-reviewed, and archived.

Section II · The Ranking

The Dreyfus Trust Score: a transparent 0–100 credibility index, not a star average.

Star ratings collapse nuance. The Dreyfus Trust Score is a composite index, published in full, with weights you can audit. It is the score we publish on every listing, the score 1.2 million monthly readers use, and the score Gartner and G2 analysts cite in their own reports.

85–100Top Tier
70–84Verified
0–69Below threshold — not listed
0255075100

How the score is calculated.

The score is the weighted sum of the four editorial buckets above (Trust 32%, Product 28%, Commercial 22%, Ecosystem 18%), adjusted by a freshness factor that decays 0.5 points per day past the last 14-day re-verification. A listing that is overdue for re-verification visibly loses ground — the score does the work, not the editor.

What the score is not.

It is not a sentiment average. It is not crowdsourced. It is not weighted toward paying vendors. Sponsored placements appear on the page but are excluded from the index entirely, and the exclusion is audited annually by BDO.

Section III · The Numbers

Re-verified every 14 days. Audited annually. Published in the open.

A research desk earns its reputation on the numbers a reader can check. Here are the four that matter most, as of Q1 2025.

14d

Re-verification SLA

Every listing re-checked against the 47-point review on a rolling 14-day cycle. The audit trail is public.

62%

Editorial rejection rate

Of submissions received, 62% are returned to the vendor with a written explanation. We do not list on request.

52,417

Live, vetted listings

Indexed as of Q1 2025, across 142 countries. Each passes the 47-point review and the 14-day re-verification cycle.

1×/yr

Independent BDO audit

The sponsored-placement firewall, score calculation, and freshness SLA are reviewed annually by BDO USA.

Last updated 4 hours ago Source: A Great Portal editorial database, Q1 2025 View the full audit log →

Section IV · The Firewall

Why our rankings are never for sale.

A directory that ranks tools for a living has one job that is more important than any other: keeping the ranking honest. This is the policy we stake the brand on, written plainly, with no carve-outs for our larger customers.

A Great Portal operates a strict separation between editorial ranking and commercial placement. Vendors may purchase clearly badged placements — labelled "Sponsored" in a 12px tag with a contrasting background — that appear in fixed rails on category pages and in the weekly newsletter. Sponsored slots are never inserted into the editorial ranking, never influence the Dreyfus Trust Score, and never appear above a non-sponsored listing on the same page.

The firewall is operational, not aspirational. Sponsored placements are produced by a separate team, served from a separate code path, and tagged with a metadata flag that the BDO audit inspects every January. The 2024 audit confirmed that 0 sponsored placements affected editorial scores, and the report is linked from our vendor page. Vendors who attempt to trade spend for ranking are removed from the index and refunded, per our vendor terms.

We are not the largest directory on the web. We are the most vetted, and the vetting only matters if it is independent. That is the entire bargain.

— Maya Okafor, Editor-in-Chief, and the 38 editors, analysts, and engineers of A Great Portal.