AgentMastery Scoring Methodology
Version: 1.0
Last Updated: October 2025
Status: Active
Our Scoring System
Every AI tool in our directory receives a composite score (0-100) based on four equally weighted dimensions:
1. Value (0-100)
- Cost-effectiveness: Price relative to features delivered
- ROI potential: Measurable impact on productivity or revenue
- Pricing transparency: Clear, accessible pricing without "Contact Sales" barriers
- Free tier quality: Usability of free/trial tier if available
Weight: 25%
2. Quality (0-100)
- Accuracy: Output quality, error rates, hallucination frequency
- Reliability: Uptime, consistency, bug frequency
- Output excellence: User satisfaction with generated content/results
- Feature completeness: Delivered features vs. promised features
Weight: 25%
3. Adoption (0-100)
- Market momentum: Growth trajectory over 12 months
- User base size: Total active users (verified when possible)
- Community engagement: Forum activity, user-generated content
- Enterprise traction: Fortune 500 customers, case studies
Weight: 25%
4. User Experience (0-100)
- Ease of use: Time to first value for new users
- Learning curve: Documentation quality, onboarding flow
- Interface quality: Design, navigation, accessibility
- Support quality: Response time, help center completeness
Weight: 25%
Composite Score Formula
Composite Score = (Value × 0.25) + (Quality × 0.25) + (Adoption × 0.25) + (UX × 0.25)
Data Sources
- First-party testing: We create accounts and test core workflows
- User reviews: G2, Capterra, Product Hunt, Reddit
- Public data: GitHub stars, Twitter followers, job listings
- Vendor data: Pricing pages, documentation, changelogs
- Third-party benchmarks: Independent accuracy tests, uptime monitors
Scoring Frequency
- Weekly snapshots: Every Monday at 00:00 UTC
- Continuous monitoring: Pricing changes, feature launches
- Manual reviews: Quarterly deep-dive for top 20 tools
Editorial Independence
- Scores are calculated algorithmically
- Affiliate relationships do not influence scores
- Vendors cannot purchase ranking improvements
- All scoring changes are logged with timestamps
Change Log
v1.0 (October 2025):
- Initial methodology published
- 47 tools scored
- 9 categories covered
Questions?
Email: methodology@agentmastery.ai
© 2025 AgentMastery.ai | Licensed under CC BY 4.0