This is the way I like to test plugins. It’s an ever evolving endeavor that takes a lot of time, which means that I only do it with a handful of plugins.
Here is what goes into the score:
Scoring Factors
The overall score is based on a weighted average of 5 scoring factors:
- Technical Performance (25%).
- User Experience (20%).
- Sound Quality (25%).
- Economic Factors (10%).
- Compatibility and Plugin Formats (10%).
- Plugin-Specific Scoring Factors (10%).
Each factor is given a weight based on its level of importance:
Understanding the Scoring system
I decided to use a 1-10 scoring system where 10 represents an exceptional score, a 8.5 represents an acceptable score, and anything below 7 is unacceptable.
- 10 – Exceptional score.
- 9.5-9.9 – Excellent score.
- 9.0-9.4 – Very good score, only some minor complaints.
- 8.5-8.9 – Good score. Good enough for most users, even professionals.
- 8.0-8.4 – Fair score. May not suit most peoples’ needs.
- 7.5-7.9 – Poor Score.
- 7.0-7.4 – Very poor score.
- <7.0 – Don’t buy (Poor in every regard).
Scoring Methodology
The methodology will keep evolving over time, but right now I do all of the technical tests I can (I test the plugins with PluginVal, AuVal, open up instances of the plugin on as many tracks as possible until the DAW can’t handle it anymore be it by crashing or when playback starts adding crackle noise, use PluginDoctor to check oversampling and latency performance, etc.).
I try to keep all of the testing and scoring as objective as possible, even though some of the usability and user interface scoring might be inevitably more subjective.
Test System and Software
- M2 Mac Mini with 8gb of ram.
- DAWs: Reaper, Studio One.
Note: I have two additional PCs (one extremely high-end with a 12-core processor, 64gb or RAM, etc., and a 10-year old laptop, none of which represent the kind of computer most home producers use, which is why the more logical choice is the Mac Mini with the M2 processor).
Technical Performance
Here I try to “stress-test” the plugins, test if oversampling can completely remove aliasing in a sine sweep test, overall latency of the plugin, and I also take into account some features, such as if they have sidechain capabilities, and others, to give them the score.
CPU consumption is the most important test in this section because in a large project with multiple plugins, this starts to add up and can introduce crackle or even crash the DAW.
What does PluginVal do: It runs a series of processing tests, such as checking multiple sample rates and block sizes, it checks all of the parameters, and some other technical things.
What does AUVal do: It runs a series of tests to validate the plugin, and LogicPro and Garageband run this test every time a new plugin is loaded, and the plugin can only be loaded if it passes.
Note about PluginVal and AUVal: 99.9% of plugins pass this test, but it’s included to find the ones that may not since this means a big red flag.
CPU Performance results per DAW (No Oversampling):
This test is done with a sample rate of 44.1khz and with a 512 block size in every DAW.
Let’s use my own THR MB-Control as an example:
Total Number of tracks per DAW
- Reaper: 324 tracks.
- Studio One: 290
Note: The CPU is given based on how it fares against the competition, in this case, multiband compressors.
| Criteria | Value | Score |
| PluginVal or AUVal (“Pass” = 10, “Fail” = 1) | Pass | 10 |
| Oversampling & Aliasing | 4x & 8x OS (removes aliasing). | 10 |
| Latency | 3ms | 10 |
| Latency with Oversampling (if it has the option) | 5.9ms | 10 |
| Sidechain Compatible | Yes | 10 |
| CPU Consumption & Scalability (Max Simultaneous tracks…) | 307 | 10 |
| Average | 10 |
User Experience
Here I include multiple factors which combine the initial experience of downloading and installing the plugin, all the way to actually using it.
| Criteria | Value | Score |
| Installation, DRM and Licensing Restrictions | Simple Installer (no extra steps) | 10 |
| Free Trial Availability | 7-days | 10 |
| Interface Look and Feel, Intuitiveness, Responsiveness | Very responsive and intuitive | 10 |
| Ease of Use | Beginner friendly | 10 |
| Documentation and Learning Resources | Written Guide | 9 |
| Update and Support Policy | Lifetime Updates | 10 |
| Average | 9.83 |
Sound Quality
| Criteria | Value | Score |
| Sound Fidelity, Sound Options | Transparent sound | 10 |
| Artifacts/Distortion | No Artifacts or Distortion | 10 |
| 10 |
Economic Factors
Economic factors will take into account the price of the competition. If there are any other similar plugins with the same features and overall quality at a much lower price, the score will be lower.
| Criteria | Value | Score |
| Price | $49.00 | |
| Min Price of Similar Plugins | $29.99 | |
| Max Price of Similar Plugins | $199.00 | |
| Average | 8.99 |
Compatibility and Formats
For formats, a 10 will be given to plugins that are available as VST3, AU, and AAX in 32- and 64-bit since this covers 99% of needs.
KONTAKT, RTAS LV2, etc. are nice extras to have.
| Category | Value | Score |
| Formats | VST3, AU, AAX, 32-bit & 64-bit | 10 |
| Compatibility | Windows, MacOS, Linux | 10 |
| Average | 10 |
Plugin-specific features
For example, if it’s a compressor, then I will be taking into account if it has auto makeup gain, auto attack and release, as well as other features.
| Criteria | Value | Score |
| Auto Makeup Gain | Maintains perceived loudness almost perfectly. | 9.5 |
| Auto Attack & Release | Yes | 10 |
| Mid/Side Processing | No | 7 |
| RMS/Peak Detection Control | No | 7.00 |
| MIDI Learn | No | 7 |
| Average | 8.10 |
Final Score
This is how the final score would get calculated with the different weights:
| Average Score: | 9.68 |
| Technical Performance | 10 |
| User Experience | 9.83 |
| Sound Quality | 10 |
| Economic Factors/Price | 8.99 |
| Compatibility/Formats | 10 |
| Plugin-specific features | 8.10 |