How we test
Last updated
Every Shipworthy review is based on using the tool for real work. This page explains what that means.
The process
- Build something real. We use the tool on an actual project, not a toy demo, for long enough to hit its rough edges.
- Keep the evidence. Screenshots, code and timings come from that project. We keep the bill.
- Check the price. Pricing comes from the vendor's pricing page and carries the date we checked it.
- Write the verdict first. Every review opens with a verdict, a rating out of 5, and the pros and cons we actually observed.
- Re-test. Reviews show a "last tested" date and are re-tested every 90 days. Tools change fast.
Ratings
Ratings run from 0 to 5 in half-point steps. A tool earns "Shipworthy" when we would use it on our own production work; "Not yet" means it is not there today.
Ratings of 4 and above carry the "Shipworthy" badge; anything below 4 shows "Not yet".
| Rating | What it means in practice |
|---|---|
| 4.5 to 5 | Best in its category for most developers. We use it on production work and would recommend it without caveats beyond price. |
| 4 | Shipworthy. We would use it on production work, with known limits that we name in the review. |
| 3 to 3.5 | Useful, but not there yet. It needs steady correction, has gaps a competitor does not, or costs more than it saves. Fine for some people; the review says who. |
| 2 to 2.5 | Works, but we would not choose it. More time fixing its output than it saves, or serious lock-in or reliability problems. |
| 0 to 1.5 | Broken, unsafe or misleading. It loses work, does something risky without asking, or does not do what its pricing page claims. |
What we weigh
- Output quality. Does the result work and pass our checks with light edits? For AI tools, how often did we have to correct it, and how badly?
- Speed. Time to a working result, including setup and the time we spent steering.
- Price. What a real month of use cost us, not the headline plan, and whether the cheaper tier would have been enough.
- Lock-in. Can you take your code, data and settings elsewhere if you stop paying?
- Privacy and safety. What happens to your code, whether it trains on your data by default, and whether it asks before doing anything with side effects.
No single factor decides a rating. A tool can be excellent and still lose half a point for a price that only makes sense for teams.
What we do not do
- No stock images or invented benchmarks.
- No reviews of tools we have not used.
- No paid placements inside reviews. Sponsorship, when it exists, is labelled and kept separate.