Customer conversation · just ended

Your company
just learned something.

The question is whether it becomes another forgotten note or the right work.

Source preservedNorthstar onboarding review32:18 · 3 participants · transcript ready
01

A customer call just ended. One sentence could change what the team ships next.

Evidence, in the open

The intelligence.
The evidence behind it.

Useful AI needs more than a convincing answer. We evaluate task relevance, engineering readiness, and code implementation separately—and publish the limits alongside the results.

Archived development results

Latest Nebius results and live workflow checks. The dated results below remain available for comparison.

01 / Relevance

Task relevance

Keep real tasks visible. Let evidence decide what belongs in the engineering queue.

14/14

investigated cases passing all checks

Synthetic development testgpt-5.4-2026-03-05

Small synthetic set, not measured real-world accuracy.

02 / Readiness

Engineering enrichment

Recognize when a request has enough context to become engineering work—and when it does not.

24/24

correct readiness decisions

Synthetic development testgpt-5.4-2026-03-05 · low reasoning

Readiness is not plan correctness. Independent plan-quality review is still pending.

03 / Implementation

Datacurve DeepSWE

Test code changes against a separate, externally defined coding benchmark.

56.6%

64/113 · full-cohort tasks passed

Separate Compress Cloud runArchived Compress Cloud coding run

A separate Compress Cloud run, not a meeting-to-code success rate.

Explore full results and methodologyBaselines, limitations, and what we still need to measure

Separate measurements of task relevance, plan preparation and coding. Development fixtures are not independently labeled real-world accuracy. These scores must not be combined into an end-to-end success rate.

Task relevance

investigated cases passing all checks
14/14100.0%
Real tasks retained
14/14100.0%
Incorrect queue admissions
0/140.0%
Source-only baseline passing all checks
8/1457.1%
  • 14 agent-authored cases; live reviewer with frozen history, code and web retrieval. Planning is an invocation spy, not an executed plan.
  • A real task may remain visible without belonging in our engineering queue. Human acceptance and execution approval remain separate.

Engineering enrichment

correct readiness decisions
24/24100.0%
Completed cases
24/24100.0%
False-ready decisions
0/120.0%
Automatic wording checks
23/2495.8%
Plain-model readiness baseline
24/24100.0%
Plain-model wording baseline
20/2483.3%
Raw reviewer contrast checks
8/1080.0%
  • 24 agent-authored ambiguous tasks (16 original + 8 transfer). Readiness is not plan correctness. The separate raw reviewer scored 8/10, incorrectly rejecting two valid briefs.
  • Same source evidence and per-call output cap; unequal total budget: up to 4 Actionairy calls versus 2 baseline calls. No claim of competitor superiority.
  • Independently reviewed usable-plan accuracy and human correction time are not yet measured.
  • Failed calls remain in the full denominator. Automatic wording checks are lexical, not semantic grading. No independent human usable-plan score is available.

Datacurve DeepSWE

full-cohort tasks passed
64/11356.6%
  • 2 tasks remained ungradeable after bounded regrading. They receive no pass credit and stay in the full-cohort denominator.
  • Sealed candidate patches and pinned verifier contexts. This is a separate coding evaluation, not an end-to-end Actionairy meeting-to-code test or a current-model leaderboard claim.

What these results do not measure

Meeting extraction: independently labeled capture precision and recall are not yet available. Transcript-upload acceptance is a user-journey check, not an accuracy estimate.

This archived report does not measure independently reviewed plan quality, human correction time, or a representative meeting-to-code success rate. Historical results do not establish current-model performance or competitor superiority.

The DeepSWE mark identifies Datacurve’s benchmark. Our result is a Light Reach evaluation, not a Datacurve endorsement or an official leaderboard entry.