Product benchmarks

Measure the effectof context.

LoCoMo measures whether PowerContext retrieves the right evidence from long conversations. SWE-bench Pro measures whether project context helps Codex resolve more repository issues.

Review both evaluations ↓
LoCoMo accuracy and search latency1,540 scored questions
SystemAccuracySearch p95
PowerContext90.78%1.38 s
PowerMem87.79%1.44 s
Full context52.9%17.12 s

LoCoMo: retrieve the right context

LoCoMo asks questions about long conversations split across sessions. PowerContext must find the relevant conversation evidence before producing an answer. This run covers 1,540 scored questions in categories 1 through 4.

Question mix1,540 total
Single-hop841
Temporal321
Multi-hop282
Open-domain96
PowerContext resultCategories 1 through 4

90.78% answer accuracy

PowerContext answered 1,398 of 1,540 questions correctly, 37.88 percentage points above full-context prompting. Search p95 was 1.38 seconds, with about 1.65k answer tokens per question.

SWE-bench Pro: turn context into working patches

To measure the effect of PowerContext, we ran the same Codex configuration twice on all 731 public v2 tasks. The two arms differed only in whether PowerContext was enabled.

SWE-bench Pro paired runsPassing the benchmark tests counts as resolved
PowerContext OFF602 / 731 · 82.35%
PowerContext ON634 / 731 · 86.73%
+4.38 pp · 32 more tasks

This is a paired run on a pinned task set, not an official SWE-bench Pro submission. Agent runs are stochastic, so the scores describe these two runs only.

Comparison with published results

LoCoMo results use different readers, judges, and answer-matching rules. SWE-bench Pro results come from the official Public leaderboard, so the two tabs require different interpretations.

Data checked August 31, 2026

15 systems · All 1,540 scored questions

  1. 1ZepVendor run94.70%
  2. 2EverMemOSThird-party run94.50%
  3. 3XMDBVendor run93.20%
  4. 4TrueMemory ProOpen harness93.00%
  5. 5Mem0Vendor run92.50%
  6. 6PowerContextProject run90.78%
  7. 7HonchoVendor run89.90%
  8. 8DakeraReproducible vendor run88.20%
  9. 9PowerMemProject run87.79%
  10. 10MemvidOpen harness85.65%
  11. 11GenesysCertified vendor run85.55%
  12. 12EngramThird-party run84.50%
  13. 13MemHQOpen harness83.20%
  14. 14Logica MindOpen harness72.50%
  15. 15SupermemoryThird-party run65.40%

Readers, judges, and answer-matching rules differ, so this is not an official ranking.

Evaluation methods and sources

Method references and reproducibility resources for both evaluations.

LoCoMo

SWE-bench Pro

Task set and graderscaleapi/SWE-bench_Pro-os

Use PowerContext in your workflow

Start with a supported agent or connect an application through the HTTP API. The repository contains the runtime, integrations, and evaluation tools.

Open the GitHub repository