analyzer is my trading record analysis project. trader switched the live trading records to a new export format v2, backfilling the historical data completely. I handed the schema review and full verification over to Opus to run autonomously. Reading the process logs it left behind, when I reached the end, this was what I said to myself:

The entire process was handled by Opus itself. However, after the whole process finished running, it makes me somewhat doubt Opus’s correctness in handling things.

Let me make this clear first: I was not present, nor did I independently verify it. The evidence this article stands on is the logs Opus wrote itself and the content of trader’s replies. But precisely because it is a self-account, those four moments are even more worth looking at—every single one in the logs shares the exact same shape: stopping as soon as the evidence appeared to support its assumption.

1. A Grep Match Taken as Verification

Opus raised 10 points during the schema review, among which A-3 was later proven wrong. It ran a grep for partial in trader’s documentation, found partial_1325, compared it against partial written in the new schema’s flattened list, and concluded:

The export layer copies the DB values exactly as they are, so the DB is the source of truth—at least one item in this schema list is wrong.

trader’s reply pointed out: partial (derived by the pipeline) and partial_1325 (from the source DB) are two distinct values from two adapters, not a typo; the real issue with the schema was mixing enums from two sources into a single table, with the source side also missing 8 values. Opus went back to check the # skip_reason closed enum (must match trade_store / plan D2) block in the strategy module, confirmed that trader was right, and explicitly admitted its mistake in the final report.

The error attribution it gave itself: it stopped after running a grep and hitting a string that supported its hypothesis, without asking, “Could this enum be per-source?”. A match only proves that a string exists somewhere; it does not prove its relationship with another string.

2. A Well-Designed Verification Rendered Impossible

One of the verification conditions was “the pipeline does not regress.” Opus’s original plan was the most direct kind: running a direct diff between v1 and v2 for the same day. But when it actually went to fetch the baseline:

$ git ls-files data/…/live/ | wc -l
       0
$ cd cronjobs && git ls-files data/…/live/ | wc -l
       0

data/ is a symlink, neither repo tracked live/, and trader’s full backfill was an in-place overwrite—at the exact moment of comparison, the original v1 content no longer existed anywhere. The well-designed verification discovered upon execution that there was nothing left to compare against.

It had to resort to three pieces of indirect evidence to evaluate it as PASS: the top-level keys of v1 were preserved, the records and summary values were 8/8 identical, and _records_from_pipeline directly consumed the summary of v1, making them structurally homologous. Before making the judgment, it paused for a moment: should this be evaluated as “unable to verify” instead of PASS? It finally passed it, because the third item was structural homology, not a statistical coincidence. However, it explicitly wrote “there is no diff behind this PASS” into the verification document, rather than quietly letting it through: if the next person did not know the baseline was gone, they would have assumed a comparison backed that PASS.

3. Almost Wronging an Honest Disclosure

trader proactively disclosed: the realized_pnl for four trading days was skewed low because the settlement leg fill_price defaulted to 0.0. Opus went to verify this, and the first thing it scanned was records[].legs:

=== Legs where fill_price == 0.0 ===
  None

Zero entries. Its immediate thought was “he listed extra dates”, preparing to write this into the verification as a finding. Where was the mistake: records[].legs only carries entry legs, while the 0.0 values belong to the exit/settlement legs, which reside in exit.fills at the v1 top level. Scanning the correct field the second time finally matched them up—the number of legs across the four dates was perfectly consistent, with no omissions and no false alarms. trader’s disclosure was correct, and it listed exactly the right amount.

There was an even more hidden one earlier: when printing the exit block for the first time, it only truncated to the first 400 characters, saw real prices, and assumed upfront that “these few days look normal.” Conclusions read from truncated outputs are false, and the truncation was added by Opus itself.

4. 24 Consistent Files, Missing 14 Days

The final one is the most valuable segment in the entire log. trader’s completion report stated “24 files completely backfilled,” and there were indeed 24 files on the disk—the numbers were internally consistent. Opus dumping the sessions table was originally just to get the true values for two specific dates to compare against the export files, but an extra row appeared in the output:

id  trade_date  day_type  entered  observed_cost  trade_seq
1   <Date>      normal    0        <Masked>

This is the seed row of the guard sequence—even the schema document itself had mentioned it. Going to the disk to check data/live/<日期>.json: this file did not exist. Only then did this trigger a complete cross-check of DB coverage (distinct dates in the source table vs. actual files): the source had 31 days of records, only exported 24 files, and was missing 14 days.

Had it not been for that time dumping the entire table for a different purpose, these 14 days would not have been discovered—the report said 24, the disk had 24, and every single number matched up. What was missing was “whether the number 24 itself was correct,” and that could only be known by checking against the source. Opus’s lesson verbatim:

Internal consistency of numbers in the report does not mean the numbers are correct. Catching what “should be there but isn’t” requires enumerating from the source, not tallying from the artifacts.

The Same Shape, The Same Antidote

All four moments were instances of “stopping as soon as the evidence appeared to support itself.” And the four antidotes are actually the exact same thing—returning to first principles:

• The antidote for a grep match is to question the structure instead of the surface string: “Is this enum per-source?”
• The antidote for truncated outputs is to look at the entire set, not a sample;
• The antidote for 24 internally consistent files is to derive completeness from the source, not tally from the artifacts;
• The antidote for an evaporated baseline is to make “how to prove this afterward” the primary constraint right during the planning phase.

This is not a case of “agents are unusable.” The reason I can see these four moments is precisely because Opus wrote every single one of them into the log; the admission of mistake for A-3 and the cross-check of the 14 days were ultimately also completed and truthfully recorded by Opus itself. But this honesty simultaneously tells me: when no one was watching, it stopped at “looks correct” four times. The responsibility of verification does not disappear just because it was handed off to an agent.

The Direction I Plan to Try: Making It Use First Principles During Planning

This direction was not something I had the moment I read the logs. After finishing the logs, all I had was that initial doubt mentioned at the beginning—I was clueless. It was only later, after organizing and generalizing the four moments into “the same shape,” that the antidote surfaced alongside that shape:

Good thing I did this organizing and generalizing today, so I could come up with a possible direction for correction. Otherwise, I originally didn’t have a clue.

The direction itself is very plain, neither implemented nor verified yet:

1. Define completeness starting from the source of truth during planning—enumerate the full set from the source, and do not accept “internal consistency of artifacts” as verification evidence.
2. For verification conditions, ask first “how to prove it from the source of truth afterward”, making verifiability the primary constraint of the plan—saving the baseline that needs saving before the overwrite happens.
3. Question the structure during investigation (per-source? which layer? which field?); treat grep matches and truncated outputs only as clues, not as conclusions.

Two Small Footnotes in the Logs

Two things are unrelated to the main storyline, but both made me trust this log a bit more. Opus promised in the document to “rerun the same verification script to confirm after the fix,” but realized right after writing it that the script was in the session’s scratchpad, which would be gone once the session ended—rendering the promise empty on the spot. It moved the script into tools/verify_live_export_v2.py and explicitly wrote down the reason. Another thing: regarding the design of pnl_summary(exclude_suspect=True) which defaults to excluding suspect days, it considered this borderline (mixing it into the average would silently skew the statistics), so it patched in three exit points and stated, “I am still not sure even now if this judgment is correct.”

Writing uncertainty as uncertainty, and turning an empty promise into a real one—paragraphs like these are exactly the reason I treat its logs as evidence.

Conclusion

Consistency does not mean correctness. Catching what “should be there but isn’t” requires enumerating from the source, not tallying from the artifacts—this is Opus’s own conclusion, and it is also the one sentence I most want to leave behind after reading the entire log.

Adding one more layer of my own: delegating does not equal absolution, and doubt itself is not a direction. I only had doubt when I finished reading the logs; it was organizing and generalizing that turned that doubt into something actionable. “Having read the logs” does not equal “having done the homework”—the post-task homework for the delegator is organizing, not just reading.