Skip to content

[Experimental] Classify each SMT-query#966

Draft
jcp19 wants to merge 8 commits intomasterfrom
claude/extend-prover-interface-n7IQJ
Draft

[Experimental] Classify each SMT-query#966
jcp19 wants to merge 8 commits intomasterfrom
claude/extend-prover-interface-n7IQJ

Conversation

@jcp19
Copy link
Copy Markdown
Contributor

@jcp19 jcp19 commented Apr 11, 2026

This PR is not meant to be merged or reviewed as is. Architecturally, the way he record the per-query data might not be the most adequate; perhaps the symbex logger would be a better place for that.

This PR introduces support for recording and classifying SMT queries (assertions and checks) issued during verification. Each query may be classified as one of the following:

  • Consistency: Well-formedness checks (permissions, injectivity)
  • Heap: Heap access and permission correctness
  • FunctionalCorrectness: User-visible proof obligations (pre/postconditions, assertions)
  • Axiomatization: Axiom consistency checks
  • PathInfeasibility: Branch feasibility and smoke checks
  • Unknown: Unclassified queries

This feature is enabled by the CLI option --recordProofQueries, which takes the path to the file storing the results of the analysis as an argument.

Notable Implementation Details

  • Query recording is only performed when Verifier.config.recordProofQueries is defined
  • Source position information is extracted from AST nodes; unknown positions default to NoPosition
  • CSV export handles missing metadata gracefully with "?" placeholders

claude added 6 commits April 10, 2026 22:03
…iming

Adds `ProofQueryKind` (5 categories: Consistency, Heap, FunctionalCorrectness,
Axiomatization, PathInfeasibility) and extends `Decider.assert`, `Decider.check`,
and `Decider.checkSmoke` to accept `kind`, `pos`, and `member` parameters with
defaults so all existing call sites compile unchanged.

When `--recordProofQueries` is enabled, each SMT query is recorded into a
thread-safe `ProofQueryCollector` with its kind, member name, source position,
wall-clock duration (ms), and success flag. Results can be exported to a CSV
file via `--proofQueryCsvFile <path>`.

All ~59 call sites across rules, resources, and decider have been updated to
pass the appropriate kind, source position, and current member name.

https://claude.ai/code/session_01Guw2LiiZWESXZ5toNLpbQ8
- Evaluator.scala: ePerm is Option[ast.Exp], use .map(_.pos).getOrElse(NoPosition)
- QuantifiedChunkSupport.scala: resource not in scope in assertReadPermission,
  use permsExp.map(_.pos).getOrElse(NoPosition) instead

https://claude.ai/code/session_01Guw2LiiZWESXZ5toNLpbQ8
…x duration formatting

- --recordProofQueries now takes the output CSV path directly; recording is
  enabled iff the option is provided (removes the separate --proofQueryCsvFile flag)
- Duration values in the CSV are now formatted as fixed-point decimal with 3
  decimal places instead of scientific notation

https://claude.ai/code/session_01Guw2LiiZWESXZ5toNLpbQ8
…o State

ConsumptionResult.apply, NonQuantifiedPropertyInterpreter, and the QCS
heuristic/findChunk functions did not receive a State and so always emitted
member = None in recorded queries.  Fix by:
- Adding member: Option[String] = None to ConsumptionResult.apply and its callers
- Adding member to NonQuantifiedPropertyInterpreter's constructor and all 5
  instantiation sites (ChunkSupporter, MoreCompleteExhaleSupporter,
  StateConsolidator ×2, QuantifiedChunkSupport)
- Adding member to qpAppChunkOrderHeuristics, singleReceiverChunkOrderHeuristic,
  and findChunk (trait + impl) in QuantifiedChunkSupport; threading it through
  StateConsolidator.singleMerge → findMatchingChunk → findChunk

https://claude.ai/code/session_01Guw2LiiZWESXZ5toNLpbQ8
…nd annotation

Callers can pass description = Some("...") at specific call sites for extra
clarity; all existing call sites default to None.

https://claude.ai/code/session_01Guw2LiiZWESXZ5toNLpbQ8
@jcp19 jcp19 marked this pull request as draft April 11, 2026 21:11
@jcp19 jcp19 changed the title Add proof query recording and classification infrastructure [Experimental] Classify each SMT-query Apr 11, 2026
claude added 2 commits April 12, 2026 19:25
- deciderAssert: do not record queries where isKnownToBeTrue short-circuits
  (no prover call was made, so timing and results are meaningless noise)
- ConsumptionResult: derive pos from the permission expression argument
- qpAppChunkOrderHeuristics: add pos parameter, thread resource.pos from caller
- singleReceiverChunkOrderHeuristic: add pos parameter, thread resourceAccess.pos from caller

Three internal helpers (findChunkWithProver, NonQuantifiedPropertyInterpreter.buildCheck,
QuantifiedChunkSupport.findChunk) still emit NoPosition because they operate at the
pure term level with no AST resource node in scope.

https://claude.ai/code/session_01Guw2LiiZWESXZ5toNLpbQ8
findChunk (ChunkSupportRules trait + chunkSupporter impl) and the private
findChunkWithProver were the only remaining call paths without member or pos.
All five callers have State in scope and can supply both:

- consumeGreedy: passes s.currentMember.map(_.name)
- lookupGreedy: passes member + resource.pos
- MoreCompleteExhaleSupporter (x2): passes member + resource.pos
- StateConsolidator.findMatchingChunk: passes the already-threaded member

findChunkWithProver now also passes kind = ProofQueryKind.Heap, eliminating
the remaining Consistency-default misclassification for alias checks.

https://claude.ai/code/session_01Guw2LiiZWESXZ5toNLpbQ8
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants