Skip to content

refine rule criteria, move mode and message to db config - #1396

Merged
iceljc merged 1 commit into
SciSharp:masterfrom
iceljc:features/refine-rule-criteria
Aug 12, 2026
Merged

refine rule criteria, move mode and message to db config#1396
iceljc merged 1 commit into
SciSharp:masterfrom
iceljc:features/refine-rule-criteria

Conversation

@iceljc

@iceljc iceljc commented Aug 12, 2026

Copy link
Copy Markdown
Collaborator

No description provided.

@qodo-code-review

Copy link
Copy Markdown
Contributor

PR Summary by Qodo

Refine agent rule criteria and move mode/message into persisted rule config

✨ Enhancement ⚙️ Configuration changes 🕐 20-40 Minutes

Grey Divider

AI Description

• Persist per-rule criteria mode and user message in agent rule configuration.
• Allow rule config to override trigger-level criteria evaluator selection.
• Render rule messages as templates using conversation states before dispatching.
Diagram

graph TD
  A["RuleEngine"] --> B[("Agent Rule (DB)")] --> C["RuleCriteria (Mode+Text)"] --> D{{"Resolve evaluator"}} --> E["LlmCriteriaEvaluator"]
  D --> F["PythonScriptEvaluator"]
  A --> G["ConversationService"] --> H["TemplateRender"]
  A --> I["RuleTriggerOptions (Mode)"] --> D

  subgraph Legend
    direction LR
    _svc["Service"] ~~~ _db[("Database")] ~~~ _dec{{"Decision"}}
  end
Loading
High-Level Assessment

The following are alternative approaches to this PR:

1. Keep backward-compatible alias fields (Type + Config)
  • ➕ Reduces migration risk for existing stored rules and callers
  • ➕ Allows gradual rollout across services/plugins
  • ➖ Increases model complexity and ambiguity (multiple sources of truth)
  • ➖ Requires deprecation plan and cleanup later
2. Use a strongly-typed enum for criteria mode
  • ➕ Prevents mode string drift (e.g., code vs python_script) at compile time
  • ➕ Improves discoverability and validation
  • ➖ Less extensible for plugins that want custom modes without core changes
  • ➖ May require custom JSON (de)serialization rules
3. Centralize message templating in ConversationService
  • ➕ Single place for templating behavior across all message sources
  • ➕ Easier to enforce consistent escaping/validation
  • ➖ Harder to keep rule-specific rendering behavior isolated
  • ➖ May introduce templating side-effects for non-rule messages

Recommendation: The PR’s direction (persisting per-rule mode/message and allowing rule-level override) is sound and increases configurability. Consider adding a short-term compatibility layer (alias fields or migration) if existing persisted rules/callers still send/expect CriteriaOptions.Type or AgentRule.Config, and document the new expected mode values (e.g., python_script) to avoid runtime evaluator resolution failures.

Files changed (8) +99 / -687

Enhancement (2) +74 / -663
AgentRule.csPersist rule message and criteria (with mode override) +20/-4

Persist rule message and criteria (with mode override)

• Replaces the old Config object with explicit Message and Criteria fields on AgentRule. Introduces RuleCriteria with a Mode (evaluator selection) and Criteria text, allowing per-rule override of trigger-provided mode.

src/Infrastructure/BotSharp.Abstraction/Agents/Models/AgentRule.cs

RuleEngine.csPrefer per-rule criteria mode and template-render outbound messages +54/-659

Prefer per-rule criteria mode and template-render outbound messages

• Resolves criteria evaluator using trigger options Mode, but allows rule.Criteria.Mode to override. Adds message selection (rule.Message fallback to trigger text) and renders the message via ITemplateRender using conversation states before sending. Also removes a large block of commented legacy graph execution code.

src/Infrastructure/BotSharp.Core.Rules/Engines/RuleEngine.cs

Refactor (3) +5 / -10
RuleCriteriaContext.csRemove trigger text from criteria evaluation context +0/-5

Remove trigger text from criteria evaluation context

• Removes the Text field from RuleCriteriaContext, leaving criteria Options and States as the evaluation inputs carried through the engine.

src/Infrastructure/BotSharp.Abstraction/Rules/Models/RuleCriteriaContext.cs

CodeCriteriaEvaluator.csAlign script evaluator identifier with python_script mode +1/-1

Align script evaluator identifier with python_script mode

• Updates the evaluator Type identifier to BuiltInRuleCriteria.PythonScript so RuleEngine can resolve it via the new mode value.

src/Infrastructure/BotSharp.Core.Rules/Criteria/Code/CodeCriteriaEvaluator.cs

LlmCriteriaEvaluator.csRead criteria text from RuleCriteria instead of legacy config +4/-4

Read criteria text from RuleCriteria instead of legacy config

• Switches LLM criteria input building to use rule.Criteria (RuleCriteria) rather than rule.Config. Keeps existing render-data behavior while aligning with the new persisted model.

src/Infrastructure/BotSharp.Core.Rules/Criteria/Llm/LlmCriteriaEvaluator.cs

Other (3) +20 / -14
BuiltInRuleCriteria.csRename built-in code criteria to python_script +1/-1

Rename built-in code criteria to python_script

• Updates the built-in criteria constant from "code" to "python_script" to better reflect the supported evaluator implementation.

src/Infrastructure/BotSharp.Abstraction/Rules/Constants/BuiltInRuleCriteria.cs

RuleTriggerOptions.csSwitch criteria selector from Type to Mode +1/-1

Switch criteria selector from Type to Mode

• Renames CriteriaOptions.Type to a nullable Mode, shifting evaluator selection terminology and enabling rule-level override precedence.

src/Infrastructure/BotSharp.Abstraction/Rules/Options/RuleTriggerOptions.cs

AgentRuleMongoElement.csUpdate Mongo persistence for message + criteria (with mode) +18/-12

Update Mongo persistence for message + criteria (with mode)

• Replaces RuleConfigMongoModel with RuleCriteriaMongoModel and adds Message persistence. Updates ToMongoElement/ToDomainElement mappings to align Mongo storage with the new AgentRule schema.

src/Plugins/BotSharp.Plugin.MongoStorage/Models/AgentRuleMongoElement.cs

@qodo-code-review

Copy link
Copy Markdown
Contributor

Code Review by Qodo

🐞 Bugs (4) 📘 Rule violations (0) 📜 Skill insights (0)

Grey Divider


Action required

1. Rule mode needs options 🐞 Bug ≡ Correctness
Description
RuleEngine resolves an evaluator from rule.Criteria.Mode but only evaluates criteria when
options.Criteria is non-null, so rule-configured criteria can be silently ignored and the rule
executes unconditionally.
Code

src/Infrastructure/BotSharp.Core.Rules/Engines/RuleEngine.cs[R55-56]

+            var evaluator = ResolveCriteriaEvaluator(rule.Criteria?.Mode) ?? criteriaEvaluator;
+            if (evaluator != null && options?.Criteria != null)
Evidence
The engine computes an evaluator from the rule’s stored mode, but the evaluation block requires
options.Criteria to be present, so rule-owned mode is ignored when callers omit criteria options.

src/Infrastructure/BotSharp.Core.Rules/Engines/RuleEngine.cs[53-73]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
RuleEngine selects an evaluator from `rule.Criteria?.Mode`, but the criteria evaluation block is gated by `options?.Criteria != null`. This prevents rule-owned criteria from being enforced unless the caller also sends a Criteria options object.

## Issue Context
This directly contradicts the comment that “the rule's own mode wins … without the caller knowing”, and it can cause rules to trigger even when their criteria should block them.

## Fix Focus Areas
- src/Infrastructure/BotSharp.Core.Rules/Engines/RuleEngine.cs[53-69]

### Suggested implementation direction
- Change the guard to evaluate criteria when `evaluator != null`.
- When `options?.Criteria` is null, pass a default `CriteriaOptions` into `RuleCriteriaContext` (so evaluators can still read settings as default/null) e.g. `Options = options?.Criteria ?? new CriteriaOptions()`.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


2. Missing default criteria mode 🐞 Bug ≡ Correctness
Description
CriteriaOptions no longer has a default mode, and ResolveCriteriaEvaluator returns null for blank
mode; when callers omit the mode (or still send legacy fields), the engine skips EvaluateCriteria
and still triggers the rule.
Code

src/Infrastructure/BotSharp.Abstraction/Rules/Options/RuleTriggerOptions.cs[26]

+    public string? Mode { get; set; }
Evidence
CriteriaOptions.Mode is nullable and has no default initializer, and RuleEngine’s resolver
explicitly returns null for blank mode; this combination means criteria evaluation can be bypassed
when mode isn’t supplied.

src/Infrastructure/BotSharp.Abstraction/Rules/Options/RuleTriggerOptions.cs[20-33]
src/Infrastructure/BotSharp.Core.Rules/Engines/RuleEngine.cs[79-88]
src/Infrastructure/BotSharp.Core.Rules/Engines/RuleEngine.cs[32-41]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
`CriteriaOptions` changed from a non-null `Type` with a default value to a nullable `Mode` with no default. In RuleEngine, a null/blank mode resolves no evaluator, and criteria evaluation is skipped while rule triggering continues.

## Issue Context
Previously, `CriteriaOptions.Type` defaulted to the built-in code criteria type. Now, `Mode` can be omitted by callers (including older clients), and the system can silently stop applying criteria.

## Fix Focus Areas
- src/Infrastructure/BotSharp.Abstraction/Rules/Options/RuleTriggerOptions.cs[20-33]
- src/Infrastructure/BotSharp.Core.Rules/Engines/RuleEngine.cs[32-41]
- src/Infrastructure/BotSharp.Core.Rules/Engines/RuleEngine.cs[79-88]

### Suggested implementation direction
Choose at least one:
1) Give `CriteriaOptions.Mode` a safe default (e.g. `BuiltInRuleCriteria.Llm` or `BuiltInRuleCriteria.PythonScript`, depending on intended behavior).
2) Add backward-compatible JSON aliasing (e.g., keep an obsolete `Type` property mapped from `"type"` and translate it into `Mode`).
3) If Mode is absent, fail closed (do not trigger) when criteria evaluation is expected.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


3. Criteria identifier renamed 🐞 Bug ☼ Reliability
Description
The built-in criteria identifier changed from "code" to "python_script" and the evaluator now
advertises only python_script; any persisted/requested mode value "code" will no longer resolve,
causing criteria to be skipped.
Code

src/Infrastructure/BotSharp.Abstraction/Rules/Constants/BuiltInRuleCriteria.cs[13]

+    public const string PythonScript = "python_script";
Evidence
BuiltInRuleCriteria now only defines python_script, and the code criteria evaluator’s Type is
python_script; since RuleEngine matches by string, legacy mode values won’t match any evaluator.

src/Infrastructure/BotSharp.Abstraction/Rules/Constants/BuiltInRuleCriteria.cs[8-19]
src/Infrastructure/BotSharp.Core.Rules/Criteria/Code/CodeCriteriaEvaluator.cs[24-33]
src/Infrastructure/BotSharp.Core.Rules/Engines/RuleEngine.cs[79-88]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
`BuiltInRuleCriteria` removed/renamed the prior identifier and `CodeCriteriaEvaluator` now reports `Type = "python_script"`. Evaluator resolution is strict string matching, so legacy `mode: "code"` configurations stop working.

## Issue Context
RuleEngine resolves evaluators by comparing evaluator.Type to the configured mode (`x.Type.IsEqualTo(mode)`). With the rename, older stored configs or API callers using `"code"` will resolve no evaluator.

## Fix Focus Areas
- src/Infrastructure/BotSharp.Abstraction/Rules/Constants/BuiltInRuleCriteria.cs[8-19]
- src/Infrastructure/BotSharp.Core.Rules/Criteria/Code/CodeCriteriaEvaluator.cs[24-34]
- src/Infrastructure/BotSharp.Core.Rules/Engines/RuleEngine.cs[79-88]

### Suggested implementation direction
- Add a legacy alias constant (e.g. `public const string Code = "code";`) and/or map `"code"` -> `"python_script"` inside `ResolveCriteriaEvaluator`.
- Alternatively, let the evaluator advertise both identifiers (e.g. by matching on a small set) via resolver mapping logic.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


View action required (1)
4. Mongo schema not migrated 🐞 Bug ☼ Reliability
Description
Mongo AgentRule storage renamed Config -> Criteria and added Message without a backward-compatible
read path; existing documents with Config will deserialize without Criteria, dropping stored
criteria/mode and altering triggering behavior.
Code

src/Plugins/BotSharp.Plugin.MongoStorage/Models/AgentRuleMongoElement.cs[R10-11]

+    public string? Message { get; set; }
+    public RuleCriteriaMongoModel? Criteria { get; set; }
Evidence
The Mongo model now only persists Message and Criteria; there is no mapping from the old Config
field into the new Criteria structure, so persisted rules using the old schema lose their criteria
on read.

src/Plugins/BotSharp.Plugin.MongoStorage/Models/AgentRuleMongoElement.cs[5-33]
src/Infrastructure/BotSharp.Abstraction/Agents/Models/AgentRule.cs[3-20]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
Mongo persistence changed the stored shape from `Config` to `Message` + `Criteria`. With no legacy field mapping, rules persisted under the previous schema will load with `Criteria == null` and lose their stored criteria/mode.

## Issue Context
`AgentRuleMongoElement` no longer has a `Config` property. With `BsonIgnoreExtraElements`, unknown fields are ignored rather than mapped, so old data is effectively dropped on read.

## Fix Focus Areas
- src/Plugins/BotSharp.Plugin.MongoStorage/Models/AgentRuleMongoElement.cs[5-33]
- src/Infrastructure/BotSharp.Abstraction/Agents/Models/AgentRule.cs[3-20]

### Suggested implementation direction
- Add a legacy `Config` property (marked obsolete) and translate it into `Criteria` during `ToDomainElement`.
- Or use BSON aliases (e.g., `[BsonElement("Config")]`) / custom deserialization to read old `Config` into new `Criteria`.
- Consider a one-time migration to rewrite stored documents from `Config` to `Criteria` and add `Message`.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


Grey Divider

Tip of the day
💡 Did you know, you can enable the Remediation agent and Qodo fixes findings in a dedicated fix PR

More tips ↗ | Customize Qodo ↗ | Qodo docs ↗

Grey Divider

Qodo Logo

Comment on lines +55 to +56
var evaluator = ResolveCriteriaEvaluator(rule.Criteria?.Mode) ?? criteriaEvaluator;
if (evaluator != null && options?.Criteria != null)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Action required

1. Rule mode needs options 🐞 Bug ≡ Correctness

RuleEngine resolves an evaluator from rule.Criteria.Mode but only evaluates criteria when
options.Criteria is non-null, so rule-configured criteria can be silently ignored and the rule
executes unconditionally.
Agent Prompt
## Issue description
RuleEngine selects an evaluator from `rule.Criteria?.Mode`, but the criteria evaluation block is gated by `options?.Criteria != null`. This prevents rule-owned criteria from being enforced unless the caller also sends a Criteria options object.

## Issue Context
This directly contradicts the comment that “the rule's own mode wins … without the caller knowing”, and it can cause rules to trigger even when their criteria should block them.

## Fix Focus Areas
- src/Infrastructure/BotSharp.Core.Rules/Engines/RuleEngine.cs[53-69]

### Suggested implementation direction
- Change the guard to evaluate criteria when `evaluator != null`.
- When `options?.Criteria` is null, pass a default `CriteriaOptions` into `RuleCriteriaContext` (so evaluators can still read settings as default/null) e.g. `Options = options?.Criteria ?? new CriteriaOptions()`.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

/// Selects which <c>IRuleCriteriaEvaluator</c> handles this criteria.
/// </summary>
public string Type { get; set; } = BuiltInRuleCriteria.Code;
public string? Mode { get; set; }

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Action required

2. Missing default criteria mode 🐞 Bug ≡ Correctness

CriteriaOptions no longer has a default mode, and ResolveCriteriaEvaluator returns null for blank
mode; when callers omit the mode (or still send legacy fields), the engine skips EvaluateCriteria
and still triggers the rule.
Agent Prompt
## Issue description
`CriteriaOptions` changed from a non-null `Type` with a default value to a nullable `Mode` with no default. In RuleEngine, a null/blank mode resolves no evaluator, and criteria evaluation is skipped while rule triggering continues.

## Issue Context
Previously, `CriteriaOptions.Type` defaulted to the built-in code criteria type. Now, `Mode` can be omitted by callers (including older clients), and the system can silently stop applying criteria.

## Fix Focus Areas
- src/Infrastructure/BotSharp.Abstraction/Rules/Options/RuleTriggerOptions.cs[20-33]
- src/Infrastructure/BotSharp.Core.Rules/Engines/RuleEngine.cs[32-41]
- src/Infrastructure/BotSharp.Core.Rules/Engines/RuleEngine.cs[79-88]

### Suggested implementation direction
Choose at least one:
1) Give `CriteriaOptions.Mode` a safe default (e.g. `BuiltInRuleCriteria.Llm` or `BuiltInRuleCriteria.PythonScript`, depending on intended behavior).
2) Add backward-compatible JSON aliasing (e.g., keep an obsolete `Type` property mapped from `"type"` and translate it into `Mode`).
3) If Mode is absent, fail closed (do not trigger) when criteria evaluation is expected.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

/// Evaluate a code script (e.g. Python) that returns a boolean result.
/// </summary>
public const string Code = "code";
public const string PythonScript = "python_script";

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Action required

3. Criteria identifier renamed 🐞 Bug ☼ Reliability

The built-in criteria identifier changed from "code" to "python_script" and the evaluator now
advertises only python_script; any persisted/requested mode value "code" will no longer resolve,
causing criteria to be skipped.
Agent Prompt
## Issue description
`BuiltInRuleCriteria` removed/renamed the prior identifier and `CodeCriteriaEvaluator` now reports `Type = "python_script"`. Evaluator resolution is strict string matching, so legacy `mode: "code"` configurations stop working.

## Issue Context
RuleEngine resolves evaluators by comparing evaluator.Type to the configured mode (`x.Type.IsEqualTo(mode)`). With the rename, older stored configs or API callers using `"code"` will resolve no evaluator.

## Fix Focus Areas
- src/Infrastructure/BotSharp.Abstraction/Rules/Constants/BuiltInRuleCriteria.cs[8-19]
- src/Infrastructure/BotSharp.Core.Rules/Criteria/Code/CodeCriteriaEvaluator.cs[24-34]
- src/Infrastructure/BotSharp.Core.Rules/Engines/RuleEngine.cs[79-88]

### Suggested implementation direction
- Add a legacy alias constant (e.g. `public const string Code = "code";`) and/or map `"code"` -> `"python_script"` inside `ResolveCriteriaEvaluator`.
- Alternatively, let the evaluator advertise both identifiers (e.g. by matching on a small set) via resolver mapping logic.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

Comment on lines +10 to +11
public string? Message { get; set; }
public RuleCriteriaMongoModel? Criteria { get; set; }

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Action required

4. Mongo schema not migrated 🐞 Bug ☼ Reliability

Mongo AgentRule storage renamed Config -> Criteria and added Message without a backward-compatible
read path; existing documents with Config will deserialize without Criteria, dropping stored
criteria/mode and altering triggering behavior.
Agent Prompt
## Issue description
Mongo persistence changed the stored shape from `Config` to `Message` + `Criteria`. With no legacy field mapping, rules persisted under the previous schema will load with `Criteria == null` and lose their stored criteria/mode.

## Issue Context
`AgentRuleMongoElement` no longer has a `Config` property. With `BsonIgnoreExtraElements`, unknown fields are ignored rather than mapped, so old data is effectively dropped on read.

## Fix Focus Areas
- src/Plugins/BotSharp.Plugin.MongoStorage/Models/AgentRuleMongoElement.cs[5-33]
- src/Infrastructure/BotSharp.Abstraction/Agents/Models/AgentRule.cs[3-20]

### Suggested implementation direction
- Add a legacy `Config` property (marked obsolete) and translate it into `Criteria` during `ToDomainElement`.
- Or use BSON aliases (e.g., `[BsonElement("Config")]`) / custom deserialization to read old `Config` into new `Criteria`.
- Consider a one-time migration to rewrite stored documents from `Config` to `Criteria` and add `Message`.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

@iceljc
iceljc merged commit 8460c30 into SciSharp:master Aug 12, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant