test(missions): add KB pipeline coverage for matcher scoring paths#14075
Conversation
matcher scoring paths Signed-off-by: ayuxsh009 <1raj.aayush@gmail.com>
kubestellar-prow
Bot
added
the
dco-signoff: yes
✅ Deploy Preview for kubestellarconsole ready!Built without sensitive environment variables
To edit notification comments on pull requests, go to your Netlify project configuration. |
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: The full list of commands accepted by this bot can be found here. DetailsNeeds approval from an approver in each of these files:Approvers can indicate their approval by writing |
|
👋 Hey @ayuxsh009 — thanks for opening this PR!
This is an automated message. |
kubestellar-prow
Bot
added
the
size/L
There was a problem hiding this comment.
Pull request overview
Adds a dedicated Vitest suite to increase unit test coverage for the mission matcher’s KB ranking/scoring behavior in matchMissionsToCluster(), focusing on resource→keyword expansion, issue-category scoring, and engagement/matchPercent normalization.
Changes:
- Added a new test file covering RESOURCE_TO_PROJECTS keyword expansion behavior for several common operators/components.
- Added tests exercising ISSUE_TO_CATEGORIES category-boost scoring (including case-insensitive matching and multi-issue inputs).
- Added tests for engagement-based scoring boosts and
matchPercentnormalization behavior.
Comments suppressed due to low confidence (2)
web/src/lib/missions/tests/matcher-kb-pipeline.test.ts:212
- This test’s inline comment implies that adding a second issue can increase the score via additional tag/category matches, but matchMissionsToCluster breaks after the first mission-keyword match against issueCategories, so additional categories won’t stack once a match is found. Please adjust the comment (or the assertion) to reflect the actual non-stacking behavior.
it('accumulates score from multiple simultaneous cluster issues', () => {
// Two issues → two category sets → potentially more category boosts
const missions = [makeMission({ tags: ['troubleshoot', 'memory'], title: 'Debug OOM crashloop' })]
const singleIssue = { name: 'test', issues: ['CrashLoopBackOff'] }
const dualIssue = { name: 'test', issues: ['CrashLoopBackOff', 'Pod was OOMKilled'] }
const single = matchMissionsToCluster(missions, singleIssue)
const dual = matchMissionsToCluster(missions, dualIssue)
// Dual issues expand the category set → additional memory/limits tags can match
expect(dual[0].score).toBeGreaterThanOrEqual(single[0].score)
})
web/src/lib/missions/tests/matcher-kb-pipeline.test.ts:267
- These tests repeatedly use double type assertions (e.g., metadata cast to MissionExport['metadata'] even though reactions/comments aren’t part of that type, and per-element null/undefined casts for issues). This makes the test harder to read and can mask typing drift; consider a small local helper/type (e.g., MissionExport & { metadata: { reactions?: number; comments?: number } }) and casting the whole issues array once (unknown as string[]) to keep intent clear.
it('skips null or undefined issue entries without throwing', () => {
const missions = [makeMission({ tags: ['troubleshoot'] })]
const cluster = { name: 'test', issues: [null as unknown as string, undefined as unknown as string, 'CrashLoopBackOff'] }
expect(() => matchMissionsToCluster(missions, cluster)).not.toThrow()
const results = matchMissionsToCluster(missions, cluster)
expect(results[0].score).toBeGreaterThan(1)
})
})
// ── 3. Engagement metadata + matchPercent normalisation ──────────────────
describe('matchMissionsToCluster — engagement scoring and matchPercent', () => {
it('boosts missions with >= 20 reactions by 10 points', () => {
const highEngagement = makeMission({ title: 'Popular mission', metadata: { reactions: 20, comments: 0 } as unknown as MissionExport['metadata'] })
const noEngagement = makeMission({ title: 'Obscure mission' })
const results = matchMissionsToCluster([highEngagement, noEngagement], null)
const popular = results.find(r => r.mission.title === 'Popular mission')!
const obscure = results.find(r => r.mission.title === 'Obscure mission')!
expect(popular.score).toBe(10) // 10 engagement (minimum floor of 1 only applies when score is 0)
expect(obscure.score).toBe(1) // baseline only — score stayed 0, floor applied
})
it('gives partial boost (5 pts) for missions with 5–19 reactions', () => {
const moderate = makeMission({ title: 'Moderate', metadata: { reactions: 5, comments: 0 } as unknown as MissionExport['metadata'] })
const results = matchMissionsToCluster([moderate], null)
expect(results[0].score).toBe(5) // 5 reactions boost (floor not applied — score > 0)
})
it('does not boost missions with fewer than 5 reactions', () => {
const low = makeMission({ title: 'Low', metadata: { reactions: 4, comments: 0 } as unknown as MissionExport['metadata'] })
const results = matchMissionsToCluster([low], null)
expect(results[0].score).toBe(1) // baseline only
})
it('boosts missions with >= 10 comments by 5 additional points', () => {
const active = makeMission({ title: 'Active', metadata: { reactions: 0, comments: 10 } as unknown as MissionExport['metadata'] })
const results = matchMissionsToCluster([active], null)
expect(results[0].score).toBe(5) // 5 comments boost (floor not applied — score > 0)
})
it('stacks reactions and comments boosts independently', () => {
const viral = makeMission({ title: 'Viral', metadata: { reactions: 20, comments: 15 } as unknown as MissionExport['metadata'] })
const results = matchMissionsToCluster([viral], null)
expect(results[0].score).toBe(15) // 10 reactions + 5 comments (no floor, score already > 0)
| * Tests for three scoring paths in matchMissionsToCluster that are untested | ||
| * in matcher.test.ts: | ||
| * |
There was a problem hiding this comment.
@copilot apply changes based on this feedback
![kubestellar-hive[bot]](https://avatars.githubusercontent.com/u/134407106?s=60&v=4){kind=small}
|
@kubestellar-hive[bot]: changing LGTM is restricted to collaborators DetailsIn response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. |
kubestellar-hive
Bot
merged commit 58de917
into
kubestellar:main
❌ Post-Merge Verification: failedCommit: |
PR Description
📌 Fixes
No issue — standalone test coverage contribution
📝 Summary of Changes
matchMissionsToCluster()that had zero coverage in the existingmatcher.test.tsChanges Made
web/src/lib/missions/__tests__/matcher-kb-pipeline.test.tswith 28 tests covering:RESOURCE_TO_PROJECTStransitive expansion —prometheus-server,istio-pilot,argocd-server,cert-manager-webhook,keda-operator,velerocorrectly expand into ecosystem keywords so related missions are surfacedISSUE_TO_CATEGORIESscoring — cluster issues (CrashLoopBackOff,OOMKilled,privileged container,pending,host network) map to fix categories and boost missions with matching tags; covers case-insensitive matching, null/undefined safety, andmulti-issue accumulation
matchPercentnormalisation — reaction/comment boost tiers, top scorer always getsmatchPercent: 100,proportional scaling verified
Checklist
git commit -s)Screenshots or Logs (if applicable)
Test Files 1 passed (1)
Tests 28 passed (28)
Duration 1.49s
👀 Reviewer Notes
These tests target the KB pipeline scoring logic used by the AI agent to rank runbooks against live cluster state — three paths that
existed in
matcher.tsbut had no test coverage. Aligned with the LFX 2026 Term 2 "AI-driven operational KB and mission controltesting" scope.