feat: Create token infrastructure

[poltorak]

feat: token infrastructure

Shared, design-system-agnostic infrastructure for reading, loading, and querying design tokens. This is a platform layer — no new MCP tools are exposed. Future tool specs (suggest-token-replacements, audit-token-usage, suggest-utility-classes) will consume it.

What's new

Configuration layer

• New optional ds.generatedStylesRoot config field — relative path to the directory containing generated token CSS files
• New ds.tokens.* config block with 6 sub-fields controlling file discovery, prefix filtering, directory interpretation, and categorisation — all with sensible defaults
• Bootstrap validation warns and continues if the path is invalid (never crashes the server)
• CLI arguments added for all new fields

CSS Custom Property Parser

• Regex-based extractor that reads --*: value; declarations from CSS files into a name→value map
• Strips comments, preserves var() references, supports optional prefix filtering

Token Dataset and Loader

• Discovers token files via configurable glob pattern under the generated styles root
• Three directory strategies: flat (single token set), brand-theme (path segments map to scope keys like brand/theme), auto (inferred from directory depth)
• Three categorisation modes: by-prefix (longest match from a configurable prefix map), by-value (pattern-matches hex/rgb/hsl → color, px/rem/em → spacing, etc.), none
• Immutable, indexed dataset with O(1) lookups by name, value, category, and scope — plus scope-filtered variants
• Lazy-loaded on first access, cached for subsequent calls

SCSS Value Parser

• PostCSS-based parser that extracts property-value pairs per selector from component SCSS files
• Handles nested selectors, :host, and ::ng-deep
• Classifies each entry as declaration (component token definition), consumption (var() reference), or plain
• Configurable component token prefix

Server integration

• Token config and generated styles root injected into every tool handler's context
• Lazy getTokenDataset() method on the server — returns an empty dataset with an actionable diagnostic when not configured

Docs updated

• README — added new configuration parameters table with detailed descriptions, updated MCP config examples and project structure diagram
• Getting Started guide — added new config field to the setup example
• Architecture doc — expanded configuration options section, documented new shared infrastructure modules

No breaking changes

All new config fields are optional with defaults. Existing configurations parse identically. No new npm dependencies.

[Copilot]

Pull request overview

Introduces a shared, design-system-agnostic “token infrastructure” layer across the Angular MCP server and shared utilities, enabling optional discovery, parsing, categorization, and querying of design tokens (plus an SCSS value parser), with server/CLI/config plumbing and documentation updates.

Changes:

• Add ds.generatedStylesRoot and ds.tokens.* configuration (schema defaults + CLI flags) with bootstrap validation that warns (and disables token features) when invalid.
• Implement token parsing/loading infrastructure (CSS custom property parser, token dataset + loader) and an SCSS value parser in styles-ast-utils.
• Update docs and repo hygiene (.gitignore + removal of Cursor flow/example files).

Reviewed changes

Copilot reviewed 35 out of 37 changed files in this pull request and generated 6 comments.

Show a summary per file

File	Description
README.md	Documents new optional token configuration and example project structure.
packages/shared/styles-ast-utils/src/lib/scss-value-parser.ts	Adds SCSS property/value extraction and classification utility.
packages/shared/styles-ast-utils/src/lib/scss-value-parser.spec.ts	Adds unit + parameterized tests for the SCSS value parser.
packages/shared/styles-ast-utils/src/index.ts	Re-exports the new SCSS value parser API.
packages/angular-mcp/src/main.ts	Adds CLI args for ds.generatedStylesRoot and ds.tokens.* and passes them into server creation.
packages/angular-mcp-server/src/lib/validation/spec/config-schema-and-bootstrap.spec.ts	Adds schema + bootstrap validation tests for token config fields and defaults.
packages/angular-mcp-server/src/lib/validation/file-existence.ts	Adds “warn and disable” validation for ds.generatedStylesRoot.
packages/angular-mcp-server/src/lib/validation/angular-mcp-server-options.schema.ts	Adds TokensConfigSchema + ds.generatedStylesRoot to the Zod options schema.
packages/angular-mcp-server/src/lib/tools/ds/shared/utils/token-dataset.ts	Introduces the queryable token dataset interfaces and indexed implementation.
packages/angular-mcp-server/src/lib/tools/ds/shared/utils/token-dataset-loader.ts	Adds file discovery, scope inference, categorization, and dataset loading.
packages/angular-mcp-server/src/lib/tools/ds/shared/utils/spec/token-dataset.spec.ts	Adds extensive dataset behavior tests.
packages/angular-mcp-server/src/lib/tools/ds/shared/utils/spec/token-dataset-loader.spec.ts	Adds loader tests covering strategies, categorization modes, and filters.
packages/angular-mcp-server/src/lib/tools/ds/shared/utils/spec/css-custom-property-parser.spec.ts	Adds tests for CSS custom property extraction and prefix filtering.
packages/angular-mcp-server/src/lib/tools/ds/shared/utils/regex-helpers.ts	Adds shared CSS custom property regex constants.
packages/angular-mcp-server/src/lib/tools/ds/shared/utils/handler-helpers.ts	Extends handler context/options to include token config and generated styles root.
packages/angular-mcp-server/src/lib/tools/ds/shared/utils/css-custom-property-parser.ts	Adds regex-based CSS --* declaration extraction + optional prefix filtering.
packages/angular-mcp-server/src/lib/spec/server-token-integration.spec.ts	Adds integration tests for server bootstrap and getTokenDataset() behavior.
packages/angular-mcp-server/src/lib/angular-mcp-server.ts	Integrates token config into server context and adds lazy getTokenDataset() caching.
package-lock.json	Lockfile update (peer metadata churn).
docs/getting-started.md	Updates setup example to include optional ds.generatedStylesRoot.
docs/architecture-internal-design.md	Documents token configuration and new shared modules.
.gitignore	Ignores Kiro settings/tmp directories.
.cursor/mcp.json.example	Removed Cursor MCP example file.
.cursor/flows/README.md	Removed Cursor flows documentation.
.cursor/flows/ds-refactoring-flow/clean-global-styles.mdc	Removed Cursor flow rule file.
.cursor/flows/ds-refactoring-flow/05-prepare-report.mdc	Removed Cursor flow rule file.
.cursor/flows/ds-refactoring-flow/04-validate-changes.mdc	Removed Cursor flow rule file.
.cursor/flows/ds-refactoring-flow/03-non-viable-cases.mdc	Removed Cursor flow rule file.
.cursor/flows/ds-refactoring-flow/03-fix-violations.mdc	Removed Cursor flow rule file.
.cursor/flows/ds-refactoring-flow/02b-plan-refactoring-for-all-violations.mdc	Removed Cursor flow rule file.
.cursor/flows/ds-refactoring-flow/02-plan-refactoring.mdc	Removed Cursor flow rule file.
.cursor/flows/ds-refactoring-flow/01b-find-all-violations.mdc	Removed Cursor flow rule file.
.cursor/flows/ds-refactoring-flow/01-find-violations.mdc	Removed Cursor flow rule file.
.cursor/flows/component-refactoring/angular-20.md	Removed Cursor flow reference doc.
.cursor/flows/component-refactoring/03-validate-component.mdc	Removed Cursor flow rule file.
.cursor/flows/component-refactoring/02-refactor-component.mdc	Removed Cursor flow rule file.
.cursor/flows/component-refactoring/01-review-component.mdc	Removed Cursor flow rule file.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Karnaukhov-kh]

Can be combined with css-custom-property-parser.ts or replace it? parseScssContent + filter getDeclarations() for custom properties?

[rapczynska]

do we need it? can't we just go all he way down through the nested dirs of a given path?

[rapczynska]

why differentiate component token from other tokens? both can be messed up equally by overriding and setting raw value

[rapczynska]

it should not focus on what kind of token that is imo - it's not it business. it should just do whatever has to be done for a given prefix

[poltorak]

Thanks, I'll remove that one. It's indeed redundant