feat: add jinja templating for prompts

Open Rajat-Ahuja1997 opened this issue 5 months ago • 1 comments

Resolves https://linear.app/plastic-labs/issue/DEV-1194

Summary by CodeRabbit

New Features
- Prompt system moved to template-driven rendering with several new customizable templates and a template rendering utility.
- New dream consolidation settings and related template path added; additional deriver and dialectic template paths added.
Documentation
- Example config and env templates updated with new template path entries and commented placeholders.
Chores
- Added Jinja2 dependency.
Tests
- Added tests for template rendering and prompt generation.

Nov 03 '25 18:11 Rajat-Ahuja1997

Walkthrough

Externalizes prompt construction into Jinja2 templates and a TemplateManager; adds templates and template-path config entries; refactors deriver, dialectic, and dreamer prompt functions to render templates; adds Jinja2 dependency and updates dependency-group format; and adds tests for template rendering and prompts.

Changes

Cohort / File(s)	Summary
Configuration & Packaging `\.env.template`, `config.toml.example`, `pyproject.toml`	Adds commented template path variables to `.env.template`; introduces `[dream]` config and several template-path keys in `config.toml.example`; adds `jinja2>=3.1.6` and migrates dev deps to `[dependency-groups]` in `pyproject.toml`.
Config Schema `src/config.py`	Adds template-path fields and defaults: `DeriverSettings` (`CRITICAL_ANALYSIS_TEMPLATE`, `PEER_CARD_TEMPLATE`), `DialecticSettings` (`DIALECTIC_TEMPLATE`, `QUERY_GENERATION_TEMPLATE`), and `DreamSettings` (`CONSOLIDATION_TEMPLATE`).
Prompt Refactor — Deriver `src/deriver/prompts.py`	Replaces inline prompt assembly with `render_template(settings.DERIVER.*, context)` calls; removes `cleandoc` and internal string assembly; imports `settings` and `render_template`.
Prompt Refactor — Dialectic `src/dialectic/prompts.py`	Replaces inline prompt construction with `render_template()` using `settings.DIALECTIC.*`; updates `dialectic_prompt` to accept `observer` and `observed` as keyword-only and to use `observer_peer_card`/`observed_peer_card` naming in context.
Prompt Refactor — Dreamer `src/dreamer/prompts.py`	Replaces inline consolidation prompt with `render_template(settings.DREAM.CONSOLIDATION_TEMPLATE, {...})`; removes `cleandoc` and returns rendered template.
Template Rendering Infra `src/utils/templates.py`	Adds `TemplateManager` using Jinja2 `PackageLoader` for `src/templates`, configures env (autoescape, whitespace, `join_lines` filter), provides cached `get_template_manager()` and `render_template()` helper with error propagation and logging.
Jinja2 Templates `src/templates/...` `src/templates/deriver/critical_analysis.jinja`, `src/templates/deriver/peer_card.jinja`, `src/templates/dialectic/dialectic.jinja`, `src/templates/dialectic/query_generation.jinja`, `src/templates/dreamer/consolidation.jinja`	Adds five templates capturing prior inline prompt logic (critical analysis, peer card, dialectic synthesis, query generation, consolidation) with placeholders and conditional blocks for optional data.
Tests `tests/utils/test_templates.py`	Adds tests for TemplateManager, `render_template`, and prompt rendering (deriver, dialectic, dreamer) covering edge cases, deterministic behavior, special-character handling, and None/empty inputs.

Sequence Diagram(s)

sequenceDiagram
    participant PromptFn as Prompt Function
    participant Settings as Config Settings
    participant API as render_template()
    participant TM as TemplateManager
    participant Jinja as Jinja2 Env

    rect rgb(235,245,255)
    Note over PromptFn,Settings: template-driven prompt generation
    PromptFn->>Settings: read TEMPLATE path (e.g. DERIVER.CRITICAL_ANALYSIS_TEMPLATE)
    PromptFn->>API: render_template(template_name, context)
    API->>TM: get_template_manager()
    TM->>Jinja: env.get_template(template_name)
    Jinja-->>TM: Template object
    TM->>Jinja: template.render(context)
    Jinja-->>TM: rendered string
    TM-->>API: trimmed rendered string
    API-->>PromptFn: return prompt text
    end

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~50 minutes

Pay special attention to:
- Behavioral parity between previous inline prompt text and the new Jinja templates (critical_analysis, dialectic).
- dialectic_prompt signature change and all call sites.
- TemplateManager error handling, template lookup paths, and the join_lines filter semantics.
- Tests covering None/empty inputs, deterministic outputs, and special-character escaping.

"I stitched my thoughts in a Jinja thread,
Templates hum where lines once spread.
Configs set, tests hop in queue,
I render, I sniff—new prompts anew.
🐇🥕"

Pre-merge checks and finishing touches

✅ Passed checks (5 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title accurately summarizes the main change: adding Jinja templating for prompts across the codebase.
Linked Issues check	✅ Passed	The PR fully addresses DEV-1194 objectives: prompts are moved to Jinja templates in dedicated files, and the template rendering system enables ML team integration for dataset generation.
Out of Scope Changes check	✅ Passed	All changes are scoped to Jinja templating: template files added, config updated with template paths, and dependency (jinja2) added. The dev-dependencies restructuring in pyproject.toml aligns with the dependency addition.
Docstring Coverage	✅ Passed	Docstring coverage is 97.14% which is sufficient. The required threshold is 80.00%.

✨ Finishing touches

[ ] 📝 Generate docstrings

🧪 Generate unit tests (beta)

[ ] Create PR with unit tests
[ ] Post copyable unit tests in a comment
[ ] Commit unit tests in branch rajat/prompt-jinja

📜 Recent review details

Configuration used: CodeRabbit UI

Review profile: ASSERTIVE

Plan: Pro

Disabled knowledge base sources:

Linear integration is disabled by default for public repositories

You can enable these sources in your CodeRabbit configuration.

📥 Commits

Reviewing files that changed from the base of the PR and between 0a81b107c6eba50ddc242ec9d378a83c0be145ba and 2e00447b67726f35317fdeee6a1e59983dcac359.

⛔ Files ignored due to path filters (1)

uv.lock is excluded by !**/*.lock

📒 Files selected for processing (14)

.env.template (3 hunks)
config.toml.example (3 hunks)
pyproject.toml (1 hunks)
src/config.py (3 hunks)
src/deriver/prompts.py (3 hunks)
src/dialectic/prompts.py (3 hunks)
src/dreamer/prompts.py (2 hunks)
src/templates/deriver/critical_analysis.jinja (1 hunks)
src/templates/deriver/peer_card.jinja (1 hunks)
src/templates/dialectic/dialectic.jinja (1 hunks)
src/templates/dialectic/query_generation.jinja (1 hunks)
src/templates/dreamer/consolidation.jinja (1 hunks)
src/utils/templates.py (1 hunks)
tests/utils/test_templates.py (1 hunks)

🧰 Additional context used

📓 Path-based instructions (5)

**/*.py