Ralph Khreish 72502416c6
chore: v0.17 features and improvements (#771)
* chore: task management and small bug fix.

* chore: task management

* feat: implement research command with enhanced context gathering - Add comprehensive research command with AI-powered queries - Implement ContextGatherer utility for reusable context extraction - Support multiple context types: tasks, files, custom text, project tree - Add fuzzy search integration for automatic task discovery - Implement detailed token breakdown display with syntax highlighting - Add enhanced UI with boxed output and code block formatting - Support different detail levels (low, medium, high) for responses - Include project-specific context for more relevant AI responses - Add token counting with gpt-tokens library integration - Create reusable patterns for future context-aware commands - Task 94.4 completed

* docs: add context gathering rule and update existing rules

- Create comprehensive context_gathering.mdc rule documenting ContextGatherer utility patterns, FuzzyTaskSearch integration, token breakdown display, code block syntax highlighting, and enhanced result display patterns
- Update new_features.mdc to include context gathering step
- Update commands.mdc with context-aware command pattern
- Update ui.mdc with enhanced display patterns and syntax highlighting
- Update utilities.mdc to document new context gathering utilities
- Update glossary.mdc to include new context_gathering rule
- Establishes standardized patterns for building intelligent, context-aware commands that can leverage project knowledge for better AI assistance

* feat(fuzzy): improves fuzzy search to introspect into subtasks as well. might still need improvement.

* fix(move): adjusts logic to prevent an issue when moving from parent to subtask if the target parent has no subtasks.

* fix(move-task): Fix critical bugs in task move functionality

- Fixed parent-to-parent task moves where original task would remain as duplicate
- Fixed moving tasks to become subtasks of empty parents (validation errors)
- Fixed moving subtasks between different parent tasks
- Improved comma-separated batch moves with proper error handling
- Updated MCP tool to use core logic instead of custom implementation
- Resolves task duplication issues and enables proper task hierarchy reorganization

* feat(research): Add subtasks to fuzzy search and follow-up questions

- Enhanced fuzzy search to include subtasks in discovery - Added interactive follow-up question functionality using inquirer
- Improved context discovery by including both tasks and subtasks
- Follow-up option for research with default to 'n' for quick workflow

* chore: removes task004 chat that had like 11k lines lol.

* chore: formatting

* feat(show): add comma-separated ID support for multi-task viewing

- Enhanced get-task/show command to support comma-separated task IDs for efficient batch operations.
- New features include multiple task retrieval, smart display logic, interactive action menu with batch operations, MCP array response for AI agent efficiency, and support for mixed parent tasks and subtasks.
- Implementation includes updated CLI show command, enhanced MCP get_task tool, modified showTaskDirect function, and maintained full backward compatibility.
- Documentation updated across all relevant files.

Benefits include faster context gathering for AI agents, improved workflow with interactive batch operations, better UX with responsive layout, and enhanced API efficiency.

* feat(research): Adds MCP tool for  command

- New MCP Tool: research tool enables AI-powered research with project context
- Context Integration: Supports task IDs, file paths, custom context, and project tree
- Fuzzy Task Discovery: Automatically finds relevant tasks using semantic search
- Token Management: Detailed token counting and breakdown by context type
- Multiple Detail Levels: Support for low, medium, and high detail research responses
- Telemetry Integration: Full cost tracking and usage analytics
- Direct Function: researchDirect with comprehensive parameter validation
- Silent Mode: Prevents console output interference with MCP JSON responses
- Error Handling: Robust error handling with proper MCP response formatting

This completes subtasks 94.5 (Direct Function) and 94.6 (MCP Tool) for the research command implementation, providing a powerful research interface for integrated development environments like Cursor.

Updated documentation across taskmaster.mdc, README.md, command-reference.md, examples.md, tutorial.md, and docs/README.md to highlight research capabilities and usage patterns.

* chore: task management

* chore: task management and removes mistakenly staged changes

* fix(move): Fix move command bug that left duplicate tasks

- Fixed logic in moveTaskToNewId function that was incorrectly treating task-to-task moves as subtask creation instead of task replacement
- Updated moveTaskToNewId to properly handle replacing existing destination tasks instead of just placeholders
- The move command now correctly replaces destination tasks and cleans up properly without leaving duplicates

- Task Management: Moved task 93 (Google Vertex AI Provider) to position 88, Moved task 94 (Azure OpenAI Provider) to position 89, Updated task dependencies and regenerated task files, Cleaned up orphaned task files automatically
- All important validations remain in place: Prevents moving tasks to themselves, Prevents moving parent tasks to their own subtasks, Prevents circular dependencies
- Resolves the issue where moving tasks would leave both source and destination tasks in tasks.json and file system

* chore: formatting

* feat: Add .taskmaster directory (#619)

* chore: apply requested changes from next branch (#629)

* chore: rc version bump

* chore: cleanup migration-guide

* fix: bedrock set model and other fixes (#641)

* Fix: MCP log errors (#648)

* fix: projectRoot duplicate .taskmaster directory (#655)

* Version Packages

* chore: add package-lock.json

* Version Packages

* Version Packages

* fix: markdown format (#622)

* Version Packages

* Version Packages

* Fixed the Typo in cursor rules Issue:#675 (#677)

Fixed the typo in the Api keys

* Add one-click MCP server installation for Cursor (#671)

* Update README.md - Remove trailing commas (#673)

JSON doesn't allow for trailing commas, so these need to be removed in order for this to work

* chore: rc version bump

* fix: findTasksPath function

* fix: update MCP tool

* feat(ui): replace emoji complexity indicators with clean filled circle characters

Replace 🟢, 🟡, 🔴 emojis with ● character in getComplexityWithColor function

Update corresponding unit tests to expect ● instead of emojis

Improves UI continuity

* fix(ai-providers): change generateObject mode from 'tool' to 'auto' for better provider compatibility

Fixes Perplexity research role failing with 'tool-mode object generation' error

The hardcoded 'tool' mode was incompatible with providers like Perplexity that support structured JSON output but not function calling/tool use

Using 'auto' mode allows the AI SDK to choose the best approach for each provider

* Adds qwen3-235n-a22b:free to supported models. Closes #687)

* chore: adds a warning when custom openrouter model is a free model which suffers from lower rate limits, restricted context, and, worst of all, no access to tool_use.

* refactor: enhance add-task fuzzy search and fix duplicate banner display

- **Remove hardcoded category system** in add-task that always matched 'Task management'
- **Eliminate arbitrary limits** in fuzzy search results (5→25 high relevance, 3→10 medium relevance, 8→20 detailed tasks)
- **Improve semantic weighting** in Fuse.js search (details=3, description=2, title=1.5) for better relevance
- **Fix duplicate banner issue** by removing console.clear() and redundant displayBanner() calls from UI functions
- **Enhance context generation** to rely on semantic similarity rather than rigid pattern matching
- **Preserve terminal history** to address GitHub issue #553 about eating terminal lines
- **Remove displayBanner() calls** from: displayHelp, displayNextTask, displayTaskById, displayComplexityReport, set-task-status, clear-subtasks, dependency-manager functions

The add-task system now provides truly relevant task context based on semantic similarity rather than arbitrary categories and limits, while maintaining a cleaner terminal experience.

Changes span: add-task.js, ui.js, set-task-status.js, clear-subtasks.js, list-tasks.js, dependency-manager.js

Closes #553

* chore: changeset

* chore: passes tests and linting

* chore: more linting

* ninja(sync): add sync-readme command for GitHub README export with UTM tracking and professional markdown formatting. Experimental

* chore: changeset adjustment

* docs: Auto-update and format models.md

* chore: updates readme with npm download badges and mentions AI Jason who is joining the taskmaster core team.

* chore: fixes urls in readme npm packages

* chore: fixes urls in readme npm packages again

* fix: readme typo

* readme: fix twitter urls.

* readme: removes the taskmaster list output which is too overwhelming given its size with subtasks. may re-add later. fixes likely issues in the json for manual config in cursor and windsurf in the readme.

* chore: small readme nitpicks

* chore: adjusts changeset from minor to patch to avoid version bump to 0.17

* readme: moves up the documentation links higher up in the readme. same with the cursor one-click install.

* Fix Cursor deeplink installation with copy-paste instructions (#723)

* solve merge conflics with next. not gonna deal with these much longer.

* chore: update task files during rebase

* chore: task management

* feat: implement research command with enhanced context gathering - Add comprehensive research command with AI-powered queries - Implement ContextGatherer utility for reusable context extraction - Support multiple context types: tasks, files, custom text, project tree - Add fuzzy search integration for automatic task discovery - Implement detailed token breakdown display with syntax highlighting - Add enhanced UI with boxed output and code block formatting - Support different detail levels (low, medium, high) for responses - Include project-specific context for more relevant AI responses - Add token counting with gpt-tokens library integration - Create reusable patterns for future context-aware commands - Task 94.4 completed

* fix(move): adjusts logic to prevent an issue when moving from parent to subtask if the target parent has no subtasks.

* fix(move-task): Fix critical bugs in task move functionality

- Fixed parent-to-parent task moves where original task would remain as duplicate
- Fixed moving tasks to become subtasks of empty parents (validation errors)
- Fixed moving subtasks between different parent tasks
- Improved comma-separated batch moves with proper error handling
- Updated MCP tool to use core logic instead of custom implementation
- Resolves task duplication issues and enables proper task hierarchy reorganization

* chore: removes task004 chat that had like 11k lines lol.

* feat(show): add comma-separated ID support for multi-task viewing

- Enhanced get-task/show command to support comma-separated task IDs for efficient batch operations.
- New features include multiple task retrieval, smart display logic, interactive action menu with batch operations, MCP array response for AI agent efficiency, and support for mixed parent tasks and subtasks.
- Implementation includes updated CLI show command, enhanced MCP get_task tool, modified showTaskDirect function, and maintained full backward compatibility.
- Documentation updated across all relevant files.

Benefits include faster context gathering for AI agents, improved workflow with interactive batch operations, better UX with responsive layout, and enhanced API efficiency.

* feat(research): Adds MCP tool for  command

- New MCP Tool: research tool enables AI-powered research with project context
- Context Integration: Supports task IDs, file paths, custom context, and project tree
- Fuzzy Task Discovery: Automatically finds relevant tasks using semantic search
- Token Management: Detailed token counting and breakdown by context type
- Multiple Detail Levels: Support for low, medium, and high detail research responses
- Telemetry Integration: Full cost tracking and usage analytics
- Direct Function: researchDirect with comprehensive parameter validation
- Silent Mode: Prevents console output interference with MCP JSON responses
- Error Handling: Robust error handling with proper MCP response formatting

This completes subtasks 94.5 (Direct Function) and 94.6 (MCP Tool) for the research command implementation, providing a powerful research interface for integrated development environments like Cursor.

Updated documentation across taskmaster.mdc, README.md, command-reference.md, examples.md, tutorial.md, and docs/README.md to highlight research capabilities and usage patterns.

* chore: task management

* fix(move): Fix move command bug that left duplicate tasks

- Fixed logic in moveTaskToNewId function that was incorrectly treating task-to-task moves as subtask creation instead of task replacement
- Updated moveTaskToNewId to properly handle replacing existing destination tasks instead of just placeholders
- The move command now correctly replaces destination tasks and cleans up properly without leaving duplicates

- Task Management: Moved task 93 (Google Vertex AI Provider) to position 88, Moved task 94 (Azure OpenAI Provider) to position 89, Updated task dependencies and regenerated task files, Cleaned up orphaned task files automatically
- All important validations remain in place: Prevents moving tasks to themselves, Prevents moving parent tasks to their own subtasks, Prevents circular dependencies
- Resolves the issue where moving tasks would leave both source and destination tasks in tasks.json and file system

* chore: moves to new task master config setup

* feat: add comma-separated status filtering to list-tasks

- supports multiple statuses like 'blocked,deferred' with comprehensive test coverage and backward compatibility

- also adjusts biome.json to stop bitching about templating.

* chore: linting ffs

* fix(generate): Fix generate command creating tasks in legacy location

- Update generate command default output directory from 'tasks' to '.taskmaster/tasks'
- Fix path.dirname() usage to properly derive output directory from tasks file location
- Update MCP tool description and documentation to reflect new structure
- Disable Biome linting rules for noUnusedTemplateLiteral and useArrowFunction
- Fixes issue where generate command was creating task files in the old 'tasks/' directory instead of the new '.taskmaster/tasks/' structure after the refactor

* chore: task management

* chore: task management some more

* fix(get-task): makes the projectRoot argument required to prevent errors when getting tasks.

* feat(tags): Implement tagged task lists migration system (Part 1/2)

This commit introduces the foundational infrastructure for tagged task lists,
enabling multi-context task management without remote storage to prevent merge conflicts.

CORE ARCHITECTURE:
• Silent migration system transforms tasks.json from old format { "tasks": [...] }
  to new tagged format { "master": { "tasks": [...] } }
• Tag resolution layer provides complete backward compatibility - existing code continues to work
• Automatic configuration and state management for seamless user experience

SILENT MIGRATION SYSTEM:
• Automatic detection and migration of legacy tasks.json format
• Complete project migration: tasks.json + config.json + state.json
• Transparent tag resolution returns old format to maintain compatibility
• Zero breaking changes - all existing functionality preserved

CONFIGURATION MANAGEMENT:
• Added global.defaultTag setting (defaults to 'master')
• New tags section with gitIntegration placeholders for future features
• Automatic config.json migration during first run
• Proper state.json creation with migration tracking

USER EXPERIENCE:
• Clean, one-time FYI notice after migration (no emojis, professional styling)
• Notice appears after 'Suggested Next Steps' and is tracked in state.json
• Silent operation - users unaware migration occurred unless explicitly shown

TECHNICAL IMPLEMENTATION:
• Enhanced readJSON() with automatic migration detection and processing
• New utility functions: getCurrentTag(), resolveTag(), getTasksForTag(), setTasksForTag()
• Complete migration orchestration via performCompleteTagMigration()
• Robust error handling and fallback mechanisms

BACKWARD COMPATIBILITY:
• 100% backward compatibility maintained
• Existing CLI commands and MCP tools continue to work unchanged
• Legacy tasks.json format automatically upgraded on first read
• All existing workflows preserved

TESTING VERIFIED:
• Complete migration from legacy state works correctly
• Config.json properly updated with tagged system settings
• State.json created with correct initial values
• Migration notice system functions as designed
• All existing functionality continues to work normally

Part 2 will implement tag management commands (add-tag, use-tag, list-tags)
and MCP tool updates for full tagged task system functionality.

Related: Task 103 - Implement Tagged Task Lists System for Multi-Context Task Management

* docs: Update documentation and rules for tagged task lists system

- Updated task-structure.md with comprehensive tagged format explanation
- Updated all .cursor/rules/*.mdc files to reflect tagged system
- Completed subtask 103.16: Update Documentation for Tagged Task Lists System

* feat(mcp): Add tagInfo to responses and integrate ContextGatherer

Enhances the MCP server to include 'tagInfo' (currentTag, availableTags) in all tool responses, providing better client-side context.

- Introduces a new 'ContextGatherer' utility to standardize the collection of file, task, and project context for AI-powered commands. This refactors several task-manager modules ('expand-task', 'research', 'update-task', etc.) to use the new utility.

- Fixes an issue in 'get-task' and 'get-tasks' MCP tools where the 'projectRoot' was not being passed correctly, preventing tag information from being included in their responses.

- Adds subtask '103.17' to track the implementation of the task template importing feature.

- Updates documentation ('.cursor/rules', 'docs/') to align with the new tagged task system and context gatherer logic.

* fix: include tagInfo in AI service responses for MCP tools

- Update all core functions that call AI services to extract and return tagInfo
- Update all direct functions to include tagInfo in MCP response data
- Fixes issue where add_task, expand_task, and other AI commands were not including current tag and available tags information
- tagInfo includes currentTag from state.json and availableTags list
- Ensures tagged task lists system information is properly propagated through the full chain: AI service -> core function -> direct function -> MCP client

* fix(move-task): Update move functionality for tagged task system compatibility

- incorporate GitHub commit fixes and resolve readJSON data handling

* feat(tagged-tasks): Complete core tag management system implementation

- Implements comprehensive tagged task lists system for multi-context task management including core tag management functions (Task 103.11), MCP integration updates, and foundational infrastructure for tagged task operations. Features tag CRUD operations, validation, metadata tracking, deep task copying, and full backward compatibility.

* fix(core): Fixed move-task.js writing _rawTaggedData directly, updated writeJSON to filter tag fields, fixed CLI move command missing projectRoot, added ensureTagMetadata utility

* fix(tasks): ensure list tasks triggers silent migration if necessary.

* feat(tags): Complete show and add-task command tag support
- show command: Added --tag flag, fixed projectRoot passing to UI functions
- add-task command: Already had proper tag support and projectRoot handling
- Both commands now work correctly with tagged task lists system
- Migration logic works properly when viewing and adding tasks
- Updated subtask 103.5 with progress on high-priority command fixes

* fix(tags): Clean up rogue created properties and fix taskCount calculation
- Enhanced writeJSON to automatically filter rogue created/description properties from tag objects
- Fixed tags command error by making taskCount calculation dynamic instead of hardcoded
- Cleaned up existing rogue created property in master tag through forced write operation
- All created properties now properly located in metadata objects only
- Tags command working perfectly with proper task count display
- Data integrity maintained with automatic cleanup during write operations

* fix(tags): Resolve critical tag deletion and migration notice bugs

Major Issues Fixed:

1. Tag Deletion Bug: Fixed critical issue where creating subtasks would delete other tags

   - Root cause: writeJSON function wasn't accepting projectRoot/tag parameters

   - Fixed writeJSON signature and logic to handle tagged data structure

   - Added proper merging of resolved tag data back into full tagged structure

2. Persistent Migration Notice: Fixed FYI notice showing after every command

   - Root cause: markMigrationForNotice was resetting migrationNoticeShown to false

   - Fixed migration logic to only trigger on actual legacy->tagged migrations

   - Added proper _rawTaggedData checks to prevent false migration detection

3. Data Corruption Prevention: Enhanced data integrity safeguards

   - Fixed writeJSON to filter out internal properties

   - Added automatic cleanup of rogue properties

   - Improved hasTaggedStructure detection logic

Commands Fixed: add-subtask, remove-subtask, and all commands now preserve tags correctly

* fix(tags): Resolve tag deletion bug in remove-task command

Refactored the core 'removeTask' function to be fully tag-aware, preventing data corruption.

- The function now correctly reads the full tagged data structure by prioritizing '_rawTaggedData' instead of operating on a resolved single-tag view.

- All subsequent operations (task removal, dependency cleanup, file writing) now correctly reference the full multi-tag data object, preserving the integrity of 'tasks.json'.

- This resolves the critical bug where removing a task would delete all other tags.

* fix(tasks): Ensure new task IDs are sequential within the target tag

Modified the ID generation logic in 'add-task.js' to calculate the next task ID based on the highest ID within the specified tag, rather than globally across all tags.

This fixes a critical bug where creating a task in a new tag would result in a high, non-sequential ID, such as ID 105 for the first task in a tag.

* fix(commands): Add missing context parameters to dependency and remove-subtask commands

- Add projectRoot and tag context to all dependency commands
- Add projectRoot and tag context to remove-subtask command
- Add --tag option to remove-subtask command
- Fixes critical bug where remove-subtask was deleting other tags due to missing context
- All dependency and subtask commands now properly handle tagged task lists

* feat(tags): Add --tag flag support to core commands for multi-context task management
- parse-prd now supports creating tasks in specific contexts
- Fixed tag preservation logic to prevent data loss
- analyze-complexity generates tag-specific reports
- Non-existent tags created automatically
- Enables rapid prototyping and parallel development workflows

* feat(tags): Complete tagged task lists system with enhanced use-tag command

- Multi-context task management with full CLI support
- Enhanced use-tag command shows next available task after switching
- Universal --tag flag support across all commands
- Seamless migration with zero disruption
- Complete tag management suite (add, delete, rename, copy, list)
- Smart confirmation logic and data integrity protection
- State management and configuration integration
- Real-world use cases for teams, features, and releases

* feat(tags): Complete tag support for remaining CLI commands

- Add --tag flag to update, move, and set-status commands
- Ensure all task operation commands now support tag context
- Fix missing tag context passing to core functions
- Complete comprehensive tag-aware command coverage

* feat(ui): add tag indicator to all CLI commands
- shows 🏷️ tag: tagname for complete context visibility across 15+ commands

* fix(ui): resolve dependency 'Not found' issue when filtering

- now correctly displays dependencies that exist but are filtered out of view

* feat(research): Add comprehensive AI-powered research command with interactive follow-ups, save functionality, intelligent context gathering, fuzzy task discovery, multi-source context support, enhanced display with syntax highlighting, clean inquirer menus, comprehensive help, and MCP integration with saveTo parameter

* feat(tags): Implement full MCP support for Tagged Task Lists and update-task append mode

* chore: task management

* feat(research): Enhance research command with follow-up menu, save functionality, and fix ContextGatherer token counting

* feat(git-workflow): Add automatic git branch-tag integration

- Implement automatic tag creation when switching to new git branches

- Add branch-tag mapping system for seamless context switching

- Enable auto-switch of task contexts based on current git branch

- Provide isolated task contexts per branch to prevent merge conflicts

- Add configuration support for enabling/disabling git workflow features

- Fix ES module compatibility issues in git-utils module

- Maintain zero migration impact with automatic 'master' tag creation

- Support parallel development with branch-specific task contexts

The git workflow system automatically detects branch changes and creates corresponding empty task tags, enabling developers to maintain separate task contexts for different features/branches while preventing task-related merge conflicts during collaborative development.

Resolves git workflow integration requirements for multi-context development.

* feat(git-workflow): Simplify git integration with --from-branch option

- Remove automatic git workflow and branch-tag switching - we are not ready for it yet

- Add --from-branch option to add-tag command for manual tag creation from git branch

- Remove git workflow configuration from config.json and assets

- Disable automatic tag switching functions in git-utils.js

- Add createTagFromBranch function for branch-based tag creation

- Support both CLI and MCP interfaces for --from-branch functionality

- Fix ES module imports in git-utils.js and utils.js

- Maintain user control over tag contexts without forced automation

The simplified approach allows users to create tags from their current git branch when desired, without the complexity and rigidity of automatic branch-tag synchronization. Users maintain full control over their tag contexts while having convenient tools for git-based workflows when needed.

* docs: Update rule files to reflect simplified git integration approach

- Remove automatic git workflow features, update to manual --from-branch option, change Part 2 references to completed status

* fix(commands): Fix add-tag --from-branch requiring tagName argument
- Made tagName optional when using --from-branch - Added validation for either tagName or --from-branch
- Fixes 'missing required argument' error with --from-branch option

* fix(mcp): Prevent tag deletion on subtask update

Adds a safety net to the writeJSON utility to prevent data loss when updating subtasks via the MCP server.

The MCP process was inadvertently causing the _rawTaggedData property, which holds the complete multi-tag structure, to be lost. When writeJSON received the data for only a single tag, it would overwrite the entire tasks.json file, deleting all other tags.

This fix makes writeJSON more robust. If it receives data that looks like a single, resolved tag without the complete structure, it re-reads the full tasks.json file from disk. It then carefully merges the updated data back into the correct tag within the full structure, preserving all other tags.

* fix: resolve all remaining test failures and improve test reliability

- Fix clear-subtasks test by implementing deep copy of mock data to prevent mutation issues between tests
- Fix add-task test by uncommenting and properly configuring generateTaskFiles call with correct parameters
- Fix analyze-task-complexity tests by properly mocking fs.writeFileSync with shared mock function
- Update test expectations to match actual function signatures and data structures
- Improve mock setup consistency across all test suites
- Ensure all tests now pass (329 total: 318 passed, 11 skipped, 0 failed)

* chore: task management

---------

Co-authored-by: Eyal Toledano <eyal@microangel.so>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: Ibrahim H. <bitsnaps@yahoo.fr>
Co-authored-by: Saksham Goel <sakshamgoel1107@gmail.com>
Co-authored-by: Joe Danziger <joe@ticc.net>
Co-authored-by: Aaron Gabriel Neyer <ag@unforced.org>
2025-06-14 11:04:26 -04:00

968 lines
27 KiB
JavaScript

/**
* contextGatherer.js
* Comprehensive context gathering utility for Task Master AI operations
* Supports task context, file context, project tree, and custom context
*/
import fs from 'fs';
import path from 'path';
import pkg from 'gpt-tokens';
import Fuse from 'fuse.js';
import {
readJSON,
findTaskById,
truncate,
flattenTasksWithSubtasks
} from '../utils.js';
const { encode } = pkg;
/**
* Context Gatherer class for collecting and formatting context from various sources
*/
export class ContextGatherer {
constructor(projectRoot) {
this.projectRoot = projectRoot;
this.tasksPath = path.join(
projectRoot,
'.taskmaster',
'tasks',
'tasks.json'
);
this.allTasks = this._loadAllTasks();
}
_loadAllTasks() {
try {
const data = readJSON(this.tasksPath, this.projectRoot);
const tasks = data?.tasks || [];
return tasks;
} catch (error) {
console.warn(
`Warning: Could not load tasks for ContextGatherer: ${error.message}`
);
return [];
}
}
/**
* Count tokens in a text string using gpt-tokens
* @param {string} text - Text to count tokens for
* @returns {number} Token count
*/
countTokens(text) {
if (!text || typeof text !== 'string') {
return 0;
}
try {
return encode(text).length;
} catch (error) {
// Fallback to rough character-based estimation if tokenizer fails
// Rough estimate: ~4 characters per token for English text
return Math.ceil(text.length / 4);
}
}
/**
* Main method to gather context from multiple sources
* @param {Object} options - Context gathering options
* @param {Array<string>} [options.tasks] - Task/subtask IDs to include
* @param {Array<string>} [options.files] - File paths to include
* @param {string} [options.customContext] - Additional custom context
* @param {boolean} [options.includeProjectTree] - Include project file tree
* @param {string} [options.format] - Output format: 'research', 'chat', 'system-prompt'
* @param {boolean} [options.includeTokenCounts] - Whether to include token breakdown
* @param {string} [options.semanticQuery] - A query string for semantic task searching.
* @param {number} [options.maxSemanticResults] - Max number of semantic results.
* @param {Array<number>} [options.dependencyTasks] - Array of task IDs to build dependency graphs from.
* @returns {Promise<Object>} Object with context string and analysis data
*/
async gather(options = {}) {
const {
tasks = [],
files = [],
customContext = '',
includeProjectTree = false,
format = 'research',
includeTokenCounts = false,
semanticQuery,
maxSemanticResults = 10,
dependencyTasks = []
} = options;
const contextSections = [];
const finalTaskIds = new Set(tasks.map(String));
let analysisData = null;
let tokenBreakdown = null;
// Initialize token breakdown if requested
if (includeTokenCounts) {
tokenBreakdown = {
total: 0,
customContext: null,
tasks: [],
files: [],
projectTree: null
};
}
// Semantic Search
if (semanticQuery && this.allTasks.length > 0) {
const semanticResults = this._performSemanticSearch(
semanticQuery,
maxSemanticResults
);
// Store the analysis data for UI display
analysisData = semanticResults.analysisData;
semanticResults.tasks.forEach((task) => {
finalTaskIds.add(String(task.id));
});
}
// Dependency Graph Analysis
if (dependencyTasks.length > 0) {
const dependencyResults = this._buildDependencyContext(dependencyTasks);
dependencyResults.allRelatedTaskIds.forEach((id) =>
finalTaskIds.add(String(id))
);
// We can format and add dependencyResults.graphVisualization later if needed
}
// Add custom context first
if (customContext && customContext.trim()) {
const formattedCustomContext = this._formatCustomContext(
customContext,
format
);
contextSections.push(formattedCustomContext);
// Calculate tokens for custom context if requested
if (includeTokenCounts) {
tokenBreakdown.customContext = {
tokens: this.countTokens(formattedCustomContext),
characters: formattedCustomContext.length
};
tokenBreakdown.total += tokenBreakdown.customContext.tokens;
}
}
// Gather context for the final list of tasks
if (finalTaskIds.size > 0) {
const taskContextResult = await this._gatherTaskContext(
Array.from(finalTaskIds),
format,
includeTokenCounts
);
if (taskContextResult.context) {
contextSections.push(taskContextResult.context);
// Add task breakdown if token counting is enabled
if (includeTokenCounts && taskContextResult.breakdown) {
tokenBreakdown.tasks = taskContextResult.breakdown;
const taskTokens = taskContextResult.breakdown.reduce(
(sum, task) => sum + task.tokens,
0
);
tokenBreakdown.total += taskTokens;
}
}
}
// Add file context
if (files.length > 0) {
const fileContextResult = await this._gatherFileContext(
files,
format,
includeTokenCounts
);
if (fileContextResult.context) {
contextSections.push(fileContextResult.context);
// Add file breakdown if token counting is enabled
if (includeTokenCounts && fileContextResult.breakdown) {
tokenBreakdown.files = fileContextResult.breakdown;
const fileTokens = fileContextResult.breakdown.reduce(
(sum, file) => sum + file.tokens,
0
);
tokenBreakdown.total += fileTokens;
}
}
}
// Add project tree context
if (includeProjectTree) {
const treeContextResult = await this._gatherProjectTreeContext(
format,
includeTokenCounts
);
if (treeContextResult.context) {
contextSections.push(treeContextResult.context);
// Add tree breakdown if token counting is enabled
if (includeTokenCounts && treeContextResult.breakdown) {
tokenBreakdown.projectTree = treeContextResult.breakdown;
tokenBreakdown.total += treeContextResult.breakdown.tokens;
}
}
}
const finalContext = this._joinContextSections(contextSections, format);
const result = {
context: finalContext,
analysisData: analysisData,
contextSections: contextSections.length,
finalTaskIds: Array.from(finalTaskIds)
};
// Only include tokenBreakdown if it was requested
if (includeTokenCounts) {
result.tokenBreakdown = tokenBreakdown;
}
return result;
}
_performSemanticSearch(query, maxResults) {
const searchableTasks = this.allTasks.map((task) => {
const dependencyTitles =
task.dependencies?.length > 0
? task.dependencies
.map((depId) => this.allTasks.find((t) => t.id === depId)?.title)
.filter(Boolean)
.join(' ')
: '';
return { ...task, dependencyTitles };
});
// Use the exact same approach as add-task.js
const searchOptions = {
includeScore: true, // Return match scores
threshold: 0.4, // Lower threshold = stricter matching (range 0-1)
keys: [
{ name: 'title', weight: 1.5 }, // Title is most important
{ name: 'description', weight: 2 }, // Description is very important
{ name: 'details', weight: 3 }, // Details is most important
// Search dependencies to find tasks that depend on similar things
{ name: 'dependencyTitles', weight: 0.5 }
],
// Sort matches by score (lower is better)
shouldSort: true,
// Allow searching in nested properties
useExtendedSearch: true,
// Return up to 50 matches
limit: 50
};
// Create search index using Fuse.js
const fuse = new Fuse(searchableTasks, searchOptions);
// Extract significant words and phrases from the prompt (like add-task.js does)
const promptWords = query
.toLowerCase()
.replace(/[^\w\s-]/g, ' ') // Replace non-alphanumeric chars with spaces
.split(/\s+/)
.filter((word) => word.length > 3); // Words at least 4 chars
// Use the user's prompt for fuzzy search
const fuzzyResults = fuse.search(query);
// Also search for each significant word to catch different aspects
const wordResults = [];
for (const word of promptWords) {
if (word.length > 5) {
// Only use significant words
const results = fuse.search(word);
if (results.length > 0) {
wordResults.push(...results);
}
}
}
// Merge and deduplicate results
const mergedResults = [...fuzzyResults];
// Add word results that aren't already in fuzzyResults
for (const wordResult of wordResults) {
if (!mergedResults.some((r) => r.item.id === wordResult.item.id)) {
mergedResults.push(wordResult);
}
}
// Group search results by relevance
const highRelevance = mergedResults
.filter((result) => result.score < 0.25)
.map((result) => result.item);
const mediumRelevance = mergedResults
.filter((result) => result.score >= 0.25 && result.score < 0.4)
.map((result) => result.item);
// Get recent tasks (newest first)
const recentTasks = [...this.allTasks]
.sort((a, b) => b.id - a.id)
.slice(0, 5);
// Combine high relevance, medium relevance, and recent tasks
// Prioritize high relevance first
const allRelevantTasks = [...highRelevance];
// Add medium relevance if not already included
for (const task of mediumRelevance) {
if (!allRelevantTasks.some((t) => t.id === task.id)) {
allRelevantTasks.push(task);
}
}
// Add recent tasks if not already included
for (const task of recentTasks) {
if (!allRelevantTasks.some((t) => t.id === task.id)) {
allRelevantTasks.push(task);
}
}
// Get top N results for context
const finalResults = allRelevantTasks.slice(0, maxResults);
return {
tasks: finalResults,
analysisData: {
highRelevance: highRelevance,
mediumRelevance: mediumRelevance,
recentTasks: recentTasks,
allRelevantTasks: allRelevantTasks
}
};
}
_buildDependencyContext(taskIds) {
const { allRelatedTaskIds, graphs, depthMap } =
this._buildDependencyGraphs(taskIds);
if (allRelatedTaskIds.size === 0) return '';
const dependentTasks = Array.from(allRelatedTaskIds)
.map((id) => this.allTasks.find((t) => t.id === id))
.filter(Boolean)
.sort((a, b) => (depthMap.get(a.id) || 0) - (depthMap.get(b.id) || 0));
const uniqueDetailedTasks = dependentTasks.slice(0, 8);
let context = `\nThis task relates to a dependency structure with ${dependentTasks.length} related tasks in the chain.`;
const directDeps = this.allTasks.filter((t) => taskIds.includes(t.id));
if (directDeps.length > 0) {
context += `\n\nDirect dependencies:\n${directDeps
.map((t) => `- Task ${t.id}: ${t.title} - ${t.description}`)
.join('\n')}`;
}
const indirectDeps = dependentTasks.filter((t) => !taskIds.includes(t.id));
if (indirectDeps.length > 0) {
context += `\n\nIndirect dependencies (dependencies of dependencies):\n${indirectDeps
.slice(0, 5)
.map((t) => `- Task ${t.id}: ${t.title} - ${t.description}`)
.join('\n')}`;
if (indirectDeps.length > 5)
context += `\n- ... and ${
indirectDeps.length - 5
} more indirect dependencies`;
}
context += `\n\nDetailed information about dependencies:`;
for (const depTask of uniqueDetailedTasks) {
const isDirect = taskIds.includes(depTask.id)
? ' [DIRECT DEPENDENCY]'
: '';
context += `\n\n------ Task ${depTask.id}${isDirect}: ${depTask.title} ------\n`;
context += `Description: ${depTask.description}\n`;
if (depTask.dependencies?.length) {
context += `Dependencies: ${depTask.dependencies.join(', ')}\n`;
}
if (depTask.details) {
context += `Implementation Details: ${truncate(
depTask.details,
400
)}\n`;
}
}
if (graphs.length > 0) {
context += '\n\nDependency Chain Visualization:';
context += graphs
.map((graph) => this._formatDependencyChain(graph))
.join('');
}
return context;
}
_buildDependencyGraphs(taskIds) {
const visited = new Set();
const depthMap = new Map();
const graphs = [];
for (const id of taskIds) {
const graph = this._buildDependencyGraph(id, visited, depthMap);
if (graph) graphs.push(graph);
}
return { allRelatedTaskIds: visited, graphs, depthMap };
}
_buildDependencyGraph(taskId, visited, depthMap, depth = 0) {
if (visited.has(taskId) || depth > 5) return null; // Limit recursion depth
const task = this.allTasks.find((t) => t.id === taskId);
if (!task) return null;
visited.add(taskId);
if (!depthMap.has(taskId) || depth < depthMap.get(taskId)) {
depthMap.set(taskId, depth);
}
const dependencies =
task.dependencies
?.map((depId) =>
this._buildDependencyGraph(depId, visited, depthMap, depth + 1)
)
.filter(Boolean) || [];
return { ...task, dependencies };
}
_formatDependencyChain(node, prefix = '', isLast = true, depth = 0) {
if (depth > 3) return '';
const connector = isLast ? '└── ' : '├── ';
let result = `${prefix}${connector}Task ${node.id}: ${node.title}`;
if (node.dependencies?.length) {
const childPrefix = prefix + (isLast ? ' ' : '│ ');
result += node.dependencies
.map((dep, index) =>
this._formatDependencyChain(
dep,
childPrefix,
index === node.dependencies.length - 1,
depth + 1
)
)
.join('');
}
return '\n' + result;
}
/**
* Parse task ID strings into structured format
* Supports formats: "15", "15.2", "16,17.1"
* @param {Array<string>} taskIds - Array of task ID strings
* @returns {Array<Object>} Parsed task identifiers
*/
_parseTaskIds(taskIds) {
const parsed = [];
for (const idStr of taskIds) {
if (idStr.includes('.')) {
// Subtask format: "15.2"
const [parentId, subtaskId] = idStr.split('.');
parsed.push({
type: 'subtask',
parentId: parseInt(parentId, 10),
subtaskId: parseInt(subtaskId, 10),
fullId: idStr
});
} else {
// Task format: "15"
parsed.push({
type: 'task',
taskId: parseInt(idStr, 10),
fullId: idStr
});
}
}
return parsed;
}
/**
* Gather context from tasks and subtasks
* @param {Array<string>} taskIds - Task/subtask IDs
* @param {string} format - Output format
* @param {boolean} includeTokenCounts - Whether to include token breakdown
* @returns {Promise<Object>} Task context result with breakdown
*/
async _gatherTaskContext(taskIds, format, includeTokenCounts = false) {
try {
if (!this.allTasks || this.allTasks.length === 0) {
return { context: null, breakdown: [] };
}
const parsedIds = this._parseTaskIds(taskIds);
const contextItems = [];
const breakdown = [];
for (const parsed of parsedIds) {
let formattedItem = null;
let itemInfo = null;
if (parsed.type === 'task') {
const result = findTaskById(this.allTasks, parsed.taskId);
if (result.task) {
formattedItem = this._formatTaskForContext(result.task, format);
itemInfo = {
id: parsed.fullId,
type: 'task',
title: result.task.title,
tokens: includeTokenCounts ? this.countTokens(formattedItem) : 0,
characters: formattedItem.length
};
}
} else if (parsed.type === 'subtask') {
const parentResult = findTaskById(this.allTasks, parsed.parentId);
if (parentResult.task && parentResult.task.subtasks) {
const subtask = parentResult.task.subtasks.find(
(st) => st.id === parsed.subtaskId
);
if (subtask) {
formattedItem = this._formatSubtaskForContext(
subtask,
parentResult.task,
format
);
itemInfo = {
id: parsed.fullId,
type: 'subtask',
title: subtask.title,
parentTitle: parentResult.task.title,
tokens: includeTokenCounts
? this.countTokens(formattedItem)
: 0,
characters: formattedItem.length
};
}
}
}
if (formattedItem && itemInfo) {
contextItems.push(formattedItem);
if (includeTokenCounts) {
breakdown.push(itemInfo);
}
}
}
if (contextItems.length === 0) {
return { context: null, breakdown: [] };
}
const finalContext = this._formatTaskContextSection(contextItems, format);
return {
context: finalContext,
breakdown: includeTokenCounts ? breakdown : []
};
} catch (error) {
console.warn(`Warning: Could not gather task context: ${error.message}`);
return { context: null, breakdown: [] };
}
}
/**
* Format a task for context inclusion
* @param {Object} task - Task object
* @param {string} format - Output format
* @returns {string} Formatted task context
*/
_formatTaskForContext(task, format) {
const sections = [];
sections.push(`**Task ${task.id}: ${task.title}**`);
sections.push(`Description: ${task.description}`);
sections.push(`Status: ${task.status || 'pending'}`);
sections.push(`Priority: ${task.priority || 'medium'}`);
if (task.dependencies && task.dependencies.length > 0) {
sections.push(`Dependencies: ${task.dependencies.join(', ')}`);
}
if (task.details) {
const details = truncate(task.details, 500);
sections.push(`Implementation Details: ${details}`);
}
if (task.testStrategy) {
const testStrategy = truncate(task.testStrategy, 300);
sections.push(`Test Strategy: ${testStrategy}`);
}
if (task.subtasks && task.subtasks.length > 0) {
sections.push(`Subtasks: ${task.subtasks.length} subtasks defined`);
}
return sections.join('\n');
}
/**
* Format a subtask for context inclusion
* @param {Object} subtask - Subtask object
* @param {Object} parentTask - Parent task object
* @param {string} format - Output format
* @returns {string} Formatted subtask context
*/
_formatSubtaskForContext(subtask, parentTask, format) {
const sections = [];
sections.push(
`**Subtask ${parentTask.id}.${subtask.id}: ${subtask.title}**`
);
sections.push(`Parent Task: ${parentTask.title}`);
sections.push(`Description: ${subtask.description}`);
sections.push(`Status: ${subtask.status || 'pending'}`);
if (subtask.dependencies && subtask.dependencies.length > 0) {
sections.push(`Dependencies: ${subtask.dependencies.join(', ')}`);
}
if (subtask.details) {
const details = truncate(subtask.details, 500);
sections.push(`Implementation Details: ${details}`);
}
return sections.join('\n');
}
/**
* Gather context from files
* @param {Array<string>} filePaths - File paths to read
* @param {string} format - Output format
* @param {boolean} includeTokenCounts - Whether to include token breakdown
* @returns {Promise<Object>} File context result with breakdown
*/
async _gatherFileContext(filePaths, format, includeTokenCounts = false) {
const fileContents = [];
const breakdown = [];
for (const filePath of filePaths) {
try {
const fullPath = path.isAbsolute(filePath)
? filePath
: path.join(this.projectRoot, filePath);
if (!fs.existsSync(fullPath)) {
continue;
}
const stats = fs.statSync(fullPath);
if (!stats.isFile()) {
continue;
}
// Check file size (limit to 50KB for context)
if (stats.size > 50 * 1024) {
continue;
}
const content = fs.readFileSync(fullPath, 'utf-8');
const relativePath = path.relative(this.projectRoot, fullPath);
const fileData = {
path: relativePath,
size: stats.size,
content: content,
lastModified: stats.mtime
};
fileContents.push(fileData);
// Calculate tokens for this individual file if requested
if (includeTokenCounts) {
const formattedFile = this._formatSingleFileForContext(
fileData,
format
);
breakdown.push({
path: relativePath,
sizeKB: Math.round(stats.size / 1024),
tokens: this.countTokens(formattedFile),
characters: formattedFile.length
});
}
} catch (error) {
console.warn(
`Warning: Could not read file ${filePath}: ${error.message}`
);
}
}
if (fileContents.length === 0) {
return { context: null, breakdown: [] };
}
const finalContext = this._formatFileContextSection(fileContents, format);
return {
context: finalContext,
breakdown: includeTokenCounts ? breakdown : []
};
}
/**
* Generate project file tree context
* @param {string} format - Output format
* @param {boolean} includeTokenCounts - Whether to include token breakdown
* @returns {Promise<Object>} Project tree context result with breakdown
*/
async _gatherProjectTreeContext(format, includeTokenCounts = false) {
try {
const tree = this._generateFileTree(this.projectRoot, 5); // Max depth 5
const finalContext = this._formatProjectTreeSection(tree, format);
const breakdown = includeTokenCounts
? {
tokens: this.countTokens(finalContext),
characters: finalContext.length,
fileCount: tree.fileCount || 0,
dirCount: tree.dirCount || 0
}
: null;
return {
context: finalContext,
breakdown: breakdown
};
} catch (error) {
console.warn(
`Warning: Could not generate project tree: ${error.message}`
);
return { context: null, breakdown: null };
}
}
/**
* Format a single file for context (used for token counting)
* @param {Object} fileData - File data object
* @param {string} format - Output format
* @returns {string} Formatted file context
*/
_formatSingleFileForContext(fileData, format) {
const header = `**File: ${fileData.path}** (${Math.round(fileData.size / 1024)}KB)`;
const content = `\`\`\`\n${fileData.content}\n\`\`\``;
return `${header}\n\n${content}`;
}
/**
* Generate file tree structure
* @param {string} dirPath - Directory path
* @param {number} maxDepth - Maximum depth to traverse
* @param {number} currentDepth - Current depth
* @returns {Object} File tree structure
*/
_generateFileTree(dirPath, maxDepth, currentDepth = 0) {
const ignoreDirs = [
'.git',
'node_modules',
'.env',
'coverage',
'dist',
'build'
];
const ignoreFiles = ['.DS_Store', '.env', '.env.local', '.env.production'];
if (currentDepth >= maxDepth) {
return null;
}
try {
const items = fs.readdirSync(dirPath);
const tree = {
name: path.basename(dirPath),
type: 'directory',
children: [],
fileCount: 0,
dirCount: 0
};
for (const item of items) {
if (ignoreDirs.includes(item) || ignoreFiles.includes(item)) {
continue;
}
const itemPath = path.join(dirPath, item);
const stats = fs.statSync(itemPath);
if (stats.isDirectory()) {
tree.dirCount++;
if (currentDepth < maxDepth - 1) {
const subtree = this._generateFileTree(
itemPath,
maxDepth,
currentDepth + 1
);
if (subtree) {
tree.children.push(subtree);
}
}
} else {
tree.fileCount++;
tree.children.push({
name: item,
type: 'file',
size: stats.size
});
}
}
return tree;
} catch (error) {
return null;
}
}
/**
* Format custom context section
* @param {string} customContext - Custom context string
* @param {string} format - Output format
* @returns {string} Formatted custom context
*/
_formatCustomContext(customContext, format) {
switch (format) {
case 'research':
return `## Additional Context\n\n${customContext}`;
case 'chat':
return `**Additional Context:**\n${customContext}`;
case 'system-prompt':
return `Additional context: ${customContext}`;
default:
return customContext;
}
}
/**
* Format task context section
* @param {Array<string>} taskItems - Formatted task items
* @param {string} format - Output format
* @returns {string} Formatted task context section
*/
_formatTaskContextSection(taskItems, format) {
switch (format) {
case 'research':
return `## Task Context\n\n${taskItems.join('\n\n---\n\n')}`;
case 'chat':
return `**Task Context:**\n\n${taskItems.join('\n\n')}`;
case 'system-prompt':
return `Task context: ${taskItems.join(' | ')}`;
default:
return taskItems.join('\n\n');
}
}
/**
* Format file context section
* @param {Array<Object>} fileContents - File content objects
* @param {string} format - Output format
* @returns {string} Formatted file context section
*/
_formatFileContextSection(fileContents, format) {
const fileItems = fileContents.map((file) => {
const header = `**File: ${file.path}** (${Math.round(file.size / 1024)}KB)`;
const content = `\`\`\`\n${file.content}\n\`\`\``;
return `${header}\n\n${content}`;
});
switch (format) {
case 'research':
return `## File Context\n\n${fileItems.join('\n\n---\n\n')}`;
case 'chat':
return `**File Context:**\n\n${fileItems.join('\n\n')}`;
case 'system-prompt':
return `File context: ${fileContents.map((f) => `${f.path} (${f.content.substring(0, 200)}...)`).join(' | ')}`;
default:
return fileItems.join('\n\n');
}
}
/**
* Format project tree section
* @param {Object} tree - File tree structure
* @param {string} format - Output format
* @returns {string} Formatted project tree section
*/
_formatProjectTreeSection(tree, format) {
const treeString = this._renderFileTree(tree);
switch (format) {
case 'research':
return `## Project Structure\n\n\`\`\`\n${treeString}\n\`\`\``;
case 'chat':
return `**Project Structure:**\n\`\`\`\n${treeString}\n\`\`\``;
case 'system-prompt':
return `Project structure: ${treeString.replace(/\n/g, ' | ')}`;
default:
return treeString;
}
}
/**
* Render file tree as string
* @param {Object} tree - File tree structure
* @param {string} prefix - Current prefix for indentation
* @returns {string} Rendered tree string
*/
_renderFileTree(tree, prefix = '') {
let result = `${prefix}${tree.name}/`;
if (tree.fileCount > 0 || tree.dirCount > 0) {
result += ` (${tree.fileCount} files, ${tree.dirCount} dirs)`;
}
result += '\n';
if (tree.children) {
tree.children.forEach((child, index) => {
const isLast = index === tree.children.length - 1;
const childPrefix = prefix + (isLast ? '└── ' : '├── ');
const nextPrefix = prefix + (isLast ? ' ' : '│ ');
if (child.type === 'directory') {
result += this._renderFileTree(child, childPrefix);
} else {
result += `${childPrefix}${child.name}\n`;
}
});
}
return result;
}
/**
* Join context sections based on format
* @param {Array<string>} sections - Context sections
* @param {string} format - Output format
* @returns {string} Joined context string
*/
_joinContextSections(sections, format) {
if (sections.length === 0) {
return '';
}
switch (format) {
case 'research':
return sections.join('\n\n---\n\n');
case 'chat':
return sections.join('\n\n');
case 'system-prompt':
return sections.join(' ');
default:
return sections.join('\n\n');
}
}
}
/**
* Factory function to create a context gatherer instance
* @param {string} projectRoot - Project root directory
* @returns {ContextGatherer} Context gatherer instance
*/
export function createContextGatherer(projectRoot) {
return new ContextGatherer(projectRoot);
}
export default ContextGatherer;