Organization Remediation Plan¶
Breaking the Create-Forget-Recreate Cycle¶
Created: 2025-11-02 Status: 🔴 ACTION REQUIRED Time Investment: 2 hours (Week 1), then 30 min/week Expected ROI: 60-80% reduction in duplicate work
Quick Start: The 2-Hour Foundation¶
Step 1: Create Tool Inventory (45 minutes)¶
Action: Create comprehensive tool catalog
cd /Users/bertfrichot/mem-agent-mcp
cat > tools/TOOL_INVENTORY.md << 'EOF'
# Tool Inventory - mem-agent-mcp
**Last Updated**: 2025-11-02
**Total Tools**: 161
**Canonical Tools**: See ⭐ markers
## Quick Navigation
- [Trading Tools](#trading-tools) - Market data, scanning, analytics
- [HubSpot Tools](#hubspot-tools) - CRM integration
- [Memory Tools](#memory-tools) - Entity management, indexing
- [Notion Tools](#notion-tools) - Extended Brain integration
- [System Tools](#system-tools) - Monitoring, health checks
---
## Trading Tools
### Portfolio Management
- ⭐ `portfolio_fetcher.py` - **CANONICAL** - Aggregates Schwab + Alpaca accounts
- `schwab_fetcher.py` - Schwab-specific portfolio and positions
- `alpaca_fetcher.py` - Alpaca-specific portfolio data
- `trading_analytics.py` - P&L, risk metrics, performance analysis
### Market Scanning
- ⭐ `opportunity_scanner_v2.py` - **CANONICAL** - Multi-strategy scanner (momentum, RSI, volume)
- `momentum_scanner.py` - Momentum-only (use if you only need momentum strategy)
- `micro_cap_screener.py` - Micro-cap specific (<$300M market cap)
- ~~`opportunity_scanner.py`~~ - **DEPRECATED** (2025-11-02) - Use v2 instead
- ~~`batch_opportunity_scanner.py`~~ - **DEPRECATED** (2025-11-02) - Functionality merged into v2
### Strategy Testing
- ⭐ `strategy_tester.py` - **CANONICAL** - Backtest trading strategies
- `genetic_optimizer.py` - Multi-objective parameter optimization (NSGA-II)
- `portfolio_genetic_optimizer.py` - Portfolio-level strategy optimization
### Authentication
- ⭐ `schwab_setup.py` - **CANONICAL** - Schwab OAuth authentication
- `schwab_fetcher.py` - Includes auth check functionality
- `alpaca_fetcher.py` - Alpaca authentication
---
## HubSpot Tools
### CRM Data
- ⭐ `hubspot_mcp_server.py` - **CANONICAL** - Real-time CRM MCP server
- `hubspot_company_contacts_exporter.py` - Export company-contact relationships
- `hubspot_data_audit.py` - Audit HubSpot CRM data quality
- `hubspot_full_export.py` - Complete CRM export to CSV
### Workflows
- `hubspot_workflow_fetcher.py` - Fetch workflow configurations
- `hubspot_workflow_router.py` - Route workflow logic
- `link_contacts_to_companies.py` - Auto-link contacts to companies
---
## Memory Tools
### System Health
- ⭐ `memory_health_monitor.py` - **CANONICAL** - Comprehensive system health check
- `session_bootstrap.py` - Session startup context loader
- `check_auto_indexer.py` - Verify auto-indexer is running
### Entity Management
- ⭐ `qdrant_store.py` - **CANONICAL** - Qdrant vector storage interface
- `index_memory.py` - Index entities to Qdrant
- `search_memory.py` - Search indexed entities
### Monitoring
- `metrics_tracker.py` - Collect system metrics
- `monitoring_dashboard.py` - Visualize metrics
---
## Notion Tools
### Project Management
- ⭐ `notion_mcp_server.py` - **CANONICAL** - Notion MCP server
- `notion_project_uploader.py` - Upload projects to Notion
- `notion_schema_inspector.py` - Inspect database schemas
- `sync_session_to_notion.py` - Sync session logs to Notion
### Database Operations
- `list_notion_databases.py` - List all Notion databases
- `get_project_database.py` - Fetch project database details
---
## System Tools
### Monitoring
- ⭐ `open_monitors.sh` - **CANONICAL** - Open all monitoring dashboards
- `health_check.sh` - System health check
- `check_services.sh` - Verify Docker services
### Development
- ⭐ `smart_commit.py` - **CANONICAL** - Organized git commits by category
- `detect_duplicates.py` - Find duplicate/similar files (TODO: CREATE)
- `update_tool_inventory.py` - Auto-update this file (TODO: CREATE)
---
## Usage Examples
### "I need to scan for trading opportunities"
```bash
# Use the canonical scanner with multiple strategies
uv run python3 tools/opportunity_scanner_v2.py
# If you only need momentum:
uv run python3 tools/momentum_scanner.py
# For micro-cap specific:
uv run python3 tools/micro_cap_screener.py
"I need portfolio data"¶
# Get all accounts (Schwab + Alpaca):
uv run python3 tools/portfolio_fetcher.py
# Schwab only:
uv run python3 tools/schwab_fetcher.py
# Alpaca only:
uv run python3 tools/alpaca_fetcher.py
"I need to check system health"¶
Maintenance¶
When adding new tool: 1. Add entry to appropriate category above 2. Mark with ⭐ if it becomes the canonical version 3. Mark old version as ~~DEPRECATED~~ with date 4. Update "Last Updated" date at top
Monthly review: - Delete files marked DEPRECATED >30 days ago - Update tool counts - Verify canonical tools are still best-in-class
Deprecated Tools Archive¶
Tools marked deprecated will be moved to /archive/ after 30 days:
opportunity_scanner.py- DEPRECATED 2025-11-02 (delete after 2025-12-02)batch_opportunity_scanner.py- DEPRECATED 2025-11-02 (delete after 2025-12-02)
EOF
echo "✅ Created tools/TOOL_INVENTORY.md"
### Step 2: Create Project Status Log (15 minutes)
```bash
cat > docs/PROJECT_STATUS.md << 'EOF'
# Project Status Log
**Purpose**: Single source of truth for project completion status
**Rule**: Append entries here instead of creating separate *_COMPLETE.md files
---
## 2025-11-02: HubSpot Multi-MCP Integration
**Status**: ✅ COMPLETE
**Components**:
- Memory Connector: Syncing every 4 hours (9,279 entities)
- Custom CRM MCP: Configured in `.claude/mcp.json` for real-time queries
- Developer MCP: Configured (awaiting Claude Code restart)
**Files Created**:
- `docs/guides/HUBSPOT_INTEGRATION_COMPARISON.md`
- `docs/guides/HUBSPOT_DEVELOPER_MCP_SETUP.md`
**Next Steps**: Restart Claude Code to activate Developer MCP
**Evidence**: Session log `sessions/2025-11/day-02.md`
---
## 2025-11-02: Background Service Health Fixes
**Status**: ✅ COMPLETE
**Components**:
- metrics-collector: Fixed HuggingFace model pre-download
- hubspot-sync: Fixed PATH configuration in LaunchAgent
- .env parsing: Corrected variable names in Claude-Micro-Cap-Experiment/.env
**Evidence**: All LaunchAgents now running cleanly
---
## 2025-10-31: Multi-Strategy Portfolio Foundation
**Status**: ✅ COMPLETE (awaiting 30-day paper trading test)
**Components**:
- Multi-agent trading system implemented
- Genetic optimizer (NSGA-II) for parameter tuning
- Strategy configs: options, crypto, micro_cap channels
- Foundation for agentic workflow (Phase 2)
**Files Created**:
- `/trading_agents/` (568KB multi-agent system)
- `/docs/trading/` (10 strategy documents)
**Next Steps**:
1. 30-day paper trading test
2. Review results on 2025-11-30
3. Decide on live deployment
**Evidence**: See `docs/status/SESSION_SUMMARY_2025_10_31.md`
---
## Template for New Entries
```markdown
## YYYY-MM-DD: Project Name
**Status**: ✅ COMPLETE | 🟡 IN PROGRESS | ⏸️ PAUSED | ❌ BLOCKED
**Components**:
- Component 1: Description
- Component 2: Description
**Files Created/Modified**:
- List of key files
**Next Steps**: (if applicable)
1. Step 1
2. Step 2
**Evidence**: Link to session log or detailed documentation
Maintenance: Review quarterly, archive entries >6 months old to PROJECT_STATUS_ARCHIVE.md EOF
echo "✅ Created docs/PROJECT_STATUS.md"
### Step 3: Create Weekly Cleanup Automation (30 minutes)
```bash
# Add to Makefile
cat >> Makefile << 'EOF'
# ==========================================
# Organization & Cleanup
# ==========================================
.PHONY: weekly-cleanup file-root consolidate-status org-health
weekly-cleanup: ## Run weekly organization cleanup
@echo "🗂️ Weekly Cleanup Report"
@echo "======================================"
@echo ""
@echo "📁 Root Directory Status:"
@ROOT_COUNT=$$(ls -1 *.md *.py 2>/dev/null | wc -l | tr -d ' '); \
echo " Files in root: $$ROOT_COUNT / 15 target"; \
if [ $$ROOT_COUNT -gt 15 ]; then \
echo " ⚠️ OVER TARGET - Run 'make file-root'"; \
else \
echo " ✅ Under target"; \
fi
@echo ""
@echo "📄 Session Summaries to File:"
@ls -1 *SESSION*.md *SUMMARY*.md 2>/dev/null | grep -v ORGANIZATION || echo " ✅ None"
@echo ""
@echo "📋 Status Documents to Consolidate:"
@ls -1 *COMPLETE*.md *SUCCESS*.md 2>/dev/null || echo " ✅ None"
@echo ""
@echo "🔧 Tools to Review:"
@ls -1 tools/*_v2*.py tools/*_backup*.py tools/*_old*.py 2>/dev/null || echo " ✅ None"
@echo ""
@echo "💡 Run 'make file-root' to file loose files automatically"
file-root: ## Automatically file loose files from root
@echo "📁 Filing root directory files..."
@# Move session summaries
@for file in *SESSION*.md *SUMMARY*.md; do \
if [ -f "$$file" ] && [ "$$file" != "ORGANIZATION_REMEDIATION_PLAN.md" ]; then \
DATE=$$(echo $$file | grep -oE '[0-9]{4}-[0-9]{2}-[0-9]{2}' | head -1); \
if [ -n "$$DATE" ]; then \
YEAR=$$(echo $$DATE | cut -d'-' -f1); \
MONTH=$$(echo $$DATE | cut -d'-' -f2); \
DAY=$$(echo $$DATE | cut -d'-' -f3); \
mkdir -p sessions/$$YEAR-$$MONTH; \
mv "$$file" "sessions/$$YEAR-$$MONTH/day-$$DAY.md"; \
echo " ✅ Filed $$file → sessions/$$YEAR-$$MONTH/day-$$DAY.md"; \
fi; \
fi; \
done
@# Move status documents
@for file in *COMPLETE*.md *SUCCESS*.md *VALIDATION*.md; do \
if [ -f "$$file" ]; then \
mv "$$file" docs/status/; \
echo " ✅ Filed $$file → docs/status/"; \
fi; \
done
@echo "✅ Root directory filed"
consolidate-status: ## Consolidate status documents into PROJECT_STATUS.md
@echo "📋 Status Document Consolidation"
@echo "======================================"
@echo ""
@echo "Found status documents:"
@ls -1 docs/status/*COMPLETE*.md docs/status/*SUCCESS*.md 2>/dev/null || echo " None to consolidate"
@echo ""
@echo "⚠️ Manual consolidation required:"
@echo "1. Read each status document"
@echo "2. Add summary entry to docs/PROJECT_STATUS.md"
@echo "3. Move original to archive/ or delete"
org-health: ## Display organizational health metrics
@echo "📊 Organizational Health Dashboard"
@echo "======================================"
@ROOT_MD=$$(ls -1 *.md 2>/dev/null | wc -l | tr -d ' '); \
ROOT_PY=$$(ls -1 *.py 2>/dev/null | wc -l | tr -d ' '); \
ROOT_TOTAL=$$((ROOT_MD + ROOT_PY)); \
echo "Root Directory: $$ROOT_TOTAL files (target: <15)"; \
if [ $$ROOT_TOTAL -gt 15 ]; then echo " Status: 🔴 OVER"; else echo " Status: ✅ GOOD"; fi
@echo ""
@TOOLS_TOTAL=$$(ls -1 tools/*.py 2>/dev/null | wc -l | tr -d ' '); \
echo "Total Tools: $$TOOLS_TOTAL"; \
if [ -f "tools/TOOL_INVENTORY.md" ]; then \
echo " Status: ✅ Inventory exists"; \
else \
echo " Status: ⚠️ No inventory"; \
fi
@echo ""
@DEPRECATED=$$(grep -r "DEPRECATED" tools/*.py 2>/dev/null | wc -l | tr -d ' '); \
echo "Deprecated Tools: $$DEPRECATED"; \
if [ $$DEPRECATED -gt 0 ]; then echo " Status: ⚠️ Review needed"; else echo " Status: ✅ Clean"; fi
@echo ""
@DUPLICATES=$$(ls -1 tools/*_v2*.py tools/*_backup*.py tools/*_old*.py 2>/dev/null | wc -l | tr -d ' '); \
echo "Potential Duplicates: $$DUPLICATES files"; \
if [ $$DUPLICATES -gt 0 ]; then echo " Status: ⚠️ Review needed"; else echo " Status: ✅ Clean"; fi
@echo ""
@if [ -f "docs/PROJECT_STATUS.md" ]; then \
ENTRIES=$$(grep -c "^## 20" docs/PROJECT_STATUS.md); \
echo "Project Status Entries: $$ENTRIES"; \
echo " Status: ✅ Log exists"; \
else \
echo "Project Status Log: Missing"; \
echo " Status: 🔴 CREATE IT"; \
fi
EOF
echo "✅ Added organization targets to Makefile"
Step 4: Update Session Bootstrap (30 minutes)¶
# Modify tools/session_bootstrap.py to include org health
cat >> tools/session_bootstrap.py << 'EOF'
def show_org_health():
"""Display organizational health metrics at session start"""
import os
import glob
print("\n" + "=" * 60)
print("📊 ORGANIZATIONAL HEALTH")
print("=" * 60)
# Root directory check
root_files = len(glob.glob("*.md")) + len(glob.glob("*.py"))
status = "✅" if root_files <= 15 else "⚠️"
print(f"{status} Root directory: {root_files} files (target: <15)")
# Tool inventory check
if os.path.exists("tools/TOOL_INVENTORY.md"):
print("✅ Tool inventory exists")
else:
print("🔴 Tool inventory MISSING - Run Setup Step 1")
# Deprecated tools check
deprecated_count = len(glob.glob("tools/*_v2*.py")) + len(glob.glob("tools/*_backup*.py"))
if deprecated_count > 0:
print(f"⚠️ {deprecated_count} potential duplicate tools - review needed")
else:
print("✅ No obvious duplicates")
# Project status log check
if os.path.exists("docs/PROJECT_STATUS.md"):
print("✅ Project status log exists")
else:
print("🔴 Project status log MISSING - Run Setup Step 2")
print("\n💡 Run 'make weekly-cleanup' for detailed cleanup report")
print("=" * 60 + "\n")
# Add to main() function call
if __name__ == "__main__":
# ... existing bootstrap code ...
show_org_health()
EOF
echo "✅ Updated session_bootstrap.py with org health check"
Implementation Timeline¶
Week 1: Foundation (2 hours)¶
- [x] Step 1: Create TOOL_INVENTORY.md (45 min)
- [x] Step 2: Create PROJECT_STATUS.md (15 min)
- [x] Step 3: Add Makefile automation (30 min)
- [x] Step 4: Update session bootstrap (30 min)
Deliverables:
- Tool discovery mechanism
- Status consolidation system
- Automated cleanup commands
- Health monitoring in /start
Week 2-4: Consolidation (1 hour/week)¶
- [ ] Week 2: Run
make file-root- Clean root directory - [ ] Week 3: Run
make consolidate-status- Merge status docs - [ ] Week 4: Archive deprecated tools, update TOOL_INVENTORY.md
Daily Habits (5 minutes each)¶
End of Session Checklist¶
□ Did I create any new files in root?
→ If yes: Move to proper location NOW
□ Did I create any new tools?
→ If yes: Update TOOL_INVENTORY.md NOW
□ Any project completed today?
→ If yes: Add entry to PROJECT_STATUS.md NOW
□ Any status documents created?
→ If yes: Consolidate to PROJECT_STATUS.md, delete standalone doc
Weekly Ritual (Friday, 30 minutes)¶
# Run the cleanup report
make weekly-cleanup
# If any issues:
make file-root # File loose files
make consolidate-status # Merge status docs
make org-health # Check metrics
# Review:
# - Did duplicates emerge this week?
# - Are tools properly documented?
# - Is root directory clean?
Success Criteria¶
After Week 1 (Foundation Complete):
- ✅ TOOL_INVENTORY.md exists and is populated
- ✅ PROJECT_STATUS.md exists with recent entries
- ✅ make weekly-cleanup command works
- ✅ Session bootstrap shows org health
After Month 1 (Habits Forming): - ✅ Root directory: <20 files (down from 63) - ✅ All new tools documented in TOOL_INVENTORY - ✅ No standalone status docs (all in PROJECT_STATUS.md) - ✅ Zero "I already built this" moments
After Quarter 1 (Sustainable): - ✅ Root directory: <15 files (maintained) - ✅ Tool inventory 100% complete - ✅ All deprecated tools archived or deleted - ✅ 60%+ of tasks solved by existing tools (not recreation)
Monitoring Progress¶
Metrics Dashboard (in /start)¶
make org-health
# Output:
# 📊 Organizational Health Dashboard
# ======================================
# Root Directory: 12 files (target: <15)
# Status: ✅ GOOD
#
# Total Tools: 161
# Status: ✅ Inventory exists
#
# Deprecated Tools: 0
# Status: ✅ Clean
#
# Potential Duplicates: 2 files
# Status: ⚠️ Review needed
#
# Project Status Entries: 8
# Status: ✅ Log exists
Troubleshooting¶
"I created a new tool but forgot to document it"¶
Fix:
# Add to tools/TOOL_INVENTORY.md under appropriate category
# Format:
# - `new_tool.py` - Description of what it does
"Root directory is cluttered again"¶
Fix:
"I can't find a tool I know I built"¶
Fix:
# Check the inventory first
cat tools/TOOL_INVENTORY.md | grep -i "keyword"
# If not there, search by name
ls tools/ | grep -i "keyword"
# If found but not in inventory
# → Add it to TOOL_INVENTORY.md NOW before you forget
"Created duplicate status document by accident"¶
Fix:
# Consolidate to PROJECT_STATUS.md
cat docs/status/MY_FEATURE_COMPLETE.md
# Copy relevant info to PROJECT_STATUS.md
echo "## $(date +%Y-%m-%d): Feature Name" >> docs/PROJECT_STATUS.md
# ... add details ...
# Delete the standalone document
rm docs/status/MY_FEATURE_COMPLETE.md
Next Actions (Choose 1-2 to start)¶
Highest Impact:
1. ✅ Create TOOL_INVENTORY.md (Step 1) - Enables discovery
2. ✅ Create PROJECT_STATUS.md (Step 2) - Stops status doc proliferation
3. ✅ Run make file-root - Immediate root cleanup
Quick Wins:
4. ✅ Add make org-health to Makefile - Visibility into problem
5. ✅ Add org health to /start command - Daily reminder
After Foundation:
6. Archive deprecated tools (opportunity_scanner.py, batch_opportunity_scanner.py)
7. Consolidate HubSpot status documents (6 files → 1 entry in PROJECT_STATUS.md)
8. Set Friday reminder for make weekly-cleanup
Status: Ready to implement
Next: Execute Step 1-4 (2 hours total)
Then: Run make weekly-cleanup every Friday
Success: Zero duplicate work within 30 days
Last Updated: 2025-11-02
Related: See docs/architecture/ULTRATHINK_ORGANIZATION_ANALYSIS.md for deep analysis