yrtv/database/L3/Roadmap/IMPLEMENTATION_ROADMAP.md

# L3 Implementation Roadmap & Checklist

> **Based on**: L3_ARCHITECTURE_PLAN.md v2.0  
> **Start Date**: 2026-01-28  
> **Estimated Duration**: 8-10 days

---

## Quick Start Checklist

### ✅ Pre-requisites
- [x] L1 database完整 (208 matches)
- [x] L2 database完整 (100% coverage, 51,860 rows)
- [x] L2 schema documented
- [x] Profile requirements analyzed
- [x] L3 architecture designed

### 🎯 Implementation Phases

---

## Phase 1: Schema & Infrastructure (Day 1-2)

### 1.1 Create L3 Database Schema
- [ ] Create `database/L3/schema.sql`
  - [ ] dm_player_features (207 columns)
  - [ ] dm_player_match_history
  - [ ] dm_player_map_stats  
  - [ ] dm_player_weapon_stats
  - [ ] All indexes

### 1.2 Initialize L3 Database
- [ ] Update `database/L3/L3_Builder.py` init_db()
- [ ] Run schema creation
- [ ] Verify tables created

### 1.3 Processor Base Classes
- [ ] Create `database/L3/processors/__init__.py`
- [ ] Create `database/L3/processors/base_processor.py`
  - [ ] BaseFeatureProcessor interface
  - [ ] SafeAggregator utility class
  - [ ] Z-score normalization functions

**验收标准**：
```bash
sqlite3 database/L3/L3.db ".tables"
# 应输出: dm_player_features, dm_player_match_history, dm_player_map_stats, dm_player_weapon_stats
```

---

## Phase 2: Tier 1 - Core Processors (Day 3-4)

### 2.1 BasicProcessor Implementation
- [ ] Create `database/L3/processors/basic_processor.py`

**Sub-tasks**:
- [ ] `calculate_basic_stats()` - 15 columns
  - [ ] AVG(rating, rating2, kd, adr, kast, rws) from fact_match_players
  - [ ] AVG(headshot_count), hs_rate = SUM(hs)/SUM(kills)
  - [ ] total_kills, total_deaths, total_assists
  - [ ] kpr, dpr, survival_rate
  
- [ ] `calculate_match_stats()` - 8 columns
  - [ ] win_rate, wins, losses
  - [ ] avg_match_duration from fact_matches
  - [ ] avg_mvps, mvp_rate
  - [ ] avg_elo_change, total_elo_gained from fact_match_teams
  
- [ ] `calculate_weapon_stats()` - 12 columns
  - [ ] avg_awp_kills, awp_usage_rate
  - [ ] avg_knife_kills, avg_zeus_kills, zeus_buy_rate
  - [ ] top_weapon (GROUP BY weapon in fact_round_events)
  - [ ] weapon_diversity (Shannon entropy)
  - [ ] rifle/pistol/smg hs_rates
  
- [ ] `calculate_objective_stats()` - 6 columns
  - [ ] avg_plants, avg_defuses, avg_flash_assists
  - [ ] plant_success_rate, defuse_success_rate
  - [ ] objective_impact (weighted score)

**测试用例**:
```python
features = BasicProcessor.calculate('76561198012345678', conn_l2)
assert 'core_avg_rating' in features
assert features['core_total_kills'] > 0
assert 0 <= features['core_hs_rate'] <= 1
```

---

## Phase 3: Tier 2 - Tactical Processors (Day 4-5)

### 3.1 TacticalProcessor Implementation
- [ ] Create `database/L3/processors/tactical_processor.py`

**Sub-tasks**:
- [ ] `calculate_opening_impact()` - 8 columns
  - [ ] avg_fk, avg_fd from fact_match_players
  - [ ] fk_rate, fd_rate
  - [ ] fk_success_rate (team win when FK)
  - [ ] entry_kill_rate, entry_death_rate
  - [ ] opening_duel_winrate
  
- [ ] `calculate_multikill()` - 6 columns
  - [ ] avg_2k, avg_3k, avg_4k, avg_5k
  - [ ] multikill_rate
  - [ ] ace_count (5k count)
  
- [ ] `calculate_clutch()` - 10 columns
  - [ ] clutch_1v1/1v2_attempts/wins/rate
  - [ ] clutch_1v3_plus aggregated
  - [ ] clutch_impact_score (weighted)
  
- [ ] `calculate_utility()` - 12 columns
  - [ ] util_X_per_round for flash/smoke/molotov/he
  - [ ] util_usage_rate
  - [ ] nade_dmg metrics
  - [ ] flash_efficiency, smoke_timing_score
  - [ ] util_impact_score
  
- [ ] `calculate_economy()` - 8 columns
  - [ ] dmg_per_1k from fact_round_player_economy
  - [ ] kpr/kd for eco/force/full rounds
  - [ ] save_discipline, force_success_rate
  - [ ] eco_efficiency_score

**测试**:
```python
features = TacticalProcessor.calculate('76561198012345678', conn_l2)
assert 'tac_fk_rate' in features
assert features['tac_multikill_rate'] >= 0
```

---

## Phase 4: Tier 3 - Intelligence Processors (Day 5-7)

### 4.1 IntelligenceProcessor Implementation
- [ ] Create `database/L3/processors/intelligence_processor.py`

**Sub-tasks**:
- [ ] `calculate_high_iq_kills()` - 8 columns
  - [ ] wallbang/smoke/blind/noscope kills from fact_round_events flags
  - [ ] Rates: X_kills / total_kills
  - [ ] high_iq_score (weighted formula)
  
- [ ] `calculate_timing_analysis()` - 12 columns
  - [ ] early/mid/late kills by event_time bins (0-30s, 30-60s, 60s+)
  - [ ] timing shares
  - [ ] avg_kill_time, avg_death_time
  - [ ] aggression_index, patience_score
  - [ ] first_contact_time (MIN(event_time) per round)
  
- [ ] `calculate_pressure_performance()` - 10 columns
  - [ ] comeback_kd/rating (when down 4+ rounds)
  - [ ] losing_streak_kd (3+ round loss streak)
  - [ ] matchpoint_kpr/rating (at 15-X or 12-X)
  - [ ] clutch_composure, entry_in_loss
  - [ ] pressure_performance_index, big_moment_score
  - [ ] tilt_resistance
  
- [ ] `calculate_position_mastery()` - 15 columns ⚠️ Complex
  - [ ] site_a/b/mid_control_rate from xyz clustering
  - [ ] favorite_position (most common cluster)
  - [ ] position_diversity (entropy)
  - [ ] rotation_speed (distance between kills)
  - [ ] map_coverage, defensive/aggressive positioning
  - [ ] lurk_tendency, site_anchor_score
  - [ ] spatial_iq_score
  
- [ ] `calculate_trade_network()` - 8 columns
  - [ ] trade_kill_count (kills within 5s of teammate death)
  - [ ] trade_kill_rate
  - [ ] trade_response_time (AVG seconds)
  - [ ] trade_given (deaths traded by teammate)
  - [ ] trade_balance, trade_efficiency
  - [ ] teamwork_score

**Position Mastery特别注意**:
```python
# 需要使用sklearn DBSCAN聚类
from sklearn.cluster import DBSCAN

def cluster_player_positions(steam_id, conn_l2):
    """从fact_round_events提取xyz坐标并聚类"""
    cursor = conn_l2.cursor()
    cursor.execute("""
        SELECT attacker_pos_x, attacker_pos_y, attacker_pos_z
        FROM fact_round_events
        WHERE attacker_steam_id = ?
        AND attacker_pos_x IS NOT NULL
    """, (steam_id,))
    
    coords = cursor.fetchall()
    # DBSCAN clustering...
```

**测试**:
```python
features = IntelligenceProcessor.calculate('76561198012345678', conn_l2)
assert 'int_high_iq_score' in features
assert features['int_timing_early_kill_share'] + features['int_timing_mid_kill_share'] + features['int_timing_late_kill_share'] <= 1.1  # Allow rounding
```

---

## Phase 5: Tier 4 - Meta Processors (Day 7-8)

### 5.1 MetaProcessor Implementation
- [ ] Create `database/L3/processors/meta_processor.py`

**Sub-tasks**:
- [ ] `calculate_stability()` - 8 columns
  - [ ] rating_volatility (STDDEV of last 20 matches)
  - [ ] recent_form_rating (AVG last 10)
  - [ ] win/loss_rating
  - [ ] rating_consistency (100 - volatility_norm)
  - [ ] time_rating_correlation (CORR(duration, rating))
  - [ ] map_stability, elo_tier_stability
  
- [ ] `calculate_side_preference()` - 14 columns
  - [ ] side_ct/t_rating from fact_match_players_ct/t
  - [ ] side_ct/t_kd, win_rate, fk_rate, kast
  - [ ] side_rating_diff, side_kd_diff
  - [ ] side_preference ('CT'/'T'/'Balanced')
  - [ ] side_balance_score
  
- [ ] `calculate_opponent_adaptation()` - 12 columns
  - [ ] vs_lower/similar/higher_elo_rating/kd
  - [ ] Based on fact_match_teams.group_origin_elo差值
  - [ ] elo_adaptation, stomping_score, upset_score
  - [ ] consistency_across_elos, rank_resistance
  - [ ] smurf_detection
  
- [ ] `calculate_map_specialization()` - 10 columns
  - [ ] best/worst_map, best/worst_rating
  - [ ] map_diversity (entropy)
  - [ ] map_pool_size (maps with 5+ matches)
  - [ ] map_specialist_score, map_versatility
  - [ ] comfort_zone_rate, map_adaptation
  
- [ ] `calculate_session_pattern()` - 8 columns
  - [ ] avg_matches_per_day
  - [ ] longest_streak (consecutive days)
  - [ ] weekend/weekday_rating
  - [ ] morning/afternoon/evening/night_rating (based on timestamp)

**测试**:
```python
features = MetaProcessor.calculate('76561198012345678', conn_l2)
assert 'meta_rating_volatility' in features
assert features['meta_side_preference'] in ['CT', 'T', 'Balanced']
```

---

## Phase 6: Tier 5 - Composite Processors (Day 8)

### 6.1 CompositeProcessor Implementation
- [ ] Create `database/L3/processors/composite_processor.py`

**Sub-tasks**:
- [ ] `normalize_and_standardize()` helper
  - [ ] Z-score normalization function
  - [ ] Global mean/std calculation from all players
  - [ ] Map Z-score to 0-100 range
  
- [ ] `calculate_radar_scores()` - 8 scores
  - [ ] score_aim: 25% Rating + 20% KD + 15% ADR + 10% DuelWin + 10% HighEloKD + 20% MultiKill
  - [ ] score_clutch: 25% 1v3+ + 20% MatchPtWin + 20% ComebackKD + 15% PressureEntry + 20% Rating
  - [ ] score_pistol: 30% PistolKills + 30% PistolWin + 20% PistolKD + 20% PistolHS%
  - [ ] score_defense: 35% CT_Rating + 35% T_Rating + 15% CT_FK + 15% T_FK
  - [ ] score_utility: 35% UsageRate + 25% NadeDmg + 20% FlashEff + 20% FlashEnemy
  - [ ] score_stability: 30% (100-Volatility) + 30% LossRating + 20% WinRating + 20% Consistency
  - [ ] score_economy: 50% Dmg/$1k + 30% EcoKPR + 20% SaveRoundKD
  - [ ] score_pace: 40% EntryTiming + 30% TradeSpeed + 30% AggressionIndex
  
- [ ] `calculate_overall_score()` - AVG of 8 scores
  
- [ ] `classify_tier()` - Performance tier
  - [ ] Elite: overall > 75
  - [ ] Advanced: 60-75
  - [ ] Intermediate: 40-60
  - [ ] Beginner: < 40
  
- [ ] `calculate_percentile()` - Rank among all players

**依赖**:
```python
def calculate(steam_id: str, conn_l2: sqlite3.Connection, pre_features: dict) -> dict:
    """
    需要前面4个Tier的特征作为输入
    
    Args:
        pre_features: 包含Tier 1-4的所有特征
    """
    pass
```

**测试**:
```python
# 需要先计算所有前置特征
features = {}
features.update(BasicProcessor.calculate(steam_id, conn_l2))
features.update(TacticalProcessor.calculate(steam_id, conn_l2))
features.update(IntelligenceProcessor.calculate(steam_id, conn_l2))
features.update(MetaProcessor.calculate(steam_id, conn_l2))
composite = CompositeProcessor.calculate(steam_id, conn_l2, features)

assert 0 <= composite['score_aim'] <= 100
assert composite['tier_classification'] in ['Elite', 'Advanced', 'Intermediate', 'Beginner']
```

---

## Phase 7: L3_Builder Integration (Day 8-9)

### 7.1 Main Builder Logic
- [ ] Update `database/L3/L3_Builder.py`
  - [ ] Import all processors
  - [ ] Main loop: iterate all players from dim_players
  - [ ] Call processors in order
  - [ ] _upsert_features() helper
  - [ ] Batch commit every 100 players
  - [ ] Progress logging

```python
def main():
    logger.info("Starting L3 Builder...")
    
    # 1. Init DB
    init_db()
    
    # 2. Connect
    conn_l2 = sqlite3.connect(L2_DB_PATH)
    conn_l3 = sqlite3.connect(L3_DB_PATH)
    
    # 3. Get all players
    cursor = conn_l2.cursor()
    cursor.execute("SELECT DISTINCT steam_id_64 FROM dim_players")
    players = cursor.fetchall()
    
    logger.info(f"Processing {len(players)} players...")
    
    for idx, (steam_id,) in enumerate(players, 1):
        try:
            # 4. Calculate features tier by tier
            features = {}
            features.update(BasicProcessor.calculate(steam_id, conn_l2))
            features.update(TacticalProcessor.calculate(steam_id, conn_l2))
            features.update(IntelligenceProcessor.calculate(steam_id, conn_l2))
            features.update(MetaProcessor.calculate(steam_id, conn_l2))
            features.update(CompositeProcessor.calculate(steam_id, conn_l2, features))
            
            # 5. Upsert to L3
            _upsert_features(conn_l3, steam_id, features)
            
            # 6. Commit batch
            if idx % 100 == 0:
                conn_l3.commit()
                logger.info(f"Processed {idx}/{len(players)} players")
        
        except Exception as e:
            logger.error(f"Error processing {steam_id}: {e}")
    
    conn_l3.commit()
    logger.info("Done!")
```

### 7.2 Auxiliary Tables Population
- [ ] Populate `dm_player_match_history`
  - [ ] FROM fact_match_players JOIN fact_matches
  - [ ] ORDER BY match date
  - [ ] Calculate match_sequence, rolling averages
  
- [ ] Populate `dm_player_map_stats`
  - [ ] GROUP BY steam_id, map_name
  - [ ] FROM fact_match_players
  
- [ ] Populate `dm_player_weapon_stats`
  - [ ] GROUP BY steam_id, weapon_name
  - [ ] FROM fact_round_events
  - [ ] TOP 10 weapons per player

### 7.3 Full Build Test
- [ ] Run: `python database/L3/L3_Builder.py`
- [ ] Verify: All players processed
- [ ] Check: Row counts in all L3 tables
- [ ] Validate: Sample features make sense

**验收标准**:
```sql
SELECT COUNT(*) FROM dm_player_features;  -- 应该 = dim_players count
SELECT AVG(core_avg_rating) FROM dm_player_features;  -- 应该接近1.0
SELECT COUNT(*) FROM dm_player_features WHERE score_aim > 0;  -- 大部分玩家有评分
```

---

## Phase 8: Web Services Refactoring (Day 9-10)

### 8.1 Create PlayerService
- [ ] Create `web/services/player_service.py`

```python
class PlayerService:
    @staticmethod
    def get_player_features(steam_id: str) -> dict:
        """获取完整特征（dm_player_features）"""
        pass
    
    @staticmethod
    def get_player_radar_data(steam_id: str) -> dict:
        """获取雷达图8维数据"""
        pass
    
    @staticmethod
    def get_player_core_stats(steam_id: str) -> dict:
        """获取核心Dashboard数据"""
        pass
    
    @staticmethod
    def get_player_history(steam_id: str, limit: int = 20) -> list:
        """获取历史趋势数据"""
        pass
    
    @staticmethod
    def get_player_map_stats(steam_id: str) -> list:
        """获取各地图统计"""
        pass
    
    @staticmethod
    def get_player_weapon_stats(steam_id: str, top_n: int = 10) -> list:
        """获取Top N武器"""
        pass
    
    @staticmethod
    def get_players_ranking(order_by: str = 'core_avg_rating', 
                           limit: int = 100, 
                           offset: int = 0) -> list:
        """获取排行榜"""
        pass
```

- [ ] Implement all methods
- [ ] Add error handling
- [ ] Add caching (optional)

### 8.2 Refactor Routes
- [ ] Update `web/routes/players.py`
  - [ ] `/profile/<steam_id>` route
  - [ ] Use PlayerService instead of direct DB queries
  - [ ] Pass features dict to template
  
- [ ] Add API endpoints
  - [ ] `/api/players/<steam_id>/features`
  - [ ] `/api/players/ranking`
  - [ ] `/api/players/<steam_id>/history`

### 8.3 Update feature_service.py
- [ ] Mark old rebuild methods as DEPRECATED
- [ ] Redirect to L3_Builder.py
- [ ] Keep query methods for backward compatibility

---

## Phase 9: Frontend Integration (Day 10-11)

### 9.1 Update profile.html Template
- [ ] Dashboard cards: use `features.core_*`
- [ ] Radar chart: use `features.score_*`
- [ ] Trend chart: use `history` data
- [ ] Core Performance section
- [ ] Gunfight section
- [ ] Opening Impact section
- [ ] Clutch section
- [ ] High IQ Kills section
- [ ] Map stats table
- [ ] Weapon stats table

### 9.2 JavaScript Integration
- [ ] Radar chart rendering (Chart.js)
- [ ] Trend chart rendering
- [ ] Dynamic data loading

### 9.3 UI Polish
- [ ] Responsive design
- [ ] Loading states
- [ ] Error handling
- [ ] Tooltips for complex metrics

---

## Phase 10: Testing & Validation (Day 11-12)

### 10.1 Unit Tests
- [ ] Test each processor independently
- [ ] Mock L2 data
- [ ] Verify calculation correctness

### 10.2 Integration Tests
- [ ] Full L3_Builder run
- [ ] Verify all tables populated
- [ ] Check data consistency

### 10.3 Performance Tests
- [ ] Benchmark L3_Builder runtime
- [ ] Profile slow queries
- [ ] Optimize if needed

### 10.4 Data Quality Checks
- [ ] Verify no NULL values where expected
- [ ] Check value ranges (e.g., 0 <= rate <= 1)
- [ ] Validate composite scores (0-100)
- [ ] Cross-check with L2 source data

---

## Success Criteria

### ✅ L3 Database
- [ ] All 4 tables created with correct schemas
- [ ] dm_player_features has 207 columns
- [ ] All players from L2 have corresponding L3 rows
- [ ] No critical NULL values

### ✅ Feature Calculation
- [ ] All 5 processors implemented and tested
- [ ] 207 features calculated correctly
- [ ] Composite scores in 0-100 range
- [ ] Tier classification working

### ✅ Services & Routes
- [ ] PlayerService provides all query methods
- [ ] Routes use services correctly
- [ ] API endpoints return valid JSON
- [ ] No direct DB queries in routes

### ✅ Frontend
- [ ] Profile page renders correctly
- [ ] Radar chart displays 8 dimensions
- [ ] Trend chart shows history
- [ ] All sections populated with data

### ✅ Performance
- [ ] L3_Builder completes in < 20 min for 1000 players
- [ ] Profile page loads in < 200ms
- [ ] No N+1 query problems

---

## Risk Mitigation

### 🔴 High Risk Items
1. **Position Mastery (xyz clustering)**
   - Mitigation: Start with simple grid-based approach, defer ML clustering
   
2. **Composite Score Standardization**
   - Mitigation: Use simple percentile-based normalization as fallback
   
3. **Performance at Scale**
   - Mitigation: Implement incremental updates, add indexes

### 🟡 Medium Risk Items
1. **Time Window Calculations (trades)**
   - Mitigation: Use efficient self-JOIN with time bounds
   
2. **Missing Data Handling**
   - Mitigation: Comprehensive NULL handling, default values

### 🟢 Low Risk Items
1. Basic aggregations (AVG, SUM, COUNT)
2. Service layer refactoring
3. Template updates

---

## Next Actions

**Immediate (Today)**:
1. Create schema.sql
2. Initialize L3.db
3. Create processor base classes

**Tomorrow**:
1. Implement BasicProcessor
2. Test with sample player
3. Start TacticalProcessor

**This Week**:
1. Complete all 5 processors
2. Full L3_Builder run
3. Service refactoring

**Next Week**:
1. Frontend integration
2. Testing & validation
3. Documentation

---

## Notes

- 保持每个processor独立，便于单元测试
- 使用动态SQL避免column count错误
- 所有rate/percentage使用0-1范围存储，UI展示时乘100
- 时间戳统一使用Unix timestamp (INTEGER)
- 遵循"查询不计算"原则：web层只SELECT，不做聚合