0.4.2: RDD uploaded. Fixed some README.
This commit is contained in:
54
README.md
54
README.md
@@ -1,4 +1,4 @@
|
||||
# YRTV 项目说明
|
||||
# YRTV 项目说明 till 0.4.1
|
||||
|
||||
## 项目概览
|
||||
yrtv这一块。
|
||||
@@ -41,47 +41,10 @@ yrtv/
|
||||
```
|
||||
|
||||
## 环境要求
|
||||
- Python 3.9+
|
||||
- Python 3.11.4+
|
||||
- Playwright(下载器依赖)
|
||||
- pandas、numpy(校验脚本依赖)
|
||||
|
||||
项目默认 Python 路径:
|
||||
```
|
||||
C:/ProgramData/anaconda3/python.exe
|
||||
```
|
||||
|
||||
## 快速开始
|
||||
### 1. 下载比赛数据
|
||||
进入项目根目录运行(默认示例 URL):
|
||||
```
|
||||
C:/ProgramData/anaconda3/python.exe downloader/downloader.py
|
||||
```
|
||||
|
||||
指定比赛 URL:
|
||||
```
|
||||
C:/ProgramData/anaconda3/python.exe downloader/downloader.py --url https://arena.5eplay.com/data/match/g161-20260118222715609322516
|
||||
```
|
||||
|
||||
批量下载(从文本文件读取 URL 列表):
|
||||
```
|
||||
C:/ProgramData/anaconda3/python.exe downloader/downloader.py --url-list downloader/gamelist/match_list_2026.txt
|
||||
```
|
||||
|
||||
### 2. 生成 L1A 数据库
|
||||
```
|
||||
C:/ProgramData/anaconda3/python.exe ETL/L1A.py
|
||||
```
|
||||
|
||||
### 3. 生成 L2 数据库
|
||||
```
|
||||
C:/ProgramData/anaconda3/python.exe ETL/L2_Builder.py
|
||||
```
|
||||
|
||||
### 4. 校验与覆盖检查
|
||||
```
|
||||
C:/ProgramData/anaconda3/python.exe ETL/verify/verify_L2.py
|
||||
C:/ProgramData/anaconda3/python.exe ETL/verify/verify_deep.py
|
||||
```
|
||||
|
||||
## 数据库层级说明
|
||||
### L1A
|
||||
@@ -108,11 +71,6 @@ C:/ProgramData/anaconda3/python.exe ETL/verify/verify_deep.py
|
||||
## JSON Schema 抽取工具
|
||||
用于分析大量 `iframe_network.json` 的字段结构与覆盖情况,支持动态 Key 归并与多格式输出。
|
||||
|
||||
常用命令:
|
||||
```
|
||||
C:/ProgramData/anaconda3/python.exe utils/json_extractor/main.py
|
||||
```
|
||||
|
||||
输出内容通常位于 `output_reports/` 或 `database/original_json_schema/`,包括:
|
||||
- `schema_summary.md`:结构概览
|
||||
- `schema_flat.csv`:扁平字段列表
|
||||
@@ -125,11 +83,3 @@ L2 中 `fact_matches.data_source_type` 用于区分数据来源与字段覆盖
|
||||
- `unknown`:无法识别来源
|
||||
|
||||
入库逻辑保持互斥:同一场比赛只会按其来源覆盖相应字段,避免重复或冲突。
|
||||
|
||||
## 常用文件定位
|
||||
- 下载器入口:[downloader.py](file:///c:/Users/Administrator/Documents/trae_projects/yrtv/downloader/downloader.py)
|
||||
- L1A 入库脚本:[L1A.py](file:///c:/Users/Administrator/Documents/trae_projects/yrtv/ETL/L1A.py)
|
||||
- L2 构建脚本:[L2_Builder.py](file:///c:/Users/Administrator/Documents/trae_projects/yrtv/ETL/L2_Builder.py)
|
||||
- L2 Schema:[schema.sql](file:///c:/Users/Administrator/Documents/trae_projects/yrtv/database/L2/schema.sql)
|
||||
- 覆盖检查:[verify_L2.py](file:///c:/Users/Administrator/Documents/trae_projects/yrtv/ETL/verify/verify_L2.py)
|
||||
- 深度校验:[verify_deep.py](file:///c:/Users/Administrator/Documents/trae_projects/yrtv/ETL/verify/verify_deep.py)
|
||||
|
||||
Reference in New Issue
Block a user