feat: moderation

This commit is contained in:
zhr 2026-09-29 16:37:36 +08:00
parent 45720ae2e1
commit 7f2567cf15
51 changed files with 3091 additions and 54 deletions

View File

@ -29,7 +29,7 @@ location ~ ^/api/admin/spark/users/export-zip {
}
# spark-server:IM / 群 / 投诉 / 排行 / 用户检索 / 音乐列表与 V3 账号运营
location ~ ^/api/admin/(spark|pop)/(im|groups|complaints|ops-catalog|rankings|music)(/|$) {
location ~ ^/api/admin/(spark|pop)/(im|groups|complaints|ops-catalog|rankings|music|sensitive)(/|$) {
proxy_pass http://spark_im_admin;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;

View File

@ -24,3 +24,145 @@
### 验证
- `make test`: pass
## 11:24 — 敏感词发送处罚详细设计
### 概要
- 对照当前发送链、用户状态、Admin 封禁与 `StatusHidden`,写成可实施的机审设计:`moderation` 模块、AccountGate、分类处罚、表结构、错误码与分期;v1 不做 hide。
### 变更范围
| 类型 | 说明 |
|------|------|
| 文档 | `docs/moderation.md` |
### 主要文件
- `docs/moderation.md` — IM 文本敏感词与处罚设计稿
### 验证
- `make test`: skipped(纯文档,无编译/测试面)
### 关联
- 对话:文本发送敏感词分类处罚
## 11:35 — 敏感词匹配算法设计
### 概要
- 补全匹配:逐 rune 折叠+丢分隔、Aho-Corasick、区间映回原文、replace 合并打码;v1 无简繁、无英文词边界。
### 变更范围
| 类型 | 说明 |
|------|------|
| 文档 | `docs/moderation.md` §6 |
### 主要文件
- `docs/moderation.md` — 匹配流水线与用例
### 验证
- `make test`: skipped(纯文档)
### 关联
- 对话:敏感词如何匹配
## 15:10 — 敏感词机审 v1 实现
### 概要
- 落地 `pkg/sensitive`(Fold + Aho-Corasick)、`moderation` 模块、发送链 AccountGate/Scan/处罚、`40302`/`40303`、Admin 词库与命中查询;封禁与机审 disable 统一踢线。
### 变更范围
| 类型 | 说明 |
|------|------|
| 功能 | 文本发送敏感词;全站禁言;封号踢线 |
| 迁移 | `0028_moderation.sql` |
| HTTP Admin | `/api/admin/spark|pop/sensitive-words*`、`sensitive-hits` |
### 主要文件
- `migrations/0028_moderation.sql`
- `internal/pkg/sensitive/`、`internal/module/moderation/`
- `internal/module/message/logic/send_moderation.go`
- `internal/module/user/logic/account_send.go`
- `internal/bootstrap/app.go`、`internal/transport/http/router_admin.go`
### 验证
- `make test`: pass
### 关联
- `docs/moderation.md`
## 15:32 — 敏感词种子 SQL 与 OpenAPI
### 概要
- 新增 `0029_sensitive_words_seed.sql`(按分类默认处罚 16 条示例词);OpenAPI 增加 `admin-moderation` 路径与 schema;envelope 增加 `40302`/`40303`。
### 变更范围
| 类型 | 说明 |
|------|------|
| 迁移 | `0029_sensitive_words_seed.sql` |
| 文档 | `internal/transport/http/docs/openapi.yaml` |
| API | 词条/命中 JSON 字段 snake_case |
### 主要文件
- `migrations/0029_sensitive_words_seed.sql`
- `internal/transport/http/docs/openapi.yaml`
- `internal/module/moderation/entity/word.go`
### 验证
- `make test`: pass
## 15:55 — OpenAPI 管理端 tag 统一为 admin
### 概要
- 不改 Gin 路由;去掉 tag `admin-music`,音乐与敏感词等管理接口统一为 `admin`,description 按 `…/spark/im|music|sensitive-words` 等路径分模块。
### 变更范围
| 类型 | 说明 |
|------|------|
| 文档 | `openapi.yaml`、`docs/moderation.md`、`docs/popadmin-music-novels-deploy.md` |
### 主要文件
- `internal/transport/http/docs/openapi.yaml`
### 验证
- `make test`: pass
## 15:55 — OpenAPI 管理端 tag 统一为 admin
### 概要
- 不改 Gin 路由;去掉 tag `admin-music`,音乐与敏感词等统一为 `admin`,description 按 `…/spark/im|music|sensitive-words` 等路径分模块。
### 变更范围
| 类型 | 说明 |
|------|------|
| 文档 | `openapi.yaml`、`docs/moderation.md`、`docs/popadmin-music-novels-deploy.md` |
### 主要文件
- `internal/transport/http/docs/openapi.yaml`
### 验证
- `make test`: pass

515
docs/moderation.md Normal file
View File

@ -0,0 +1,515 @@
# 敏感词与发送处罚
> 对照:IM 文本发送机审(词库 + 分类处罚)
> 形态:模块化单体内独立业务 `moderation`,挂在现有 `message/logic.SendMessage` 上
> 状态:**v1 已落地**(发送闸 + 文本机审 + Admin 词库/命中 + 封禁踢线)。部署前执行 `migrations/0028_moderation.sql`、`0029_sensitive_words_seed.sql`。
> 硬约束:`docs/architecture-v3.md` C1–C7。不上 Kafka、K8s、微服务。表一律 `t_*`。
本文按 **2026-09-29 代码** 写成可实施说明,不是空想架构。HTTP 与 TCP 已共用 `internal/module/message/logic`;Gin / gnet 只绑参。`logic` 测试禁止连 MySQL。
---
# 0. 范围
## 0.1 做(v1)
| 能力 | 说明 |
| --- | --- |
| 文本机审 | 仅 `content.t = 1` 的 `text.body`(单聊 `t_private_message` / 群聊 `t_group_message`) |
| 分类 | 涉黑 / 涉毒 / 涉黄 / 涉赌 / 涉政 / 宗教 / 辱骂 / 广告 / 其他 |
| 处罚 | 按词条 `action` 执行;一条消息多命中取 **最严** action |
| 账号闸 | 发送任意类型前检查 `t_users.status` 与 `send_muted_until` |
| 封号踢线 | `disable` 与 Admin 封禁:`status=0` + JWT `LogoutAll` + TCP `KickUser` |
| 词库运营 | Admin CRUD + 进程内自动机热替换 |
| 审计 | `t_sensitive_hits`(片段,不存全文) |
## 0.2 明确不做(v1)
- 图片 OCR、语音 ASR、视频抽帧、谐音/拼音/变形字模型
- 朋友圈 `moment`、广场 `square`、昵称、群公告、文件名、位置名称(同一 `Scan` 二期再挂)
- 独立审核微服务、Kafka、按词 `LIKE` 打发送热路径
- 把处罚做成群 `speak_muted` / `muted_all`(那是群维度,已由 `group/logic.CanSend` 处理)
- 用现有 Admin「白名单 / 异常标记」接口(二者目前是空实现,忽略写入)
- 给客户端回传命中原文(避免探测词库)
## 0.3 和现码的关系
| 现码 | 敏感词怎么接 |
| --- | --- |
| `SendMessage`:解 content → `prepareSend` → persist → `deliver:queue` → ACK | 在 persist **之前** 扫词;reject / mute / disable **不 INSERT、不 ACK**(C5) |
| HTTP `handler/message`、TCP `handleSendMessage` | 零改业务,只多注入依赖 |
| `FriendGate` / `GroupGate` | 并列增加 `AccountGate`、`Filter` |
| `users.status` 0/1;登录拒绝停用账号 | 发送侧补同一检查;停用 JWT 残留不能再发 |
| Admin `BanByEmail` 只改 status | 与 `disable` 共用踢线;封禁补 `LogoutAll`+`KickUser` |
| 注销 `revokeAccess` | 封号复用同一对接口 |
| `Message.StatusHidden=3`;历史 `status <> 3` | `hide`:落库 Hidden、**不** Push;会话同步已看不到 |
| 广场 `AuditPending/Approved` | **不要**和 IM Hidden 混用;广场二期自己打 Audit |
| `friend.Report` | 用户举人对人,与词库命中分流;hits 给运营 |
| `reqctx.RealIP` | 命中审计写 PROXY IP(C6) |
| `GET /v1/meta/enums` | 增加 category / action / envelope `40302` |
| 表前缀 | `t_sensitive_words`、`t_sensitive_hits`;用户列加在 `t_users` |
| V3 | 怎么守 |
| --- | --- |
| C1 | 聊天仍是 MsgId;词库行 id 自增;hits 用 Snowflake |
| C2 | 扫词只在 TaskPool 的 SEND 上;EventLoop 不碰词库 |
| C5 | 未 COMMIT+enqueue 不 ACK;reject 路径无行 |
| C6 | hits.real_ip 用 RealIP |
| C7 | 不写会话 ZSET;hide 不 fanout |
---
# 1. 总览
```text
客户端 SEND_MESSAGE / POST 发消息
│ TaskPool(已有)
▼
message/logic.SendMessage
│
├─ AccountGate.CanSend(uid) 停用 / 全站禁言未到期 → 403
├─ prepareSend 好友 / 群禁言(已有)
├─ 若文本:Filter.Scan(body)
│ └─ Apply(hit):处罚写库 → 改写或拒绝
├─ persistSend CreateAndBump
└─ DeliverQueue.Push hide 跳过
│
Admin ── moderation/logic ── t_sensitive_words
内存 AC 自动机(热替换)
```
匹配器无 I/O:`internal/pkg/sensitive`(stdlib)。词表从 MySQL 加载到进程;单进程单体改词后直接 Rebuild,不上 Redis PubSub。
---
# 2. 模块与目录
```text
internal/pkg/sensitive/ Aho-Corasick + 文本规范化(无 DSN)
internal/module/moderation/
entity/ Category, Action, Word, Hit, ScanResult
model/ word.go, hit.go, matcher.go(持有自动机;TableName → t_*)
logic/ scan.go, apply.go, words.go, hits.go, reload.go, service.go
internal/transport/http/handler/admin/ 词库与命中(薄 MVC)
```
`message/logic.Service` 增加字段,**不** import `moderation/model`:
```go
type AccountGate interface {
AssertCanSend(ctx context.Context, userID int64) error
}
type TextFilter interface {
// Scan 无命中返回 (nil, nil)。
Scan(ctx context.Context, text string) (*moderationentity.ScanResult, error)
}
type Penalty interface {
// Apply 在拒绝发送前落处罚。失败返回 error(不 ACK)。
Apply(ctx context.Context, userID, conversationID int64, in *moderationentity.ApplyInput) error
}
```
`bootstrap`:`moderation.NewService(...)` → 注入 `NewService(..., account, filter, penalty)`。`filter`/`penalty` 可为同一 `*moderation.Service`。测试用内存词表 stub。
`logic` 一文件一职能:`scan.go`、`apply.go`、`words_admin.go`、`hits.go`。
---
# 3. 发送时序(改 `send.go`)
与现有函数对齐,插入点固定:
```text
ready / validate / UnmarshalContent // 已有
AccountGate.AssertCanSend(userID) // 新增:所有消息类型
prepareSend // 已有:好友、群 CanSend
checkReplyTo / checkMentions // 已有
若 content.Type == TypeText && Text != nil:
res, err := Filter.Scan(ctx, Text.Body)
按 res.Action:
log → Apply 只记 hits,body 不变
replace → 改 Text.Body 为打码串,再 Marshal
reject → Apply 记 hits,return 40302
hide → Apply 记 hits;msg.Status = StatusHidden;Marshal 原文或打码(见 5.4)
mute → Apply 写 send_muted_until + hits,return 40302
disable → Apply 停用+踢线 + hits,return 40302
Marshal(replace 之后必须重新 Marshal)
stampTargets / assignID / persistSend // 已有
若 Status != Hidden:enqueueMessage // hide 不入队
SEND_ACK
```
**幂等:** `client_msg_id` 在 reject 时无行,客户端重试会再次 Scan。`replace` 成功后重试走现有 `persistSend` 的 Duplicate → 返回已落库行,不再二次打码。
**处罚写库失败:** 返回 `50001`,不 persist 消息(与 C5 一致:没有成功处罚就当发送失败)。
**多命中:** `Scan` 返回全部 hit,`Action = max(severity)`;hits 表一行一词或一行一消息带 JSON word_ids(v1 建议 **一行一词** 便于按类统计)。
---
# 4. 闭集 enum
数字稳定,不回收。entity `Valid()` + OpenAPI `$ref` + `GET /v1/meta/enums` + i18n `enum.sensitive_category.*` / `enum.sensitive_action.*`。
## 4.1 Category(词条分类,运营统计)
| 值 | name | 中文 |
| --- | --- | --- |
| 1 | `crime` | 涉黑 |
| 2 | `drug` | 涉毒 |
| 3 | `porn` | 涉黄 |
| 4 | `gambling` | 涉赌 |
| 5 | `politics` | 涉政 |
| 6 | `religion` | 宗教 |
| 7 | `abuse` | 辱骂 |
| 8 | `ad` | 广告 |
| 9 | `other` | 其他 |
与 `friend.ReportReason`(1 垃圾 2 骚扰 3 假身份 4 其他)**不是同一套**,禁止复用。
## 4.2 Action(处罚,按严厉程度递增)
| 值 | name | 行为摘要 |
| --- | --- | --- |
| 1 | `log` | 发送成功;只审计 |
| 2 | `replace` | 命中处替换为 `***` 后发送 |
| 3 | `reject` | 不落库;40302 |
| 4 | `hide` | 落库 `status=3`;不 deliver |
| 5 | `mute` | 全站禁言至 `now+mute_seconds`;不落库;40302 |
| 6 | `disable` | `status=0` + LogoutAll + KickUser;不落库;40302 |
比较用整数大小即可(1&lt;…&lt;6)。
## 4.3 类别默认 action(词条可覆盖)
实施时写入种子数据或 Admin 缺省,**不是**写死在 Scan 里:
| Category | 默认 Action | mute_seconds |
| --- | --- | --- |
| 涉政 / 涉黑 / 涉毒 | `disable` | — |
| 涉赌 / 宗教 / 涉黄 | `reject` | — |
| 辱骂 | `replace` | — |
| 广告 | `log` | — |
| 其他 | `reject` | — |
产品若改「涉政先禁言再人工」,只改默认/词条,不改发送链。
---
# 5. 处罚细节(对齐现码)
## 5.1 AccountGate(所有消息类型)
`t_users.status != 1` → `40301` 或与登录一致的 `ErrForbidden`(停用账号)。
`send_muted_until != NULL && until > now` → `40303` `send_muted`(与内容违规分开,便于客户端展示「禁言至某时」)。无该列或已到期则放行。
发送图片时若仍在禁言期,同样拦截(禁言是账号能力,不是「只禁文本」)。
## 5.2 mute
`UPDATE t_users SET send_muted_until = GREATEST(IFNULL(send_muted_until,0), :until)` 避免短禁言覆盖长禁言。`mute_seconds` 来自 **胜出那条词**;多词 mute 取最长秒数。
不调用群 `speak_muted`。
## 5.3 disable
与注销踢线同序、不删用户行:
1. `status = 0`
2. `JWT.LogoutAll`
3. `Session.KickUser`
4. KICK 文案新 key `notify.kick.banned`(现有仅 replaced / logout / deleted)
Admin `BanByEmail` **改为同一套**,否则运营手点封禁与机审封号行为不一致。
## 5.4 hide
- `msg.Status = StatusHidden` 后 `CreateAndBump`(会话 last_msg 是否 bump:建议 **仍 bump**,否则对端游标异常;但 Push 跳过,对端靠同步也过滤 Hidden,last 预览可能短暂指向不可见消息——v1 接受;二期可 hide 不 bump)。
- **不** `queue.Push`。
- 发送方历史同样 `status <> 3`,自己也看不到。v1 不做成「仅自己可见」。
- body:默认存原文便于运营抽查 Admin IM;客户端不可见即达到目的。
## 5.5 replace
按 §6 把规范化命中映回原文 byte 区间,合并重叠后每段换成 `***`。映射断言失败则按 reject。重新 `content.Marshal()` 再入库。
## 5.6 log
正常 persist + enqueue;只写 hits。
---
# 6. 匹配
发送热路径禁止「对每个词 `strings.Contains` / SQL `LIKE` / 用户正则」。词是字面量;一次线性扫描同时找全部词。匹配发生在 **规范化后的 rune 流** 上,再把命中区间 **映回原文**,供 `replace` 和 `excerpt`。
## 6.1 目标与不做
| 要挡住 | 例子 | v1 是否覆盖 |
| --- | --- | --- |
| 连续子串 | `网络赌博` 含词 `赌博` | 是 |
| 夹空白 / 标点 | `赌 博`、`赌。博` | 是(规范化时丢掉分隔符) |
| 全角 / 兼容字 | `賭博`、`AD` | 是(逐 rune 折叠) |
| 大小写英文 | `Casino` | 是(ToLower) |
| 零宽插入 | `赌\u200b博` | 是(丢掉 Cf) |
| 英文词中词 | `ass` ⊂ `class` | **否**(无词边界;中文 IM 优先)。二期可加 `whole_word` |
| 谐音 / 拆字 / 火星文模型 | `杜博`、`氵+者` | **否** |
| 图片里的字 | — | **否** |
词条 **只存字面**,没有正则,避免 ReDoS。
## 6.2 两段式流水线
```text
text.body(UTF-8 原文,最多按现有消息体上限截断扫描)
│
▼ FoldAndIndex(词库入库与 Scan 同一函数)
norm runes + origFrom[] / origTo[]
│ origFrom[i] = 第 i 个保留 rune 在原文的 byte 起点
│ origTo[i] = 该 rune 的 byte 终点(开区间)
▼
Aho-Corasick.Find(norm) 输出 [n0, n1) 规范化下标 + word_id
│
▼
映回原文 [origFrom[n0], origTo[n1-1])
多命中:审计保留全部;处罚取 max(action)
replace:原文区间合并后再打码
```
**禁止**先对整句做 NFKC 再匹配:兼容分解会改变长度(`fi`→`fi`),区间无法稳定映回,`replace` 只能整句 `***`。v1 用 **逐 rune 折叠 + 丢弃分隔**,映射一一对应。
## 6.3 FoldAndIndex(规范化)
对原文按 UTF-8 解成 rune,从左到右:
1. **丢掉**:Unicode 类别 `Cf`(含 U+200B/U+FEFF)、`Zs`、`Zl`、`Zp`;ASCII 空白;常见标点(ASCII ``!"#$%&'()*+,-./:;<=>?@[\]^_`{|}~`` 以及 `,。!?、;:…·「」『』【】()`)。
2. **保留则折叠**:`unicode.ToLower`;全角拉丁/数字收到半角(`A`→`a`,可用 `golang.org/x/text/width` 的 `Narrow`,或一张小映射表)。不把一个 rune 拆成多个;若折叠结果多于一个 rune,只取第一个(极端兼容字,可测)。
3. 输出 `norm []rune` 与平行数组 `origFrom` / `origTo`。
**入库:** Admin 提交的 `word_raw` 走同一函数得到 `word`(规范化串)。`word` 为空(全是标点)→ 400 拒绝。`uk_word` 建在规范化串上,避免「赌博」与「赌 博」两条。
**扫描:** `text.body` 同样 Fold;`len(norm)==0` → 无命中。
**最短词:** 允许 1 个 rune,但种子词建议 ≥2。单字「黄」误伤由运营自己负责。
## 6.4 自动机
标准 **Aho-Corasick**:
- 模式 = 各 `enabled=1` 的 `word`(已是规范化串)。
- 输出链:某位置结束时,报告所有落在该节点上的词(含后缀,例如同时命中 `赌` 与 `赌博`)。
- 复杂度:建树 O(Σ|word|);扫一句 O(|norm| + 命中次数)。千~万词、单条约 2KB 正文,CPU 可接受。
- 实现放 `internal/pkg/sensitive`,无 DSN。节点上挂 `[]patternID`(下标进内存词表:id/category/action/mute_seconds)。
`enabled=0` 不进树。Reload 时整树重建,`atomic.Pointer` 交换,Scan 无锁读。
## 6.5 命中如何变成处罚和打码
对每次 AC 命中 `(word_id, n0, n1)`:
- 原文区间:`[origFrom[n0], origTo[n1-1])`(byte)。
- `excerpt`:原文该区间,两端最多再扩 8 rune,总长截断 128(审计用,不回客户端)。
- `ScanResult.Hits` 保留全部;`ScanResult.Action = max(hits.action)`;mute 秒数取 **胜出 action 为 mute 的那些词里最长**。
**replace(仅当最严 action 就是 replace):**
1. 收集所有命中的原文 byte 区间。
2. 按起点排序,合并重叠或相邻区间。
3. 每个合并区间替换成三个 ASCII `*`(固定宽度,不按原词长度拉长)。
4. 拼回新 `body`,再 `content.Marshal()`。
若最严已是 reject/mute/disable,不打码、不落库。
**映射失败:** Fold 保证 `n1-1 < len(origTo)`;实现上断言失败则该次 Scan 当 reject(保守),打日志。不要静默放行。
## 6.6 词边界(v1 明确放弃)
中文无空格,用「规范化后的连续子串」= 产品语义。英文 `scunthorpe` 类误伤不在 v1 修。运营用更长短语,或二期 `whole_word`:仅当命中两端在原文上为非字母。
## 6.7 扫描上限
与消息体校验同一上限(现有 content 长度闸)。Matcher 再设硬顶(例如 8192 rune),超长截断再扫——只为绑 CPU,截断本身不算绕过(超长消息应已被发送校验拒绝)。
## 6.8 热更新
Admin 写 `t_sensitive_words` 成功后 `Reload()`:`WHERE enabled=1` 取出全部 `word`,Fold 校验与库内一致(不一致则跳过该行并打错误日志),建新 AC,交换指针。单进程,不上 Redis。logic 测试用内存 `[]Word` 调 `Build`,不连 MySQL。
## 6.9 包职责
| 包 | 职责 |
| --- | --- |
| `pkg/sensitive` | `FoldAndIndex`、`Matcher.Find`、区间合并;纯函数 + 结构体 |
| `moderation/model` | 持有 `atomic` 自动机、从 MySQL 装词 |
| `moderation/logic` | Scan → entity.ScanResult;Apply 处罚 |
| `message/logic` | 调 Filter,不关心 AC |
## 6.10 匹配用例(实施时写成 `pkg/sensitive` 测试)
| 原文 | 词库 | 期望 |
| --- | --- | --- |
| `今晚赌博` | `赌博` | 命中;replace 后含 `***` |
| `今晚赌 博啊` | `赌博` | 命中(空白已丢) |
| `賭博` | `赌博` | 命中(繁简若未折则 **不** 命中:v1 **不做简繁转换**,简繁当两条词) |
| `Casino` | `casino` | 命中 |
| `class` | `ass` | **命中**(v1 无词边界,文档接受) |
| `你好` | `赌博` | 无命中 |
| `赌\u200b博` | `赌博` | 命中 |
| 空串 / 纯标点 | 任意 | 无命中 |
---
# 7. 表
迁移新文件(当前最大约 `0027`,实施时接下一个序号)。`cmd/migrate` 全量重放。列 COMMENT 必填。GORM `TableName()` 返回 `t_*`。
## 7.1 `t_users` 增量
```sql
ALTER TABLE t_users
ADD COLUMN send_muted_until DATETIME(3) NULL
COMMENT '全站禁言截止时间,空或已过期表示可发消息';
```
## 7.2 `t_sensitive_words`
```sql
CREATE TABLE t_sensitive_words (
id BIGINT NOT NULL AUTO_INCREMENT COMMENT '行 id,表内自增',
word_raw VARCHAR(64) NOT NULL COMMENT '运营录入原文,展示用',
word VARCHAR(64) NOT NULL COMMENT 'FoldAndIndex 后的词面,全局唯一,进自动机',
category TINYINT NOT NULL COMMENT '1涉黑 2涉毒 3涉黄 4涉赌 5涉政 6宗教 7辱骂 8广告 9其他',
action TINYINT NOT NULL COMMENT '1log 2replace 3reject 4hide 5mute 6disable',
mute_seconds INT NOT NULL DEFAULT 0 COMMENT 'action=mute 时的禁言秒数',
enabled TINYINT NOT NULL DEFAULT 1 COMMENT '1启用 0下线(不进自动机)',
created_at DATETIME(3) NOT NULL COMMENT '创建时间',
updated_at DATETIME(3) NOT NULL COMMENT '更新时间',
PRIMARY KEY (id),
UNIQUE KEY uk_word (word)
) COMMENT='敏感词;发送热路径使用进程内自动机,本表为真源';
```
`word` 最长 64;更长的短语运营拆词。
## 7.3 `t_sensitive_hits`
```sql
CREATE TABLE t_sensitive_hits (
id BIGINT NOT NULL COMMENT 'Snowflake',
user_id BIGINT NOT NULL COMMENT '发送者 uid',
conversation_id BIGINT NOT NULL DEFAULT 0 COMMENT '会话 id,拒绝时已 prepareSend 则有值',
word_id BIGINT NOT NULL COMMENT 't_sensitive_words.id',
category TINYINT NOT NULL COMMENT '命中时的类别快照',
action TINYINT NOT NULL COMMENT '本行对应词的 action;消息级最严 action 另见同批 rows',
excerpt VARCHAR(128) NOT NULL DEFAULT '' COMMENT '命中片段,截断,禁止整篇正文',
real_ip VARCHAR(64) NOT NULL DEFAULT '' COMMENT 'PROXY RealIP',
created_at DATETIME(3) NOT NULL COMMENT '命中时间',
PRIMARY KEY (id),
KEY idx_user_created (user_id, created_at),
KEY idx_created (created_at),
KEY idx_word (word_id)
) COMMENT='敏感词命中审计,供 Admin 查询';
```
hits 的 `id` 用 Snowflake(与 MsgId 同类,时间有序)。`word_id` 允许词已删除后仍留快照(不建 FK,避免运营删词失败)。
---
# 8. 错误码与 i18n
| code | name | 何时 |
| --- | --- | --- |
| `40301` | forbidden | 停用账号、好友/群权限(已有) |
| `40302` | content_blocked | reject / mute / disable(内容或因此触发的拒绝发送) |
| `40303` | send_muted | 仅「禁言未到期、尚未扫词」 |
`40302` 的 `msg`:`i18n` key `error.40302` / 业务 `moderation.blocked`:「消息包含违规内容」。
`data`:v1 空对象。不返回 word、不返回 category。
`GET /v1/meta/enums` 的 `envelope_code` 增加 40302、40303。
踢线:`notify.kick.banned`。
---
# 9. Admin HTTP
挂在现有 `/api/admin/spark` 与 `/api/admin/pop`(与 music 相同双前缀),JWT AdminAuth。Handler 只 bind + 调 `moderation/logic`。
| 方法 | 路径 | 作用 |
| --- | --- | --- |
| GET | `/sensitive/words` | 分页;query:`q`、`category`、`enabled` |
| POST | `/sensitive/words` | 创建;body:word、category、action、mute_seconds、enabled |
| PATCH | `/sensitive/words/:id` | 改分类/处罚/启用 |
| DELETE | `/sensitive/words/:id` | 物理删或 enabled=0(推荐软下线) |
| GET | `/sensitive/hits` | 分页;query:uid、category、since、until |
| POST | `/sensitive/reload` | 强制 Rebuild 自动机(写入接口已自动 Reload 时可省略) |
校验:`oneof` 与 enum 数字一致;`label`/`errmsg` i18n。
OpenAPI 写在 `openapi.yaml` 的 `admin-music` tag;路径前缀 `/api/admin/spark/sensitive*`(`pop` 别名)。
词库变更后必须 Reload;失败则接口返回 500,避免 DB 与自动机分叉。
---
# 10. 客户端(IM)
- 发送失败认 `40302` / `40303`,展示服务器 `msg`。
- `replace` / `log` / `hide`:发送接口仍成功 ACK(hide 也 ACK,但自己同步看不到——产品需知)。若 hide 要让发送方感知失败,应改用 reject;**v1 hide 仅用于「落库留证、对端不可见」**,与 ACK 成功并存,文档需让客户端不要用 ACK 当「对端已收到」。
**更干净的 v1:砍掉 hide,运营留证用 hits + reject。** 推荐 **v1 不实现 hide**,避免 ACK 语义撕裂。需要留证时 reject 已有 excerpt。`StatusHidden` 留给以后人工审核。
修订:**v1 Action 不含 hide**(表 enum 仍占位 4,词条禁止配置或配置后当 reject)。二期人工审核再启用 hide。
---
# 11. 测试
`make test`:logic ≥65%。
| 包 | 用例 |
| --- | --- |
| `pkg/sensitive` | 规范化、多词、最严 action、enabled=0 不匹配、空串 |
| `moderation/logic` | 内存词表 Scan;Apply mute/disable 调 fake user/session |
| `message/logic` | stub Filter:命中 reject 无 persist、无 queue;replace 后 content 含 `***`;AccountGate 失败不 persist |
| HTTP | 可选:发文本 40302 |
禁止 moderation logic 测试连真实 MySQL。
---
# 12. 实施顺序
1. 迁移:`send_muted_until`、`t_sensitive_words`、`t_sensitive_hits`;跑 `cmd/migrate`
2. `pkg/sensitive` + entity enum + meta/i18n/OpenAPI
3. `moderation` model 内存自动机 + logic Scan/Reload
4. `AccountGate` 接入 `SendMessage`;Admin 封禁补踢线
5. 文本 Filter:log / replace / reject
6. mute / disable
7. Admin 词库与 hits
8. (二期)moment/square `Publish` 调同一 Scan;广场可打 `AuditPending`
9. (二期)Action=hide + 人工审核队列
---
# 13. 默认拍板(实施按此,除非产品改口)
1. 涉政 / 涉黑 / 涉毒默认 **disable**(踢线)。
2. 客户端 **不** 返回 category。
3. **v1 不做 hide**(enum 4 保留给二期)。
4. 禁言列名 `send_muted_until`。
5. 词面 UNIQUE 用规范化串。
---
# 14. 参考代码锚点
- 发送:`internal/module/message/logic/send.go`、`send_perm.go`、`send_persist.go`
- 注入:`internal/bootstrap/app.go` `msglogic.NewService`
- 停用登录:`user/logic/login.go` `Status != StatusActive`
- 踢线:`user/logic/delete.go` `revokeAccess`
- Admin 封禁:`admin/logic/users.go` `BanByEmail`
- Hidden 过滤:`message/model/query.go`
- 群发言:`group/logic/members.go` `CanSend`
- 表名:`t_users`、`t_private_message`、`t_group_message`

View File

@ -41,6 +41,8 @@ import (
inboxmodel "spark-server/internal/module/inbox/model"
msglogic "spark-server/internal/module/message/logic"
msgmodel "spark-server/internal/module/message/model"
modlogic "spark-server/internal/module/moderation/logic"
modmodel "spark-server/internal/module/moderation/model"
momentlogic "spark-server/internal/module/moment/logic"
momentmodel "spark-server/internal/module/moment/model"
musiclogic "spark-server/internal/module/music/logic"
@ -205,7 +207,8 @@ func finishApp(cfg *Config, log *zap.Logger, logClose func() error, idgen *id.Ge
chat := wireChat(idgen, db)
friends := wireFriends(idgen, db, users, deliverQueue, chat)
groups := wireGroups(idgen, db, users, friends, deliverQueue, objects, chat)
messages := wireMessages(idgen, friends, groups, deliverQueue, chat)
moderation := wireModeration(db, users, idgen)
messages := wireMessages(idgen, friends, groups, deliverQueue, chat, moderation, users)
files := wireFiles(cfg, log, objects, groups)
favorites := wireFavorites(idgen, db, objects, chat)
moments := wireMoments(idgen, db, objects, friends, users)
@ -214,7 +217,7 @@ func finishApp(cfg *Config, log *zap.Logger, logClose func() error, idgen *id.Ge
app.squareBot = wireSquareBot(cfg, log, rdb, objects, square, users)
music := wireMusic(cfg, db, objects)
users.BindAccountCleanup(jwtSvc, sessions, accountCleaners(friends, moments, square, groups, favorites, inbox, mailbox, chat)...)
adminSvc, adminTokens := wireAdmin(cfg, db, groups, idgen)
adminSvc, adminTokens := wireAdmin(cfg, db, groups, idgen, users)
app.tcp = tcp.NewServer(log, pool, sessions, &tcp.Options{
Addr: cfg.Server.TCP.Addr,
Multicore: cfg.Server.TCP.Multicore,
@ -235,7 +238,7 @@ func finishApp(cfg *Config, log *zap.Logger, logClose func() error, idgen *id.Ge
Sync: syncSvc,
})
app.http = httpx.New(log, cfg.Server.HTTP.Addr, httpEngineDeps(
jwtSvc, login, smsSvc, users, friends, groups, files, favorites, moments, square, music, client, messages, syncSvc, adminSvc, adminTokens, probe, authLimit,
jwtSvc, login, smsSvc, users, friends, groups, files, favorites, moments, square, music, client, messages, syncSvc, adminSvc, adminTokens, moderation, probe, authLimit,
))
return app, nil
}
@ -257,24 +260,43 @@ func httpEngineDeps(
syncSvc *synclogic.Service,
adminSvc *adminlogic.Service,
adminTokens *adminlogic.TokenService,
moderation *modlogic.Service,
probe *Probe,
authLimit *ratelimit.Limiter,
) *httpx.EngineDeps {
return &httpx.EngineDeps{
Auth: jwtSvc, Login: login, SMS: sms, Users: users, Friends: friends, Groups: groups,
Files: files, Favorites: favorites, Moments: moments, Square: square, Music: music, Client: client,
Messages: messages, Sync: syncSvc, Admin: adminSvc, AdminTokens: adminTokens,
Messages: messages, Sync: syncSvc, Admin: adminSvc, AdminTokens: adminTokens, Moderation: moderation,
Health: probe, AuthLimiter: authLimit,
}
}
func wireAdmin(cfg *Config, db *mysql.DB, groups *grouplogic.Service, idgen *id.Generator) (*adminlogic.Service, *adminlogic.TokenService) {
func wireAdmin(cfg *Config, db *mysql.DB, groups *grouplogic.Service, idgen *id.Generator, users *userlogic.Service) (*adminlogic.Service, *adminlogic.TokenService) {
tokens := adminlogic.NewTokenService(cfg.Admin.JWTSecret)
var repo *adminmodel.Repo
if db != nil && db.Ready() {
repo = adminmodel.NewRepo(db.Gorm())
}
return adminlogic.NewService(repo, groups, idgen), tokens
svc := adminlogic.NewService(repo, groups, idgen)
if users != nil {
svc.BindUserDisabler(users)
}
return svc, tokens
}
func wireModeration(db *mysql.DB, users *userlogic.Service, idgen *id.Generator) *modlogic.Service {
if db == nil || !db.Ready() || idgen == nil {
return nil
}
repo := modmodel.NewRepo(db.Gorm())
if repo == nil {
return nil
}
bridge := modmodel.NewLogicBridge(repo)
svc := modlogic.NewService(bridge, users, idgen)
_ = svc.Reload(context.Background())
return svc
}
func loadTLS(t TLSConfig) (*tls.Config, error) {
@ -490,11 +512,23 @@ func wireDeliver(cfg *Config, log *zap.Logger, rdb *redisx.Client, sessions sess
}), q
}
func wireMessages(idgen *id.Generator, friends *friendlogic.Service, groups *grouplogic.Service, q msglogic.DeliverQueue, chat chatStores) *msglogic.Service {
func wireMessages(
idgen *id.Generator,
friends *friendlogic.Service,
groups *grouplogic.Service,
q msglogic.DeliverQueue,
chat chatStores,
moderation *modlogic.Service,
users *userlogic.Service,
) *msglogic.Service {
if idgen == nil {
return nil
}
return msglogic.NewService(chat.msgs, chat.chats, chat.p2p, friends, groups, idgen, q)
svc := msglogic.NewService(chat.msgs, chat.chats, chat.p2p, friends, groups, idgen, q)
if moderation != nil && users != nil {
svc.BindModeration(users, moderation, moderation)
}
return svc
}
func accountCleaners(

View File

@ -15,10 +15,15 @@ import (
var ErrNotReady = errors.New("admin: repository not ready")
// Service orchestrates admin use-cases. SQL lives in model.Repo.
type UserDisabler interface {
DisableAccount(ctx context.Context, userID int64) error
}
type Service struct {
repo Repository
groups *grouplogic.Service
ids IDGen
repo Repository
groups *grouplogic.Service
ids IDGen
disabler UserDisabler
}
type IDGen interface {
@ -54,6 +59,13 @@ func NewService(repo Repository, groups *grouplogic.Service, ids IDGen) *Service
return &Service{repo: repo, groups: groups, ids: ids}
}
func (s *Service) BindUserDisabler(d UserDisabler) {
if s == nil {
return
}
s.disabler = d
}
func (s *Service) ready() bool {
return s != nil && s.repo != nil
}

View File

@ -118,11 +118,25 @@ func (s *Service) BanByEmail(ctx context.Context, email string, banned bool) err
if !s.ready() {
return ErrNotReady
}
status := userentity.StatusActive
if banned {
status = userentity.StatusDisabled
if !banned {
return s.repo.SetUserStatusByEmail(ctx, email, userentity.StatusActive)
}
return s.repo.SetUserStatusByEmail(ctx, email, status)
rows, err := s.repo.UsersByEmail(ctx, email)
if err != nil {
return err
}
if len(rows) == 0 {
return errs.ErrNotFound
}
if s.disabler == nil {
return s.repo.SetUserStatusByEmail(ctx, email, userentity.StatusDisabled)
}
for i := range rows {
if err := s.disabler.DisableAccount(ctx, rows[i].ID); err != nil {
return err
}
}
return nil
}
func (s *Service) ResetNicknameByEmail(ctx context.Context, email, nickname string) error {

View File

@ -0,0 +1,23 @@
package logic
import (
"context"
modentity "spark-server/internal/module/moderation/entity"
)
// AccountGate checks account status before send.
type AccountGate interface {
AssertCanSend(ctx context.Context, userID int64) error
}
// TextFilter scans outbound text. No hit returns (nil, nil).
type TextFilter interface {
Scan(ctx context.Context, text string) (*modentity.ScanResult, error)
}
// Penalty records hits and enforces mute/disable/reject.
type Penalty interface {
Apply(ctx context.Context, userID, conversationID int64, in *modentity.ApplyInput) error
Enforce(ctx context.Context, userID int64, res *modentity.ScanResult) error
}

View File

@ -43,6 +43,9 @@ func (s *Service) SendMessage(ctx context.Context, userID int64, deviceID uint64
if err != nil {
return nil, errs.ErrInvalidArgument
}
if err := s.gateAccount(ctx, userID); err != nil {
return nil, err
}
conv, memberIDs, err := s.prepareSend(ctx, userID, in)
if err != nil {
return nil, err
@ -53,6 +56,9 @@ func (s *Service) SendMessage(ctx context.Context, userID int64, deviceID uint64
if err := s.checkMentions(ctx, userID, conv, content); err != nil {
return nil, err
}
if err := s.moderateText(ctx, userID, conv.ID, content); err != nil {
return nil, err
}
raw, err := content.Marshal()
if err != nil {
return nil, errs.ErrInvalidArgument

View File

@ -0,0 +1,41 @@
package logic
import (
"context"
"spark-server/internal/module/message/entity"
modentity "spark-server/internal/module/moderation/entity"
)
func (s *Service) gateAccount(ctx context.Context, userID int64) error {
if s.account == nil {
return nil
}
return s.account.AssertCanSend(ctx, userID)
}
func (s *Service) moderateText(ctx context.Context, userID, conversationID int64, content *entity.Content) error {
if content == nil || content.Type != entity.TypeText || content.Text == nil || s.filter == nil {
return nil
}
body := content.Text.Body
res, err := s.filter.Scan(ctx, body)
if err != nil {
return err
}
if res == nil || len(res.Hits) == 0 {
return nil
}
if s.penalty != nil {
if err := s.penalty.Apply(ctx, userID, conversationID, &modentity.ApplyInput{Hits: res.Hits}); err != nil {
return err
}
if err := s.penalty.Enforce(ctx, userID, res); err != nil {
return err
}
}
if res.Action == modentity.ActionReplace && res.ReplacedBody != "" {
content.Text.Body = res.ReplacedBody
}
return nil
}

View File

@ -0,0 +1,39 @@
package logic
import (
"context"
"encoding/json"
"errors"
"testing"
friendlogic "spark-server/internal/module/friend/logic"
modentity "spark-server/internal/module/moderation/entity"
modlogic "spark-server/internal/module/moderation/logic"
"spark-server/internal/pkg/errs"
"spark-server/internal/pkg/sensitive"
)
func TestSendMessageContentBlocked(t *testing.T) {
svc, users, friends, _ := testMessageSvc(t)
word := sensitive.FoldWord("赌博")
repo := modlogic.NewMemoryRepo(&modentity.Word{
ID: 1, WordRaw: "赌博", Word: word, Category: modentity.CategoryGambling,
Action: modentity.ActionReject, Enabled: true,
})
mod := modlogic.NewService(repo, users, svc.ids)
svc.BindModeration(users, mod, mod)
ctx := context.Background()
a, b := twoUsers(t, users)
applied, err := friends.Apply(ctx, a.ID, &friendlogic.ApplyInput{UID: b.ID, Reason: "hi"})
if err != nil {
t.Fatal(err)
}
if _, err := friends.Accept(ctx, b.ID, friendlogic.AcceptInput{RequestID: applied.RequestID}); err != nil {
t.Fatal(err)
}
text := json.RawMessage(`{"t":1,"text":{"body":"今晚赌博"}}`)
_, err = svc.SendMessage(ctx, a.ID, 1, &SendInput{PeerID: b.ID, ClientMsgID: "bad", Content: text})
if !errors.Is(err, errs.ErrContentBlocked) {
t.Fatalf("expected blocked got %v", err)
}
}

View File

@ -64,12 +64,25 @@ type Service struct {
groups GroupGate
ids IDGen
queue DeliverQueue
account AccountGate
filter TextFilter
penalty Penalty
}
func NewService(repo Repository, chats Conversations, p2p P2POpener, friends FriendGate, groups GroupGate, ids IDGen, queue DeliverQueue) *Service {
return &Service{repo: repo, chats: chats, p2p: p2p, friends: friends, groups: groups, ids: ids, queue: queue}
}
// BindModeration wires account gate and text filter (optional; nil skips).
func (s *Service) BindModeration(account AccountGate, filter TextFilter, penalty Penalty) {
if s == nil {
return
}
s.account = account
s.filter = filter
s.penalty = penalty
}
func (s *Service) ready() bool {
return s != nil && s.repo != nil && s.chats != nil && s.ids != nil
}

View File

@ -43,6 +43,8 @@ func envelopeCode(lang i18n.Lang) []entity.Item {
code(lang, "not_authenticated", errs.ErrNotAuthenticated),
code(lang, "verify_code_invalid", errs.ErrVerifyCodeInvalid),
code(lang, "forbidden", errs.ErrForbidden),
code(lang, "content_blocked", errs.ErrContentBlocked),
code(lang, "send_muted", errs.ErrSendMuted),
code(lang, "not_found", errs.ErrNotFound),
code(lang, "media_expired", errs.ErrMediaExpired),
code(lang, "duplicate", errs.ErrDuplicate),

View File

@ -0,0 +1,39 @@
package entity
type Action int8
const (
ActionLog Action = 1
ActionReplace Action = 2
ActionReject Action = 3
ActionHide Action = 4
ActionMute Action = 5
ActionDisable Action = 6
)
func (a Action) Valid() bool {
return a >= ActionLog && a <= ActionDisable
}
// Effective maps v1 unsupported actions (hide) to reject.
func (a Action) Effective() Action {
if a == ActionHide {
return ActionReject
}
return a
}
func (a Action) Severity() int {
return int(a.Effective())
}
func MaxAction(actions ...Action) Action {
var best Action
for _, a := range actions {
e := a.Effective()
if e.Severity() > best.Severity() {
best = e
}
}
return best
}

View File

@ -0,0 +1,19 @@
package entity
type Category int8
const (
CategoryCrime Category = 1
CategoryDrug Category = 2
CategoryPorn Category = 3
CategoryGambling Category = 4
CategoryPolitics Category = 5
CategoryReligion Category = 6
CategoryAbuse Category = 7
CategoryAd Category = 8
CategoryOther Category = 9
)
func (c Category) Valid() bool {
return c >= CategoryCrime && c <= CategoryOther
}

View File

@ -0,0 +1,21 @@
package entity
type HitMatch struct {
WordID int64
Category Category
Action Action
Excerpt string
ByteFrom int
ByteTo int
}
type ScanResult struct {
Action Action
MuteSeconds int
Hits []HitMatch
ReplacedBody string
}
type ApplyInput struct {
Hits []HitMatch
}

View File

@ -0,0 +1,15 @@
package entity
import "time"
type Word struct {
ID int64 `json:"id"`
WordRaw string `json:"word_raw"`
Word string `json:"word"`
Category Category `json:"category"`
Action Action `json:"action"`
MuteSeconds int `json:"mute_seconds"`
Enabled bool `json:"enabled"`
CreatedAt time.Time `json:"created_at"`
UpdatedAt time.Time `json:"updated_at"`
}

View File

@ -0,0 +1,73 @@
package logic
import (
"context"
"time"
modentity "spark-server/internal/module/moderation/entity"
"spark-server/internal/pkg/errs"
"spark-server/internal/pkg/reqctx"
)
// Apply records hits and runs account penalties. Call before rejecting send.
func (s *Service) Apply(ctx context.Context, userID, conversationID int64, in *modentity.ApplyInput) error {
if in == nil || len(in.Hits) == 0 {
return nil
}
if !s.Ready() {
return errs.ErrInner
}
return s.recordHits(ctx, userID, conversationID, in.Hits)
}
// Enforce runs mute/disable after hits are recorded. Returns ErrContentBlocked when send must stop.
func (s *Service) Enforce(ctx context.Context, userID int64, res *modentity.ScanResult) error {
if res == nil {
return nil
}
switch res.Action {
case modentity.ActionReject, modentity.ActionHide:
return errs.ErrContentBlocked
case modentity.ActionMute:
if s.users == nil || res.MuteSeconds <= 0 {
return errs.ErrContentBlocked
}
until := time.Now().Add(time.Duration(res.MuteSeconds) * time.Second)
if err := s.users.ExtendSendMutedUntil(ctx, userID, until); err != nil {
return err
}
return errs.ErrContentBlocked
case modentity.ActionDisable:
if s.users == nil {
return errs.ErrContentBlocked
}
if err := s.users.DisableAccount(ctx, userID); err != nil {
return err
}
return errs.ErrContentBlocked
default:
return nil
}
}
func (s *Service) recordHits(ctx context.Context, userID, conversationID int64, hits []modentity.HitMatch) error {
ip := ""
if rc := reqctx.From(ctx); rc != nil {
ip = rc.RealIP
}
now := time.Now()
rows := make([]HitRecord, 0, len(hits))
for i := range hits {
h := hits[i]
id, err := s.ids.NextID()
if err != nil {
return err
}
rows = append(rows, HitRecord{
ID: id, UserID: userID, ConversationID: conversationID,
WordID: h.WordID, Category: h.Category, Action: h.Action.Effective(),
Excerpt: h.Excerpt, RealIP: ip, CreatedAt: now,
})
}
return s.repo.InsertHits(ctx, rows)
}

View File

@ -0,0 +1,13 @@
package logic
import (
"context"
"time"
)
func (s *Service) ListHits(ctx context.Context, userID int64, category int, since, until *time.Time, page, size int) ([]HitRecord, int64, error) {
if !s.Ready() {
return nil, 0, ErrNotReady
}
return s.repo.ListHits(ctx, userID, category, since, until, page, size)
}

View File

@ -0,0 +1,168 @@
package logic
import (
"context"
"sync"
"time"
modentity "spark-server/internal/module/moderation/entity"
"spark-server/internal/pkg/sensitive"
)
// MemoryRepo is an in-process word list for logic tests (no MySQL).
type MemoryRepo struct {
mu sync.Mutex
words map[int64]*modentity.Word
nextID int64
m *sensitive.Matcher
hits []HitRecord
}
func NewMemoryRepo(words ...*modentity.Word) *MemoryRepo {
r := &MemoryRepo{words: map[int64]*modentity.Word{}, nextID: 1}
for _, w := range words {
if w == nil {
continue
}
if w.ID == 0 {
w.ID = r.nextID
r.nextID++
}
cp := *w
r.words[cp.ID] = &cp
}
r.rebuild()
return r
}
func (r *MemoryRepo) Ready() bool { return r != nil }
func (r *MemoryRepo) ReloadMatcher(ctx context.Context) error {
_ = ctx
r.rebuild()
return nil
}
func (r *MemoryRepo) rebuild() {
entries := make([]sensitive.Entry, 0, len(r.words))
for _, w := range r.words {
if w == nil || !w.Enabled || w.Word == "" {
continue
}
entries = append(entries, sensitive.Entry{ID: w.ID, Word: w.Word})
}
r.m = sensitive.Build(entries)
}
func (r *MemoryRepo) ListWords(ctx context.Context, _ string, _ int, _ *bool, _ int, _ int) ([]modentity.Word, int64, error) {
_ = ctx
r.mu.Lock()
defer r.mu.Unlock()
var all []modentity.Word
for _, w := range r.words {
if w == nil {
continue
}
all = append(all, *w)
}
return all, int64(len(all)), nil
}
func (r *MemoryRepo) CreateWord(ctx context.Context, w *modentity.Word) error {
_ = ctx
r.mu.Lock()
defer r.mu.Unlock()
if w.ID == 0 {
w.ID = r.nextID
r.nextID++
}
cp := *w
r.words[cp.ID] = &cp
r.rebuild()
return nil
}
func (r *MemoryRepo) UpdateWord(ctx context.Context, id int64, patch WordPatch) error {
_ = ctx
r.mu.Lock()
defer r.mu.Unlock()
w := r.words[id]
if w == nil {
return ErrNotReady
}
cp := *w
if patch != nil {
if err := patch(&cp); err != nil {
return err
}
}
r.words[id] = &cp
r.rebuild()
return nil
}
func (r *MemoryRepo) SoftDeleteWord(ctx context.Context, id int64) error {
return r.UpdateWord(ctx, id, func(w *modentity.Word) error {
w.Enabled = false
return nil
})
}
func (r *MemoryRepo) InsertHits(ctx context.Context, hits []HitRecord) error {
_ = ctx
r.mu.Lock()
defer r.mu.Unlock()
r.hits = append(r.hits, hits...)
return nil
}
func (r *MemoryRepo) ListHits(ctx context.Context, _ int64, _ int, _, _ *time.Time, _ int, _ int) ([]HitRecord, int64, error) {
_ = ctx
r.mu.Lock()
defer r.mu.Unlock()
return r.hits, int64(len(r.hits)), nil
}
func (r *MemoryRepo) LookupWord(id int64) *modentity.Word {
r.mu.Lock()
defer r.mu.Unlock()
w := r.words[id]
if w == nil {
return nil
}
cp := *w
return &cp
}
func (r *MemoryRepo) MatcherFind(text string) ([]MatchDetail, string, bool) {
idx := sensitive.FoldAndIndex(text)
if r.m == nil || len(idx.Norm) == 0 {
return nil, text, true
}
raw := r.m.FindAll(idx.Norm)
if len(raw) == 0 {
return nil, text, true
}
intervals, ok := sensitive.IntervalsFromMatches(&idx, raw)
masked := text
if ok {
masked = sensitive.Mask(idx.OrigBytes, intervals, "***")
}
details := make([]MatchDetail, 0, len(raw))
for _, rm := range raw {
w := r.LookupWord(rm.EntryID)
if w == nil {
continue
}
from, to := 0, 0
if ok && rm.Start >= 0 && rm.End <= len(idx.OrigFrom) {
from = idx.OrigFrom[rm.Start]
to = idx.OrigTo[rm.End-1]
}
details = append(details, MatchDetail{
WordID: w.ID, Category: w.Category, Action: w.Action,
Excerpt: sensitive.Excerpt(idx.OrigBytes, from, to, 128), From: from, To: to,
})
}
return details, masked, ok
}

View File

@ -0,0 +1,120 @@
package logic
import (
"context"
"errors"
"testing"
"time"
modentity "spark-server/internal/module/moderation/entity"
"spark-server/internal/pkg/errs"
"spark-server/internal/pkg/sensitive"
)
type fakeUsers struct {
mutedUntil time.Time
disabled bool
}
func (f *fakeUsers) AssertCanSend(_ context.Context, _ int64) error { return nil }
func (f *fakeUsers) DisableAccount(_ context.Context, _ int64) error {
f.disabled = true
return nil
}
func (f *fakeUsers) ExtendSendMutedUntil(_ context.Context, _ int64, until time.Time) error {
if until.After(f.mutedUntil) {
f.mutedUntil = until
}
return nil
}
type fakeIDs struct{ n int64 }
func (f *fakeIDs) NextID() (int64, error) {
f.n++
return f.n, nil
}
func TestScanReplace(t *testing.T) {
word := sensitive.FoldWord("赌博")
repo := NewMemoryRepo(&modentity.Word{
ID: 1, WordRaw: "赌博", Word: word, Category: modentity.CategoryGambling,
Action: modentity.ActionReplace, Enabled: true,
})
svc := NewService(repo, nil, &fakeIDs{})
res, err := svc.Scan(context.Background(), "今晚赌博")
if err != nil || res == nil || res.Action != modentity.ActionReplace || res.ReplacedBody != "今晚***" {
t.Fatalf("scan %+v %v", res, err)
}
}
func TestCreateWordAndReload(t *testing.T) {
repo := NewMemoryRepo()
svc := NewService(repo, nil, &fakeIDs{})
w, err := svc.CreateWord(context.Background(), &CreateWordInput{
WordRaw: "广告词", Category: int8(modentity.CategoryAd), Action: int8(modentity.ActionLog),
})
if err != nil || w == nil || w.Word == "" {
t.Fatalf("create %+v %v", w, err)
}
list, total, err := svc.ListWords(context.Background(), "", 0, nil, 0, 10)
if err != nil || total != 1 || len(list) != 1 {
t.Fatalf("list %d %+v %v", total, list, err)
}
}
func TestEnforceReject(t *testing.T) {
repo := NewMemoryRepo(&modentity.Word{
ID: 2, Word: sensitive.FoldWord("毒品"), WordRaw: "毒品",
Category: modentity.CategoryDrug, Action: modentity.ActionReject, Enabled: true,
})
ids := &fakeIDs{}
svc := NewService(repo, &fakeUsers{}, ids)
res, _ := svc.Scan(context.Background(), "涉毒品")
if err := svc.Apply(context.Background(), 9, 1, &modentity.ApplyInput{Hits: res.Hits}); err != nil {
t.Fatal(err)
}
if err := svc.Enforce(context.Background(), 9, res); !errors.Is(err, errs.ErrContentBlocked) {
t.Fatalf("enforce %v", err)
}
if len(repo.hits) != 1 {
t.Fatalf("hits %d", len(repo.hits))
}
}
func TestEnforceMute(t *testing.T) {
repo := NewMemoryRepo(&modentity.Word{
ID: 3, Word: sensitive.FoldWord("骂人"), WordRaw: "骂人",
Category: modentity.CategoryAbuse, Action: modentity.ActionMute, MuteSeconds: 60, Enabled: true,
})
fu := &fakeUsers{}
svc := NewService(repo, fu, &fakeIDs{})
res, _ := svc.Scan(context.Background(), "骂人啊")
if err := svc.Apply(context.Background(), 1, 2, &modentity.ApplyInput{Hits: res.Hits}); err != nil {
t.Fatal(err)
}
if err := svc.Enforce(context.Background(), 1, res); !errors.Is(err, errs.ErrContentBlocked) {
t.Fatalf("%v", err)
}
if fu.mutedUntil.IsZero() {
t.Fatal("expected mute")
}
}
func TestPatchAndDeleteWord(t *testing.T) {
repo := NewMemoryRepo()
svc := NewService(repo, nil, &fakeIDs{})
w, err := svc.CreateWord(context.Background(), &CreateWordInput{
WordRaw: "临时", Category: int8(modentity.CategoryOther), Action: int8(modentity.ActionReject),
})
if err != nil {
t.Fatal(err)
}
act := int8(modentity.ActionLog)
if err := svc.PatchWord(context.Background(), w.ID, &PatchWordInput{Action: &act}); err != nil {
t.Fatal(err)
}
if err := svc.DeleteWord(context.Background(), w.ID); err != nil {
t.Fatal(err)
}
}

View File

@ -0,0 +1,57 @@
package logic
import (
"context"
modentity "spark-server/internal/module/moderation/entity"
)
// Scan implements message TextFilter. No hit returns (nil, nil).
func (s *Service) Scan(ctx context.Context, text string) (*modentity.ScanResult, error) {
if !s.Ready() {
return nil, nil
}
_ = ctx
details, masked, ok := s.repo.MatcherFind(text)
if len(details) == 0 {
return nil, nil
}
if !ok {
return &modentity.ScanResult{Action: modentity.ActionReject, Hits: toHits(details)}, nil
}
var actions []modentity.Action
var muteSec int
hits := make([]modentity.HitMatch, 0, len(details))
for i := range details {
d := details[i]
act := d.Action.Effective()
actions = append(actions, act)
if act == modentity.ActionMute {
if w := s.repo.LookupWord(d.WordID); w != nil && w.MuteSeconds > muteSec {
muteSec = w.MuteSeconds
}
}
hits = append(hits, modentity.HitMatch{
WordID: d.WordID, Category: d.Category, Action: d.Action,
Excerpt: d.Excerpt, ByteFrom: d.From, ByteTo: d.To,
})
}
best := modentity.MaxAction(actions...)
res := &modentity.ScanResult{Action: best, Hits: hits, MuteSeconds: muteSec}
if best == modentity.ActionReplace {
res.ReplacedBody = masked
}
return res, nil
}
func toHits(details []MatchDetail) []modentity.HitMatch {
out := make([]modentity.HitMatch, 0, len(details))
for i := range details {
d := details[i]
out = append(out, modentity.HitMatch{
WordID: d.WordID, Category: d.Category, Action: d.Action,
Excerpt: d.Excerpt, ByteFrom: d.From, ByteTo: d.To,
})
}
return out
}

View File

@ -0,0 +1,79 @@
// Package logic is sensitive-word scan, penalties, and admin word CRUD.
package logic
import (
"context"
"errors"
"time"
modentity "spark-server/internal/module/moderation/entity"
)
var ErrNotReady = errors.New("moderation: not ready")
type Repository interface {
Ready() bool
ReloadMatcher(ctx context.Context) error
ListWords(ctx context.Context, q string, category int, enabled *bool, page, size int) ([]modentity.Word, int64, error)
CreateWord(ctx context.Context, w *modentity.Word) error
UpdateWord(ctx context.Context, id int64, patch WordPatch) error
SoftDeleteWord(ctx context.Context, id int64) error
InsertHits(ctx context.Context, hits []HitRecord) error
ListHits(ctx context.Context, userID int64, category int, since, until *time.Time, page, size int) ([]HitRecord, int64, error)
LookupWord(id int64) *modentity.Word
MatcherFind(text string) ([]MatchDetail, string, bool)
}
type WordPatch func(w *modentity.Word) error
type HitRecord struct {
ID int64 `json:"id"`
UserID int64 `json:"user_id"`
ConversationID int64 `json:"conversation_id"`
WordID int64 `json:"word_id"`
Category modentity.Category `json:"category"`
Action modentity.Action `json:"action"`
Excerpt string `json:"excerpt"`
RealIP string `json:"real_ip"`
CreatedAt time.Time `json:"created_at"`
}
type MatchDetail struct {
WordID int64
Category modentity.Category
Action modentity.Action
Excerpt string
From int
To int
}
type UserModeration interface {
AssertCanSend(ctx context.Context, userID int64) error
DisableAccount(ctx context.Context, userID int64) error
ExtendSendMutedUntil(ctx context.Context, userID int64, until time.Time) error
}
type IDGen interface {
NextID() (int64, error)
}
type Service struct {
repo Repository
users UserModeration
ids IDGen
}
func NewService(repo Repository, users UserModeration, ids IDGen) *Service {
return &Service{repo: repo, users: users, ids: ids}
}
func (s *Service) Ready() bool {
return s != nil && s.repo != nil && s.repo.Ready()
}
func (s *Service) Reload(ctx context.Context) error {
if !s.Ready() {
return ErrNotReady
}
return s.repo.ReloadMatcher(ctx)
}

View File

@ -0,0 +1,102 @@
package logic
import (
"context"
modentity "spark-server/internal/module/moderation/entity"
"spark-server/internal/pkg/errs"
"spark-server/internal/pkg/sensitive"
)
type CreateWordInput struct {
WordRaw string `json:"word" validate:"required,max=64" label:"field.sensitive_word" errmsg:"required:validate.moderation.word_required"`
Category int8 `json:"category" validate:"required,min=1,max=9" label:"field.sensitive_category"`
Action int8 `json:"action" validate:"required,min=1,max=6" label:"field.sensitive_action"`
MuteSeconds int `json:"mute_seconds" validate:"min=0" label:"field.mute_seconds"`
Enabled *bool `json:"enabled"`
}
type PatchWordInput struct {
Category *int8 `json:"category" validate:"omitempty,min=1,max=9"`
Action *int8 `json:"action" validate:"omitempty,min=1,max=6"`
MuteSeconds *int `json:"mute_seconds" validate:"omitempty,min=0"`
Enabled *bool `json:"enabled"`
WordRaw *string `json:"word" validate:"omitempty,max=64"`
}
func (s *Service) ListWords(ctx context.Context, q string, category int, enabled *bool, page, size int) ([]modentity.Word, int64, error) {
if !s.Ready() {
return nil, 0, ErrNotReady
}
return s.repo.ListWords(ctx, q, category, enabled, page, size)
}
func (s *Service) CreateWord(ctx context.Context, in *CreateWordInput) (*modentity.Word, error) {
if !s.Ready() || in == nil {
return nil, ErrNotReady
}
cat := modentity.Category(in.Category)
act := modentity.Action(in.Action).Effective()
if !cat.Valid() || !act.Valid() {
return nil, errs.ErrInvalidArgument
}
norm := sensitive.FoldWord(in.WordRaw)
if norm == "" {
return nil, errs.ErrInvalidArgument
}
enabled := true
if in.Enabled != nil {
enabled = *in.Enabled
}
w := &modentity.Word{
WordRaw: in.WordRaw, Word: norm, Category: cat, Action: act,
MuteSeconds: in.MuteSeconds, Enabled: enabled,
}
if err := s.repo.CreateWord(ctx, w); err != nil {
return nil, err
}
return w, nil
}
func (s *Service) PatchWord(ctx context.Context, id int64, in *PatchWordInput) error {
if !s.Ready() || id <= 0 || in == nil {
return errs.ErrInvalidArgument
}
return s.repo.UpdateWord(ctx, id, func(w *modentity.Word) error {
if in.WordRaw != nil {
w.WordRaw = *in.WordRaw
w.Word = sensitive.FoldWord(w.WordRaw)
if w.Word == "" {
return errs.ErrInvalidArgument
}
}
if in.Category != nil {
c := modentity.Category(*in.Category)
if !c.Valid() {
return errs.ErrInvalidArgument
}
w.Category = c
}
if in.Action != nil {
a := modentity.Action(*in.Action).Effective()
if !a.Valid() {
return errs.ErrInvalidArgument
}
w.Action = a
}
if in.MuteSeconds != nil {
w.MuteSeconds = *in.MuteSeconds
}
if in.Enabled != nil {
w.Enabled = *in.Enabled
}
return nil
})
}
func (s *Service) DeleteWord(ctx context.Context, id int64) error {
if !s.Ready() || id <= 0 {
return errs.ErrInvalidArgument
}
return s.repo.SoftDeleteWord(ctx, id)
}

View File

@ -0,0 +1,119 @@
package model
import (
"context"
"time"
modentity "spark-server/internal/module/moderation/entity"
modlogic "spark-server/internal/module/moderation/logic"
"spark-server/internal/pkg/sensitive"
)
// LogicBridge adapts Repo to moderation/logic.Repository.
type LogicBridge struct {
*Repo
}
func NewLogicBridge(r *Repo) *LogicBridge {
if r == nil {
return nil
}
return &LogicBridge{Repo: r}
}
func (b *LogicBridge) LookupWord(id int64) *modentity.Word {
if b == nil || b.matcher == nil {
return nil
}
row := b.matcher.WordByID(id)
if row == nil {
return nil
}
return row.entity()
}
func (b *LogicBridge) MatcherFind(text string) ([]modlogic.MatchDetail, string, bool) {
m := b.matcher.Get()
idx := sensitive.FoldAndIndex(text)
if m == nil || len(idx.Norm) == 0 {
return nil, text, true
}
rawMatches := m.FindAll(idx.Norm)
if len(rawMatches) == 0 {
return nil, text, true
}
intervals, ok := sensitive.IntervalsFromMatches(&idx, rawMatches)
masked := text
if ok {
masked = sensitive.Mask(idx.OrigBytes, intervals, "***")
}
details := make([]modlogic.MatchDetail, 0, len(rawMatches))
for _, rm := range rawMatches {
row := b.matcher.WordByID(rm.EntryID)
if row == nil {
continue
}
ent := row.entity()
from, to := 0, 0
if ok && rm.Start >= 0 && rm.End <= len(idx.OrigFrom) {
from = idx.OrigFrom[rm.Start]
to = idx.OrigTo[rm.End-1]
}
ex := sensitive.Excerpt(idx.OrigBytes, from, to, 128)
details = append(details, modlogic.MatchDetail{
WordID: ent.ID, Category: ent.Category, Action: ent.Action,
Excerpt: ex, From: from, To: to,
})
}
if len(details) == 0 {
return nil, text, true
}
return details, masked, ok
}
func (b *LogicBridge) UpdateWord(ctx context.Context, id int64, patch modlogic.WordPatch) error {
return b.Repo.UpdateWord(ctx, id, func(row *WordRow) error {
if patch == nil {
return nil
}
ent := row.entity()
if err := patch(ent); err != nil {
return err
}
row.WordRaw = ent.WordRaw
row.Word = ent.Word
row.Category = int8(ent.Category)
row.Action = int8(ent.Action)
row.MuteSeconds = ent.MuteSeconds
row.Enabled = boolToTiny(ent.Enabled)
return nil
})
}
func (b *LogicBridge) InsertHits(ctx context.Context, hits []modlogic.HitRecord) error {
rows := make([]HitRow, 0, len(hits))
for i := range hits {
h := hits[i]
rows = append(rows, hitFromEntity(h.ID, h.UserID, h.ConversationID, modentity.HitMatch{
WordID: h.WordID, Category: h.Category, Action: h.Action, Excerpt: h.Excerpt,
}, h.RealIP, h.CreatedAt))
}
return b.Repo.InsertHits(ctx, rows)
}
func (b *LogicBridge) ListHits(ctx context.Context, userID int64, category int, since, until *time.Time, page, size int) ([]modlogic.HitRecord, int64, error) {
rows, total, err := b.Repo.ListHits(ctx, userID, category, since, until, page, size)
if err != nil {
return nil, 0, err
}
out := make([]modlogic.HitRecord, 0, len(rows))
for i := range rows {
r := rows[i]
out = append(out, modlogic.HitRecord{
ID: r.ID, UserID: r.UserID, ConversationID: r.ConversationID,
WordID: r.WordID, Category: modentity.Category(r.Category), Action: modentity.Action(r.Action),
Excerpt: r.Excerpt, RealIP: r.RealIP, CreatedAt: r.CreatedAt,
})
}
return out, total, nil
}

View File

@ -0,0 +1,35 @@
package model
import (
"time"
modentity "spark-server/internal/module/moderation/entity"
)
type HitRow struct {
ID int64 `gorm:"column:id;primaryKey"`
UserID int64 `gorm:"column:user_id"`
ConversationID int64 `gorm:"column:conversation_id"`
WordID int64 `gorm:"column:word_id"`
Category int8 `gorm:"column:category"`
Action int8 `gorm:"column:action"`
Excerpt string `gorm:"column:excerpt"`
RealIP string `gorm:"column:real_ip"`
CreatedAt time.Time `gorm:"column:created_at"`
}
func (HitRow) TableName() string { return "t_sensitive_hits" }
func hitFromEntity(id int64, userID, convID int64, h modentity.HitMatch, realIP string, at time.Time) HitRow {
return HitRow{
ID: id,
UserID: userID,
ConversationID: convID,
WordID: h.WordID,
Category: int8(h.Category),
Action: int8(h.Action),
Excerpt: h.Excerpt,
RealIP: realIP,
CreatedAt: at,
}
}

View File

@ -0,0 +1,50 @@
package model
import (
"sync/atomic"
"spark-server/internal/pkg/sensitive"
)
// MatcherStore holds a hot-swappable sensitive.Matcher.
type MatcherStore struct {
ptr atomic.Pointer[sensitive.Matcher]
byID map[int64]*WordRow
}
func NewMatcherStore() *MatcherStore {
return &MatcherStore{byID: map[int64]*WordRow{}}
}
func (s *MatcherStore) Get() *sensitive.Matcher {
if s == nil {
return nil
}
return s.ptr.Load()
}
func (s *MatcherStore) Replace(rows []WordRow) {
entries := make([]sensitive.Entry, 0, len(rows))
byID := make(map[int64]*WordRow, len(rows))
for i := range rows {
r := rows[i]
if r.Enabled != 1 || r.Word == "" {
continue
}
byID[r.ID] = &rows[i]
entries = append(entries, sensitive.Entry{ID: r.ID, Word: r.Word})
}
m := sensitive.Build(entries)
if s == nil {
return
}
s.byID = byID
s.ptr.Store(m)
}
func (s *MatcherStore) WordByID(id int64) *WordRow {
if s == nil {
return nil
}
return s.byID[id]
}

View File

@ -0,0 +1,210 @@
package model
import (
"context"
"errors"
"time"
mysqldriver "github.com/go-sql-driver/mysql"
"gorm.io/gorm"
modentity "spark-server/internal/module/moderation/entity"
"spark-server/internal/pkg/errs"
)
type Repo struct {
db *gorm.DB
matcher *MatcherStore
}
func NewRepo(db *gorm.DB) *Repo {
if db == nil {
return nil
}
return &Repo{db: db, matcher: NewMatcherStore()}
}
func (r *Repo) Ready() bool {
return r != nil && r.db != nil
}
func (r *Repo) Matcher() *MatcherStore {
if r == nil {
return nil
}
return r.matcher
}
func (r *Repo) ReloadMatcher(ctx context.Context) error {
if !r.Ready() {
return errs.ErrAvailable
}
var rows []WordRow
if err := r.db.WithContext(ctx).Where("enabled = ?", 1).Find(&rows).Error; err != nil {
return err
}
r.matcher.Replace(rows)
return nil
}
func (r *Repo) ListWords(ctx context.Context, q string, category int, enabled *bool, page, size int) ([]modentity.Word, int64, error) {
if !r.Ready() {
return nil, 0, errs.ErrAvailable
}
if size <= 0 || size > 200 {
size = 50
}
if page < 0 {
page = 0
}
db := r.db.WithContext(ctx).Model(&WordRow{})
q = trim(q)
if q != "" {
like := "%" + q + "%"
db = db.Where("word_raw LIKE ? OR word LIKE ?", like, like)
}
if category > 0 {
db = db.Where("category = ?", category)
}
if enabled != nil {
if *enabled {
db = db.Where("enabled = ?", 1)
} else {
db = db.Where("enabled = ?", 0)
}
}
var total int64
if err := db.Count(&total).Error; err != nil {
return nil, 0, err
}
var rows []WordRow
if err := db.Order("id DESC").Offset(page * size).Limit(size).Find(&rows).Error; err != nil {
return nil, 0, err
}
out := make([]modentity.Word, 0, len(rows))
for i := range rows {
out = append(out, *rows[i].entity())
}
return out, total, nil
}
func (r *Repo) CreateWord(ctx context.Context, w *modentity.Word) error {
if !r.Ready() || w == nil || w.Word == "" {
return errs.ErrInvalidArgument
}
now := time.Now()
row := WordRow{
WordRaw: w.WordRaw, Word: w.Word, Category: int8(w.Category), Action: int8(w.Action),
MuteSeconds: w.MuteSeconds, Enabled: boolToTiny(w.Enabled), CreatedAt: now, UpdatedAt: now,
}
if err := r.db.WithContext(ctx).Create(&row).Error; err != nil {
return mapDB(err)
}
w.ID = row.ID
w.CreatedAt = now
w.UpdatedAt = now
return r.ReloadMatcher(ctx)
}
func (r *Repo) UpdateWord(ctx context.Context, id int64, patch func(*WordRow) error) error {
if !r.Ready() || id <= 0 {
return errs.ErrInvalidArgument
}
var row WordRow
if err := r.db.WithContext(ctx).Where("id = ?", id).Take(&row).Error; err != nil {
return mapDB(err)
}
if patch != nil {
if err := patch(&row); err != nil {
return err
}
}
row.UpdatedAt = time.Now()
if err := r.db.WithContext(ctx).Save(&row).Error; err != nil {
return mapDB(err)
}
return r.ReloadMatcher(ctx)
}
func (r *Repo) SoftDeleteWord(ctx context.Context, id int64) error {
return r.UpdateWord(ctx, id, func(row *WordRow) error {
row.Enabled = 0
return nil
})
}
func (r *Repo) InsertHits(ctx context.Context, rows []HitRow) error {
if !r.Ready() || len(rows) == 0 {
return nil
}
return r.db.WithContext(ctx).Create(&rows).Error
}
func (r *Repo) ListHits(ctx context.Context, userID int64, category int, since, until *time.Time, page, size int) ([]HitRow, int64, error) {
if !r.Ready() {
return nil, 0, errs.ErrAvailable
}
if size <= 0 || size > 200 {
size = 50
}
if page < 0 {
page = 0
}
db := r.db.WithContext(ctx).Model(&HitRow{})
if userID > 0 {
db = db.Where("user_id = ?", userID)
}
if category > 0 {
db = db.Where("category = ?", category)
}
if since != nil {
db = db.Where("created_at >= ?", *since)
}
if until != nil {
db = db.Where("created_at <= ?", *until)
}
var total int64
if err := db.Count(&total).Error; err != nil {
return nil, 0, err
}
var rows []HitRow
if err := db.Order("created_at DESC").Offset(page * size).Limit(size).Find(&rows).Error; err != nil {
return nil, 0, err
}
return rows, total, nil
}
func boolToTiny(v bool) int8 {
if v {
return 1
}
return 0
}
func trim(s string) string {
for len(s) > 0 && (s[0] == ' ' || s[0] == '\t') {
s = s[1:]
}
for len(s) > 0 {
last := s[len(s)-1]
if last != ' ' && last != '\t' {
break
}
s = s[:len(s)-1]
}
return s
}
func mapDB(err error) error {
if err == nil {
return nil
}
if errors.Is(err, gorm.ErrRecordNotFound) {
return errs.ErrNotFound
}
var me *mysqldriver.MySQLError
if errors.As(err, &me) && me.Number == 1062 {
return errs.ErrDuplicate
}
return err
}

View File

@ -0,0 +1,38 @@
package model
import (
"time"
modentity "spark-server/internal/module/moderation/entity"
)
type WordRow struct {
ID int64 `gorm:"column:id;primaryKey;autoIncrement"`
WordRaw string `gorm:"column:word_raw"`
Word string `gorm:"column:word"`
Category int8 `gorm:"column:category"`
Action int8 `gorm:"column:action"`
MuteSeconds int `gorm:"column:mute_seconds"`
Enabled int8 `gorm:"column:enabled"`
CreatedAt time.Time `gorm:"column:created_at"`
UpdatedAt time.Time `gorm:"column:updated_at"`
}
func (WordRow) TableName() string { return "t_sensitive_words" }
func (r *WordRow) entity() *modentity.Word {
if r == nil {
return nil
}
return &modentity.Word{
ID: r.ID,
WordRaw: r.WordRaw,
Word: r.Word,
Category: modentity.Category(r.Category),
Action: modentity.Action(r.Action),
MuteSeconds: r.MuteSeconds,
Enabled: r.Enabled == 1,
CreatedAt: r.CreatedAt,
UpdatedAt: r.UpdatedAt,
}
}

View File

@ -32,16 +32,17 @@ func ValidGender(g int) bool {
// User is the account. ID is uid (Snowflake), never AUTO_INCREMENT.
type User struct {
ID int64
SparkID string
Nickname string
Avatar string
QRCode string
Birthday time.Time
Gender Gender
Status Status
LastLoginAt time.Time
LastActiveAt time.Time
CreatedAt time.Time
UpdatedAt time.Time
ID int64
SparkID string
Nickname string
Avatar string
QRCode string
Birthday time.Time
Gender Gender
Status Status
SendMutedUntil time.Time
LastLoginAt time.Time
LastActiveAt time.Time
CreatedAt time.Time
UpdatedAt time.Time
}

View File

@ -0,0 +1,72 @@
package logic
import (
"context"
"time"
"spark-server/internal/module/user/entity"
"spark-server/internal/pkg/errs"
)
// AccountModeration is implemented by user persistence for send gates and bans.
type AccountModeration interface {
SetUserStatus(ctx context.Context, userID int64, status entity.Status) error
ExtendSendMutedUntil(ctx context.Context, userID int64, until time.Time) error
}
type BannedKicker interface {
KickUserBanned(userID int64)
}
// AssertCanSend is the AccountGate for message send (all content types).
func (s *Service) AssertCanSend(ctx context.Context, userID int64) error {
if !s.ready() || userID <= 0 {
return errs.ErrUnauthorized
}
u, err := s.repo.GetByID(ctx, userID)
if err != nil {
return err
}
if u.Status != entity.StatusActive {
return errs.ErrForbidden
}
if !u.SendMutedUntil.IsZero() && u.SendMutedUntil.After(time.Now()) {
return errs.ErrSendMuted
}
return nil
}
// ExtendSendMutedUntil implements moderation.UserModeration.
func (s *Service) ExtendSendMutedUntil(ctx context.Context, userID int64, until time.Time) error {
mod, ok := s.repo.(AccountModeration)
if !ok || mod == nil {
return errs.ErrAvailable
}
return mod.ExtendSendMutedUntil(ctx, userID, until)
}
// DisableAccount disables the user and revokes live sessions (moderation / admin ban).
func (s *Service) DisableAccount(ctx context.Context, userID int64) error {
mod, ok := s.repo.(AccountModeration)
if !ok || mod == nil {
return errs.ErrAvailable
}
if err := mod.SetUserStatus(ctx, userID, entity.StatusDisabled); err != nil {
return err
}
s.revokeAccessBanned(ctx, userID)
return nil
}
func (s *Service) revokeAccessBanned(ctx context.Context, userID int64) {
if s.tokens != nil {
_ = s.tokens.LogoutAll(ctx, userID)
}
if bk, ok := s.sessions.(BannedKicker); ok && bk != nil {
bk.KickUserBanned(userID)
return
}
if s.sessions != nil {
s.sessions.KickUser(userID)
}
}

View File

@ -0,0 +1,55 @@
package logic
import (
"context"
"errors"
"testing"
"time"
"spark-server/internal/infrastructure/id"
"spark-server/internal/module/user/entity"
usermodel "spark-server/internal/module/user/model"
"spark-server/internal/pkg/errs"
)
func TestAssertCanSendMuted(t *testing.T) {
ids, err := id.New(1)
if err != nil {
t.Fatal(err)
}
repo := usermodel.NewMemoryRepo()
users := NewService(repo, nil, ids, nil)
ctx := context.Background()
u, err := users.AuthenticateEmail(ctx, "test11@spark.com", "1", &entity.LoginDevice{DeviceID: 11, DeviceType: entity.DeviceWeb})
if err != nil {
t.Fatal(err)
}
until := time.Now().Add(time.Hour)
if err := repo.ExtendSendMutedUntil(ctx, u.ID, until); err != nil {
t.Fatal(err)
}
if err := users.AssertCanSend(ctx, u.ID); !errors.Is(err, errs.ErrSendMuted) {
t.Fatalf("muted %v", err)
}
}
func TestDisableAccount(t *testing.T) {
ids, err := id.New(2)
if err != nil {
t.Fatal(err)
}
repo := usermodel.NewMemoryRepo()
users := NewService(repo, nil, ids, nil)
ctx := context.Background()
u, err := users.AuthenticateEmail(ctx, "test12@spark.com", "1", &entity.LoginDevice{DeviceID: 12, DeviceType: entity.DeviceWeb})
if err != nil {
t.Fatal(err)
}
if err := users.DisableAccount(ctx, u.ID); err != nil {
t.Fatal(err)
}
got, err := repo.GetByID(ctx, u.ID)
if err != nil || got.Status != entity.StatusDisabled {
t.Fatalf("status %+v %v", got, err)
}
}

View File

@ -0,0 +1,33 @@
package model
import (
"context"
"time"
"spark-server/internal/module/user/entity"
"spark-server/internal/pkg/errs"
)
func (m *MemoryRepo) SetUserStatus(_ context.Context, userID int64, status entity.Status) error {
m.mu.Lock()
defer m.mu.Unlock()
u := m.byID[userID]
if u == nil {
return errs.ErrNotFound
}
u.Status = status
return nil
}
func (m *MemoryRepo) ExtendSendMutedUntil(_ context.Context, userID int64, until time.Time) error {
m.mu.Lock()
defer m.mu.Unlock()
u := m.byID[userID]
if u == nil {
return errs.ErrNotFound
}
if u.SendMutedUntil.IsZero() || until.After(u.SendMutedUntil) {
u.SendMutedUntil = until
}
return nil
}

View File

@ -0,0 +1,44 @@
package model
import (
"context"
"time"
"spark-server/internal/module/user/entity"
"spark-server/internal/pkg/errs"
)
func (m *Repo) SetUserStatus(ctx context.Context, userID int64, status entity.Status) error {
db, err := m.session(ctx)
if err != nil {
return err
}
res := db.Model(&User{}).Where("id = ?", userID).Updates(map[string]any{
"status": int(status),
"updated_at": time.Now(),
})
if res.Error != nil {
return mapDBErr(res.Error)
}
if res.RowsAffected == 0 {
return errs.ErrNotFound
}
return nil
}
func (m *Repo) ExtendSendMutedUntil(ctx context.Context, userID int64, until time.Time) error {
db, err := m.session(ctx)
if err != nil {
return err
}
// GREATEST(IFNULL(send_muted_until, '1970-01-01'), ?)
res := db.Exec(`UPDATE t_users SET send_muted_until = GREATEST(COALESCE(send_muted_until, '1970-01-01 00:00:00.000'), ?), updated_at = ? WHERE id = ?`,
until, time.Now(), userID)
if res.Error != nil {
return mapDBErr(res.Error)
}
if res.RowsAffected == 0 {
return errs.ErrNotFound
}
return nil
}

View File

@ -10,18 +10,19 @@ import (
// User maps table users: account row, Snowflake uid, mutable spark_id.
type User struct {
ID int64 `gorm:"column:id;primaryKey;comment:Snowflake 用户 id(uid),时间有序,非自增"`
SparkID string `gorm:"column:spark_id;comment:对外展示的账号,用户可改,全局唯一"`
Nickname string `gorm:"column:nickname;comment:昵称"`
Avatar string `gorm:"column:avatar;comment:头像对象存储 key 或 URL,不存图片二进制"`
QRCode string `gorm:"column:qrcode;comment:个人二维码扫描票据,随机生成,不含 uid 或 spark_id"`
Birthday *time.Time `gorm:"column:birthday;type:date;comment:生日,未填为空"`
Gender int `gorm:"column:gender;comment:0=未知 1=男 2=女"`
Status int `gorm:"column:status;comment:1=正常 0=禁用"`
LastLoginAt *time.Time `gorm:"column:last_login_at;comment:最近一次登录时间"`
LastActiveAt *time.Time `gorm:"column:last_active_at;comment:最近一次活跃时间(登录、心跳、发消息等)"`
CreatedAt time.Time `gorm:"column:created_at;comment:注册时间(create_time)"`
UpdatedAt time.Time `gorm:"column:updated_at;comment:资料最近更新时间(update_time)"`
ID int64 `gorm:"column:id;primaryKey;comment:Snowflake 用户 id(uid),时间有序,非自增"`
SparkID string `gorm:"column:spark_id;comment:对外展示的账号,用户可改,全局唯一"`
Nickname string `gorm:"column:nickname;comment:昵称"`
Avatar string `gorm:"column:avatar;comment:头像对象存储 key 或 URL,不存图片二进制"`
QRCode string `gorm:"column:qrcode;comment:个人二维码扫描票据,随机生成,不含 uid 或 spark_id"`
Birthday *time.Time `gorm:"column:birthday;type:date;comment:生日,未填为空"`
Gender int `gorm:"column:gender;comment:0=未知 1=男 2=女"`
Status int `gorm:"column:status;comment:1=正常 0=禁用"`
SendMutedUntil *time.Time `gorm:"column:send_muted_until;comment:全站禁言截止时间"`
LastLoginAt *time.Time `gorm:"column:last_login_at;comment:最近一次登录时间"`
LastActiveAt *time.Time `gorm:"column:last_active_at;comment:最近一次活跃时间(登录、心跳、发消息等)"`
CreatedAt time.Time `gorm:"column:created_at;comment:注册时间(create_time)"`
UpdatedAt time.Time `gorm:"column:updated_at;comment:资料最近更新时间(update_time)"`
}
func (User) TableName() string { return "t_users" }
@ -31,18 +32,19 @@ func (r *User) entity() *entity.User {
return nil
}
return &entity.User{
ID: r.ID,
SparkID: r.SparkID,
Nickname: r.Nickname,
Avatar: r.Avatar,
QRCode: r.QRCode,
Birthday: timeOrZero(r.Birthday),
Gender: entity.Gender(r.Gender),
Status: entity.Status(r.Status),
LastLoginAt: timeOrZero(r.LastLoginAt),
LastActiveAt: timeOrZero(r.LastActiveAt),
CreatedAt: r.CreatedAt,
UpdatedAt: r.UpdatedAt,
ID: r.ID,
SparkID: r.SparkID,
Nickname: r.Nickname,
Avatar: r.Avatar,
QRCode: r.QRCode,
Birthday: timeOrZero(r.Birthday),
Gender: entity.Gender(r.Gender),
Status: entity.Status(r.Status),
SendMutedUntil: timeOrZero(r.SendMutedUntil),
LastLoginAt: timeOrZero(r.LastLoginAt),
LastActiveAt: timeOrZero(r.LastActiveAt),
CreatedAt: r.CreatedAt,
UpdatedAt: r.UpdatedAt,
}
}

View File

@ -12,6 +12,8 @@ var (
ErrNotAuthenticated = NewError("40102", "not authenticated", "")
ErrVerifyCodeInvalid = NewError("40103", "verification code invalid", "")
ErrForbidden = NewError("40301", "forbidden", "")
ErrContentBlocked = NewError("40302", "content blocked", "")
ErrSendMuted = NewError("40303", "send muted", "")
ErrNotFound = NewError("40401", "not found", "")
ErrMediaExpired = NewError("40402", "media expired", "")
ErrDuplicate = NewError("40901", "duplicate", "")

View File

@ -32,6 +32,12 @@ func init() {
"err.40301": {
ZhCN: "无权限", En: "forbidden", ZhTW: "無權限",
},
"err.40302": {
ZhCN: "消息包含违规内容", En: "message blocked", ZhTW: "訊息包含違規內容",
},
"err.40303": {
ZhCN: "账号禁言中", En: "send muted", ZhTW: "帳號禁言中",
},
"err.40401": {
ZhCN: "未找到", En: "not found", ZhTW: "未找到",
},

View File

@ -11,6 +11,9 @@ func init() {
NotifyKickDeleted: {
ZhCN: "账号已注销", En: "Account deleted", ZhTW: "帳號已註銷",
},
NotifyKickBanned: {
ZhCN: "账号已被封禁", En: "Account banned", ZhTW: "帳號已被封禁",
},
NotifyFriendRequest + ".title": {
ZhCN: "好友请求", En: "Friend request", ZhTW: "好友請求",
},

View File

@ -187,6 +187,7 @@ const (
NotifyKickReplaced = "notify.kick.replaced"
NotifyKickLogout = "notify.kick.logout"
NotifyKickDeleted = "notify.kick.deleted"
NotifyKickBanned = "notify.kick.banned"
NotifyFriendRequest = "notify.friend.request"
NotifyFriendBecame = "notify.friend.became"
NotifyGroupInvite = "notify.group.invite"

View File

@ -0,0 +1,99 @@
package sensitive
import (
"strings"
"unicode"
"unicode/utf8"
)
const maxScanRunes = 8192
// Index maps normalized rune position to original UTF-8 byte offsets [from, to).
type Index struct {
Norm []rune
OrigFrom []int
OrigTo []int
OrigBytes []byte
}
// FoldWord normalizes a dictionary entry (same rules as FoldAndIndex on short input).
func FoldWord(s string) string {
idx := FoldAndIndex(s)
if len(idx.Norm) == 0 {
return ""
}
return string(idx.Norm)
}
// FoldAndIndex builds the normalized rune stream and byte mapping into the original text.
func FoldAndIndex(text string) Index {
raw := []byte(text)
var norm []rune
var from, to []int
runes := 0
for i := 0; i < len(raw); {
r, w := decodeRune(raw, i)
if w == 0 {
break
}
start := i
end := i + w
i = end
if dropRune(r) {
continue
}
r = foldRune(r)
if r == 0 {
continue
}
runes++
if runes > maxScanRunes {
break
}
norm = append(norm, r)
from = append(from, start)
to = append(to, end)
}
return Index{Norm: norm, OrigFrom: from, OrigTo: to, OrigBytes: raw}
}
func dropRune(r rune) bool {
if r <= unicode.MaxASCII {
if unicode.IsSpace(r) || isASCIIPunct(r) {
return true
}
}
if unicode.In(r, unicode.Cf, unicode.Zs, unicode.Zl, unicode.Zp) {
return true
}
if isCJKPunct(r) {
return true
}
return false
}
func isASCIIPunct(r rune) bool {
return strings.ContainsRune(`!"#$%&'()*+,-./:;<=>?@[\]^_`+"`"+`{|}~`, r)
}
func isCJKPunct(r rune) bool {
return strings.ContainsRune(",。!?、;:…·「」『』【】()", r)
}
func foldRune(r rune) rune {
if r >= 0xFF01 && r <= 0xFF5E {
return r - 0xFEE0
}
return unicode.ToLower(r)
}
func decodeRune(b []byte, i int) (rune, int) {
if i >= len(b) {
return 0, 0
}
r, w := utf8.DecodeRune(b[i:])
if w == 0 {
return utf8.RuneError, 1
}
return r, w
}

View File

@ -0,0 +1,111 @@
package sensitive
// Entry is one enabled dictionary pattern (normalized word).
type Entry struct {
ID int64
Word string
RuneLen int
}
// Match is a hit on the normalized stream [Start, End) rune indices.
type Match struct {
EntryID int64
Start int
End int
}
// Matcher is an Aho-Corasick automaton over normalized patterns.
type Matcher struct {
entries []Entry
root *acNode
}
type acNode struct {
next map[rune]*acNode
fail *acNode
outputs []int // indices into entries
}
// Build constructs a matcher from patterns. Empty patterns are skipped.
func Build(entries []Entry) *Matcher {
m := &Matcher{entries: make([]Entry, 0, len(entries)), root: &acNode{next: map[rune]*acNode{}}}
for i := range entries {
e := entries[i]
w := e.Word
if w == "" {
continue
}
idx := len(m.entries)
rl := len([]rune(w))
e.RuneLen = rl
m.entries = append(m.entries, e)
m.addPattern(w, rl, idx)
}
m.buildFails()
return m
}
func (m *Matcher) addPattern(word string, _ int, outIdx int) {
node := m.root
for _, r := range word {
child := node.next[r]
if child == nil {
child = &acNode{next: map[rune]*acNode{}}
node.next[r] = child
}
node = child
}
node.outputs = append(node.outputs, outIdx)
}
func (m *Matcher) buildFails() {
queue := make([]*acNode, 0, 16)
for _, child := range m.root.next {
child.fail = m.root
queue = append(queue, child)
}
for qi := 0; qi < len(queue); qi++ {
cur := queue[qi]
for r, child := range cur.next {
queue = append(queue, child)
f := cur.fail
for f != nil && f.next[r] == nil {
f = f.fail
}
if f == nil {
child.fail = m.root
} else {
child.fail = f.next[r]
}
child.outputs = append(child.outputs, child.fail.outputs...)
}
}
}
// FindAll returns every match in norm (inclusive start, exclusive end).
func (m *Matcher) FindAll(norm []rune) []Match {
if m == nil || len(norm) == 0 {
return nil
}
var out []Match
node := m.root
for i, r := range norm {
for node != m.root && node.next[r] == nil {
node = node.fail
}
next := node.next[r]
if next == nil {
next = m.root
}
node = next
for _, idx := range node.outputs {
ent := m.entries[idx]
start := i + 1 - ent.RuneLen
if start < 0 {
start = 0
}
out = append(out, Match{EntryID: ent.ID, Start: start, End: i + 1})
}
}
return out
}

View File

@ -0,0 +1,104 @@
package sensitive
import "sort"
// ByteInterval is a span in the original UTF-8 text.
type ByteInterval struct {
From int
To int
}
// IntervalsFromMatches maps normalized matches back to original byte intervals.
func IntervalsFromMatches(idx *Index, matches []Match) ([]ByteInterval, bool) {
if idx == nil || len(matches) == 0 {
return nil, true
}
intervals := make([]ByteInterval, 0, len(matches))
for i := range matches {
m := matches[i]
if m.Start < 0 || m.End <= m.Start || m.End > len(idx.OrigFrom) {
return nil, false
}
last := m.End - 1
intervals = append(intervals, ByteInterval{
From: idx.OrigFrom[m.Start],
To: idx.OrigTo[last],
})
}
return mergeIntervals(intervals), true
}
func mergeIntervals(in []ByteInterval) []ByteInterval {
if len(in) == 0 {
return nil
}
sort.Slice(in, func(i, j int) bool { return in[i].From < in[j].From })
out := []ByteInterval{in[0]}
for i := 1; i < len(in); i++ {
last := &out[len(out)-1]
cur := in[i]
if cur.From <= last.To {
if cur.To > last.To {
last.To = cur.To
}
continue
}
out = append(out, cur)
}
return out
}
// Mask replaces each interval with repl (default "***").
func Mask(orig []byte, intervals []ByteInterval, repl string) string {
if len(intervals) == 0 {
return string(orig)
}
if repl == "" {
repl = "***"
}
var b []byte
pos := 0
for _, iv := range intervals {
if iv.From < pos || iv.To < iv.From || iv.To > len(orig) {
return string(orig)
}
b = append(b, orig[pos:iv.From]...)
b = append(b, repl...)
pos = iv.To
}
b = append(b, orig[pos:]...)
return string(b)
}
// Excerpt returns a short slice of orig around [from,to).
func Excerpt(orig []byte, from, to int, maxLen int) string {
if maxLen <= 0 {
maxLen = 128
}
if from < 0 {
from = 0
}
if to > len(orig) {
to = len(orig)
}
if from >= to {
return ""
}
const pad = 24
start := from - pad
if start < 0 {
start = 0
}
end := to + pad
if end > len(orig) {
end = len(orig)
}
s := string(orig[start:end])
if len(s) > maxLen {
if maxLen <= 3 {
return s[:maxLen]
}
return s[:maxLen-3] + "..."
}
return s
}

View File

@ -0,0 +1,18 @@
package sensitive
// ScanText folds text, runs the matcher, and optionally masks the original body.
func ScanText(m *Matcher, text string) (matches []Match, masked string, ok bool) {
idx := FoldAndIndex(text)
if len(idx.Norm) == 0 || m == nil {
return nil, text, true
}
matches = m.FindAll(idx.Norm)
if len(matches) == 0 {
return nil, text, true
}
intervals, ok := IntervalsFromMatches(&idx, matches)
if !ok {
return matches, text, false
}
return matches, Mask(idx.OrigBytes, intervals, "***"), true
}

View File

@ -0,0 +1,41 @@
package sensitive
import (
"testing"
)
func TestFoldAndMatch(t *testing.T) {
m := Build([]Entry{{ID: 1, Word: FoldWord("赌博")}})
matches, masked, ok := ScanText(m, "今晚赌博")
if !ok || len(matches) != 1 || masked == "今晚赌博" {
t.Fatalf("match %+v masked=%q ok=%v", matches, masked, ok)
}
if masked != "今晚***" {
t.Fatalf("masked=%q", masked)
}
matches, _, ok = ScanText(m, "今晚赌 博啊")
if !ok || len(matches) != 1 {
t.Fatalf("spaced %+v", matches)
}
matches, _, ok = ScanText(m, "赌\u200b博")
if !ok || len(matches) != 1 {
t.Fatalf("zwsp %+v", matches)
}
m2 := Build([]Entry{{ID: 2, Word: FoldWord("casino")}})
matches, _, ok = ScanText(m2, "Casino")
if !ok || len(matches) != 1 {
t.Fatalf("case %+v", matches)
}
matches, _, _ = ScanText(m, "你好")
if len(matches) != 0 {
t.Fatalf("no hit %+v", matches)
}
}
func TestAssInClass(t *testing.T) {
m := Build([]Entry{{ID: 1, Word: FoldWord("ass")}})
matches, _, ok := ScanText(m, "class")
if !ok || len(matches) != 1 {
t.Fatalf("expected substring hit %+v", matches)
}
}

View File

@ -381,6 +381,7 @@ tags:
description: |
popadmin 音乐管理:`/api/admin/spark/music*`(`/api/admin/pop/music*` 别名)。
曲目 CRUD、封面/背景/音频上传、分类 CRUD、曲目分类绑定(多对多)、全局试听秒数。
敏感词:`/api/admin/spark/sensitive*`(`/api/admin/pop/sensitive*` 别名);词库与命中审计、热重载。
Admin JWT;响应 `{success,message,data}`。Host 常用 `https://popadmin.wyrs365.com`。
- name: novels
description: |
@ -2434,6 +2435,157 @@ paths:
schema:
$ref: "#/components/schemas/AdminApiResponse"
/api/admin/spark/sensitive/words:
get:
tags: [admin-music]
summary: 敏感词分页列表
description: |
Query:`q`(匹配 word_raw/word)、`category`(1–9)、`enabled`(`true`/`false`/`1`/`0`)、`page`、`size`。
别名:`GET /api/admin/pop/sensitive/words`。
operationId: adminListSensitiveWords
security:
- bearerAuth: []
parameters:
- name: q
in: query
schema: { type: string }
- name: category
in: query
schema: { $ref: "#/components/schemas/SensitiveCategory" }
- name: enabled
in: query
schema: { type: string }
description: true/false 或 1/0
- name: page
in: query
schema: { type: integer, default: 0 }
- name: size
in: query
schema: { type: integer, default: 50, maximum: 200 }
responses:
"200":
description: "`data.items` 为词条,`data.total` 为总数"
content:
application/json:
schema:
$ref: "#/components/schemas/AdminApiResponse"
post:
tags: [admin-music]
summary: 新建敏感词
description: |
`word` 为运营录入原文;服务端 Fold 后写入 `word` 列(唯一键)。`action=4`(hide)在 v1 会按 reject 处理。
成功后自动 Reload 自动机。
operationId: adminCreateSensitiveWord
security:
- bearerAuth: []
requestBody:
required: true
content:
application/json:
schema:
$ref: "#/components/schemas/SensitiveWordCreateRequest"
responses:
"200":
description: 返回新建词条
content:
application/json:
schema:
$ref: "#/components/schemas/AdminApiResponse"
/api/admin/spark/sensitive/reload:
post:
tags: [admin-music]
summary: 从数据库重建敏感词自动机
operationId: adminReloadSensitiveWords
security:
- bearerAuth: []
responses:
"200":
description: OK
content:
application/json:
schema:
$ref: "#/components/schemas/AdminApiResponse"
/api/admin/spark/sensitive/words/{id}:
patch:
tags: [admin-music]
summary: 更新敏感词
operationId: adminPatchSensitiveWord
security:
- bearerAuth: []
parameters:
- name: id
in: path
required: true
schema: { type: integer, format: int64 }
requestBody:
required: true
content:
application/json:
schema:
$ref: "#/components/schemas/SensitiveWordPatchRequest"
responses:
"200":
description: OK
content:
application/json:
schema:
$ref: "#/components/schemas/AdminApiResponse"
delete:
tags: [admin-music]
summary: 下线敏感词(enabled=0)
operationId: adminDeleteSensitiveWord
security:
- bearerAuth: []
parameters:
- name: id
in: path
required: true
schema: { type: integer, format: int64 }
responses:
"200":
description: OK
content:
application/json:
schema:
$ref: "#/components/schemas/AdminApiResponse"
/api/admin/spark/sensitive/hits:
get:
tags: [admin-music]
summary: 敏感词命中审计分页
description: Query:`uid`、`category`、`since`/`until`(RFC3339)、`page`、`size`。
operationId: adminListSensitiveHits
security:
- bearerAuth: []
parameters:
- name: uid
in: query
schema: { type: integer, format: int64 }
- name: category
in: query
schema: { $ref: "#/components/schemas/SensitiveCategory" }
- name: since
in: query
schema: { type: string, format: date-time }
- name: until
in: query
schema: { type: string, format: date-time }
- name: page
in: query
schema: { type: integer, default: 0 }
- name: size
in: query
schema: { type: integer, default: 50, maximum: 200 }
responses:
"200":
description: "`data.items` 为命中行,`data.total` 为总数"
content:
application/json:
schema:
$ref: "#/components/schemas/AdminApiResponse"
# —— FullStoryNovel 公开小说 API(基址选 admin.fullstorynovel.com 或 fullstorynovel.com)——
/api/home:
get:
@ -3262,7 +3414,7 @@ components:
12 SYNC_MESSAGES 13 SYNC_MESSAGES_FINISH 14 KICK 15 DELETE_MESSAGE
EnvelopeCode:
type: string
enum: ["0", "40001", "40002", "40003", "40004", "40005", "40101", "40102", "40103", "40301", "40401", "40402", "40901", "42901", "42902", "42903", "49901", "50001", "50002", "50003", "50301", "50401"]
enum: ["0", "40001", "40002", "40003", "40004", "40005", "40101", "40102", "40103", "40301", "40302", "40303", "40401", "40402", "40901", "42901", "42902", "42903", "49901", "50001", "50002", "50003", "50301", "50401"]
description: |
envelope.code。成功为 "0"。
`40005` media_oversized:媒体分辨率/体积超限(开启 media.user_reject 时拒收)。
@ -3364,6 +3516,60 @@ components:
$ref: "#/components/schemas/MusicCategory"
description: 启用中的分类树
memberPreviewSeconds: { type: integer }
SensitiveCategory:
type: integer
enum: [1, 2, 3, 4, 5, 6, 7, 8, 9]
description: |
1=涉黑 2=涉毒 3=涉黄 4=涉赌 5=涉政 6=宗教 7=辱骂 8=广告 9=其他
SensitiveAction:
type: integer
enum: [1, 2, 3, 4, 5, 6]
description: |
1=log 2=replace 3=reject 4=hide(v1 按 reject)5=mute 6=disable
SensitiveWord:
type: object
properties:
id: { type: integer, format: int64 }
word_raw: { type: string, description: 运营录入原文 }
word: { type: string, description: 规范化词面,进自动机 }
category: { $ref: "#/components/schemas/SensitiveCategory" }
action: { $ref: "#/components/schemas/SensitiveAction" }
mute_seconds: { type: integer, description: action=mute 时生效 }
enabled: { type: boolean }
created_at: { type: string, format: date-time }
updated_at: { type: string, format: date-time }
SensitiveWordCreateRequest:
type: object
required: [word, category, action]
properties:
word:
type: string
maxLength: 64
description: 词面原文(入库后 Fold 为 word 列)
category: { $ref: "#/components/schemas/SensitiveCategory" }
action: { $ref: "#/components/schemas/SensitiveAction" }
mute_seconds: { type: integer, minimum: 0, default: 0 }
enabled: { type: boolean, default: true }
SensitiveWordPatchRequest:
type: object
properties:
word: { type: string, maxLength: 64 }
category: { $ref: "#/components/schemas/SensitiveCategory" }
action: { $ref: "#/components/schemas/SensitiveAction" }
mute_seconds: { type: integer, minimum: 0 }
enabled: { type: boolean }
SensitiveHit:
type: object
properties:
id: { type: integer, format: int64 }
user_id: { type: integer, format: int64 }
conversation_id: { type: integer, format: int64 }
word_id: { type: integer, format: int64 }
category: { $ref: "#/components/schemas/SensitiveCategory" }
action: { $ref: "#/components/schemas/SensitiveAction" }
excerpt: { type: string, maxLength: 128 }
real_ip: { type: string }
created_at: { type: string, format: date-time }
MusicUpsertRequest:
type: object
description: |

View File

@ -0,0 +1,157 @@
package adminhandler
import (
"errors"
"net/http"
"strconv"
"time"
"github.com/gin-gonic/gin"
modlogic "spark-server/internal/module/moderation/logic"
"spark-server/internal/pkg/errs"
"spark-server/internal/pkg/reqctx"
"spark-server/internal/pkg/validate"
"spark-server/internal/transport/http/respond"
)
type ModerationHandler struct {
svc *modlogic.Service
}
func NewModeration(svc *modlogic.Service) *ModerationHandler {
return &ModerationHandler{svc: svc}
}
func (h *ModerationHandler) ready(c *gin.Context) bool {
if h == nil || h.svc == nil || !h.svc.Ready() {
Fail(c, http.StatusServiceUnavailable, "moderation 服务未就绪")
return false
}
return true
}
func (h *ModerationHandler) ListWords(c *gin.Context) {
if !h.ready(c) {
return
}
page, _ := strconv.Atoi(c.DefaultQuery("page", "0"))
size, _ := strconv.Atoi(c.DefaultQuery("size", "50"))
cat, _ := strconv.Atoi(c.Query("category"))
var enabled *bool
if v := c.Query("enabled"); v == "1" || v == "true" {
enabled = ptrBool(true)
} else if v == "0" || v == "false" {
enabled = ptrBool(false)
}
list, total, err := h.svc.ListWords(c.Request.Context(), c.Query("q"), cat, enabled, page, size)
if err != nil {
Fail(c, http.StatusInternalServerError, err.Error())
return
}
OK(c, gin.H{"items": list, "total": total})
}
func (h *ModerationHandler) CreateWord(c *gin.Context) {
if !h.ready(c) {
return
}
var in modlogic.CreateWordInput
if err := respond.BindJSON(c, &in); err != nil {
return
}
if err := validate.StructLang(reqctx.Lang(c.Request.Context()), &in); err != nil {
respond.JSONError(c, err)
return
}
out, err := h.svc.CreateWord(c.Request.Context(), &in)
if err != nil {
failModeration(c, err)
return
}
OK(c, out)
}
func (h *ModerationHandler) PatchWord(c *gin.Context) {
if !h.ready(c) {
return
}
id, err := strconv.ParseInt(c.Param("id"), 10, 64)
if err != nil || id <= 0 {
Fail(c, http.StatusBadRequest, "invalid id")
return
}
var in modlogic.PatchWordInput
if err := respond.BindJSON(c, &in); err != nil {
return
}
if err := h.svc.PatchWord(c.Request.Context(), id, &in); err != nil {
failModeration(c, err)
return
}
OK(c, gin.H{"ok": true})
}
func (h *ModerationHandler) DeleteWord(c *gin.Context) {
if !h.ready(c) {
return
}
id, err := strconv.ParseInt(c.Param("id"), 10, 64)
if err != nil || id <= 0 {
Fail(c, http.StatusBadRequest, "invalid id")
return
}
if err := h.svc.DeleteWord(c.Request.Context(), id); err != nil {
failModeration(c, err)
return
}
OK(c, gin.H{"ok": true})
}
func (h *ModerationHandler) Reload(c *gin.Context) {
if !h.ready(c) {
return
}
if err := h.svc.Reload(c.Request.Context()); err != nil {
Fail(c, http.StatusInternalServerError, err.Error())
return
}
OK(c, gin.H{"ok": true})
}
func (h *ModerationHandler) ListHits(c *gin.Context) {
if !h.ready(c) {
return
}
page, _ := strconv.Atoi(c.DefaultQuery("page", "0"))
size, _ := strconv.Atoi(c.DefaultQuery("size", "50"))
uid, _ := strconv.ParseInt(c.Query("uid"), 10, 64)
cat, _ := strconv.Atoi(c.Query("category"))
var since, until *time.Time
if s := c.Query("since"); s != "" {
if t, err := time.Parse(time.RFC3339, s); err == nil {
since = &t
}
}
if s := c.Query("until"); s != "" {
if t, err := time.Parse(time.RFC3339, s); err == nil {
until = &t
}
}
list, total, err := h.svc.ListHits(c.Request.Context(), uid, cat, since, until, page, size)
if err != nil {
Fail(c, http.StatusInternalServerError, err.Error())
return
}
OK(c, gin.H{"items": list, "total": total})
}
func failModeration(c *gin.Context, err error) {
if errors.Is(err, errs.ErrInvalidArgument) || errors.Is(err, errs.ErrDuplicate) || errors.Is(err, errs.ErrNotFound) {
respond.JSONError(c, err)
return
}
Fail(c, http.StatusInternalServerError, err.Error())
}
func ptrBool(v bool) *bool { return &v }

View File

@ -17,6 +17,7 @@ func registerAdminRoutes(r *gin.Engine, deps *EngineDeps) {
}
mh := adminhandler.NewMusic(deps.Music)
ch := adminhandler.NewClient(deps.Client)
modh := adminhandler.NewModeration(deps.Moderation)
// Shared user ops (popadmin Host 可将 /api/admin/users* 切到本服务)
if h != nil {
@ -49,9 +50,23 @@ func registerAdminRoutes(r *gin.Engine, deps *EngineDeps) {
}
registerMusicAdmin(g, mh)
registerClientAdmin(g, ch)
registerModerationAdmin(g, modh)
}
}
func registerModerationAdmin(g *gin.RouterGroup, h *adminhandler.ModerationHandler) {
if h == nil {
return
}
sg := g.Group("/sensitive")
sg.GET("/words", h.ListWords)
sg.POST("/words", h.CreateWord)
sg.POST("/reload", h.Reload)
sg.PATCH("/words/:id", h.PatchWord)
sg.DELETE("/words/:id", h.DeleteWord)
sg.GET("/hits", h.ListHits)
}
func registerSparkAdmin(g *gin.RouterGroup, h *adminhandler.Handler) {
g.GET("/api-doc", h.APIDoc)
g.GET("/ops-catalog", h.OpsCatalog)

View File

@ -19,6 +19,7 @@ import (
friendlogic "spark-server/internal/module/friend/logic"
grouplogic "spark-server/internal/module/group/logic"
msglogic "spark-server/internal/module/message/logic"
modlogic "spark-server/internal/module/moderation/logic"
momentlogic "spark-server/internal/module/moment/logic"
musiclogic "spark-server/internal/module/music/logic"
smslogic "spark-server/internal/module/sms/logic"
@ -60,6 +61,7 @@ type EngineDeps struct {
AdminTokens *adminlogic.TokenService
Music *musiclogic.Service
Client *clientlogic.Service
Moderation *modlogic.Service
Health Health
AuthLimiter *ratelimit.Limiter
}

View File

@ -54,7 +54,15 @@ func (m *memManager) KickDevice(userID int64, deviceID uint64) {
}
}
func (m *memManager) KickUserBanned(userID int64) {
m.kickUserKey(userID, i18n.NotifyKickBanned)
}
func (m *memManager) KickUser(userID int64) {
m.kickUserKey(userID, i18n.NotifyKickDeleted)
}
func (m *memManager) kickUserKey(userID int64, key string) {
m.mu.Lock()
defer m.mu.Unlock()
userMap := m.byUser[userID]
@ -62,7 +70,7 @@ func (m *memManager) KickUser(userID int64) {
return
}
for id, existing := range userMap {
reason := i18n.T(i18n.Parse(existing.Language()), i18n.NotifyKickDeleted, nil)
reason := i18n.T(i18n.Parse(existing.Language()), key, nil)
_ = existing.Send(&Packet{
Command: protocol.CmdKick,
Body: protocol.EncodeKick(existing.RequestID(), reason),

View File

@ -32,6 +32,7 @@ type Manager interface {
GetUserSessions(userID int64) []Session
KickDevice(userID int64, deviceID uint64)
KickUser(userID int64)
KickUserBanned(userID int64)
SendToUserExceptDevice(ctx context.Context, userID int64, exceptDeviceID uint64, packet *Packet) error
}

View File

@ -0,0 +1,35 @@
-- Sensitive words, hit audit, account send mute.
ALTER TABLE t_users
ADD COLUMN send_muted_until DATETIME(3) NULL
COMMENT '全站禁言截止时间,空或已过期表示可发消息';
CREATE TABLE IF NOT EXISTS t_sensitive_words (
id BIGINT NOT NULL AUTO_INCREMENT COMMENT '行 id,表内自增',
word_raw VARCHAR(64) NOT NULL COMMENT '运营录入原文,展示用',
word VARCHAR(64) NOT NULL COMMENT 'FoldAndIndex 后的词面,全局唯一,进自动机',
category TINYINT NOT NULL COMMENT '1涉黑 2涉毒 3涉黄 4涉赌 5涉政 6宗教 7辱骂 8广告 9其他',
action TINYINT NOT NULL COMMENT '1log 2replace 3reject 4hide 5mute 6disable',
mute_seconds INT NOT NULL DEFAULT 0 COMMENT 'action=mute 时的禁言秒数',
enabled TINYINT NOT NULL DEFAULT 1 COMMENT '1启用 0下线(不进自动机)',
created_at DATETIME(3) NOT NULL COMMENT '创建时间',
updated_at DATETIME(3) NOT NULL COMMENT '更新时间',
PRIMARY KEY (id),
UNIQUE KEY uk_word (word)
) COMMENT='敏感词;发送热路径使用进程内自动机,本表为真源';
CREATE TABLE IF NOT EXISTS t_sensitive_hits (
id BIGINT NOT NULL COMMENT 'Snowflake',
user_id BIGINT NOT NULL COMMENT '发送者 uid',
conversation_id BIGINT NOT NULL DEFAULT 0 COMMENT '会话 id',
word_id BIGINT NOT NULL COMMENT 't_sensitive_words.id',
category TINYINT NOT NULL COMMENT '命中时的类别快照',
action TINYINT NOT NULL COMMENT '本行对应词的 action',
excerpt VARCHAR(128) NOT NULL DEFAULT '' COMMENT '命中片段,截断',
real_ip VARCHAR(64) NOT NULL DEFAULT '' COMMENT 'PROXY RealIP',
created_at DATETIME(3) NOT NULL COMMENT '命中时间',
PRIMARY KEY (id),
KEY idx_user_created (user_id, created_at),
KEY idx_created (created_at),
KEY idx_word (word_id)
) COMMENT='敏感词命中审计,供 Admin 查询';

View File

@ -0,0 +1,22 @@
-- 敏感词种子(按 docs/moderation.md §4.3 默认处罚)。word 为 FoldAndIndex 后的词面,与入库规则一致。
-- 可重复执行:uk_word 冲突时仅刷新 updated_at。
INSERT INTO t_sensitive_words (word_raw, word, category, action, mute_seconds, enabled, created_at, updated_at)
VALUES
('黑社会', '黑社会', 1, 6, 0, 1, CURRENT_TIMESTAMP(3), CURRENT_TIMESTAMP(3)),
('黑帮', '黑帮', 1, 6, 0, 1, CURRENT_TIMESTAMP(3), CURRENT_TIMESTAMP(3)),
('冰毒', '冰毒', 2, 6, 0, 1, CURRENT_TIMESTAMP(3), CURRENT_TIMESTAMP(3)),
('海洛因', '海洛因', 2, 6, 0, 1, CURRENT_TIMESTAMP(3), CURRENT_TIMESTAMP(3)),
('色情', '色情', 3, 3, 0, 1, CURRENT_TIMESTAMP(3), CURRENT_TIMESTAMP(3)),
('裸聊', '裸聊', 3, 3, 0, 1, CURRENT_TIMESTAMP(3), CURRENT_TIMESTAMP(3)),
('赌博', '赌博', 4, 3, 0, 1, CURRENT_TIMESTAMP(3), CURRENT_TIMESTAMP(3)),
('博彩', '博彩', 4, 3, 0, 1, CURRENT_TIMESTAMP(3), CURRENT_TIMESTAMP(3)),
('分裂国家', '分裂国家', 5, 6, 0, 1, CURRENT_TIMESTAMP(3), CURRENT_TIMESTAMP(3)),
('邪教', '邪教', 6, 3, 0, 1, CURRENT_TIMESTAMP(3), CURRENT_TIMESTAMP(3)),
('傻逼', '傻逼', 7, 2, 0, 1, CURRENT_TIMESTAMP(3), CURRENT_TIMESTAMP(3)),
('去死', '去死', 7, 2, 0, 1, CURRENT_TIMESTAMP(3), CURRENT_TIMESTAMP(3)),
('恶意刷屏', '恶意刷屏', 7, 5, 3600, 1, CURRENT_TIMESTAMP(3), CURRENT_TIMESTAMP(3)),
('加微信', '加微信', 8, 1, 0, 1, CURRENT_TIMESTAMP(3), CURRENT_TIMESTAMP(3)),
('代理加盟', '代理加盟', 8, 1, 0, 1, CURRENT_TIMESTAMP(3), CURRENT_TIMESTAMP(3)),
('违禁测试', '违禁测试', 9, 3, 0, 1, CURRENT_TIMESTAMP(3), CURRENT_TIMESTAMP(3))
ON DUPLICATE KEY UPDATE updated_at = VALUES(updated_at);