feat: update object key

This commit is contained in:
zhr 2026-09-18 17:14:38 +08:00
parent 8aa8e0bad9
commit c11fbda141
40 changed files with 1014 additions and 162 deletions

View File

@ -95,7 +95,7 @@ Media: HTTP upload credential, message content is object key only. Frame max 64K
Message types: text / image / voice / video / emoji / gif / file / location / system (`message/entity.Type` 1–9). `messages.content` stores serialized JSON bytes (BLOB), never a raw string. Closed type/status values: [enums.md](enums.md). Message types: text / image / voice / video / emoji / gif / file / location / system (`message/entity.Type` 1–9). `messages.content` stores serialized JSON bytes (BLOB), never a raw string. Closed type/status values: [enums.md](enums.md).
TCP body is JSON envelope `{code,msg,request_id,data}`. AUTH request is `{"token","device_id","app_version","language"}` (not protobuf). TCP body is JSON envelope `{code,msg,request_id,data}`. AUTH request is `{"token","device_id","app_version","language"}` (not protobuf).
Public TLS is Nginx `:443` SNI: HTTP cert on loopback `:8443`, IM cert on loopback `:19001` then PROXY to gnet `:9090` (host `:9000` is rustfs). See `deploy/nginx/`. Public TLS is Nginx `:443` SNI: HTTP cert on loopback `:8443` (`sparkapi` + `sparkr`/RustFS), IM cert on loopback `:19001` then PROXY to gnet `:9090` (host `:9000` is rustfs). See `deploy/nginx/`.
AUTH/login rate-limit uses `reqctx.RealIP`. TaskPool workers recover panics. Deliver workers start with `RequeueStale` + `SETNX deliver:done:{msgId}`. AUTH/login rate-limit uses `reqctx.RealIP`. TaskPool workers recover panics. Deliver workers start with `RequeueStale` + `SETNX deliver:done:{msgId}`.
## When adding a feature ## When adding a feature

329
README.md
View File

@ -1,21 +1,322 @@
# spark-server # spark-server
Spark IM modular monolith. Architecture: [docs/architecture-v3.md](docs/architecture-v3.md). Spark IM 服务端。Go 1.26 **模块化单体**(一台机器、一个进程)。公网只开 **443**,HTTP / IM / 对象存储用三个域名、Nginx SNI 分流。
- Go 1.26 仓库:<https://gitea.wyrs365.com/Spark/spark-server.git>
- Gin HTTP on `127.0.0.1:8080`. JSON envelope `{"code","msg","request_id","data"}`. Docs: `GET /docs` (API), `/docs/enums`, `/docs/errors`. Spec: `GET /openapi.yaml`. 完整设计:[docs/architecture-v3.md](docs/architecture-v3.md)
- Probes: `GET /healthz` liveness; `GET /readyz` readiness (HTTP 503 when MySQL/Redis/task pool/accepting are not ready). `env: prod` (or `SPARK_ENV=prod`) rejects empty MySQL DSN, empty Redis, and the default JWT secret. 在线协议:`GET /docs`(Swagger)、`/docs/enums`、`/docs/errors`,规范 `GET /openapi.yaml`。
- Logs: zap to console and/or rotating file (`log.console` / `log.file`). GORM SQL goes through the same zap. `request_id` is server-generated and returned in the response body (also as `X-Request-ID`).
- gnet TCP on loopback (local `127.0.0.1:8090`; ECS Docker `127.0.0.1:9090` because host `:9000` is rustfs). gnet v2.10 has no built-in TLS; Nginx terminates IM TLS and forwards PROXY+plaintext. Sample: `deploy/nginx/`. `make deploy` writes image/config/logs under `/data/spark-server/`.
- TCP Frame body is JSON `{code,msg,request_id,data}`. AUTH body: `{"token","device_id","app_version","language"}`.
- Public entry is Nginx `:443` only (SNI split)
```bash 不上 Kafka、K8s、微服务。不要给 gnet 绑公网端口,不要在 EventLoop 里打 MySQL。
make start
make migrate ---
make lint
## 它做什么
即时通讯:登录、好友、群、发消息、增量同步、撤回删除、媒体直传。
| 层 | 职责 |
| --- | --- |
| 实时 | TCP 长连接(gnet)。在线推送 `MESSAGE`,心跳 30s,空闲 120s 断开 |
| 正文 | MySQL。Snowflake `MsgId` 是主键、排序、同步游标。没有 `ConversationSeq` |
| 会话列表 / 未读 | Redis ZSET。群热路径不写 MySQL inbox 行 |
| 扇出 | Redis `deliver:queue` + worker。`SEND_ACK` = MySQL COMMIT **且** 入队成功 |
| 媒体 | 客户端拿 Presigned URL 直传 RustFS。消息里只存 `object_key` |
HTTP 和 TCP 调同一套 `internal/module/*/logic`。Gin handler 只做绑定和写信封。
---
## 生产入口
| 用途 | 域名 | 说明 |
| --- | --- | --- |
| HTTP API | `https://sparkapi.wyrs365.com` | REST `/v1/...`,文档 `/docs` |
| IM 长连接 | `sparkim.wyrs365.com:443` | TLS + 自定义 Frame,不是 WebSocket |
| 对象存储 | `https://sparkr.wyrs365.com` | RustFS S3 path-style,Presigned PUT/GET |
本机开发:HTTP `http://127.0.0.1:8080`,TCP 明文 `127.0.0.1:8090`。
ECS 上进程只绑回环,由 Nginx 终结 TLS:
```text
客户端 :443 (ssl_preread)
├─ sparkapi.wyrs365.com → 127.0.0.1:8443 → Gin 127.0.0.1:18080
├─ sparkr.wyrs365.com → 127.0.0.1:8443 → RustFS 127.0.0.1:9000
└─ sparkim.wyrs365.com → 127.0.0.1:19001 (IM TLS)
→ PROXY → gnet 127.0.0.1:9090
``` ```
`make start` runs `cmd/server` against `configs/config.yaml` (override with `make start CONFIG=path/to.yaml`). `make run` builds `bin/spark-server` first, then starts that binary. 主机 `:9000` 是 RustFS S3,所以 IM gnet 用 **9090**。安全组只放行 443。
Agent conventions live in `.cursor/rules` and `.cursor/skills/spark-im-v3`. Lint conventions live in `.golangci.yaml`. ---
## 硬约束(不要改)
| ID | 规则 |
| --- | --- |
| C1 | 消息正文在 MySQL。Snowflake `MsgId` 是 PK / 排序 / 同步游标。没有 ConversationSeq |
| C2 | gnet EventLoop 不调 MySQL / Redis / 对象存储 / 扇出。入站进 TaskPool;扇出走 `deliver:queue` |
| C3 | 保密靠 TLS。Frame 只有薄头 + 可选 gzip。没有应用层 ECDH/AES |
| C4 | 公网只有 443。gnet / Gin 绑回环 |
| C5 | `SEND_ACK` 必须在 COMMIT **且** deliver `Push` 成功之后。入队失败不 ACK |
| C6 | 限流 / 审计用 PROXY / `X-Real-IP` 的 `RealIP`,不用 gnet `RemoteAddr`(那是 127.0.0.1) |
| C7 | 会话列表 / 未读 / 预览在 Redis ZSET。群热路径不 upsert MySQL inbox |
热路径禁止 `context.TODO()`。推送超时不踢连接。不要把媒体字节写进 `messages.content`。
---
## 统一响应
HTTP 与 TCP **应答 Body** 同一套信封。业务成败只看 `code` 字符串,不要用数字 `0` 比较。除 `/readyz` 未就绪是 HTTP 503 外,业务失败也是 HTTP 200。
```json
{"code":"0","msg":"ok","request_id":"3H8kPq2mN1R","data":{}}
```
| 字段 | 规则 |
| --- | --- |
| `code` | `"0"` 成功;其它失败。全表见 [/docs/errors](https://sparkapi.wyrs365.com/docs/errors) |
| `msg` | 展示文案,随语言变。成功一般为 `ok` |
| `request_id` | 服务端生成,**11 位 base62**(`0-9A-Za-z`)。客户端不用传。HTTP 同时写 Header `X-Request-ID` |
| `data` | 成功为业务对象;失败固定 `{}`。缺字段当空对象,不要当 `null` |
`request_id` 和 TCP Frame 头里的 `RequestID`(uint64)不是一回事:后者是客户端填的配对号,服务端原样带回。
### HTTP 请求放哪
| 位置 | 内容 |
| --- | --- |
| Header | `Authorization: Bearer …`、`X-Device-ID`、`X-Language` 或 `Accept-Language`、`X-App-Version`、`Content-Type` |
| Body | POST/PATCH 业务字段。不要把 token / 语言 / device_id 塞进业务 JSON(登录 Body 带 `device_id`,刷新 Body 带 `refresh_token`) |
| Path | 资源 id:好友 uid、群号、MsgId |
| Query | GET 筛选 / 分页 |
语言:`zh-CN`(缺省)/ `en` / `zh-TW`。网关写 `X-Real-IP`,客户端不要发。
---
## 登录
1. (可选)`POST /v1/auth/email-codes` Body `{"email"}`。
2. `POST /v1/auth/login` 拿 `access_token` / `refresh_token`。`device_id` 由**客户端生成并持久化**,服务端不分配;写入 JWT `did`。
3. 其它 HTTP:`Authorization: Bearer …`,可选 `X-Device-ID`(须等于登录时的 device_id)。
4. 续期:`POST /v1/auth/token/refresh`,Body `{"refresh_token"}`,不要带 Bearer。
5. 在线发消息、收推送、增量同步走 **TCP**。`POST /v1/messages` 只是 HTTP 调试通道,不要当 App 主路径。
测试号 `test01@spark.com`~`test10@spark.com` 可跳过发码,登录任意 `verify_code`。
`uid` 就是 Snowflake `users.id`。对外展示用可变的 `spark_id`,不要把 uid 写进对象存储路径。
---
## HTTP 资源
前缀 `/v1/{resource}`。
| 组 | 路径 | 说明 |
| --- | --- | --- |
| 探活 | `/healthz` `/readyz` | 存活;未就绪 HTTP 503 |
| 文档 | `/docs` `/docs/enums` `/docs/errors` `/openapi.yaml` | 无需登录 |
| 枚举 | `GET /v1/meta/enums` | 无需登录 |
| 认证 | `/v1/auth` | 发码、登录、刷新、登出 |
| 用户 | `/v1/users/me` | 资料 |
| 好友 | `/v1/friends` | 申请、同意、备注、置顶、拉黑、举报 |
| 群 | `/v1/groups` | 创建、资料、成员、邀请、踢人、转让、解散 |
| 文件 | `/v1/files` | 上传/下载凭证、批量、图片派生 |
| 收藏 | `/v1/favorites` | 收藏;媒体会 Copy 到 `perm/fav/` |
| 会话 | `/v1/conversations` | 列表、免打扰 |
| 消息 | `/v1/messages` | 历史 / 撤回 / 删除(主路径仍是 TCP) |
---
## TCP 协议
不是 WebSocket、不是 HTTP、不是 Protobuf。对 `sparkim.wyrs365.com:443` 做 TLS(SNI 必须是该域名),之后写二进制 Frame。
### Frame(大端,固定 23 字节头)
```text
offset 0 Magic uint16 0x5350(ASCII "SP")
offset 2 Version uint8 1
offset 3 Command uint16
offset 5 Flags uint16 bit0=1 表示 Body 是 gzip
offset 7 RequestID uint64 客户端配对,可 0;响应原样带回
offset 15 SequenceID uint32 请求/响应配对;PUSH_ACK 必须带回推送帧的 SequenceID
offset 19 BodyLength uint32 Body 字节数(压缩后)
offset 23 Body bytes UTF-8 JSON
```
先读 23 字节,再按 `BodyLength` 读 Body。Magic 不对或总长超过 64KB 应断开。Body ≥ 512B 且 gzip 更小时可压缩。
**请求 Body 是业务 JSON,不要包信封。** 应答和 `MESSAGE` 推送才是 `{code,msg,request_id,data}`。失败时 Command 仍为请求 Command(发送失败不是 `SEND_ACK`)。
| 值 | 名 | 方向 |
| --- | --- | --- |
| 1 | AUTH | → 请求 `{"token","device_id","app_version","language"}` |
| 2 | HEARTBEAT | → 空 Body |
| 3 | GENERATE_MSG_ID | → 空;应答 `data.msg_id` |
| 4 | SEND_MESSAGE | → 与 HTTP 发消息同一 JSON |
| 5 | SEND_ACK | ← 仅 4 的成功应答 |
| 6 | MESSAGE | ← 推送 |
| 7 | PUSH_ACK | → 空;`SequenceID` = 推送帧的 SequenceID |
| 8 | READ_MESSAGE | → |
| 9 | RECALL_MESSAGE | → `{"msg_id"}` |
| 10 | SYNC_INBOX | → 会话列表增量 |
| 12 | SYNC_MESSAGES | → 某会话消息增量 |
| 14 | KICK | ← 同设备顶号 |
| 15 | DELETE_MESSAGE | → `{"msg_id"}` |
除 AUTH / HEARTBEAT / PUSH_ACK 外须先 AUTH,否则 `40102`。
上线同步顺序:**先 `SYNC_INBOX`,再按会话 `SYNC_MESSAGES`**。游标是 Snowflake `MsgId` / inbox score,不是 ConversationSeq。
Java / Swift 示例:`docs/clients/`,OpenAPI 里也可点开。
---
## 发消息路径
必须按这个顺序(C5):
1. TaskPool(禁止在 EventLoop 里做)
2. 权限(好友 / 拉黑 / 群禁言)——失败则不 INSERT
3. Snowflake MsgId(或客户端 MsgId,须在 ±5 分钟内)
4. MySQL 事务:消息表 + `conversations.last_msg_id`
5. `DeliverQueue.Push`
6. `SEND_ACK`
第 5 步失败不 ACK。客户端用同一个 `client_msg_id` 重试。发送设备不回推 `MESSAGE`,其它设备必须推。同 `device_id` 顶旧连接,下发 `KICK`。
私聊与群消息分表存储。消息类型 1–9:文本 / 图片 / 语音 / 视频 / 表情 / gif / 文件 / 位置 / 系统。`content` 是 JSON 序列化后的 **bytes**,不是字符串。
---
## 媒体 `object_key`
客户端 **不自己拼 key**。`POST /v1/files` 只传 `purpose` / `mime` / `size`(群头像再加 `group_id`),服务端签发 Presigned PUT。PUT 必须原样带返回的 `headers`(`Content-Type`、`Content-Length`、`If-None-Match: *`)。同一 key 第二次 PUT 为 412。
文件名是 **16 位 base62 随机串**(12 字节),不占用 Snowflake。路径不含 uid。
| purpose | object_key | 生命周期 |
| --- | --- | --- |
| `avatar` | `perm/avatar/{yyyyMMdd}/{name}.{ext}` | 永久 |
| `group_avatar` | `perm/group/{群号}/{name}.{ext}` | 永久 |
| `chat_image` | `chat/image/{yyyyMMdd}/{name}.{ext}` | 桶生命周期 |
| `chat_voice` | `chat/voice/{yyyyMMdd}/{name}.{ext}` | 桶生命周期 |
| `chat_video` | `chat/video/{yyyyMMdd}/{name}.{ext}` | 桶生命周期 |
| `chat_gif` | `chat/gif/{yyyyMMdd}/{name}.{ext}` | 桶生命周期 |
| `chat_file` | `chat/file/{yyyyMMdd}/{name}.{ext}` | 桶生命周期 |
日期 UTC `yyyyMMdd`。`ext` 由 mime 映射,未知为 `bin`。收藏复制到 `perm/fav/{yyyyMMdd}/{name}.{ext}`。
合法 key:以 `perm/` 或 `chat/` 开头;禁止 `..`、空白、`\`、前导 `/`;最长 512。
发图片:客户端只 PUT 原图,再 `POST /v1/files/variants` 让服务端出缩略图和预览。消息里只存 key。
进程访问 RustFS 用内网 `object_storage.endpoint`(Docker 里是 `rustfs:9000`),签给客户端的 URL 用 `public_endpoint`(`https://sparkr.wyrs365.com`)。
---
## 目录
```text
cmd/server 进程入口
cmd/migrate 按序执行 migrations/*.sql
configs/config.yaml 本机开发配置
deploy/docker/ 生产镜像、compose、deploy.sh
deploy/nginx/ 443 SNI(host/ 是这台 ECS 的 drop-in)
docs/architecture-v3.md 架构全文
docs/clients/ TCP Java / Swift demo
internal/bootstrap 读配置、接线、生命周期
internal/transport/http Gin;handler 只绑定
internal/transport/tcp gnet Frame + TaskPool
internal/module/{biz}/entity 领域类型,无 I/O
internal/module/{biz}/model 一张表一个 Go 文件(GORM + COMMENT)
internal/module/{biz}/logic 一个功能一个文件,禁止 import model
internal/pkg errs / i18n / reply / reqctx / validate
internal/infrastructure MySQL / Redis / Snowflake / zap / RustFS
migrations/ SQL,改完必须在本机跑 migrate
```
`logic` 不 import `model`。接线在 `internal/bootstrap`。表和列必须有 COMMENT。
---
## 本机开发
需要:Go 1.26、MySQL 8、Redis。对象存储可空(上传接口会不可用)。
```bash
# 配置 configs/config.yaml(env: dev)
make migrate # 应用 migrations/*.sql
make start # go run ./cmd/server -config configs/config.yaml
# 或
make run # 先编 bin/spark-server 再启动
```
覆盖配置:`make start CONFIG=path/to.yaml`。
```bash
make test
make lint # funlen 100,lll 240
```
探活:
```bash
curl -sS http://127.0.0.1:8080/healthz
curl -sS http://127.0.0.1:8080/readyz
```
`env: prod`(或 `SPARK_ENV=prod`)会拒绝空 MySQL DSN、空 Redis、以及默认 JWT secret。
改了 `migrations/*.sql` 后自己跑 `make migrate`,不要留给部署。
---
## 部署(ECS)
`make deploy` → `deploy/docker/deploy.sh`。默认 `root@47.236.66.9`,密钥 `~/.ssh/spark-ecs`。
主机目录:
| 路径 | 内容 |
| --- | --- |
| `/data/spark-server/app` | 构建源码 |
| `/data/spark-server/configs/config.yaml` | 运行配置(来自 `deploy/docker/config.docker.yaml`) |
| `/data/spark-server/logs` | 进程日志 |
| `/data/spark-server/images/spark-server.tar.gz` | `docker save` |
容器加入已有 `shell_default` 网络(mysql / redis / rustfs)。发布:
- `127.0.0.1:18080` → 容器 HTTP 8080
- `127.0.0.1:9090` → 容器 gnet 9090
Nginx drop-in:`deploy/nginx/host/`,安装脚本 `bash deploy/nginx/install-host.sh`。不要覆盖整份 `/etc/nginx/nginx.conf`。
Snowflake `snowflake_worker_id` 每进程唯一。时钟回拨会拒绝发号。
---
## 日志与 ID
- zap 控制台和/或滚动文件(`log.console` / `log.file`)。GORM SQL 走同一套 zap。
- HTTP `request_id`:11 位 base62,响应体 + `X-Request-ID`。
- 对象文件名:16 位 base62,与 Snowflake 号段分开。
- Snowflake 只给 uid、MsgId、会话 id、收藏行 id 等业务主键。
---
## 相关文档
| 文档 | 内容 |
| --- | --- |
| [docs/architecture-v3.md](docs/architecture-v3.md) | 架构全文 |
| `GET /docs` | OpenAPI / Swagger |
| `GET /docs/enums` | 可跳转枚举 |
| `GET /docs/errors` | 信封错误码 |
| [deploy/nginx/README.md](deploy/nginx/README.md) | 443 SNI |
| `docs/clients/` | TCP 客户端示例 |
| `.cursor/skills/spark-im-v3/` | 实现约定(给 Agent) |
| `.golangci.yaml` | 代码规范 |

View File

@ -84,6 +84,7 @@ message:
object_storage: object_storage:
endpoint: "47.236.66.9:9000" endpoint: "47.236.66.9:9000"
public_endpoint: "https://sparkr.wyrs365.com"
access_key: "rustfs_spark" access_key: "rustfs_spark"
secret_key: 'kWvGtYT/~q+35q;3dSI<^P!MH+bgrpBg' secret_key: 'kWvGtYT/~q+35q;3dSI<^P!MH+bgrpBg'
bucket: "spark" bucket: "spark"

View File

@ -84,6 +84,7 @@ message:
object_storage: object_storage:
endpoint: "rustfs:9000" endpoint: "rustfs:9000"
public_endpoint: "https://sparkr.wyrs365.com"
access_key: "rustfs_spark" access_key: "rustfs_spark"
secret_key: 'kWvGtYT/~q+35q;3dSI<^P!MH+bgrpBg' secret_key: 'kWvGtYT/~q+35q;3dSI<^P!MH+bgrpBg'
bucket: "spark" bucket: "spark"

View File

@ -33,6 +33,6 @@ rsync -az --delete \
"${SSH[@]}" "docker save spark-server:latest | gzip -c > $DATA/images/spark-server.tar.gz" "${SSH[@]}" "docker save spark-server:latest | gzip -c > $DATA/images/spark-server.tar.gz"
"${SSH[@]}" "cd $REMOTE/deploy/docker && docker compose -f docker-compose.yaml up -d spark-server" "${SSH[@]}" "cd $REMOTE/deploy/docker && docker compose -f docker-compose.yaml up -d spark-server"
"${SSH[@]}" "docker exec spark-server /usr/local/bin/spark-migrate -config /app/configs/config.yaml -dir /app/migrations" "${SSH[@]}" "docker exec spark-server /usr/local/bin/spark-migrate -config /app/configs/config.yaml -dir /app/migrations"
"${SSH[@]}" "cp $REMOTE/deploy/nginx/host/spark-sni.conf /etc/nginx/stream.d/spark-sni.conf && nginx -t && systemctl reload nginx" "${SSH[@]}" "cp $REMOTE/deploy/nginx/host/spark-sni.conf /etc/nginx/stream.d/spark-sni.conf && if [ -f /etc/letsencrypt/live/sparkr.wyrs365.com/fullchain.pem ]; then cp $REMOTE/deploy/nginx/host/sparkr.conf /etc/nginx/conf.d/sparkr.conf; fi && nginx -t && systemctl reload nginx"
"${SSH[@]}" "docker ps --filter name=spark-server --format 'table {{.Names}}\t{{.Status}}\t{{.Ports}}'" "${SSH[@]}" "docker ps --filter name=spark-server --format 'table {{.Names}}\t{{.Status}}\t{{.Ports}}'"
"${SSH[@]}" "curl -fsS http://127.0.0.1:18080/healthz && echo && curl -fsS http://127.0.0.1:18080/readyz && echo" "${SSH[@]}" "curl -fsS http://127.0.0.1:18080/healthz && echo && curl -fsS http://127.0.0.1:18080/readyz && echo"

View File

@ -6,6 +6,8 @@ gnet v2 has no native TLS. IM certificates live here, not in spark-server `tcp.t
Client --TLS--> :443 ssl_preread Client --TLS--> :443 ssl_preread
├─ SNI sparkapi.wyrs365.com → 127.0.0.1:8443 (HTTP TLS) ├─ SNI sparkapi.wyrs365.com → 127.0.0.1:8443 (HTTP TLS)
│ → proxy_pass http://127.0.0.1:18080 + X-Real-IP │ → proxy_pass http://127.0.0.1:18080 + X-Real-IP
├─ SNI sparkr.wyrs365.com → 127.0.0.1:8443 (HTTP TLS)
│ → proxy_pass http://127.0.0.1:9000 RustFS S3
└─ SNI sparkim.wyrs365.com → 127.0.0.1:19001 (IM TLS) └─ SNI sparkim.wyrs365.com → 127.0.0.1:19001 (IM TLS)
→ PROXY protocol → 127.0.0.1:9090 gnet → PROXY protocol → 127.0.0.1:9090 gnet
``` ```
@ -13,6 +15,7 @@ Client --TLS--> :443 ssl_preread
On this ECS, `:443` is shared with other wyrs HTTPS sites. Do **not** replace `/etc/nginx/nginx.conf`. Drop-ins: On this ECS, `:443` is shared with other wyrs HTTPS sites. Do **not** replace `/etc/nginx/nginx.conf`. Drop-ins:
- `host/sparkapi.conf` → `/etc/nginx/conf.d/` - `host/sparkapi.conf` → `/etc/nginx/conf.d/`
- `host/sparkr.conf` → `/etc/nginx/conf.d/` (RustFS S3; DNS A record `sparkr.wyrs365.com` → this host)
- `host/spark-sni.conf` → `/etc/nginx/stream.d/` - `host/spark-sni.conf` → `/etc/nginx/stream.d/`
Existing `listen 443 ssl` vhosts move to `127.0.0.1:8443 ssl proxy_protocol` so SNI can take public 443. `00-spark-http.conf` sets `real_ip_header proxy_protocol` and `port_in_redirect off` so `$remote_addr` is the client and redirects stay on 443. Existing `listen 443 ssl` vhosts move to `127.0.0.1:8443 ssl proxy_protocol` so SNI can take public 443. `00-spark-http.conf` sets `real_ip_header proxy_protocol` and `port_in_redirect off` so `$remote_addr` is the client and redirects stay on 443.
@ -20,9 +23,10 @@ Existing `listen 443 ssl` vhosts move to `127.0.0.1:8443 ssl proxy_protocol` so
spark-server (Docker): spark-server (Docker):
- HTTP `127.0.0.1:18080` → container `:8080` - HTTP `127.0.0.1:18080` → container `:8080`
- TCP `127.0.0.1:9090` with `proxy_protocol: true` → container `:9090` (host `:9000` is rustfs S3) - TCP `127.0.0.1:9090` with `proxy_protocol: true` → container `:9090` (host `:9000` is rustfs S3; bind `127.0.0.1:9000`, not public)
- Image / config / logs: `/data/spark-server/{images,configs,logs}` - Image / config / logs: `/data/spark-server/{images,configs,logs}`
- Public security group: 443 only - Public security group: 443 only
- Clients upload/download at `https://sparkr.wyrs365.com` (path-style). spark-server talks to `rustfs:9000` on the Docker network and signs URLs with `object_storage.public_endpoint`.
```bash ```bash
bash deploy/nginx/install-host.sh bash deploy/nginx/install-host.sh

View File

@ -3,7 +3,7 @@
server { server {
listen 80; listen 80;
listen [::]:80; listen [::]:80;
server_name sparkapi.wyrs365.com sparkim.wyrs365.com; server_name sparkapi.wyrs365.com sparkim.wyrs365.com sparkr.wyrs365.com;
location /.well-known/acme-challenge/ { location /.well-known/acme-challenge/ {
root /var/www/certbot; root /var/www/certbot;

View File

@ -1,6 +1,6 @@
# Spark IM + shared HTTPS on :443 (ssl_preread). # Spark IM + shared HTTPS on :443 (ssl_preread).
# sparkim.wyrs365.com → IM TLS terminator → PROXY plaintext to gnet :9090 # sparkim.wyrs365.com → IM TLS terminator → PROXY plaintext to gnet :9090
# default SNI → existing HTTP sites on 127.0.0.1:8443 (proxy_protocol restores RealIP) # default SNI → 127.0.0.1:8443 HTTP TLS (sparkapi / sparkr / other wyrs sites; PROXY restores RealIP)
map $ssl_preread_server_name $spark_sni_backend { map $ssl_preread_server_name $spark_sni_backend {
sparkim.wyrs365.com 127.0.0.1:19001; sparkim.wyrs365.com 127.0.0.1:19001;

View File

@ -0,0 +1,58 @@
# Spark RustFS S3 — sparkr.wyrs365.com
# Public :443 is stream ssl_preread (see stream.d/spark-sni.conf).
# TLS terminates on loopback :8443; proxy HTTP to Docker rustfs on :9000.
# Do not rewrite the URI: SigV4 signs host + path.
upstream spark_rustfs {
server 127.0.0.1:9000;
keepalive 32;
}
server {
listen 80;
listen [::]:80;
server_name sparkr.wyrs365.com;
location /.well-known/acme-challenge/ {
root /var/www/certbot;
}
location / {
return 301 https://$host$request_uri;
}
}
server {
listen 127.0.0.1:8443 ssl proxy_protocol;
listen [::1]:8443 ssl proxy_protocol;
http2 on;
server_name sparkr.wyrs365.com;
ssl_certificate /etc/letsencrypt/live/sparkr.wyrs365.com/fullchain.pem;
ssl_certificate_key /etc/letsencrypt/live/sparkr.wyrs365.com/privkey.pem;
ssl_session_timeout 1d;
ssl_session_cache shared:SparkSSL:10m;
add_header Strict-Transport-Security "max-age=31536000; includeSubDomains" always;
add_header X-Content-Type-Options nosniff always;
ignore_invalid_headers off;
client_max_body_size 100m;
proxy_buffering off;
proxy_request_buffering off;
proxy_connect_timeout 300s;
proxy_send_timeout 300s;
proxy_read_timeout 300s;
proxy_http_version 1.1;
chunked_transfer_encoding off;
location / {
proxy_pass http://spark_rustfs;
proxy_set_header Host $http_host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto https;
proxy_set_header Connection "";
proxy_cache_convert_head off;
}
}

View File

@ -5,33 +5,61 @@ set -euo pipefail
ROOT="$(cd "$(dirname "$0")" && pwd)" ROOT="$(cd "$(dirname "$0")" && pwd)"
HOST_DIR="$ROOT/host" HOST_DIR="$ROOT/host"
BACKUP="/root/nginx-backup-$(date +%Y%m%d%H%M%S)" BACKUP="/root/nginx-backup-$(date +%Y%m%d%H%M%S)"
DOMAINS=(sparkapi.wyrs365.com sparkim.wyrs365.com sparkr.wyrs365.com)
need_certs() { cert_ok() {
[[ -f /etc/letsencrypt/live/sparkapi.wyrs365.com/fullchain.pem ]] \ [[ -f "/etc/letsencrypt/live/$1/fullchain.pem" ]]
&& [[ -f /etc/letsencrypt/live/sparkim.wyrs365.com/fullchain.pem ]]
} }
install_acme() { all_certs_ok() {
local d
for d in "${DOMAINS[@]}"; do
cert_ok "$d" || return 1
done
}
install_acme_domain() {
local d=$1
mkdir -p /var/www/certbot mkdir -p /var/www/certbot
cp "$HOST_DIR/spark-acme.conf" /etc/nginx/conf.d/spark-acme.conf cat > "/etc/nginx/conf.d/spark-acme-${d}.conf" <<EOF
server {
listen 80;
listen [::]:80;
server_name ${d};
location /.well-known/acme-challenge/ {
root /var/www/certbot;
}
location / {
return 404;
}
}
EOF
nginx -t nginx -t
systemctl reload nginx systemctl reload nginx
} }
issue_certs() { issue_missing_certs() {
certbot certonly --webroot -w /var/www/certbot \ local d
--non-interactive --agree-tos --keep-until-expiring \ for d in "${DOMAINS[@]}"; do
-d sparkapi.wyrs365.com if cert_ok "$d"; then
certbot certonly --webroot -w /var/www/certbot \ continue
--non-interactive --agree-tos --keep-until-expiring \ fi
-d sparkim.wyrs365.com install_acme_domain "$d"
certbot certonly --webroot -w /var/www/certbot \
--non-interactive --agree-tos --keep-until-expiring \
-d "$d"
rm -f "/etc/nginx/conf.d/spark-acme-${d}.conf"
done
} }
rewrite_http_443() { rewrite_http_443() {
local f local f
for f in /etc/nginx/conf.d/*.conf; do for f in /etc/nginx/conf.d/*.conf; do
case "$(basename "$f")" in case "$(basename "$f")" in
sparkapi.conf|spark-acme.conf) continue ;; sparkapi.conf|sparkr.conf|spark-acme.conf|00-spark-http.conf) continue ;;
spark-acme-*) continue ;;
esac esac
if grep -qE 'listen[[:space:]]+127\.0\.0\.1:8443' "$f"; then if grep -qE 'listen[[:space:]]+127\.0\.0\.1:8443' "$f"; then
continue continue
@ -48,8 +76,10 @@ rewrite_http_443() {
install_sni() { install_sni() {
rewrite_http_443 rewrite_http_443
rm -f /etc/nginx/conf.d/spark-acme.conf rm -f /etc/nginx/conf.d/spark-acme.conf
rm -f /etc/nginx/conf.d/spark-acme-*.conf
cp "$HOST_DIR/00-spark-http.conf" /etc/nginx/conf.d/00-spark-http.conf cp "$HOST_DIR/00-spark-http.conf" /etc/nginx/conf.d/00-spark-http.conf
cp "$HOST_DIR/sparkapi.conf" /etc/nginx/conf.d/sparkapi.conf cp "$HOST_DIR/sparkapi.conf" /etc/nginx/conf.d/sparkapi.conf
cp "$HOST_DIR/sparkr.conf" /etc/nginx/conf.d/sparkr.conf
cp "$HOST_DIR/spark-sni.conf" /etc/nginx/stream.d/spark-sni.conf cp "$HOST_DIR/spark-sni.conf" /etc/nginx/stream.d/spark-sni.conf
mkdir -p /etc/letsencrypt/renewal-hooks/deploy mkdir -p /etc/letsencrypt/renewal-hooks/deploy
cat > /etc/letsencrypt/renewal-hooks/deploy/reload-nginx.sh <<'HOOK' cat > /etc/letsencrypt/renewal-hooks/deploy/reload-nginx.sh <<'HOOK'
@ -69,12 +99,11 @@ mkdir -p "$BACKUP"
cp -a /etc/nginx "$BACKUP/nginx" cp -a /etc/nginx "$BACKUP/nginx"
echo "backup $BACKUP" echo "backup $BACKUP"
if ! need_certs; then if ! all_certs_ok; then
install_acme issue_missing_certs
issue_certs
fi fi
if ! need_certs; then if ! all_certs_ok; then
echo "certificates missing" >&2 echo "certificates missing" >&2
exit 1 exit 1
fi fi

View File

@ -138,6 +138,7 @@ func finishApp(cfg *Config, log *zap.Logger, logClose func() error, idgen *id.Ge
authLimit := ratelimit.New(cfg.RateLimit.AuthPerSecond) authLimit := ratelimit.New(cfg.RateLimit.AuthPerSecond)
objects, err := objectstore.Open(&objectstore.Options{ objects, err := objectstore.Open(&objectstore.Options{
Endpoint: cfg.ObjectStorage.Endpoint, Endpoint: cfg.ObjectStorage.Endpoint,
PublicEndpoint: cfg.ObjectStorage.PublicEndpoint,
AccessKey: cfg.ObjectStorage.AccessKey, AccessKey: cfg.ObjectStorage.AccessKey,
SecretKey: cfg.ObjectStorage.SecretKey, SecretKey: cfg.ObjectStorage.SecretKey,
Bucket: cfg.ObjectStorage.Bucket, Bucket: cfg.ObjectStorage.Bucket,
@ -166,7 +167,7 @@ func finishApp(cfg *Config, log *zap.Logger, logClose func() error, idgen *id.Ge
friends := wireFriends(idgen, db, users, deliverQueue, chat) friends := wireFriends(idgen, db, users, deliverQueue, chat)
groups := wireGroups(idgen, db, users, friends, deliverQueue, objects, chat) groups := wireGroups(idgen, db, users, friends, deliverQueue, objects, chat)
messages := wireMessages(idgen, friends, groups, deliverQueue, chat) messages := wireMessages(idgen, friends, groups, deliverQueue, chat)
files := wireFiles(cfg, objects, idgen, groups) files := wireFiles(cfg, objects, groups)
favorites := wireFavorites(idgen, db, objects, chat) favorites := wireFavorites(idgen, db, objects, chat)
app.tcp = tcp.NewServer(log, pool, sessions, &tcp.Options{ app.tcp = tcp.NewServer(log, pool, sessions, &tcp.Options{
Addr: cfg.Server.TCP.Addr, Addr: cfg.Server.TCP.Addr,
@ -276,11 +277,11 @@ func wireGroups(idgen *id.Generator, db *mysql.DB, users *userlogic.Service, fri
return grouplogic.NewService(repo, users, friends, idgen, chat.msgs, q, objects) return grouplogic.NewService(repo, users, friends, idgen, chat.msgs, q, objects)
} }
func wireFiles(cfg *Config, store *objectstore.Store, idgen *id.Generator, groups *grouplogic.Service) *uploadlogic.Service { func wireFiles(cfg *Config, store *objectstore.Store, groups *grouplogic.Service) *uploadlogic.Service {
if store == nil || !store.Ready() || idgen == nil { if store == nil || !store.Ready() {
return nil return nil
} }
return uploadlogic.NewService(store, idgen, uploadlogic.Options{ return uploadlogic.NewService(store, uploadlogic.Options{
TTL: cfg.ObjectStorage.PresignTTL, TTL: cfg.ObjectStorage.PresignTTL,
Limit: ratelimit.New(cfg.RateLimit.UploadPerSecond), Limit: ratelimit.New(cfg.RateLimit.UploadPerSecond),
Groups: groups, Groups: groups,

View File

@ -118,6 +118,7 @@ type RateLimitConfig struct {
type ObjectStorageConfig struct { type ObjectStorageConfig struct {
Endpoint string `yaml:"endpoint"` Endpoint string `yaml:"endpoint"`
PublicEndpoint string `yaml:"public_endpoint"`
AccessKey string `yaml:"access_key"` AccessKey string `yaml:"access_key"`
SecretKey string `yaml:"secret_key"` SecretKey string `yaml:"secret_key"`
Bucket string `yaml:"bucket"` Bucket string `yaml:"bucket"`
@ -229,6 +230,9 @@ func applyDefaults(cfg *Config) {
if v := os.Getenv("SPARK_S3_ENDPOINT"); v != "" { if v := os.Getenv("SPARK_S3_ENDPOINT"); v != "" {
cfg.ObjectStorage.Endpoint = v cfg.ObjectStorage.Endpoint = v
} }
if v := os.Getenv("SPARK_S3_PUBLIC_ENDPOINT"); v != "" {
cfg.ObjectStorage.PublicEndpoint = v
}
if v := os.Getenv("SPARK_S3_ACCESS_KEY"); v != "" { if v := os.Getenv("SPARK_S3_ACCESS_KEY"); v != "" {
cfg.ObjectStorage.AccessKey = v cfg.ObjectStorage.AccessKey = v
} }

View File

@ -18,13 +18,24 @@ import (
"spark-server/internal/pkg/errs" "spark-server/internal/pkg/errs"
) )
const maxObjectBytes = 4 << 20 const (
maxGetBytes = 32 << 20
maxPutBytes = 4 << 20
)
func (s *Store) PresignPut(ctx context.Context, key, contentType string, ttl time.Duration) (string, error) { func (s *Store) PresignPut(ctx context.Context, key, contentType string, size int64, ttl time.Duration) (string, error) {
if !s.Ready() { if !s.Ready() {
return "", errs.ErrAvailable return "", errs.ErrAvailable
} }
in := &s3.PutObjectInput{Bucket: aws.String(s.bucket), Key: aws.String(key)} if size < 1 {
return "", errs.ErrInvalidArgument
}
in := &s3.PutObjectInput{
Bucket: aws.String(s.bucket),
Key: aws.String(key),
ContentLength: aws.Int64(size),
IfNoneMatch: aws.String("*"),
}
if contentType != "" { if contentType != "" {
in.ContentType = aws.String(contentType) in.ContentType = aws.String(contentType)
} }
@ -61,11 +72,11 @@ func (s *Store) Get(ctx context.Context, key string) ([]byte, error) {
return nil, mapObjectErr(err) return nil, mapObjectErr(err)
} }
defer func() { _ = out.Body.Close() }() defer func() { _ = out.Body.Close() }()
data, err := io.ReadAll(io.LimitReader(out.Body, maxObjectBytes+1)) data, err := io.ReadAll(io.LimitReader(out.Body, maxGetBytes+1))
if err != nil { if err != nil {
return nil, mapObjectErr(err) return nil, mapObjectErr(err)
} }
if len(data) > maxObjectBytes { if len(data) > maxGetBytes {
return nil, errs.ErrInvalidArgument return nil, errs.ErrInvalidArgument
} }
return data, nil return data, nil
@ -75,7 +86,7 @@ func (s *Store) Put(ctx context.Context, key string, body []byte, contentType st
if !s.Ready() { if !s.Ready() {
return errs.ErrAvailable return errs.ErrAvailable
} }
if len(body) == 0 || len(body) > maxObjectBytes { if len(body) == 0 || len(body) > maxPutBytes {
return errs.ErrInvalidArgument return errs.ErrInvalidArgument
} }
in := &s3.PutObjectInput{ in := &s3.PutObjectInput{

View File

@ -29,6 +29,7 @@ type Store struct {
// Options configure Open. Empty Endpoint skips the connection. // Options configure Open. Empty Endpoint skips the connection.
type Options struct { type Options struct {
Endpoint string Endpoint string
PublicEndpoint string
AccessKey string AccessKey string
SecretKey string SecretKey string
Bucket string Bucket string
@ -42,7 +43,7 @@ func Open(o *Options) (*Store, error) {
if o == nil || strings.TrimSpace(o.Endpoint) == "" { if o == nil || strings.TrimSpace(o.Endpoint) == "" {
return &Store{}, nil return &Store{}, nil
} }
host, secure, err := parseEndpoint(o.Endpoint, o.UseSSL) client, presignBase, err := newClients(o)
if err != nil { if err != nil {
return nil, err return nil, err
} }
@ -50,19 +51,14 @@ func Open(o *Options) (*Store, error) {
if bucket == "" { if bucket == "" {
bucket = "spark" bucket = "spark"
} }
s := &Store{client: client, presign: s3.NewPresignClient(presignBase), bucket: bucket, log: o.Log}
if s.log == nil {
s.log = zap.NewNop()
}
region := strings.TrimSpace(o.Region) region := strings.TrimSpace(o.Region)
if region == "" { if region == "" {
region = defaultRegion region = defaultRegion
} }
scheme := "http"
if secure {
scheme = "https"
}
client := newS3Client(region, scheme+"://"+host, o.AccessKey, o.SecretKey)
s := &Store{client: client, presign: s3.NewPresignClient(client), bucket: bucket, log: o.Log}
if s.log == nil {
s.log = zap.NewNop()
}
ctx, cancel := context.WithTimeout(context.Background(), defaultTimeout) ctx, cancel := context.WithTimeout(context.Background(), defaultTimeout)
defer cancel() defer cancel()
if err := s.ensureBucket(ctx, region, o.ChatExpireDays); err != nil { if err := s.ensureBucket(ctx, region, o.ChatExpireDays); err != nil {
@ -83,6 +79,26 @@ func (s *Store) Ping(ctx context.Context) error {
return err return err
} }
func newClients(o *Options) (*s3.Client, *s3.Client, error) {
region := strings.TrimSpace(o.Region)
if region == "" {
region = defaultRegion
}
host, secure, err := parseEndpoint(o.Endpoint, o.UseSSL)
if err != nil {
return nil, nil, err
}
client := newS3Client(region, endpointURL(host, secure), o.AccessKey, o.SecretKey)
if pub := strings.TrimSpace(o.PublicEndpoint); pub != "" {
phost, psecure, err := parseEndpoint(pub, true)
if err != nil {
return nil, nil, err
}
return client, newS3Client(region, endpointURL(phost, psecure), o.AccessKey, o.SecretKey), nil
}
return client, client, nil
}
func newS3Client(region, baseURL, access, secret string) *s3.Client { func newS3Client(region, baseURL, access, secret string) *s3.Client {
return s3.New(s3.Options{ return s3.New(s3.Options{
Region: region, Region: region,
@ -94,8 +110,15 @@ func newS3Client(region, baseURL, access, secret string) *s3.Client {
}) })
} }
func endpointURL(host string, secure bool) string {
if secure {
return "https://" + host
}
return "http://" + host
}
func parseEndpoint(raw string, useSSL bool) (host string, secure bool, err error) { func parseEndpoint(raw string, useSSL bool) (host string, secure bool, err error) {
raw = strings.TrimSpace(raw) raw = strings.TrimSpace(strings.TrimRight(raw, "/"))
switch { switch {
case raw == "": case raw == "":
return "", false, fmt.Errorf("objectstore: empty endpoint") return "", false, fmt.Errorf("objectstore: empty endpoint")

View File

@ -40,13 +40,17 @@ func TestOpenEmpty(t *testing.T) {
func TestPresignLocal(t *testing.T) { func TestPresignLocal(t *testing.T) {
client := newS3Client(defaultRegion, "http://127.0.0.1:9000", "ak", "sk") client := newS3Client(defaultRegion, "http://127.0.0.1:9000", "ak", "sk")
s := &Store{client: client, presign: s3.NewPresignClient(client), bucket: "spark"} s := &Store{client: client, presign: s3.NewPresignClient(client), bucket: "spark"}
u, err := s.PresignPut(t.Context(), "chat/a.jpg", "image/jpeg", time.Hour) u, err := s.PresignPut(t.Context(), "chat/a.jpg", "image/jpeg", 1024, time.Hour)
if err != nil { if err != nil {
t.Fatal(err) t.Fatal(err)
} }
if !strings.Contains(u, "127.0.0.1:9000/spark/chat/a.jpg") { if !strings.Contains(u, "127.0.0.1:9000/spark/chat/a.jpg") {
t.Fatalf("%s", u) t.Fatalf("%s", u)
} }
signed := strings.ToLower(u)
if !strings.Contains(signed, "content-length") || !strings.Contains(signed, "if-none-match") {
t.Fatalf("unsigned size/overwrite headers: %s", u)
}
g, err := s.PresignGet(t.Context(), "perm/avatar/1.jpg", time.Minute) g, err := s.PresignGet(t.Context(), "perm/avatar/1.jpg", time.Minute)
if err != nil { if err != nil {
t.Fatal(err) t.Fatal(err)
@ -56,6 +60,37 @@ func TestPresignLocal(t *testing.T) {
} }
} }
func TestPresignPutRejectsZeroSize(t *testing.T) {
client := newS3Client(defaultRegion, "http://127.0.0.1:9000", "ak", "sk")
s := &Store{client: client, presign: s3.NewPresignClient(client), bucket: "spark"}
if _, err := s.PresignPut(t.Context(), "chat/a.jpg", "image/jpeg", 0, time.Hour); !errors.Is(err, errs.ErrInvalidArgument) {
t.Fatalf("%v", err)
}
}
func TestPresignPublicEndpoint(t *testing.T) {
internal, public, err := newClients(&Options{
Endpoint: "http://rustfs:9000",
PublicEndpoint: "https://sparkr.wyrs365.com/",
AccessKey: "ak",
SecretKey: "sk",
})
if err != nil {
t.Fatal(err)
}
s := &Store{client: internal, presign: s3.NewPresignClient(public), bucket: "spark"}
u, err := s.PresignPut(t.Context(), "chat/a.jpg", "image/jpeg", 1024, time.Hour)
if err != nil {
t.Fatal(err)
}
if !strings.Contains(u, "https://sparkr.wyrs365.com/spark/chat/a.jpg") {
t.Fatalf("%s", u)
}
if strings.Contains(u, "rustfs:9000") {
t.Fatalf("internal host leaked: %s", u)
}
}
func TestGetPutNotReady(t *testing.T) { func TestGetPutNotReady(t *testing.T) {
s, err := Open(&Options{}) s, err := Open(&Options{})
if err != nil { if err != nil {

View File

@ -7,9 +7,10 @@ import (
msgentity "spark-server/internal/module/message/entity" msgentity "spark-server/internal/module/message/entity"
uploadentity "spark-server/internal/module/upload/entity" uploadentity "spark-server/internal/module/upload/entity"
"spark-server/internal/pkg/errs" "spark-server/internal/pkg/errs"
"spark-server/internal/pkg/reqctx"
) )
func (s *Service) copyMedia(ctx context.Context, userID int64, payload *msgentity.Content) ([]string, error) { func (s *Service) copyMedia(ctx context.Context, payload *msgentity.Content) ([]string, error) {
keys := payload.ObjectKeys() keys := payload.ObjectKeys()
if len(keys) == 0 { if len(keys) == 0 {
return nil, nil return nil, nil
@ -24,12 +25,7 @@ func (s *Service) copyMedia(ctx context.Context, userID int64, payload *msgentit
if _, ok := mapping[src]; ok { if _, ok := mapping[src]; ok {
continue continue
} }
objectID, err := s.ids.NextID() dst := uploadentity.BuildFavKey(reqctx.NewObjectName(), day, uploadentity.ExtOfKey(src))
if err != nil {
s.deleteKeys(ctx, copied)
return nil, err
}
dst := uploadentity.BuildFavKey(userID, objectID, day, uploadentity.ExtOfKey(src))
if err := s.objects.Copy(ctx, src, dst); err != nil { if err := s.objects.Copy(ctx, src, dst); err != nil {
s.deleteKeys(ctx, copied) s.deleteKeys(ctx, copied)
return nil, err return nil, err

View File

@ -47,7 +47,7 @@ func (s *Service) Create(ctx context.Context, userID int64, in *CreateInput) (*I
if err != nil { if err != nil {
return nil, err return nil, err
} }
copied, err := s.copyMedia(ctx, userID, payload) copied, err := s.copyMedia(ctx, payload)
if err != nil { if err != nil {
return nil, err return nil, err
} }

View File

@ -56,8 +56,12 @@ func TestFavoriteCopiesMediaAndExpires(t *testing.T) {
t.Fatal(err) t.Fatal(err)
} }
src := "chat/20260917/1/99.jpg" src := "chat/20260917/1/99.jpg"
thumb := "chat/20260917/1/t.jpg"
orig := "chat/20260917/1/o.jpg"
objs.keys[src] = true objs.keys[src] = true
raw, err := msgentity.NewImage(&msgentity.Image{Key: src, Mime: "image/jpeg"}).Marshal() objs.keys[thumb] = true
objs.keys[orig] = true
raw, err := msgentity.NewImage(&msgentity.Image{Key: src, ThumbKey: thumb, OrigKey: orig, Mime: "image/jpeg"}).Marshal()
if err != nil { if err != nil {
t.Fatal(err) t.Fatal(err)
} }
@ -76,10 +80,10 @@ func TestFavoriteCopiesMediaAndExpires(t *testing.T) {
if err != nil { if err != nil {
t.Fatal(err) t.Fatal(err)
} }
if item.MsgID != msgID || !strings.Contains(string(item.Content), "perm/fav/1/") { if item.MsgID != msgID || !strings.Contains(string(item.Content), "perm/fav/") {
t.Fatalf("%+v", item) t.Fatalf("%+v", item)
} }
if strings.Contains(string(item.Content), src) { if strings.Contains(string(item.Content), src) || strings.Contains(string(item.Content), orig) {
t.Fatal("still points at chat key") t.Fatal("still points at chat key")
} }
again, err := svc.Create(ctx, 1, &CreateInput{MsgID: msgID}) again, err := svc.Create(ctx, 1, &CreateInput{MsgID: msgID})

View File

@ -13,6 +13,7 @@ import (
"spark-server/internal/module/group/entity" "spark-server/internal/module/group/entity"
uploadentity "spark-server/internal/module/upload/entity" uploadentity "spark-server/internal/module/upload/entity"
userentity "spark-server/internal/module/user/entity" userentity "spark-server/internal/module/user/entity"
"spark-server/internal/pkg/reqctx"
) )
const collageJPEG = "image/jpeg" const collageJPEG = "image/jpeg"
@ -63,10 +64,7 @@ func (s *Service) putAvatar(ctx context.Context, g *entity.Group, raw []byte, mi
if g == nil || len(raw) == 0 { if g == nil || len(raw) == 0 {
return false return false
} }
key := uploadentity.PrefixGroup + strconv.FormatInt(g.ID, 10) + "/default." + ext key := uploadentity.BuildObjectKey(uploadentity.PurposeGroupAvatar, g.ID, reqctx.NewObjectName(), "", ext)
if objectID, err := s.ids.NextID(); err == nil {
key = uploadentity.BuildObjectKey(uploadentity.PurposeGroupAvatar, g.ID, objectID, "", ext)
}
if err := s.objects.Put(ctx, key, raw, mime); err != nil { if err := s.objects.Put(ctx, key, raw, mime); err != nil {
return false return false
} }

View File

@ -121,7 +121,7 @@ func (c *Content) Validate() error {
case TypeText: case TypeText:
return validateText(c.Text) return validateText(c.Text)
case TypeImage: case TypeImage:
return validateMediaKey("image", c.Image.Key) return validateImage(c.Image)
case TypeVoice: case TypeVoice:
return validateVoice(c.Voice) return validateVoice(c.Voice)
case TypeVideo: case TypeVideo:
@ -220,6 +220,16 @@ func validateMentions(c *Content) error {
return nil return nil
} }
func validateImage(img *Image) error {
if err := validateMediaKey("image", img.Key); err != nil {
return err
}
if img.Size < 0 || img.OrigSize < 0 || img.Width < 0 || img.Height < 0 || img.OrigW < 0 || img.OrigH < 0 {
return fmt.Errorf("content: image size")
}
return nil
}
func validateMediaKey(kind, key string) error { func validateMediaKey(kind, key string) error {
if strings.TrimSpace(key) == "" { if strings.TrimSpace(key) == "" {
return fmt.Errorf("content: %s key empty", kind) return fmt.Errorf("content: %s key empty", kind)
@ -279,14 +289,21 @@ func (c *Content) ObjectKeys() []string {
var keys []string var keys []string
add := func(k string) { add := func(k string) {
k = strings.TrimSpace(k) k = strings.TrimSpace(k)
if k != "" { if k == "" {
keys = append(keys, k) return
} }
for _, existing := range keys {
if existing == k {
return
}
}
keys = append(keys, k)
} }
switch { switch {
case c.Image != nil: case c.Image != nil:
add(c.Image.Key) add(c.Image.Key)
add(c.Image.ThumbKey) add(c.Image.ThumbKey)
add(c.Image.OrigKey)
case c.Voice != nil: case c.Voice != nil:
add(c.Voice.Key) add(c.Voice.Key)
case c.Video != nil: case c.Video != nil:
@ -316,6 +333,7 @@ func (c *Content) ReplaceObjectKeys(mapping map[string]string) {
if c.Image != nil { if c.Image != nil {
repl(&c.Image.Key) repl(&c.Image.Key)
repl(&c.Image.ThumbKey) repl(&c.Image.ThumbKey)
repl(&c.Image.OrigKey)
} }
if c.Voice != nil { if c.Voice != nil {
repl(&c.Voice.Key) repl(&c.Voice.Key)

View File

@ -80,6 +80,9 @@ func TestRejectsRawStringAndEmptyKey(t *testing.T) {
if _, err := NewImage(&Image{}).Marshal(); err == nil || !strings.Contains(err.Error(), "key") { if _, err := NewImage(&Image{}).Marshal(); err == nil || !strings.Contains(err.Error(), "key") {
t.Fatalf("image key: %v", err) t.Fatalf("image key: %v", err)
} }
if _, err := NewImage(&Image{Key: "k", OrigSize: -1}).Marshal(); err == nil {
t.Fatal("negative orig_size")
}
bad := &Content{Type: TypeText, Text: &Text{Body: "x"}, Image: &Image{Key: "k"}} bad := &Content{Type: TypeText, Text: &Text{Body: "x"}, Image: &Image{Key: "k"}}
if _, err := bad.Marshal(); err == nil { if _, err := bad.Marshal(); err == nil {
t.Fatal("two payloads") t.Fatal("two payloads")
@ -122,13 +125,21 @@ func TestMentionsRoundTrip(t *testing.T) {
} }
func TestObjectKeysAndReplace(t *testing.T) { func TestObjectKeysAndReplace(t *testing.T) {
img := NewImage(&Image{Key: "chat/a.jpg", ThumbKey: "chat/t.jpg"}) img := NewImage(&Image{Key: "chat/a.jpg", ThumbKey: "chat/t.jpg", OrigKey: "chat/o.jpg", OrigSize: 4 << 20})
keys := img.ObjectKeys() keys := img.ObjectKeys()
if len(keys) != 2 || keys[0] != "chat/a.jpg" { if len(keys) != 3 || keys[0] != "chat/a.jpg" || keys[2] != "chat/o.jpg" {
t.Fatalf("%v", keys) t.Fatalf("%v", keys)
} }
img.ReplaceObjectKeys(map[string]string{"chat/a.jpg": "perm/fav/1/x.jpg", "chat/t.jpg": "perm/fav/1/y.jpg"}) img.ReplaceObjectKeys(map[string]string{
if img.Image.Key != "perm/fav/1/x.jpg" || img.Image.ThumbKey != "perm/fav/1/y.jpg" { "chat/a.jpg": "perm/fav/1/x.jpg",
"chat/t.jpg": "perm/fav/1/y.jpg",
"chat/o.jpg": "perm/fav/1/z.jpg",
})
if img.Image.Key != "perm/fav/1/x.jpg" || img.Image.ThumbKey != "perm/fav/1/y.jpg" || img.Image.OrigKey != "perm/fav/1/z.jpg" {
t.Fatalf("%+v", img.Image) t.Fatalf("%+v", img.Image)
} }
same := NewImage(&Image{Key: "chat/a.jpg", OrigKey: "chat/a.jpg"})
if got := same.ObjectKeys(); len(got) != 1 {
t.Fatalf("dedupe %v", got)
}
} }

View File

@ -5,7 +5,10 @@ type Text struct {
Body string `json:"body"` Body string `json:"body"`
} }
// Image references an uploaded object. Bytes themselves are not stored in messages.content. // Image references uploaded objects. Bytes are not stored in messages.content.
// WeChat-style: bubble uses thumb_key, tap-to-view uses key (compressed),
// 「查看原图」uses orig_key when the sender opted in.
// Clients PUT JPEG/PNG only (convert HEIC/WebP locally); the server does not transcode.
type Image struct { type Image struct {
Key string `json:"key"` Key string `json:"key"`
Mime string `json:"mime,omitempty"` Mime string `json:"mime,omitempty"`
@ -13,6 +16,10 @@ type Image struct {
Height int `json:"h,omitempty"` Height int `json:"h,omitempty"`
Size int64 `json:"size,omitempty"` Size int64 `json:"size,omitempty"`
ThumbKey string `json:"thumb_key,omitempty"` ThumbKey string `json:"thumb_key,omitempty"`
OrigKey string `json:"orig_key,omitempty"`
OrigSize int64 `json:"orig_size,omitempty"`
OrigW int `json:"orig_w,omitempty"`
OrigH int `json:"orig_h,omitempty"`
} }
// Voice is an audio clip in object storage. // Voice is an audio clip in object storage.

View File

@ -9,6 +9,11 @@ type Credential struct {
ExpiresIn int64 `json:"expires_in"` ExpiresIn int64 `json:"expires_in"`
} }
// CredentialSet is one HTTP round-trip for up to three PUTs (thumb / preview / orig).
type CredentialSet struct {
Items []Credential `json:"items"`
}
type Download struct { type Download struct {
ObjectKey string `json:"object_key"` ObjectKey string `json:"object_key"`
DownloadURL string `json:"download_url"` DownloadURL string `json:"download_url"`

View File

@ -16,11 +16,16 @@ const (
PurposeChatGIF = "chat_gif" PurposeChatGIF = "chat_gif"
PurposeChatFile = "chat_file" PurposeChatFile = "chat_file"
PrefixPerm = "perm/" PrefixPerm = "perm/"
PrefixChat = "chat/" PrefixChat = "chat/"
PrefixAvatar = "perm/avatar/" PrefixAvatar = "perm/avatar/"
PrefixGroup = "perm/group/" PrefixGroup = "perm/group/"
PrefixFav = "perm/fav/" PrefixFav = "perm/fav/"
PrefixChatImage = "chat/image/"
PrefixChatVoice = "chat/voice/"
PrefixChatVideo = "chat/video/"
PrefixChatGIF = "chat/gif/"
PrefixChatFile = "chat/file/"
MethodPUT = "PUT" MethodPUT = "PUT"
MethodGET = "GET" MethodGET = "GET"
@ -28,7 +33,8 @@ const (
const ( const (
maxAvatar = 2 << 20 maxAvatar = 2 << 20
maxImage = 10 << 20 maxImage = 32 << 20
maxGIF = 10 << 20
maxVoice = 16 << 20 maxVoice = 16 << 20
maxVideo = 100 << 20 maxVideo = 100 << 20
maxFile = 50 << 20 maxFile = 50 << 20
@ -75,25 +81,46 @@ func ExtOfKey(key string) string {
return ext return ext
} }
func BuildObjectKey(purpose string, ownerID, objectID int64, day, ext string) string { func BuildObjectKey(purpose string, ownerID int64, name, day, ext string) string {
if ext == "" { if ext == "" {
ext = extDefault ext = extDefault
} }
if name == "" {
name = "0"
}
switch purpose { switch purpose {
case PurposeAvatar: case PurposeAvatar:
return fmt.Sprintf("%s%d/%s/%d.%s", PrefixAvatar, ownerID, day, objectID, ext) return fmt.Sprintf("%s%s/%s.%s", PrefixAvatar, day, name, ext)
case PurposeGroupAvatar: case PurposeGroupAvatar:
return fmt.Sprintf("%s%d/%d.%s", PrefixGroup, ownerID, objectID, ext) return fmt.Sprintf("%s%d/%s.%s", PrefixGroup, ownerID, name, ext)
default: default:
return fmt.Sprintf("%s%s/%d/%d.%s", PrefixChat, day, ownerID, objectID, ext) return fmt.Sprintf("%s%s/%s/%s.%s", PrefixChat, chatKind(purpose), day, name, ext)
} }
} }
func BuildFavKey(userID, objectID int64, day, ext string) string { func BuildFavKey(name, day, ext string) string {
if ext == "" { if ext == "" {
ext = extDefault ext = extDefault
} }
return fmt.Sprintf("%s%d/%s/%d.%s", PrefixFav, userID, day, objectID, ext) if name == "" {
name = "0"
}
return fmt.Sprintf("%s%s/%s.%s", PrefixFav, day, name, ext)
}
func chatKind(purpose string) string {
switch purpose {
case PurposeChatImage:
return "image"
case PurposeChatVoice:
return "voice"
case PurposeChatVideo:
return "video"
case PurposeChatGIF:
return "gif"
default:
return "file"
}
} }
func ValidObjectKey(key string) bool { func ValidObjectKey(key string) bool {
@ -110,13 +137,13 @@ var specs = map[string]Spec{
PurposeAvatar: {Purpose: PurposeAvatar, MaxSize: maxAvatar, MIMEs: imageMIMEs}, PurposeAvatar: {Purpose: PurposeAvatar, MaxSize: maxAvatar, MIMEs: imageMIMEs},
PurposeGroupAvatar: {Purpose: PurposeGroupAvatar, MaxSize: maxAvatar, MIMEs: imageMIMEs}, PurposeGroupAvatar: {Purpose: PurposeGroupAvatar, MaxSize: maxAvatar, MIMEs: imageMIMEs},
PurposeChatImage: {Purpose: PurposeChatImage, MaxSize: maxImage, MIMEs: imageMIMEs}, PurposeChatImage: {Purpose: PurposeChatImage, MaxSize: maxImage, MIMEs: imageMIMEs},
PurposeChatGIF: {Purpose: PurposeChatGIF, MaxSize: maxImage, MIMEs: gifMIMEs}, PurposeChatGIF: {Purpose: PurposeChatGIF, MaxSize: maxGIF, MIMEs: gifMIMEs},
PurposeChatVoice: {Purpose: PurposeChatVoice, MaxSize: maxVoice, MIMEs: voiceMIMEs}, PurposeChatVoice: {Purpose: PurposeChatVoice, MaxSize: maxVoice, MIMEs: voiceMIMEs},
PurposeChatVideo: {Purpose: PurposeChatVideo, MaxSize: maxVideo, MIMEs: videoMIMEs}, PurposeChatVideo: {Purpose: PurposeChatVideo, MaxSize: maxVideo, MIMEs: videoMIMEs},
PurposeChatFile: {Purpose: PurposeChatFile, MaxSize: maxFile, MIMEs: fileMIMEs}, PurposeChatFile: {Purpose: PurposeChatFile, MaxSize: maxFile, MIMEs: fileMIMEs},
} }
var imageMIMEs = set("image/jpeg", "image/jpg", "image/png", "image/webp", "image/heic", "image/heif") var imageMIMEs = set("image/jpeg", "image/jpg", "image/png")
var gifMIMEs = set("image/gif", "image/webp") var gifMIMEs = set("image/gif", "image/webp")
var voiceMIMEs = set("audio/mpeg", "audio/mp4", "audio/aac", "audio/amr", "audio/ogg", "audio/wav", "audio/webm", "audio/x-m4a") var voiceMIMEs = set("audio/mpeg", "audio/mp4", "audio/aac", "audio/amr", "audio/ogg", "audio/wav", "audio/webm", "audio/x-m4a")
var videoMIMEs = set("video/mp4", "video/quicktime", "video/webm") var videoMIMEs = set("video/mp4", "video/quicktime", "video/webm")
@ -129,7 +156,7 @@ var fileMIMEs = set(
var mimeExt = map[string]string{ var mimeExt = map[string]string{
"image/jpeg": "jpg", "image/jpg": "jpg", "image/png": "png", "image/webp": "webp", "image/jpeg": "jpg", "image/jpg": "jpg", "image/png": "png", "image/webp": "webp",
"image/heic": "heic", "image/heif": "heif", "image/gif": "gif", "image/gif": "gif",
"audio/mpeg": "mp3", "audio/mp4": "m4a", "audio/aac": "aac", "audio/amr": "amr", "audio/mpeg": "mp3", "audio/mp4": "m4a", "audio/aac": "aac", "audio/amr": "amr",
"audio/ogg": "ogg", "audio/wav": "wav", "audio/webm": "weba", "audio/x-m4a": "m4a", "audio/ogg": "ogg", "audio/wav": "wav", "audio/webm": "weba", "audio/x-m4a": "m4a",
"video/mp4": "mp4", "video/quicktime": "mov", "video/webm": "webm", "video/mp4": "mp4", "video/quicktime": "mov", "video/webm": "webm",

View File

@ -6,20 +6,24 @@ import (
) )
func TestBuildObjectKeyByPurpose(t *testing.T) { func TestBuildObjectKeyByPurpose(t *testing.T) {
avatar := BuildObjectKey(PurposeAvatar, 11, 99, "20260917", "jpg") avatar := BuildObjectKey(PurposeAvatar, 11, "99", "20260917", "jpg")
if avatar != "perm/avatar/11/20260917/99.jpg" { if avatar != "perm/avatar/20260917/99.jpg" {
t.Fatalf("avatar %s", avatar) t.Fatalf("avatar %s", avatar)
} }
chat := BuildObjectKey(PurposeChatImage, 11, 99, "20260917", "jpg") chat := BuildObjectKey(PurposeChatImage, 11, "99", "20260917", "jpg")
if chat != "chat/20260917/11/99.jpg" { if chat != "chat/image/20260917/99.jpg" {
t.Fatalf("chat %s", chat) t.Fatalf("chat %s", chat)
} }
group := BuildObjectKey(PurposeGroupAvatar, 7, 99, "20260917", "png") voice := BuildObjectKey(PurposeChatVoice, 11, "aa", "20260917", "m4a")
if voice != "chat/voice/20260917/aa.m4a" {
t.Fatalf("voice %s", voice)
}
group := BuildObjectKey(PurposeGroupAvatar, 7, "99", "20260917", "png")
if group != "perm/group/7/99.png" { if group != "perm/group/7/99.png" {
t.Fatalf("group %s", group) t.Fatalf("group %s", group)
} }
fav := BuildFavKey(11, 7, "20260917", "m4a") fav := BuildFavKey("99", "20260917", "m4a")
if fav != "perm/fav/11/20260917/7.m4a" { if fav != "perm/fav/20260917/99.m4a" {
t.Fatalf("fav %s", fav) t.Fatalf("fav %s", fav)
} }
} }
@ -49,7 +53,19 @@ func TestNormalizeMIMEAndExt(t *testing.T) {
if _, ok := spec.MIMEs["audio/mpeg"]; !ok { if _, ok := spec.MIMEs["audio/mpeg"]; !ok {
t.Fatal("voice mime") t.Fatal("voice mime")
} }
if strings.Contains(BuildObjectKey(PurposeAvatar, 1, 1, "d", "jpg"), PrefixChat) { img, ok := LookupSpec(PurposeChatImage)
if !ok {
t.Fatal("image spec")
}
if _, ok := img.MIMEs["image/jpeg"]; !ok {
t.Fatal("jpeg")
}
for _, mime := range []string{"image/heic", "image/heif", "image/webp"} {
if _, ok := img.MIMEs[mime]; ok {
t.Fatalf("interchange must not allow %s", mime)
}
}
if strings.Contains(BuildObjectKey(PurposeAvatar, 1, "1", "d", "jpg"), PrefixChat) {
t.Fatal("avatar must be perm") t.Fatal("avatar must be perm")
} }
} }

View File

@ -0,0 +1,49 @@
package logic
import (
"context"
"spark-server/internal/module/upload/entity"
"spark-server/internal/pkg/errs"
"spark-server/internal/pkg/reqctx"
"spark-server/internal/pkg/validate"
)
const maxCredentialBatch = 3
// BatchCreateInput issues up to three Presigned PUTs in one call (thumb / preview / orig).
type BatchCreateInput struct {
Items []CreateInput `json:"items" validate:"required,min=1,max=3,dive" label:"field.upload_items" errmsg:"required:validate.upload.items_required;min:validate.upload.items_required;max:validate.upload.items_invalid"`
}
func (s *Service) CreateCredentials(ctx context.Context, userID int64, in *BatchCreateInput) (*entity.CredentialSet, error) {
if !s.ready() {
return nil, errs.ErrAvailable
}
if userID <= 0 {
return nil, errs.ErrUnauthorized
}
if in == nil {
return nil, errs.ErrInvalidArgument
}
if err := validate.StructLang(reqctx.Lang(ctx), in); err != nil {
return nil, err
}
if n := len(in.Items); n < 1 || n > maxCredentialBatch {
return nil, errs.ErrInvalidArgument
}
for range in.Items {
if !s.allow(userID) {
return nil, errs.ErrRateLimited
}
}
items := make([]entity.Credential, 0, len(in.Items))
for i := range in.Items {
cred, err := s.issuePut(ctx, userID, &in.Items[i])
if err != nil {
return nil, err
}
items = append(items, *cred)
}
return &entity.CredentialSet{Items: items}, nil
}

View File

@ -2,6 +2,7 @@ package logic
import ( import (
"context" "context"
"strconv"
"spark-server/internal/module/upload/entity" "spark-server/internal/module/upload/entity"
"spark-server/internal/pkg/errs" "spark-server/internal/pkg/errs"
@ -34,14 +35,15 @@ func (s *Service) CreateCredential(ctx context.Context, userID int64, in *Create
if !s.allow(userID) { if !s.allow(userID) {
return nil, errs.ErrRateLimited return nil, errs.ErrRateLimited
} }
return s.issuePut(ctx, userID, in)
}
func (s *Service) issuePut(ctx context.Context, userID int64, in *CreateInput) (*entity.Credential, error) {
_, mime, err := matchSpec(ctx, in) _, mime, err := matchSpec(ctx, in)
if err != nil { if err != nil {
return nil, err return nil, err
} }
objectID, err := s.ids.NextID() name := reqctx.NewObjectName()
if err != nil {
return nil, err
}
ownerID := userID ownerID := userID
if in.Purpose == entity.PurposeGroupAvatar { if in.Purpose == entity.PurposeGroupAvatar {
if err := s.readyGroupAvatar(ctx, userID, in.GroupID); err != nil { if err := s.readyGroupAvatar(ctx, userID, in.GroupID); err != nil {
@ -49,8 +51,8 @@ func (s *Service) CreateCredential(ctx context.Context, userID int64, in *Create
} }
ownerID = in.GroupID ownerID = in.GroupID
} }
key := entity.BuildObjectKey(in.Purpose, ownerID, objectID, dayOf(s.now()), entity.ExtForMIME(mime)) key := entity.BuildObjectKey(in.Purpose, ownerID, name, dayOf(s.now()), entity.ExtForMIME(mime))
url, err := s.store.PresignPut(ctx, key, mime, s.ttl) url, err := s.store.PresignPut(ctx, key, mime, in.Size, s.ttl)
if err != nil { if err != nil {
return nil, err return nil, err
} }
@ -58,7 +60,11 @@ func (s *Service) CreateCredential(ctx context.Context, userID int64, in *Create
ObjectKey: key, ObjectKey: key,
UploadURL: url, UploadURL: url,
Method: entity.MethodPUT, Method: entity.MethodPUT,
Headers: map[string]string{"Content-Type": mime}, Headers: map[string]string{
"Content-Type": mime,
"Content-Length": strconv.FormatInt(in.Size, 10),
"If-None-Match": "*",
},
ExpiresIn: s.expiresIn(), ExpiresIn: s.expiresIn(),
}, nil }, nil
} }

View File

@ -7,7 +7,6 @@ import (
"testing" "testing"
"time" "time"
"spark-server/internal/infrastructure/id"
"spark-server/internal/module/upload/entity" "spark-server/internal/module/upload/entity"
"spark-server/internal/pkg/errs" "spark-server/internal/pkg/errs"
"spark-server/internal/pkg/reqctx" "spark-server/internal/pkg/reqctx"
@ -18,7 +17,7 @@ type fakeStore struct {
putErr error putErr error
} }
func (f *fakeStore) PresignPut(_ context.Context, key, _ string, _ time.Duration) (string, error) { func (f *fakeStore) PresignPut(_ context.Context, key, _ string, _ int64, _ time.Duration) (string, error) {
if f.putErr != nil { if f.putErr != nil {
return "", f.putErr return "", f.putErr
} }
@ -37,11 +36,7 @@ func (f *fakeStore) PresignGet(_ context.Context, key string, _ time.Duration) (
func testUpload(t *testing.T) *Service { func testUpload(t *testing.T) *Service {
t.Helper() t.Helper()
ids, err := id.New(1) return NewService(&fakeStore{}, Options{
if err != nil {
t.Fatal(err)
}
return NewService(&fakeStore{}, ids, Options{
TTL: 5 * time.Minute, TTL: 5 * time.Minute,
Now: func() time.Time { return time.Date(2026, 9, 17, 8, 0, 0, 0, time.UTC) }, Now: func() time.Time { return time.Date(2026, 9, 17, 8, 0, 0, 0, time.UTC) },
}) })
@ -54,11 +49,13 @@ func TestCreateCredentialChatVsAvatar(t *testing.T) {
if err != nil { if err != nil {
t.Fatal(err) t.Fatal(err)
} }
if !strings.HasPrefix(chat.ObjectKey, "chat/20260917/42/") || chat.Method != "PUT" || chat.Headers["Content-Type"] != "image/jpeg" { if !strings.HasPrefix(chat.ObjectKey, "chat/image/20260917/") || chat.Method != "PUT" ||
chat.Headers["Content-Type"] != "image/jpeg" || chat.Headers["Content-Length"] != "1024" ||
chat.Headers["If-None-Match"] != "*" || strings.Contains(chat.ObjectKey, "/42/") {
t.Fatalf("%+v", chat) t.Fatalf("%+v", chat)
} }
avatar, err := svc.CreateCredential(ctx, 42, &CreateInput{Purpose: entity.PurposeAvatar, Mime: "image/png", Size: 2048}) avatar, err := svc.CreateCredential(ctx, 42, &CreateInput{Purpose: entity.PurposeAvatar, Mime: "image/png", Size: 2048})
if err != nil || !strings.HasPrefix(avatar.ObjectKey, "perm/avatar/42/20260917/") { if err != nil || !strings.HasPrefix(avatar.ObjectKey, "perm/avatar/20260917/") {
t.Fatalf("%+v %v", avatar, err) t.Fatalf("%+v %v", avatar, err)
} }
group, err := svc.CreateCredential(ctx, 42, &CreateInput{Purpose: entity.PurposeGroupAvatar, Mime: "image/jpeg", Size: 1024, GroupID: 7}) group, err := svc.CreateCredential(ctx, 42, &CreateInput{Purpose: entity.PurposeGroupAvatar, Mime: "image/jpeg", Size: 1024, GroupID: 7})
@ -79,6 +76,11 @@ func TestCreateCredentialRejectsBadMIMEAndSize(t *testing.T) {
if _, err := svc.CreateCredential(ctx, 1, &CreateInput{Purpose: entity.PurposeChatImage, Mime: "image/jpeg", Size: 50 << 20}); err == nil { if _, err := svc.CreateCredential(ctx, 1, &CreateInput{Purpose: entity.PurposeChatImage, Mime: "image/jpeg", Size: 50 << 20}); err == nil {
t.Fatal("huge image") t.Fatal("huge image")
} }
for _, mime := range []string{"image/heic", "image/heif", "image/webp"} {
if _, err := svc.CreateCredential(ctx, 1, &CreateInput{Purpose: entity.PurposeChatImage, Mime: mime, Size: 1024}); err == nil {
t.Fatalf("accepted %s", mime)
}
}
if _, err := svc.CreateCredential(ctx, 0, &CreateInput{Purpose: entity.PurposeChatImage, Mime: "image/jpeg", Size: 1}); !errors.Is(err, errs.ErrUnauthorized) { if _, err := svc.CreateCredential(ctx, 0, &CreateInput{Purpose: entity.PurposeChatImage, Mime: "image/jpeg", Size: 1}); !errors.Is(err, errs.ErrUnauthorized) {
t.Fatalf("uid %v", err) t.Fatalf("uid %v", err)
} }
@ -95,3 +97,24 @@ func TestCreateDownloadValidatesKey(t *testing.T) {
t.Fatal("path escape") t.Fatal("path escape")
} }
} }
func TestCreateCredentialsBatch(t *testing.T) {
svc := testUpload(t)
ctx := reqctx.With(t.Context(), &reqctx.RequestContext{UserID: 42, Language: "zh-CN"})
part := func(size int64) CreateInput {
return CreateInput{Purpose: entity.PurposeChatImage, Mime: "image/jpeg", Size: size}
}
got, err := svc.CreateCredentials(ctx, 42, &BatchCreateInput{Items: []CreateInput{part(1024), part(2048), part(4096)}})
if err != nil || got == nil || len(got.Items) != 3 {
t.Fatalf("%+v %v", got, err)
}
if got.Items[0].ObjectKey == got.Items[1].ObjectKey || got.Items[1].Headers["Content-Length"] != "2048" {
t.Fatalf("%+v", got.Items)
}
if _, err := svc.CreateCredentials(ctx, 42, &BatchCreateInput{}); err == nil {
t.Fatal("empty")
}
if _, err := svc.CreateCredentials(ctx, 42, &BatchCreateInput{Items: []CreateInput{part(1), part(1), part(1), part(1)}}); err == nil {
t.Fatal("too many")
}
}

View File

@ -10,15 +10,10 @@ import (
// Presigner issues short-lived object-storage URLs. Infrastructure implements it. // Presigner issues short-lived object-storage URLs. Infrastructure implements it.
type Presigner interface { type Presigner interface {
PresignPut(ctx context.Context, key, contentType string, ttl time.Duration) (string, error) PresignPut(ctx context.Context, key, contentType string, size int64, ttl time.Duration) (string, error)
PresignGet(ctx context.Context, key string, ttl time.Duration) (string, error) PresignGet(ctx context.Context, key string, ttl time.Duration) (string, error)
} }
type IDGen interface {
NextID() (int64, error)
}
// AvatarACL gates group_avatar uploads. Nil skips the check (tests).
type AvatarACL interface { type AvatarACL interface {
CanSetAvatar(ctx context.Context, userID, groupID int64) error CanSetAvatar(ctx context.Context, userID, groupID int64) error
} }
@ -33,14 +28,13 @@ type Options struct {
// Service issues upload/download credentials. HTTP and TCP share it. // Service issues upload/download credentials. HTTP and TCP share it.
type Service struct { type Service struct {
store Presigner store Presigner
ids IDGen
ttl time.Duration ttl time.Duration
limit *ratelimit.Limiter limit *ratelimit.Limiter
now func() time.Time now func() time.Time
groups AvatarACL groups AvatarACL
} }
func NewService(store Presigner, ids IDGen, opt Options) *Service { func NewService(store Presigner, opt Options) *Service {
ttl := opt.TTL ttl := opt.TTL
if ttl <= 0 { if ttl <= 0 {
ttl = 15 * time.Minute ttl = 15 * time.Minute
@ -49,11 +43,11 @@ func NewService(store Presigner, ids IDGen, opt Options) *Service {
if clock == nil { if clock == nil {
clock = func() time.Time { return time.Now().UTC() } clock = func() time.Time { return time.Now().UTC() }
} }
return &Service{store: store, ids: ids, ttl: ttl, limit: opt.Limit, now: clock, groups: opt.Groups} return &Service{store: store, ttl: ttl, limit: opt.Limit, now: clock, groups: opt.Groups}
} }
func (s *Service) ready() bool { func (s *Service) ready() bool {
return s != nil && s.store != nil && s.ids != nil return s != nil && s.store != nil
} }
func (s *Service) expiresIn() int64 { func (s *Service) expiresIn() int64 {

View File

@ -7,6 +7,7 @@ import (
uploadentity "spark-server/internal/module/upload/entity" uploadentity "spark-server/internal/module/upload/entity"
"spark-server/internal/module/user/entity" "spark-server/internal/module/user/entity"
"spark-server/internal/pkg/reqctx"
) )
const defaultAvatarMIME = "image/png" const defaultAvatarMIME = "image/png"
@ -42,18 +43,14 @@ func (s *Service) stampDefaultAvatar(ctx context.Context, u *entity.User) {
} }
func (s *Service) putLetterPNG(ctx context.Context, u *entity.User) string { func (s *Service) putLetterPNG(ctx context.Context, u *entity.User) string {
if s == nil || s.objects == nil || s.ids == nil || u == nil || u.ID <= 0 { if s == nil || s.objects == nil || u == nil || u.ID <= 0 {
return "" return ""
} }
raw, err := entity.LetterAvatarPNG(u.Nickname) raw, err := entity.LetterAvatarPNG(u.Nickname)
if err != nil || len(raw) == 0 { if err != nil || len(raw) == 0 {
return "" return ""
} }
objectID, err := s.ids.NextID() key := uploadentity.BuildObjectKey(uploadentity.PurposeAvatar, u.ID, reqctx.NewObjectName(), time.Now().UTC().Format("20060102"), "png")
if err != nil {
return ""
}
key := uploadentity.BuildObjectKey(uploadentity.PurposeAvatar, u.ID, objectID, time.Now().Format("20060102"), "png")
if err := s.objects.Put(ctx, key, raw, defaultAvatarMIME); err != nil { if err := s.objects.Put(ctx, key, raw, defaultAvatarMIME); err != nil {
return "" return ""
} }

View File

@ -27,10 +27,15 @@ func TestAuthenticateTestEmailAnyCode(t *testing.T) {
if u1.ID <= 0 || !entity.ValidSparkID(u1.SparkID) || u1.Nickname != "test01" || !entity.ValidQRCode(u1.QRCode) { if u1.ID <= 0 || !entity.ValidSparkID(u1.SparkID) || u1.Nickname != "test01" || !entity.ValidQRCode(u1.QRCode) {
t.Fatalf("%+v", u1) t.Fatalf("%+v", u1)
} }
prefix := "perm/avatar/" + strconv.FormatInt(u1.ID, 10) + "/" if !strings.HasPrefix(u1.Avatar, "perm/avatar/") || !strings.HasSuffix(u1.Avatar, ".png") {
if !strings.HasPrefix(u1.Avatar, prefix) || !strings.HasSuffix(u1.Avatar, ".png") {
t.Fatalf("default avatar %s", u1.Avatar) t.Fatalf("default avatar %s", u1.Avatar)
} }
uid := strconv.FormatInt(u1.ID, 10)
for _, part := range strings.Split(u1.Avatar, "/") {
if part == uid {
t.Fatalf("avatar leaked uid %s", u1.Avatar)
}
}
store, ok := svc.objects.(*memObjects) store, ok := svc.objects.(*memObjects)
if !ok { if !ok {
t.Fatal("mem objects") t.Fatal("mem objects")

View File

@ -11,6 +11,9 @@ func init() {
KeyFieldSize: { KeyFieldSize: {
ZhCN: "文件大小", En: "File size", ZhTW: "檔案大小", ZhCN: "文件大小", En: "File size", ZhTW: "檔案大小",
}, },
KeyFieldUploadItems: {
ZhCN: "上传列表", En: "Upload list", ZhTW: "上傳列表",
},
KeyFieldObjectKey: { KeyFieldObjectKey: {
ZhCN: "对象键", En: "Object key", ZhTW: "物件鍵", ZhCN: "对象键", En: "Object key", ZhTW: "物件鍵",
}, },
@ -50,6 +53,12 @@ func init() {
KeyUploadGroupRequired: { KeyUploadGroupRequired: {
ZhCN: "请选择群", En: "Please choose a group", ZhTW: "請選擇群", ZhCN: "请选择群", En: "Please choose a group", ZhTW: "請選擇群",
}, },
KeyUploadItemsRequired: {
ZhCN: "请填写要上传的文件", En: "Please provide files to upload", ZhTW: "請填寫要上傳的檔案",
},
KeyUploadItemsInvalid: {
ZhCN: "一次最多签发 3 个上传凭证", En: "At most 3 upload credentials per request", ZhTW: "一次最多簽發 3 個上傳憑證",
},
KeyFavoriteMsgRequired: { KeyFavoriteMsgRequired: {
ZhCN: "请选择消息", En: "Please choose a message", ZhTW: "請選擇訊息", ZhCN: "请选择消息", En: "Please choose a message", ZhTW: "請選擇訊息",
}, },

View File

@ -42,6 +42,7 @@ const (
KeyFieldPurpose = "field.purpose" KeyFieldPurpose = "field.purpose"
KeyFieldMime = "field.mime" KeyFieldMime = "field.mime"
KeyFieldSize = "field.size" KeyFieldSize = "field.size"
KeyFieldUploadItems = "field.upload_items"
KeyFieldObjectKey = "field.object_key" KeyFieldObjectKey = "field.object_key"
KeyFieldMsgID = "field.msg_id" KeyFieldMsgID = "field.msg_id"
KeyFieldClientMsgID = "field.client_msg_id" KeyFieldClientMsgID = "field.client_msg_id"
@ -90,6 +91,8 @@ const (
KeyUploadKeyRequired = "validate.upload.key_required" KeyUploadKeyRequired = "validate.upload.key_required"
KeyUploadKeyInvalid = "validate.upload.key_invalid" KeyUploadKeyInvalid = "validate.upload.key_invalid"
KeyUploadGroupRequired = "validate.upload.group_required" KeyUploadGroupRequired = "validate.upload.group_required"
KeyUploadItemsRequired = "validate.upload.items_required"
KeyUploadItemsInvalid = "validate.upload.items_invalid"
KeyFavoriteMsgRequired = "validate.favorite.msg_required" KeyFavoriteMsgRequired = "validate.favorite.msg_required"
KeyFavoriteMsgInvalid = "validate.favorite.msg_invalid" KeyFavoriteMsgInvalid = "validate.favorite.msg_invalid"
KeyMessageClientMsgRequired = "validate.message.client_msg_required" KeyMessageClientMsgRequired = "validate.message.client_msg_required"

View File

@ -3,7 +3,8 @@ package reqctx
import ( import (
"context" "context"
"crypto/rand" "crypto/rand"
"encoding/hex" "encoding/binary"
"math/big"
"net" "net"
"strings" "strings"
"time" "time"
@ -91,11 +92,51 @@ func stripHost(v string) string {
return v return v
} }
// NewID returns a 16-byte hex id for request_id. const (
base62Alphabet = "0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz"
base62Len = 11
objectNameLen = 16
objectNameSrc = 12
)
// NewID returns an 8-byte base62 id (11 chars) for request_id and JWT jti.
func NewID() string { func NewID() string {
var b [16]byte var b [8]byte
if _, err := rand.Read(b[:]); err != nil { if _, err := rand.Read(b[:]); err != nil {
return "00000000000000000000000000000000" return "00000000000"
} }
return hex.EncodeToString(b[:]) return encodeBase62(binary.BigEndian.Uint64(b[:]))
}
// NewObjectName returns a 12-byte base62 id (16 chars) for object_key filenames. Not Snowflake.
func NewObjectName() string {
return randomBase62(objectNameSrc, objectNameLen)
}
func encodeBase62(n uint64) string {
var out [base62Len]byte
for i := base62Len - 1; i >= 0; i-- {
out[i] = base62Alphabet[n%62]
n /= 62
}
return string(out[:])
}
func randomBase62(byteN, width int) string {
if byteN < 1 || width < 1 {
return ""
}
b := make([]byte, byteN)
if _, err := rand.Read(b); err != nil {
return strings.Repeat("0", width)
}
n := new(big.Int).SetBytes(b)
out := make([]byte, width)
base := big.NewInt(62)
r := new(big.Int)
for i := width - 1; i >= 0; i-- {
n.DivMod(n, base, r)
out[i] = base62Alphabet[r.Int64()]
}
return string(out)
} }

View File

@ -1,6 +1,9 @@
package reqctx package reqctx
import "testing" import (
"strings"
"testing"
)
func TestParseHTTPRealIP(t *testing.T) { func TestParseHTTPRealIP(t *testing.T) {
got := ParseHTTPRealIP(" 203.0.113.10 ", "198.51.100.1, 192.0.2.1", "192.0.2.1:1234") got := ParseHTTPRealIP(" 203.0.113.10 ", "198.51.100.1, 192.0.2.1", "192.0.2.1:1234")
@ -27,6 +30,48 @@ func TestRealIPFromContext(t *testing.T) {
} }
} }
func TestNewIDLength(t *testing.T) {
id := NewID()
if len(id) != 11 {
t.Fatalf("len %d %q", len(id), id)
}
for i := 0; i < len(id); i++ {
if !strings.ContainsRune(base62Alphabet, rune(id[i])) {
t.Fatalf("char %q in %q", id[i], id)
}
}
if id == NewID() {
t.Fatal("not unique")
}
}
func TestNewObjectNameLength(t *testing.T) {
id := NewObjectName()
if len(id) != 16 {
t.Fatalf("len %d %q", len(id), id)
}
for i := 0; i < len(id); i++ {
if !strings.ContainsRune(base62Alphabet, rune(id[i])) {
t.Fatalf("char %q in %q", id[i], id)
}
}
if id == NewObjectName() {
t.Fatal("not unique")
}
}
func TestEncodeBase62(t *testing.T) {
if got := encodeBase62(0); got != "00000000000" {
t.Fatalf("%q", got)
}
if got := encodeBase62(61); got != "0000000000z" {
t.Fatalf("%q", got)
}
if got := encodeBase62(62); got != "00000000010" {
t.Fatalf("%q", got)
}
}
func TestRequestIDFromContext(t *testing.T) { func TestRequestIDFromContext(t *testing.T) {
ctx := With(t.Context(), &RequestContext{RequestID: "abc"}) ctx := With(t.Context(), &RequestContext{RequestID: "abc"})
if RequestID(ctx) != "abc" { if RequestID(ctx) != "abc" {

View File

@ -7,7 +7,7 @@ info:
## 环境与地址 ## 环境与地址
公网只开放 `443`。Nginx 按 TLS SNI 分流:HTTP 证书打到 `sparkapi.wyrs365.com`,IM 证书打到 `sparkim.wyrs365.com`。进程本身只绑本机。 公网只开放 `443`。Nginx 按 TLS SNI 分流:HTTP 证书打到 `sparkapi.wyrs365.com`,IM 证书打到 `sparkim.wyrs365.com`,对象存储(RustFS S3)打到 `sparkr.wyrs365.com`。进程本身只绑本机。
| 环境 | 用途 | HTTP Base URL | IM 长连接 | | 环境 | 用途 | HTTP Base URL | IM 长连接 |
| --- | --- | --- | --- | | --- | --- | --- | --- |
@ -36,7 +36,7 @@ info:
| --- | --- | --- | | --- | --- | --- |
| `code` | **字符串** | `"0"` 成功;其它失败。不要用数字 `0` 比较。全表见 [错误码](/docs/errors)。 | | `code` | **字符串** | `"0"` 成功;其它失败。不要用数字 `0` 比较。全表见 [错误码](/docs/errors)。 |
| `msg` | 字符串 | 给人看的文案,随请求语言变化(`zh-CN` / `en` / `zh-TW`)。成功一般为 `ok`。 | | `msg` | 字符串 | 给人看的文案,随请求语言变化(`zh-CN` / `en` / `zh-TW`)。成功一般为 `ok`。 |
| `request_id` | 字符串 | 服务端生成的本次请求 id。客户端不用传;从响应体读取。与 Header `X-Request-ID` 相同。 | | `request_id` | 字符串 | 服务端生成,11 位 base62(`0-9A-Za-z`)。客户端不用传;从响应体读取。与 Header `X-Request-ID` 相同。 |
| `data` | object / array | 成功时为业务对象;失败时固定 `{}`。字段缺失当空对象,不要当 `null`。 | | `data` | object / array | 成功时为业务对象;失败时固定 `{}`。字段缺失当空对象,不要当 `null`。 |
**判定成功只看 `code === "0"`。** 不要只看 HTTP 状态码。 **判定成功只看 `code === "0"`。** 不要只看 HTTP 状态码。
@ -655,7 +655,7 @@ paths:
uid: 1940000000000000000 uid: 1940000000000000000
spark_id: alice_ok spark_id: alice_ok
nickname: test01 nickname: test01
avatar: perm/avatar/1940000000000000000/20260917/1940000000000000001.png avatar: perm/avatar/20260917/3H8kPq2mN1Rabcde.png
qrcode: "-NDveu-9Q" qrcode: "-NDveu-9Q"
birthday: "" birthday: ""
gender: 0 gender: 0
@ -1106,9 +1106,13 @@ paths:
description: | description: |
客户端只传 `purpose` / `mime` / `size`,服务端生成 object_key 并签发短时 Presigned PUT。 客户端只传 `purpose` / `mime` / `size`,服务端生成 object_key 并签发短时 Presigned PUT。
文件直传 RustFS,不经过 Gin。 文件直传 RustFS,不经过 Gin。
`avatar` 写入 `perm/avatar/{uid}/…`(不过期);`group_avatar` 写入 `perm/group/{群号}/…`(需 `group_id`,群主/管理员); `avatar` 写入 `perm/avatar/{yyyyMMdd}/{name}.{ext}`(不过期,路径不含 uid);
聊天媒体写入 `chat/{yyyyMMdd}/{uid}/…`(桶生命周期过期)。 `group_avatar` 写入 `perm/group/{群号}/{name}.{ext}`(需 `group_id`,群主/管理员);
客户端 PUT 时必须带返回的 `headers`(至少 `Content-Type`)。消息与头像只存 `object_key`。 聊天按类型:`chat/image|voice|video|gif|file/{yyyyMMdd}/{name}.{ext}`(桶生命周期按前缀过期)。
`{name}` 为服务端 16 位 base62 随机串,不占用 Snowflake。
发图片:客户端压成 JPEG/PNG 后 `POST /v1/files/batch` 直传缩略图、压缩大图、原图(未勾选原图只传前两项)。
服务端不转码。iOS HEIC、Android WebP 须客户端先转 JPEG/PNG,两端才能互看。
客户端 PUT 必须原样带返回的 `headers`。消息只存 object_key。
operationId: createUploadCredential operationId: createUploadCredential
requestBody: requestBody:
required: true required: true
@ -1131,12 +1135,61 @@ paths:
code: "0" code: "0"
msg: ok msg: ok
data: data:
object_key: chat/20260917/1/2.jpg object_key: chat/image/20260917/3H8kPq2mN1Rabcde.jpg
upload_url: http://47.236.66.9:9000/spark/chat/20260917/1/2.jpg?X-Amz-Algorithm=AWS4-HMAC-SHA256 upload_url: https://sparkr.wyrs365.com/spark/chat/image/20260917/3H8kPq2mN1Rabcde.jpg?X-Amz-Algorithm=AWS4-HMAC-SHA256
method: PUT method: PUT
headers: headers:
Content-Type: image/jpeg Content-Type: image/jpeg
Content-Length: "245760"
If-None-Match: "*"
expires_in: 900 expires_in: 900
/v1/files/batch:
post:
tags: [files]
summary: 批量签发直传凭证
description: |
一次返回 1~3 个 Presigned PUT,顺序与 `items` 相同。聊天图片请按「缩略图、压缩大图、原图」排列;未勾选原图只传前两项。
mime 仅 `image/jpeg` / `image/png`(HEIC、WebP 请客户端先转)。限流按条数计入 `upload_per_second`。文件仍直传 RustFS。
operationId: createUploadCredentials
requestBody:
required: true
content:
application/json:
schema:
$ref: "#/components/schemas/CreateUploadBatchRequest"
example:
items:
- purpose: chat_image
mime: image/jpeg
size: 8192
- purpose: chat_image
mime: image/jpeg
size: 245760
- purpose: chat_image
mime: image/jpeg
size: 3500000
responses:
"200":
description: 统一 envelope
content:
application/json:
schema:
$ref: "#/components/schemas/Envelope"
example:
code: "0"
msg: ok
data:
items:
- object_key: chat/image/20260917/3H8kPq2mN1Rabcde.jpg
method: PUT
headers:
Content-Type: image/jpeg
Content-Length: "8192"
If-None-Match: "*"
- object_key: chat/image/20260917/4kPq2mN1RabcdeXY.jpg
method: PUT
- object_key: chat/image/20260917/5mN1RabcdeXY12ab.jpg
method: PUT
/v1/files/downloads: /v1/files/downloads:
post: post:
tags: [files] tags: [files]
@ -1181,7 +1234,7 @@ paths:
tags: [favorites] tags: [favorites]
summary: 收藏一条消息 summary: 收藏一条消息
description: | description: |
会话成员可将消息收入收藏。图片/语音/视频/文件会 CopyObject 到 `perm/fav/{uid}/…`。 会话成员可将消息收入收藏。图片/语音/视频/文件会 CopyObject 到 `perm/fav/{yyyyMMdd}/{name}.{ext}`。
源对象已从 `chat/` 过期删除则失败(`code=40402`),不提供本地补传。 源对象已从 `chat/` 过期删除则失败(`code=40402`),不提供本地补传。
同一 `msg_id` 重复收藏返回已有记录。撤回不影响已收藏快照。 同一 `msg_id` 重复收藏返回已有记录。撤回不影响已收藏快照。
operationId: createFavorite operationId: createFavorite
@ -1477,8 +1530,8 @@ components:
example: ok example: ok
request_id: request_id:
type: string type: string
description: 服务端生成的本次请求 id。客户端不用传,从响应体读取。 description: 服务端生成的本次请求 id(11 位 base62,`0-9A-Za-z`)。客户端不用传,从响应体读取。
example: a1b2c3d4e5f6a7b8a1b2c3d4e5f6a7b8 example: 3H8kPq2mN1R
data: data:
type: object type: object
additionalProperties: true additionalProperties: true
@ -1743,6 +1796,9 @@ components:
mime: mime:
type: string type: string
example: image/jpeg example: image/jpeg
description: |
avatar / group_avatar / chat_image 仅 image/jpeg、image/jpg、image/png。
iOS HEIC、Android WebP 须客户端转成 JPEG/PNG 再申请凭证。
size: size:
type: integer type: integer
format: int64 format: int64
@ -1752,6 +1808,16 @@ components:
format: int64 format: int64
minimum: 1 minimum: 1
description: purpose=group_avatar 时必填,为自增群号 description: purpose=group_avatar 时必填,为自增群号
CreateUploadBatchRequest:
type: object
required: [items]
properties:
items:
type: array
minItems: 1
maxItems: 3
items:
$ref: "#/components/schemas/CreateUploadRequest"
CreateDownloadRequest: CreateDownloadRequest:
type: object type: object
required: [object_key] required: [object_key]
@ -1792,6 +1858,39 @@ components:
properties: properties:
body: body:
type: string type: string
image:
type: object
description: |
t=2。点开看 `key`(压缩预览);气泡用 `thumb_key`;`orig_key` 有则显示「查看原图」。
三档由客户端压成 JPEG/PNG 后 `POST /v1/files/batch` 直传。服务端不转码;不要传 HEIC/WebP。
required: [key]
properties:
key:
type: string
description: 点开查看用的压缩图 object_key
thumb_key:
type: string
description: 气泡缩略图,可省略
orig_key:
type: string
description: 原图 object_key;空则没有「查看原图」
orig_size:
type: integer
format: int64
description: 原图像素文件大小,供「查看原图 3.2MB」按钮
orig_w:
type: integer
orig_h:
type: integer
mime:
type: string
w:
type: integer
h:
type: integer
size:
type: integer
format: int64
at_all: at_all:
type: boolean type: boolean
description: "@所有人,仅群主/管理员" description: "@所有人,仅群主/管理员"

View File

@ -27,11 +27,11 @@ import (
type stubPresign struct{} type stubPresign struct{}
func (stubPresign) PresignPut(_ context.Context, key, _ string, _ time.Duration) (string, error) { func (s *stubPresign) PresignPut(_ context.Context, key, _ string, _ int64, _ time.Duration) (string, error) {
return "http://rustfs.test/" + key + "?sig=1", nil return "http://rustfs.test/" + key + "?sig=1", nil
} }
func (stubPresign) PresignGet(_ context.Context, key string, _ time.Duration) (string, error) { func (s *stubPresign) PresignGet(_ context.Context, key string, _ time.Duration) (string, error) {
return "http://rustfs.test/" + key + "?get=1", nil return "http://rustfs.test/" + key + "?get=1", nil
} }
@ -48,7 +48,7 @@ func testFileEngine(t *testing.T) (*ginEngineBundle, *id.Generator, *msgmodel.Me
users := userlogic.NewService(usermodel.NewMemoryRepo(), usermodel.NewMemoryCodes(), ids, nil) users := userlogic.NewService(usermodel.NewMemoryRepo(), usermodel.NewMemoryCodes(), ids, nil)
msgs := msgmodel.NewMemoryRepo() msgs := msgmodel.NewMemoryRepo()
convs := convmodel.NewMemoryP2P() convs := convmodel.NewMemoryP2P()
files := uploadlogic.NewService(stubPresign{}, ids, uploadlogic.Options{ files := uploadlogic.NewService(&stubPresign{}, uploadlogic.Options{
Now: func() time.Time { return time.Date(2026, 9, 17, 0, 0, 0, 0, time.UTC) }, Now: func() time.Time { return time.Date(2026, 9, 17, 0, 0, 0, 0, time.UTC) },
}) })
favs := favlogic.NewService(favmodel.NewMemoryRepo(), msgs, convs, nil, ids) favs := favlogic.NewService(favmodel.NewMemoryRepo(), msgs, convs, nil, ids)
@ -71,13 +71,25 @@ func TestUploadCredentialHTTP(t *testing.T) {
ObjectKey string `json:"object_key"` ObjectKey string `json:"object_key"`
Method string `json:"method"` Method string `json:"method"`
} }
if err := json.Unmarshal(env.Data, &cred); err != nil || cred.Method != "PUT" || !strings.Contains(cred.ObjectKey, "chat/20260917/") { if err := json.Unmarshal(env.Data, &cred); err != nil || cred.Method != "PUT" || !strings.Contains(cred.ObjectKey, "chat/image/20260917/") {
t.Fatalf("%s %v", env.Data, err) t.Fatalf("%s %v", env.Data, err)
} }
bad := authJSON(t, h, &tok, http.MethodPost, "/v1/files", `{"purpose":"avatar","mime":"application/pdf","size":10}`) bad := authJSON(t, h, &tok, http.MethodPost, "/v1/files", `{"purpose":"avatar","mime":"application/pdf","size":10}`)
if bad.Code != errs.ErrInvalidArgument.Code { if bad.Code != errs.ErrInvalidArgument.Code {
t.Fatalf("mime %+v", bad) t.Fatalf("mime %+v", bad)
} }
batch := authJSON(t, h, &tok, http.MethodPost, "/v1/files/batch", `{"items":[{"purpose":"chat_image","mime":"image/jpeg","size":1024},{"purpose":"chat_image","mime":"image/jpeg","size":2048}]}`)
if batch.Code != "0" {
t.Fatalf("batch %+v", batch)
}
var set struct {
Items []struct {
ObjectKey string `json:"object_key"`
} `json:"items"`
}
if err := json.Unmarshal(batch.Data, &set); err != nil || len(set.Items) != 2 || set.Items[0].ObjectKey == set.Items[1].ObjectKey {
t.Fatalf("%s %v", batch.Data, err)
}
} }
func TestFavoriteTextHTTP(t *testing.T) { func TestFavoriteTextHTTP(t *testing.T) {

View File

@ -36,6 +36,24 @@ func (h *Handler) Create(c *gin.Context) {
respond.JSON(c, out) respond.JSON(c, out)
} }
func (h *Handler) CreateBatch(c *gin.Context) {
uid, ok := currentUID(c)
if !ok {
return
}
var in uploadlogic.BatchCreateInput
if err := respond.BindJSON(c, &in); err != nil {
respond.JSONError(c, err)
return
}
out, err := h.files.CreateCredentials(c.Request.Context(), uid, &in)
if err != nil {
respond.JSONError(c, err)
return
}
respond.JSON(c, out)
}
func (h *Handler) Download(c *gin.Context) { func (h *Handler) Download(c *gin.Context) {
uid, ok := currentUID(c) uid, ok := currentUID(c)
if !ok { if !ok {

View File

@ -132,6 +132,7 @@ func registerFileRoutes(v1 *gin.RouterGroup, deps *EngineDeps) {
h := filehandler.New(deps.Files) h := filehandler.New(deps.Files)
g := v1.Group("/files") g := v1.Group("/files")
g.POST("", h.Create) g.POST("", h.Create)
g.POST("/batch", h.CreateBatch)
g.POST("/downloads", h.Download) g.POST("/downloads", h.Download)
} }

View File

@ -34,7 +34,7 @@ func TestHealthzEnvelopeAndTraceHeaders(t *testing.T) {
t.Fatalf("status %d", w.Code) t.Fatalf("status %d", w.Code)
} }
got := w.Header().Get("X-Request-ID") got := w.Header().Get("X-Request-ID")
if len(got) != 32 { if len(got) != 11 {
t.Fatalf("server request id %q headers %v", got, w.Header()) t.Fatalf("server request id %q headers %v", got, w.Header())
} }
if w.Header().Get("X-Trace-ID") != "" { if w.Header().Get("X-Trace-ID") != "" {
@ -47,7 +47,7 @@ func TestHealthzEnvelopeAndTraceHeaders(t *testing.T) {
if env.Code != "0" || env.Msg != "ok" { if env.Code != "0" || env.Msg != "ok" {
t.Fatalf("%+v", env) t.Fatalf("%+v", env)
} }
if env.RequestID != got || len(env.RequestID) != 32 { if env.RequestID != got || len(env.RequestID) != 11 {
t.Fatalf("body request_id %q header %q", env.RequestID, got) t.Fatalf("body request_id %q header %q", env.RequestID, got)
} }
if !bytes.Contains(env.Data, []byte(`"status":"ok"`)) { if !bytes.Contains(env.Data, []byte(`"status":"ok"`)) {
@ -82,7 +82,7 @@ func TestReadyzHTTPStatus(t *testing.T) {
if env.Code != errs.ErrAvailable.Code || !bytes.Contains(env.Data, []byte(`"mysql":"down"`)) { if env.Code != errs.ErrAvailable.Code || !bytes.Contains(env.Data, []byte(`"mysql":"down"`)) {
t.Fatalf("%+v", env) t.Fatalf("%+v", env)
} }
if len(env.RequestID) != 32 { if len(env.RequestID) != 11 {
t.Fatalf("readyz request_id %q", env.RequestID) t.Fatalf("readyz request_id %q", env.RequestID)
} }
} }