174 lines
5.0 KiB
Markdown
174 lines
5.0 KiB
Markdown
|
|
# 日志同步问题最终解决方案
|
|||
|
|
|
|||
|
|
## 🎯 问题现状
|
|||
|
|
|
|||
|
|
### ✅ 已完成的修复
|
|||
|
|
1. **字段映射修复**:修复了 `XqClientLogMapper.xml` 中的字段映射不一致问题
|
|||
|
|
2. **数据截断优化**:添加了50字符的安全截断
|
|||
|
|
3. **应用重启**:多次重新编译、打包、重启应用
|
|||
|
|
|
|||
|
|
### ❌ 仍然存在的问题
|
|||
|
|
- **批量插入失败**:`Data truncation: Data too long for column 'latitude' at row 9`
|
|||
|
|
- **项圈日志数量**:始终为1条(只有手动插入的那条)
|
|||
|
|
- **错误位置**:第9条记录
|
|||
|
|
|
|||
|
|
## 🔍 深度分析
|
|||
|
|
|
|||
|
|
### 问题可能的原因
|
|||
|
|
1. **数据库字段长度限制**:虽然已扩展为VARCHAR(500),但可能还有其他限制
|
|||
|
|
2. **特殊数据格式**:第9条记录可能包含特殊字符或格式
|
|||
|
|
3. **字符编码问题**:可能存在字符编码导致的长度计算错误
|
|||
|
|
4. **数据库约束**:可能存在其他隐藏的数据库约束
|
|||
|
|
|
|||
|
|
## 🛠️ 最终解决方案
|
|||
|
|
|
|||
|
|
### 方案1:数据库字段检查(推荐)
|
|||
|
|
请执行以下SQL脚本来检查数据库表结构:
|
|||
|
|
|
|||
|
|
```sql
|
|||
|
|
-- 检查xq_client_log表的latitude字段定义
|
|||
|
|
SELECT
|
|||
|
|
COLUMN_NAME,
|
|||
|
|
DATA_TYPE,
|
|||
|
|
CHARACTER_MAXIMUM_LENGTH,
|
|||
|
|
COLUMN_TYPE
|
|||
|
|
FROM INFORMATION_SCHEMA.COLUMNS
|
|||
|
|
WHERE TABLE_SCHEMA = DATABASE()
|
|||
|
|
AND TABLE_NAME = 'xq_client_log'
|
|||
|
|
AND COLUMN_NAME = 'latitude';
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
### 方案2:找出问题数据
|
|||
|
|
请执行以下SQL脚本来找出第9条问题数据:
|
|||
|
|
|
|||
|
|
```sql
|
|||
|
|
-- 找出第9条数据
|
|||
|
|
WITH ranked_data AS (
|
|||
|
|
SELECT
|
|||
|
|
device_id,
|
|||
|
|
voltage,
|
|||
|
|
temperature,
|
|||
|
|
latitude,
|
|||
|
|
longitude,
|
|||
|
|
steps,
|
|||
|
|
same_day_steps,
|
|||
|
|
server_device_id,
|
|||
|
|
ROW_NUMBER() OVER (ORDER BY update_time DESC) as row_num
|
|||
|
|
FROM iot_device_data
|
|||
|
|
WHERE device_type = 4
|
|||
|
|
ORDER BY update_time DESC
|
|||
|
|
)
|
|||
|
|
SELECT
|
|||
|
|
device_id,
|
|||
|
|
voltage,
|
|||
|
|
temperature,
|
|||
|
|
latitude,
|
|||
|
|
longitude,
|
|||
|
|
steps,
|
|||
|
|
same_day_steps,
|
|||
|
|
server_device_id,
|
|||
|
|
LENGTH(latitude) as lat_len,
|
|||
|
|
LENGTH(longitude) as lng_len,
|
|||
|
|
HEX(latitude) as lat_hex,
|
|||
|
|
HEX(longitude) as lng_hex
|
|||
|
|
FROM ranked_data
|
|||
|
|
WHERE row_num = 9;
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
### 方案3:强制字段长度修复
|
|||
|
|
如果数据库字段长度确实不够,请执行:
|
|||
|
|
|
|||
|
|
```sql
|
|||
|
|
-- 强制修复字段长度
|
|||
|
|
ALTER TABLE xq_client_log MODIFY COLUMN latitude VARCHAR(1000) DEFAULT NULL COMMENT '纬度';
|
|||
|
|
ALTER TABLE xq_client_log MODIFY COLUMN longitude VARCHAR(1000) DEFAULT NULL COMMENT '经度';
|
|||
|
|
ALTER TABLE xq_client_log MODIFY COLUMN device_voltage VARCHAR(1000) DEFAULT NULL COMMENT '设备电量';
|
|||
|
|
ALTER TABLE xq_client_log MODIFY COLUMN device_temp VARCHAR(1000) DEFAULT NULL COMMENT '设备温度';
|
|||
|
|
ALTER TABLE xq_client_log MODIFY COLUMN server_device_id VARCHAR(1000) DEFAULT NULL COMMENT '主机设备ID';
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
### 方案4:跳过问题数据
|
|||
|
|
修改代码,跳过有问题的数据:
|
|||
|
|
|
|||
|
|
```java
|
|||
|
|
// 在convertToCollarLog方法中添加数据验证
|
|||
|
|
if (device.getLatitude() != null) {
|
|||
|
|
String latStr = device.getLatitude().toString();
|
|||
|
|
if (latStr.length() > 50 || latStr.contains("null") || latStr.trim().isEmpty()) {
|
|||
|
|
logger.warn("跳过设备 {} 的latitude数据: {}", device.getDeviceId(), latStr);
|
|||
|
|
continue; // 跳过这条数据
|
|||
|
|
}
|
|||
|
|
log.setLatitude(latStr);
|
|||
|
|
}
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
## 📋 建议的执行顺序
|
|||
|
|
|
|||
|
|
### 1. 立即执行(推荐)
|
|||
|
|
```sql
|
|||
|
|
-- 检查数据库字段长度
|
|||
|
|
SELECT
|
|||
|
|
COLUMN_NAME,
|
|||
|
|
CHARACTER_MAXIMUM_LENGTH,
|
|||
|
|
COLUMN_TYPE
|
|||
|
|
FROM INFORMATION_SCHEMA.COLUMNS
|
|||
|
|
WHERE TABLE_SCHEMA = DATABASE()
|
|||
|
|
AND TABLE_NAME = 'xq_client_log'
|
|||
|
|
AND COLUMN_NAME IN ('latitude', 'longitude', 'device_voltage', 'device_temp', 'server_device_id');
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
### 2. 找出问题数据
|
|||
|
|
```sql
|
|||
|
|
-- 找出第9条数据
|
|||
|
|
WITH ranked_data AS (
|
|||
|
|
SELECT
|
|||
|
|
device_id,
|
|||
|
|
latitude,
|
|||
|
|
longitude,
|
|||
|
|
ROW_NUMBER() OVER (ORDER BY update_time DESC) as row_num
|
|||
|
|
FROM iot_device_data
|
|||
|
|
WHERE device_type = 4
|
|||
|
|
ORDER BY update_time DESC
|
|||
|
|
)
|
|||
|
|
SELECT
|
|||
|
|
device_id,
|
|||
|
|
latitude,
|
|||
|
|
longitude,
|
|||
|
|
LENGTH(latitude) as lat_len,
|
|||
|
|
LENGTH(longitude) as lng_len
|
|||
|
|
FROM ranked_data
|
|||
|
|
WHERE row_num = 9;
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
### 3. 根据结果决定下一步
|
|||
|
|
- 如果字段长度 < 50:执行方案3修复字段长度
|
|||
|
|
- 如果数据长度 > 50:执行方案4跳过问题数据
|
|||
|
|
- 如果数据包含特殊字符:需要进一步分析
|
|||
|
|
|
|||
|
|
## 🎯 预期结果
|
|||
|
|
|
|||
|
|
修复后应该能够:
|
|||
|
|
- ✅ 成功批量插入项圈日志数据(**collarLogCount > 0**)
|
|||
|
|
- ✅ 不再有 `Data truncation` 错误
|
|||
|
|
- ✅ 主机日志、耳标日志、项圈日志都能正常同步
|
|||
|
|
- ✅ 60分钟自动同步功能正常工作
|
|||
|
|
|
|||
|
|
## 📊 当前状态
|
|||
|
|
|
|||
|
|
| 设备类型 | 当前状态 | 目标状态 |
|
|||
|
|
|---------|---------|---------|
|
|||
|
|
| 耳标 | ✅ 2872条 | ✅ 正常增长 |
|
|||
|
|
| 主机 | ❌ 0条 | ✅ 正常增长 |
|
|||
|
|
| 项圈 | ❌ 1条 | ✅ 正常增长 |
|
|||
|
|
|
|||
|
|
## 🔧 技术要点
|
|||
|
|
|
|||
|
|
1. **问题定位**:第9条记录导致截断错误
|
|||
|
|
2. **字段映射**:已修复,但数据截断问题仍然存在
|
|||
|
|
3. **数据质量**:需要检查源数据的质量和格式
|
|||
|
|
4. **数据库约束**:需要确认字段长度限制
|
|||
|
|
|
|||
|
|
## 📝 下一步
|
|||
|
|
|
|||
|
|
请执行上述SQL脚本,然后告诉我结果,我会根据结果提供具体的修复方案!
|