积分安全交互保障 · 数据回收报告

📅 报告日期: 2026-06-15 🚀 Phase 1 上线: 2026-05-05 (9 家) 🚀 Phase 2 上线: 2026-05-29 (11 家) 📊 数据截至: 2026-06-14 (T+4 订单完整至 6/11)

📋 目录

一、核心指标 KPI 摘要

Phase 1 (9 家积分商家) · 5/5–6/14 累计 41 天

B − A 点击率
−0.37pp
A: 21.58% / B: 21.21%
B − A CB 订单数
−193 (−2.3%)
A: 8,216 / B: 8,023
B − A 佣金
−$1,517 (−19.2%)
A: $7,919 / B: $6,402
B 真实正向商家增量
+$365
tesco/rewe/kohls/allegro

Phase 2 (11 家推荐商家) · 5/29–6/14 累计 17 天

B − A 点击率
−0.01pp
A: 22.11% / B: 22.10%(持平)
B − A CB 订单数
−194 (−17.2%)
A: 1,126 / B: 932
B − A 佣金
−$130 (−18.1%)
A: $718 / B: $589
B 真实正向商家增量
+$25
doordash/moonpig/vistaprint
⚠️ 核心结论: Phase 1 click rate 已衰减并转负(+0.58pp → −0.37pp),佣金大幅负向 −19.2% 主要由 agoda B 桶 enabled=1 命中率仅 25% 拖累(−$1,224)。Phase 2 click 持平,但真实正向集中在 vistaprint/moonpig/shop-apotheke/mediaexpert 等少数商家。整体效应不及 PRD 预期 +3~+8pp。

二、实验背景

9 家积分体系商家(Phase 1) + 11 家扩展商家(Phase 2) 因用户担心激活 CB 影响自家积分/会员权益,click rate 显著低于全局均值。本实验在 icon / CB Popup / FCB 弹窗三处展示"积分不受影响"文案,50/50 AB 灰度。

实验组 B: extra.core_tips_style = '6.19.90-2' AND core_tips_enabled = 1 (真实看到文案)
对照组 A: extra.core_tips_style = '6.19.90-1' (无文案)

9 家 Phase 1 商家: tesco.com / allegro.pl / samsclub.com / kohls.com / target.com / marksandspencer.com / agoda.com / rewe.de / argos.co.uk

11 家 Phase 2 商家: sephora.com / doordash.com / gap.com / diy.com / shop-apotheke.com / ocado.com / moonpig.com / just-eat.co.uk / mediaexpert.pl / vistaprint.com / westlotto.de

三、数据口径与时间窗

说明
曝光表v_dwd_log_parse_extension_service_impress_di
点击表v_dwd_log_parse_extension_service_click_di
激活成功(出站)dwd_log_event_outbound_result_det_d
SP journeydws_log_sp_user_shopping_journey_detail_di
订单表dwd_trd_user_orders_di (bu='CP' AND is_cb_order=1)
场景过滤type='CB' AND scene IN ('Popup','FirstPage','Icon')
Phase 2 版本过滤3 段式版本号 ≥ 6.20.60
订单延迟T+0=4% / T+1=50% / T+4=95% → PRE/POST 避开 3 天内日期
窗口对齐同 DOW + 同长度 + T+4 完整

四、AB 点击层累计

Scope窗口A 曝光A 点击A 率B 曝光B 点击B 率B − A
Phase 1 (9 家)5/5–6/14 (41 天) 55,02311,87421.58% 55,04311,67721.21% −0.37pp
Phase 2 (11 家,版本过滤)5/29–6/14 (17 天) 16,5053,64922.11% 16,3273,60922.10% −0.01pp

五、Phase 1 按商家 click rate (5/5–6/14)

商家A 曝光A 点击A 率B 曝光B 点击B 率B − A
agoda.com4,8761,12223.01%5,0451,17423.27%+0.26
target.com10,9622,02518.47%11,0352,08918.93%+0.46
rewe.de4,34647210.86%4,33647010.84%−0.02
marksandspencer.com7,2291,67623.18%7,1531,64923.05%−0.13
kohls.com5,02853510.64%4,82348810.12%−0.52
allegro.pl12,7863,31025.89%12,6303,18125.19%−0.70
samsclub.com6,4771,05916.35%6,5071,01615.61%−0.74
tesco.com6,8941,92527.92%7,0621,89926.89%−1.03
argos.co.uk仍零数据 — 配置问题严重

六、Phase 2 按商家 click rate (版本过滤,5/29–6/14)

商家A 曝光A 点击A 率B 曝光B 点击B 率B − A
vistaprint.com3093711.97%3345315.87%+3.90
shop-apotheke.com1,50631620.98%1,49535123.48%+2.50
moonpig.com1,57148731.00%1,61252932.82%+1.82
mediaexpert.pl1,51720413.45%1,39620714.83%+1.38
doordash.com5,12093718.30%5,19395618.41%+0.11
diy.com (B&Q)1,39036025.90%1,45436725.24%−0.66
sephora.com1,14517114.93%1,01314414.22%−0.71
ocado.com1,49838925.97%1,43236025.14%−0.83
gap.com1,30127020.75%1,24823018.43%−2.32
just-eat.co.uk1,43149334.45%1,40943630.94%−3.51
westlotto.de591322.03%531018.87%−3.16

七、Phase 1 严格漏斗 + 订单佣金 (5/5–6/11, T+4 完整)

口径: B 实验组 = 真实看到文案 (style=2 AND enabled=1) → 点击 popup → 同 domain CB 订单

商家A 曝光A 点击A 订单A 佣金B 曝光B 点击B 订单B 佣金Δ订单Δ佣金
allegro.pl12,3463,1684,456$2,110.8512,1653,0294,569$2,082.82+113−$28.03
tesco.com6,6131,8321,692$745.886,3321,6511,571$967.51−121+$221.63
rewe.de4,304454260$480.564,081411320$597.89+60+$117.33
kohls.com4,88951351$131.714,29443652$157.36+1+$25.65
marksandspencer.com7,0101,599554$445.656,6081,491496$423.34−58−$22.31
samsclub.com6,234997440$1,005.435,472848407$503.50−33−$501.93
target.com10,4711,876644$1,207.288,7241,571576$1,101.42−68−$105.86
agoda.com4,225925119$1,792.111,04321532$568.51−87−$1,223.60
Phase 1 合计56,09211,3648,216$7,919.4748,7199,6528,023$6,402.35−193 (−2.3%)−$1,517.12 (−19.2%)
主要拖累: agoda B 桶曝光仅 A 的 25%(1,043 vs 4,225),enabled=1 命中率严重偏低,导致 −$1,224 佣金缺口。samsclub-US 持续负向 −$502。两者合计抵消 Phase 1 其他商家全部正向。

八、Phase 2 严格漏斗 + 订单佣金 (5/29–6/11, T+4 完整,版本过滤)

商家A 曝光A 点击A 订单A 佣金B 曝光B 点击B 订单B 佣金Δ订单Δ佣金
doordash.com4,359742417$137.003,128548353$152.26−64+$15.26
just-eat.co.uk1,211404224$77.45753261198$66.64−26−$10.81
moonpig.com1,337443240$26.02978326179$31.79−61+$5.77
sephora.com87112541$138.866548561$107.84+20−$31.02
diy.com1,15628678$126.1284322570$98.45−8−$27.67
mediaexpert.pl1,34018147$104.2982815937$87.02−10−$17.27
ocado.com1,35634221$43.5399720310$19.96−11−$23.57
shop-apotheke.com1,2822648$11.686521673$5.92−5−$5.76
vistaprint.com2733212$9.211923613$13.66+1+$4.45
gap.com1,10523012$12.939221658$4.97−4−$7.96
westlotto.de591326$31.19000$0−26−$31.19
Phase 2 合计14,3493,0621,126$718.288,9472,175932$588.51−194 (−17.2%)−$129.77 (−18.1%)
桶量失衡: Phase 2 B/A 曝光比例仅 62%,虽然 click rate 持平,但下游订单/佣金的绝对值差距主要来自桶量失衡而非真实负向效应。

九、效应演化轨迹

Phase 1 演化

时间窗天数click B−A订单 B−A佣金 B−A
5/1410+0.58pp
5/1511+0.22pp
5/3127+0.15pp+105 (+1.9%)+$73 (+1.6%)
6/3300.00pp
6/835−0.35pp−156 (−2.1%)−$1,360 (−18.9%)
6/14/1138–41−0.37pp−193 (−2.3%)−$1,517 (−19.2%)

Phase 2 演化(版本过滤)

时间窗天数click B−A订单 B−A佣金 B−A
6/36+0.34pp
6/1114−0.11pp−218 (−27.7%)−$185 (−32.6%)
6/14/1114–17−0.01pp−194 (−17.2%)−$130 (−18.1%)

十、真实正向商家(剔除桶量失衡 + 负向干扰)

Phase商家Δ佣金备注
1tesco.com+$221.63桶量平衡(96%),订单 −121 但佣金涨,B 单均高
1rewe.de+$117.33桶量平衡(95%),订单 +60(+23%)
1kohls.com+$25.65订单几乎持平,佣金小幅正向
1allegro.pl+113 单桶量平衡,订单 +2.5%,佣金 −$28(单均下降)
2doordash.com+$15.26桶量失衡但佣金正向
2moonpig.com+$5.77click +1.82pp 稳定
2vistaprint.com+$4.45click +3.90pp 最强,样本小
真实正向合计+$390约占大盘佣金 5-6%

十一、结论与建议

真实结论: 文案确实在 tesco/rewe/kohls/allegro/doordash/moonpig/vistaprint 等约 7 家商家持续带来订单或佣金正向,合计 ~$390/期。但整体 PRE/POST 累计效果被 agoda(B 桶失衡 −$1,224)和 samsclub(持续负向 −$502)严重抵消,大盘表现 −19% 佣金。

紧急行动清单

优先级商家/问题行动
🔥 立即agoda B 桶 enabled=1 命中率仅 25%排查文案下发逻辑,agoda 是高佣金商家,严重拖累
🔥 立即argos.co.uk 41 天零数据BD/PM 必须介入修复曝光下发
⚠️ 排查samsclub-US, gap-US, just-eat 持续负向文案对北美超市/餐饮场景适配性差,需重做或精准下发
✅ 保留tesco/rewe/kohls/allegro/doordash/moonpig/vistaprint持续观察 + 单店全量推荐
❌ 关停Phase 1 整体效应已转负考虑全量回滚或重做文案 V2,只保留正向商家
💡 验证shop-apotheke / mediaexpert 反转(+2.50 / +1.38)确认文案是否在 6/3 后调整(可能 DE/PL 文案重新下发)

十二、统计逻辑详解

1. AB 桶定义

2. 严格 6 层漏斗

每层都是前一层子集,严格按"看到文案→点击→激活→SP→下单"链路:

数据源过滤条件
① 曝光impress 表extra JSON 含 AB tag
② 点击click 表同 AB tag + guid 在①
③ 激活成功outbound 表outbound_state='1' AND NOT (reason_type='0' AND reason='1')
④ SP 用户journey 表guid 在③
⑤ 下单用户order 表bu='CP' AND is_cb_order=1
⑥ 订单数 + 佣金order 表聚合count(1), sum(commission)

3. 时间窗对齐规则

4. 版本号过滤(Phase 2)

Phase 2 仅统计扩展版本 ≥ 6.20.60 的用户:

WHERE (
  cast(substring_index(version, '.', 1) AS unsigned) * 10000
  + cast(substring_index(substring_index(version, '.', 2), '.', -1) AS unsigned) * 100
  + cast(substring_index(version, '.', -1) AS unsigned)
) >= 62060

3 段式版本号数值化: major×10000 + minor×100 + patch

5. ITT vs 严格漏斗

6. 真实正向商家筛选

十三、SQL 全集

📝 SQL 1 · Phase 1 累计 click rate
SELECT
  i.arm, i.impr_users,
  COALESCE(c.click_users, 0) AS click_users,
  ROUND(100.0 * COALESCE(c.click_users, 0) / NULLIF(i.impr_users, 0), 2) AS click_rate_pct
FROM (
  SELECT
    CASE WHEN get_json_object(extra, '$.core_tips_style') = '6.19.90-2' THEN 'B_exp'
         WHEN get_json_object(extra, '$.core_tips_style') = '6.19.90-1' THEN 'A_ctrl' END AS arm,
    count(distinct guid) AS impr_users
  FROM v_dwd_log_parse_extension_service_impress_di
  WHERE pt BETWEEN '2026-05-05' AND '2026-06-14'
    AND domain IN ('tesco.com','allegro.pl','samsclub.com','kohls.com','target.com',
                   'marksandspencer.com','agoda.com','rewe.de','argos.co.uk')
    AND type = 'CB' AND scene IN ('Popup','FirstPage','Icon')
    AND get_json_object(extra, '$.core_tips_style') IN ('6.19.90-1','6.19.90-2')
  GROUP BY 1
) i
LEFT JOIN (
  -- 同结构换 v_dwd_log_parse_extension_service_click_di
  SELECT
    CASE WHEN get_json_object(extra, '$.core_tips_style') = '6.19.90-2' THEN 'B_exp'
         WHEN get_json_object(extra, '$.core_tips_style') = '6.19.90-1' THEN 'A_ctrl' END AS arm,
    count(distinct guid) AS click_users
  FROM v_dwd_log_parse_extension_service_click_di
  WHERE pt BETWEEN '2026-05-05' AND '2026-06-14'
    AND domain IN (...同上...)
    AND type = 'CB' AND scene IN ('Popup','FirstPage','Icon')
    AND get_json_object(extra, '$.core_tips_style') IN ('6.19.90-1','6.19.90-2')
  GROUP BY 1
) c ON i.arm = c.arm
ORDER BY i.arm;
📝 SQL 2 · Phase 2 累计 click rate(版本过滤)
SELECT i.arm, i.impr_users, COALESCE(c.click_users,0) AS click_users,
       ROUND(100.0*COALESCE(c.click_users,0)/NULLIF(i.impr_users,0),2) AS click_rate_pct
FROM (
  SELECT CASE WHEN get_json_object(extra,'$.core_tips_style')='6.19.90-2' THEN 'B_exp'
              WHEN get_json_object(extra,'$.core_tips_style')='6.19.90-1' THEN 'A_ctrl' END AS arm,
         count(distinct guid) AS impr_users
  FROM v_dwd_log_parse_extension_service_impress_di
  WHERE pt BETWEEN '2026-05-29' AND '2026-06-14'
    AND domain IN ('sephora.com','doordash.com','gap.com','diy.com','shop-apotheke.com',
                   'ocado.com','moonpig.com','just-eat.co.uk','mediaexpert.pl',
                   'vistaprint.com','westlotto.de')
    AND type='CB' AND scene IN ('Popup','FirstPage','Icon')
    AND get_json_object(extra,'$.core_tips_style') IN ('6.19.90-1','6.19.90-2')
    AND (cast(substring_index(version,'.',1) AS unsigned)*10000
       + cast(substring_index(substring_index(version,'.',2),'.',-1) AS unsigned)*100
       + cast(substring_index(version,'.',-1) AS unsigned)) >= 62060
  GROUP BY 1
) i
LEFT JOIN (
  -- 同结构换 click 表
) c ON i.arm=c.arm
ORDER BY i.arm;
📝 SQL 3 · Phase 1 严格漏斗 + 订单佣金(B 实验组)
SELECT
  'B_exp' AS arm, i.domain,
  count(distinct i.guid) AS impr_users,
  count(distinct case when c.guid is not null then i.guid end) AS click_users,
  count(distinct case when c.guid is not null AND o.guid is not null then i.guid end) AS order_users,
  sum(case when c.guid is not null AND o.guid is not null then o.cb_orders else 0 end) AS orders,
  ROUND(sum(case when c.guid is not null AND o.guid is not null then o.commission else 0 end),2) AS commission
FROM (
  -- 看到了文案的用户 (style=2 AND enabled=1)
  SELECT DISTINCT guid, domain FROM v_dwd_log_parse_extension_service_impress_di
  WHERE pt BETWEEN '2026-05-05' AND '2026-06-11'
    AND domain IN (... 9 家 ...)
    AND type='CB' AND scene IN ('Popup','FirstPage','Icon')
    AND get_json_object(extra,'$.core_tips_style')='6.19.90-2'
    AND get_json_object(extra,'$.core_tips_enabled')='1'
) i
LEFT JOIN (
  -- 点击了带文案的 popup
  SELECT DISTINCT guid, domain FROM v_dwd_log_parse_extension_service_click_di
  WHERE pt BETWEEN '2026-05-05' AND '2026-06-11'
    AND domain IN (... 9 家 ...)
    AND type='CB' AND scene IN ('Popup','FirstPage','Icon')
    AND get_json_object(extra,'$.core_tips_style')='6.19.90-2'
    AND get_json_object(extra,'$.core_tips_enabled')='1'
) c ON i.guid=c.guid AND i.domain=c.domain
LEFT JOIN (
  -- CB 订单 GROUP BY guid 算佣金
  SELECT guid, domain, count(1) AS cb_orders, sum(commission) AS commission
  FROM dwd_trd_user_orders_di
  WHERE create_date BETWEEN '2026-05-05' AND '2026-06-11'
    AND domain IN (... 9 家 ...)
    AND bu='CP' AND is_cb_order=1
  GROUP BY guid, domain
) o ON i.guid=o.guid AND i.domain=o.domain
GROUP BY i.domain
ORDER BY i.domain;

-- A 对照组同结构, 去掉 enabled=1 过滤, style 改为 '6.19.90-1'
📝 SQL 4 · Phase 2 严格漏斗(含版本过滤)
-- 与 SQL 3 同结构, 增加版本过滤,domain 改 11 家 Phase 2,日期改 5/29 起
SELECT
  'B_exp' AS arm, i.domain,
  count(distinct i.guid) AS impr_users,
  count(distinct case when c.guid is not null then i.guid end) AS click_users,
  count(distinct case when c.guid is not null AND o.guid is not null then i.guid end) AS order_users,
  sum(case when c.guid is not null AND o.guid is not null then o.cb_orders else 0 end) AS orders,
  ROUND(sum(case when c.guid is not null AND o.guid is not null then o.commission else 0 end),2) AS commission
FROM (
  SELECT DISTINCT guid, domain FROM v_dwd_log_parse_extension_service_impress_di
  WHERE pt BETWEEN '2026-05-29' AND '2026-06-11'
    AND domain IN ('sephora.com','doordash.com','gap.com','diy.com','shop-apotheke.com',
                   'ocado.com','moonpig.com','just-eat.co.uk','mediaexpert.pl',
                   'vistaprint.com','westlotto.de')
    AND type='CB' AND scene IN ('Popup','FirstPage','Icon')
    AND get_json_object(extra,'$.core_tips_style')='6.19.90-2'
    AND get_json_object(extra,'$.core_tips_enabled')='1'
    AND (cast(substring_index(version,'.',1) AS unsigned)*10000
       + cast(substring_index(substring_index(version,'.',2),'.',-1) AS unsigned)*100
       + cast(substring_index(version,'.',-1) AS unsigned)) >= 62060
) i
LEFT JOIN ( -- click 表同结构 ) c ON i.guid=c.guid AND i.domain=c.domain
LEFT JOIN ( -- order 表 GROUP BY guid,domain ) o ON i.guid=o.guid AND i.domain=o.domain
GROUP BY i.domain
ORDER BY i.domain;
📝 SQL 5 · 完整漏斗(含 outbound 激活成功 + SP journey)
SELECT '<arm>' AS arm,
       count(distinct i.guid) AS impr_users,
       count(distinct case when c.guid is not null then i.guid end) AS click_users,
       count(distinct case when c.guid is not null AND ob.client_id is not null then i.guid end) AS act_users,
       count(distinct case when c.guid is not null AND ob.client_id is not null AND sp.guid is not null then i.guid end) AS sp_users,
       count(distinct case when c.guid is not null AND ob.client_id is not null AND sp.guid is not null AND o.guid is not null then i.guid end) AS order_users,
       sum(case when c.guid is not null AND ob.client_id is not null AND sp.guid is not null AND o.guid is not null then o.cb_orders else 0 end) AS orders
FROM (impress style=2 AND enabled=1) i
LEFT JOIN (click 同 tag) c ON i.guid=c.guid
LEFT JOIN (
  -- outbound: 激活成功
  SELECT DISTINCT client_id FROM dwd_log_event_outbound_result_det_d
  WHERE access_at >= '2026-05-05' AND access_at < '2026-06-12'  -- 注意:用 access_at 不是 pt
    AND domain IN (...) AND business='2' AND outbound_state='1'
    AND NOT (outbound_state_reason_type='0' AND outbound_state_reason='1')
) ob ON i.guid=ob.client_id
LEFT JOIN (
  SELECT DISTINCT guid FROM dws_log_sp_user_shopping_journey_detail_di
  WHERE pt BETWEEN '2026-05-05' AND '2026-06-11' AND domain IN (...)
) sp ON i.guid=sp.guid
LEFT JOIN (
  SELECT guid, count(1) AS cb_orders FROM dwd_trd_user_orders_di
  WHERE create_date BETWEEN '2026-05-05' AND '2026-06-11' AND domain IN (...)
    AND bu='CP' AND is_cb_order=1 GROUP BY guid
) o ON i.guid=o.guid;
📝 SQL 6 · 版本号 ≥ 6.20.60 过滤片段
-- 3 段式版本号数值化: major×10000 + minor×100 + patch >= 62060
AND (
  cast(substring_index(version, '.', 1) AS unsigned) * 10000
  + cast(substring_index(substring_index(version, '.', 2), '.', -1) AS unsigned) * 100
  + cast(substring_index(version, '.', -1) AS unsigned)
) >= 62060

-- 示例:
--   '6.20.60' → 6*10000 + 20*100 + 60 = 62060 ✓
--   '6.20.69' → 62069 ✓
--   '6.30.9'  → 6*10000 + 30*100 + 9 = 63009 ✓
--   '6.19.89' → 61989 ✗ 不符合

关键陷阱速查

陷阱避坑
outbound 表无 pt 字段access_at 时间戳过滤
激活成功判定outbound_state='1' AND NOT (reason_type='0' AND reason='1') 不要光看 state=1
CB outbound 标识business='2' 是字符串不是数字
订单延迟T+0=4%, T+4=95%, PRE/POST 切勿包含 3 天内日期
kpis 聚合表无 extra JSON做 AB 必回 raw 表 v_dwd_log_parse_extension_service_*
chained LEFT JOIN 级联陷阱所有 LEFT JOIN 都从 base 表 i.guid 直接 JOIN,不要 c.guid → o.guid
订单表 country 字段名visit_country 不是 country
同 DOW 对齐PRE/POST 必须同长度同起始 DOW(周一对周一)