CB 横条弹窗 A/B · 数据回收(6/19-20 早期)
结账页遮挡假设验证 · 实验组横条 vs 对照组原 CB 弹窗 · 含回收逻辑 / 计算逻辑 / 完整 SQL
一句话结论(早期 · 样本极小 · 不可下结论):横条 关闭率 34.5% < 弹窗 50.2%(更不挡操作、秒关↓,贴合假设),但 CTR 11.9% < 弹窗 28.8%(点击也腰斩)→ 更像横条不够显眼(banner-blindness)。且实验组 2 天仅 84 曝光 / 10 点击,两臂曝光 ≈ 1:10 不均衡,shein 开关未开(曝光 0)→ 需推动铺量 + 等订单 T+4 再复评。
① 回收逻辑
分组(人群标识 s_popup + 版本门槛)
- 实验组 =
s_popup='1'(接口下发的人群标识,看到横条);对照组 =s_popup='0'(看到原 CB 弹窗)。s_popup 在曝光事件extraJSON 里。 - 必须加
version >= '6.30.69':横条是 6.30.69(20% 放量)版本功能,不加版本门槛,s_popup=0 会混入老版本默认值(污染对照)。 - 共同限定:
type='CB' AND scene='Popup'(仅业务类型+弹窗场景限定,不区分组)+ 7 家商家。两组唯一区别是 extra 的 s_popup。 - 组人数(ITT 分母):取 DMP 镜像表
dws_dmp_guid_group_list_daily_d的分桶 assignment(100000426_2=实验 /_1=对照,干净 50/50 各 ~3.16 万)。
严格漏斗串联(看到 → 点击/关闭同一条弹窗 → 激活 → SP → 下单)
- 点击/关闭靠
impress_id回连曝光:impress_id 是单条弹窗实例的唯一 ID(曝光/点击/关闭共享),保证"点/关的就是被曝光的那条弹窗",排除隔天或跨弹窗的点击。
⚠️ 关闭/点击事件自身 extra 的 s_popup 可能为空 → 分组只能靠 impress_id 回连曝光,不能靠事件自己的 s_popup。 - 激活/订单靠
click_id = 出站.tracking_id = 订单.tracking_id串联到"点击的那条弹窗"。 - 每级是上一级子集:点击⊆曝光、激活⊆点击、SP⊆激活、下单⊆SP。
数据源表
| 环节 | 表 | 关键字段 |
|---|---|---|
| 曝光 | v_dwd_log_parse_extension_service_impress_di | guid / type / scene / version / impress_id / extra(s_popup,cb_small_popup) |
| 点击·关闭 | v_dwd_log_cp_click_cb_log_di | client_id / impress_id / click_id / action(click/close) |
| 激活成功(出站) | dwd_log_event_outbound_result_det_d | tracking_id / business / outbound_state / _reason / _reason_type / access_at |
| SP | dws_log_sp_user_shopping_journey_detail_di | guid / domain |
| 订单 | dwd_trd_user_orders_di | guid / tracking_id / is_cb_order / sales / commission / created_time |
| 分桶人数 | dws_dmp_guid_group_list_daily_d | guid / group_id_list(array) |
② 计算逻辑(每指标定义)
| 指标 | 定义 / 公式(均按 guid 去重) |
|---|---|
| 组人数 | DMP 分桶 assignment 的 UV(exp=arm2 / ctrl=arm1) |
| 曝光UV | 该组(s_popup)+ version≥6.30.69 + CB Popup + 7家 的 distinct guid |
| 曝光率 | 曝光UV ÷ 组人数 |
| 点击UV | 曝光人群中,存在 action='click' 且 impress_id 命中曝光的 distinct guid |
| CTR | 点击UV ÷ 曝光UV |
| 关闭UV | 曝光人群中,存在 action='close' 且 impress_id 命中曝光的 distinct guid |
| 关闭率 | 关闭UV ÷ 曝光UV |
| 激活成功UV | 点击人群中,click_id=出站tracking_id 且 business='2'(CB) outbound_state='1' 且排除 (state=1 AND reason=0 AND type=1) 的 distinct guid |
| 激活成功率 | 激活成功UV ÷ 点击UV |
| SP人数 | 激活人群中,在 7 家有 SP 事件的 distinct guid |
| 下单人数 / 订单数 | click_id=订单tracking_id 的 CB 单:distinct guid / distinct order_id |
| sales / commission | 上述 CB 单的 sum(sales) / sum(commission)(⚠️ T+0~2 不全) |
③ 汇总数据
| 组 | 组人数 | 曝光UV | 曝光率 | 点击UV | CTR | 关闭UV | 关闭率 | 激活成功UV | 激活成功率 | SP | 下单 | 订单 | sales | comm |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 实验(横条) | 31,575 | 84 | 0.266% | 10 | 11.90% | 29 | 34.52% | 10 | 100% | 4 | 2 | 2 | $12.68 | $0.76 |
| 对照(弹窗) | 31,764 | 861 | 2.711% | 248 | 28.80% | 432 | 50.17% | 245 | 98.79% | 122 | 41 | 121 | $2,279.77 | $75.74 |
对照曝光 861 已与 ava 口径校准一致。激活成功率两组 ~100%(点击即激活有效)。
④ 按天
| 日期 | 组 | 组人数 | 曝光UV | 曝光率 | 点击UV | CTR | 关闭UV | 关闭率 | 激活成功UV | 激活成功率 | SP | 下单 | 订单 | sales | comm |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 6/19 | 实验 | 31,575 | 50 | 0.158% | 7 | 14.00% | 17 | 34.00% | 7 | 100% | 2 | 1 | 1 | $5.23 | $0.36 |
| 6/19 | 对照 | 31,764 | 461 | 1.451% | 139 | 30.15% | 224 | 48.59% | 137 | 98.56% | 65 | 21 | 67 | $1,419.06 | $53.08 |
| 6/20 | 实验 | 31,575 | 38 | 0.120% | 3 | 7.89% | 12 | 31.58% | 3 | 100% | 2 | 1 | 1 | $7.45 | $0.40 |
| 6/20 | 对照 | 31,764 | 448 | 1.410% | 119 | 26.56% | 221 | 49.33% | 118 | 99.16% | 62 | 21 | 54 | $860.71 | $22.66 |
⑤ 按商家
| 商家 | 组 | 曝光UV | 曝光率 | 点击UV | CTR | 关闭UV | 关闭率 | 激活成功率 | SP | 下单 | 订单 | sales | comm |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| aliexpress | 实验 | 42 | 0.133% | 7 | 16.67% | 12 | 28.57% | 100% | 3 | 2 | 2 | $12.68 | $0.76 |
| aliexpress | 对照 | 506 | 1.593% | 170 | 33.60% | 230 | 45.45% | 99.4% | 92 | 26 | 101 | $1,219.27 | $60.94 |
| shopee.com.br | 实验 | 18 | 0.057% | 1 | 5.56% | 7 | 38.89% | 100% | 0 | 0 | 0 | — | — |
| shopee.com.br | 对照 | 123 | 0.387% | 39 | 31.71% | 58 | 47.15% | 100% | 11 | 6 | 9 | $152.13 | $0.97 |
| etsy | 实验 | 6 | 0.019% | 0 | 0% | 3 | 50.00% | — | 0 | 0 | 0 | — | — |
| etsy | 对照 | 85 | 0.268% | 8 | 9.41% | 45 | 52.94% | 100% | 3 | 2 | 4 | $33.11 | $1.66 |
| lowes | 实验 | 8 | 0.025% | 1 | 12.50% | 5 | 62.50% | 100% | 0 | 0 | 0 | — | — |
| lowes | 对照 | 64 | 0.201% | 10 | 15.63% | 43 | 67.19% | 90% | 2 | 1 | 1 | $545.16 | $10.90 |
| samsclub | 实验 | 6 | 0.019% | 1 | 16.67% | 2 | 33.33% | 100% | 1 | 0 | 0 | — | — |
| samsclub | 对照 | 35 | 0.110% | 5 | 14.29% | 23 | 65.71% | 100% | 3 | 3 | 3 | $0 | $0 |
| tesco | 实验 | 5 | 0.016% | 0 | 0% | 0 | 0% | — | 0 | 0 | 0 | — | — |
| tesco | 对照 | 51 | 0.161% | 16 | 31.37% | 33 | 64.71% | 93.8% | 11 | 3 | 3 | $330.10 | $1.27 |
| shein | 对照 | 2 | 0.006% | 0 | 0% | 1 | 50.00% | — | 0 | 0 | 0 | — | — |
| shein | 实验 | 0 | 0% | — | — | — | — | — | — | — | — | — | — |
⚠️ shein 实验组曝光 0 = 商家横条开关(cb:cb:small:frame)未开;aliexpress 是唯一有像样量的商家。
⑥ 完整 SQL(可复跑 · StarRocks / cp_dw db35)
汇总查询如下;按天 / 按商家只需在最外层 SELECT 与 GROUP BY 增加 e.dt / e.mch。
WITH
-- 组人数:DMP 镜像表分桶 (50/50 assignment)
arm_size AS (
SELECT CASE WHEN array_contains(group_id_list,'100000426_2') THEN 'exp' ELSE 'ctrl' END grp,
COUNT(DISTINCT guid) arm_n
FROM dws_dmp_guid_group_list_daily_d
WHERE pt='2026-06-20'
AND (array_contains(group_id_list,'100000426_1') OR array_contains(group_id_list,'100000426_2'))
GROUP BY 1),
-- 曝光:人群标识 s_popup 分组 + version>=6.30.69 + CB Popup + 7家
expo AS (
SELECT i.guid, i.impress_id, i.pt dt, i.sub_domain mch,
CASE WHEN get_json_string(i.extra,'$.s_popup')='1' THEN 'exp' ELSE 'ctrl' END grp
FROM v_dwd_log_parse_extension_service_impress_di i
WHERE i.pt >= '2026-06-19' AND i.type='CB' AND i.scene='Popup' AND i.version >= '6.30.69'
AND i.sub_domain IN ('shein.com','shopee.com.br','tesco.com','lowes.com','aliexpress.com','etsy.com','samsclub.com')
AND get_json_string(i.extra,'$.s_popup') IN ('0','1')),
-- 点击/关闭:靠 impress_id 回连"被曝光的同一条弹窗"
clk AS (SELECT DISTINCT impress_id, click_id, client_id guid FROM v_dwd_log_cp_click_cb_log_di
WHERE pt >= '2026-06-19' AND action='click' AND impress_id IS NOT NULL
AND sub_domain IN ('shein.com','shopee.com.br','tesco.com','lowes.com','aliexpress.com','etsy.com','samsclub.com')),
cls AS (SELECT DISTINCT impress_id, client_id guid FROM v_dwd_log_cp_click_cb_log_di
WHERE pt >= '2026-06-19' AND action='close' AND impress_id IS NOT NULL
AND sub_domain IN ('shein.com','shopee.com.br','tesco.com','lowes.com','aliexpress.com','etsy.com','samsclub.com')),
-- 激活成功:出站 CB(business=2) 成功,排除(state=1&reason=0&type=1)
act AS (SELECT DISTINCT tracking_id FROM dwd_log_event_outbound_result_det_d
WHERE DATE(access_at) >= '2026-06-19' AND business='2' AND outbound_state='1'
AND NOT (outbound_state_reason='0' AND outbound_state_reason_type='1')
AND subdomain IN ('shein.com','shopee.com.br','tesco.com','lowes.com','aliexpress.com','etsy.com','samsclub.com')),
sp AS (SELECT DISTINCT guid FROM dws_log_sp_user_shopping_journey_detail_di
WHERE pt >= '2026-06-19'
AND domain IN ('shein.com','shopee.com.br','tesco.com','lowes.com','aliexpress.com','etsy.com','samsclub.com')),
ord AS (SELECT tracking_id, order_id, sales, commission FROM dwd_trd_user_orders_di
WHERE DATE(created_time) >= '2026-06-19' AND is_cb_order=1
AND root_domain IN ('shein.com','shopee.com.br','tesco.com','lowes.com','aliexpress.com','etsy.com','samsclub.com'))
SELECT e.grp 组,
MAX(z.arm_n) 组人数,
COUNT(DISTINCT e.guid) 曝光UV,
ROUND(COUNT(DISTINCT e.guid)*100.0/NULLIF(MAX(z.arm_n),0),3) 曝光率,
COUNT(DISTINCT CASE WHEN c.impress_id IS NOT NULL THEN e.guid END) 点击UV,
ROUND(COUNT(DISTINCT CASE WHEN c.impress_id IS NOT NULL THEN e.guid END)*100.0
/NULLIF(COUNT(DISTINCT e.guid),0),2) CTR,
COUNT(DISTINCT CASE WHEN cl.impress_id IS NOT NULL THEN e.guid END) 关闭UV,
ROUND(COUNT(DISTINCT CASE WHEN cl.impress_id IS NOT NULL THEN e.guid END)*100.0
/NULLIF(COUNT(DISTINCT e.guid),0),2) 关闭率,
COUNT(DISTINCT CASE WHEN a.tracking_id IS NOT NULL THEN e.guid END) 激活成功UV,
ROUND(COUNT(DISTINCT CASE WHEN a.tracking_id IS NOT NULL THEN e.guid END)*100.0
/NULLIF(COUNT(DISTINCT CASE WHEN c.impress_id IS NOT NULL THEN e.guid END),0),2) 激活成功率,
COUNT(DISTINCT CASE WHEN a.tracking_id IS NOT NULL AND s.guid IS NOT NULL THEN e.guid END) SP人数,
COUNT(DISTINCT CASE WHEN o.tracking_id IS NOT NULL THEN e.guid END) 下单人数,
COUNT(DISTINCT o.order_id) 订单数,
ROUND(SUM(o.sales),2) sales, ROUND(SUM(o.commission),2) commission
FROM expo e
JOIN arm_size z ON e.grp=z.grp
LEFT JOIN clk c ON e.impress_id=c.impress_id -- 点击⊆曝光(同一条弹窗)
LEFT JOIN cls cl ON e.impress_id=cl.impress_id -- 关闭⊆曝光(同一条弹窗)
LEFT JOIN act a ON c.click_id=a.tracking_id -- 激活⊆点击
LEFT JOIN sp s ON e.guid=s.guid
LEFT JOIN ord o ON c.click_id=o.tracking_id -- 下单⊆点击(CB归因)
GROUP BY e.grp
ORDER BY e.grp;
⑦ 读数与数据质量
- 关闭率:实验 34.5% < 对照 50.2%(-15.7pp)——最贴合假设的信号:横条更不挡操作,反射秒关明显减少(按天/aliexpress 一致)。
- 但 CTR:实验 11.9% < 对照 28.8%——点击也腰斩。"关闭也低、点击也低"= 横条不够显眼(banner-blindness),正是 PRD 预设的"未达预期"情形之一。
- 激活成功率两组 ~100%(点击即激活有效,瓶颈在点击不在激活)。
- 不可下结论:实验组仅 84 曝光 / 10 点击(n 极小);两臂曝光 ≈ 1:10 不均衡(横条只在小部分实验流量真渲染,曝光率 0.27% vs 对照 2.71%);shein 开关未开。
- 订单不全:6/19-20 订单为 T+0~2,联盟回调延迟下仅 ~4-50% 落库,sales/commission/订单 别作对比,需等 T+4(6/25 后)。
- 6/21 事件尚未 ETL 落库(按 PT 日落表)。
下一步:推动 shein 等商家横条开关全开 + 排查实验臂仅 0.27% 触发横条的原因 → 放量到两臂曝光均衡(每臂≥几百 UV)+ 等订单 T+4 → 用本页 SQL(s_popup 分组 + version≥6.30.69 + impress_id/click_id 严格串联)复评。