CB 横条弹窗 A/B · 数据回收(自上线 6/19 ~ 6/25)
extra.s_popup='0' 当对照组是错的——s_popup='0' 是全站所有用户普通 CB 弹窗的默认值,不是人群包对照臂。DMP 交叉验证:人群包对照臂当天仅 ~580 人有弹窗曝光,而 s_popup='0' 全站有 5,047 人 → 旧"对照组"混进了 ~4,467 个人群包之外的用户,把曝光做成假的「1:10 不均衡」、把对照 CTR 抬到全站的 25%。本版两臂都限定 DMP 人群包(_1 vs _2),曝光均衡(2,483 vs 2,527),对照 CTR 回落到秒关人群真实的 ~14%。实验组 s_popup='1' 本就≈纯净(554/555 落在 _2),无需改。
① 三组对比(核心 · DMP 人群包口径 · 6/19~6/25)
两臂均按 DMP 分桶分配(ITT):实验=横条臂(_2),对照=弹窗臂(_1)。实验组拆 横条 与 横条+悬浮(同次访问悬浮球去重并集)。曝光率分母=DMP 分桶人数(exp 29,652 / ctrl 29,894)。下单率/曝光、佣金/曝光为归一化指标。
| 组 | 曝光UV | 曝光率 | 点击UV | CTR | 关闭率 | 激活 | 激活率 | 下单UV | 订单 | sales | 佣金 | 下单率/曝光 | 佣金/曝光 |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 对照 · 弹窗 | 2,527 | 8.45% | 357 | 14.13% | 80.17% | 340 | 95.24% | 71 | 333 | $6,373.33 | $197.45 | 2.81% | $0.0781 |
| 对照 · 弹窗+悬浮 | 2,527 | 8.45% | 384 | 15.2% | — | 366 | 95.31% | 80 | 370 | $6,779.79 | $217.00 | 3.17% | $0.0859 |
| 实验 · 横条 | 2,483 | 8.37% | 231 | 9.3% | 57.87% | 222 | 96.1% | 41 | 94 | $1,811.23 | $58.85 | 1.65% | $0.0237 |
| 实验 · 横条+悬浮 | 2,483 | 8.37% | 262 | 10.55% | — | 252 | 96.18% | 52 | 114 | $5,579.67 | $97.91 | 2.09% | $0.0394 |
两臂曝光率几乎相等(8.37% vs 8.45%)= A/B 分桶健康、弹窗/横条触发覆盖均衡。激活率两组 ~95-96%(点击即激活,瓶颈在点击)。订单 6/23-25 T+1~3 不全,佣金对比仅供参考(小样本+延迟)。
② 悬浮球归因阶梯(口径松紧)
| 实验组悬浮球口径 | 点击UV |
|---|---|
| loose 任意站点(高估) | 64 |
| 同商家+同天(同次访问·采用) | 34 |
| 同页 exact-url(偏低) | 12 |
| 对照组悬浮球口径 | 点击UV |
|---|---|
| loose 任意站点(高估) | 66 |
| 同商家+同天(同次访问·采用) | 33 |
| 同页 exact-url(偏低) | 15 |
loose(任意站点)会把用户在别的页/别的站点的悬浮球也算进来(高估);同页 exact-url 因悬浮球跨页跟随会偏低。取中间「同次访问」。悬浮球完整漏斗已并入①的「横条+悬浮」「弹窗+悬浮」行。
③ 按天(DMP 口径 · 两臂每天均衡)
| 日期 | 组 | 曝光UV | 点击UV | CTR | 关闭率 | 下单UV | 订单 | 佣金 |
|---|---|---|---|---|---|---|---|---|
| 2026-06-19 | 对照 | 103 | 17 | 16.5% | 70.87% | 3 | 16 | $2.95 |
| 2026-06-19 | 实验 | 87 | 8 | 9.2% | 58.62% | 1 | 1 | $0.36 |
| 2026-06-20 | 对照 | 87 | 12 | 13.79% | 73.56% | 3 | 7 | $1.19 |
| 2026-06-20 | 实验 | 88 | 6 | 6.82% | 65.91% | 2 | 2 | $0.40 |
| 2026-06-21 | 对照 | 114 | 14 | 12.28% | 83.33% | 3 | 5 | $56.59 |
| 2026-06-21 | 实验 | 98 | 7 | 7.14% | 64.29% | 4 | 13 | $12.18 |
| 2026-06-22 | 对照 | 760 | 90 | 11.84% | 78.68% | 21 | 55 | $33.89 |
| 2026-06-22 | 实验 | 719 | 60 | 8.34% | 60.92% | 7 | 16 | $7.24 |
| 2026-06-23 | 对照 | 939 | 110 | 11.71% | 80.62% | 32 | 145 | $67.88 |
| 2026-06-23 | 实验 | 944 | 68 | 7.2% | 61.33% | 14 | 30 | $22.22 |
| 2026-06-24 | 对照 | 544 | 82 | 15.07% | 66.73% | 17 | 67 | $22.53 |
| 2026-06-24 | 实验 | 568 | 55 | 9.68% | 38.56% | 13 | 19 | $11.70 |
| 2026-06-25 | 对照 | 582 | 100 | 17.18% | 66.67% | 10 | 38 | $12.42 |
| 2026-06-25 | 实验 | 558 | 54 | 9.68% | 31.0% | 5 | 13 | $4.75 |
两臂逐日曝光基本相等(如 6/23 对照 939 / 实验 944)。6/22 起量升是大盘活跃度,两臂同步,非分桶问题。横条关闭率每天都明显低于弹窗,CTR 每天都略低于弹窗。6/23-25 订单 T+1~3 偏低。
④ 按商家(DMP 口径)
| 商家 | 组 | 曝光UV | 点击UV | CTR | 关闭率 | 下单UV | 订单 | 佣金 |
|---|---|---|---|---|---|---|---|---|
| aliexpress.com | 对照 | 1044 | 194 | 18.58% | 77.87% | 37 | 293 | $124.87 |
| aliexpress.com | 实验 | 1038 | 140 | 13.49% | 57.32% | 25 | 78 | $52.53 |
| etsy.com | 对照 | 241 | 7 | 2.9% | 84.65% | 3 | 3 | $43.59 |
| etsy.com | 实验 | 245 | 5 | 2.04% | 76.33% | 1 | 1 | $0.41 |
| lowes.com | 对照 | 125 | 12 | 9.6% | 84.0% | 3 | 4 | $19.77 |
| lowes.com | 实验 | 107 | 7 | 6.54% | 66.36% | 2 | 2 | $2.52 |
| samsclub.com | 对照 | 160 | 7 | 4.38% | 82.5% | 3 | 3 | $0.00 |
| samsclub.com | 实验 | 194 | 7 | 3.61% | 60.82% | 3 | 3 | $0.16 |
| shein.com | 对照 | 7 | 0 | 0.0% | 85.71% | 0 | 0 | $— |
| shein.com | 实验 | 14 | 0 | 0.0% | 57.14% | 0 | 0 | $— |
| shopee.com.br | 对照 | 514 | 94 | 18.29% | 74.71% | 14 | 17 | $1.62 |
| shopee.com.br | 实验 | 485 | 39 | 8.04% | 42.68% | 4 | 4 | $0.41 |
| target.com | 对照 | 315 | 27 | 8.57% | 81.9% | 3 | 4 | $2.98 |
| target.com | 实验 | 273 | 18 | 6.59% | 65.57% | 3 | 3 | $1.50 |
| tesco.com | 对照 | 201 | 21 | 10.45% | 84.08% | 8 | 9 | $4.63 |
| tesco.com | 实验 | 194 | 16 | 8.25% | 45.88% | 3 | 3 | $1.32 |
各商家两臂曝光均衡。横条 CTR 普遍略低于弹窗、关闭率普遍明显低于弹窗。aliexpress 量最大(实验 1038 曝光 / CTR 13.5% vs 弹窗 18.6%);shein 两臂都几乎无量(7/14 曝光)= 商家横条/弹窗在该站基本没触发。
⑤ 读数与结论(修正版)
- 关闭率:横条 57.9% ≪ 弹窗 80.2%(-22.3pp) —— 最强信号,且比污染口径更显著(真对照=秒关人群,弹窗下 80% 关闭;横条降到 58%)。✅「遮挡假设」成立。
- CTR:横条 9.3% < 弹窗 14.1%(-4.8pp) —— 横条点击仍偏低,但差距只有此前的约 1/3(之前 -14pp 是拿全站当对照的假象)。横条少关掉的 22pp 用户大多变成「不关也不点」= banner-blindness。
- 悬浮球二次召回真实但有限(同次访问):实验组 34 点击 / 12 下单 / 20 单 / $39 佣金,把横条拉到 横条+悬浮 CTR 10.6%、下单率/曝光 2.09%。
- 单位曝光转化:横条+悬浮 下单率 2.09% vs 弹窗+悬浮 3.17%(≈66%);佣金/曝光 $0.039 vs $0.086(≈46%,受小样本大单影响,看下单率更稳)。横条仍跑不平弹窗。
- 激活率两组 ~95-96%(瓶颈在点击不在激活)。
- 订单未收口:6/23-25 为 T+1~3,佣金/订单偏低,最终需等 T+4(6/29 后)复评。
建议:①横条「不挡操作」已强证实(关闭率↓22pp),方向正确;但 CTR 仍偏低、悬浮球只补 +1.3pp → 按 PRD 既定应对加「显眼度」(首屏微动效、放大预估金额、强化 Coupert 标识)把被"放过"的 22pp 忽略用户转化为点击;②等 6/29 T+4 订单收口后,用 DMP 口径看下单率/曝光是否随放量稳定(当前 66%)再定全量;③shein 等无量商家排查触发条件。
⑥ 计算逻辑 & 口径
- 分组(ITT,关键修正):用 DMP 镜像
dws_dmp_guid_group_list_daily_d的分桶 ——array_contains(group_id_list,'100000426_2')=实验 /_1=对照。不能用extra.s_popup='0'当对照(那是全站默认值,会混入人群包外用户)。 - 曝光/点击/关闭:DMP 臂内用户的
type='CB' AND scene='Popup' AND version>='6.30.69'8 家曝光;点击/关闭靠 impress_id 回连。横条实际记为 scene='Popup' + extra.cb_small_popup='1'(PRD 计划的新 scene 'Small_Popup' 实际未启用)。 - 激活/订单:click_id = 出站 tracking_id = 订单 tracking_id(is_cb_order=1)。
- 悬浮球:scene='FloatWin' click,client_id∈臂内 cohort 且 (sub_domain,当天) 与其横条曝光一致(同次访问近似)。横条+悬浮 = 两者点击 guid 去重并集,订单去重 order_id。
⑦ 完整 SQL(DMP 口径 · 可复跑)
WITH dmp AS ( -- ⭐分组用 DMP 分桶,不用 s_popup
SELECT guid, CASE WHEN array_contains(group_id_list,'100000426_2') THEN 'exp' ELSE 'ctrl' END grp
FROM dws_dmp_guid_group_list_daily_d WHERE pt='2026-06-24'
AND (array_contains(group_id_list,'100000426_1') OR array_contains(group_id_list,'100000426_2'))),
expo AS (SELECT i.impress_id, i.guid, i.sub_domain mch, substr(i.create_time,1,10) d, dm.grp
FROM v_dwd_log_parse_extension_service_impress_di i JOIN dmp dm ON i.guid=dm.guid
WHERE i.pt BETWEEN '2026-06-19' AND '2026-06-25' AND i.type='CB' AND i.scene='Popup' AND i.version>='6.30.69'
AND i.sub_domain IN ('shein.com','shopee.com.br','tesco.com','lowes.com','target.com','aliexpress.com','etsy.com','samsclub.com')
AND i.impress_id IS NOT NULL),
bp AS (SELECT DISTINCT grp, guid, mch, d FROM expo),
clkraw AS (SELECT client_id guid, click_id, impress_id, scene, sub_domain mch, substr(access_time,1,10) d
FROM v_dwd_log_cp_click_cb_log_di WHERE pt BETWEEN '2026-06-19' AND '2026-06-25' AND action='click' AND click_id IS NOT NULL),
barclk AS (SELECT e.grp, c.guid, c.click_id, 'bar' src FROM clkraw c JOIN expo e ON c.impress_id=e.impress_id),
floclk AS (SELECT DISTINCT bp.grp, c.guid, c.click_id, 'float' src
FROM clkraw c JOIN bp ON c.guid=bp.guid AND c.mch=bp.mch AND c.d=bp.d WHERE c.scene='FloatWin'),
allclk AS (SELECT DISTINCT grp,guid,click_id,src FROM (SELECT * FROM barclk UNION ALL SELECT * FROM floclk) u),
act AS (SELECT DISTINCT tracking_id FROM dwd_log_event_outbound_result_det_d
WHERE DATE(access_at) BETWEEN '2026-06-19' AND '2026-06-30' AND business='2' AND outbound_state='1'
AND NOT(outbound_state_reason='0' AND outbound_state_reason_type='1')),
ord AS (SELECT tracking_id, order_id, sales, commission FROM dwd_trd_user_orders_di
WHERE create_date BETWEEN '2026-06-19' AND '2026-06-27' AND is_cb_order=1)
SELECT a.grp, a.src, COUNT(DISTINCT a.guid) clk_uv,
COUNT(DISTINCT CASE WHEN ac.tracking_id IS NOT NULL THEN a.guid END) act_uv,
COUNT(DISTINCT CASE WHEN o.order_id IS NOT NULL THEN a.guid END) ord_uv,
COUNT(DISTINCT o.order_id) orders, ROUND(SUM(o.commission),2) comm
FROM allclk a LEFT JOIN act ac ON a.click_id=ac.tracking_id LEFT JOIN ord o ON a.click_id=o.tracking_id
GROUP BY GROUPING SETS ((a.grp,a.src),(a.grp)) ORDER BY a.grp,a.src;
-- 曝光UV/关闭UV:expo LEFT JOIN (action='close' DISTINCT impress_id) ON impress_id, GROUP BY grp