安卓输入窃取窗口分析
本文辑录AOSP 14中spy window之理念、调用脉络、分派机理、pilferPointers()抢占之理,及其与InputMonitor之关联。要旨所陈:
spy window乃一种InputWindow之设,其枢要乃InputConfig.SPY。InputManager.monitorGestureInput()所返之android.view.InputMonitor于AOSP 14中,其下实由GestureMonitorSpyWindow为之,故所创者实为spy window。InputManagerService.monitorInput()/ 本地createInputMonitor()乃另类之全局监控机制,不隶于窗体系统,无 Z-order 与 touchable region 之概念。
1. Spy Window 之定义
方法: android.os.InputConfig
文件: frameworks/native/libs/input/android/os/InputConfig.aidl
/**
* An input spy window. This window will receive all pointer events within its touchable
* area, but will not stop events from being sent to other windows below it in z-order.
* An input event will be dispatched to all spy windows above the top non-spy window at the
* event's coordinates.
*/
SPY = 1 << 14,
含义:
- spy window 能收受其 touchable area 内之 pointer event。
- 其不拦阻、不阻截 Z-order 下方窗体接收事件。
- 惟位于“命中之最上层非 spy window”之上之 spy windows 方能收受事件。
例如 Z-order 自上而下为:
spy1
spy2
appWindow
spy3
若触摸点命中appWindow者,分发之的乃:
appWindow + spy1 + spy2
spy3,此不承之,盖因其潜于非 spy 之窗下而未中。
2. Java API 至 Spy Window 之创生链
2.1 法:InputManager.monitorGestureInput(String name, int displayId)
文:frameworks/base/core/java/android/hardware/input/InputManager.java
/**
* Monitor input on the specified display for gestures.
*
* @hide
*/
public InputMonitor monitorGestureInput(String name, int displayId) {
return mGlobal.monitorGestureInput(name, displayId);
}
此乃 SystemUI、Shell 等系统构件所常叩之入口。
2.2 法:InputManagerGlobal.monitorGestureInput(String name, int displayId)
文:frameworks/base/core/java/android/hardware/input/InputManagerGlobal.java
/**
* @see InputManager#monitorGestureInput(String, int)
*/
public InputMonitor monitorGestureInput(String name, int displayId) {
try {
return mIm.monitorGestureInput(new Binder(), name, displayId);
} catch (RemoteException ex) {
throw ex.rethrowFromSystemServer();
}
}
此间以 Binder 呼 IInputManager.monitorGestureInput(...),入 system_server 之 InputManagerService。
2.3 法:InputManagerService.monitorGestureInput(...)
文:frameworks/base/services/core/java/com/android/server/input/InputManagerService.java
@Override // Binder call
public InputMonitor monitorGestureInput(IBinder monitorToken, @NonNull String requestedName,
int displayId) {
if (!checkCallingPermission(android.Manifest.permission.MONITOR_INPUT,
"monitorGestureInput()")) {
throw new SecurityException("Requires MONITOR_INPUT permission");
}
final SurfaceControl sc = mWindowManagerCallbacks.createSurfaceForGestureMonitor(name,
displayId);
final InputChannel inputChannel = createSpyWindowGestureMonitor(
monitorToken, name, sc, displayId, pid, uid);
return new InputMonitor(inputChannel,
new InputMonitorHost(inputChannel.getToken()),
new SurfaceControl(sc, "IMS.monitorGestureInput"));
}
要旨:
- 调用者须具
MONITOR_INPUT之权。 - WMS立一器,以监手势,名曰
SurfaceControl。 - IMS召
createSpyWindowGestureMonitor(...),立一窥窗。 - 所返于调用者者,乃
android.view.InputMonitor。
2.4法:InputManagerService.createSpyWindowGestureMonitor(...)
文:frameworks/base/services/core/java/com/android/server/input/InputManagerService.java
@NonNull
private InputChannel createSpyWindowGestureMonitor(IBinder monitorToken, String name,
SurfaceControl sc, int displayId, int pid, int uid) {
final InputChannel channel = createInputChannel(name);
monitorToken.linkToDeath(() -> removeSpyWindowGestureMonitor(channel.getToken()), 0);
synchronized (mInputMonitors) {
mInputMonitors.put(channel.getToken(),
new GestureMonitorSpyWindow(monitorToken, name, displayId, pid, uid, sc,
channel));
}
final InputChannel outInputChannel = new InputChannel();
channel.copyTo(outInputChannel);
return outInputChannel;
}
要旨:
- 立一偶,为输入之渠。
- 以
GestureMonitorSpyWindow装束渠道与表。 - 以渠道令牌为钥,存于
mInputMonitors。 - 返 client 侧
InputChannel于唤者。
2.5 法:GestureMonitorSpyWindow.GestureMonitorSpyWindow(...)
文:frameworks/base/services/core/java/com/android/server/input/GestureMonitorSpyWindow.java
GestureMonitorSpyWindow(IBinder token, String name, int displayId, int pid, int uid,
SurfaceControl sc, InputChannel inputChannel) {
mWindowHandle = new InputWindowHandle(mApplicationHandle, displayId);
mWindowHandle.name = name;
mWindowHandle.token = mClientChannel.getToken();
mWindowHandle.layoutParamsType = WindowManager.LayoutParams.TYPE_SECURE_SYSTEM_OVERLAY;
mWindowHandle.ownerPid = pid;
mWindowHandle.ownerUid = uid;
mWindowHandle.replaceTouchableRegionWithCrop(null /* use this surface's bounds */);
mWindowHandle.inputConfig =
InputConfig.NOT_FOCUSABLE | InputConfig.SPY | InputConfig.TRUSTED_OVERLAY;
final SurfaceControl.Transaction t = new SurfaceControl.Transaction();
t.setInputWindowInfo(mInputSurface, mWindowHandle);
t.setLayer(mInputSurface, InputManagerService.INPUT_OVERLAY_LAYER_GESTURE_MONITOR);
t.show(mInputSurface);
t.apply();
}
此乃 monitorGestureInput() 与窥窗相系之明证:
monitorGestureInput()
-> createSpyWindowGestureMonitor()
-> new GestureMonitorSpyWindow()
-> InputConfig.SPY
GestureMonitorSpyWindow 为无图形缓之输入表,然以 setInputWindowInfo(...) 入输入窗列。
3. 直用 INPUT_FEATURE_SPY 之窗径。
非惟monitorGestureInput()自造窺視之窗,系統之窗亦可直設WindowManager.LayoutParams.INPUT_FEATURE_SPY。
3.1 方術/域: WindowManager.LayoutParams.INPUT_FEATURE_SPY
檔: frameworks/base/core/java/android/view/WindowManager.java
/**
* An input spy window. This window will receive all pointer events within its touchable
* area, but will not stop events from being sent to other windows below it in z-order.
* An input event will be dispatched to all spy windows above the top non-spy window at the
* event's coordinates.
*
* @hide
*/
@RequiresPermission(permission.MONITOR_INPUT)
public static final int INPUT_FEATURE_SPY = 1 << 2;
3.2 方術: WindowManagerService.sanitizeSpyWindow(...)
檔: frameworks/base/services/core/java/com/android/server/wm/WindowManagerService.java
/**
* You need MONITOR_INPUT permission to be able to set INPUT_FEATURE_SPY.
*/
private int sanitizeSpyWindow(int inputFeatures, String windowName, int callingUid,
int callingPid) {
if ((inputFeatures & INPUT_FEATURE_SPY) == 0) {
return inputFeatures;
}
final int permissionResult = mContext.checkPermission(
permission.MONITOR_INPUT, callingPid, callingUid);
if (permissionResult != PackageManager.PERMISSION_GRANTED) {
throw new IllegalArgumentException("Cannot use INPUT_FEATURE_SPY from '" + windowName
+ "' because it doesn't the have MONITOR_INPUT permission");
}
return inputFeatures;
}
直設INPUT_FEATURE_SPY亦需MONITOR_INPUT之權。
3.3 方術/靜態對映: InputConfigAdapter.INPUT_FEATURE_TO_CONFIG_MAP
檔: frameworks/base/services/core/java/com/android/server/wm/InputConfigAdapter.java
private static final List<FlagMapping> INPUT_FEATURE_TO_CONFIG_MAP = List.of(
new FlagMapping(
LayoutParams.INPUT_FEATURE_NO_INPUT_CHANNEL,
InputConfig.NO_INPUT_CHANNEL, false /* inverted */),
new FlagMapping(
LayoutParams.INPUT_FEATURE_DISABLE_USER_ACTIVITY,
InputConfig.DISABLE_USER_ACTIVITY, false /* inverted */),
new FlagMapping(
LayoutParams.INPUT_FEATURE_SPY,
InputConfig.SPY, false /* inverted */));
是故Java層LayoutParams.INPUT_FEATURE_SPY終將化為原生/輸入層可見之形。InputConfig.SPY。
3.4 示例: UdfpsControllerOverlay.coreLayoutParams
文件: frameworks/base/packages/SystemUI/src/com/android/systemui/biometrics/UdfpsControllerOverlay.kt
private val coreLayoutParams = WindowManager.LayoutParams(
WindowManager.LayoutParams.TYPE_NAVIGATION_BAR_PANEL,
0 /* flags set in computeLayoutParams() */,
PixelFormat.TRANSLUCENT
).apply {
privateFlags = WindowManager.LayoutParams.PRIVATE_FLAG_TRUSTED_OVERLAY
if (featureFlags.isEnabled(Flags.UDFPS_NEW_TOUCH_DETECTION)) {
inputFeatures = WindowManager.LayoutParams.INPUT_FEATURE_SPY
}
}
此乃将既有系统 overlay 窗设为 spy 窗之例。宜于屏下指纹此般“overlay 自需监听触控,然不欲吞没寻常窗口事件”之境。
4. Native 侧之安全约束
方法: InputDispatcher::setInputWindowsLocked(...)
文件: frameworks/native/services/inputflinger/dispatcher/InputDispatcher.cpp
// Ensure all spy windows are trusted overlays
LOG_ALWAYS_FATAL_IF(info.isSpy() &&
!info.inputConfig.test(
WindowInfo::InputConfig::TRUSTED_OVERLAY),
"%s has feature SPY, but is not a trusted overlay.",
window->getName().c_str());
native 侧强令所有 spy 窗必兼为 TRUSTED_OVERLAY。此乃安全界:spy 窗得监听触控流,然不可予寻常应用窗。
5. InputDispatcher 如何择寻常窗与 Spy Window
第五一法:InputDispatcher::findTouchedWindowAtLocked(...)
之文件:frameworks/native/services/inputflinger/dispatcher/InputDispatcher.cpp
std::pair<sp<WindowInfoHandle>, std::vector<InputTarget>>
InputDispatcher::findTouchedWindowAtLocked(int32_t displayId, float x, float y, bool isStylus,
bool ignoreDragWindow) const {
// Traverse windows from front to back to find touched window.
const auto& windowHandles = getWindowHandlesLocked(displayId);
for (const sp<WindowInfoHandle>& windowHandle : windowHandles) {
const WindowInfo& info = *windowHandle->getInfo();
if (!info.isSpy() &&
windowAcceptsTouchAt(info, displayId, x, y, isStylus, getTransformLocked(displayId))) {
return {windowHandle, outsideTargets};
}
}
return {nullptr, {}};
}
凡寻常触控目标,必明除之 spy window:
!info.isSpy()
是故 spy window 不得为常序前景触控之标。
第五二法:InputDispatcher::findTouchedSpyWindowsAtLocked(...)
之文件:frameworks/native/services/inputflinger/dispatcher/InputDispatcher.cpp
std::vector<sp<WindowInfoHandle>> InputDispatcher::findTouchedSpyWindowsAtLocked(
int32_t displayId, float x, float y, bool isStylus) const {
// Traverse windows from front to back and gather the touched spy windows.
std::vector<sp<WindowInfoHandle>> spyWindows;
const auto& windowHandles = getWindowHandlesLocked(displayId);
for (const sp<WindowInfoHandle>& windowHandle : windowHandles) {
const WindowInfo& info = *windowHandle->getInfo();
if (!windowAcceptsTouchAt(info, displayId, x, y, isStylus, getTransformLocked(displayId))) {
continue;
}
if (!info.isSpy()) {
// The first touched non-spy window was found, so return the spy windows touched so far.
return spyWindows;
}
spyWindows.push_back(windowHandle);
}
return spyWindows;
}
此法显 spy window 之 Z-序之则:
- 自上而下遍历诸窗。
- 所中 spy window 皆当集之。
- 一旦遇首所中非 spy window,即止而返所已集之 spy windows。
- 故非 spy window 之下之 spy window 不得受此事件。
5.3 法:InputDispatcher::findTouchedWindowTargetsLocked(...)
文:frameworks/native/services/inputflinger/dispatcher/InputDispatcher.cpp
auto [newTouchedWindowHandle, outsideTargets] =
findTouchedWindowAtLocked(displayId, x, y, isStylus);
std::vector<sp<WindowInfoHandle>> newTouchedWindows =
findTouchedSpyWindowsAtLocked(displayId, x, y, isStylus);
if (newTouchedWindowHandle != nullptr) {
// Process the foreground window first so that it is the first to receive the event.
newTouchedWindows.insert(newTouchedWindows.begin(), newTouchedWindowHandle);
}
for (const sp<WindowInfoHandle>& windowHandle : newTouchedWindows) {
ftl::Flags<InputTarget::Flags> targetFlags = InputTarget::Flags::DISPATCH_AS_IS;
if (canReceiveForegroundTouches(*windowHandle->getInfo())) {
targetFlags |= InputTarget::Flags::FOREGROUND;
}
tempTouchState.addOrUpdateWindow(windowHandle, targetFlags, entry.deviceId, pointerIds,
isDownOrPointerDown
? std::make_optional(entry.eventTime)
: std::nullopt);
}
调用之序乃:
findTouchedWindowTargetsLocked()
-> findTouchedWindowAtLocked() // 找真正的 foreground window,排除 spy
-> findTouchedSpyWindowsAtLocked() // 找 foreground window 上方的 spy windows
-> insert foreground at begin // foreground 优先分发
-> addOrUpdateWindow(...) // 都加入 TouchState
由是 spy window 得受同 pointer stream,然不更 foreground window 之择。
6. Spy Window 何以非 Foreground Touch Target
法:canReceiveForegroundTouches(...)
文:frameworks/native/services/inputflinger/dispatcher/InputDispatcher.cpp
bool canReceiveForegroundTouches(const WindowInfo& info) {
// A non-touchable window can still receive touch events (e.g. in the case of
// STYLUS_INTERCEPTOR), so prevent such windows from receiving foreground events for touches.
return !info.inputConfig.test(gui::WindowInfo::InputConfig::NOT_TOUCHABLE) && !info.isSpy();
}
spy window 可受事件之副,然不携 InputTarget::Flags::FOREGROUND。此保系统手势监听不毁寻常 app 之常规触摸目标之择。
7. 盗指之变:自旁听而主之
7.1 法:InputMonitor.pilferPointers()
文:frameworks/base/core/java/android/view/InputMonitor.java
/**
* Takes all of the current pointer events streams that are currently being sent to this
* monitor and generates appropriate cancellations for the windows that would normally get
* them.
*/
public void pilferPointers() {
try {
mHost.pilferPointers();
} catch (RemoteException e) {
e.rethrowFromSystemServer();
}
}
此乃 Java InputMonitor所显之抢夺之接口也
7.2 法:InputManagerService.InputMonitorHost.pilferPointers()
文:frameworks/base/services/core/java/com/android/server/input/InputManagerService.java
private final class InputMonitorHost extends IInputMonitorHost.Stub {
private final IBinder mInputChannelToken;
@Override
public void pilferPointers() {
mNative.pilferPointers(mInputChannelToken);
}
}
Java 层之调用,将入于 natvie InputDispatcher::pilferPointers(...)
7.3 法:InputDispatcher::pilferPointersLocked(...)
文:frameworks/native/services/inputflinger/dispatcher/InputDispatcher.cpp
status_t InputDispatcher::pilferPointersLocked(const sp<IBinder>& token) {
auto [statePtr, windowPtr, displayId] = findTouchStateWindowAndDisplayLocked(token);
TouchState& state = *statePtr;
TouchedWindow& window = *windowPtr;
// Send cancel events to all the input channels we're stealing from.
CancelationOptions options(CancelationOptions::Mode::CANCEL_POINTER_EVENTS,
"input channel stole pointer stream");
std::bitset<MAX_POINTER_ID + 1> pointerIds = window.getTouchingPointers(deviceId);
options.pointerIds = pointerIds;
for (const TouchedWindow& w : state.windows) {
const std::shared_ptr<InputChannel> channel =
getInputChannelLocked(w.windowHandle->getToken());
if (channel != nullptr && channel->getConnectionToken() != token) {
synthesizeCancelationEventsForInputChannelLocked(channel, options);
}
}
// Prevent the gesture from being sent to any other windows.
window.addPilferingPointers(deviceId, pointerIds);
state.cancelPointersForWindowsExcept(deviceId, pointerIds, token);
return OK;
}
行止之要:
- 觅得调用者 token 所应之现触之窗也
- 取此窗所受 pointer ids。
- 为其他受此 pointer 之窗合成
ACTION_CANCEL。 - 将此 pointer 标为 pilfering。
- 后继同 pointer stream 惟续予 pilfering 窗。
7.4 法:InputDispatcher::findTouchedWindowTargetsLocked(...)
文:frameworks/native/services/inputflinger/dispatcher/InputDispatcher.cpp
// If a window is already pilfering some pointers, give it this new pointer as well and
// make it pilfering. This will prevent other non-spy windows from getting this pointer,
// which is a specific behaviour that we want.
const int32_t pointerId = entry.pointerProperties[pointerIndex].id;
for (TouchedWindow& touchedWindow : tempTouchState.windows) {
if (touchedWindow.hasTouchingPointer(entry.deviceId, pointerId) &&
touchedWindow.hasPilferingPointers(entry.deviceId)) {
touchedWindow.addPilferingPointer(entry.deviceId, pointerId);
}
}
// Restrict all pilfered pointers to the pilfering windows.
tempTouchState.cancelPointersForNonPilferingWindows();
此段治多指之境:若 spy 窗已 pilfer pointer 之部,新坠而亦中其 pointer 者,续归其掌。
8. 用境与真码
8.1 返手势
法:EdgeBackGestureHandler.onInputEvent(...)内理
文:frameworks/base/packages/SystemUI/src/com/android/systemui/navigationbar/gestural/EdgeBackGestureHandler.java
} else if (dx > dy && dx > mTouchSlop) {
if (mAllowGesture) {
mThresholdCrossed = true;
// Capture inputs
mInputMonitor.pilferPointers();
mInputEventReceiver.setBatchingEnabled(true);
}
}
呼链:
SystemUI EdgeBackGestureHandler
-> InputManager.monitorGestureInput(...)
-> InputMonitor receives pointer stream through spy window
-> gesture crosses threshold
-> InputMonitor.pilferPointers()
-> InputDispatcher sends ACTION_CANCEL to app
-> remaining MOVE/UP go to SystemUI
宜境:初不可阻 app,否则边鄙常击将隳;惟确为返势,乃攫之。
8.2 剪贴板叠外点击
法:ClipboardOverlayController.monitorOutsideTouches()
文:frameworks/base/packages/SystemUI/src/com/android/systemui/clipboardoverlay/ClipboardOverlayController.java
private void monitorOutsideTouches() {
InputManager inputManager = mContext.getSystemService(InputManager.class);
mInputMonitor = inputManager.monitorGestureInput("clipboard overlay", 0);
mInputEventReceiver = new InputEventReceiver(
mInputMonitor.getInputChannel(), Looper.getMainLooper()) {
@Override
public void onInputEvent(InputEvent event) {
if (event instanceof MotionEvent) {
MotionEvent motionEvent = (MotionEvent) event;
if (motionEvent.getActionMasked() == MotionEvent.ACTION_DOWN) {
if (!mView.isInTouchRegion(
(int) motionEvent.getRawX(), (int) motionEvent.getRawY())) {
animateOut();
}
}
}
}
};
}
宜境:叠欲知用户是否击外域,以闭 UI,然初毋吞 app 触。
8.3 UDFPS 叠
法/属:UdfpsControllerOverlay.coreLayoutParams
文:frameworks/base/packages/SystemUI/src/com/android/systemui/biometrics/UdfpsControllerOverlay.kt
if (featureFlags.isEnabled(Flags.UDFPS_NEW_TOUCH_DETECTION)) {
inputFeatures = WindowManager.LayoutParams.INPUT_FEATURE_SPY
}
宜于场景:既存系统之overlay窗口,需监听触控,且不碍底层窗口之寻常输入。
9. 与monitorInput全局监控之别
9.1 方法: InputManagerService.monitorInput(...)
文件: frameworks/base/services/core/java/com/android/server/input/InputManagerService.java
/**
* Creates an input channel that will receive all input from the input dispatcher.
*/
public InputChannel monitorInput(String inputChannelName, int displayId) {
Objects.requireNonNull(inputChannelName, "inputChannelName not be null");
if (displayId < Display.DEFAULT_DISPLAY) {
throw new IllegalArgumentException("displayId must >= 0.");
}
return mNative.createInputMonitor(displayId, inputChannelName, Binder.getCallingPid());
}
此乃旧式全局监控路径,所返者非InputChannel,乃android.view.InputMonitor。
9.2 方法: InputDispatcher::createInputMonitor(...)
文件: frameworks/native/services/inputflinger/dispatcher/InputDispatcher.cpp
Result<std::unique_ptr<InputChannel>> InputDispatcher::createInputMonitor(int32_t displayId,
const std::string& name,
gui::Pid pid) {
openInputChannelPair(name, serverChannel, clientChannel);
std::shared_ptr<Connection> connection =
std::make_shared<Connection>(serverChannel, /*monitor=*/true, mIdGenerator);
mConnectionsByToken.emplace(token, connection);
mGlobalMonitorsByDisplay[displayId].emplace_back(serverChannel, pid);
mLooper->addFd(fd, 0, ALOOPER_EVENT_INPUT, ...);
return clientChannel;
}
此将channel置入mGlobalMonitorsByDisplay,不设InputWindowHandle,亦不入窗口Z-order。
第九节 方法:InputDispatcher::addGlobalMonitoringTargetsLocked(...)
文件:frameworks/native/services/inputflinger/dispatcher/InputDispatcher.cpp
void InputDispatcher::addGlobalMonitoringTargetsLocked(std::vector<InputTarget>& inputTargets,
int32_t displayId) {
auto monitorsIt = mGlobalMonitorsByDisplay.find(displayId);
if (monitorsIt == mGlobalMonitorsByDisplay.end()) return;
for (const Monitor& monitor : selectResponsiveMonitorsLocked(monitorsIt->second)) {
InputTarget target;
target.inputChannel = monitor.inputChannel;
target.flags = InputTarget::Flags::DISPATCH_AS_IS;
inputTargets.push_back(target);
}
}
全局监视器乃于键/动作调度之际,额外增设之全局目标,不与窗口命中测试相干。
第九节 示例:DisplayContent创建PointerEventDispatcher
方法/构造流程:DisplayContent.DisplayContent(...)
文件:frameworks/base/services/core/java/com/android/server/wm/DisplayContent.java
final InputChannel inputChannel = mWmService.mInputManager.monitorInput(
"PointerEventDispatcher" + mDisplayId, mDisplayId);
mPointerEventDispatcher = new PointerEventDispatcher(inputChannel);
适宜之境:WMS内部欲察display上之pointer event,如任务轻触侦测、鼠标位置追踪等。
第十节 对比总括
| 机制 | 本质 | 是否为窗口 | 受 Z-order 影否? | 受 touchable region 影否? | 适于 pilfer 乎? | 典型之境何在? |
|---|---|---|---|---|---|---|
InputConfig.SPY | InputWindow 之配置若何? | 是 | 是 | 是 | 是 | 系统可信 overlay 旁听触摸 |
monitorGestureInput() 返回之 InputMonitor | Java API,其下实为 GestureMonitorSpyWindow | 是 | 是 | 是 | 者, | 所返手势、Shell 手势、overlay 外部点击 |
monitorInput()之全局监控也, | 之原生全局监控信道也, | 否, | 否, | 否, | 此通常不适于 | WMS 之内部全局 pointer 观察, |
简化之选:
- 已有系统 overlay 窗口,欲旁听触摸者,当用
INPUT_FEATURE_SPY。 - 之组件,当监听显示上之手势,或可于识别后攫取之:当用
InputManager.monitorGestureInput()。 - WMS于内,需观全局输入,不须问窗口命中之谓:当用
monitorInput()global monitor。












