[SOLVED] Android goldfish kernel IOCTL system call hook kernel panic

Issue

I modified some system calls. Except "ioctl", all of them are working good. When i try to "rmmod" on my kernel module, i see kernel panic. I'm using android-goldfish-3.4 kernel in the android emulator.

Run emulator with custom kernel:

emulator @nexus4 -kernel goldfish/arch/arm/boot/zImage -wipe-data -show-kernel

In kernel module:

void **sys_call_table;
asmlinkage long(*original_call_ioctl)(unsigned int, unsigned int, unsigned long);

asmlinkage long our_sys_ioctl(unsigned int fd, unsigned int cmd, unsigned long arg){
    return original_call_ioctl(fd, cmd, arg);
}


static int __init my_trap_init(void){
    sys_call_table = (void*)0xc000d984;

    original_call_ioctl = sys_call_table[__NR_ioctl];
    sys_call_table[__NR_ioctl] = our_sys_ioctl;

    return 0;
}

static void __exit my_trap_exit(void){
    sys_call_table[__NR_ioctl] = original_call_ioctl;
}

module_init(my_trap_init);
module_exit(my_trap_exit);

Install module (running without error):

insmod trapcall.ko

Remove module (kernel panic):

rmmod trapcall.ko

After remove:

Unable to handle kernel paging request at virtual address bf000010
pgd = ed484000
[bf000010] *pgd=2d012811, *pte=00000000, *ppte=00000000
Internal error: Oops: 80000017 [#1] PREEMPT ARM
Modules linked in: [last unloaded: trapcall]
CPU: 0    Tainted: P           O  (3.4.67-g2011b1c #6)
PC is at 0xbf000010
LR is at sys_ioctl+0x64/0x6c
pc : [<bf000010>]    lr : [<c00c4e78>]    psr: 20000013
sp : ecd55fa0  ip : 00000000  fp : 9eae0db0
r10: 00000000  r9 : ecd54000  r8 : c000d984
r7 : 00000036  r6 : acc94140  r5 : acc941a0  r4 : acc94170
r3 : 00000001  r2 : 0000000f  r1 : 00000000  r0 : 00000000
Flags: nzCv  IRQs on  FIQs on  Mode SVC_32  ISA ARM  Segment user
Control: 10c53c7d  Table: 2d484059  DAC: 00000015

LR: 0xc00c4df8:
4df8  eaffff6f c0045877 c0045878 c020660b 04924924 40305828 fffffdfd e92d41f3
4e18  e1a06001 e28d1004 e1a07000 e1a05002 ebffc654 e2504000 03e08008 0a00000f
4e38  e1a01006 e1a02005 eb0332a0 e2508000 1a000005 e1a00004 e1a01007 e1a02006
4e58  e1a03005 ebfffe77 e1a08000 e59d3004 e3530000 0a000001 e1a00004 ebffc701
4e78  e1a00008 e8bd81fc e92d43f8 e1a04000 e282600f e5943008 e3c66003 e3e00015
4e98  e1560003 e1a05002 e59dc020 e1cd82d8 e584000c c8bd83f8 e1a02008 e3a03000
4eb8  e1530009 01520008 e1a00008 13e0004a 1584000c 18bd83f8 e5943004 e3530000
4ed8  0a000004 e2832004 e3a03000 e482c000 e3530000 1a00002f e5947000 e3a03000

SP: 0xecd55f20:
5f20  ffffffda 00000001 00000000 00000000 ed93a190 edd8af18 ffff6201 0000000b
5f40  00000000 bf000010 20000013 ffffffff ecd55f8c c000d4f8 00000000 00000000
5f60  0000000f 00000001 acc94170 acc941a0 acc94140 00000036 c000d984 ecd54000
5f80  00000000 9eae0db0 00000000 ecd55fa0 c00c4e78 bf000010 20000013 ffffffff
5fa0  9eae0cc8 c000d800 acc94170 acc941a0 0000000f c0186201 9eae0cd0 9eae0cc8
5fc0  acc94170 acc941a0 acc94140 00000036 00000001 acd18308 af243f0d 9eae0db0
5fe0  acecd6a0 9eae0cb8 af27f9cd af267884 60000010 0000000f 9b1647f0 e3c8f8df
6000  ed751000 ee014800 00000050 ecd56050 0000000b 00000003 deca0000 ffffffff

R8: 0xc000d904:
d904  e31a0c03 1a000008 e3570d06 e24fef46 3798f107 e28d1008 e3a08000 e357080f
d924  e2270000 2a000bbd ea00a118 e1a02007 e28d1008 e3a00000 eb00068f e28fe018
d944  e1a07000 e28d1008 e3570d06 3891007f 388d0030 3798f107 eaffffee e5ad0008
d964  e1a02007 e1a0100d e3a00001 eb000682 eaffffb6 e320f000 e320f000 c04cdd74
d984  c0029558 c001ce1c c000dfac c00b59b8 c00b5a1c c00b4cf4 c00b3e78 c0035d94
d9a4  c00b4d1c c00c2f44 c00c2d8c c000dfbc c00b473c c0035d94 c00c2c8c c00b493c
d9c4  c0053828 c0035d94 c0035d94 c00b537c c0025c90 c00d09ec c0035d94 c0053898
d9e4  c0053c70 c0035d94 c0022bec c0035d94 c0035d94 c002a728 c0035d94 c0035d94

R9: 0xecd53f80:
3f80  6e1c3a3f 6e1c4a3f 6e1c5a3f 6e1c6a3f 6e1c7a3f 00000000 00000000 00000000
3fa0  00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
3fc0  00000000 00000000 00000000 6e1f6a3f 00000000 00000000 00000000 00000000
3fe0  00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
4000  00000000 00000002 00000000 ed4dfc00 c04cea6c 00000000 00000015 edb34e00
4020  ed4dfc00 c04d1c78 edab9540 ec436800 00000000 ecd54000 ecd55e2c ecd55df8
4040  c036d728 00000000 00000000 00000000 00000000 00000000 01010000 01000000
4060  9eae0db0 00000000 00000000 00000000 00000000 00000000 00000000 00000000
Process Binder_1 (pid: 365, stack limit = 0xecd542e8)
Stack: (0xecd55fa0 to 0xecd56000)
5fa0: 9eae0cc8 c000d800 acc94170 acc941a0 0000000f c0186201 9eae0cd0 9eae0cc8
5fc0: acc94170 acc941a0 acc94140 00000036 00000001 acd18308 af243f0d 9eae0db0
5fe0: acecd6a0 9eae0cb8 af27f9cd af267884 60000010 0000000f 9b1647f0 e3c8f8df
[<c00c4e78>] (sys_ioctl+0x64/0x6c) from [<9eae0cc8>] (0x9eae0cc8)
Code: bad PC value
---[ end trace a19740dedb23f37c ]---
Kernel panic - not syncing: Fatal exception

I want to say again, other system call hookings are working great. Just failing with "ioctl" call.

Solution

Call stack address sys_ioctl+0x64/0x6c looks like ret command from the sys_ioctl.

Possible workflow which leads to kernel panic:

1. Someone calls ioctl. According to syscall table it resolves into our_sys_ioctl.

2. Function our_sys_ioctl calls sys_ioctl via pointer original_call_ioctl. Return address is stored in the stack, as usual.

3. sys_ioctl at the end calls filesystem-specific .ioctl function, which may require a long time for execute(e.g., wait).

4. Another process calls rmmod for your module. It frees code of our_sys_ioctl among other things.

5. Filesystem specific .ioctl function returns into sys_ioctl. sys_ioctl executes its last ret instruction which jumps inside former our_sys_ioctl. But its code is already freed, so this leads to undefined behavior, which may cause kernel panic.

Such race condition may be eliminated by using tail call, when address, stored at step 2 on the stack, is actually return address of our_sys_ioctl.

But race between calling our_sys_ioctl and freeing code of this function is unavoidable. It is better to not unload module which replaces system call at all.

Answered By - Tsyvarev