Gentle introduction xHCI debug capabilities
- published
- reading time
- 5 minutes
Introduction
The XHCI specification has a optional debugging capabilities that is available via the extended capabilities set. This set could be enabled by vendor to pass along additional information around the hardware - especially information that would help in debugging. In this post, we are going to identify a USB controller (on a target) that supports this debug capability and try to enable/verify it on the host side. At last, we are going to also look at the debug capabilites associated with early boot phase of the kernel
Requirements
- USB 3.1 Superspeed cable A-A or A-C cable
- Target with supported xHCI debug capability
- Host machine
Checking if xHCI debug capability is supported
We need to verify if the target machine has xHCI debug capability supported. We can simply find out by peeking into our sysfs device information:
($) find "/sys/devices" -type f -name "dbc"
If this command returns a device, we can be sure that the xHCI controller is present with debug capability.
Enable the xHCI debug capability on target
($) sudo bash -c "echo enable > /sys/devices/pci0000:00/0000:00:0d.0/dbc"
You should expect the following confirmation via dmesg on the target:
[ 3400.545281] xhci_hcd 0000:04:00.3: DbC connected
[ 3400.905302] xhci_hcd 0000:04:00.3: DbC configured
On the host machine which is connected via the USB 3.1 SuperSpeed cable,
we can verify that the USB host controller recognises the debug device.
Running dmesg on host:
[110351.933236] usb 4-2: new SuperSpeed USB device number 6 using xhci_hcd
[110351.944701] usb 4-2: New USB device found, idVendor=1d6b, idProduct=0010, bcdDevice= 0.10
[110351.944732] usb 4-2: New USB device strings: Mfr=1, Product=2, SerialNumber=3
[110351.944742] usb 4-2: Product: Linux USB Debug Target
[110351.944750] usb 4-2: Manufacturer: Linux Foundation
[110351.944756] usb 4-2: SerialNumber: 0001
[110351.990401] usbcore: registered new interface driver usb_debug
[110351.990438] usbserial: USB Serial support registered for debug
[110351.990452] usbserial: USB Serial support registered for xhci_dbc
[110351.990502] usb_debug 4-2:1.0: xhci_dbc converter detected
[110351.990786] usb 4-2: xhci_dbc converter now attached to ttyUSB0
You can attach any serial console utility to interact with target device using xHCI debug capabilities.
Linux kernel early boot setup
xHCI is also helpful to output early kernel boot stage logs. This phase is the intialization process just after the bootloaders hands over control. To enable early kernel debug, we need to configure the kernel with proper kconfig and pass debug parameters in the kernel commandline.
Build the kernel with following kconfig enabled:
CONFIG_EARLY_PRINTK_USB_XDBC
And for the kernel commandline, append:
earlyprintk=xdbc
Early boot IO remap
Now that you have had gentle introduction to this debugging capability, we are going briefly mention about fixed boot-time mappings known as fixmaps. The early boot has a special set of operations that are carried out before the standard virtual memory and device drivers are active. In this phase, hardware registers are allowed to be accessed via fixed boot time mapping and page tables.
Generally the xHCI debug capability reside under the extended capabilities given by vendor and can be mapped in fixed boot time mappings. However, we have such devices coming up, that have extended capabilites size much bigger than the fixmaps.
In this case, the PCIe memory range would fail to get IO remapped using early_ioremap() call.
kernel: xhci_dbc:early_xdbc_parse_parameter: dbgp_num: 0
kernel: ------------[ cut here ]------------
ay 19 13:42:57 kernel: WARNING: CPU: 0 PID: 0 at mm/early_ioremap.c:139 __early_ioremap+0xae/0x180
kernel: Modules linked in:
kernel: CPU: 0 UID: 0 PID: 0 Comm: swapper Not tainted 6.18.25-valve1-gcab630e7af50 #13 PREEMPT(undef) 4b70872d9de6788b7f2d10dce46ada89e6cd177b
kernel: RIP: 0010:__early_ioremap+0xae/0x180
kernel: Code: 60 ba 3f a0 4c 89 ca 48 81 e3 00 f0 ff ff 48 81 e2 00 f0 ff ff 48 29 d3 48 89 14 24 48 89 da 48 c1 ea 0c 89 d5 83 fa 40 76 04 <0f> 0b eb a2 6b c0 c0 4d 89 ce 41 81 e6 ff 0f 00 00 44 8d b8 ff 05
kernel: RSP: 0000:ffffffff9fa03cb8 EFLAGS: 00010006 ORIG_RAX: 0000000000000000
kernel: RAX: 0000000000000000 RBX: 0000000000100000 RCX: 0000000000100000
kernel: RDX: 0000000000000100 RSI: 0000000000100000 RDI: 0000000080200000
kernel: RBP: 0000000000000100 R08: 0000000000000000 R09: 0000000080200000
kernel: R10: 0000000000000004 R11: ffffffff9fa03ad0 R12: 8000000000000163
kernel: R13: 0000000000000000 R14: 0000000080200000 R15: 0000000000000000
kernel: FS: 0000000000000000(0000) GS:0000000000000000(0000) knlGS:0000000000000000
kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
kernel: CR2: ffff888000000413 CR3: 000000020ed02000 CR4: 00000000000000b0
kernel: Call Trace:
kernel: <TASK>
kernel: ? early_xdbc_parse_parameter+0x32c/0x360
kernel: ? setup_early_printk+0x4f5/0x520
kernel: ? do_early_param+0x44/0x70
kernel: ? parse_args+0x233/0x420
kernel: ? __pfx_do_early_param+0x10/0x10
kernel: ? parse_early_options+0x29/0x30
kernel: ? __pfx_do_early_param+0x10/0x10
kernel: ? parse_early_param+0x64/0xc0
kernel: ? setup_arch+0x542/0xbc0
kernel: ? _printk+0x6b/0x90
kernel: ? start_kernel+0x66/0x9a0
kernel: ? x86_64_start_reservations+0x24/0x30
kernel: ? x86_64_start_kernel+0xcc/0xd0
kernel: ? common_startup_64+0x13e/0x141
kernel: </TASK>
kernel: ---[ end trace 0000000000000000 ]---
And this is the exact place that is triggered at in mm/early_ioremap.c
/*
* Mappings have to fit in the FIX_BTMAP area.
*/
nrpages = size >> PAGE_SHIFT;
if (WARN_ON(nrpages > NR_FIX_BTMAPS))
return NULL;
We shall call this issue as extra extended capabilities issue for lack of a better work!
Solution to extra extended capbilities for xHCI
Now that we are seeing such hardware in the market where the fixmaps can be a limiting factor, we have to find a solution for it. This has been a focus of my work and after a few iterations, we have this patch for review! Feel free for review comments or follow it for curiousity. There might be few more followups on this particular approach – so stay tuned for that.
Thank you for reading!