0001 =====================
0002 DAWR issues on POWER9
0003 =====================
0004
0005 On older POWER9 processors, the Data Address Watchpoint Register (DAWR) can
0006 cause a checkstop if it points to cache inhibited (CI) memory. Currently Linux
0007 has no way to distinguish CI memory when configuring the DAWR, so on affected
0008 systems, the DAWR is disabled.
0009
0010 Affected processor revisions
0011 ============================
0012
0013 This issue is only present on processors prior to v2.3. The revision can be
0014 found in /proc/cpuinfo::
0015
0016 processor : 0
0017 cpu : POWER9, altivec supported
0018 clock : 3800.000000MHz
0019 revision : 2.3 (pvr 004e 1203)
0020
0021 On a system with the issue, the DAWR is disabled as detailed below.
0022
0023 Technical Details:
0024 ==================
0025
0026 DAWR has 6 different ways of being set.
0027 1) ptrace
0028 2) h_set_mode(DAWR)
0029 3) h_set_dabr()
0030 4) kvmppc_set_one_reg()
0031 5) xmon
0032
0033 For ptrace, we now advertise zero breakpoints on POWER9 via the
0034 PPC_PTRACE_GETHWDBGINFO call. This results in GDB falling back to
0035 software emulation of the watchpoint (which is slow).
0036
0037 h_set_mode(DAWR) and h_set_dabr() will now return an error to the
0038 guest on a POWER9 host. Current Linux guests ignore this error, so
0039 they will silently not get the DAWR.
0040
0041 kvmppc_set_one_reg() will store the value in the vcpu but won't
0042 actually set it on POWER9 hardware. This is done so we don't break
0043 migration from POWER8 to POWER9, at the cost of silently losing the
0044 DAWR on the migration.
0045
0046 For xmon, the 'bd' command will return an error on P9.
0047
0048 Consequences for users
0049 ======================
0050
0051 For GDB watchpoints (ie 'watch' command) on POWER9 bare metal , GDB
0052 will accept the command. Unfortunately since there is no hardware
0053 support for the watchpoint, GDB will software emulate the watchpoint
0054 making it run very slowly.
0055
0056 The same will also be true for any guests started on a POWER9
0057 host. The watchpoint will fail and GDB will fall back to software
0058 emulation.
0059
0060 If a guest is started on a POWER8 host, GDB will accept the watchpoint
0061 and configure the hardware to use the DAWR. This will run at full
0062 speed since it can use the hardware emulation. Unfortunately if this
0063 guest is migrated to a POWER9 host, the watchpoint will be lost on the
0064 POWER9. Loads and stores to the watchpoint locations will not be
0065 trapped in GDB. The watchpoint is remembered, so if the guest is
0066 migrated back to the POWER8 host, it will start working again.
0067
0068 Force enabling the DAWR
0069 =======================
0070 Kernels (since ~v5.2) have an option to force enable the DAWR via::
0071
0072 echo Y > /sys/kernel/debug/powerpc/dawr_enable_dangerous
0073
0074 This enables the DAWR even on POWER9.
0075
0076 This is a dangerous setting, USE AT YOUR OWN RISK.
0077
0078 Some users may not care about a bad user crashing their box
0079 (ie. single user/desktop systems) and really want the DAWR. This
0080 allows them to force enable DAWR.
0081
0082 This flag can also be used to disable DAWR access. Once this is
0083 cleared, all DAWR access should be cleared immediately and your
0084 machine once again safe from crashing.
0085
0086 Userspace may get confused by toggling this. If DAWR is force
0087 enabled/disabled between getting the number of breakpoints (via
0088 PTRACE_GETHWDBGINFO) and setting the breakpoint, userspace will get an
0089 inconsistent view of what's available. Similarly for guests.
0090
0091 For the DAWR to be enabled in a KVM guest, the DAWR needs to be force
0092 enabled in the host AND the guest. For this reason, this won't work on
0093 POWERVM as it doesn't allow the HCALL to work. Writes of 'Y' to the
0094 dawr_enable_dangerous file will fail if the hypervisor doesn't support
0095 writing the DAWR.
0096
0097 To double check the DAWR is working, run this kernel selftest:
0098
0099 tools/testing/selftests/powerpc/ptrace/ptrace-hwbreak.c
0100
0101 Any errors/failures/skips mean something is wrong.
