Back to home page

OSCL-LXR

 
 

    


0001 ===================================================
0002 Scalable Matrix Extension support for AArch64 Linux
0003 ===================================================
0004 
0005 This document outlines briefly the interface provided to userspace by Linux in
0006 order to support use of the ARM Scalable Matrix Extension (SME).
0007 
0008 This is an outline of the most important features and issues only and not
0009 intended to be exhaustive.  It should be read in conjunction with the SVE
0010 documentation in sve.rst which provides details on the Streaming SVE mode
0011 included in SME.
0012 
0013 This document does not aim to describe the SME architecture or programmer's
0014 model.  To aid understanding, a minimal description of relevant programmer's
0015 model features for SME is included in Appendix A.
0016 
0017 
0018 1.  General
0019 -----------
0020 
0021 * PSTATE.SM, PSTATE.ZA, the streaming mode vector length, the ZA
0022   register state and TPIDR2_EL0 are tracked per thread.
0023 
0024 * The presence of SME is reported to userspace via HWCAP2_SME in the aux vector
0025   AT_HWCAP2 entry.  Presence of this flag implies the presence of the SME
0026   instructions and registers, and the Linux-specific system interfaces
0027   described in this document.  SME is reported in /proc/cpuinfo as "sme".
0028 
0029 * Support for the execution of SME instructions in userspace can also be
0030   detected by reading the CPU ID register ID_AA64PFR1_EL1 using an MRS
0031   instruction, and checking that the value of the SME field is nonzero. [3]
0032 
0033   It does not guarantee the presence of the system interfaces described in the
0034   following sections: software that needs to verify that those interfaces are
0035   present must check for HWCAP2_SME instead.
0036 
0037 * There are a number of optional SME features, presence of these is reported
0038   through AT_HWCAP2 through:
0039 
0040         HWCAP2_SME_I16I64
0041         HWCAP2_SME_F64F64
0042         HWCAP2_SME_I8I32
0043         HWCAP2_SME_F16F32
0044         HWCAP2_SME_B16F32
0045         HWCAP2_SME_F32F32
0046         HWCAP2_SME_FA64
0047 
0048   This list may be extended over time as the SME architecture evolves.
0049 
0050   These extensions are also reported via the CPU ID register ID_AA64SMFR0_EL1,
0051   which userspace can read using an MRS instruction.  See elf_hwcaps.txt and
0052   cpu-feature-registers.txt for details.
0053 
0054 * Debuggers should restrict themselves to interacting with the target via the
0055   NT_ARM_SVE, NT_ARM_SSVE and NT_ARM_ZA regsets.  The recommended way
0056   of detecting support for these regsets is to connect to a target process
0057   first and then attempt a
0058 
0059         ptrace(PTRACE_GETREGSET, pid, NT_ARM_<regset>, &iov).
0060 
0061 * Whenever ZA register values are exchanged in memory between userspace and
0062   the kernel, the register value is encoded in memory as a series of horizontal
0063   vectors from 0 to VL/8-1 stored in the same endianness invariant format as is
0064   used for SVE vectors.
0065 
0066 * On thread creation TPIDR2_EL0 is preserved unless CLONE_SETTLS is specified,
0067   in which case it is set to 0.
0068 
0069 2.  Vector lengths
0070 ------------------
0071 
0072 SME defines a second vector length similar to the SVE vector length which is
0073 controls the size of the streaming mode SVE vectors and the ZA matrix array.
0074 The ZA matrix is square with each side having as many bytes as a streaming
0075 mode SVE vector.
0076 
0077 
0078 3.  Sharing of streaming and non-streaming mode SVE state
0079 ---------------------------------------------------------
0080 
0081 It is implementation defined which if any parts of the SVE state are shared
0082 between streaming and non-streaming modes.  When switching between modes
0083 via software interfaces such as ptrace if no register content is provided as
0084 part of switching no state will be assumed to be shared and everything will
0085 be zeroed.
0086 
0087 
0088 4.  System call behaviour
0089 -------------------------
0090 
0091 * On syscall PSTATE.ZA is preserved, if PSTATE.ZA==1 then the contents of the
0092   ZA matrix are preserved.
0093 
0094 * On syscall PSTATE.SM will be cleared and the SVE registers will be handled
0095   as per the standard SVE ABI.
0096 
0097 * Neither the SVE registers nor ZA are used to pass arguments to or receive
0098   results from any syscall.
0099 
0100 * On process creation (eg, clone()) the newly created process will have
0101   PSTATE.SM cleared.
0102 
0103 * All other SME state of a thread, including the currently configured vector
0104   length, the state of the PR_SME_VL_INHERIT flag, and the deferred vector
0105   length (if any), is preserved across all syscalls, subject to the specific
0106   exceptions for execve() described in section 6.
0107 
0108 
0109 5.  Signal handling
0110 -------------------
0111 
0112 * Signal handlers are invoked with streaming mode and ZA disabled.
0113 
0114 * A new signal frame record za_context encodes the ZA register contents on
0115   signal delivery. [1]
0116 
0117 * The signal frame record for ZA always contains basic metadata, in particular
0118   the thread's vector length (in za_context.vl).
0119 
0120 * The ZA matrix may or may not be included in the record, depending on
0121   the value of PSTATE.ZA.  The registers are present if and only if:
0122   za_context.head.size >= ZA_SIG_CONTEXT_SIZE(sve_vq_from_vl(za_context.vl))
0123   in which case PSTATE.ZA == 1.
0124 
0125 * If matrix data is present, the remainder of the record has a vl-dependent
0126   size and layout.  Macros ZA_SIG_* are defined [1] to facilitate access to
0127   them.
0128 
0129 * The matrix is stored as a series of horizontal vectors in the same format as
0130   is used for SVE vectors.
0131 
0132 * If the ZA context is too big to fit in sigcontext.__reserved[], then extra
0133   space is allocated on the stack, an extra_context record is written in
0134   __reserved[] referencing this space.  za_context is then written in the
0135   extra space.  Refer to [1] for further details about this mechanism.
0136 
0137 
0138 5.  Signal return
0139 -----------------
0140 
0141 When returning from a signal handler:
0142 
0143 * If there is no za_context record in the signal frame, or if the record is
0144   present but contains no register data as described in the previous section,
0145   then ZA is disabled.
0146 
0147 * If za_context is present in the signal frame and contains matrix data then
0148   PSTATE.ZA is set to 1 and ZA is populated with the specified data.
0149 
0150 * The vector length cannot be changed via signal return.  If za_context.vl in
0151   the signal frame does not match the current vector length, the signal return
0152   attempt is treated as illegal, resulting in a forced SIGSEGV.
0153 
0154 
0155 6.  prctl extensions
0156 --------------------
0157 
0158 Some new prctl() calls are added to allow programs to manage the SME vector
0159 length:
0160 
0161 prctl(PR_SME_SET_VL, unsigned long arg)
0162 
0163     Sets the vector length of the calling thread and related flags, where
0164     arg == vl | flags.  Other threads of the calling process are unaffected.
0165 
0166     vl is the desired vector length, where sve_vl_valid(vl) must be true.
0167 
0168     flags:
0169 
0170         PR_SME_VL_INHERIT
0171 
0172             Inherit the current vector length across execve().  Otherwise, the
0173             vector length is reset to the system default at execve().  (See
0174             Section 9.)
0175 
0176         PR_SME_SET_VL_ONEXEC
0177 
0178             Defer the requested vector length change until the next execve()
0179             performed by this thread.
0180 
0181             The effect is equivalent to implicit execution of the following
0182             call immediately after the next execve() (if any) by the thread:
0183 
0184                 prctl(PR_SME_SET_VL, arg & ~PR_SME_SET_VL_ONEXEC)
0185 
0186             This allows launching of a new program with a different vector
0187             length, while avoiding runtime side effects in the caller.
0188 
0189             Without PR_SME_SET_VL_ONEXEC, the requested change takes effect
0190             immediately.
0191 
0192 
0193     Return value: a nonnegative on success, or a negative value on error:
0194         EINVAL: SME not supported, invalid vector length requested, or
0195             invalid flags.
0196 
0197 
0198     On success:
0199 
0200     * Either the calling thread's vector length or the deferred vector length
0201       to be applied at the next execve() by the thread (dependent on whether
0202       PR_SME_SET_VL_ONEXEC is present in arg), is set to the largest value
0203       supported by the system that is less than or equal to vl.  If vl ==
0204       SVE_VL_MAX, the value set will be the largest value supported by the
0205       system.
0206 
0207     * Any previously outstanding deferred vector length change in the calling
0208       thread is cancelled.
0209 
0210     * The returned value describes the resulting configuration, encoded as for
0211       PR_SME_GET_VL.  The vector length reported in this value is the new
0212       current vector length for this thread if PR_SME_SET_VL_ONEXEC was not
0213       present in arg; otherwise, the reported vector length is the deferred
0214       vector length that will be applied at the next execve() by the calling
0215       thread.
0216 
0217     * Changing the vector length causes all of ZA, P0..P15, FFR and all bits of
0218       Z0..Z31 except for Z0 bits [127:0] .. Z31 bits [127:0] to become
0219       unspecified, including both streaming and non-streaming SVE state.
0220       Calling PR_SME_SET_VL with vl equal to the thread's current vector
0221       length, or calling PR_SME_SET_VL with the PR_SVE_SET_VL_ONEXEC flag,
0222       does not constitute a change to the vector length for this purpose.
0223 
0224     * Changing the vector length causes PSTATE.ZA and PSTATE.SM to be cleared.
0225       Calling PR_SME_SET_VL with vl equal to the thread's current vector
0226       length, or calling PR_SME_SET_VL with the PR_SVE_SET_VL_ONEXEC flag,
0227       does not constitute a change to the vector length for this purpose.
0228 
0229 
0230 prctl(PR_SME_GET_VL)
0231 
0232     Gets the vector length of the calling thread.
0233 
0234     The following flag may be OR-ed into the result:
0235 
0236         PR_SME_VL_INHERIT
0237 
0238             Vector length will be inherited across execve().
0239 
0240     There is no way to determine whether there is an outstanding deferred
0241     vector length change (which would only normally be the case between a
0242     fork() or vfork() and the corresponding execve() in typical use).
0243 
0244     To extract the vector length from the result, bitwise and it with
0245     PR_SME_VL_LEN_MASK.
0246 
0247     Return value: a nonnegative value on success, or a negative value on error:
0248         EINVAL: SME not supported.
0249 
0250 
0251 7.  ptrace extensions
0252 ---------------------
0253 
0254 * A new regset NT_ARM_SSVE is defined for access to streaming mode SVE
0255   state via PTRACE_GETREGSET and  PTRACE_SETREGSET, this is documented in
0256   sve.rst.
0257 
0258 * A new regset NT_ARM_ZA is defined for ZA state for access to ZA state via
0259   PTRACE_GETREGSET and PTRACE_SETREGSET.
0260 
0261   Refer to [2] for definitions.
0262 
0263 The regset data starts with struct user_za_header, containing:
0264 
0265     size
0266 
0267         Size of the complete regset, in bytes.
0268         This depends on vl and possibly on other things in the future.
0269 
0270         If a call to PTRACE_GETREGSET requests less data than the value of
0271         size, the caller can allocate a larger buffer and retry in order to
0272         read the complete regset.
0273 
0274     max_size
0275 
0276         Maximum size in bytes that the regset can grow to for the target
0277         thread.  The regset won't grow bigger than this even if the target
0278         thread changes its vector length etc.
0279 
0280     vl
0281 
0282         Target thread's current streaming vector length, in bytes.
0283 
0284     max_vl
0285 
0286         Maximum possible streaming vector length for the target thread.
0287 
0288     flags
0289 
0290         Zero or more of the following flags, which have the same
0291         meaning and behaviour as the corresponding PR_SET_VL_* flags:
0292 
0293             SME_PT_VL_INHERIT
0294 
0295             SME_PT_VL_ONEXEC (SETREGSET only).
0296 
0297 * The effects of changing the vector length and/or flags are equivalent to
0298   those documented for PR_SME_SET_VL.
0299 
0300   The caller must make a further GETREGSET call if it needs to know what VL is
0301   actually set by SETREGSET, unless is it known in advance that the requested
0302   VL is supported.
0303 
0304 * The size and layout of the payload depends on the header fields.  The
0305   SME_PT_ZA_*() macros are provided to facilitate access to the data.
0306 
0307 * In either case, for SETREGSET it is permissible to omit the payload, in which
0308   case the vector length and flags are changed and PSTATE.ZA is set to 0
0309   (along with any consequences of those changes).  If a payload is provided
0310   then PSTATE.ZA will be set to 1.
0311 
0312 * For SETREGSET, if the requested VL is not supported, the effect will be the
0313   same as if the payload were omitted, except that an EIO error is reported.
0314   No attempt is made to translate the payload data to the correct layout
0315   for the vector length actually set.  It is up to the caller to translate the
0316   payload layout for the actual VL and retry.
0317 
0318 * The effect of writing a partial, incomplete payload is unspecified.
0319 
0320 
0321 8.  ELF coredump extensions
0322 ---------------------------
0323 
0324 * NT_ARM_SSVE notes will be added to each coredump for
0325   each thread of the dumped process.  The contents will be equivalent to the
0326   data that would have been read if a PTRACE_GETREGSET of the corresponding
0327   type were executed for each thread when the coredump was generated.
0328 
0329 * A NT_ARM_ZA note will be added to each coredump for each thread of the
0330   dumped process.  The contents will be equivalent to the data that would have
0331   been read if a PTRACE_GETREGSET of NT_ARM_ZA were executed for each thread
0332   when the coredump was generated.
0333 
0334 
0335 9.  System runtime configuration
0336 --------------------------------
0337 
0338 * To mitigate the ABI impact of expansion of the signal frame, a policy
0339   mechanism is provided for administrators, distro maintainers and developers
0340   to set the default vector length for userspace processes:
0341 
0342 /proc/sys/abi/sme_default_vector_length
0343 
0344     Writing the text representation of an integer to this file sets the system
0345     default vector length to the specified value, unless the value is greater
0346     than the maximum vector length supported by the system in which case the
0347     default vector length is set to that maximum.
0348 
0349     The result can be determined by reopening the file and reading its
0350     contents.
0351 
0352     At boot, the default vector length is initially set to 32 or the maximum
0353     supported vector length, whichever is smaller and supported.  This
0354     determines the initial vector length of the init process (PID 1).
0355 
0356     Reading this file returns the current system default vector length.
0357 
0358 * At every execve() call, the new vector length of the new process is set to
0359   the system default vector length, unless
0360 
0361     * PR_SME_VL_INHERIT (or equivalently SME_PT_VL_INHERIT) is set for the
0362       calling thread, or
0363 
0364     * a deferred vector length change is pending, established via the
0365       PR_SME_SET_VL_ONEXEC flag (or SME_PT_VL_ONEXEC).
0366 
0367 * Modifying the system default vector length does not affect the vector length
0368   of any existing process or thread that does not make an execve() call.
0369 
0370 
0371 Appendix A.  SME programmer's model (informative)
0372 =================================================
0373 
0374 This section provides a minimal description of the additions made by SME to the
0375 ARMv8-A programmer's model that are relevant to this document.
0376 
0377 Note: This section is for information only and not intended to be complete or
0378 to replace any architectural specification.
0379 
0380 A.1.  Registers
0381 ---------------
0382 
0383 In A64 state, SME adds the following:
0384 
0385 * A new mode, streaming mode, in which a subset of the normal FPSIMD and SVE
0386   features are available.  When supported EL0 software may enter and leave
0387   streaming mode at any time.
0388 
0389   For best system performance it is strongly encouraged for software to enable
0390   streaming mode only when it is actively being used.
0391 
0392 * A new vector length controlling the size of ZA and the Z registers when in
0393   streaming mode, separately to the vector length used for SVE when not in
0394   streaming mode.  There is no requirement that either the currently selected
0395   vector length or the set of vector lengths supported for the two modes in
0396   a given system have any relationship.  The streaming mode vector length
0397   is referred to as SVL.
0398 
0399 * A new ZA matrix register.  This is a square matrix of SVLxSVL bits.  Most
0400   operations on ZA require that streaming mode be enabled but ZA can be
0401   enabled without streaming mode in order to load, save and retain data.
0402 
0403   For best system performance it is strongly encouraged for software to enable
0404   ZA only when it is actively being used.
0405 
0406 * Two new 1 bit fields in PSTATE which may be controlled via the SMSTART and
0407   SMSTOP instructions or by access to the SVCR system register:
0408 
0409   * PSTATE.ZA, if this is 1 then the ZA matrix is accessible and has valid
0410     data while if it is 0 then ZA can not be accessed.  When PSTATE.ZA is
0411     changed from 0 to 1 all bits in ZA are cleared.
0412 
0413   * PSTATE.SM, if this is 1 then the PE is in streaming mode.  When the value
0414     of PSTATE.SM is changed then it is implementation defined if the subset
0415     of the floating point register bits valid in both modes may be retained.
0416     Any other bits will be cleared.
0417 
0418 
0419 References
0420 ==========
0421 
0422 [1] arch/arm64/include/uapi/asm/sigcontext.h
0423     AArch64 Linux signal ABI definitions
0424 
0425 [2] arch/arm64/include/uapi/asm/ptrace.h
0426     AArch64 Linux ptrace ABI definitions
0427 
0428 [3] Documentation/arm64/cpu-feature-registers.rst