0001 ===============================
0002 LIBNVDIMM: Non-Volatile Devices
0003 ===============================
0004
0005 libnvdimm - kernel / libndctl - userspace helper library
0006
0007 nvdimm@lists.linux.dev
0008
0009 Version 13
0010
0011 .. contents:
0012
0013 Glossary
0014 Overview
0015 Supporting Documents
0016 Git Trees
0017 LIBNVDIMM PMEM
0018 PMEM-REGIONs, Atomic Sectors, and DAX
0019 Example NVDIMM Platform
0020 LIBNVDIMM Kernel Device Model and LIBNDCTL Userspace API
0021 LIBNDCTL: Context
0022 libndctl: instantiate a new library context example
0023 LIBNVDIMM/LIBNDCTL: Bus
0024 libnvdimm: control class device in /sys/class
0025 libnvdimm: bus
0026 libndctl: bus enumeration example
0027 LIBNVDIMM/LIBNDCTL: DIMM (NMEM)
0028 libnvdimm: DIMM (NMEM)
0029 libndctl: DIMM enumeration example
0030 LIBNVDIMM/LIBNDCTL: Region
0031 libnvdimm: region
0032 libndctl: region enumeration example
0033 Why Not Encode the Region Type into the Region Name?
0034 How Do I Determine the Major Type of a Region?
0035 LIBNVDIMM/LIBNDCTL: Namespace
0036 libnvdimm: namespace
0037 libndctl: namespace enumeration example
0038 libndctl: namespace creation example
0039 Why the Term "namespace"?
0040 LIBNVDIMM/LIBNDCTL: Block Translation Table "btt"
0041 libnvdimm: btt layout
0042 libndctl: btt creation example
0043 Summary LIBNDCTL Diagram
0044
0045
0046 Glossary
0047 ========
0048
0049 PMEM:
0050 A system-physical-address range where writes are persistent. A
0051 block device composed of PMEM is capable of DAX. A PMEM address range
0052 may span an interleave of several DIMMs.
0053
0054 DPA:
0055 DIMM Physical Address, is a DIMM-relative offset. With one DIMM in
0056 the system there would be a 1:1 system-physical-address:DPA association.
0057 Once more DIMMs are added a memory controller interleave must be
0058 decoded to determine the DPA associated with a given
0059 system-physical-address.
0060
0061 DAX:
0062 File system extensions to bypass the page cache and block layer to
0063 mmap persistent memory, from a PMEM block device, directly into a
0064 process address space.
0065
0066 DSM:
0067 Device Specific Method: ACPI method to control specific
0068 device - in this case the firmware.
0069
0070 DCR:
0071 NVDIMM Control Region Structure defined in ACPI 6 Section 5.2.25.5.
0072 It defines a vendor-id, device-id, and interface format for a given DIMM.
0073
0074 BTT:
0075 Block Translation Table: Persistent memory is byte addressable.
0076 Existing software may have an expectation that the power-fail-atomicity
0077 of writes is at least one sector, 512 bytes. The BTT is an indirection
0078 table with atomic update semantics to front a PMEM block device
0079 driver and present arbitrary atomic sector sizes.
0080
0081 LABEL:
0082 Metadata stored on a DIMM device that partitions and identifies
0083 (persistently names) capacity allocated to different PMEM namespaces. It
0084 also indicates whether an address abstraction like a BTT is applied to
0085 the namepsace. Note that traditional partition tables, GPT/MBR, are
0086 layered on top of a PMEM namespace, or an address abstraction like BTT
0087 if present, but partition support is deprecated going forward.
0088
0089
0090 Overview
0091 ========
0092
0093 The LIBNVDIMM subsystem provides support for PMEM described by platform
0094 firmware or a device driver. On ACPI based systems the platform firmware
0095 conveys persistent memory resource via the ACPI NFIT "NVDIMM Firmware
0096 Interface Table" in ACPI 6. While the LIBNVDIMM subsystem implementation
0097 is generic and supports pre-NFIT platforms, it was guided by the
0098 superset of capabilities need to support this ACPI 6 definition for
0099 NVDIMM resources. The original implementation supported the
0100 block-window-aperture capability described in the NFIT, but that support
0101 has since been abandoned and never shipped in a product.
0102
0103 Supporting Documents
0104 --------------------
0105
0106 ACPI 6:
0107 https://www.uefi.org/sites/default/files/resources/ACPI_6.0.pdf
0108 NVDIMM Namespace:
0109 https://pmem.io/documents/NVDIMM_Namespace_Spec.pdf
0110 DSM Interface Example:
0111 https://pmem.io/documents/NVDIMM_DSM_Interface_Example.pdf
0112 Driver Writer's Guide:
0113 https://pmem.io/documents/NVDIMM_Driver_Writers_Guide.pdf
0114
0115 Git Trees
0116 ---------
0117
0118 LIBNVDIMM:
0119 https://git.kernel.org/cgit/linux/kernel/git/nvdimm/nvdimm.git
0120 LIBNDCTL:
0121 https://github.com/pmem/ndctl.git
0122
0123
0124 LIBNVDIMM PMEM
0125 ==============
0126
0127 Prior to the arrival of the NFIT, non-volatile memory was described to a
0128 system in various ad-hoc ways. Usually only the bare minimum was
0129 provided, namely, a single system-physical-address range where writes
0130 are expected to be durable after a system power loss. Now, the NFIT
0131 specification standardizes not only the description of PMEM, but also
0132 platform message-passing entry points for control and configuration.
0133
0134 PMEM (nd_pmem.ko): Drives a system-physical-address range. This range is
0135 contiguous in system memory and may be interleaved (hardware memory controller
0136 striped) across multiple DIMMs. When interleaved the platform may optionally
0137 provide details of which DIMMs are participating in the interleave.
0138
0139 It is worth noting that when the labeling capability is detected (a EFI
0140 namespace label index block is found), then no block device is created
0141 by default as userspace needs to do at least one allocation of DPA to
0142 the PMEM range. In contrast ND_NAMESPACE_IO ranges, once registered,
0143 can be immediately attached to nd_pmem. This latter mode is called
0144 label-less or "legacy".
0145
0146 PMEM-REGIONs, Atomic Sectors, and DAX
0147 -------------------------------------
0148
0149 For the cases where an application or filesystem still needs atomic sector
0150 update guarantees it can register a BTT on a PMEM device or partition. See
0151 LIBNVDIMM/NDCTL: Block Translation Table "btt"
0152
0153
0154 Example NVDIMM Platform
0155 =======================
0156
0157 For the remainder of this document the following diagram will be
0158 referenced for any example sysfs layouts::
0159
0160
0161 (a) (b) DIMM
0162 +-------------------+--------+--------+--------+
0163 +------+ | pm0.0 | free | pm1.0 | free | 0
0164 | imc0 +--+- - - region0- - - +--------+ +--------+
0165 +--+---+ | pm0.0 | free | pm1.0 | free | 1
0166 | +-------------------+--------v v--------+
0167 +--+---+ | |
0168 | cpu0 | region1
0169 +--+---+ | |
0170 | +----------------------------^ ^--------+
0171 +--+---+ | free | pm1.0 | free | 2
0172 | imc1 +--+----------------------------| +--------+
0173 +------+ | free | pm1.0 | free | 3
0174 +----------------------------+--------+--------+
0175
0176 In this platform we have four DIMMs and two memory controllers in one
0177 socket. Each PMEM interleave set is identified by a region device with
0178 a dynamically assigned id.
0179
0180 1. The first portion of DIMM0 and DIMM1 are interleaved as REGION0. A
0181 single PMEM namespace is created in the REGION0-SPA-range that spans most
0182 of DIMM0 and DIMM1 with a user-specified name of "pm0.0". Some of that
0183 interleaved system-physical-address range is left free for
0184 another PMEM namespace to be defined.
0185
0186 2. In the last portion of DIMM0 and DIMM1 we have an interleaved
0187 system-physical-address range, REGION1, that spans those two DIMMs as
0188 well as DIMM2 and DIMM3. Some of REGION1 is allocated to a PMEM namespace
0189 named "pm1.0".
0190
0191 This bus is provided by the kernel under the device
0192 /sys/devices/platform/nfit_test.0 when the nfit_test.ko module from
0193 tools/testing/nvdimm is loaded. This module is a unit test for
0194 LIBNVDIMM and the acpi_nfit.ko driver.
0195
0196
0197 LIBNVDIMM Kernel Device Model and LIBNDCTL Userspace API
0198 ========================================================
0199
0200 What follows is a description of the LIBNVDIMM sysfs layout and a
0201 corresponding object hierarchy diagram as viewed through the LIBNDCTL
0202 API. The example sysfs paths and diagrams are relative to the Example
0203 NVDIMM Platform which is also the LIBNVDIMM bus used in the LIBNDCTL unit
0204 test.
0205
0206 LIBNDCTL: Context
0207 -----------------
0208
0209 Every API call in the LIBNDCTL library requires a context that holds the
0210 logging parameters and other library instance state. The library is
0211 based on the libabc template:
0212
0213 https://git.kernel.org/cgit/linux/kernel/git/kay/libabc.git
0214
0215 LIBNDCTL: instantiate a new library context example
0216 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
0217
0218 ::
0219
0220 struct ndctl_ctx *ctx;
0221
0222 if (ndctl_new(&ctx) == 0)
0223 return ctx;
0224 else
0225 return NULL;
0226
0227 LIBNVDIMM/LIBNDCTL: Bus
0228 -----------------------
0229
0230 A bus has a 1:1 relationship with an NFIT. The current expectation for
0231 ACPI based systems is that there is only ever one platform-global NFIT.
0232 That said, it is trivial to register multiple NFITs, the specification
0233 does not preclude it. The infrastructure supports multiple busses and
0234 we use this capability to test multiple NFIT configurations in the unit
0235 test.
0236
0237 LIBNVDIMM: control class device in /sys/class
0238 ---------------------------------------------
0239
0240 This character device accepts DSM messages to be passed to DIMM
0241 identified by its NFIT handle::
0242
0243 /sys/class/nd/ndctl0
0244 |-- dev
0245 |-- device -> ../../../ndbus0
0246 |-- subsystem -> ../../../../../../../class/nd
0247
0248
0249
0250 LIBNVDIMM: bus
0251 --------------
0252
0253 ::
0254
0255 struct nvdimm_bus *nvdimm_bus_register(struct device *parent,
0256 struct nvdimm_bus_descriptor *nfit_desc);
0257
0258 ::
0259
0260 /sys/devices/platform/nfit_test.0/ndbus0
0261 |-- commands
0262 |-- nd
0263 |-- nfit
0264 |-- nmem0
0265 |-- nmem1
0266 |-- nmem2
0267 |-- nmem3
0268 |-- power
0269 |-- provider
0270 |-- region0
0271 |-- region1
0272 |-- region2
0273 |-- region3
0274 |-- region4
0275 |-- region5
0276 |-- uevent
0277 `-- wait_probe
0278
0279 LIBNDCTL: bus enumeration example
0280 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
0281
0282 Find the bus handle that describes the bus from Example NVDIMM Platform::
0283
0284 static struct ndctl_bus *get_bus_by_provider(struct ndctl_ctx *ctx,
0285 const char *provider)
0286 {
0287 struct ndctl_bus *bus;
0288
0289 ndctl_bus_foreach(ctx, bus)
0290 if (strcmp(provider, ndctl_bus_get_provider(bus)) == 0)
0291 return bus;
0292
0293 return NULL;
0294 }
0295
0296 bus = get_bus_by_provider(ctx, "nfit_test.0");
0297
0298
0299 LIBNVDIMM/LIBNDCTL: DIMM (NMEM)
0300 -------------------------------
0301
0302 The DIMM device provides a character device for sending commands to
0303 hardware, and it is a container for LABELs. If the DIMM is defined by
0304 NFIT then an optional 'nfit' attribute sub-directory is available to add
0305 NFIT-specifics.
0306
0307 Note that the kernel device name for "DIMMs" is "nmemX". The NFIT
0308 describes these devices via "Memory Device to System Physical Address
0309 Range Mapping Structure", and there is no requirement that they actually
0310 be physical DIMMs, so we use a more generic name.
0311
0312 LIBNVDIMM: DIMM (NMEM)
0313 ^^^^^^^^^^^^^^^^^^^^^^
0314
0315 ::
0316
0317 struct nvdimm *nvdimm_create(struct nvdimm_bus *nvdimm_bus, void *provider_data,
0318 const struct attribute_group **groups, unsigned long flags,
0319 unsigned long *dsm_mask);
0320
0321 ::
0322
0323 /sys/devices/platform/nfit_test.0/ndbus0
0324 |-- nmem0
0325 | |-- available_slots
0326 | |-- commands
0327 | |-- dev
0328 | |-- devtype
0329 | |-- driver -> ../../../../../bus/nd/drivers/nvdimm
0330 | |-- modalias
0331 | |-- nfit
0332 | | |-- device
0333 | | |-- format
0334 | | |-- handle
0335 | | |-- phys_id
0336 | | |-- rev_id
0337 | | |-- serial
0338 | | `-- vendor
0339 | |-- state
0340 | |-- subsystem -> ../../../../../bus/nd
0341 | `-- uevent
0342 |-- nmem1
0343 [..]
0344
0345
0346 LIBNDCTL: DIMM enumeration example
0347 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
0348
0349 Note, in this example we are assuming NFIT-defined DIMMs which are
0350 identified by an "nfit_handle" a 32-bit value where:
0351
0352 - Bit 3:0 DIMM number within the memory channel
0353 - Bit 7:4 memory channel number
0354 - Bit 11:8 memory controller ID
0355 - Bit 15:12 socket ID (within scope of a Node controller if node
0356 controller is present)
0357 - Bit 27:16 Node Controller ID
0358 - Bit 31:28 Reserved
0359
0360 ::
0361
0362 static struct ndctl_dimm *get_dimm_by_handle(struct ndctl_bus *bus,
0363 unsigned int handle)
0364 {
0365 struct ndctl_dimm *dimm;
0366
0367 ndctl_dimm_foreach(bus, dimm)
0368 if (ndctl_dimm_get_handle(dimm) == handle)
0369 return dimm;
0370
0371 return NULL;
0372 }
0373
0374 #define DIMM_HANDLE(n, s, i, c, d) \
0375 (((n & 0xfff) << 16) | ((s & 0xf) << 12) | ((i & 0xf) << 8) \
0376 | ((c & 0xf) << 4) | (d & 0xf))
0377
0378 dimm = get_dimm_by_handle(bus, DIMM_HANDLE(0, 0, 0, 0, 0));
0379
0380 LIBNVDIMM/LIBNDCTL: Region
0381 --------------------------
0382
0383 A generic REGION device is registered for each PMEM interleave-set /
0384 range. Per the example there are 2 PMEM regions on the "nfit_test.0"
0385 bus. The primary role of regions are to be a container of "mappings". A
0386 mapping is a tuple of <DIMM, DPA-start-offset, length>.
0387
0388 LIBNVDIMM provides a built-in driver for REGION devices. This driver
0389 is responsible for all parsing LABELs, if present, and then emitting NAMESPACE
0390 devices for the nd_pmem driver to consume.
0391
0392 In addition to the generic attributes of "mapping"s, "interleave_ways"
0393 and "size" the REGION device also exports some convenience attributes.
0394 "nstype" indicates the integer type of namespace-device this region
0395 emits, "devtype" duplicates the DEVTYPE variable stored by udev at the
0396 'add' event, "modalias" duplicates the MODALIAS variable stored by udev
0397 at the 'add' event, and finally, the optional "spa_index" is provided in
0398 the case where the region is defined by a SPA.
0399
0400 LIBNVDIMM: region::
0401
0402 struct nd_region *nvdimm_pmem_region_create(struct nvdimm_bus *nvdimm_bus,
0403 struct nd_region_desc *ndr_desc);
0404
0405 ::
0406
0407 /sys/devices/platform/nfit_test.0/ndbus0
0408 |-- region0
0409 | |-- available_size
0410 | |-- btt0
0411 | |-- btt_seed
0412 | |-- devtype
0413 | |-- driver -> ../../../../../bus/nd/drivers/nd_region
0414 | |-- init_namespaces
0415 | |-- mapping0
0416 | |-- mapping1
0417 | |-- mappings
0418 | |-- modalias
0419 | |-- namespace0.0
0420 | |-- namespace_seed
0421 | |-- numa_node
0422 | |-- nfit
0423 | | `-- spa_index
0424 | |-- nstype
0425 | |-- set_cookie
0426 | |-- size
0427 | |-- subsystem -> ../../../../../bus/nd
0428 | `-- uevent
0429 |-- region1
0430 [..]
0431
0432 LIBNDCTL: region enumeration example
0433 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
0434
0435 Sample region retrieval routines based on NFIT-unique data like
0436 "spa_index" (interleave set id).
0437
0438 ::
0439
0440 static struct ndctl_region *get_pmem_region_by_spa_index(struct ndctl_bus *bus,
0441 unsigned int spa_index)
0442 {
0443 struct ndctl_region *region;
0444
0445 ndctl_region_foreach(bus, region) {
0446 if (ndctl_region_get_type(region) != ND_DEVICE_REGION_PMEM)
0447 continue;
0448 if (ndctl_region_get_spa_index(region) == spa_index)
0449 return region;
0450 }
0451 return NULL;
0452 }
0453
0454
0455 LIBNVDIMM/LIBNDCTL: Namespace
0456 -----------------------------
0457
0458 A REGION, after resolving DPA aliasing and LABEL specified boundaries, surfaces
0459 one or more "namespace" devices. The arrival of a "namespace" device currently
0460 triggers the nd_pmem driver to load and register a disk/block device.
0461
0462 LIBNVDIMM: namespace
0463 ^^^^^^^^^^^^^^^^^^^^
0464
0465 Here is a sample layout from the 2 major types of NAMESPACE where namespace0.0
0466 represents DIMM-info-backed PMEM (note that it has a 'uuid' attribute), and
0467 namespace1.0 represents an anonymous PMEM namespace (note that has no 'uuid'
0468 attribute due to not support a LABEL)
0469
0470 ::
0471
0472 /sys/devices/platform/nfit_test.0/ndbus0/region0/namespace0.0
0473 |-- alt_name
0474 |-- devtype
0475 |-- dpa_extents
0476 |-- force_raw
0477 |-- modalias
0478 |-- numa_node
0479 |-- resource
0480 |-- size
0481 |-- subsystem -> ../../../../../../bus/nd
0482 |-- type
0483 |-- uevent
0484 `-- uuid
0485 /sys/devices/platform/nfit_test.1/ndbus1/region1/namespace1.0
0486 |-- block
0487 | `-- pmem0
0488 |-- devtype
0489 |-- driver -> ../../../../../../bus/nd/drivers/pmem
0490 |-- force_raw
0491 |-- modalias
0492 |-- numa_node
0493 |-- resource
0494 |-- size
0495 |-- subsystem -> ../../../../../../bus/nd
0496 |-- type
0497 `-- uevent
0498
0499 LIBNDCTL: namespace enumeration example
0500 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
0501 Namespaces are indexed relative to their parent region, example below.
0502 These indexes are mostly static from boot to boot, but subsystem makes
0503 no guarantees in this regard. For a static namespace identifier use its
0504 'uuid' attribute.
0505
0506 ::
0507
0508 static struct ndctl_namespace
0509 *get_namespace_by_id(struct ndctl_region *region, unsigned int id)
0510 {
0511 struct ndctl_namespace *ndns;
0512
0513 ndctl_namespace_foreach(region, ndns)
0514 if (ndctl_namespace_get_id(ndns) == id)
0515 return ndns;
0516
0517 return NULL;
0518 }
0519
0520 LIBNDCTL: namespace creation example
0521 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
0522
0523 Idle namespaces are automatically created by the kernel if a given
0524 region has enough available capacity to create a new namespace.
0525 Namespace instantiation involves finding an idle namespace and
0526 configuring it. For the most part the setting of namespace attributes
0527 can occur in any order, the only constraint is that 'uuid' must be set
0528 before 'size'. This enables the kernel to track DPA allocations
0529 internally with a static identifier::
0530
0531 static int configure_namespace(struct ndctl_region *region,
0532 struct ndctl_namespace *ndns,
0533 struct namespace_parameters *parameters)
0534 {
0535 char devname[50];
0536
0537 snprintf(devname, sizeof(devname), "namespace%d.%d",
0538 ndctl_region_get_id(region), paramaters->id);
0539
0540 ndctl_namespace_set_alt_name(ndns, devname);
0541 /* 'uuid' must be set prior to setting size! */
0542 ndctl_namespace_set_uuid(ndns, paramaters->uuid);
0543 ndctl_namespace_set_size(ndns, paramaters->size);
0544 /* unlike pmem namespaces, blk namespaces have a sector size */
0545 if (parameters->lbasize)
0546 ndctl_namespace_set_sector_size(ndns, parameters->lbasize);
0547 ndctl_namespace_enable(ndns);
0548 }
0549
0550
0551 Why the Term "namespace"?
0552 ^^^^^^^^^^^^^^^^^^^^^^^^^
0553
0554 1. Why not "volume" for instance? "volume" ran the risk of confusing
0555 ND (libnvdimm subsystem) to a volume manager like device-mapper.
0556
0557 2. The term originated to describe the sub-devices that can be created
0558 within a NVME controller (see the nvme specification:
0559 https://www.nvmexpress.org/specifications/), and NFIT namespaces are
0560 meant to parallel the capabilities and configurability of
0561 NVME-namespaces.
0562
0563
0564 LIBNVDIMM/LIBNDCTL: Block Translation Table "btt"
0565 -------------------------------------------------
0566
0567 A BTT (design document: https://pmem.io/2014/09/23/btt.html) is a
0568 personality driver for a namespace that fronts entire namespace as an
0569 'address abstraction'.
0570
0571 LIBNVDIMM: btt layout
0572 ^^^^^^^^^^^^^^^^^^^^^
0573
0574 Every region will start out with at least one BTT device which is the
0575 seed device. To activate it set the "namespace", "uuid", and
0576 "sector_size" attributes and then bind the device to the nd_pmem or
0577 nd_blk driver depending on the region type::
0578
0579 /sys/devices/platform/nfit_test.1/ndbus0/region0/btt0/
0580 |-- namespace
0581 |-- delete
0582 |-- devtype
0583 |-- modalias
0584 |-- numa_node
0585 |-- sector_size
0586 |-- subsystem -> ../../../../../bus/nd
0587 |-- uevent
0588 `-- uuid
0589
0590 LIBNDCTL: btt creation example
0591 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
0592
0593 Similar to namespaces an idle BTT device is automatically created per
0594 region. Each time this "seed" btt device is configured and enabled a new
0595 seed is created. Creating a BTT configuration involves two steps of
0596 finding and idle BTT and assigning it to consume a namespace.
0597
0598 ::
0599
0600 static struct ndctl_btt *get_idle_btt(struct ndctl_region *region)
0601 {
0602 struct ndctl_btt *btt;
0603
0604 ndctl_btt_foreach(region, btt)
0605 if (!ndctl_btt_is_enabled(btt)
0606 && !ndctl_btt_is_configured(btt))
0607 return btt;
0608
0609 return NULL;
0610 }
0611
0612 static int configure_btt(struct ndctl_region *region,
0613 struct btt_parameters *parameters)
0614 {
0615 btt = get_idle_btt(region);
0616
0617 ndctl_btt_set_uuid(btt, parameters->uuid);
0618 ndctl_btt_set_sector_size(btt, parameters->sector_size);
0619 ndctl_btt_set_namespace(btt, parameters->ndns);
0620 /* turn off raw mode device */
0621 ndctl_namespace_disable(parameters->ndns);
0622 /* turn on btt access */
0623 ndctl_btt_enable(btt);
0624 }
0625
0626 Once instantiated a new inactive btt seed device will appear underneath
0627 the region.
0628
0629 Once a "namespace" is removed from a BTT that instance of the BTT device
0630 will be deleted or otherwise reset to default values. This deletion is
0631 only at the device model level. In order to destroy a BTT the "info
0632 block" needs to be destroyed. Note, that to destroy a BTT the media
0633 needs to be written in raw mode. By default, the kernel will autodetect
0634 the presence of a BTT and disable raw mode. This autodetect behavior
0635 can be suppressed by enabling raw mode for the namespace via the
0636 ndctl_namespace_set_raw_mode() API.
0637
0638
0639 Summary LIBNDCTL Diagram
0640 ------------------------
0641
0642 For the given example above, here is the view of the objects as seen by the
0643 LIBNDCTL API::
0644
0645 +---+
0646 |CTX|
0647 +-+-+
0648 |
0649 +-------+ |
0650 | DIMM0 <-+ | +---------+ +--------------+ +---------------+
0651 +-------+ | | +-> REGION0 +---> NAMESPACE0.0 +--> PMEM8 "pm0.0" |
0652 | DIMM1 <-+ +-v--+ | +---------+ +--------------+ +---------------+
0653 +-------+ +-+BUS0+-| +---------+ +--------------+ +----------------------+
0654 | DIMM2 <-+ +----+ +-> REGION1 +---> NAMESPACE1.0 +--> PMEM6 "pm1.0" | BTT1 |
0655 +-------+ | | +---------+ +--------------+ +---------------+------+
0656 | DIMM3 <-+
0657 +-------+
