Back to home page




0001 Remote Processor Framework
0003 1. Introduction
0005 Modern SoCs typically have heterogeneous remote processor devices in asymmetric
0006 multiprocessing (AMP) configurations, which may be running different instances
0007 of operating system, whether it's Linux or any other flavor of real-time OS.
0009 OMAP4, for example, has dual Cortex-A9, dual Cortex-M3 and a C64x+ DSP.
0010 In a typical configuration, the dual cortex-A9 is running Linux in a SMP
0011 configuration, and each of the other three cores (two M3 cores and a DSP)
0012 is running its own instance of RTOS in an AMP configuration.
0014 The remoteproc framework allows different platforms/architectures to
0015 control (power on, load firmware, power off) those remote processors while
0016 abstracting the hardware differences, so the entire driver doesn't need to be
0017 duplicated. In addition, this framework also adds rpmsg virtio devices
0018 for remote processors that supports this kind of communication. This way,
0019 platform-specific remoteproc drivers only need to provide a few low-level
0020 handlers, and then all rpmsg drivers will then just work
0021 (for more information about the virtio-based rpmsg bus and its drivers,
0022 please read Documentation/rpmsg.txt).
0023 Registration of other types of virtio devices is now also possible. Firmwares
0024 just need to publish what kind of virtio devices do they support, and then
0025 remoteproc will add those devices. This makes it possible to reuse the
0026 existing virtio drivers with remote processor backends at a minimal development
0027 cost.
0029 2. User API
0031   int rproc_boot(struct rproc *rproc)
0032     - Boot a remote processor (i.e. load its firmware, power it on, ...).
0033       If the remote processor is already powered on, this function immediately
0034       returns (successfully).
0035       Returns 0 on success, and an appropriate error value otherwise.
0036       Note: to use this function you should already have a valid rproc
0037       handle. There are several ways to achieve that cleanly (devres, pdata,
0038       the way remoteproc_rpmsg.c does this, or, if this becomes prevalent, we
0039       might also consider using dev_archdata for this).
0041   void rproc_shutdown(struct rproc *rproc)
0042     - Power off a remote processor (previously booted with rproc_boot()).
0043       In case @rproc is still being used by an additional user(s), then
0044       this function will just decrement the power refcount and exit,
0045       without really powering off the device.
0046       Every call to rproc_boot() must (eventually) be accompanied by a call
0047       to rproc_shutdown(). Calling rproc_shutdown() redundantly is a bug.
0048       Notes:
0049       - we're not decrementing the rproc's refcount, only the power refcount.
0050         which means that the @rproc handle stays valid even after
0051         rproc_shutdown() returns, and users can still use it with a subsequent
0052         rproc_boot(), if needed.
0054   struct rproc *rproc_get_by_phandle(phandle phandle)
0055     - Find an rproc handle using a device tree phandle. Returns the rproc
0056       handle on success, and NULL on failure. This function increments
0057       the remote processor's refcount, so always use rproc_put() to
0058       decrement it back once rproc isn't needed anymore.
0060 3. Typical usage
0062 #include <linux/remoteproc.h>
0064 /* in case we were given a valid 'rproc' handle */
0065 int dummy_rproc_example(struct rproc *my_rproc)
0066 {
0067         int ret;
0069         /* let's power on and boot our remote processor */
0070         ret = rproc_boot(my_rproc);
0071         if (ret) {
0072                 /*
0073                  * something went wrong. handle it and leave.
0074                  */
0075         }
0077         /*
0078          * our remote processor is now powered on... give it some work
0079          */
0081         /* let's shut it down now */
0082         rproc_shutdown(my_rproc);
0083 }
0085 4. API for implementors
0087   struct rproc *rproc_alloc(struct device *dev, const char *name,
0088                                 const struct rproc_ops *ops,
0089                                 const char *firmware, int len)
0090     - Allocate a new remote processor handle, but don't register
0091       it yet. Required parameters are the underlying device, the
0092       name of this remote processor, platform-specific ops handlers,
0093       the name of the firmware to boot this rproc with, and the
0094       length of private data needed by the allocating rproc driver (in bytes).
0096       This function should be used by rproc implementations during
0097       initialization of the remote processor.
0098       After creating an rproc handle using this function, and when ready,
0099       implementations should then call rproc_add() to complete
0100       the registration of the remote processor.
0101       On success, the new rproc is returned, and on failure, NULL.
0103       Note: _never_ directly deallocate @rproc, even if it was not registered
0104       yet. Instead, when you need to unroll rproc_alloc(), use rproc_free().
0106   void rproc_free(struct rproc *rproc)
0107     - Free an rproc handle that was allocated by rproc_alloc.
0108       This function essentially unrolls rproc_alloc(), by decrementing the
0109       rproc's refcount. It doesn't directly free rproc; that would happen
0110       only if there are no other references to rproc and its refcount now
0111       dropped to zero.
0113   int rproc_add(struct rproc *rproc)
0114     - Register @rproc with the remoteproc framework, after it has been
0115       allocated with rproc_alloc().
0116       This is called by the platform-specific rproc implementation, whenever
0117       a new remote processor device is probed.
0118       Returns 0 on success and an appropriate error code otherwise.
0119       Note: this function initiates an asynchronous firmware loading
0120       context, which will look for virtio devices supported by the rproc's
0121       firmware.
0122       If found, those virtio devices will be created and added, so as a result
0123       of registering this remote processor, additional virtio drivers might get
0124       probed.
0126   int rproc_del(struct rproc *rproc)
0127     - Unroll rproc_add().
0128       This function should be called when the platform specific rproc
0129       implementation decides to remove the rproc device. it should
0130       _only_ be called if a previous invocation of rproc_add()
0131       has completed successfully.
0133       After rproc_del() returns, @rproc is still valid, and its
0134       last refcount should be decremented by calling rproc_free().
0136       Returns 0 on success and -EINVAL if @rproc isn't valid.
0138   void rproc_report_crash(struct rproc *rproc, enum rproc_crash_type type)
0139     - Report a crash in a remoteproc
0140       This function must be called every time a crash is detected by the
0141       platform specific rproc implementation. This should not be called from a
0142       non-remoteproc driver. This function can be called from atomic/interrupt
0143       context.
0145 5. Implementation callbacks
0147 These callbacks should be provided by platform-specific remoteproc
0148 drivers:
0150 /**
0151  * struct rproc_ops - platform-specific device handlers
0152  * @start:      power on the device and boot it
0153  * @stop:       power off the device
0154  * @kick:       kick a virtqueue (virtqueue id given as a parameter)
0155  */
0156 struct rproc_ops {
0157         int (*start)(struct rproc *rproc);
0158         int (*stop)(struct rproc *rproc);
0159         void (*kick)(struct rproc *rproc, int vqid);
0160 };
0162 Every remoteproc implementation should at least provide the ->start and ->stop
0163 handlers. If rpmsg/virtio functionality is also desired, then the ->kick handler
0164 should be provided as well.
0166 The ->start() handler takes an rproc handle and should then power on the
0167 device and boot it (use rproc->priv to access platform-specific private data).
0168 The boot address, in case needed, can be found in rproc->bootaddr (remoteproc
0169 core puts there the ELF entry point).
0170 On success, 0 should be returned, and on failure, an appropriate error code.
0172 The ->stop() handler takes an rproc handle and powers the device down.
0173 On success, 0 is returned, and on failure, an appropriate error code.
0175 The ->kick() handler takes an rproc handle, and an index of a virtqueue
0176 where new message was placed in. Implementations should interrupt the remote
0177 processor and let it know it has pending messages. Notifying remote processors
0178 the exact virtqueue index to look in is optional: it is easy (and not
0179 too expensive) to go through the existing virtqueues and look for new buffers
0180 in the used rings.
0182 6. Binary Firmware Structure
0184 At this point remoteproc only supports ELF32 firmware binaries. However,
0185 it is quite expected that other platforms/devices which we'd want to
0186 support with this framework will be based on different binary formats.
0188 When those use cases show up, we will have to decouple the binary format
0189 from the framework core, so we can support several binary formats without
0190 duplicating common code.
0192 When the firmware is parsed, its various segments are loaded to memory
0193 according to the specified device address (might be a physical address
0194 if the remote processor is accessing memory directly).
0196 In addition to the standard ELF segments, most remote processors would
0197 also include a special section which we call "the resource table".
0199 The resource table contains system resources that the remote processor
0200 requires before it should be powered on, such as allocation of physically
0201 contiguous memory, or iommu mapping of certain on-chip peripherals.
0202 Remotecore will only power up the device after all the resource table's
0203 requirement are met.
0205 In addition to system resources, the resource table may also contain
0206 resource entries that publish the existence of supported features
0207 or configurations by the remote processor, such as trace buffers and
0208 supported virtio devices (and their configurations).
0210 The resource table begins with this header:
0212 /**
0213  * struct resource_table - firmware resource table header
0214  * @ver: version number
0215  * @num: number of resource entries
0216  * @reserved: reserved (must be zero)
0217  * @offset: array of offsets pointing at the various resource entries
0218  *
0219  * The header of the resource table, as expressed by this structure,
0220  * contains a version number (should we need to change this format in the
0221  * future), the number of available resource entries, and their offsets
0222  * in the table.
0223  */
0224 struct resource_table {
0225         u32 ver;
0226         u32 num;
0227         u32 reserved[2];
0228         u32 offset[0];
0229 } __packed;
0231 Immediately following this header are the resource entries themselves,
0232 each of which begins with the following resource entry header:
0234 /**
0235  * struct fw_rsc_hdr - firmware resource entry header
0236  * @type: resource type
0237  * @data: resource data
0238  *
0239  * Every resource entry begins with a 'struct fw_rsc_hdr' header providing
0240  * its @type. The content of the entry itself will immediately follow
0241  * this header, and it should be parsed according to the resource type.
0242  */
0243 struct fw_rsc_hdr {
0244         u32 type;
0245         u8 data[0];
0246 } __packed;
0248 Some resources entries are mere announcements, where the host is informed
0249 of specific remoteproc configuration. Other entries require the host to
0250 do something (e.g. allocate a system resource). Sometimes a negotiation
0251 is expected, where the firmware requests a resource, and once allocated,
0252 the host should provide back its details (e.g. address of an allocated
0253 memory region).
0255 Here are the various resource types that are currently supported:
0257 /**
0258  * enum fw_resource_type - types of resource entries
0259  *
0260  * @RSC_CARVEOUT:   request for allocation of a physically contiguous
0261  *                  memory region.
0262  * @RSC_DEVMEM:     request to iommu_map a memory-based peripheral.
0263  * @RSC_TRACE:      announces the availability of a trace buffer into which
0264  *                  the remote processor will be writing logs.
0265  * @RSC_VDEV:       declare support for a virtio device, and serve as its
0266  *                  virtio header.
0267  * @RSC_LAST:       just keep this one at the end
0268  *
0269  * Please note that these values are used as indices to the rproc_handle_rsc
0270  * lookup table, so please keep them sane. Moreover, @RSC_LAST is used to
0271  * check the validity of an index before the lookup table is accessed, so
0272  * please update it as needed.
0273  */
0274 enum fw_resource_type {
0275         RSC_CARVEOUT    = 0,
0276         RSC_DEVMEM      = 1,
0277         RSC_TRACE       = 2,
0278         RSC_VDEV        = 3,
0279         RSC_LAST        = 4,
0280 };
0282 For more details regarding a specific resource type, please see its
0283 dedicated structure in include/linux/remoteproc.h.
0285 We also expect that platform-specific resource entries will show up
0286 at some point. When that happens, we could easily add a new RSC_PLATFORM
0287 type, and hand those resources to the platform-specific rproc driver to handle.
0289 7. Virtio and remoteproc
0291 The firmware should provide remoteproc information about virtio devices
0292 that it supports, and their configurations: a RSC_VDEV resource entry
0293 should specify the virtio device id (as in virtio_ids.h), virtio features,
0294 virtio config space, vrings information, etc.
0296 When a new remote processor is registered, the remoteproc framework
0297 will look for its resource table and will register the virtio devices
0298 it supports. A firmware may support any number of virtio devices, and
0299 of any type (a single remote processor can also easily support several
0300 rpmsg virtio devices this way, if desired).
0302 Of course, RSC_VDEV resource entries are only good enough for static
0303 allocation of virtio devices. Dynamic allocations will also be made possible
0304 using the rpmsg bus (similar to how we already do dynamic allocations of
0305 rpmsg channels; read more about it in rpmsg.txt).