0001 .. SPDX-License-Identifier: GPL-2.0
0002
0003 .. include:: <isonum.txt>
0004
0005 Qualcomm Camera Subsystem driver
0006 ================================
0007
0008 Introduction
0009 ------------
0010
0011 This file documents the Qualcomm Camera Subsystem driver located under
0012 drivers/media/platform/qcom/camss.
0013
0014 The current version of the driver supports the Camera Subsystem found on
0015 Qualcomm MSM8916/APQ8016 and MSM8996/APQ8096 processors.
0016
0017 The driver implements V4L2, Media controller and V4L2 subdev interfaces.
0018 Camera sensor using V4L2 subdev interface in the kernel is supported.
0019
0020 The driver is implemented using as a reference the Qualcomm Camera Subsystem
0021 driver for Android as found in Code Aurora [#f1]_ [#f2]_.
0022
0023
0024 Qualcomm Camera Subsystem hardware
0025 ----------------------------------
0026
0027 The Camera Subsystem hardware found on 8x16 / 8x96 processors and supported by
0028 the driver consists of:
0029
0030 - 2 / 3 CSIPHY modules. They handle the Physical layer of the CSI2 receivers.
0031 A separate camera sensor can be connected to each of the CSIPHY module;
0032 - 2 / 4 CSID (CSI Decoder) modules. They handle the Protocol and Application
0033 layer of the CSI2 receivers. A CSID can decode data stream from any of the
0034 CSIPHY. Each CSID also contains a TG (Test Generator) block which can generate
0035 artificial input data for test purposes;
0036 - ISPIF (ISP Interface) module. Handles the routing of the data streams from
0037 the CSIDs to the inputs of the VFE;
0038 - 1 / 2 VFE (Video Front End) module(s). Contain a pipeline of image processing
0039 hardware blocks. The VFE has different input interfaces. The PIX (Pixel) input
0040 interface feeds the input data to the image processing pipeline. The image
0041 processing pipeline contains also a scale and crop module at the end. Three
0042 RDI (Raw Dump Interface) input interfaces bypass the image processing
0043 pipeline. The VFE also contains the AXI bus interface which writes the output
0044 data to memory.
0045
0046
0047 Supported functionality
0048 -----------------------
0049
0050 The current version of the driver supports:
0051
0052 - Input from camera sensor via CSIPHY;
0053 - Generation of test input data by the TG in CSID;
0054 - RDI interface of VFE
0055
0056 - Raw dump of the input data to memory.
0057
0058 Supported formats:
0059
0060 - YUYV/UYVY/YVYU/VYUY (packed YUV 4:2:2 - V4L2_PIX_FMT_YUYV /
0061 V4L2_PIX_FMT_UYVY / V4L2_PIX_FMT_YVYU / V4L2_PIX_FMT_VYUY);
0062 - MIPI RAW8 (8bit Bayer RAW - V4L2_PIX_FMT_SRGGB8 /
0063 V4L2_PIX_FMT_SGRBG8 / V4L2_PIX_FMT_SGBRG8 / V4L2_PIX_FMT_SBGGR8);
0064 - MIPI RAW10 (10bit packed Bayer RAW - V4L2_PIX_FMT_SBGGR10P /
0065 V4L2_PIX_FMT_SGBRG10P / V4L2_PIX_FMT_SGRBG10P / V4L2_PIX_FMT_SRGGB10P /
0066 V4L2_PIX_FMT_Y10P);
0067 - MIPI RAW12 (12bit packed Bayer RAW - V4L2_PIX_FMT_SRGGB12P /
0068 V4L2_PIX_FMT_SGBRG12P / V4L2_PIX_FMT_SGRBG12P / V4L2_PIX_FMT_SRGGB12P).
0069 - (8x96 only) MIPI RAW14 (14bit packed Bayer RAW - V4L2_PIX_FMT_SRGGB14P /
0070 V4L2_PIX_FMT_SGBRG14P / V4L2_PIX_FMT_SGRBG14P / V4L2_PIX_FMT_SRGGB14P).
0071
0072 - (8x96 only) Format conversion of the input data.
0073
0074 Supported input formats:
0075
0076 - MIPI RAW10 (10bit packed Bayer RAW - V4L2_PIX_FMT_SBGGR10P / V4L2_PIX_FMT_Y10P).
0077
0078 Supported output formats:
0079
0080 - Plain16 RAW10 (10bit unpacked Bayer RAW - V4L2_PIX_FMT_SBGGR10 / V4L2_PIX_FMT_Y10).
0081
0082 - PIX interface of VFE
0083
0084 - Format conversion of the input data.
0085
0086 Supported input formats:
0087
0088 - YUYV/UYVY/YVYU/VYUY (packed YUV 4:2:2 - V4L2_PIX_FMT_YUYV /
0089 V4L2_PIX_FMT_UYVY / V4L2_PIX_FMT_YVYU / V4L2_PIX_FMT_VYUY).
0090
0091 Supported output formats:
0092
0093 - NV12/NV21 (two plane YUV 4:2:0 - V4L2_PIX_FMT_NV12 / V4L2_PIX_FMT_NV21);
0094 - NV16/NV61 (two plane YUV 4:2:2 - V4L2_PIX_FMT_NV16 / V4L2_PIX_FMT_NV61).
0095 - (8x96 only) YUYV/UYVY/YVYU/VYUY (packed YUV 4:2:2 - V4L2_PIX_FMT_YUYV /
0096 V4L2_PIX_FMT_UYVY / V4L2_PIX_FMT_YVYU / V4L2_PIX_FMT_VYUY).
0097
0098 - Scaling support. Configuration of the VFE Encoder Scale module
0099 for downscalling with ratio up to 16x.
0100
0101 - Cropping support. Configuration of the VFE Encoder Crop module.
0102
0103 - Concurrent and independent usage of two (8x96: three) data inputs -
0104 could be camera sensors and/or TG.
0105
0106
0107 Driver Architecture and Design
0108 ------------------------------
0109
0110 The driver implements the V4L2 subdev interface. With the goal to model the
0111 hardware links between the modules and to expose a clean, logical and usable
0112 interface, the driver is split into V4L2 sub-devices as follows (8x16 / 8x96):
0113
0114 - 2 / 3 CSIPHY sub-devices - each CSIPHY is represented by a single sub-device;
0115 - 2 / 4 CSID sub-devices - each CSID is represented by a single sub-device;
0116 - 2 / 4 ISPIF sub-devices - ISPIF is represented by a number of sub-devices
0117 equal to the number of CSID sub-devices;
0118 - 4 / 8 VFE sub-devices - VFE is represented by a number of sub-devices equal to
0119 the number of the input interfaces (3 RDI and 1 PIX for each VFE).
0120
0121 The considerations to split the driver in this particular way are as follows:
0122
0123 - representing CSIPHY and CSID modules by a separate sub-device for each module
0124 allows to model the hardware links between these modules;
0125 - representing VFE by a separate sub-devices for each input interface allows
0126 to use the input interfaces concurrently and independently as this is
0127 supported by the hardware;
0128 - representing ISPIF by a number of sub-devices equal to the number of CSID
0129 sub-devices allows to create linear media controller pipelines when using two
0130 cameras simultaneously. This avoids branches in the pipelines which otherwise
0131 will require a) userspace and b) media framework (e.g. power on/off
0132 operations) to make assumptions about the data flow from a sink pad to a
0133 source pad on a single media entity.
0134
0135 Each VFE sub-device is linked to a separate video device node.
0136
0137 The media controller pipeline graph is as follows (with connected two / three
0138 OV5645 camera sensors):
0139
0140 .. _qcom_camss_graph:
0141
0142 .. kernel-figure:: qcom_camss_graph.dot
0143 :alt: qcom_camss_graph.dot
0144 :align: center
0145
0146 Media pipeline graph 8x16
0147
0148 .. kernel-figure:: qcom_camss_8x96_graph.dot
0149 :alt: qcom_camss_8x96_graph.dot
0150 :align: center
0151
0152 Media pipeline graph 8x96
0153
0154
0155 Implementation
0156 --------------
0157
0158 Runtime configuration of the hardware (updating settings while streaming) is
0159 not required to implement the currently supported functionality. The complete
0160 configuration on each hardware module is applied on STREAMON ioctl based on
0161 the current active media links, formats and controls set.
0162
0163 The output size of the scaler module in the VFE is configured with the actual
0164 compose selection rectangle on the sink pad of the 'msm_vfe0_pix' entity.
0165
0166 The crop output area of the crop module in the VFE is configured with the actual
0167 crop selection rectangle on the source pad of the 'msm_vfe0_pix' entity.
0168
0169
0170 Documentation
0171 -------------
0172
0173 APQ8016 Specification:
0174 https://developer.qualcomm.com/download/sd410/snapdragon-410-processor-device-specification.pdf
0175 Referenced 2016-11-24.
0176
0177 APQ8096 Specification:
0178 https://developer.qualcomm.com/download/sd820e/qualcomm-snapdragon-820e-processor-apq8096sge-device-specification.pdf
0179 Referenced 2018-06-22.
0180
0181 References
0182 ----------
0183
0184 .. [#f1] https://source.codeaurora.org/quic/la/kernel/msm-3.10/
0185 .. [#f2] https://source.codeaurora.org/quic/la/kernel/msm-3.18/
