0001 For both Cherrytrail (CHT) and Baytrail (BHT) the driver
0002 requires the "candrpv_0415_20150521_0458" firmware version.
0003 It should be noticed that the firmware file is different,
0004 depending on the ISP model, so they're stored with different
0005 names:
0006
0007 - for BHT: /lib/firmware/shisp_2400b0_v21.bin
0008
0009 Warning: The driver was not tested yet for BHT.
0010
0011 - for CHT: /lib/firmware/shisp_2401a0_v21.bin
0012
0013 https://github.com/intel-aero/meta-intel-aero-base/blob/master/recipes-kernel/linux/linux-yocto/shisp_2401a0_v21.bin
0014
0015 NOTE:
0016 =====
0017
0018 This driver currently doesn't work with most V4L2 applications,
0019 as there are still some issues with regards to implementing
0020 certain APIs at the standard way.
0021
0022 Also, currently only USERPTR streaming mode is working.
0023
0024 In order to test, it is needed to know what's the sensor's
0025 resolution. This can be checked with:
0026
0027 $ v4l2-ctl --get-fmt-video
0028 Format Video Capture:
0029 Width/Height : 1600/1200
0030 ...
0031
0032 It is known to work with:
0033
0034 - v4l2grab at contrib/test directory at https://git.linuxtv.org/v4l-utils.git/
0035
0036 The resolution should not be bigger than the max resolution
0037 supported by the sensor, or it will fail. So, if the sensor
0038 reports:
0039
0040 The driver can be tested with:
0041
0042 v4l2grab -f YUYV -x 1600 -y 1200 -d /dev/video2 -u
0043
0044 - NVT at https://github.com/intel/nvt
0045
0046 $ ./v4l2n -o testimage_@.raw \
0047 --device /dev/video2 \
0048 --input 0 \
0049 --exposure=30000,30000,30000,30000 \
0050 --parm type=1,capturemode=CI_MODE_PREVIEW \
0051 --fmt type=1,width=1600,height=1200,pixelformat=YUYV \
0052 --reqbufs count=2,memory=USERPTR \
0053 --parameters=wb_config.r=32768,wb_config.gr=21043,wb_config.gb=21043,wb_config.b=30863 \
0054 --capture=20
0055
0056 As the output is in raw format, images need to be converted with:
0057
0058 $ for i in $(seq 0 19); do
0059 name="testimage_$(printf "%03i" $i)"
0060 ./raw2pnm -x$WIDTH -y$HEIGHT -f$FORMAT $name.raw $name.pnm
0061 rm $name.raw
0062 done
0063
0064 TODO
0065 ====
0066
0067 1. Fix support for MMAP streaming mode. This is required for most
0068 V4L2 applications;
0069
0070 2. Implement and/or fix V4L2 ioctls in order to allow a normal app to
0071 use it;
0072
0073 3. Ensure that the driver will pass v4l2-compliance tests;
0074
0075 4. Get manufacturer's authorization to redistribute the binaries for
0076 the firmware files;
0077
0078 5. remove VIDEO_ATOMISP_ISP2401, making the driver to auto-detect the
0079 register address differences between ISP2400 and ISP2401;
0080
0081 6. Cleanup the driver code, removing the abstraction layers inside it;
0082
0083 7. The atomisp doesn't rely at the usual i2c stuff to discover the
0084 sensors. Instead, it calls a function from atomisp_gmin_platform.c.
0085 There are some hacks added there for it to wait for sensors to be
0086 probed (with a timeout of 2 seconds or so). This should be converted
0087 to the usual way, using V4L2 async subdev framework to wait for
0088 cameras to be probed;
0089
0090 8. Switch to standard V4L2 sub-device API for sensor and lens. In
0091 particular, the user space API needs to support V4L2 controls as
0092 defined in the V4L2 spec and references to atomisp must be removed from
0093 these drivers.
0094
0095 9. Use LED flash API for flash LED drivers such as LM3554 (which already
0096 has a LED class driver).
0097
0098 10. Migrate the sensor drivers out of staging or re-using existing
0099 drivers;
0100
0101 11. Switch the driver to use pm_runtime stuff. Right now, it probes the
0102 existing PMIC code and sensors call it directly.
0103
0104 12. There's a problem on sensor drivers: when trying to set a video
0105 format, the atomisp main driver calls the sensor drivers with the
0106 sensor turned off. This causes them to fail.
0107
0108 This was fixed at atomisp-ov2880, which has a hack inside it
0109 to turn it on when VIDIOC_S_FMT is called, but this has to be
0110 cheked on other drivers as well.
0111
0112 The right fix seems to power on the sensor when a video device is
0113 opened (or at the first VIDIOC_ ioctl - except for VIDIOC_QUERYCAP),
0114 powering it down at close() syscall.
0115
0116 Such kind of control would need to be done inside the atomisp driver,
0117 not at the sensors code.
0118
0119 13. There are several issues related to memory management, that can
0120 cause crashes and/or memory leaks. The atomisp splits the memory
0121 management on three separate regions:
0122
0123 - dynamic pool;
0124 - reserved pool;
0125 - generic pool
0126
0127 The code implementing it is at:
0128
0129 drivers/staging/media/atomisp/pci/hmm/
0130
0131 It also has a separate code for managing DMA buffers at:
0132
0133 drivers/staging/media/atomisp/pci/mmu/
0134
0135 The code there is really dirty, ugly and probably wrong. I fixed
0136 one bug there already, but the best would be to just trash it and use
0137 something else. Maybe the code from the newer intel driver could
0138 serve as a model:
0139
0140 drivers/staging/media/ipu3/ipu3-mmu.c
0141
0142 But converting it to use something like that is painful and may
0143 cause some breakages.
0144
0145 14. The file structure needs to get tidied up to resemble a normal Linux
0146 driver.
0147
0148 15. Lots of the midlayer glue. Unused code and abstraction needs removing.
0149
0150 16. The AtomISP driver includes some special IOCTLS (ATOMISP_IOC_XXXX_XXXX)
0151 and controls that require some cleanup. Some of those code may have
0152 been removed during the cleanups. They could be needed in order to
0153 properly support 3A algorithms.
0154
0155 Such IOCTL interface needs more documentation. The better would
0156 be to use something close to the interface used by the IPU3 IMGU driver.
0157
0158 17. The ISP code has some dependencies of the exact FW version.
0159 The version defined in pci/sh_css_firmware.c:
0160
0161 BYT (isp2400): "irci_stable_candrpv_0415_20150521_0458"
0162
0163 CHT (isp2401): "irci_ecr - master_20150911_0724"
0164
0165 Those versions don't seem to be available anymore. On the tests we've
0166 done so far, this version also seems to work for CHT:
0167
0168 "irci_stable_candrpv_0415_20150521_0458"
0169
0170 Which can be obtainable from Yocto Atom ISP respository.
0171
0172 but this was not thoroughly tested.
0173
0174 At some point we may need to round up a few driver versions and see if
0175 there are any specific things that can be done to fold in support for
0176 multiple firmware versions.
0177
0178
0179 18. Switch from videobuf1 to videobuf2. Videobuf1 is being removed!
0180
0181 19. Correct Coding Style. Please refrain sending coding style patches
0182 for this driver until the other work is done, as there will be a lot
0183 of code churn until this driver becomes functional again.
0184
0185 20. Remove the logic which sets up pipelines inside it, moving it to
0186 libcamera and implement MC support.
0187
0188
0189 Limitations
0190 ===========
0191
0192 1. To test the patches, you also need the ISP firmware
0193
0194 for BYT: /lib/firmware/shisp_2400b0_v21.bin
0195 for CHT: /lib/firmware/shisp_2401a0_v21.bin
0196
0197 The firmware files will usually be found in /etc/firmware on an Android
0198 device but can also be extracted from the upgrade kit if you've managed
0199 to lose them somehow.
0200
0201 2. Without a 3A library the capture behaviour is not very good. To take a good
0202 picture, you need tune ISP parameters by IOCTL functions or use a 3A library
0203 such as libxcam.
0204
0205 3. The driver is intended to drive the PCI exposed versions of the device.
0206 It will not detect those devices enumerated via ACPI as a field of the
0207 i915 GPU driver.
0208
0209 There are some patches adding i915 GPU support floating at the Yocto's
0210 Aero repository (so far, untested upstream).
0211
0212 4. The driver supports only v2 of the IPU/Camera. It will not work with the
0213 versions of the hardware in other SoCs.
