Mauro Carvalho Chehab | b956109 | 2016-07-17 18:31:51 -0300 | [diff] [blame] | 1 | .. include:: <isonum.txt> |
| 2 | |
Sakari Ailus | e83dd48 | 2011-03-07 14:20:52 -0300 | [diff] [blame] | 3 | OMAP 3 Image Signal Processor (ISP) driver |
Mauro Carvalho Chehab | b956109 | 2016-07-17 18:31:51 -0300 | [diff] [blame] | 4 | ========================================== |
Sakari Ailus | e83dd48 | 2011-03-07 14:20:52 -0300 | [diff] [blame] | 5 | |
Mauro Carvalho Chehab | b956109 | 2016-07-17 18:31:51 -0300 | [diff] [blame] | 6 | Copyright |copy| 2010 Nokia Corporation |
Sakari Ailus | e83dd48 | 2011-03-07 14:20:52 -0300 | [diff] [blame] | 7 | |
Mauro Carvalho Chehab | b956109 | 2016-07-17 18:31:51 -0300 | [diff] [blame] | 8 | Copyright |copy| 2009 Texas Instruments, Inc. |
| 9 | |
| 10 | Contacts: Laurent Pinchart <laurent.pinchart@ideasonboard.com>, |
| 11 | Sakari Ailus <sakari.ailus@iki.fi>, David Cohen <dacohen@gmail.com> |
Sakari Ailus | e83dd48 | 2011-03-07 14:20:52 -0300 | [diff] [blame] | 12 | |
| 13 | |
| 14 | Introduction |
Mauro Carvalho Chehab | b956109 | 2016-07-17 18:31:51 -0300 | [diff] [blame] | 15 | ------------ |
Sakari Ailus | e83dd48 | 2011-03-07 14:20:52 -0300 | [diff] [blame] | 16 | |
| 17 | This file documents the Texas Instruments OMAP 3 Image Signal Processor (ISP) |
Lad, Prabhakar | 83c7353 | 2012-09-14 05:17:52 -0300 | [diff] [blame] | 18 | driver located under drivers/media/platform/omap3isp. The original driver was |
Sakari Ailus | e83dd48 | 2011-03-07 14:20:52 -0300 | [diff] [blame] | 19 | written by Texas Instruments but since that it has been rewritten (twice) at |
| 20 | Nokia. |
| 21 | |
| 22 | The driver has been successfully used on the following versions of OMAP 3: |
| 23 | |
Mauro Carvalho Chehab | b956109 | 2016-07-17 18:31:51 -0300 | [diff] [blame] | 24 | - 3430 |
| 25 | - 3530 |
| 26 | - 3630 |
Sakari Ailus | e83dd48 | 2011-03-07 14:20:52 -0300 | [diff] [blame] | 27 | |
| 28 | The driver implements V4L2, Media controller and v4l2_subdev interfaces. |
| 29 | Sensor, lens and flash drivers using the v4l2_subdev interface in the kernel |
| 30 | are supported. |
| 31 | |
| 32 | |
| 33 | Split to subdevs |
Mauro Carvalho Chehab | b956109 | 2016-07-17 18:31:51 -0300 | [diff] [blame] | 34 | ---------------- |
Sakari Ailus | e83dd48 | 2011-03-07 14:20:52 -0300 | [diff] [blame] | 35 | |
| 36 | The OMAP 3 ISP is split into V4L2 subdevs, each of the blocks inside the ISP |
| 37 | having one subdev to represent it. Each of the subdevs provide a V4L2 subdev |
| 38 | interface to userspace. |
| 39 | |
Mauro Carvalho Chehab | b956109 | 2016-07-17 18:31:51 -0300 | [diff] [blame] | 40 | - OMAP3 ISP CCP2 |
| 41 | - OMAP3 ISP CSI2a |
| 42 | - OMAP3 ISP CCDC |
| 43 | - OMAP3 ISP preview |
| 44 | - OMAP3 ISP resizer |
| 45 | - OMAP3 ISP AEWB |
| 46 | - OMAP3 ISP AF |
| 47 | - OMAP3 ISP histogram |
Sakari Ailus | e83dd48 | 2011-03-07 14:20:52 -0300 | [diff] [blame] | 48 | |
| 49 | Each possible link in the ISP is modelled by a link in the Media controller |
Mauro Carvalho Chehab | b956109 | 2016-07-17 18:31:51 -0300 | [diff] [blame] | 50 | interface. For an example program see [#f2]_. |
Sakari Ailus | e83dd48 | 2011-03-07 14:20:52 -0300 | [diff] [blame] | 51 | |
| 52 | |
| 53 | Controlling the OMAP 3 ISP |
Mauro Carvalho Chehab | b956109 | 2016-07-17 18:31:51 -0300 | [diff] [blame] | 54 | -------------------------- |
Sakari Ailus | e83dd48 | 2011-03-07 14:20:52 -0300 | [diff] [blame] | 55 | |
| 56 | In general, the settings given to the OMAP 3 ISP take effect at the beginning |
| 57 | of the following frame. This is done when the module becomes idle during the |
| 58 | vertical blanking period on the sensor. In memory-to-memory operation the pipe |
| 59 | is run one frame at a time. Applying the settings is done between the frames. |
| 60 | |
| 61 | All the blocks in the ISP, excluding the CSI-2 and possibly the CCP2 receiver, |
| 62 | insist on receiving complete frames. Sensors must thus never send the ISP |
| 63 | partial frames. |
| 64 | |
| 65 | Autoidle does have issues with some ISP blocks on the 3430, at least. |
| 66 | Autoidle is only enabled on 3630 when the omap3isp module parameter autoidle |
| 67 | is non-zero. |
| 68 | |
| 69 | |
| 70 | Events |
Mauro Carvalho Chehab | b956109 | 2016-07-17 18:31:51 -0300 | [diff] [blame] | 71 | ------ |
Sakari Ailus | e83dd48 | 2011-03-07 14:20:52 -0300 | [diff] [blame] | 72 | |
| 73 | The OMAP 3 ISP driver does support the V4L2 event interface on CCDC and |
| 74 | statistics (AEWB, AF and histogram) subdevs. |
| 75 | |
Sakari Ailus | 69d232a | 2011-06-15 15:58:48 -0300 | [diff] [blame] | 76 | The CCDC subdev produces V4L2_EVENT_FRAME_SYNC type event on HS_VS |
| 77 | interrupt which is used to signal frame start. Earlier version of this |
| 78 | driver used V4L2_EVENT_OMAP3ISP_HS_VS for this purpose. The event is |
| 79 | triggered exactly when the reception of the first line of the frame starts |
| 80 | in the CCDC module. The event can be subscribed on the CCDC subdev. |
Sakari Ailus | e83dd48 | 2011-03-07 14:20:52 -0300 | [diff] [blame] | 81 | |
| 82 | (When using parallel interface one must pay account to correct configuration |
| 83 | of the VS signal polarity. This is automatically correct when using the serial |
| 84 | receivers.) |
| 85 | |
| 86 | Each of the statistics subdevs is able to produce events. An event is |
| 87 | generated whenever a statistics buffer can be dequeued by a user space |
| 88 | application using the VIDIOC_OMAP3ISP_STAT_REQ IOCTL. The events available |
| 89 | are: |
| 90 | |
Mauro Carvalho Chehab | b956109 | 2016-07-17 18:31:51 -0300 | [diff] [blame] | 91 | - V4L2_EVENT_OMAP3ISP_AEWB |
| 92 | - V4L2_EVENT_OMAP3ISP_AF |
| 93 | - V4L2_EVENT_OMAP3ISP_HIST |
Sakari Ailus | e83dd48 | 2011-03-07 14:20:52 -0300 | [diff] [blame] | 94 | |
| 95 | The type of the event data is struct omap3isp_stat_event_status for these |
| 96 | ioctls. If there is an error calculating the statistics, there will be an |
| 97 | event as usual, but no related statistics buffer. In this case |
| 98 | omap3isp_stat_event_status.buf_err is set to non-zero. |
| 99 | |
| 100 | |
| 101 | Private IOCTLs |
Mauro Carvalho Chehab | b956109 | 2016-07-17 18:31:51 -0300 | [diff] [blame] | 102 | -------------- |
Sakari Ailus | e83dd48 | 2011-03-07 14:20:52 -0300 | [diff] [blame] | 103 | |
| 104 | The OMAP 3 ISP driver supports standard V4L2 IOCTLs and controls where |
| 105 | possible and practical. Much of the functions provided by the ISP, however, |
| 106 | does not fall under the standard IOCTLs --- gamma tables and configuration of |
| 107 | statistics collection are examples of such. |
| 108 | |
| 109 | In general, there is a private ioctl for configuring each of the blocks |
| 110 | containing hardware-dependent functions. |
| 111 | |
| 112 | The following private IOCTLs are supported: |
| 113 | |
Mauro Carvalho Chehab | b956109 | 2016-07-17 18:31:51 -0300 | [diff] [blame] | 114 | - VIDIOC_OMAP3ISP_CCDC_CFG |
| 115 | - VIDIOC_OMAP3ISP_PRV_CFG |
| 116 | - VIDIOC_OMAP3ISP_AEWB_CFG |
| 117 | - VIDIOC_OMAP3ISP_HIST_CFG |
| 118 | - VIDIOC_OMAP3ISP_AF_CFG |
| 119 | - VIDIOC_OMAP3ISP_STAT_REQ |
| 120 | - VIDIOC_OMAP3ISP_STAT_EN |
Sakari Ailus | e83dd48 | 2011-03-07 14:20:52 -0300 | [diff] [blame] | 121 | |
| 122 | The parameter structures used by these ioctls are described in |
| 123 | include/linux/omap3isp.h. The detailed functions of the ISP itself related to |
| 124 | a given ISP block is described in the Technical Reference Manuals (TRMs) --- |
| 125 | see the end of the document for those. |
| 126 | |
| 127 | While it is possible to use the ISP driver without any use of these private |
| 128 | IOCTLs it is not possible to obtain optimal image quality this way. The AEWB, |
| 129 | AF and histogram modules cannot be used without configuring them using the |
| 130 | appropriate private IOCTLs. |
| 131 | |
| 132 | |
| 133 | CCDC and preview block IOCTLs |
Mauro Carvalho Chehab | b956109 | 2016-07-17 18:31:51 -0300 | [diff] [blame] | 134 | ----------------------------- |
Sakari Ailus | e83dd48 | 2011-03-07 14:20:52 -0300 | [diff] [blame] | 135 | |
| 136 | The VIDIOC_OMAP3ISP_CCDC_CFG and VIDIOC_OMAP3ISP_PRV_CFG IOCTLs are used to |
| 137 | configure, enable and disable functions in the CCDC and preview blocks, |
| 138 | respectively. Both IOCTLs control several functions in the blocks they |
| 139 | control. VIDIOC_OMAP3ISP_CCDC_CFG IOCTL accepts a pointer to struct |
| 140 | omap3isp_ccdc_update_config as its argument. Similarly VIDIOC_OMAP3ISP_PRV_CFG |
| 141 | accepts a pointer to struct omap3isp_prev_update_config. The definition of |
Mauro Carvalho Chehab | b956109 | 2016-07-17 18:31:51 -0300 | [diff] [blame] | 142 | both structures is available in [#f1]_. |
Sakari Ailus | e83dd48 | 2011-03-07 14:20:52 -0300 | [diff] [blame] | 143 | |
| 144 | The update field in the structures tells whether to update the configuration |
| 145 | for the specific function and the flag tells whether to enable or disable the |
| 146 | function. |
| 147 | |
| 148 | The update and flag bit masks accept the following values. Each separate |
| 149 | functions in the CCDC and preview blocks is associated with a flag (either |
| 150 | disable or enable; part of the flag field in the structure) and a pointer to |
| 151 | configuration data for the function. |
| 152 | |
| 153 | Valid values for the update and flag fields are listed here for |
| 154 | VIDIOC_OMAP3ISP_CCDC_CFG. Values may be or'ed to configure more than one |
| 155 | function in the same IOCTL call. |
| 156 | |
Mauro Carvalho Chehab | b956109 | 2016-07-17 18:31:51 -0300 | [diff] [blame] | 157 | - OMAP3ISP_CCDC_ALAW |
| 158 | - OMAP3ISP_CCDC_LPF |
| 159 | - OMAP3ISP_CCDC_BLCLAMP |
| 160 | - OMAP3ISP_CCDC_BCOMP |
| 161 | - OMAP3ISP_CCDC_FPC |
| 162 | - OMAP3ISP_CCDC_CULL |
| 163 | - OMAP3ISP_CCDC_CONFIG_LSC |
| 164 | - OMAP3ISP_CCDC_TBL_LSC |
Sakari Ailus | e83dd48 | 2011-03-07 14:20:52 -0300 | [diff] [blame] | 165 | |
| 166 | The corresponding values for the VIDIOC_OMAP3ISP_PRV_CFG are here: |
| 167 | |
Mauro Carvalho Chehab | b956109 | 2016-07-17 18:31:51 -0300 | [diff] [blame] | 168 | - OMAP3ISP_PREV_LUMAENH |
| 169 | - OMAP3ISP_PREV_INVALAW |
| 170 | - OMAP3ISP_PREV_HRZ_MED |
| 171 | - OMAP3ISP_PREV_CFA |
| 172 | - OMAP3ISP_PREV_CHROMA_SUPP |
| 173 | - OMAP3ISP_PREV_WB |
| 174 | - OMAP3ISP_PREV_BLKADJ |
| 175 | - OMAP3ISP_PREV_RGB2RGB |
| 176 | - OMAP3ISP_PREV_COLOR_CONV |
| 177 | - OMAP3ISP_PREV_YC_LIMIT |
| 178 | - OMAP3ISP_PREV_DEFECT_COR |
| 179 | - OMAP3ISP_PREV_GAMMABYPASS |
| 180 | - OMAP3ISP_PREV_DRK_FRM_CAPTURE |
| 181 | - OMAP3ISP_PREV_DRK_FRM_SUBTRACT |
| 182 | - OMAP3ISP_PREV_LENS_SHADING |
| 183 | - OMAP3ISP_PREV_NF |
| 184 | - OMAP3ISP_PREV_GAMMA |
Sakari Ailus | e83dd48 | 2011-03-07 14:20:52 -0300 | [diff] [blame] | 185 | |
| 186 | The associated configuration pointer for the function may not be NULL when |
| 187 | enabling the function. When disabling a function the configuration pointer is |
| 188 | ignored. |
| 189 | |
| 190 | |
| 191 | Statistic blocks IOCTLs |
Mauro Carvalho Chehab | b956109 | 2016-07-17 18:31:51 -0300 | [diff] [blame] | 192 | ----------------------- |
Sakari Ailus | e83dd48 | 2011-03-07 14:20:52 -0300 | [diff] [blame] | 193 | |
| 194 | The statistics subdevs do offer more dynamic configuration options than the |
| 195 | other subdevs. They can be enabled, disable and reconfigured when the pipeline |
| 196 | is in streaming state. |
| 197 | |
| 198 | The statistics blocks always get the input image data from the CCDC (as the |
| 199 | histogram memory read isn't implemented). The statistics are dequeueable by |
| 200 | the user from the statistics subdev nodes using private IOCTLs. |
| 201 | |
| 202 | The private IOCTLs offered by the AEWB, AF and histogram subdevs are heavily |
| 203 | reflected by the register level interface offered by the ISP hardware. There |
| 204 | are aspects that are purely related to the driver implementation and these are |
| 205 | discussed next. |
| 206 | |
| 207 | VIDIOC_OMAP3ISP_STAT_EN |
| 208 | ----------------------- |
| 209 | |
| 210 | This private IOCTL enables/disables a statistic module. If this request is |
| 211 | done before streaming, it will take effect as soon as the pipeline starts to |
| 212 | stream. If the pipeline is already streaming, it will take effect as soon as |
| 213 | the CCDC becomes idle. |
| 214 | |
| 215 | VIDIOC_OMAP3ISP_AEWB_CFG, VIDIOC_OMAP3ISP_HIST_CFG and VIDIOC_OMAP3ISP_AF_CFG |
| 216 | ----------------------------------------------------------------------------- |
| 217 | |
| 218 | Those IOCTLs are used to configure the modules. They require user applications |
| 219 | to have an in-depth knowledge of the hardware. Most of the fields explanation |
| 220 | can be found on OMAP's TRMs. The two following fields common to all the above |
| 221 | configure private IOCTLs require explanation for better understanding as they |
| 222 | are not part of the TRM. |
| 223 | |
Mauro Carvalho Chehab | b956109 | 2016-07-17 18:31:51 -0300 | [diff] [blame] | 224 | omap3isp_[h3a_af/h3a_aewb/hist]\_config.buf_size: |
Sakari Ailus | e83dd48 | 2011-03-07 14:20:52 -0300 | [diff] [blame] | 225 | |
| 226 | The modules handle their buffers internally. The necessary buffer size for the |
| 227 | module's data output depends on the requested configuration. Although the |
| 228 | driver supports reconfiguration while streaming, it does not support a |
| 229 | reconfiguration which requires bigger buffer size than what is already |
| 230 | internally allocated if the module is enabled. It will return -EBUSY on this |
| 231 | case. In order to avoid such condition, either disable/reconfigure/enable the |
| 232 | module or request the necessary buffer size during the first configuration |
| 233 | while the module is disabled. |
| 234 | |
| 235 | The internal buffer size allocation considers the requested configuration's |
| 236 | minimum buffer size and the value set on buf_size field. If buf_size field is |
| 237 | out of [minimum, maximum] buffer size range, it's clamped to fit in there. |
| 238 | The driver then selects the biggest value. The corrected buf_size value is |
| 239 | written back to user application. |
| 240 | |
Mauro Carvalho Chehab | b956109 | 2016-07-17 18:31:51 -0300 | [diff] [blame] | 241 | omap3isp_[h3a_af/h3a_aewb/hist]\_config.config_counter: |
Sakari Ailus | e83dd48 | 2011-03-07 14:20:52 -0300 | [diff] [blame] | 242 | |
| 243 | As the configuration doesn't take effect synchronously to the request, the |
| 244 | driver must provide a way to track this information to provide more accurate |
| 245 | data. After a configuration is requested, the config_counter returned to user |
| 246 | space application will be an unique value associated to that request. When |
| 247 | user application receives an event for buffer availability or when a new |
| 248 | buffer is requested, this config_counter is used to match a buffer data and a |
| 249 | configuration. |
| 250 | |
| 251 | VIDIOC_OMAP3ISP_STAT_REQ |
| 252 | ------------------------ |
| 253 | |
| 254 | Send to user space the oldest data available in the internal buffer queue and |
| 255 | discards such buffer afterwards. The field omap3isp_stat_data.frame_number |
| 256 | matches with the video buffer's field_count. |
| 257 | |
| 258 | |
| 259 | Technical reference manuals (TRMs) and other documentation |
Mauro Carvalho Chehab | b956109 | 2016-07-17 18:31:51 -0300 | [diff] [blame] | 260 | ---------------------------------------------------------- |
Sakari Ailus | e83dd48 | 2011-03-07 14:20:52 -0300 | [diff] [blame] | 261 | |
| 262 | OMAP 3430 TRM: |
| 263 | <URL:http://focus.ti.com/pdfs/wtbu/OMAP34xx_ES3.1.x_PUBLIC_TRM_vZM.zip> |
| 264 | Referenced 2011-03-05. |
| 265 | |
| 266 | OMAP 35xx TRM: |
| 267 | <URL:http://www.ti.com/litv/pdf/spruf98o> Referenced 2011-03-05. |
| 268 | |
| 269 | OMAP 3630 TRM: |
| 270 | <URL:http://focus.ti.com/pdfs/wtbu/OMAP36xx_ES1.x_PUBLIC_TRM_vQ.zip> |
| 271 | Referenced 2011-03-05. |
| 272 | |
| 273 | DM 3730 TRM: |
| 274 | <URL:http://www.ti.com/litv/pdf/sprugn4h> Referenced 2011-03-06. |
| 275 | |
| 276 | |
| 277 | References |
Mauro Carvalho Chehab | b956109 | 2016-07-17 18:31:51 -0300 | [diff] [blame] | 278 | ---------- |
Sakari Ailus | e83dd48 | 2011-03-07 14:20:52 -0300 | [diff] [blame] | 279 | |
Mauro Carvalho Chehab | b956109 | 2016-07-17 18:31:51 -0300 | [diff] [blame] | 280 | .. [#f1] include/linux/omap3isp.h |
Sakari Ailus | e83dd48 | 2011-03-07 14:20:52 -0300 | [diff] [blame] | 281 | |
Mauro Carvalho Chehab | b956109 | 2016-07-17 18:31:51 -0300 | [diff] [blame] | 282 | .. [#f2] http://git.ideasonboard.org/?p=media-ctl.git;a=summary |