From 76737ade515e3da8bd16f3534a8b83a9f64f32d9 Mon Sep 17 00:00:00 2001
From: Shunqian Zheng <zhengsq@rock-chips.com>
Date: Thu, 8 Mar 2018 17:47:51 +0800
Subject: [PATCH] media: videodev2.h, v4l2-ioctl: add rkisp1 meta buffer format
Add the Rockchip ISP1 specific processing parameter format
V4L2_META_FMT_RK_ISP1_PARAMS and metadata format
V4L2_META_FMT_RK_ISP1_STAT_3A for 3A.
Signed-off-by: Shunqian Zheng <zhengsq@rock-chips.com>
Signed-off-by: Jacob Chen <jacob2.chen@rock-chips.com>
Acked-by: Hans Verkuil <hans.verkuil@cisco.com>
[update for upstream]
Signed-off-by: Helen Koike <helen.koike@collabora.com>
Series-to: linux-rockchip@lists.infradead.org
Series-cc: linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org
Series-cc: mchehab@kernel.org, linux-media@vger.kernel.org
Series-cc: sakari.ailus@linux.intel.com, hans.verkuil@cisco.com
Series-cc: tfiga@chromium.org, zhengsq@rock-chips.com
Series-cc: laurent.pinchart@ideasonboard.com, zyc@rock-chips.com
Series-cc: eddie.cai.linux@gmail.com, jeffy.chen@rock-chips.com
Series-cc: devicetree@vger.kernel.org, heiko@sntech.de
Series-cc: jacob2.chen@rock-chips.com
Series-cc: kernel@collabora.com, ezequiel@collabora.com
Series-version: 7
Series-changes: 7
- s/IPU3/RK_ISP1
Cover-letter:
Rockchip ISP1 Driver
Hello,
I'm re-sending a new version of ISP(Camera) v4l2 driver for rockchip
rk3399 SoC.
It is not perfect yet (see known issues below), but I'm sending in case
some other people already wants to start playing with it.
I believe de main design is ok (please let me know if
it is not) and it would be great to get some reviews already.
This patchset is also available at:
https://gitlab.collabora.com/koike/linux/tree/rockchip/isp/v7
Libcamera patched to work with this version:
https://gitlab.collabora.com/koike/libcamera
(I'll also sent it to the libcamera dev mailing list)
I tested on the rockpi 4 with a rpi v1.3 sensor and also with the
Scarlet Chromebook.
Images from the Scarlet are a bit dark and green for some reason, but I
believe it's a problem in the sensor's drivers as images from the
rockpi looks ok.
The main differences from previous version are (in a macro pov):
----------------------------------------------------------------
- dphy specific code migrated to drivers/phy
- design change: droped the subdevice for the interface. Now, in the
media topology, the sensors connect directly to the ISP1.
- v4l2-compliance fixes
- dropped rk3288 (as I'm not testing it)
- dropped txrx dphy support (as I'm not testing it and it requires a bit
more work to support dsi too)
- interrupts and hw config fixes
- minor bug fixes and cleanups
- I added myself in the MAINTAINERS, as I'm not sure if previous people
still wants to maintain it, please let me know if I should keep the old
names there.
Known issues:
-------------
- Reloading the module doesn't work (there is some missing cleanup when
unloading)
- When capturing in bayer format, changing the size doesn't seem to
affect the image.
- crop needs more tests
- v4l2-compliance error:
fail: v4l2-test-controls.cpp(824): subscribe event for control 'Image Processing Controls' failed
test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: FAIL
It seems that if controls are supported, v4l2-compliance says that
controls of type 'Image Processing Controls' are mandatory, is this
correct?
- It seems there are still some issues with interrupts, but I couldn't
isolate them yet.
Reasoning for the design change:
--------------------------------
In the previous version, the isp subdevice call the mipidphy_g_mbus
from rkisp1.c, so it can get
informations from the sensor such as the type of mbus (V4L2_MBUS_BT656,
V4L2_MBUS_PARALLEL or V4L2_MBUS_CSI2_CPHY), the number of csi2 lanes,
flags (V4L2_MBUS_PCLK_SAMPLE_RISING, V4L2_MBUS_VSYNC_ACTIVE_LOW or
V4L2_MBUS_HSYNC_ACTIVE_LOW). And the isp driver uses those info to configure
the hardware properly.
These information come from the DT node of the sensor. And the current
implementation is propagating this information from the sensor to the isp
through this mipidphy_g_mbus_config() (thus the hack)
* 1st attempt to solve this hack) Separating the interface code from the isp.
With the topology:
isp -> csi2 -> sensor
I was trying to migrate the CSI2 hardware configuration to the csi2 subdevice code.
But the problem I found was that in the DT, the csi2 regs is in the middle of the isp1
regs, and declaring the same "regs = <>" in both nodes (isp0 and csi2) is no good.
* 2nd attempt) flatening the DT
So instead of having two DT nodes (isp0 and csi2), we can have a single node, similar
to omap3isp.
And we can have a property, "rk,phy-type" that defines the interface (csi2, bt656 or parallel).
But, now my question is: can the isp be connected to multiple interfaces at a
time? If yes, then this is not a good solution (as we won't be able to describe
multiple interfaces in the DT node).
* 3rd attemp - WIP) get rid of the interface subdevice (chosen design)
Is there a reason to have the topology like:
isp -> interface - - -> sensor1
| |--------> sensor2
| - - - - - - > sensor3
(only one sensor can be active at a time)
?
Would it be ok if I just hide the interface from the topology? Like:
isp - - - - - -> sensor1
| |-----------> sensor2
|- - - - - - - > sensor3
Like this, I could cleanup a bunch of v4l2 code from the interface node,
the isp would know the active sensor (and its configs), and it can
configure everything itself.
I don't really see a big reason to expose the interface (csi2,
bt656 or parallel) in the topology.
Unless I'm missing something.
Previous changelog:
-------------------
changes in V6:
- add mipi txrx phy support
- remove bool and enum from uapi header
- add buf_prepare op
- correct some spelling problems
- return all queued buffers when starting stream failed
changes in V5: Sync with local changes,
- fix the SP height limit
- speed up the second stream capture
- the second stream can't force sync for rsz when start/stop streaming
- add frame id to param vb2 buf
- enable luminance maximum threshold
changes in V4:
- fix some bugs during development
- move quantization settings to rkisp1 subdev
- correct some spelling problems
- describe ports in dt-binding documents
changes in V3:
- add some comments
- fix wrong use of v4l2_async_subdev_notifier_register
- optimize two paths capture at a time
- remove compose
- re-struct headers
- add a tmp wiki page: http://opensource.rock-chips.com/wiki_Rockchip-isp1
changes in V2:
mipi-phy:
- use async probing
- make it be a child device of the GRF
isp:
- add dummy buffer
- change the way to get bus configuration, which make it possible to
add parallel sensor support in the future(without mipi-phy driver).
------------------
END
---
drivers/media/v4l2-core/v4l2-ioctl.c | 2 ++
include/uapi/linux/videodev2.h | 4 ++++
2 files changed, 6 insertions(+)
diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c b/drivers/media/v4l2-core/v4l2-ioctl.c
index b1f4b991dba633..248eb9d3bf4248 100644
--- a/drivers/media/v4l2-core/v4l2-ioctl.c
+++ b/drivers/media/v4l2-core/v4l2-ioctl.c
@@ -1308,6 +1308,8 @@ static void v4l_fill_fmtdesc(struct v4l2_fmtdesc *fmt)
case V4L2_META_FMT_VSP1_HGO: descr = "R-Car VSP1 1-D Histogram"; break;
case V4L2_META_FMT_VSP1_HGT: descr = "R-Car VSP1 2-D Histogram"; break;
case V4L2_META_FMT_UVC: descr = "UVC payload header metadata"; break;
+ case V4L2_META_FMT_RK_ISP1_PARAMS: descr = "Rockchip ISP1 3A params"; break;
+ case V4L2_META_FMT_RK_ISP1_STAT_3A: descr = "Rockchip ISP1 3A statistics"; break;
default:
/* Compressed formats */
diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
index 9d9705ceda768b..e1fa8e3089f83e 100644
--- a/include/uapi/linux/videodev2.h
+++ b/include/uapi/linux/videodev2.h
@@ -750,6 +750,10 @@ struct v4l2_pix_format {
#define V4L2_META_FMT_UVC v4l2_fourcc('U', 'V', 'C', 'H') /* UVC Payload Header metadata */
#define V4L2_META_FMT_D4XX v4l2_fourcc('D', '4', 'X', 'X') /* D4XX Payload Header metadata */
+/* Vendor specific - used for RK_ISP1 camera sub-system */
+#define V4L2_META_FMT_RK_ISP1_PARAMS v4l2_fourcc('R', 'K', '1', 'P') /* Rockchip ISP1 params */
+#define V4L2_META_FMT_RK_ISP1_STAT_3A v4l2_fourcc('R', 'K', '1', 'S') /* Rockchip ISP1 3A statistics */
+
/* priv field value to indicates that subsequent fields are valid. */
#define V4L2_PIX_FMT_PRIV_MAGIC 0xfeedcafe
--
GitLab