Skip to content
Snippets Groups Projects
  1. Jul 03, 2019
    • Helen Koike's avatar
      MAINTAINERS: add entry for Rockchip ISP1 driver · f38830c9
      Helen Koike authored
      
      Add MAINTAINERS entry for the rockchip isp1 driver.
      
      Signed-off-by: default avatarHelen Koike <helen.koike@collabora.com>
      f38830c9
    • Shunqian Zheng's avatar
      arm64: dts: rockchip: add rx0 mipi-phy for rk3399 · 4371f68c
      Shunqian Zheng authored
      
      It's a Designware MIPI D-PHY, used for ISP0 in rk3399.
      
      Signed-off-by: default avatarShunqian Zheng <zhengsq@rock-chips.com>
      Signed-off-by: default avatarJacob Chen <jacob2.chen@rock-chips.com>
      [update for upstream]
      Signed-off-by: default avatarHelen Koike <helen.koike@collabora.com>
      
      Series-changes: 7
      - add phy-cells
      4371f68c
    • Shunqian Zheng's avatar
      arm64: dts: rockchip: add isp0 node for rk3399 · 4cf2aa22
      Shunqian Zheng authored
      
      rk3399 have two ISP, but we havn't test isp1, so just add isp0 at present.
      
      Signed-off-by: default avatarShunqian Zheng <zhengsq@rock-chips.com>
      Signed-off-by: default avatarJacob Chen <jacob2.chen@rock-chips.com>
      [update for upstream]
      Signed-off-by: default avatarHelen Koike <helen.koike@collabora.com>
      
      Series-changes: 7
      - add phy properties
      - add ports
      4cf2aa22
    • Jacob Chen's avatar
      dt-bindings: Document the Rockchip MIPI RX D-PHY bindings · d4b5d064
      Jacob Chen authored
      
      Add DT bindings documentation for Rockchip MIPI D-PHY RX
      
      Signed-off-by: default avatarJacob Chen <jacob2.chen@rock-chips.com>
      Reviewed-by: default avatarRob Herring <robh@kernel.org>
      [update for upstream]
      Signed-off-by: default avatarHelen Koike <helen.koike@collabora.com>
      
      Series-changes: 7
      - updated doc with new design and tested example
      d4b5d064
    • Jacob Chen's avatar
      dt-bindings: Document the Rockchip ISP1 bindings · 9c1db8a5
      Jacob Chen authored
      
      Add DT bindings documentation for Rockchip ISP1
      
      Signed-off-by: default avatarJacob Chen <jacob2.chen@rock-chips.com>
      Reviewed-by: default avatarRob Herring <robh@kernel.org>
      [update for upstream]
      Signed-off-by: default avatarHelen Koike <helen.koike@collabora.com>
      
      Series-changes: 7
      - update document with new design and tested example
      9c1db8a5
    • Jacob Chen's avatar
      media: rkisp1: add rockchip isp1 core driver · 51d9a916
      Jacob Chen authored
      
      Add the core driver for rockchip isp1.
      
      Signed-off-by: default avatarJacob Chen <jacob2.chen@rock-chips.com>
      Signed-off-by: default avatarShunqian Zheng <zhengsq@rock-chips.com>
      Signed-off-by: default avatarYichong Zhong <zyc@rock-chips.com>
      Signed-off-by: default avatarJacob Chen <cc@rock-chips.com>
      Signed-off-by: default avatarEddie Cai <eddie.cai.linux@gmail.com>
      Signed-off-by: default avatarJeffy Chen <jeffy.chen@rock-chips.com>
      Signed-off-by: default avatarAllon Huang <allon.huang@rock-chips.com>
      Signed-off-by: default avatarTomasz Figa <tfiga@chromium.org>
      [fixed compilation and run time errors regarding new v4l2 async API]
      Signed-off-by: default avatarLaurent Pinchart <laurent.pinchart@ideasonboard.com>
      [Add missing module device table]
      Signed-off-by: default avatarEzequiel Garcia <ezequiel@collabora.com>
      [update for upstream]
      Signed-off-by: default avatarHelen Koike <helen.koike@collabora.com>
      
      Series-changes: 7
      - VIDEO_ROCKCHIP_ISP1 selects VIDEOBUF2_VMALLOC
      - add PHY_ROCKCHIP_DPHY as a dependency for VIDEO_ROCKCHIP_ISP1
      - Fix compilation and runtime errors due to bitrotting
      The code has bit-rotten since March 2018, fix compilation errors.
      The new V4L2 async notifier API requires notifiers to be initialized by
      a call to v4l2_async_notifier_init() before being used, do so.
      - Add missing module device table
      - use clk_bulk framework
      - add missing notifiers cleanups
      - s/strlcpy/strscpy
      - normalize bus_info name
      - fix s_stream error path, stream_cnt wans't being decremented properly
      - use devm_platform_ioremap_resource() helper
      - s/deice/device
      - redesign: remove mipi/csi subdevice, sensors connect directly to the
      isp subdevice in the media topology now.
      - remove "saved_state" member from rkisp1_stream struct
      - Reverse the order of MIs
      - Simplify MI interrupt handling
      Rather than adding unnecessary indirection, just use stream index to
      handle MI interrupt enable/disable/clear, since the stream index matches
      the order of bits now, thanks to previous patch. While at it, remove
      some dead code.
      - code styling and checkpatch fixes
      51d9a916
    • Jacob Chen's avatar
      media: rkisp1: add capture device driver · f8fc6c7d
      Jacob Chen authored
      
      This is the capture device interface driver that provides the v4l2
      user interface. Frames can be received from ISP1.
      
      Signed-off-by: default avatarJacob Chen <jacob2.chen@rock-chips.com>
      Signed-off-by: default avatarShunqian Zheng <zhengsq@rock-chips.com>
      Signed-off-by: default avatarYichong Zhong <zyc@rock-chips.com>
      Signed-off-by: default avatarJacob Chen <cc@rock-chips.com>
      Signed-off-by: default avatarEddie Cai <eddie.cai.linux@gmail.com>
      Signed-off-by: default avatarJeffy Chen <jeffy.chen@rock-chips.com>
      Signed-off-by: default avatarAllon Huang <allon.huang@rock-chips.com>
      Signed-off-by: default avatarTomasz Figa <tfiga@chromium.org>
      [updated for upstream]
      Signed-off-by: default avatarHelen Koike <helen.koike@collabora.com>
      
      Series-changes: 7
      - s/strlcpy/strscpy
      - Fix v4l2-compliance issues:
      	* remove input ioctls
      media api can be used to define the topology, this input api is not
      required. Besides it, if an input is enumerated, v4l2-compliance is not
      happy with G_FMT returning the default colorspace instead of something
      more specific.
      	* return the pixelformat to the userspace
      G_/S_/TRY_ FORMAT should return a valid pixelformat to the user, even if
      the user gave an invalid one
      	* add missing default colorspace and ycbcr
      	* fix wrong pixformat in mp_fmts[] table
      	* add buf type check in s_/g_selection
      	* queue_setup - check sizes
      	* normalize bus_info name
      	* fix field any v4l2-compliance -s complain - set field none
      	when streaming
      - Fix compiling error: s/vidioc_enum_fmt_vid_cap_mplane/vidioc_enum_fmt_vid_cap
      - Replace stream state with a boolean
      The rkisp1_state enum consists only of 3 entries, where 1 is completely
      unused and the other two respectively mean not streaming or streaming.
      Replace it with a boolean called "streaming".
      - Simplify MI interrupt handling
      Rather than adding unnecessary indirection, just use stream index to
      handle MI interrupt enable/disable/clear, since the stream index matches
      the order of bits now, thanks to previous patch. While at it, remove
      some dead code.
      - code styling and checkpatch fixes
      - add link_validate: don't allow a link with bayer/non-bayer mismatch
      
      Commit-notes:
      Hi,
      
      Hans, I removed your Acked-by tag since some things changed. Please let
      me know if I can keep it.
      
      Thanks
      Helen
      END
      f8fc6c7d
    • Jacob Chen's avatar
      media: rkisp1: add ISP1 params driver · 3f80f8e7
      Jacob Chen authored
      
      Add the output video driver that accept params from userspace.
      
      Signed-off-by: default avatarJacob Chen <jacob2.chen@rock-chips.com>
      Signed-off-by: default avatarShunqian Zheng <zhengsq@rock-chips.com>
      Signed-off-by: default avatarYichong Zhong <zyc@rock-chips.com>
      Signed-off-by: default avatarJacob Chen <cc@rock-chips.com>
      Signed-off-by: default avatarEddie Cai <eddie.cai.linux@gmail.com>
      Signed-off-by: default avatarJeffy Chen <jeffy.chen@rock-chips.com>
      Signed-off-by: default avatarAllon Huang <allon.huang@rock-chips.com>
      Signed-off-by: default avatarTomasz Figa <tfiga@chromium.org>
      [update for upstream]
      Signed-off-by: default avatarHelen Koike <helen.koike@collabora.com>
      
      Series-changes: 7
      - s/strlcpy/strscpy
      - s/strcpy/strscpy
      - fix config lsc error
      LSC data table size is 17x17, but when configuring data to ISP,
      should be aligned to 18x17. That means every last data of last
      line should be filled with 0, and not filled with the data of
      next line.
      - Update new ISP parameters immediately
      For those sub modules that have shadow registers in core isp, the
      new programing parameters would not be active if both
      CIF_ISP_CTRL_ISP_CFG_UPD_PERMANENT and CFG_UPD are not set. Now
      we configure CFG_UPD to force update the shadow registers when new
      ISP parameters are configured.
      - fix some ISP parameters config error
      Some ISP parameter config functions may override the old enable
      bit value, because the enable bits of these modules are in the
      same registers with parameters. So we should save the old enable
      bits firstly.
      - code styling and checkpatch fixes
      3f80f8e7
    • Jacob Chen's avatar
      media: rkisp1: add ISP1 statistics driver · d286ec94
      Jacob Chen authored
      
      Add the capture video driver for rockchip isp1 statistics block.
      
      Signed-off-by: default avatarJacob Chen <jacob2.chen@rock-chips.com>
      Signed-off-by: default avatarShunqian Zheng <zhengsq@rock-chips.com>
      Signed-off-by: default avatarYichong Zhong <zyc@rock-chips.com>
      Signed-off-by: default avatarJacob Chen <cc@rock-chips.com>
      Signed-off-by: default avatarEddie Cai <eddie.cai.linux@gmail.com>
      Signed-off-by: default avatarJeffy Chen <jeffy.chen@rock-chips.com>
      Signed-off-by: default avatarAllon Huang <allon.huang@rock-chips.com>
      Signed-off-by: default avatarTomasz Figa <tfiga@chromium.org>
      [update for upstream]
      Signed-off-by: default avatarHelen Koike <helen.koike@collabora.com>
      Acked-by: default avatarHans Verkuil <hans.verkuil@cisco.com>
      
      Series-changes: 7
      - s/strlcpy/strscpy
      - sort out the locks in isp stats
      - code styling and checkpatch fixes
      d286ec94
    • Jacob Chen's avatar
      media: rkisp1: add Rockchip ISP1 subdev driver · 642a4e42
      Jacob Chen authored
      
      Add the subdev driver for rockchip isp1.
      
      Signed-off-by: default avatarJacob Chen <jacob2.chen@rock-chips.com>
      Signed-off-by: default avatarShunqian Zheng <zhengsq@rock-chips.com>
      Signed-off-by: default avatarYichong Zhong <zyc@rock-chips.com>
      Signed-off-by: default avatarJacob Chen <cc@rock-chips.com>
      Signed-off-by: default avatarEddie Cai <eddie.cai.linux@gmail.com>
      Signed-off-by: default avatarJeffy Chen <jeffy.chen@rock-chips.com>
      Signed-off-by: default avatarAllon Huang <allon.huang@rock-chips.com>
      Signed-off-by: default avatarTomasz Figa <tfiga@chromium.org>
      [fixed unknown entity type / switched to PIXEL_RATE]
      Signed-off-by: default avatarEzequiel Garcia <ezequiel@collabora.com>
      [update for upstream]
      Signed-off-by: default avatarHelen Koike <helen.koike@collabora.com>
      
      Series-changes: 7
      - fixed warning because of unknown entity type
      - fixed v4l2-compliance errors regarding rkisp1 formats, try formats
      and default values
      - fix typo riksp1/rkisp1
      - redesign: remove mipi/csi subdevice, sensors connect directly to the
      isp subdevice in the media topology now. As a consequence, remove the
      hack in mipidphy_g_mbus_config() where information from the sensor was
      being propagated through the topology.
      - From the old dphy:
      	* cache get_remote_sensor() in s_stream
      	* use V4L2_CID_PIXEL_RATE instead of V4L2_CID_LINK_FREQ
      - Replace stream state with a boolean
      - code styling and checkpatch fixes
      - fix stop_stream (return after calling stop, do not reenable the stream)
      - fix rkisp1_isp_sd_get_selection when V4L2_SUBDEV_FORMAT_TRY is set
      - fix get format in output (isp_sd->out_fmt.mbus_code was being ignored)
      - s/intput/input
      - remove #define sd_to_isp_sd(_sd), add a static inline as it will be
      reused by the capture
      642a4e42
    • Jacob Chen's avatar
      media: rkisp1: add Rockchip MIPI Synopsys DPHY driver · c09331d5
      Jacob Chen authored
      
      This commit adds a subdev driver for Rockchip MIPI Synopsys DPHY driver
      
      Signed-off-by: default avatarJacob Chen <jacob2.chen@rock-chips.com>
      Signed-off-by: default avatarShunqian Zheng <zhengsq@rock-chips.com>
      Signed-off-by: default avatarTomasz Figa <tfiga@chromium.org>
      [migrate to phy framework]
      Signed-off-by: default avatarEzequiel Garcia <ezequiel@collabora.com>
      [update for upstream]
      Signed-off-by: default avatarHelen Koike <helen.koike@collabora.com>
      
      Series-changes: 7
      - Migrate dphy specific code from
      drivers/media/platform/rockchip/isp1/mipi_dphy_sy.c
      to drivers/phy/rockchip/phy-rockchip-dphy.c
      - Drop support for rk3288
      - Drop support for dphy txrx
      - code styling and checkpatch fixes
      c09331d5
    • Jeffy Chen's avatar
      media: rkisp1: Add user space ABI definitions · bc6e4888
      Jeffy Chen authored
      
      Add the header for userspace
      
      Signed-off-by: default avatarJeffy Chen <jeffy.chen@rock-chips.com>
      Signed-off-by: default avatarJacob Chen <jacob2.chen@rock-chips.com>
      [update for upstream]
      Signed-off-by: default avatarHelen Koike <helen.koike@collabora.com>
      
      Series-changes: 7
      - Fix checkpatch errors (lines over 80 and SPDX)
      - Add TODO to improve docs
      
      Commit-notes:
      Hi,
      
      I don't have the "REF_01 - ISP_user_manual, Rev 2.57" document that was
      mentioned in previous version, so I'm adding a TODO for now to improve
      the docs once we collect the information.
      If Rockchip people could help here it would be great.
      
      Thanks
      Helen
      END
      bc6e4888
    • Jacob Chen's avatar
      media: doc: add document for rkisp1 meta buffer format · 8e862bc7
      Jacob Chen authored
      
      This commit add document for rkisp1 meta buffer format
      
      Signed-off-by: default avatarJacob Chen <jacob-chen@rock-chips.com>
      Acked-by: default avatarHans Verkuil <hans.verkuil@cisco.com>
      [update for upstream]
      Signed-off-by: default avatarHelen Koike <helen.koike@collabora.com>
      
      Series-changes: 7
      - s/correspond/corresponding
      - s/use/uses
      - s/docuemnt/document
      8e862bc7
    • Shunqian Zheng's avatar
      media: videodev2.h, v4l2-ioctl: add rkisp1 meta buffer format · 76737ade
      Shunqian Zheng authored
      
      Add the Rockchip ISP1 specific processing parameter format
      V4L2_META_FMT_RK_ISP1_PARAMS and metadata format
      V4L2_META_FMT_RK_ISP1_STAT_3A for 3A.
      
      Signed-off-by: default avatarShunqian Zheng <zhengsq@rock-chips.com>
      Signed-off-by: default avatarJacob Chen <jacob2.chen@rock-chips.com>
      Acked-by: default avatarHans Verkuil <hans.verkuil@cisco.com>
      [update for upstream]
      Signed-off-by: default avatarHelen Koike <helen.koike@collabora.com>
      
      Series-to: linux-rockchip@lists.infradead.org
      Series-cc: linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org
      Series-cc: mchehab@kernel.org, linux-media@vger.kernel.org
      Series-cc: sakari.ailus@linux.intel.com, hans.verkuil@cisco.com
      Series-cc: tfiga@chromium.org, zhengsq@rock-chips.com
      Series-cc: laurent.pinchart@ideasonboard.com, zyc@rock-chips.com
      Series-cc: eddie.cai.linux@gmail.com, jeffy.chen@rock-chips.com
      Series-cc: devicetree@vger.kernel.org, heiko@sntech.de
      Series-cc: jacob2.chen@rock-chips.com
      Series-cc: kernel@collabora.com, ezequiel@collabora.com
      Series-version: 7
      Series-changes: 7
      - s/IPU3/RK_ISP1
      
      Cover-letter:
      Rockchip ISP1 Driver
      Hello,
      
      I'm re-sending a new version of ISP(Camera) v4l2 driver for rockchip
      rk3399 SoC.
      
      It is not perfect yet (see known issues below), but I'm sending in case
      some other people already wants to start playing with it.
      I believe de main design is ok (please let me know if
      it is not) and it would be great to get some reviews already.
      
      This patchset is also available at:
      https://gitlab.collabora.com/koike/linux/tree/rockchip/isp/v7
      
      Libcamera patched to work with this version:
      https://gitlab.collabora.com/koike/libcamera
      (I'll also sent it to the libcamera dev mailing list)
      
      I tested on the rockpi 4 with a rpi v1.3 sensor and also with the
      Scarlet Chromebook.
      Images from the Scarlet are a bit dark and green for some reason, but I
      believe it's a problem in the sensor's drivers as images from the
      rockpi looks ok.
      
      The main differences from previous version are (in a macro pov):
      ----------------------------------------------------------------
      - dphy specific code migrated to drivers/phy
      - design change: droped the subdevice for the interface. Now, in the
      media topology, the sensors connect directly to the ISP1.
      - v4l2-compliance fixes
      - dropped rk3288 (as I'm not testing it)
      - dropped txrx dphy support (as I'm not testing it and it requires a bit
      more work to support dsi too)
      - interrupts and hw config fixes
      - minor bug fixes and cleanups
      - I added myself in the MAINTAINERS, as I'm not sure if previous people
      still wants to maintain it, please let me know if I should keep the old
      names there.
      
      Known issues:
      -------------
      - Reloading the module doesn't work (there is some missing cleanup when
      unloading)
      - When capturing in bayer format, changing the size doesn't seem to
      affect the image.
      - crop needs more tests
      - v4l2-compliance error:
              fail: v4l2-test-controls.cpp(824): subscribe event for control 'Image Processing Controls' failed
      test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: FAIL
      It seems that if controls are supported, v4l2-compliance says that
      controls of type 'Image Processing Controls' are mandatory, is this
      correct?
      - It seems there are still some issues with interrupts, but I couldn't
      isolate them yet.
      
      Reasoning for the design change:
      --------------------------------
      In the previous version, the isp subdevice call the mipidphy_g_mbus
      from rkisp1.c, so it can get
      informations from the sensor such as the type of mbus (V4L2_MBUS_BT656,
      V4L2_MBUS_PARALLEL or V4L2_MBUS_CSI2_CPHY), the number of csi2 lanes,
      flags (V4L2_MBUS_PCLK_SAMPLE_RISING, V4L2_MBUS_VSYNC_ACTIVE_LOW or
      V4L2_MBUS_HSYNC_ACTIVE_LOW). And the isp driver uses those info to configure
      the hardware properly.
      
      These information come from the DT node of the sensor. And the current
      implementation is propagating this information from the sensor to the isp
      through this mipidphy_g_mbus_config() (thus the hack)
      
      * 1st attempt to solve this hack) Separating the interface code from the isp.
      
      With the topology:
      
      isp -> csi2 -> sensor
      
      I was trying to migrate the CSI2 hardware configuration to the csi2 subdevice code.
      But the problem I found was that in the DT, the csi2 regs is in the middle of the isp1
      regs, and declaring the same "regs = <>" in both nodes (isp0 and csi2) is no good.
      
      * 2nd attempt) flatening the DT
      So instead of having two DT nodes (isp0 and csi2), we can have a single node, similar
      to omap3isp.
      And we can have a property, "rk,phy-type" that defines the interface (csi2, bt656 or parallel).
      
      But, now my question is: can the isp be connected to multiple interfaces at a
      time? If yes, then this is not a good solution (as we won't be able to describe
      multiple interfaces in the DT node).
      
      * 3rd attemp - WIP) get rid of the interface subdevice (chosen design)
      Is there a reason to have the topology like:
      
      isp -> interface - - -> sensor1
              |   |-------->  sensor2
              | - - - - - - > sensor3
      
      (only one sensor can be active at a time)
      
      ?
      
      Would it be ok if I just hide the interface from the topology? Like:
      
      isp - - - - - -> sensor1
      | |----------->  sensor2
      |- - - - - - - > sensor3
      
      Like this, I could cleanup a bunch of v4l2 code from the interface node,
      the isp would know the active sensor (and its configs), and it can
      configure everything itself.
      
      I don't really see a big reason to expose the interface (csi2,
      bt656 or parallel) in the topology.
      Unless I'm missing something.
      
      Previous changelog:
      -------------------
      
      changes in V6:
        - add mipi txrx phy support
        - remove bool and enum from uapi header
        - add buf_prepare op
        - correct some spelling problems
        - return all queued buffers when starting stream failed
      
      changes in V5: Sync with local changes,
        - fix the SP height limit
        - speed up the second stream capture
        - the second stream can't force sync for rsz when start/stop streaming
        - add frame id to param vb2 buf
        - enable luminance maximum threshold
      
      changes in V4:
        - fix some bugs during development
        - move quantization settings to rkisp1 subdev
        - correct some spelling problems
        - describe ports in dt-binding documents
      
      changes in V3:
        - add some comments
        - fix wrong use of v4l2_async_subdev_notifier_register
        - optimize two paths capture at a time
        - remove compose
        - re-struct headers
        - add a tmp wiki page: http://opensource.rock-chips.com/wiki_Rockchip-isp1
      
      changes in V2:
        mipi-phy:
          - use async probing
          - make it be a child device of the GRF
        isp:
          - add dummy buffer
          - change the way to get bus configuration, which make it possible to
                  add parallel sensor support in the future(without mipi-phy driver).
      
      ------------------
      END
      76737ade
    • Helen Koike's avatar
      tmp: rk3399-gru-scarlet.dtsi · 1c893b18
      Helen Koike authored
      1c893b18
    • Helen Koike's avatar
      tmp: ov5647 / rockpi 4 · bcf7f995
      Helen Koike authored
      bcf7f995
    • Helen Koike's avatar
      tmp: rk3399.dtsi iommu · ba8478a9
      Helen Koike authored
      ba8478a9
  2. Jun 27, 2019
  3. Jun 24, 2019
Loading