Add documentation and update references in current documentation for the
Extended API.

Signed-off-by: Helen Koike <helen.koike@collabora.com>
---
Changes in v6:
- Update note saying ext_api should be used for new applications on
  newer kernels (Tomasz and Hans)
- Fix typos pointed in v5 (Hand and Tomasz)
- Change order, mention Ext first in format.rst (Tomasz)
- Mention planes[i].offset should be set to zero for userptr
- Remove ext_create_buf and ext_prep_buf from the docs
- s/displayed/consumed for output (Tomasz)
- Remove references for plane length
- Drop EIO sentence mentioning signal loss (Hans)
- Removed first half of the note in EIO (Tomas and Hans)
- Update text to mention EXT_TRY_FMT is mandatory (Hans and Tomasz)
- Remove requirement to fill `memory` field for dqbuf (Tomasz)
- EXT_DQBUF sets `m` field to zero (Tomasz for DMA-fd)

Changes in v5:
- new patch

Simplify Multi/Single planer API handling by converting to v4l2_ext_pix_format.

Signed-off-by: Boris Brezillon <boris.brezillon@collabora.com>
Signed-off-by: Helen Koike <helen.koike@collabora.com>
---
Changes in v6:
- Update with new format and buffer structs

Changes in v4:
- Update with new format and buffer structs
- Rebased on top of media/master (post 5.8-rc1)

Changes in v3:
- Rebased on top of media/master (post 5.4-rc1)

Changes in v2:
- New patch

Simplify Multi/Single planer API handling by converting to v4l2_ext_pix_format.

Duplicate v4l2_ioctl_ops for touch devices. This is done to force the
framework to use the ext hooks when the classic Api is used from
userspace in Vid devices, and to keep touch devices with classic hook.

Signed-off-by: Boris Brezillon <boris.brezillon@collabora.com>
Signed-off-by: Helen Koike <helen.koike@collabora.com>
---
Changes in v6:
- Update with new format and buffer structs
- duplicate v4l2_ioctl_ops for touch devices.

Changes in v4:
- Update with new format and buffer structs
- Rebased on top of media/master (post 5.8-rc1)

Changes in v3:
- Rebased on top of media/master (post 5.4-rc1)

Changes in v2:
- New patch
---

Just a new version of v4l2_fill_mbus_format() and v4l2_fill_ext_pix_format()
to deal with the new v4l2_ext_pix_format struct.
This is needed to convert the VIMC driver to the EXT_FMT/EXT_BUF iocts.

Signed-off-by: Boris Brezillon <boris.brezillon@collabora.com>
Signed-off-by: Helen Koike <helen.koike@collabora.com>
---
Changes in v6:
- Rename v4l2_fill_ext_pix_format() to v4l2_fill_ext_pix_format_from_mbus() (Tomasz)

Changes in v4:
- Add helper v4l2_fill_ext_pix_format()
- Rebased on top of media/master (post 5.8-rc1)

Changes in v3:
- Rebased on top of media/master (post 5.4-rc1)

Changes in v2:
- New patch

Add vb2 ext hooks and call vb2_set_pixelformat().
This allows more flexibility with buffer handling.

Signed-off-by: Helen Koike <helen.koike@collabora.com>

---
Changes in v6:
- New patch to exemplify how drivers would easily support features from Ext Buf

Add vb2 ext hooks and call vb2_set_pixelformat().
This allows more flexibility with buffer handling.

Signed-off-by: Helen Koike <helen.koike@collabora.com>
---
Changes in v6:
- New patch to exemplify how drivers would easily support features from Ext Buf

To overcome the limitations of Ext ioctls, that is being converted to
classic hooks, add helpers to allow applications support layouts such as
using the same buffer with planes in different offsets.

To use the new hooks, drivers should:

 static const struct v4l2_ioctl_ops ioctl_ops = {
 ...
 +	.vidioc_ext_qbuf = vb2_ioctl_ext_qbuf,
 +	.vidioc_ext_dqbuf = vb2_ioctl_ext_dqbuf,
 ...
 }

 +	vb2_set_pixelformat(dev->pixelformat)

The old hooks should be kept to keep the driver compatible with classic
Api.

Add mem_offset field to struct vb2_plane, to allow tracking where the plane
starts in a buffer, as defined from userspace.
When returning the buffer to userspace, this offset can be adjusted
depending on the data_offset returned from the driver.

Add pixelformat field to struct vb2_buffer, to allow vb2 to know how to
decompose the payload set with vb2_set_plane_payload() into color planes
when a single memory buffer is used.

Signed-off-by: Boris Brezillon <boris.brezillon@collabora.com>
Signed-off-by: Helen Koike <helen.koike@collabora.com>

---
Changes in v6:
This patch is based on original "media: videobuf2: Expose helpers to implement the _ext_fmt and _ext_buf hooks"

- This patch was completly refactored
- Conversions from v4l2_buffer to v4l2_ext_buffer was removed from vb2.
  Both v4l2_buffer and v4l2_ext_buffer need to be supported, since Ext is only valid
  for video devices, v4l2_buffer needs to be supported for vbi, meta, and others.
- Zero v4l2_ext_buffer.planes.m field (Tomasz for DMA-fd)

Changes in v5:
- Update with new format and buffer structs
- Updated commit message with the uAPI prefix

Changes in v4:
- Update with new format and buffer structs
- Fix some bugs caught by v4l2-compliance
- Rebased on top of media/master (post 5.8-rc1)

Changes in v3:
- Rebased on top of media/master (post 5.4-rc1)

Changes in v2:
- New patch

Reorganize flags handling to be easily reuseble when Ext functions get
added.
No logic is changed, just moving around code.

- Two new functions:
	v4l2_clear_buffer_flags()
	vb2_fill_vb2_v4l2_buffer_flags()
- set_buffer_cache_hints() receives a pointer to flags instead of the
  v4l2_buffer object, making it undependent of this struct.

Signed-off-by: Helen Koike <helen.koike@collabora.com>

---
Changes in v6:
- New patch

Those extended buffer ops have several purpose:
1/ Fix y2038 issues by converting the timestamp into an u64 counting
   the number of ns elapsed since 1970
2/ Unify single/multiplanar handling
3/ Add a new start offset field to each v4l2 plane buffer info struct
   to support the case where a single buffer object is storing all
   planes data, each one being placed at a different offset

New hooks are created in v4l2_ioctl_ops so that drivers can start using
these new objects.

Note that the timecode field is gone, since there doesn't seem to be
in-kernel users. We can be added back in the reserved area if needed or
use the Request API to collect more metadata information from the
frame.

Signed-off-by: Hans Verkuil <hans.verkuil@cisco.com>
Signed-off-by: Boris Brezillon <boris.brezillon@collabora.com>
Signed-off-by: Helen Koike <helen.koike@collabora.com>
---

Changes in v6:
This patch was completely refactored, and based on previous version from
Hans and Boris.
- Refactor conversions v4l2_buffer <-> v4l2_ext_buffer for (d)qbuf
- I removed EXT_CREATE_BUFS since it is basically only usefull to MMAP.
  If this is going towards DMA-fd centric, then we can use the current
  REQUESTBUF to switch to it, and we can think a better way to support
  MMAP later if there are usecases.
  I also moved memory field from v4l2_ext_plane to v4l2_ext_buffer,
  since it is very unlikely to mix memory types, and REQUESTBUF can
  switch the whole buffer object to a given type.
- I removed EXT_QUERYBUF, since it is only useful to MMAP, for the
  same reason above.
- I removed EXT_PREPARE_BUF, since it is basically just an optimization,
  we can add it later (my intention is to simplify this patchset).
- These ioctls are only valid for video types (and not for overlay,
  vbi, touch, meta, etc).
- Refactor struct v4l2_ext_buffer and struct v4l2_ext_planes as
  discussed with Tomasz:
	- add bytesused back
	- remove lenght field
	- move memory field from planes to buffer object
- Fix order in documentation of struct v4l2_ext_buffer (Tomasz)
- Fix flags documentation of struct v4l2_ext_buffer, don't say when flags are ignored (Tomasz)
- v4l_print_ext_buffer(): print request_fd and offset/userptr (Tomasz)

Changes in v5:
- migrate memory from v4l2_ext_buffer to v4l2_ext_plane
- return mem_offset to struct v4l2_ext_plane
- change sizes and reorder fields to avoid holes in the struct and make
  it the same for 32 and 64 bits

Changes in v4:
- Use v4l2_ext_pix_format directly in the ioctl, drop v4l2_ext_format,
making V4L2_BUF_TYPE_VIDEO_[OUTPUT,CAPTURE] the only valid types.
- Drop VIDIOC_EXT_EXPBUF, since the only difference from VIDIOC_EXPBUF
was that with VIDIOC_EXT_EXPBUF we could export multiple planes at once.
I think we can add this later, so I removed it from this RFC to simplify it.
- Remove num_planes field from struct v4l2_ext_buffer
- Add flags field to struct v4l2_ext_create_buffers
- Reformulate struct v4l2_ext_plane
- Fix some bugs caught by v4l2-compliance
- Rebased on top of media/master (post 5.8-rc1)

Changes in v3:
- Rebased on top of media/master (post 5.4-rc1)

Changes in v2:
- Add reserved space to v4l2_ext_buffer so that new fields can be added
  later on

Signed-off-by: Helen Koike <helen.koike@collabora.com>

This is part of the multiplanar and singleplanar unification process.
v4l2_ext_pix_format is supposed to work for both cases.

We also add the concept of modifiers already employed in DRM to expose
HW-specific formats (like tiled or compressed formats) and allow
exchanging this information with the DRM subsystem in a consistent way.

Note that only V4L2_BUF_TYPE_VIDEO_[OUTPUT,CAPTURE] are accepted in
v4l2_ext_format, other types will be rejected if you use the
{G,S,TRY}_EXT_PIX_FMT ioctls.

New hooks have been added to v4l2_ioctl_ops to support those new ioctls
in drivers, but, in the meantime, the core takes care of converting
{S,G,TRY}_EXT_PIX_FMT requests into {S,G,TRY}_FMT so that old drivers can
still work if the userspace app/lib uses the new ioctls.

The conversion is also done the other around to allow userspace
apps/libs using {S,G,TRY}_FMT to work with drivers implementing the
_ext_ hooks.

Signed-off-by: Boris Brezillon <boris.brezillon@collabora.com>
Signed-off-by: Helen Koike <helen.koike@collabora.com>
---

Changes in v6:
 The main change here was fixing the conversion, so planes reflects color planes,
 and to implement this properly I made major refactors compared to the previous
 version.
- struct v4l2_plane_ext_pix_format removed, using struct v4l2_plane_pix_format instead (Tomasz)
- refer to drm_fourcc.h in struct v4l2_ext_pix_format docs (Hans)
- reorder colorimetry fields in struct v4l2_ext_pix_format (Hans)
- do not set Ext ioctls as valid for vid_out_overlay (Tomasz)
- refactor conversion functions, so planes are color planes (Tomasz)
- Don't explicitly check for e->modifier != 0 in v4l2_ext_pix_format_to_format() (Tomasz)
- Use "ef" for extended formats in the framework for consistency (Tomasz)
- Handle xfer_func field in conversions (Tomasz)
- Zero reserved fields in v4l_s_ext_pix_fmt() and v4l_try_ext_pix_fmt() (Tomasz)
- Refactor format functions to use v4l_fmt_ioctl_via_ext()
- Several fixes/refactoring/changes
- Remove EXT API for touch devices

Changes in v5:
- change sizes and reorder fields to avoid holes in the struct and make
  it the same for 32 and 64 bits
- removed __attribute__ ((packed)) from uapi structs
- Fix doc warning from make htmldocs
- Updated commit message with EXT_PIX prefix for the ioctls.

Changes in v4:
- Use v4l2_ext_pix_format directly in the ioctl, drop v4l2_ext_format,
making V4L2_BUF_TYPE_VIDEO_[OUTPUT,CAPTURE] the only valid types.
- Add reserved fields
- Removed num_planes from struct v4l2_ext_pix_format
- Removed flag field from struct v4l2_ext_pix_format, since the only
  defined value is V4L2_PIX_FMT_FLAG_PREMUL_ALPHA only used by vsp1,
  where we can use modifiers, or add it back later through the reserved
  bits.
- In v4l2_ext_format_to_format(), check if modifier is != MOD_LINEAR &&
  != MOD_INVALID
- Fix type assignment in v4l_g_fmt_ext_pix()
- Rebased on top of media/master (post 5.8-rc1)

Changes in v3:
- Rebased on top of media/master (post 5.4-rc1)

Changes in v2:
- Move the modifier in v4l2_ext_format (was formerly placed in
  v4l2_ext_plane)
- Fix a few bugs in the converters and add a strict parameter to
  allow conversion of uninitialized/mis-initialized objects

Add normalization to pixelformats, so we can fallback to it when using
Ext API, and eliminating the handling of two variantes (M and non-M
formats).

Signed-off-by: Helen Koike <helen.koike@collabora.com>

---
Changes in v6:
- New patch

request_fd is validated under media_request_get_by_fd() just below this
check. Thus remove it.

Suggested-by: Tomasz Figa <tfiga@chromium.org>
Signed-off-by: Helen Koike <helen.koike@collabora.com>

sizeimage field should be set to zero for unused planes, even when
v4l2_pix_format_mplane.num_planes is smaller then the index of planes.

Signed-off-by: Helen Koike <helen.koike@collabora.com>

---

I caught this with v4l2-compliance, which throws an error if we dirty
planes, even if invalid, so I would like to make it clear in the docs.

Print capabilities field from struct v4l2_create_buffers for better
debugging.

Signed-off-by: Helen Koike <helen.koike@collabora.com>

Replace a comma between expression statements by a semicolon.

Signed-off-by: Zheng Yongjun <zhengyongjun3@huawei.com>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>

Replace a comma between expression statements by a semicolon.

Signed-off-by: Zheng Yongjun <zhengyongjun3@huawei.com>
Acked-by: Corentin Labbe <clabbe@baylibre.com>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>

The asd allocated with v4l2_async_notifier_add_fwnode_subdev() must be
of size cal_v4l2_async_subdev, otherwise access to
cal_v4l2_async_subdev->phy will go to unallocated memory.

Fixes: 8fcb7576 ("media: ti-vpe: cal: Allow multiple contexts per subdev notifier")
Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>

Fix the following sparse warning:

drivers/media/platform/ti-vpe/cal-camerarx.c:783:32: warning: symbol 'cal_camerarx_media_ops' was not declared. Should it be static?

Signed-off-by: Zou Wei <zou_wei@huawei.com>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>

We can not make the assumption that the bound subdev is always a CSI
mux, in i.MX6UL/i.MX6ULL that is not the case. So, just get the entity
selected by source directly upstream from the CSI.

Fixes: 86e02d07 ("media: imx5/6/7: csi: Mark a bound video mux as a CSI mux")
Reported-by: Fabio Estevam <festevam@gmail.com>
Signed-off-by: Rui Miguel Silva <rmfrfs@gmail.com>
Tested-by: Fabio Estevam <festevam@gmail.com>
Tested-by: Sébastien Szymanski <sebastien.szymanski@armadeus.com>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>

Commit 86e02d07 ("media: imx5/6/7: csi: Mark a bound video mux as
a CSI mux") made an incorrect assumption that for imx7-media-csi,
the bound subdev must always be a CSI mux. On i.MX6UL/i.MX6ULL there
is no CSI mux at all, so do not return an error when the entity is not a
video mux and assign the IMX_MEDIA_GRP_ID_CSI_MUX group id only when
appropriate.

This is the same approach as done in imx-media-csi.c and it fixes the
csi probe regression on i.MX6UL.

Tested on a imx6ull-evk board.

Fixes: 86e02d07 ("media: imx5/6/7: csi: Mark a bound video mux as a CSI mux")
Signed-off-by: Fabio Estevam <festevam@gmail.com>
Signed-off-by: Rui Miguel Silva <rmfrfs@gmail.com>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>

Add support for base layer priority ID control in encoder.
This is a preparation patch to support v6.

[hverkuil: changed 54 to 51 in v4l2_ctrl_handler_init]

Signed-off-by: Dikshita Agarwal <dikshita@codeaurora.org>
Acked-by: Stanimir Varbanov <stanimir.varbanov@linaro.org>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>

This control indicates the priority id to be applied
to base layer.

[hverkuil: renumbered V4L2_CID_MPEG_VIDEO_BASELAYER_PRIORITY_ID]

Signed-off-by: Dikshita Agarwal <dikshita@codeaurora.org>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>

Add support for frame type specific min and max qp controls
for encoder.
This is a preparation patch to support v6.

Signed-off-by: Dikshita Agarwal <dikshita@codeaurora.org>
Acked-by: Stanimir Varbanov <stanimir.varbanov@linaro.org>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>

Adds bitrate control for all coding layers for h264
same as hevc.

Signed-off-by: Dikshita Agarwal <dikshita@codeaurora.org>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>

- Adds min/max qp controls for B frame for h264.
- Adds min/max qp controls for I/P/B frames for hevc similar to h264.
- Update valid range of min/max qp for hevc to accommodate 10 bit.

Signed-off-by: Dikshita Agarwal <dikshita@codeaurora.org>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>

GCC with W=2 level of kernel compiler warnings warns about the use of
uninitialised variables in a few locations. While these uninitialised
variables were not used in reality, this still produced compiler warnings.

Address this by assigning the variables to NULL and checking for NULL in
places it is not expected, returning -EIO in that case. This provides
at least some sanity checking at runtime as the compiler appears unable to
do that at compile time.

Reported-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>

Don't open-code DIV_ROUND_UP() kernel macro.

Signed-off-by: Zheng Yongjun <zhengyongjun3@huawei.com>
Reported-by: kernel test robot <lkp@intel.com>
Signed-off-by: Sean Young <sean@mess.org>
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>

When processing a MCE_RSP_GETPORTSTATUS command, the bit index to set in
ir->txports_cabled comes from response data, and isn't validated.

As ir->txports_cabled is a u8, nothing should be done if the bit index
is greater than 7.

Cc: stable@vger.kernel.org
Reported-by:  <syzbot+ec3b3128c576e109171d@syzkaller.appspotmail.com>
Signed-off-by: James Reynolds <jr@memlen.com>
Signed-off-by: Sean Young <sean@mess.org>
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>

This device is also supported.

Cc: stable@vger.kernel.org
Tested-by: Georgi Bakalski <georgi.bakalski@gmail.com>
Reported-by: Georgi Bakalski <georgi.bakalski@gmail.com>
Signed-off-by: Sean Young <sean@mess.org>
Reviewed-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>

Now when everything is in place wire up buffer requirements
from hfi platform buffers to the buffer requirements helper.

Signed-off-by: Stanimir Varbanov <stanimir.varbanov@linaro.org>
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>

Handle progressive/interlaced bitstream event by similar way as
bit depth.

Signed-off-by: Stanimir Varbanov <stanimir.varbanov@linaro.org>
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>

Wire up hfi platform codec and capabilities instead of
getting them from firmware.

Signed-off-by: Stanimir Varbanov <stanimir.varbanov@linaro.org>
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>

Add a new file for hfi platform buffer size and count calculations
for v6.

Signed-off-by: Stanimir Varbanov <stanimir.varbanov@linaro.org>
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>

>From Venus v6 and beyond the buffer size and count have to be
calculated in the v4l2 driver instead of getting them from
firmware.

Signed-off-by: Stanimir Varbanov <stanimir.varbanov@linaro.org>
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>

Add new hfi platform file with capabilities of hfi v6.

Signed-off-by: Stanimir Varbanov <stanimir.varbanov@linaro.org>
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>

Starting from v6 we have one more hfi property which will be needed
to calculate buffer sizes/count for particular codec and session type.

Signed-off-by: Stanimir Varbanov <stanimir.varbanov@linaro.org>
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>

Add new file for Venus hfi v4 with supported codecs and capabilities.

Signed-off-by: Stanimir Varbanov <stanimir.varbanov@linaro.org>
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>

Add ops to get the supported by the platform codecs and capabilities.

Signed-off-by: Stanimir Varbanov <stanimir.varbanov@linaro.org>
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>

Now when we have hfi platform make venus capabilities an
hfi platform capabilities.

Signed-off-by: Stanimir Varbanov <stanimir.varbanov@linaro.org>
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>

Introduce a new hfi platform to cover differences between hfi
versions. As a start move vpp/vsp freq data in that hfi
platform, more platform data will come later.

Signed-off-by: Stanimir Varbanov <stanimir.varbanov@linaro.org>
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>