GitLab

This is to guide upcoming contributors toward what is left to do to get
toward a production element.
Signed-off-by: Nicolas Dufresne <nicolas.dufresne@collabora.com>

With this patch, the element is not able to push buffers to the next
element in the graph. The buffers are currently missing any metadata
like timestamp, sequence number. The handling of the GstFlowReturn
for multiple pads isn't using a GstFlowCombiner as it should.
Signed-off-by: Nicolas Dufresne <nicolas.dufresne@collabora.com>

These will be useful for streaming. The requestComplete callback will
store the buffers on each pads so that the _run() can pick them up
and push them through the pads from a streaming thread.
Signed-off-by: Nicolas Dufresne <nicolas.dufresne@collabora.com>

This add getters on pad/pool/allocator so that we can retrieve the
Stream or FrameBuffer.
Signed-off-by: Nicolas Dufresne <nicolas.dufresne@collabora.com>

Setup the allocation and the release of buffers in the
element. We have one pooling GstAllocator that wraps the
FrameBufferAllocator and tracks the live time of FrameBuffer
object. Then for each pads we have a GstBufferPool object
which is only used to avoid re-allocation the GstBuffer
shell everything.
Signed-off-by: Nicolas Dufresne <nicolas.dufresne@collabora.com>

This add get/set helper to store a pool on the pad.
Signed-off-by: Nicolas Dufresne <nicolas.dufresne@collabora.com>

This is needed to track the livetime of the FrameBufferAllocator in relation to
the GstBuffer/GstMemory objects travelling inside GStreamer.
Signed-off-by: Nicolas Dufresne <nicolas.dufresne@collabora.com>

Now that we have stream-start and caps, we can now push a segment event to
announce what time will our buffer correlate to. For live sources this is just
an open segment in time format.
Signed-off-by: Nicolas Dufresne <nicolas.dufresne@collabora.com>

This is not expected to work in every possible cases, but should be sufficient as
an initial implementation. What is does it that it turns the StreamFormats into
caps and query downstream caps with that as a filter.

The result is the subset of caps that can be used. We then keep the first
structure in that result and fixate using the default values found in
StreamConfiguration as a default in case a range is available.

We then validate this configuration and turn the potentially modified
configuration into caps that we push downstream. Note that we strust the order
in StreamFormats as being sorted best first, but this is not currently in
libcamera.
Signed-off-by: Nicolas Dufresne <nicolas.dufresne@collabora.com>

This add helpers to deal with the conversion from StreamConfiguration
to caps and vis-versa. This is needed to implement caps negotiation.
Signed-off-by: Nicolas Dufresne <nicolas.dufresne@collabora.com>

Prior to sending caps, we need to send a stream-start event. This requires
generating a stream and a group id. The stream id is random for live sources and
the group id is shared across all pads.
Signed-off-by: Nicolas Dufresne <nicolas.dufresne@collabora.com>

This will allow implementing generic algorithm even if we cannot
request pads yet.
Signed-off-by: Nicolas Dufresne <nicolas.dufresne@collabora.com>

Signed-off-by: Nicolas Dufresne <nicolas.dufresne@collabora.com>

Use a GstTask as our internal streaming thread. Unlike GstBaseSrc, we
will be running an streaming thread at the element level rather then
per pad. This is needed to combine buffer request for multiple pads.
Signed-off-by: Nicolas Dufresne <nicolas.dufresne@collabora.com>

This add code to select and acquire a camera. With this, it is now
possible to run pipeline like:

   gst-launch-1.0 libcamerasrc ! fakesink

Though no buffer will be streamed yet. In this function, we implement the
change_state() virtual method to trigger actions on specific state transitions.
Note that we also return GST_STATE_CHANGE_NO_PREROLL in
GST_STATE_CHANGE_READY_TO_PAUSED and GST_STATE_CHANGE_PLAYING_TO_PAUSED
transitions as this is required for all live sources.
Signed-off-by: Nicolas Dufresne <nicolas.dufresne@collabora.com>

This will allow selecting libcamerasrc traces with the following
environment:

  GST_DEBUG=libcamerasrc:7

Or all libcamera GStreamer element traces using

  GST_DEBUG="libcamera*:7"
Signed-off-by: Nicolas Dufresne <nicolas.dufresne@collabora.com>

This property will be used to select by name the camera to use.
Signed-off-by: Nicolas Dufresne <nicolas.dufresne@collabora.com>

Signed-off-by: Nicolas Dufresne <nicolas.dufresne@collabora.com>

This simply add the boiler plate for pads on the source element. The
design is that we have one pad, called "src", that will always be
present, and then more pads can be requested prior in READY or less
state. Initially pads have one property "stream-role" that let you
decide which role this pad will have.
Signed-off-by: Nicolas Dufresne <nicolas.dufresne@collabora.com>

The GMutexLocker uses GCC/CLANG C feature to implement a smart lock,
that unlocks the mutex when the GMutexLocker goes out of scope. This
could have been implemented in C++, but as this is already implemented
I decided to just reuse it. This is particularly handy to avoid gotos
in error handling cases and to prevent unbalanced locking.
Signed-off-by: Nicolas Dufresne <nicolas.dufresne@collabora.com>

This feature is used with GstDeviceMonitor in order to enumerate
and monitor devices to be used with the source element. The resulting
GstDevice implementation is also used by application to abstract the
configuration of the source element.

Implementations notes:
  - libcamera does not support polling yet
  - The device ID isn't unique in libcamera
  - The "name" property does not yet exist in libcamerasrc
Signed-off-by: Nicolas Dufresne <nicolas.dufresne@collabora.com>

This transform the basic information found in StreamFormats to GstCaps.
This can be handy to reply to early caps query or inside a device
provider. Note that we ignored generated range as they are harmful to
caps negotiation. We also don't simplify the caps for readability
reasons, so some of the discrete value may be included in a range.
Signed-off-by: Nicolas Dufresne <nicolas.dufresne@collabora.com>

Signed-off-by: Nicolas Dufresne <nicolas.dufresne@collabora.com>

Now that libcamera creates threads internally and doesn't rely on an
application-provided event loop, remove the thread from the Android
Camera HAL layer. The CameraProxy class becomes meaningless, remove it
and communicate directly from the CameraHalManager to the CameraDevice.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
---
Changes since v1:

- Remove unused CameraDevice::cameraDevice_
- Include stddef.h in camera_hal_manager.h to fix compilation of
  Android's hardware/hardware.h header

Now that libcamera creates threads internally and doesn't rely on an
application-provided event loop, remove the thread from the V4L2
compatibility layer. The split between the V4L2CameraProxy and
V4L2Camera classes is still kept to separate the V4L2 adaptation from
camera operation. This may be further refactored later.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Niklas Söderlund <niklas.soderlund@ragnatech.se>

Document the threading model of the PipelineHandler class (and all its
derived classes). The model is already enforced by the Camera class, so
no change in the implementation is required. As for the Camera class,
disconnection is currently left out.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Niklas Söderlund <niklas.soderlund@ragnatech.se>

Document the threading model of the Camera class and implement it.
Selected functions become thread-safe, and require a few functions of
the PipelineHandler class to be called through cross-thread invocation
as the pipeline handlers live in the camera manager thread, while the
Camera class is mostly accessed from the application thread. The
PipelineHandler is made to inherit from the Object class to support
this.

Disconnection is currently left out as it is not implemented in pipeline
handlers, and isn't fully supported in the Camera class either. This
will be revisited when implementing proper hotplug support.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Niklas Söderlund <niklas.soderlund@ragnatech.se>

Relying on the application event loop to process all our internal events
is a bad idea for multiple reasons. In many cases the user of libcamera
can't provide an event loop, for instance when running through one of
the adaptation layers. The Android camera HAL and V4L2 compatibility
layer create a thread for this reason, and the GStreamer element would
need to do so as well. Furthermore, relying on the application event
loop pushes libcamera's realtime constraints to the application, which
isn't manageable.

For these reasons it's desirable to always run the camera manager, the
pipeline handlers and the cameras in a separate thread. Doing so isn't
too complicated, it only involves creating the thread internally when
starting the camera manager, and synchronizing a few methods of the
Camera class. Do so as a first step towards defining the threading model
of libcamera.

The event dispatcher interface is still exposed to applications, to
enable cross-thread signal delivery if desired.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
---
Changes since v1:

- Move "libcamera: camera_manager: Move private data members to private
  implementation" and "libcamera: camera_manager: Return a copy of the
  vector from cameras()" to separate patches

Make the signal connection and disconnection thread-safe, and document
them as such. This is required to make objects connectable from
different threads.

The connect(), disconnect() and emit() methods are now all protected by
a global mutex, which may generate a high lock contention. This could be
improved with finer-grained locks or with a pool of mutexes.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
Reviewed-by: Niklas Söderlund <niklas.soderlund@ragnatech.se>

Define the thread-safety attributes of the classes and methods that are
either thread-safe or thread-bound. The CameraManager, Camera and
PipelineHandler will be addressed separately.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Niklas Söderlund <niklas.soderlund@ragnatech.se>

Document the design of libcamera's threading support, and prepare to
document thread-safety of classes and functions with a doxygen alias
command.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Niklas Söderlund <niklas.soderlund@ragnatech.se>
---
Changes since v1:

- Move the thread-reentrancy after thread-objects and thread-signals
- Add a sentence that was mistakenly part of another patch
- Fix typos

Move all accesses to the state_ and disconnected_ members to functions
of the Private class. This will make it easier to implement
synchronization, and simplifies the Camera class implementation.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Niklas Söderlund <niklas.soderlund@ragnatech.se>

Use the d-pointer idiom ([1], [2]) to hide the private data members from
the Camera class interface. This will ease maintaining ABI
compatibility, and prepares for the implementation of the Camera class
threading model.

The FrameBufferAllocator class accesses the Camera private data members
directly. In order to hide them, this pattern is replaced with new
private member functions in the Camera class, and the
FrameBufferAllocator is updated accordingly.

[1] https://wiki.qt.io/D-Pointer
[2] https://en.cppreference.com/w/cpp/language/pimplSigned-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Niklas Söderlund <niklas.soderlund@ragnatech.se>

Making CameraManager::cameras() thread-safe requires returning a copy of
the cameras vector instead of a reference. This is also required for
hot-plugging support and is thus desirable.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Niklas Söderlund <niklas.soderlund@ragnatech.se>

Use the d-pointer idiom ([1], [2]) to hide the private data members from
the CameraManager class interface. This will ease maintaining ABI
compatibility, and prepares for the implementation of the CameraManager
class threading model.

[1] https://wiki.qt.io/D-Pointer
[2] https://en.cppreference.com/w/cpp/language/pimplSigned-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Niklas Söderlund <niklas.soderlund@ragnatech.se>

The V4L2VideoDevice::exportBuffers(),
PipelineHandler::exportFrameBuffers() and
FrameBufferAllocator::allocate() functions all return the number of
allocated buffers on success, but are documented as returning 0 in that
case. Fix their documentation.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Niklas Söderlund <niklas.soderlund@ragnatech.se>

The slots list is touched from most of the Signal template functions. In
order to prepare for thread-safety, move handling of the list to a small
number of non-template functions in the SignalBase class.

This incidently fixes a bug in signal disconnection handling where the
signal wasn't removed from the object's signals list, as pointed out by
the signals unit test.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
Reviewed-by: Niklas Söderlund <niklas.soderlund@ragnatech.se>

Add two tests that exercise the Signal::disconnect(Object *) and
Signal::disconnect() methods, to verify that they correctly remove the
signal from the connected object's list of signals. This triggers an
issue that was detected through manual code inspection, and is expected
to crash or at least generate valgrind warnings.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
Reviewed-by: Niklas Söderlund <niklas.soderlund@ragnatech.se>

We use static local variables to indicate errors in methods that return
a const reference. The local variables can thus be const, make them so.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
Reviewed-by: Niklas Söderlund <niklas.soderlund@ragnatech.se>

Now that we're using C++-14, replace the manual implementation of
std::integer_sequence with std::index_sequence, a specialization of
std::integer_sequence with the integer type equal to std::size_t. The
template parameter S that denotes a sequence is replaced with I to align
with the usage examples of cppreference.com.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
Reviewed-by: Niklas Söderlund <niklas.soderlund@ragnatech.se>