Camera WG

Camera Working Group is focused on camera components and use-cases across the Dronecode platform

• Define and Implement new camera related features (image-capture/video-capture/video-streaming)
• Define and Implement protocols for communication between multiple camera entity
• Define and Implement camera related MAVLink messages
• Review camera related pull-requests
• Discuss camera related issues raised

Lead: Lalit Begani (Intel)

Contact: lalit.kumar.begani@intel.com

When: Meetings are Weekly, every Wednesday at 9:15am (Pacific Time), 9:45pm (IST)

Where: www.uberconference.com/lf-dronecode-wg | Dial in: 415-754-9912 | No PIN needed

Who: Open for all, anyone interested

• Export Camera Parameters to users : http://discuss.px4.io/t/mavlink-camera-api-discussion/3278
  • As using json resulted in complexity in defining rules, camera definition will switch to xml format
  • Different cameras in the system to be addressed using different component ID, camera ID to go
  • Mavlink messages to get/set camera param along with possible acks discussed & defined.
  • Existing storage related mavlink messages to be used with camera comp id to target camera storage.
• Video Stream discovery over MAVLink : https://github.com/mavlink/qgroundcontrol/pull/5271
  • Discussion to continue on the PR

• Scope/Roadmap :
  • Scope for discussion be kept minimal to stay focused.
  • Roadmap will be prepared soon
• Export Camera Parameters to users : http://discuss.px4.io/t/mavlink-camera-api-discussion/3278
  • Had a detailed discussion on the protocol and its various aspects
  • Everyone has agreed to the Gus’s proposal, implementation to follow
  • Discussed and agreed on the MAVlink messages to be used.

• Roadmap :
  • There need to be a roadmap for the camera related activities.
  • Gus would prepare and put it up in the DroneCode Camera-WG section
• Export Camera Parameters to users : http://discuss.px4.io/t/mavlink-camera-api-discussion/3278
  • Discussion to continue on the discuss.px4.io forum
  • Existing WIP Mavlink messages for setting camera parameters to be deprecated
  • mavlink.io will be updated with new Mavlink messages and json definition file once finalized
• Video Stream discovery over MAVLink : https://github.com/mavlink/qgroundcontrol/pull/5271
  • Will take up this feature after 3.2 release ( suppose to happen in a week)
  • Need clarity on the mavlink communication between camera daemon (that manages all cameras in drone & runs on the companion board) and QGC. Otavio will open an issue in QGC to get inputs
• Camera related User Interface:
  • There is some work ongoing to revamp the UI by Gus
  • Need to collaborate

• Regular Weekly Meeting : Every Wednesday after PX4 dev call
  • I will check on how to schedule a meeting and then send out the meeting invite.
  • Agenda for the meeting to be published a day before. Let me know if anything needs to added to the agenda.
• Video Stream discovery over MAVLink : https://github.com/mavlink/qgroundcontrol/pull/5271
  • There is a PR open for review that implements the MAVLink messages for video stream discovery/start/stop on QGC
  • Gus will review and provide inputs
  • Two key things to discuss on the PR
    • The way QGC communicates (Heartbeat messages with unique sys-id and camera comp-id) with Drone over MAVLink for the usecase
    • User interface (Video Streaming Widget)
• Export Camera Parameters to users : http://discuss.px4.io/t/mavlink-camera-api-discussion/3278
  • There is proposal on how the camera parameters can be exported by the drone to the GCS and eventually to the user.
  • Everyone need to review and provide inputs
  • Key thing to focus on –
    • Content of the JSON definition file
    • For a given usecase involving def file, sequence of specific MAVLink messages that will be sent & received.
• Support Multiple Video Stream : https://github.com/mavlink/qgroundcontrol/issues/5260
  • There is a proposal by Gus to support multiple video streams on QGC
  • Gus is working on this feature
  • 3 things that needs to be addressed to support this feature –
    • Enable QGC to instantiate multiple VideoReceivers, each for one video stream (PR merged #5318)
    • Implement/Invoke MAVLink messages to start/stop multiple streams
    • Add changes to the user interface
• Camera/Video User Experience : https://github.com/mavlink/qgroundcontrol/issues/5316
  • There is an issue with the way how/where the video source & settings are selected
  • There is a PR that addressed the issue by implementing a Widget #5271
  • Need to finalize on the final UI, perhaps after (3) & (4)
• Documentation : https://mavlink.io/en/camera_protocol.html
  • There must only be one place where all the details are published, identified the place as mavlink.io
  • Need to put together the list of camera related MAVLink messages, use cases (image/video capture, streaming etc.) supported and for a given use case, the sequence of specific MAVLink messages that are exchanged.

Camera API WG is focused on platform independent developer APIs for cameras and remote control of camera and gimbal.

APIs under investigation include:

• Remote Camera and Gimbal Control APIs (Julian Oes)
• Video Streaming APIs
• C/C++ developer camera APIs (FPV, Snapshot, Recording, Optic Flow, and Stereo)

The following submissions have been made with a request for comments:

Qualcomm Technologies Inc provided links to the platform independent Camera APIs used in Snapdragon Flight, for consideration and comment from TSC, Camera WG and the community:

• libcamera/inc/camera.h (codeaurora)
• libcamera/inc/camera_parameters.h (codeaurora)
• There is a sample app using the API on Github:
  • https://github.com/PX4/snap_cam/

Intel open-sourced their work on video streaming. The repository and initial documentation can be found at:

• https://github.com/01org/camera-streaming-daemon
• [Review] Camera Streaming Daemon (discuss.dronecode.org)

This initial version is based only on discovery via Zeroconf and therefore requires an active WiFi connection with the drone, but one of the items for future implementation is support for the current set of MAVLink messages there.

The Intel Aero and Snapdragon Flight platforms both use RTSP for video streaming. The implementation difference is that Intel Aero is using GStreamer and Snapdragon Flight uses live555. The Dronecode Camera/Video streaming API does not (and should not) expose implementation details.

Feedback is requested and appreciated