diff options
139 files changed, 15872 insertions, 2403 deletions
diff --git a/Documentation/admin-guide/media/cec-drivers.rst b/Documentation/admin-guide/media/cec-drivers.rst deleted file mode 100644 index 8d9686c08df9..000000000000 --- a/Documentation/admin-guide/media/cec-drivers.rst +++ /dev/null @@ -1,10 +0,0 @@ -.. SPDX-License-Identifier: GPL-2.0 - -================================= -CEC driver-specific documentation -================================= - -.. toctree:: - :maxdepth: 2 - - pulse8-cec diff --git a/Documentation/admin-guide/media/cec.rst b/Documentation/admin-guide/media/cec.rst new file mode 100644 index 000000000000..5c7259371494 --- /dev/null +++ b/Documentation/admin-guide/media/cec.rst @@ -0,0 +1,369 @@ +.. SPDX-License-Identifier: GPL-2.0 + +======== +HDMI CEC +======== + +Supported hardware in mainline +============================== + +HDMI Transmitters: + +- Exynos4 +- Exynos5 +- STIH4xx HDMI CEC +- V4L2 adv7511 (same HW, but a different driver from the drm adv7511) +- stm32 +- Allwinner A10 (sun4i) +- Raspberry Pi +- dw-hdmi (Synopsis IP) +- amlogic (meson ao-cec and ao-cec-g12a) +- drm adv7511/adv7533 +- omap4 +- tegra +- rk3288, rk3399 +- tda998x +- DisplayPort CEC-Tunneling-over-AUX on i915, nouveau and amdgpu +- ChromeOS EC CEC +- CEC for SECO boards (UDOO x86). +- Chrontel CH7322 + + +HDMI Receivers: + +- adv7604/11/12 +- adv7842 +- tc358743 + +USB Dongles (see below for additional information on how to use these +dongles): + +- Pulse-Eight: the pulse8-cec driver implements the following module option: + ``persistent_config``: by default this is off, but when set to 1 the driver + will store the current settings to the device's internal eeprom and restore + it the next time the device is connected to the USB port. +- RainShadow Tech. Note: this driver does not support the persistent_config + module option of the Pulse-Eight driver. The hardware supports it, but I + have no plans to add this feature. But I accept patches :-) + +Miscellaneous: + +- vivid: emulates a CEC receiver and CEC transmitter. + Can be used to test CEC applications without actual CEC hardware. + +- cec-gpio. If the CEC pin is hooked up to a GPIO pin then + you can control the CEC line through this driver. This supports error + injection as well. + + +Utilities +========= + +Utilities are available here: https://git.linuxtv.org/v4l-utils.git + +``utils/cec-ctl``: control a CEC device + +``utils/cec-compliance``: test compliance of a remote CEC device + +``utils/cec-follower``: emulate a CEC follower device + +Note that ``cec-ctl`` has support for the CEC Hospitality Profile as is +used in some hotel displays. See http://www.htng.org. + +Note that the libcec library (https://github.com/Pulse-Eight/libcec) supports +the linux CEC framework. + +If you want to get the CEC specification, then look at the References of +the HDMI wikipedia page: https://en.wikipedia.org/wiki/HDMI. CEC is part +of the HDMI specification. HDMI 1.3 is freely available (very similar to +HDMI 1.4 w.r.t. CEC) and should be good enough for most things. + + +DisplayPort to HDMI Adapters with working CEC +============================================= + +Background: most adapters do not support the CEC Tunneling feature, +and of those that do many did not actually connect the CEC pin. +Unfortunately, this means that while a CEC device is created, it +is actually all alone in the world and will never be able to see other +CEC devices. + +This is a list of known working adapters that have CEC Tunneling AND +that properly connected the CEC pin. If you find adapters that work +but are not in this list, then drop me a note. + +To test: hook up your DP-to-HDMI adapter to a CEC capable device +(typically a TV), then run:: + + cec-ctl --playback # Configure the PC as a CEC Playback device + cec-ctl -S # Show the CEC topology + +The ``cec-ctl -S`` command should show at least two CEC devices, +ourselves and the CEC device you are connected to (i.e. typically the TV). + +General note: I have only seen this work with the Parade PS175, PS176 and +PS186 chipsets and the MegaChips 2900. While MegaChips 28x0 claims CEC support, +I have never seen it work. + +USB-C to HDMI +------------- + +Samsung Multiport Adapter EE-PW700: https://www.samsung.com/ie/support/model/EE-PW700BBEGWW/ + +Kramer ADC-U31C/HF: https://www.kramerav.com/product/ADC-U31C/HF + +Club3D CAC-2504: https://www.club-3d.com/en/detail/2449/usb_3.1_type_c_to_hdmi_2.0_uhd_4k_60hz_active_adapter/ + +DisplayPort to HDMI +------------------- + +Club3D CAC-1080: https://www.club-3d.com/en/detail/2442/displayport_1.4_to_hdmi_2.0b_hdr/ + +CableCreation (SKU: CD0712): https://www.cablecreation.com/products/active-displayport-to-hdmi-adapter-4k-hdr + +HP DisplayPort to HDMI True 4k Adapter (P/N 2JA63AA): https://www.hp.com/us-en/shop/pdp/hp-displayport-to-hdmi-true-4k-adapter + +Mini-DisplayPort to HDMI +------------------------ + +Club3D CAC-1180: https://www.club-3d.com/en/detail/2443/mini_displayport_1.4_to_hdmi_2.0b_hdr/ + +Note that passive adapters will never work, you need an active adapter. + +The Club3D adapters in this list are all MegaChips 2900 based. Other Club3D adapters +are PS176 based and do NOT have the CEC pin hooked up, so only the three Club3D +adapters above are known to work. + +I suspect that MegaChips 2900 based designs in general are likely to work +whereas with the PS176 it is more hit-and-miss (mostly miss). The PS186 is +likely to have the CEC pin hooked up, it looks like they changed the reference +design for that chipset. + + +USB CEC Dongles +=============== + +These dongles appear as ``/dev/ttyACMX`` devices and need the ``inputattach`` +utility to create the ``/dev/cecX`` devices. Support for the Pulse-Eight +has been added to ``inputattach`` 1.6.0. Support for the Rainshadow Tech has +been added to ``inputattach`` 1.6.1. + +You also need udev rules to automatically start systemd services:: + + SUBSYSTEM=="tty", KERNEL=="ttyACM[0-9]*", ATTRS{idVendor}=="2548", ATTRS{idProduct}=="1002", ACTION=="add", TAG+="systemd", ENV{SYSTEMD_WANTS}+="pulse8-cec-inputattach@%k.service" + SUBSYSTEM=="tty", KERNEL=="ttyACM[0-9]*", ATTRS{idVendor}=="2548", ATTRS{idProduct}=="1001", ACTION=="add", TAG+="systemd", ENV{SYSTEMD_WANTS}+="pulse8-cec-inputattach@%k.service" + SUBSYSTEM=="tty", KERNEL=="ttyACM[0-9]*", ATTRS{idVendor}=="04d8", ATTRS{idProduct}=="ff59", ACTION=="add", TAG+="systemd", ENV{SYSTEMD_WANTS}+="rainshadow-cec-inputattach@%k.service" + +and these systemd services: + +For Pulse-Eight make /lib/systemd/system/[email protected]:: + + [Unit] + Description=inputattach for pulse8-cec device on %I + + [Service] + Type=simple + ExecStart=/usr/bin/inputattach --pulse8-cec /dev/%I + +For the RainShadow Tech make /lib/systemd/system/[email protected]:: + + [Unit] + Description=inputattach for rainshadow-cec device on %I + + [Service] + Type=simple + ExecStart=/usr/bin/inputattach --rainshadow-cec /dev/%I + + +For proper suspend/resume support create: /lib/systemd/system/restart-cec-inputattach.service:: + + [Unit] + Description=restart inputattach for cec devices + After=suspend.target + + [Service] + Type=forking + ExecStart=/bin/bash -c 'for d in /dev/serial/by-id/usb-Pulse-Eight*; do /usr/bin/inputattach --daemon --pulse8-cec $d; done; for d in /dev/serial/by-id/usb-RainShadow_Tech*; do /usr/bin/inputattach --daemon --rainshadow-cec $d; done' + + [Install] + WantedBy=suspend.target + +And run ``systemctl enable restart-cec-inputattach``. + +To automatically set the physical address of the CEC device whenever the +EDID changes, you can use ``cec-ctl`` with the ``-E`` option:: + + cec-ctl -E /sys/class/drm/card0-DP-1/edid + +This assumes the dongle is connected to the card0-DP-1 output (``xrandr`` will tell +you which output is used) and it will poll for changes to the EDID and update +the Physical Address whenever they occur. + +To automatically run this command you can use cron. Edit crontab with +``crontab -e`` and add this line:: + + @reboot /usr/local/bin/cec-ctl -E /sys/class/drm/card0-DP-1/edid + +This only works for display drivers that expose the EDID in ``/sys/class/drm``, +such as the i915 driver. + + +CEC Without HPD +=============== + +Some displays when in standby mode have no HDMI Hotplug Detect signal, but +CEC is still enabled so connected devices can send an <Image View On> CEC +message in order to wake up such displays. Unfortunately, not all CEC +adapters can support this. An example is the Odroid-U3 SBC that has a +level-shifter that is powered off when the HPD signal is low, thus +blocking the CEC pin. Even though the SoC can use CEC without a HPD, +the level-shifter will prevent this from functioning. + +There is a CEC capability flag to signal this: ``CEC_CAP_NEEDS_HPD``. +If set, then the hardware cannot wake up displays with this behavior. + +Note for CEC application implementers: the <Image View On> message must +be the first message you send, don't send any other messages before. +Certain very bad but unfortunately not uncommon CEC implementations +get very confused if they receive anything else but this message and +they won't wake up. + +When writing a driver it can be tricky to test this. There are two +ways to do this: + +1) Get a Pulse-Eight USB CEC dongle, connect an HDMI cable from your + device to the Pulse-Eight, but do not connect the Pulse-Eight to + the display. + + Now configure the Pulse-Eight dongle:: + + cec-ctl -p0.0.0.0 --tv + + and start monitoring:: + + sudo cec-ctl -M + + On the device you are testing run:: + + cec-ctl --playback + + It should report a physical address of f.f.f.f. Now run this + command:: + + cec-ctl -t0 --image-view-on + + The Pulse-Eight should see the <Image View On> message. If not, + then something (hardware and/or software) is preventing the CEC + message from going out. + + To make sure you have the wiring correct just connect the + Pulse-Eight to a CEC-enabled display and run the same command + on your device: now there is a HPD, so you should see the command + arriving at the Pulse-Eight. + +2) If you have another linux device supporting CEC without HPD, then + you can just connect your device to that device. Yes, you can connect + two HDMI outputs together. You won't have a HPD (which is what we + want for this test), but the second device can monitor the CEC pin. + + Otherwise use the same commands as in 1. + +If CEC messages do not come through when there is no HPD, then you +need to figure out why. Typically it is either a hardware restriction +or the software powers off the CEC core when the HPD goes low. The +first cannot be corrected of course, the second will likely required +driver changes. + + +Microcontrollers & CEC +====================== + +We have seen some CEC implementations in displays that use a microcontroller +to sample the bus. This does not have to be a problem, but some implementations +have timing issues. This is hard to discover unless you can hook up a low-level +CEC debugger (see the next section). + +You will see cases where the CEC transmitter holds the CEC line high or low for +a longer time than is allowed. For directed messages this is not a problem since +if that happens the message will not be Acked and it will be retransmitted. +For broadcast messages no such mechanism exists. + +It's not clear what to do about this. It is probably wise to transmit some +broadcast messages twice to reduce the chance of them being lost. Specifically +<Standby> and <Active Source> are candidates for that. + + +Making a CEC debugger +===================== + +By using a Raspberry Pi 2B/3/4 and some cheap components you can make +your own low-level CEC debugger. + +Here is a picture of my setup: + +https://hverkuil.home.xs4all.nl/rpi3-cec.jpg + +It's a Raspberry Pi 3 together with a breadboard and some breadboard wires: + +http://www.dx.com/p/diy-40p-male-to-female-male-to-male-female-to-female-dupont-line-wire-3pcs-356089#.WYLOOXWGN7I + +Finally on of these HDMI female-female passthrough connectors (full soldering type 1): + +https://elabbay.myshopify.com/collections/camera/products/hdmi-af-af-v1a-hdmi-type-a-female-to-hdmi-type-a-female-pass-through-adapter-breakout-board?variant=45533926147 + +We've tested this and it works up to 4kp30 (297 MHz). The quality is not high +enough to pass-through 4kp60 (594 MHz). + +I also added an RTC and a breakout shield: + +https://www.amazon.com/Makerfire%C2%AE-Raspberry-Module-DS1307-Battery/dp/B00ZOXWHK4 + +https://www.dx.com/p/raspberry-pi-gpio-expansion-board-breadboard-easy-multiplexing-board-one-to-three-with-screw-for-raspberry-pi-2-3-b-b-2729992.html#.YGRCG0MzZ7I + +These two are not needed but they make life a bit easier. + +If you want to monitor the HPD line as well, then you need one of these +level shifters: + +https://www.adafruit.com/product/757 + +(This is just where I got these components, there are many other places you +can get similar things). + +The CEC pin of the HDMI connector needs to be connected to these pins: +CE0/IO8 and CE1/IO7 (pull-up GPIOs). The (optional) HPD pin of the HDMI +connector should be connected (via a level shifter to convert the 5V +to 3.3V) to these pins: IO17 and IO27. The (optional) 5V pin of the HDMI +connector should be connected (via a level shifter) to these pins: IO22 +and IO24. Monitoring the HPD an 5V lines is not necessary, but it is helpful. + +This kernel patch will hook up the cec-gpio driver correctly to +e.g. ``arch/arm/boot/dts/bcm2837-rpi-3-b-plus.dts``:: + + cec-gpio@7 { + compatible = "cec-gpio"; + cec-gpios = <&gpio 7 (GPIO_ACTIVE_HIGH|GPIO_OPEN_DRAIN)>; + hpd-gpios = <&gpio 17 GPIO_ACTIVE_HIGH>; + v5-gpios = <&gpio 22 GPIO_ACTIVE_HIGH>; + }; + + cec-gpio@8 { + compatible = "cec-gpio"; + cec-gpios = <&gpio 8 (GPIO_ACTIVE_HIGH|GPIO_OPEN_DRAIN)>; + hpd-gpios = <&gpio 27 GPIO_ACTIVE_HIGH>; + v5-gpios = <&gpio 24 GPIO_ACTIVE_HIGH>; + }; + +This dts change will enable two cec GPIO devices: I typically use one to +send/receive CEC commands and the other to monitor. If you monitor using +an unconfigured CEC adapter then it will use GPIO interrupts which makes +monitoring very accurate. + +The documentation on how to use the error injection is here: :ref:`cec_pin_error_inj`. + +``cec-ctl --monitor-pin`` will do low-level CEC bus sniffing and analysis. +You can also store the CEC traffic to file using ``--store-pin`` and analyze +it later using ``--analyze-pin``. + +You can also use this as a full-fledged CEC device by configuring it +using ``cec-ctl --tv -p0.0.0.0`` or ``cec-ctl --playback -p1.0.0.0``. diff --git a/Documentation/admin-guide/media/index.rst b/Documentation/admin-guide/media/index.rst index c676af665111..43f4a292b245 100644 --- a/Documentation/admin-guide/media/index.rst +++ b/Documentation/admin-guide/media/index.rst @@ -38,13 +38,14 @@ The media subsystem remote-controller + cec + dvb cardlist v4l-drivers dvb-drivers - cec-drivers **Copyright** |copy| 1999-2020 : LinuxTV Developers diff --git a/Documentation/admin-guide/media/pulse8-cec.rst b/Documentation/admin-guide/media/pulse8-cec.rst deleted file mode 100644 index 356d08b519f3..000000000000 --- a/Documentation/admin-guide/media/pulse8-cec.rst +++ /dev/null @@ -1,13 +0,0 @@ -.. SPDX-License-Identifier: GPL-2.0 - -Pulse-Eight CEC Adapter driver -============================== - -The pulse8-cec driver implements the following module option: - -``persistent_config`` ---------------------- - -By default this is off, but when set to 1 the driver will store the current -settings to the device's internal eeprom and restore it the next time the -device is connected to the USB port. diff --git a/Documentation/admin-guide/media/v4l-drivers.rst b/Documentation/admin-guide/media/v4l-drivers.rst index 9c7ebe2ca3bd..90a026ee05c6 100644 --- a/Documentation/admin-guide/media/v4l-drivers.rst +++ b/Documentation/admin-guide/media/v4l-drivers.rst @@ -31,4 +31,5 @@ Video4Linux (V4L) driver-specific documentation si4713 si476x vimc + visl vivid diff --git a/Documentation/admin-guide/media/visl.rst b/Documentation/admin-guide/media/visl.rst new file mode 100644 index 000000000000..7d2dc78341c9 --- /dev/null +++ b/Documentation/admin-guide/media/visl.rst @@ -0,0 +1,175 @@ +.. SPDX-License-Identifier: GPL-2.0 + +The Virtual Stateless Decoder Driver (visl) +=========================================== + +A virtual stateless decoder device for stateless uAPI development +purposes. + +This tool's objective is to help the development and testing of +userspace applications that use the V4L2 stateless API to decode media. + +A userspace implementation can use visl to run a decoding loop even when +no hardware is available or when the kernel uAPI for the codec has not +been upstreamed yet. This can reveal bugs at an early stage. + +This driver can also trace the contents of the V4L2 controls submitted +to it. It can also dump the contents of the vb2 buffers through a +debugfs interface. This is in many ways similar to the tracing +infrastructure available for other popular encode/decode APIs out there +and can help develop a userspace application by using another (working) +one as a reference. + +.. note:: + + No actual decoding of video frames is performed by visl. The + V4L2 test pattern generator is used to write various debug information + to the capture buffers instead. + +Module parameters +----------------- + +- visl_debug: Activates debug info, printing various debug messages through + dprintk. Also controls whether per-frame debug info is shown. Defaults to off. + Note that enabling this feature can result in slow performance through serial. + +- visl_transtime_ms: Simulated process time in milliseconds. Slowing down the + decoding speed can be useful for debugging. + +- visl_dprintk_frame_start, visl_dprintk_frame_nframes: Dictates a range of + frames where dprintk is activated. This only controls the dprintk tracing on a + per-frame basis. Note that printing a lot of data can be slow through serial. + +- keep_bitstream_buffers: Controls whether bitstream (i.e. OUTPUT) buffers are + kept after a decoding session. Defaults to false so as to reduce the amount of + clutter. keep_bitstream_buffers == false works well when live debugging the + client program with GDB. + +- bitstream_trace_frame_start, bitstream_trace_nframes: Similar to + visl_dprintk_frame_start, visl_dprintk_nframes, but controls the dumping of + buffer data through debugfs instead. + +What is the default use case for this driver? +--------------------------------------------- + +This driver can be used as a way to compare different userspace implementations. +This assumes that a working client is run against visl and that the ftrace and +OUTPUT buffer data is subsequently used to debug a work-in-progress +implementation. + +Information on reference frames, their timestamps, the status of the OUTPUT and +CAPTURE queues and more can be read directly from the CAPTURE buffers. + +Supported codecs +---------------- + +The following codecs are supported: + +- FWHT +- MPEG2 +- VP8 +- VP9 +- H.264 +- HEVC + +visl trace events +----------------- +The trace events are defined on a per-codec basis, e.g.: + +.. code-block:: bash + + $ ls /sys/kernel/debug/tracing/events/ | grep visl + visl_fwht_controls + visl_h264_controls + visl_hevc_controls + visl_mpeg2_controls + visl_vp8_controls + visl_vp9_controls + +For example, in order to dump HEVC SPS data: + +.. code-block:: bash + + $ echo 1 > /sys/kernel/debug/tracing/events/visl_hevc_controls/v4l2_ctrl_hevc_sps/enable + +The SPS data will be dumped to the trace buffer, i.e.: + +.. code-block:: bash + + $ cat /sys/kernel/debug/tracing/trace + video_parameter_set_id 0 + seq_parameter_set_id 0 + pic_width_in_luma_samples 1920 + pic_height_in_luma_samples 1080 + bit_depth_luma_minus8 0 + bit_depth_chroma_minus8 0 + log2_max_pic_order_cnt_lsb_minus4 4 + sps_max_dec_pic_buffering_minus1 6 + sps_max_num_reorder_pics 2 + sps_max_latency_increase_plus1 0 + log2_min_luma_coding_block_size_minus3 0 + log2_diff_max_min_luma_coding_block_size 3 + log2_min_luma_transform_block_size_minus2 0 + log2_diff_max_min_luma_transform_block_size 3 + max_transform_hierarchy_depth_inter 2 + max_transform_hierarchy_depth_intra 2 + pcm_sample_bit_depth_luma_minus1 0 + pcm_sample_bit_depth_chroma_minus1 0 + log2_min_pcm_luma_coding_block_size_minus3 0 + log2_diff_max_min_pcm_luma_coding_block_size 0 + num_short_term_ref_pic_sets 0 + num_long_term_ref_pics_sps 0 + chroma_format_idc 1 + sps_max_sub_layers_minus1 0 + flags AMP_ENABLED|SAMPLE_ADAPTIVE_OFFSET|TEMPORAL_MVP_ENABLED|STRONG_INTRA_SMOOTHING_ENABLED + + +Dumping OUTPUT buffer data through debugfs +------------------------------------------ + +If the **VISL_DEBUGFS** Kconfig is enabled, visl will populate +**/sys/kernel/debug/visl/bitstream** with OUTPUT buffer data according to the +values of bitstream_trace_frame_start and bitstream_trace_nframes. This can +highlight errors as broken clients may fail to fill the buffers properly. + +A single file is created for each processed OUTPUT buffer. Its name contains an +integer that denotes the buffer sequence, i.e.: + +.. code-block:: c + + snprintf(name, 32, "bitstream%d", run->src->sequence); + +Dumping the values is simply a matter of reading from the file, i.e.: + +For the buffer with sequence == 0: + +.. code-block:: bash + + $ xxd /sys/kernel/debug/visl/bitstream/bitstream0 + 00000000: 2601 af04 d088 bc25 a173 0e41 a4f2 3274 &......%.s.A..2t + 00000010: c668 cb28 e775 b4ac f53a ba60 f8fd 3aa1 .h.(.u...:.`..:. + 00000020: 46b4 bcfc 506c e227 2372 e5f5 d7ea 579f F...Pl.'#r....W. + 00000030: 6371 5eb5 0eb8 23b5 ca6a 5de5 983a 19e4 cq^...#..j]..:.. + 00000040: e8c3 4320 b4ba a226 cbc1 4138 3a12 32d6 ..C ...&..A8:.2. + 00000050: fef3 247b 3523 4e90 9682 ac8e eb0c a389 ..${5#N......... + 00000060: ddd0 6cfc 0187 0e20 7aae b15b 1812 3d33 ..l.... z..[..=3 + 00000070: e1c5 f425 a83a 00b7 4f18 8127 3c4c aefb ...%.:..O..'<L.. + +For the buffer with sequence == 1: + +.. code-block:: bash + + $ xxd /sys/kernel/debug/visl/bitstream/bitstream1 + 00000000: 0201 d021 49e1 0c40 aa11 1449 14a6 01dc [email protected].... + 00000010: 7023 889a c8cd 2cd0 13b4 dab0 e8ca 21fe p#....,.......!. + 00000020: c4c8 ab4c 486e 4e2f b0df 96cc c74e 8dde ...LHnN/.....N.. + 00000030: 8ce7 ee36 d880 4095 4d64 30a0 ff4f 0c5e [email protected].^ + 00000040: f16b a6a1 d806 ca2a 0ece a673 7bea 1f37 .k.....*...s{..7 + 00000050: 370f 5bb9 1dc4 ba21 6434 bc53 0173 cba0 7.[....!d4.S.s.. + 00000060: dfe6 bc99 01ea b6e0 346b 92b5 c8de 9f5d ........4k.....] + 00000070: e7cc 3484 1769 fef2 a693 a945 2c8b 31da ..4..i.....E,.1. + +And so on. + +By default, the files are removed during STREAMOFF. This is to reduce the amount +of clutter. diff --git a/Documentation/devicetree/bindings/media/allwinner,sun6i-a31-csi.yaml b/Documentation/devicetree/bindings/media/allwinner,sun6i-a31-csi.yaml index f1ccca35a790..b3d6db922693 100644 --- a/Documentation/devicetree/bindings/media/allwinner,sun6i-a31-csi.yaml +++ b/Documentation/devicetree/bindings/media/allwinner,sun6i-a31-csi.yaml @@ -73,6 +73,10 @@ properties: $ref: /schemas/graph.yaml#/properties/port description: MIPI CSI-2 bridge input port + port@2: + $ref: /schemas/graph.yaml#/properties/port + description: Internal output port to the ISP + anyOf: - required: - port@0 diff --git a/Documentation/devicetree/bindings/media/allwinner,sun6i-a31-isp.yaml b/Documentation/devicetree/bindings/media/allwinner,sun6i-a31-isp.yaml new file mode 100644 index 000000000000..6bda4f2b94c2 --- /dev/null +++ b/Documentation/devicetree/bindings/media/allwinner,sun6i-a31-isp.yaml @@ -0,0 +1,101 @@ +# SPDX-License-Identifier: (GPL-2.0 OR BSD-2-Clause) +%YAML 1.2 +--- +$id: http://devicetree.org/schemas/media/allwinner,sun6i-a31-isp.yaml# +$schema: http://devicetree.org/meta-schemas/core.yaml# + +title: Allwinner A31 Image Signal Processor Driver (ISP) Device Tree Bindings + +maintainers: + - Paul Kocialkowski <[email protected]> + +properties: + compatible: + enum: + - allwinner,sun6i-a31-isp + - allwinner,sun8i-v3s-isp + + reg: + maxItems: 1 + + interrupts: + maxItems: 1 + + clocks: + items: + - description: Bus Clock + - description: Module Clock + - description: DRAM Clock + + clock-names: + items: + - const: bus + - const: mod + - const: ram + + resets: + maxItems: 1 + + ports: + $ref: /schemas/graph.yaml#/properties/ports + + properties: + port@0: + $ref: /schemas/graph.yaml#/properties/port + description: CSI0 input port + + port@1: + $ref: /schemas/graph.yaml#/properties/port + description: CSI1 input port + + if: + properties: + compatible: + contains: + enum: + - allwinner,sun8i-v3s-isp + then: + required: + - port@0 + +required: + - compatible + - reg + - interrupts + - clocks + - clock-names + - resets + +additionalProperties: false + +examples: + - | + #include <dt-bindings/interrupt-controller/arm-gic.h> + #include <dt-bindings/clock/sun8i-v3s-ccu.h> + #include <dt-bindings/reset/sun8i-v3s-ccu.h> + + isp: isp@1cb8000 { + compatible = "allwinner,sun8i-v3s-isp"; + reg = <0x01cb8000 0x1000>; + interrupts = <GIC_SPI 83 IRQ_TYPE_LEVEL_HIGH>; + clocks = <&ccu CLK_BUS_CSI>, + <&ccu CLK_CSI1_SCLK>, + <&ccu CLK_DRAM_CSI>; + clock-names = "bus", "mod", "ram"; + resets = <&ccu RST_BUS_CSI>; + + ports { + #address-cells = <1>; + #size-cells = <0>; + + port@0 { + reg = <0>; + + isp_in_csi0: endpoint { + remote-endpoint = <&csi0_out_isp>; + }; + }; + }; + }; + +... diff --git a/Documentation/devicetree/bindings/media/i2c/ov5645.txt b/Documentation/devicetree/bindings/media/i2c/ov5645.txt deleted file mode 100644 index 72ad992f77be..000000000000 --- a/Documentation/devicetree/bindings/media/i2c/ov5645.txt +++ /dev/null @@ -1,54 +0,0 @@ -* Omnivision 1/4-Inch 5Mp CMOS Digital Image Sensor - -The Omnivision OV5645 is a 1/4-Inch CMOS active pixel digital image sensor with -an active array size of 2592H x 1944V. It is programmable through a serial I2C -interface. - -Required Properties: -- compatible: Value should be "ovti,ov5645". -- clocks: Reference to the xclk clock. -- clock-names: Should be "xclk". -- clock-frequency: Frequency of the xclk clock. -- enable-gpios: Chip enable GPIO. Polarity is GPIO_ACTIVE_HIGH. This corresponds - to the hardware pin PWDNB which is physically active low. -- reset-gpios: Chip reset GPIO. Polarity is GPIO_ACTIVE_LOW. This corresponds to - the hardware pin RESETB. -- vdddo-supply: Chip digital IO regulator. -- vdda-supply: Chip analog regulator. -- vddd-supply: Chip digital core regulator. - -The device node must contain one 'port' child node for its digital output -video port, in accordance with the video interface bindings defined in -Documentation/devicetree/bindings/media/video-interfaces.txt. - -Example: - - &i2c1 { - ... - - ov5645: ov5645@3c { - compatible = "ovti,ov5645"; - reg = <0x3c>; - - enable-gpios = <&gpio1 6 GPIO_ACTIVE_HIGH>; - reset-gpios = <&gpio5 20 GPIO_ACTIVE_LOW>; - pinctrl-names = "default"; - pinctrl-0 = <&camera_rear_default>; - - clocks = <&clks 200>; - clock-names = "xclk"; - clock-frequency = <24000000>; - - vdddo-supply = <&camera_dovdd_1v8>; - vdda-supply = <&camera_avdd_2v8>; - vddd-supply = <&camera_dvdd_1v2>; - - port { - ov5645_ep: endpoint { - clock-lanes = <1>; - data-lanes = <0 2>; - remote-endpoint = <&csi0_ep>; - }; - }; - }; - }; diff --git a/Documentation/devicetree/bindings/media/i2c/ovti,ov5645.yaml b/Documentation/devicetree/bindings/media/i2c/ovti,ov5645.yaml new file mode 100644 index 000000000000..52c6281a6684 --- /dev/null +++ b/Documentation/devicetree/bindings/media/i2c/ovti,ov5645.yaml @@ -0,0 +1,104 @@ +# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause) +%YAML 1.2 +--- +$id: http://devicetree.org/schemas/media/i2c/ovti,ov5645.yaml# +$schema: http://devicetree.org/meta-schemas/core.yaml# + +title: OmniVision OV5645 Image Sensor Device Tree Bindings + +maintainers: + - Lad Prabhakar <[email protected]> + +properties: + compatible: + const: ovti,ov5645 + + reg: + maxItems: 1 + + clocks: + description: XCLK Input Clock + + clock-frequency: + description: Frequency of the xclk clock in Hz. + + vdda-supply: + description: Analog voltage supply, 2.8 volts + + vddd-supply: + description: Digital core voltage supply, 1.5 volts + + vdddo-supply: + description: Digital I/O voltage supply, 1.8 volts + + enable-gpios: + maxItems: 1 + description: + Reference to the GPIO connected to the PWDNB pin, if any. + + reset-gpios: + maxItems: 1 + description: + Reference to the GPIO connected to the RESETB pin, if any. + + port: + description: Digital Output Port + $ref: /schemas/graph.yaml#/$defs/port-base + additionalProperties: false + + properties: + endpoint: + $ref: /schemas/media/video-interfaces.yaml# + unevaluatedProperties: false + + properties: + data-lanes: + minItems: 1 + maxItems: 2 + items: + enum: [1, 2] + + required: + - data-lanes + +required: + - compatible + - reg + - clocks + - vdddo-supply + - vdda-supply + - vddd-supply + - port + +additionalProperties: false + +examples: + - | + #include <dt-bindings/gpio/gpio.h> + + i2c { + #address-cells = <1>; + #size-cells = <0>; + + camera@3c { + compatible = "ovti,ov5645"; + reg = <0x3c>; + clocks = <&clks 1>; + clock-frequency = <24000000>; + vdddo-supply = <&ov5645_vdddo_1v8>; + vdda-supply = <&ov5645_vdda_2v8>; + vddd-supply = <&ov5645_vddd_1v5>; + enable-gpios = <&gpio1 19 GPIO_ACTIVE_HIGH>; + reset-gpios = <&gpio1 20 GPIO_ACTIVE_LOW>; + pinctrl-names = "default"; + pinctrl-0 = <&pinctrl_ov5645>; + + port { + ov5645_ep: endpoint { + remote-endpoint = <&csi0_ep>; + data-lanes = <1 2>; + }; + }; + }; + }; +... diff --git a/Documentation/devicetree/bindings/media/i2c/sony,imx412.yaml b/Documentation/devicetree/bindings/media/i2c/sony,imx412.yaml index 26d1807d0bb6..60dc25ff2b9e 100644 --- a/Documentation/devicetree/bindings/media/i2c/sony,imx412.yaml +++ b/Documentation/devicetree/bindings/media/i2c/sony,imx412.yaml @@ -19,7 +19,9 @@ description: properties: compatible: - const: sony,imx412 + enum: + - sony,imx412 + - sony,imx577 reg: description: I2C address maxItems: 1 diff --git a/Documentation/userspace-api/media/cec/cec-pin-error-inj.rst b/Documentation/userspace-api/media/cec/cec-pin-error-inj.rst index b0efce40471f..411d42a742f3 100644 --- a/Documentation/userspace-api/media/cec/cec-pin-error-inj.rst +++ b/Documentation/userspace-api/media/cec/cec-pin-error-inj.rst @@ -1,5 +1,7 @@ .. SPDX-License-Identifier: GFDL-1.1-no-invariants-or-later +.. _cec_pin_error_inj: + CEC Pin Framework Error Injection ================================= diff --git a/MAINTAINERS b/MAINTAINERS index 6f4ff0ef4523..6f1b811eed91 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -775,6 +775,24 @@ T: git git://linuxtv.org/media_tree.git F: Documentation/devicetree/bindings/media/allwinner,sun4i-a10-csi.yaml F: drivers/media/platform/sunxi/sun4i-csi/ +ALLWINNER A31 CSI DRIVER +M: Yong Deng <[email protected]> +M: Paul Kocialkowski <[email protected]> +S: Maintained +T: git git://linuxtv.org/media_tree.git +F: Documentation/devicetree/bindings/media/allwinner,sun6i-a31-csi.yaml +F: drivers/media/platform/sunxi/sun6i-csi/ + +ALLWINNER A31 ISP DRIVER +M: Paul Kocialkowski <[email protected]> +S: Maintained +T: git git://linuxtv.org/media_tree.git +F: Documentation/devicetree/bindings/media/allwinner,sun6i-a31-isp.yaml +F: drivers/staging/media/sunxi/sun6i-isp/ +F: drivers/staging/media/sunxi/sun6i-isp/uapi/sun6i-isp-config.h + ALLWINNER A31 MIPI CSI-2 BRIDGE DRIVER M: Paul Kocialkowski <[email protected]> @@ -5502,14 +5520,6 @@ M: Jaya Kumar <[email protected]> S: Maintained F: sound/pci/cs5535audio/ -CSI DRIVERS FOR ALLWINNER V3s -M: Yong Deng <[email protected]> -S: Maintained -T: git git://linuxtv.org/media_tree.git -F: Documentation/devicetree/bindings/media/allwinner,sun6i-a31-csi.yaml -F: drivers/media/platform/sunxi/sun6i-csi/ - CTU CAN FD DRIVER M: Pavel Pisa <[email protected]> M: Ondrej Ille <[email protected]> @@ -13466,7 +13476,7 @@ M: Eugen Hristev <[email protected]> S: Supported F: Documentation/devicetree/bindings/media/microchip,csi2dc.yaml -F: drivers/media/platform/atmel/microchip-csi2dc.c +F: drivers/media/platform/microchip/microchip-csi2dc.c MICROCHIP ECC DRIVER M: Tudor Ambarus <[email protected]> @@ -13493,8 +13503,10 @@ L: [email protected] S: Supported F: Documentation/devicetree/bindings/media/atmel,isc.yaml F: Documentation/devicetree/bindings/media/microchip,xisc.yaml -F: drivers/media/platform/atmel/atmel-isc* -F: drivers/media/platform/atmel/atmel-sama*-isc* +F: drivers/staging/media/deprecated/atmel/atmel-isc* +F: drivers/staging/media/deprecated/atmel/atmel-sama*-isc* +F: drivers/media/platform/microchip/microchip-isc* +F: drivers/media/platform/microchip/microchip-sama*-isc* F: include/linux/atmel-isc-media.h MICROCHIP ISI DRIVER @@ -16722,7 +16734,6 @@ M: Hans Verkuil <[email protected]> S: Maintained T: git git://linuxtv.org/media_tree.git -F: Documentation/admin-guide/media/pulse8-cec.rst F: drivers/media/cec/usb/pulse8/ PURELIFI PLFXLC DRIVER @@ -21747,6 +21758,12 @@ F: include/linux/virtio*.h F: include/uapi/linux/virtio_*.h F: tools/virtio/ +VISL VIRTUAL STATELESS DECODER DRIVER +M: Daniel Almeida <[email protected]> +S: Supported +F: drivers/media/test-drivers/visl + IFCVF VIRTIO DATA PATH ACCELERATOR R: Zhu Lingshan <[email protected]> F: drivers/vdpa/ifcvf/ diff --git a/arch/arm/boot/dts/imx6qdl-pico.dtsi b/arch/arm/boot/dts/imx6qdl-pico.dtsi index f7a56d6b160c..c39a9ebdaba1 100644 --- a/arch/arm/boot/dts/imx6qdl-pico.dtsi +++ b/arch/arm/boot/dts/imx6qdl-pico.dtsi @@ -233,7 +233,6 @@ pinctrl-0 = <&pinctrl_ov5645>; reg = <0x3c>; clocks = <&clks IMX6QDL_CLK_CKO2>; - clock-names = "xclk"; clock-frequency = <24000000>; vdddo-supply = <®_1p8v>; vdda-supply = <®_2p8v>; diff --git a/arch/arm/boot/dts/imx6qdl-wandboard.dtsi b/arch/arm/boot/dts/imx6qdl-wandboard.dtsi index ec6fba5ee8fd..e4f63423d8ee 100644 --- a/arch/arm/boot/dts/imx6qdl-wandboard.dtsi +++ b/arch/arm/boot/dts/imx6qdl-wandboard.dtsi @@ -131,7 +131,6 @@ pinctrl-0 = <&pinctrl_ov5645>; reg = <0x3c>; clocks = <&clks IMX6QDL_CLK_CKO2>; - clock-names = "xclk"; clock-frequency = <24000000>; vdddo-supply = <®_1p8v>; vdda-supply = <®_2p8v>; diff --git a/arch/arm64/boot/dts/renesas/aistarvision-mipi-adapter-2.1.dtsi b/arch/arm64/boot/dts/renesas/aistarvision-mipi-adapter-2.1.dtsi index 7ce986f0a06f..7cb5c958aece 100644 --- a/arch/arm64/boot/dts/renesas/aistarvision-mipi-adapter-2.1.dtsi +++ b/arch/arm64/boot/dts/renesas/aistarvision-mipi-adapter-2.1.dtsi @@ -65,7 +65,6 @@ ov5645: ov5645@3c { compatible = "ovti,ov5645"; reg = <0x3c>; - clock-names = "xclk"; clocks = <&osc25250_clk>; clock-frequency = <24000000>; vdddo-supply = <&ov5645_vdddo_1v8>; diff --git a/drivers/media/common/videobuf2/videobuf2-core.c b/drivers/media/common/videobuf2/videobuf2-core.c index ab9697f3b5f1..2cb2a3b544a1 100644 --- a/drivers/media/common/videobuf2/videobuf2-core.c +++ b/drivers/media/common/videobuf2/videobuf2-core.c @@ -544,6 +544,7 @@ static int __vb2_queue_free(struct vb2_queue *q, unsigned int buffers) */ if (q->num_buffers) { bool unbalanced = q->cnt_start_streaming != q->cnt_stop_streaming || + q->cnt_prepare_streaming != q->cnt_unprepare_streaming || q->cnt_wait_prepare != q->cnt_wait_finish; if (unbalanced || debug) { @@ -552,14 +553,18 @@ static int __vb2_queue_free(struct vb2_queue *q, unsigned int buffers) pr_info(" setup: %u start_streaming: %u stop_streaming: %u\n", q->cnt_queue_setup, q->cnt_start_streaming, q->cnt_stop_streaming); + pr_info(" prepare_streaming: %u unprepare_streaming: %u\n", + q->cnt_prepare_streaming, q->cnt_unprepare_streaming); pr_info(" wait_prepare: %u wait_finish: %u\n", q->cnt_wait_prepare, q->cnt_wait_finish); } q->cnt_queue_setup = 0; q->cnt_wait_prepare = 0; q->cnt_wait_finish = 0; + q->cnt_prepare_streaming = 0; q->cnt_start_streaming = 0; q->cnt_stop_streaming = 0; + q->cnt_unprepare_streaming = 0; } for (buffer = 0; buffer < q->num_buffers; ++buffer) { struct vb2_buffer *vb = q->bufs[buffer]; @@ -1991,6 +1996,9 @@ static void __vb2_queue_cancel(struct vb2_queue *q) if (q->start_streaming_called) call_void_qop(q, stop_streaming, q); + if (q->streaming) + call_void_qop(q, unprepare_streaming, q); + /* * If you see this warning, then the driver isn't cleaning up properly * in stop_streaming(). See the stop_streaming() documentation in @@ -2102,23 +2110,29 @@ int vb2_core_streamon(struct vb2_queue *q, unsigned int type) return -EINVAL; } + ret = call_qop(q, prepare_streaming, q); + if (ret) + return ret; + + q->streaming = 1; + /* * Tell driver to start streaming provided sufficient buffers * are available. */ if (q->queued_count >= q->min_buffers_needed) { - ret = v4l_vb2q_enable_media_source(q); - if (ret) - return ret; ret = vb2_start_streaming(q); if (ret) - return ret; + goto unprepare; } - q->streaming = 1; - dprintk(q, 3, "successful\n"); return 0; + +unprepare: + call_void_qop(q, unprepare_streaming, q); + q->streaming = 0; + return ret; } EXPORT_SYMBOL_GPL(vb2_core_streamon); diff --git a/drivers/media/dvb-frontends/cxd2820r_priv.h b/drivers/media/dvb-frontends/cxd2820r_priv.h index 09c42bcef971..9b4d9cf8563d 100644 --- a/drivers/media/dvb-frontends/cxd2820r_priv.h +++ b/drivers/media/dvb-frontends/cxd2820r_priv.h @@ -52,8 +52,6 @@ struct cxd2820r_priv { /* cxd2820r_core.c */ -extern int cxd2820r_debug; - int cxd2820r_gpio(struct dvb_frontend *fe, u8 *gpio); int cxd2820r_wr_reg_val_mask_tab(struct cxd2820r_priv *priv, diff --git a/drivers/media/dvb-frontends/drx39xyj/drx_dap_fasi.h b/drivers/media/dvb-frontends/drx39xyj/drx_dap_fasi.h index 739dc5590fa4..9df34c10d22b 100644 --- a/drivers/media/dvb-frontends/drx39xyj/drx_dap_fasi.h +++ b/drivers/media/dvb-frontends/drx39xyj/drx_dap_fasi.h @@ -234,8 +234,6 @@ /*-------- Public API functions ----------------------------------------------*/ -extern struct drx_access_func drx_dap_fasi_funct_g; - #define DRXDAP_FASI_RMW 0x10000000 #define DRXDAP_FASI_BROADCAST 0x20000000 #define DRXDAP_FASI_CLEARCRC 0x80000000 diff --git a/drivers/media/i2c/adv748x/adv748x-afe.c b/drivers/media/i2c/adv748x/adv748x-afe.c index 02eabe10ab97..00095c7762c2 100644 --- a/drivers/media/i2c/adv748x/adv748x-afe.c +++ b/drivers/media/i2c/adv748x/adv748x-afe.c @@ -521,6 +521,10 @@ int adv748x_afe_init(struct adv748x_afe *afe) } } + adv748x_afe_s_input(afe, afe->input); + + adv_dbg(state, "AFE Default input set to %d\n", afe->input); + /* Entity pads and sinks are 0-indexed to match the pads */ for (i = ADV748X_AFE_SINK_AIN0; i <= ADV748X_AFE_SINK_AIN7; i++) afe->pads[i].flags = MEDIA_PAD_FL_SINK; diff --git a/drivers/media/i2c/adv748x/adv748x.h b/drivers/media/i2c/adv748x/adv748x.h index d75eb3d8be5a..6f90f78f58cf 100644 --- a/drivers/media/i2c/adv748x/adv748x.h +++ b/drivers/media/i2c/adv748x/adv748x.h @@ -428,9 +428,6 @@ void adv748x_subdev_init(struct v4l2_subdev *sd, struct adv748x_state *state, const struct v4l2_subdev_ops *ops, u32 function, const char *ident); -int adv748x_register_subdevs(struct adv748x_state *state, - struct v4l2_device *v4l2_dev); - int adv748x_tx_power(struct adv748x_csi2 *tx, bool on); int adv748x_afe_init(struct adv748x_afe *afe); diff --git a/drivers/media/i2c/imx412.c b/drivers/media/i2c/imx412.c index 7f6d29e0e7c4..e1e986dc8856 100644 --- a/drivers/media/i2c/imx412.c +++ b/drivers/media/i2c/imx412.c @@ -1172,6 +1172,7 @@ static int imx412_init_controls(struct imx412 *imx412) static int imx412_probe(struct i2c_client *client) { struct imx412 *imx412; + const char *name; int ret; imx412 = devm_kzalloc(&client->dev, sizeof(*imx412), GFP_KERNEL); @@ -1179,6 +1180,9 @@ static int imx412_probe(struct i2c_client *client) return -ENOMEM; imx412->dev = &client->dev; + name = device_get_match_data(&client->dev); + if (!name) + return -ENODEV; /* Initialize subdev */ v4l2_i2c_subdev_init(&imx412->sd, client, &imx412_subdev_ops); @@ -1218,6 +1222,8 @@ static int imx412_probe(struct i2c_client *client) imx412->sd.flags |= V4L2_SUBDEV_FL_HAS_DEVNODE; imx412->sd.entity.function = MEDIA_ENT_F_CAM_SENSOR; + v4l2_i2c_subdev_set_name(&imx412->sd, client, name, NULL); + /* Initialize source pad */ imx412->pad.flags = MEDIA_PAD_FL_SOURCE; ret = media_entity_pads_init(&imx412->sd.entity, 1, &imx412->pad); @@ -1279,7 +1285,8 @@ static const struct dev_pm_ops imx412_pm_ops = { }; static const struct of_device_id imx412_of_match[] = { - { .compatible = "sony,imx412" }, + { .compatible = "sony,imx412", .data = "imx412" }, + { .compatible = "sony,imx577", .data = "imx577" }, { } }; diff --git a/drivers/media/i2c/ov5645.c b/drivers/media/i2c/ov5645.c index 47451238ca05..c8999fc4f26f 100644 --- a/drivers/media/i2c/ov5645.c +++ b/drivers/media/i2c/ov5645.c @@ -14,9 +14,6 @@ * https://www.mail-archive.com/linux-media%40vger.kernel.org/msg92671.html */ -/* - */ - #include <linux/bitops.h> #include <linux/clk.h> #include <linux/delay.h> @@ -27,6 +24,7 @@ #include <linux/module.h> #include <linux/of.h> #include <linux/of_graph.h> +#include <linux/pm_runtime.h> #include <linux/regulator/consumer.h> #include <linux/slab.h> #include <linux/types.h> @@ -108,7 +106,6 @@ struct ov5645 { u8 timing_tc_reg21; struct mutex power_lock; /* lock to protect power state */ - int power_count; struct gpio_desc *enable_gpio; struct gpio_desc *rst_gpio; @@ -635,8 +632,24 @@ static int ov5645_set_register_array(struct ov5645 *ov5645, return 0; } -static int ov5645_set_power_on(struct ov5645 *ov5645) +static int ov5645_set_power_off(struct device *dev) +{ + struct v4l2_subdev *sd = dev_get_drvdata(dev); + struct ov5645 *ov5645 = to_ov5645(sd); + + ov5645_write_reg(ov5645, OV5645_IO_MIPI_CTRL00, 0x58); + gpiod_set_value_cansleep(ov5645->rst_gpio, 1); + gpiod_set_value_cansleep(ov5645->enable_gpio, 0); + clk_disable_unprepare(ov5645->xclk); + regulator_bulk_disable(OV5645_NUM_SUPPLIES, ov5645->supplies); + + return 0; +} + +static int ov5645_set_power_on(struct device *dev) { + struct v4l2_subdev *sd = dev_get_drvdata(dev); + struct ov5645 *ov5645 = to_ov5645(sd); int ret; ret = regulator_bulk_enable(OV5645_NUM_SUPPLIES, ov5645->supplies); @@ -658,57 +671,19 @@ static int ov5645_set_power_on(struct ov5645 *ov5645) msleep(20); - return 0; -} - -static void ov5645_set_power_off(struct ov5645 *ov5645) -{ - gpiod_set_value_cansleep(ov5645->rst_gpio, 1); - gpiod_set_value_cansleep(ov5645->enable_gpio, 0); - clk_disable_unprepare(ov5645->xclk); - regulator_bulk_disable(OV5645_NUM_SUPPLIES, ov5645->supplies); -} - -static int ov5645_s_power(struct v4l2_subdev *sd, int on) -{ - struct ov5645 *ov5645 = to_ov5645(sd); - int ret = 0; - - mutex_lock(&ov5645->power_lock); - - /* If the power count is modified from 0 to != 0 or from != 0 to 0, - * update the power state. - */ - if (ov5645->power_count == !on) { - if (on) { - ret = ov5645_set_power_on(ov5645); - if (ret < 0) - goto exit; - - ret = ov5645_set_register_array(ov5645, - ov5645_global_init_setting, + ret = ov5645_set_register_array(ov5645, ov5645_global_init_setting, ARRAY_SIZE(ov5645_global_init_setting)); - if (ret < 0) { - dev_err(ov5645->dev, - "could not set init registers\n"); - ov5645_set_power_off(ov5645); - goto exit; - } - - usleep_range(500, 1000); - } else { - ov5645_write_reg(ov5645, OV5645_IO_MIPI_CTRL00, 0x58); - ov5645_set_power_off(ov5645); - } + if (ret < 0) { + dev_err(ov5645->dev, "could not set init registers\n"); + goto exit; } - /* Update the power count. */ - ov5645->power_count += on ? 1 : -1; - WARN_ON(ov5645->power_count < 0); + usleep_range(500, 1000); -exit: - mutex_unlock(&ov5645->power_lock); + return 0; +exit: + ov5645_set_power_off(dev); return ret; } @@ -795,7 +770,7 @@ static int ov5645_s_ctrl(struct v4l2_ctrl *ctrl) int ret; mutex_lock(&ov5645->power_lock); - if (!ov5645->power_count) { + if (!pm_runtime_get_if_in_use(ov5645->dev)) { mutex_unlock(&ov5645->power_lock); return 0; } @@ -827,6 +802,8 @@ static int ov5645_s_ctrl(struct v4l2_ctrl *ctrl) break; } + pm_runtime_mark_last_busy(ov5645->dev); + pm_runtime_put_autosuspend(ov5645->dev); mutex_unlock(&ov5645->power_lock); return ret; @@ -991,6 +968,10 @@ static int ov5645_s_stream(struct v4l2_subdev *subdev, int enable) int ret; if (enable) { + ret = pm_runtime_resume_and_get(ov5645->dev); + if (ret < 0) + return ret; + ret = ov5645_set_register_array(ov5645, ov5645->current_mode->data, ov5645->current_mode->data_size); @@ -998,39 +979,44 @@ static int ov5645_s_stream(struct v4l2_subdev *subdev, int enable) dev_err(ov5645->dev, "could not set mode %dx%d\n", ov5645->current_mode->width, ov5645->current_mode->height); - return ret; + goto err_rpm_put; } ret = v4l2_ctrl_handler_setup(&ov5645->ctrls); if (ret < 0) { dev_err(ov5645->dev, "could not sync v4l2 controls\n"); - return ret; + goto err_rpm_put; } ret = ov5645_write_reg(ov5645, OV5645_IO_MIPI_CTRL00, 0x45); if (ret < 0) - return ret; + goto err_rpm_put; ret = ov5645_write_reg(ov5645, OV5645_SYSTEM_CTRL0, OV5645_SYSTEM_CTRL0_START); if (ret < 0) - return ret; + goto err_rpm_put; } else { ret = ov5645_write_reg(ov5645, OV5645_IO_MIPI_CTRL00, 0x40); if (ret < 0) - return ret; + goto stream_off_rpm_put; ret = ov5645_write_reg(ov5645, OV5645_SYSTEM_CTRL0, OV5645_SYSTEM_CTRL0_STOP); - if (ret < 0) - return ret; + + goto stream_off_rpm_put; } return 0; -} -static const struct v4l2_subdev_core_ops ov5645_core_ops = { - .s_power = ov5645_s_power, -}; +err_rpm_put: + pm_runtime_put_sync(ov5645->dev); + return ret; + +stream_off_rpm_put: + pm_runtime_mark_last_busy(ov5645->dev); + pm_runtime_put_autosuspend(ov5645->dev); + return ret; +} static const struct v4l2_subdev_video_ops ov5645_video_ops = { .s_stream = ov5645_s_stream, @@ -1046,7 +1032,6 @@ static const struct v4l2_subdev_pad_ops ov5645_subdev_pad_ops = { }; static const struct v4l2_subdev_ops ov5645_subdev_ops = { - .core = &ov5645_core_ops, .video = &ov5645_video_ops, .pad = &ov5645_subdev_pad_ops, }; @@ -1188,11 +1173,9 @@ static int ov5645_probe(struct i2c_client *client) goto free_ctrl; } - ret = ov5645_s_power(&ov5645->sd, true); - if (ret < 0) { - dev_err(dev, "could not power up OV5645\n"); + ret = ov5645_set_power_on(dev); + if (ret) goto free_entity; - } ret = ov5645_read_reg(ov5645, OV5645_CHIP_ID_HIGH, &chip_id_high); if (ret < 0 || chip_id_high != OV5645_CHIP_ID_HIGH_BYTE) { @@ -1233,20 +1216,30 @@ static int ov5645_probe(struct i2c_client *client) goto power_down; } - ov5645_s_power(&ov5645->sd, false); + pm_runtime_set_active(dev); + pm_runtime_get_noresume(dev); + pm_runtime_enable(dev); + + ov5645_entity_init_cfg(&ov5645->sd, NULL); ret = v4l2_async_register_subdev(&ov5645->sd); if (ret < 0) { dev_err(dev, "could not register v4l2 device\n"); - goto free_entity; + goto err_pm_runtime; } - ov5645_entity_init_cfg(&ov5645->sd, NULL); + pm_runtime_set_autosuspend_delay(dev, 1000); + pm_runtime_use_autosuspend(dev); + pm_runtime_mark_last_busy(dev); + pm_runtime_put_autosuspend(dev); return 0; +err_pm_runtime: + pm_runtime_disable(dev); + pm_runtime_put_noidle(dev); power_down: - ov5645_s_power(&ov5645->sd, false); + ov5645_set_power_off(dev); free_entity: media_entity_cleanup(&ov5645->sd.entity); free_ctrl: @@ -1264,6 +1257,10 @@ static void ov5645_remove(struct i2c_client *client) v4l2_async_unregister_subdev(&ov5645->sd); media_entity_cleanup(&ov5645->sd.entity); v4l2_ctrl_handler_free(&ov5645->ctrls); + pm_runtime_disable(ov5645->dev); + if (!pm_runtime_status_suspended(ov5645->dev)) + ov5645_set_power_off(ov5645->dev); + pm_runtime_set_suspended(ov5645->dev); mutex_destroy(&ov5645->power_lock); } @@ -1279,10 +1276,15 @@ static const struct of_device_id ov5645_of_match[] = { }; MODULE_DEVICE_TABLE(of, ov5645_of_match); +static const struct dev_pm_ops ov5645_pm_ops = { + SET_RUNTIME_PM_OPS(ov5645_set_power_off, ov5645_set_power_on, NULL) +}; + static struct i2c_driver ov5645_i2c_driver = { .driver = { .of_match_table = ov5645_of_match, .name = "ov5645", + .pm = &ov5645_pm_ops, }, .probe_new = ov5645_probe, .remove = ov5645_remove, diff --git a/drivers/media/i2c/ov9282.c b/drivers/media/i2c/ov9282.c index df144a2f6eda..f2ec92deb5ec 100644 --- a/drivers/media/i2c/ov9282.c +++ b/drivers/media/i2c/ov9282.c @@ -13,6 +13,7 @@ #include <linux/pm_runtime.h> #include <media/v4l2-ctrls.h> +#include <media/v4l2-event.h> #include <media/v4l2-fwnode.h> #include <media/v4l2-subdev.h> @@ -21,6 +22,13 @@ #define OV9282_MODE_STANDBY 0x00 #define OV9282_MODE_STREAMING 0x01 +#define OV9282_REG_PLL_CTRL_0D 0x030d +#define OV9282_PLL_CTRL_0D_RAW8 0x60 +#define OV9282_PLL_CTRL_0D_RAW10 0x50 + +#define OV9282_REG_TIMING_HTS 0x380c +#define OV9282_TIMING_HTS_MAX 0x7fff + /* Lines per frame */ #define OV9282_REG_LPFR 0x380e @@ -45,6 +53,17 @@ /* Group hold register */ #define OV9282_REG_HOLD 0x3308 +#define OV9282_REG_ANA_CORE_2 0x3662 +#define OV9282_ANA_CORE2_RAW8 0x07 +#define OV9282_ANA_CORE2_RAW10 0x05 + +#define OV9282_REG_TIMING_FORMAT_1 0x3820 +#define OV9282_REG_TIMING_FORMAT_2 0x3821 +#define OV9282_FLIP_BIT BIT(2) + +#define OV9282_REG_MIPI_CTRL00 0x4800 +#define OV9282_GATED_CLOCK BIT(5) + /* Input clock rate */ #define OV9282_INCLK_RATE 24000000 @@ -52,6 +71,23 @@ #define OV9282_LINK_FREQ 400000000 #define OV9282_NUM_DATA_LANES 2 +/* Pixel rate */ +#define OV9282_PIXEL_RATE_10BIT (OV9282_LINK_FREQ * 2 * \ + OV9282_NUM_DATA_LANES / 10) +#define OV9282_PIXEL_RATE_8BIT (OV9282_LINK_FREQ * 2 * \ + OV9282_NUM_DATA_LANES / 8) + +/* + * OV9282 native and active pixel array size. + * 8 dummy rows/columns on each edge of a 1280x800 active array + */ +#define OV9282_NATIVE_WIDTH 1296U +#define OV9282_NATIVE_HEIGHT 816U +#define OV9282_PIXEL_ARRAY_LEFT 8U +#define OV9282_PIXEL_ARRAY_TOP 8U +#define OV9282_PIXEL_ARRAY_WIDTH 1280U +#define OV9282_PIXEL_ARRAY_HEIGHT 800U + #define OV9282_REG_MIN 0x00 #define OV9282_REG_MAX 0xfffff @@ -79,25 +115,23 @@ struct ov9282_reg_list { * struct ov9282_mode - ov9282 sensor mode structure * @width: Frame width * @height: Frame height - * @code: Format code - * @hblank: Horizontal blanking in lines + * @hblank_min: Minimum horizontal blanking in lines for non-continuous[0] and + * continuous[1] clock modes * @vblank: Vertical blanking in lines * @vblank_min: Minimum vertical blanking in lines * @vblank_max: Maximum vertical blanking in lines - * @pclk: Sensor pixel clock * @link_freq_idx: Link frequency index * @reg_list: Register list for sensor mode */ struct ov9282_mode { u32 width; u32 height; - u32 code; - u32 hblank; + u32 hblank_min[2]; u32 vblank; u32 vblank_min; u32 vblank_max; - u64 pclk; u32 link_freq_idx; + struct v4l2_rect crop; struct ov9282_reg_list reg_list; }; @@ -111,13 +145,13 @@ struct ov9282_mode { * @inclk: Sensor input clock * @ctrl_handler: V4L2 control handler * @link_freq_ctrl: Pointer to link frequency control - * @pclk_ctrl: Pointer to pixel clock control * @hblank_ctrl: Pointer to horizontal blanking control * @vblank_ctrl: Pointer to vertical blanking control * @exp_ctrl: Pointer to exposure control * @again_ctrl: Pointer to analog gain control * @vblank: Vertical blanking in lines * @cur_mode: Pointer to current selected sensor mode + * @code: Mbus code currently selected * @mutex: Mutex for serializing sensor controls * @streaming: Flag indicating streaming state */ @@ -130,15 +164,17 @@ struct ov9282 { struct clk *inclk; struct v4l2_ctrl_handler ctrl_handler; struct v4l2_ctrl *link_freq_ctrl; - struct v4l2_ctrl *pclk_ctrl; struct v4l2_ctrl *hblank_ctrl; struct v4l2_ctrl *vblank_ctrl; struct { struct v4l2_ctrl *exp_ctrl; struct v4l2_ctrl *again_ctrl; }; + struct v4l2_ctrl *pixel_rate; u32 vblank; + bool noncontinuous_clock; const struct ov9282_mode *cur_mode; + u32 code; struct mutex mutex; bool streaming; }; @@ -147,10 +183,15 @@ static const s64 link_freq[] = { OV9282_LINK_FREQ, }; -/* Sensor mode registers */ -static const struct ov9282_reg mode_1280x720_regs[] = { +/* + * Common registers + * + * Note: Do NOT include a software reset (0x0103, 0x01) in any of these + * register arrays as some settings are written as part of ov9282_power_on, + * and the reset will clear them. + */ +static const struct ov9282_reg common_regs[] = { {0x0302, 0x32}, - {0x030d, 0x50}, {0x030e, 0x02}, {0x3001, 0x00}, {0x3004, 0x00}, @@ -163,14 +204,10 @@ static const struct ov9282_reg mode_1280x720_regs[] = { {0x3030, 0x10}, {0x3039, 0x32}, {0x303a, 0x00}, - {0x3500, 0x00}, - {0x3501, 0x5f}, - {0x3502, 0x1e}, {0x3503, 0x08}, {0x3505, 0x8c}, {0x3507, 0x03}, {0x3508, 0x00}, - {0x3509, 0x10}, {0x3610, 0x80}, {0x3611, 0xa0}, {0x3620, 0x6e}, @@ -183,13 +220,85 @@ static const struct ov9282_reg mode_1280x720_regs[] = { {0x372d, 0x22}, {0x3731, 0x80}, {0x3732, 0x30}, - {0x3778, 0x00}, {0x377d, 0x22}, {0x3788, 0x02}, {0x3789, 0xa4}, {0x378a, 0x00}, {0x378b, 0x4a}, {0x3799, 0x20}, + {0x3881, 0x42}, + {0x38a8, 0x02}, + {0x38a9, 0x80}, + {0x38b1, 0x00}, + {0x38c4, 0x00}, + {0x38c5, 0xc0}, + {0x38c6, 0x04}, + {0x38c7, 0x80}, + {0x3920, 0xff}, + {0x4010, 0x40}, + {0x4043, 0x40}, + {0x4307, 0x30}, + {0x4317, 0x00}, + {0x4501, 0x00}, + {0x450a, 0x08}, + {0x4601, 0x04}, + {0x470f, 0x00}, + {0x4f07, 0x00}, + {0x5000, 0x9f}, + {0x5001, 0x00}, + {0x5e00, 0x00}, + {0x5d00, 0x07}, + {0x5d01, 0x00}, + {0x0101, 0x01}, + {0x1000, 0x03}, + {0x5a08, 0x84}, +}; + +struct ov9282_reg_list common_regs_list = { + .num_of_regs = ARRAY_SIZE(common_regs), + .regs = common_regs, +}; + +#define MODE_1280_800 0 +#define MODE_1280_720 1 +#define MODE_640_400 2 + +#define DEFAULT_MODE MODE_1280_720 + +/* Sensor mode registers */ +static const struct ov9282_reg mode_1280x800_regs[] = { + {0x3778, 0x00}, + {0x3800, 0x00}, + {0x3801, 0x00}, + {0x3802, 0x00}, + {0x3803, 0x00}, + {0x3804, 0x05}, + {0x3805, 0x0f}, + {0x3806, 0x03}, + {0x3807, 0x2f}, + {0x3808, 0x05}, + {0x3809, 0x00}, + {0x380a, 0x03}, + {0x380b, 0x20}, + {0x3810, 0x00}, + {0x3811, 0x08}, + {0x3812, 0x00}, + {0x3813, 0x08}, + {0x3814, 0x11}, + {0x3815, 0x11}, + {0x3820, 0x40}, + {0x3821, 0x00}, + {0x4003, 0x40}, + {0x4008, 0x04}, + {0x4009, 0x0b}, + {0x400c, 0x00}, + {0x400d, 0x07}, + {0x4507, 0x00}, + {0x4509, 0x00}, +}; + +static const struct ov9282_reg mode_1280x720_regs[] = { + {0x3778, 0x00}, {0x3800, 0x00}, {0x3801, 0x00}, {0x3802, 0x00}, @@ -202,10 +311,6 @@ static const struct ov9282_reg mode_1280x720_regs[] = { {0x3809, 0x00}, {0x380a, 0x02}, {0x380b, 0xd0}, - {0x380c, 0x05}, - {0x380d, 0xfa}, - {0x380e, 0x06}, - {0x380f, 0xce}, {0x3810, 0x00}, {0x3811, 0x08}, {0x3812, 0x00}, @@ -214,56 +319,107 @@ static const struct ov9282_reg mode_1280x720_regs[] = { {0x3815, 0x11}, {0x3820, 0x3c}, {0x3821, 0x84}, - {0x3881, 0x42}, - {0x38a8, 0x02}, - {0x38a9, 0x80}, - {0x38b1, 0x00}, - {0x38c4, 0x00}, - {0x38c5, 0xc0}, - {0x38c6, 0x04}, - {0x38c7, 0x80}, - {0x3920, 0xff}, {0x4003, 0x40}, {0x4008, 0x02}, {0x4009, 0x05}, {0x400c, 0x00}, {0x400d, 0x03}, - {0x4010, 0x40}, - {0x4043, 0x40}, - {0x4307, 0x30}, - {0x4317, 0x00}, - {0x4501, 0x00}, {0x4507, 0x00}, {0x4509, 0x80}, - {0x450a, 0x08}, - {0x4601, 0x04}, - {0x470f, 0x00}, - {0x4f07, 0x00}, - {0x4800, 0x20}, - {0x5000, 0x9f}, - {0x5001, 0x00}, - {0x5e00, 0x00}, - {0x5d00, 0x07}, - {0x5d01, 0x00}, - {0x0101, 0x01}, - {0x1000, 0x03}, - {0x5a08, 0x84}, +}; + +static const struct ov9282_reg mode_640x400_regs[] = { + {0x3778, 0x10}, + {0x3800, 0x00}, + {0x3801, 0x00}, + {0x3802, 0x00}, + {0x3803, 0x00}, + {0x3804, 0x05}, + {0x3805, 0x0f}, + {0x3806, 0x03}, + {0x3807, 0x2f}, + {0x3808, 0x02}, + {0x3809, 0x80}, + {0x380a, 0x01}, + {0x380b, 0x90}, + {0x3810, 0x00}, + {0x3811, 0x04}, + {0x3812, 0x00}, + {0x3813, 0x04}, + {0x3814, 0x31}, + {0x3815, 0x22}, + {0x3820, 0x60}, + {0x3821, 0x01}, + {0x4008, 0x02}, + {0x4009, 0x05}, + {0x400c, 0x00}, + {0x400d, 0x03}, + {0x4507, 0x03}, + {0x4509, 0x80}, }; /* Supported sensor mode configurations */ -static const struct ov9282_mode supported_mode = { - .width = 1280, - .height = 720, - .hblank = 250, - .vblank = 1022, - .vblank_min = 151, - .vblank_max = 51540, - .pclk = 160000000, - .link_freq_idx = 0, - .code = MEDIA_BUS_FMT_Y10_1X10, - .reg_list = { - .num_of_regs = ARRAY_SIZE(mode_1280x720_regs), - .regs = mode_1280x720_regs, +static const struct ov9282_mode supported_modes[] = { + [MODE_1280_800] = { + .width = 1280, + .height = 800, + .hblank_min = { 250, 176 }, + .vblank = 1022, + .vblank_min = 110, + .vblank_max = 51540, + .link_freq_idx = 0, + .crop = { + .left = OV9282_PIXEL_ARRAY_LEFT, + .top = OV9282_PIXEL_ARRAY_TOP, + .width = 1280, + .height = 800 + }, + .reg_list = { + .num_of_regs = ARRAY_SIZE(mode_1280x800_regs), + .regs = mode_1280x800_regs, + }, + }, + [MODE_1280_720] = { + .width = 1280, + .height = 720, + .hblank_min = { 250, 176 }, + .vblank = 1022, + .vblank_min = 41, + .vblank_max = 51540, + .link_freq_idx = 0, + .crop = { + /* + * Note that this mode takes the top 720 lines from the + * 800 of the sensor. It does not take a middle crop. + */ + .left = OV9282_PIXEL_ARRAY_LEFT, + .top = OV9282_PIXEL_ARRAY_TOP, + .width = 1280, + .height = 720 + }, + .reg_list = { + .num_of_regs = ARRAY_SIZE(mode_1280x720_regs), + .regs = mode_1280x720_regs, + }, + }, + [MODE_640_400] = { + .width = 640, + .height = 400, + .hblank_min = { 890, 816 }, + .vblank = 1022, + .vblank_min = 22, + .vblank_max = 51540, + .link_freq_idx = 0, + .crop = { + .left = OV9282_PIXEL_ARRAY_LEFT, + .top = OV9282_PIXEL_ARRAY_TOP, + .width = 1280, + .height = 800 + }, + .reg_list = { + .num_of_regs = ARRAY_SIZE(mode_640x400_regs), + .regs = mode_640x400_regs, + }, }, }; @@ -373,19 +529,33 @@ static int ov9282_write_regs(struct ov9282 *ov9282, * ov9282_update_controls() - Update control ranges based on streaming mode * @ov9282: pointer to ov9282 device * @mode: pointer to ov9282_mode sensor mode + * @fmt: pointer to the requested mode * * Return: 0 if successful, error code otherwise. */ static int ov9282_update_controls(struct ov9282 *ov9282, - const struct ov9282_mode *mode) + const struct ov9282_mode *mode, + const struct v4l2_subdev_format *fmt) { + u32 hblank_min; + s64 pixel_rate; int ret; ret = __v4l2_ctrl_s_ctrl(ov9282->link_freq_ctrl, mode->link_freq_idx); if (ret) return ret; - ret = __v4l2_ctrl_s_ctrl(ov9282->hblank_ctrl, mode->hblank); + pixel_rate = (fmt->format.code == MEDIA_BUS_FMT_Y10_1X10) ? + OV9282_PIXEL_RATE_10BIT : OV9282_PIXEL_RATE_8BIT; + ret = __v4l2_ctrl_modify_range(ov9282->pixel_rate, pixel_rate, + pixel_rate, 1, pixel_rate); + if (ret) + return ret; + + hblank_min = mode->hblank_min[ov9282->noncontinuous_clock ? 0 : 1]; + ret = __v4l2_ctrl_modify_range(ov9282->hblank_ctrl, hblank_min, + OV9282_TIMING_HTS_MAX - mode->width, 1, + hblank_min); if (ret) return ret; @@ -403,22 +573,15 @@ static int ov9282_update_controls(struct ov9282 *ov9282, */ static int ov9282_update_exp_gain(struct ov9282 *ov9282, u32 exposure, u32 gain) { - u32 lpfr; int ret; - lpfr = ov9282->vblank + ov9282->cur_mode->height; - - dev_dbg(ov9282->dev, "Set exp %u, analog gain %u, lpfr %u", - exposure, gain, lpfr); + dev_dbg(ov9282->dev, "Set exp %u, analog gain %u", + exposure, gain); ret = ov9282_write_reg(ov9282, OV9282_REG_HOLD, 1, 1); if (ret) return ret; - ret = ov9282_write_reg(ov9282, OV9282_REG_LPFR, 2, lpfr); - if (ret) - goto error_release_group_hold; - ret = ov9282_write_reg(ov9282, OV9282_REG_EXPOSURE, 3, exposure << 4); if (ret) goto error_release_group_hold; @@ -431,6 +594,40 @@ error_release_group_hold: return ret; } +static int ov9282_set_ctrl_hflip(struct ov9282 *ov9282, int value) +{ + u32 current_val; + int ret = ov9282_read_reg(ov9282, OV9282_REG_TIMING_FORMAT_2, 1, + ¤t_val); + if (ret) + return ret; + + if (value) + current_val |= OV9282_FLIP_BIT; + else + current_val &= ~OV9282_FLIP_BIT; + + return ov9282_write_reg(ov9282, OV9282_REG_TIMING_FORMAT_2, 1, + current_val); +} + +static int ov9282_set_ctrl_vflip(struct ov9282 *ov9282, int value) +{ + u32 current_val; + int ret = ov9282_read_reg(ov9282, OV9282_REG_TIMING_FORMAT_1, 1, + ¤t_val); + if (ret) + return ret; + + if (value) + current_val |= OV9282_FLIP_BIT; + else + current_val &= ~OV9282_FLIP_BIT; + + return ov9282_write_reg(ov9282, OV9282_REG_TIMING_FORMAT_1, 1, + current_val); +} + /** * ov9282_set_ctrl() - Set subdevice control * @ctrl: pointer to v4l2_ctrl structure @@ -449,6 +646,7 @@ static int ov9282_set_ctrl(struct v4l2_ctrl *ctrl) container_of(ctrl->handler, struct ov9282, ctrl_handler); u32 analog_gain; u32 exposure; + u32 lpfr; int ret; switch (ctrl->id) { @@ -466,11 +664,14 @@ static int ov9282_set_ctrl(struct v4l2_ctrl *ctrl) OV9282_EXPOSURE_OFFSET, 1, OV9282_EXPOSURE_DEFAULT); break; - case V4L2_CID_EXPOSURE: - /* Set controls only if sensor is in power on state */ - if (!pm_runtime_get_if_in_use(ov9282->dev)) - return 0; + } + + /* Set controls only if sensor is in power on state */ + if (!pm_runtime_get_if_in_use(ov9282->dev)) + return 0; + switch (ctrl->id) { + case V4L2_CID_EXPOSURE: exposure = ctrl->val; analog_gain = ov9282->again_ctrl->val; @@ -478,15 +679,28 @@ static int ov9282_set_ctrl(struct v4l2_ctrl *ctrl) exposure, analog_gain); ret = ov9282_update_exp_gain(ov9282, exposure, analog_gain); - - pm_runtime_put(ov9282->dev); - + break; + case V4L2_CID_VBLANK: + lpfr = ov9282->vblank + ov9282->cur_mode->height; + ret = ov9282_write_reg(ov9282, OV9282_REG_LPFR, 2, lpfr); + break; + case V4L2_CID_HFLIP: + ret = ov9282_set_ctrl_hflip(ov9282, ctrl->val); + break; + case V4L2_CID_VFLIP: + ret = ov9282_set_ctrl_vflip(ov9282, ctrl->val); + break; + case V4L2_CID_HBLANK: + ret = ov9282_write_reg(ov9282, OV9282_REG_TIMING_HTS, 2, + (ctrl->val + ov9282->cur_mode->width) >> 1); break; default: dev_err(ov9282->dev, "Invalid control %d", ctrl->id); ret = -EINVAL; } + pm_runtime_put(ov9282->dev); + return ret; } @@ -507,10 +721,16 @@ static int ov9282_enum_mbus_code(struct v4l2_subdev *sd, struct v4l2_subdev_state *sd_state, struct v4l2_subdev_mbus_code_enum *code) { - if (code->index > 0) + switch (code->index) { + case 0: + code->code = MEDIA_BUS_FMT_Y10_1X10; + break; + case 1: + code->code = MEDIA_BUS_FMT_Y8_1X8; + break; + default: return -EINVAL; - - code->code = supported_mode.code; + } return 0; } @@ -527,15 +747,16 @@ static int ov9282_enum_frame_size(struct v4l2_subdev *sd, struct v4l2_subdev_state *sd_state, struct v4l2_subdev_frame_size_enum *fsize) { - if (fsize->index > 0) + if (fsize->index >= ARRAY_SIZE(supported_modes)) return -EINVAL; - if (fsize->code != supported_mode.code) + if (fsize->code != MEDIA_BUS_FMT_Y10_1X10 && + fsize->code != MEDIA_BUS_FMT_Y8_1X8) return -EINVAL; - fsize->min_width = supported_mode.width; + fsize->min_width = supported_modes[fsize->index].width; fsize->max_width = fsize->min_width; - fsize->min_height = supported_mode.height; + fsize->min_height = supported_modes[fsize->index].height; fsize->max_height = fsize->min_height; return 0; @@ -546,15 +767,17 @@ static int ov9282_enum_frame_size(struct v4l2_subdev *sd, * from selected sensor mode * @ov9282: pointer to ov9282 device * @mode: pointer to ov9282_mode sensor mode + * @code: mbus code to be stored * @fmt: V4L2 sub-device format need to be filled */ static void ov9282_fill_pad_format(struct ov9282 *ov9282, const struct ov9282_mode *mode, + u32 code, struct v4l2_subdev_format *fmt) { fmt->format.width = mode->width; fmt->format.height = mode->height; - fmt->format.code = mode->code; + fmt->format.code = code; fmt->format.field = V4L2_FIELD_NONE; fmt->format.colorspace = V4L2_COLORSPACE_RAW; fmt->format.ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT; @@ -584,7 +807,8 @@ static int ov9282_get_pad_format(struct v4l2_subdev *sd, framefmt = v4l2_subdev_get_try_format(sd, sd_state, fmt->pad); fmt->format = *framefmt; } else { - ov9282_fill_pad_format(ov9282, ov9282->cur_mode, fmt); + ov9282_fill_pad_format(ov9282, ov9282->cur_mode, ov9282->code, + fmt); } mutex_unlock(&ov9282->mutex); @@ -606,12 +830,22 @@ static int ov9282_set_pad_format(struct v4l2_subdev *sd, { struct ov9282 *ov9282 = to_ov9282(sd); const struct ov9282_mode *mode; + u32 code; int ret = 0; mutex_lock(&ov9282->mutex); - mode = &supported_mode; - ov9282_fill_pad_format(ov9282, mode, fmt); + mode = v4l2_find_nearest_size(supported_modes, + ARRAY_SIZE(supported_modes), + width, height, + fmt->format.width, + fmt->format.height); + if (fmt->format.code == MEDIA_BUS_FMT_Y8_1X8) + code = MEDIA_BUS_FMT_Y8_1X8; + else + code = MEDIA_BUS_FMT_Y10_1X10; + + ov9282_fill_pad_format(ov9282, mode, code, fmt); if (fmt->which == V4L2_SUBDEV_FORMAT_TRY) { struct v4l2_mbus_framefmt *framefmt; @@ -619,9 +853,11 @@ static int ov9282_set_pad_format(struct v4l2_subdev *sd, framefmt = v4l2_subdev_get_try_format(sd, sd_state, fmt->pad); *framefmt = fmt->format; } else { - ret = ov9282_update_controls(ov9282, mode); - if (!ret) + ret = ov9282_update_controls(ov9282, mode, fmt); + if (!ret) { ov9282->cur_mode = mode; + ov9282->code = code; + } } mutex_unlock(&ov9282->mutex); @@ -643,11 +879,64 @@ static int ov9282_init_pad_cfg(struct v4l2_subdev *sd, struct v4l2_subdev_format fmt = { 0 }; fmt.which = sd_state ? V4L2_SUBDEV_FORMAT_TRY : V4L2_SUBDEV_FORMAT_ACTIVE; - ov9282_fill_pad_format(ov9282, &supported_mode, &fmt); + ov9282_fill_pad_format(ov9282, &supported_modes[DEFAULT_MODE], + ov9282->code, &fmt); return ov9282_set_pad_format(sd, sd_state, &fmt); } +static const struct v4l2_rect * +__ov9282_get_pad_crop(struct ov9282 *ov9282, + struct v4l2_subdev_state *sd_state, + unsigned int pad, enum v4l2_subdev_format_whence which) +{ + switch (which) { + case V4L2_SUBDEV_FORMAT_TRY: + return v4l2_subdev_get_try_crop(&ov9282->sd, sd_state, pad); + case V4L2_SUBDEV_FORMAT_ACTIVE: + return &ov9282->cur_mode->crop; + } + + return NULL; +} + +static int ov9282_get_selection(struct v4l2_subdev *sd, + struct v4l2_subdev_state *sd_state, + struct v4l2_subdev_selection *sel) +{ + switch (sel->target) { + case V4L2_SEL_TGT_CROP: { + struct ov9282 *ov9282 = to_ov9282(sd); + + mutex_lock(&ov9282->mutex); + sel->r = *__ov9282_get_pad_crop(ov9282, sd_state, sel->pad, + sel->which); + mutex_unlock(&ov9282->mutex); + + return 0; + } + + case V4L2_SEL_TGT_NATIVE_SIZE: + sel->r.top = 0; + sel->r.left = 0; + sel->r.width = OV9282_NATIVE_WIDTH; + sel->r.height = OV9282_NATIVE_HEIGHT; + + return 0; + + case V4L2_SEL_TGT_CROP_DEFAULT: + case V4L2_SEL_TGT_CROP_BOUNDS: + sel->r.top = OV9282_PIXEL_ARRAY_TOP; + sel->r.left = OV9282_PIXEL_ARRAY_LEFT; + sel->r.width = OV9282_PIXEL_ARRAY_WIDTH; + sel->r.height = OV9282_PIXEL_ARRAY_HEIGHT; + + return 0; + } + + return -EINVAL; +} + /** * ov9282_start_streaming() - Start sensor stream * @ov9282: pointer to ov9282 device @@ -656,9 +945,34 @@ static int ov9282_init_pad_cfg(struct v4l2_subdev *sd, */ static int ov9282_start_streaming(struct ov9282 *ov9282) { + const struct ov9282_reg bitdepth_regs[2][2] = { + { + {OV9282_REG_PLL_CTRL_0D, OV9282_PLL_CTRL_0D_RAW10}, + {OV9282_REG_ANA_CORE_2, OV9282_ANA_CORE2_RAW10}, + }, { + {OV9282_REG_PLL_CTRL_0D, OV9282_PLL_CTRL_0D_RAW8}, + {OV9282_REG_ANA_CORE_2, OV9282_ANA_CORE2_RAW8}, + } + }; const struct ov9282_reg_list *reg_list; + int bitdepth_index; int ret; + /* Write common registers */ + ret = ov9282_write_regs(ov9282, common_regs_list.regs, + common_regs_list.num_of_regs); + if (ret) { + dev_err(ov9282->dev, "fail to write common registers"); + return ret; + } + + bitdepth_index = ov9282->code == MEDIA_BUS_FMT_Y10_1X10 ? 0 : 1; + ret = ov9282_write_regs(ov9282, bitdepth_regs[bitdepth_index], 2); + if (ret) { + dev_err(ov9282->dev, "fail to write bitdepth regs"); + return ret; + } + /* Write sensor mode registers */ reg_list = &ov9282->cur_mode->reg_list; ret = ov9282_write_regs(ov9282, reg_list->regs, reg_list->num_of_regs); @@ -818,6 +1132,9 @@ static int ov9282_parse_hw_config(struct ov9282 *ov9282) if (ret) return ret; + ov9282->noncontinuous_clock = + bus_cfg.bus.mipi_csi2.flags & V4L2_MBUS_CSI2_NONCONTINUOUS_CLOCK; + if (bus_cfg.bus.mipi_csi2.num_data_lanes != OV9282_NUM_DATA_LANES) { dev_err(ov9282->dev, "number of CSI2 data lanes %d is not supported", @@ -845,6 +1162,11 @@ done_endpoint_free: } /* V4l2 subdevice ops */ +static const struct v4l2_subdev_core_ops ov9282_core_ops = { + .subscribe_event = v4l2_ctrl_subdev_subscribe_event, + .unsubscribe_event = v4l2_event_subdev_unsubscribe, +}; + static const struct v4l2_subdev_video_ops ov9282_video_ops = { .s_stream = ov9282_set_stream, }; @@ -855,9 +1177,11 @@ static const struct v4l2_subdev_pad_ops ov9282_pad_ops = { .enum_frame_size = ov9282_enum_frame_size, .get_fmt = ov9282_get_pad_format, .set_fmt = ov9282_set_pad_format, + .get_selection = ov9282_get_selection, }; static const struct v4l2_subdev_ops ov9282_subdev_ops = { + .core = &ov9282_core_ops, .video = &ov9282_video_ops, .pad = &ov9282_pad_ops, }; @@ -886,6 +1210,14 @@ static int ov9282_power_on(struct device *dev) usleep_range(400, 600); + ret = ov9282_write_reg(ov9282, OV9282_REG_MIPI_CTRL00, 1, + ov9282->noncontinuous_clock ? + OV9282_GATED_CLOCK : 0); + if (ret) { + dev_err(ov9282->dev, "fail to write MIPI_CTRL00"); + return ret; + } + return 0; error_reset: @@ -922,10 +1254,12 @@ static int ov9282_init_controls(struct ov9282 *ov9282) { struct v4l2_ctrl_handler *ctrl_hdlr = &ov9282->ctrl_handler; const struct ov9282_mode *mode = ov9282->cur_mode; + struct v4l2_fwnode_device_properties props; + u32 hblank_min; u32 lpfr; int ret; - ret = v4l2_ctrl_handler_init(ctrl_hdlr, 6); + ret = v4l2_ctrl_handler_init(ctrl_hdlr, 10); if (ret) return ret; @@ -959,12 +1293,18 @@ static int ov9282_init_controls(struct ov9282 *ov9282) mode->vblank_max, 1, mode->vblank); + v4l2_ctrl_new_std(ctrl_hdlr, &ov9282_ctrl_ops, V4L2_CID_VFLIP, + 0, 1, 1, 1); + + v4l2_ctrl_new_std(ctrl_hdlr, &ov9282_ctrl_ops, V4L2_CID_HFLIP, + 0, 1, 1, 1); + /* Read only controls */ - ov9282->pclk_ctrl = v4l2_ctrl_new_std(ctrl_hdlr, - &ov9282_ctrl_ops, - V4L2_CID_PIXEL_RATE, - mode->pclk, mode->pclk, - 1, mode->pclk); + ov9282->pixel_rate = v4l2_ctrl_new_std(ctrl_hdlr, &ov9282_ctrl_ops, + V4L2_CID_PIXEL_RATE, + OV9282_PIXEL_RATE_10BIT, + OV9282_PIXEL_RATE_10BIT, 1, + OV9282_PIXEL_RATE_10BIT); ov9282->link_freq_ctrl = v4l2_ctrl_new_int_menu(ctrl_hdlr, &ov9282_ctrl_ops, @@ -976,16 +1316,22 @@ static int ov9282_init_controls(struct ov9282 *ov9282) if (ov9282->link_freq_ctrl) ov9282->link_freq_ctrl->flags |= V4L2_CTRL_FLAG_READ_ONLY; + hblank_min = mode->hblank_min[ov9282->noncontinuous_clock ? 0 : 1]; ov9282->hblank_ctrl = v4l2_ctrl_new_std(ctrl_hdlr, &ov9282_ctrl_ops, V4L2_CID_HBLANK, - OV9282_REG_MIN, - OV9282_REG_MAX, - 1, mode->hblank); - if (ov9282->hblank_ctrl) - ov9282->hblank_ctrl->flags |= V4L2_CTRL_FLAG_READ_ONLY; + hblank_min, + OV9282_TIMING_HTS_MAX - mode->width, + 1, hblank_min); + + ret = v4l2_fwnode_device_parse(ov9282->dev, &props); + if (!ret) { + /* Failure sets ctrl_hdlr->error, which we check afterwards anyway */ + v4l2_ctrl_new_fwnode_properties(ctrl_hdlr, &ov9282_ctrl_ops, + &props); + } - if (ctrl_hdlr->error) { + if (ctrl_hdlr->error || ret) { dev_err(ov9282->dev, "control init failed: %d", ctrl_hdlr->error); v4l2_ctrl_handler_free(ctrl_hdlr); @@ -1038,8 +1384,9 @@ static int ov9282_probe(struct i2c_client *client) goto error_power_off; } - /* Set default mode to max resolution */ - ov9282->cur_mode = &supported_mode; + /* Set default mode to first mode */ + ov9282->cur_mode = &supported_modes[DEFAULT_MODE]; + ov9282->code = MEDIA_BUS_FMT_Y10_1X10; ov9282->vblank = ov9282->cur_mode->vblank; ret = ov9282_init_controls(ov9282); @@ -1049,7 +1396,8 @@ static int ov9282_probe(struct i2c_client *client) } /* Initialize subdev */ - ov9282->sd.flags |= V4L2_SUBDEV_FL_HAS_DEVNODE; + ov9282->sd.flags |= V4L2_SUBDEV_FL_HAS_DEVNODE | + V4L2_SUBDEV_FL_HAS_EVENTS; ov9282->sd.entity.function = MEDIA_ENT_F_CAM_SENSOR; /* Initialize source pad */ diff --git a/drivers/media/pci/bt8xx/bttv.h b/drivers/media/pci/bt8xx/bttv.h index d24b9ef9f59f..eed7eeb3b963 100644 --- a/drivers/media/pci/bt8xx/bttv.h +++ b/drivers/media/pci/bt8xx/bttv.h @@ -289,7 +289,6 @@ extern void bttv_init_card2(struct bttv *btv); extern void bttv_init_tuner(struct bttv *btv); /* card-specific functions */ -extern void tea5757_set_freq(struct bttv *btv, unsigned short freq); extern u32 bttv_tda9880_setnorm(struct bttv *btv, u32 gpiobits); /* extra tweaks for some chipsets */ diff --git a/drivers/media/pci/cx25821/cx25821-video.h b/drivers/media/pci/cx25821/cx25821-video.h index cf0d3f5509e7..080c8490a250 100644 --- a/drivers/media/pci/cx25821/cx25821-video.h +++ b/drivers/media/pci/cx25821/cx25821-video.h @@ -36,9 +36,6 @@ do { \ } while (0) #define FORMAT_FLAGS_PACKED 0x01 -extern void cx25821_video_wakeup(struct cx25821_dev *dev, - struct cx25821_dmaqueue *q, u32 count); - extern int cx25821_start_video_dma(struct cx25821_dev *dev, struct cx25821_dmaqueue *q, struct cx25821_buffer *buf, diff --git a/drivers/media/pci/saa7134/saa7134.h b/drivers/media/pci/saa7134/saa7134.h index 49fe0f6bacba..5c9b2912a9d1 100644 --- a/drivers/media/pci/saa7134/saa7134.h +++ b/drivers/media/pci/saa7134/saa7134.h @@ -866,7 +866,6 @@ int saa7134_ts_stop(struct saa7134_dev *dev); /* saa7134-vbi.c */ extern const struct vb2_ops saa7134_vbi_qops; -extern struct video_device saa7134_vbi_template; int saa7134_vbi_init1(struct saa7134_dev *dev); int saa7134_vbi_fini(struct saa7134_dev *dev); @@ -897,9 +896,6 @@ void saa7134_enable_i2s(struct saa7134_dev *dev); /* ----------------------------------------------------------- */ /* saa7134-oss.c */ -extern const struct file_operations saa7134_dsp_fops; -extern const struct file_operations saa7134_mixer_fops; - int saa7134_oss_init1(struct saa7134_dev *dev); int saa7134_oss_fini(struct saa7134_dev *dev); void saa7134_irq_oss_done(struct saa7134_dev *dev, unsigned long status); diff --git a/drivers/media/pci/saa7164/saa7164-core.c b/drivers/media/pci/saa7164/saa7164-core.c index d5f32e3ff544..01d75ef2342d 100644 --- a/drivers/media/pci/saa7164/saa7164-core.c +++ b/drivers/media/pci/saa7164/saa7164-core.c @@ -352,7 +352,7 @@ static void saa7164_work_enchandler(struct work_struct *w) container_of(w, struct saa7164_port, workenc); struct saa7164_dev *dev = port->dev; - u32 wp, mcb, rp, cnt = 0; + u32 wp, mcb, rp; port->last_svc_msecs_diff = port->last_svc_msecs; port->last_svc_msecs = jiffies_to_msecs(jiffies); @@ -405,7 +405,6 @@ static void saa7164_work_enchandler(struct work_struct *w) saa7164_work_enchandler_helper(port, rp); port->last_svc_rp = rp; - cnt++; if (rp == mcb) break; @@ -429,7 +428,7 @@ static void saa7164_work_vbihandler(struct work_struct *w) container_of(w, struct saa7164_port, workenc); struct saa7164_dev *dev = port->dev; - u32 wp, mcb, rp, cnt = 0; + u32 wp, mcb, rp; port->last_svc_msecs_diff = port->last_svc_msecs; port->last_svc_msecs = jiffies_to_msecs(jiffies); @@ -481,7 +480,6 @@ static void saa7164_work_vbihandler(struct work_struct *w) saa7164_work_enchandler_helper(port, rp); port->last_svc_rp = rp; - cnt++; if (rp == mcb) break; diff --git a/drivers/media/pci/saa7164/saa7164.h b/drivers/media/pci/saa7164/saa7164.h index 4b4eb156e214..eede47b686a3 100644 --- a/drivers/media/pci/saa7164/saa7164.h +++ b/drivers/media/pci/saa7164/saa7164.h @@ -493,8 +493,6 @@ int saa7164_downloadfirmware(struct saa7164_dev *dev); /* saa7164-i2c.c */ extern int saa7164_i2c_register(struct saa7164_i2c *bus); extern int saa7164_i2c_unregister(struct saa7164_i2c *bus); -extern void saa7164_call_i2c_clients(struct saa7164_i2c *bus, - unsigned int cmd, void *arg); /* ----------------------------------------------------------- */ /* saa7164-bus.c */ diff --git a/drivers/media/pci/solo6x10/solo6x10-core.c b/drivers/media/pci/solo6x10/solo6x10-core.c index 4a546eeefe38..6d87fbb0ee04 100644 --- a/drivers/media/pci/solo6x10/solo6x10-core.c +++ b/drivers/media/pci/solo6x10/solo6x10-core.c @@ -420,6 +420,7 @@ static int solo_sysfs_init(struct solo_dev *solo_dev) solo_dev->nr_chans); if (device_register(dev)) { + put_device(dev); dev->parent = NULL; return -ENOMEM; } diff --git a/drivers/media/pci/zoran/zoran_device.h b/drivers/media/pci/zoran/zoran_device.h index 34fd5cc914eb..237e830ae726 100644 --- a/drivers/media/pci/zoran/zoran_device.h +++ b/drivers/media/pci/zoran/zoran_device.h @@ -47,8 +47,6 @@ void zr36057_restart(struct zoran *zr); extern const struct zoran_format zoran_formats[]; -extern int v4l_bufsize; -extern int jpg_bufsize; extern int pass_through; /* i2c */ diff --git a/drivers/media/platform/Kconfig b/drivers/media/platform/Kconfig index a9334263fa9b..ee579916f874 100644 --- a/drivers/media/platform/Kconfig +++ b/drivers/media/platform/Kconfig @@ -72,6 +72,7 @@ source "drivers/media/platform/chips-media/Kconfig" source "drivers/media/platform/intel/Kconfig" source "drivers/media/platform/marvell/Kconfig" source "drivers/media/platform/mediatek/Kconfig" +source "drivers/media/platform/microchip/Kconfig" source "drivers/media/platform/nvidia/Kconfig" source "drivers/media/platform/nxp/Kconfig" source "drivers/media/platform/qcom/Kconfig" diff --git a/drivers/media/platform/Makefile b/drivers/media/platform/Makefile index a91f42024273..5453bb868e67 100644 --- a/drivers/media/platform/Makefile +++ b/drivers/media/platform/Makefile @@ -15,6 +15,7 @@ obj-y += chips-media/ obj-y += intel/ obj-y += marvell/ obj-y += mediatek/ +obj-y += microchip/ obj-y += nvidia/ obj-y += nxp/ obj-y += qcom/ diff --git a/drivers/media/platform/amphion/vdec.c b/drivers/media/platform/amphion/vdec.c index feb75dc204de..b27e6bed85f0 100644 --- a/drivers/media/platform/amphion/vdec.c +++ b/drivers/media/platform/amphion/vdec.c @@ -286,6 +286,7 @@ static int vdec_g_fmt(struct file *file, void *fh, struct v4l2_format *f) struct vpu_format *cur_fmt; int i; + vpu_inst_lock(inst); cur_fmt = vpu_get_format(inst, f->type); pixmp->pixelformat = cur_fmt->pixfmt; @@ -303,6 +304,7 @@ static int vdec_g_fmt(struct file *file, void *fh, struct v4l2_format *f) f->fmt.pix_mp.xfer_func = vdec->codec_info.transfer_chars; f->fmt.pix_mp.ycbcr_enc = vdec->codec_info.matrix_coeffs; f->fmt.pix_mp.quantization = vdec->codec_info.full_range; + vpu_inst_unlock(inst); return 0; } @@ -753,6 +755,9 @@ static bool vdec_check_source_change(struct vpu_inst *inst) if (!inst->fh.m2m_ctx) return false; + if (vdec->reset_codec) + return false; + if (!vb2_is_streaming(v4l2_m2m_get_dst_vq(inst->fh.m2m_ctx))) return true; fmt = vpu_helper_find_format(inst, inst->cap_format.type, vdec->codec_info.pixfmt); @@ -1088,7 +1093,8 @@ static void vdec_event_seq_hdr(struct vpu_inst *inst, struct vpu_dec_codec_info vdec->seq_tag = vdec->codec_info.tag; if (vdec->is_source_changed) { vdec_update_state(inst, VPU_CODEC_STATE_DYAMIC_RESOLUTION_CHANGE, 0); - vpu_notify_source_change(inst); + vdec->source_change++; + vdec_handle_resolution_change(inst); vdec->is_source_changed = false; } } @@ -1335,6 +1341,8 @@ static void vdec_abort(struct vpu_inst *inst) vdec->decoded_frame_count, vdec->display_frame_count, vdec->sequence); + if (!vdec->seq_hdr_found) + vdec->reset_codec = true; vdec->params.end_flag = 0; vdec->drain = 0; vdec->params.frame_count = 0; @@ -1342,6 +1350,7 @@ static void vdec_abort(struct vpu_inst *inst) vdec->display_frame_count = 0; vdec->sequence = 0; vdec->aborting = false; + inst->extra_size = 0; } static void vdec_stop(struct vpu_inst *inst, bool free) @@ -1464,8 +1473,7 @@ static int vdec_start_session(struct vpu_inst *inst, u32 type) } if (V4L2_TYPE_IS_OUTPUT(type)) { - if (inst->state == VPU_CODEC_STATE_SEEK) - vdec_update_state(inst, vdec->state, 1); + vdec_update_state(inst, vdec->state, 1); vdec->eos_received = 0; vpu_process_output_buffer(inst); } else { @@ -1629,6 +1637,7 @@ static int vdec_open(struct file *file) return ret; vdec->fixed_fmt = false; + vdec->state = VPU_CODEC_STATE_ACTIVE; inst->min_buffer_cap = VDEC_MIN_BUFFER_CAP; inst->min_buffer_out = VDEC_MIN_BUFFER_OUT; vdec_init(file); diff --git a/drivers/media/platform/amphion/vpu_drv.c b/drivers/media/platform/amphion/vpu_drv.c index 9d5a5075343d..f01ce49d27e8 100644 --- a/drivers/media/platform/amphion/vpu_drv.c +++ b/drivers/media/platform/amphion/vpu_drv.c @@ -245,7 +245,11 @@ static int __init vpu_driver_init(void) if (ret) return ret; - return vpu_core_driver_init(); + ret = vpu_core_driver_init(); + if (ret) + platform_driver_unregister(&hion_vpu_driver); + + return ret; } static void __exit vpu_driver_exit(void) diff --git a/drivers/media/platform/amphion/vpu_v4l2.c b/drivers/media/platform/amphion/vpu_v4l2.c index b779e0ba916c..4b714fab4c6b 100644 --- a/drivers/media/platform/amphion/vpu_v4l2.c +++ b/drivers/media/platform/amphion/vpu_v4l2.c @@ -65,18 +65,11 @@ unsigned int vpu_get_buffer_state(struct vb2_v4l2_buffer *vbuf) void vpu_v4l2_set_error(struct vpu_inst *inst) { - struct vb2_queue *src_q; - struct vb2_queue *dst_q; - vpu_inst_lock(inst); dev_err(inst->dev, "some error occurs in codec\n"); if (inst->fh.m2m_ctx) { - src_q = v4l2_m2m_get_src_vq(inst->fh.m2m_ctx); - dst_q = v4l2_m2m_get_dst_vq(inst->fh.m2m_ctx); - src_q->error = 1; - dst_q->error = 1; - wake_up(&src_q->done_wq); - wake_up(&dst_q->done_wq); + vb2_queue_error(v4l2_m2m_get_src_vq(inst->fh.m2m_ctx)); + vb2_queue_error(v4l2_m2m_get_dst_vq(inst->fh.m2m_ctx)); } vpu_inst_unlock(inst); } diff --git a/drivers/media/platform/aspeed/aspeed-video.c b/drivers/media/platform/aspeed/aspeed-video.c index f08b7884424f..794d4dc3a654 100644 --- a/drivers/media/platform/aspeed/aspeed-video.c +++ b/drivers/media/platform/aspeed/aspeed-video.c @@ -586,13 +586,13 @@ static int aspeed_video_start_frame(struct aspeed_video *video) bool bcd_buf_need = (video->format != VIDEO_FMT_STANDARD); if (video->v4l2_input_status) { - v4l2_warn(&video->v4l2_dev, "No signal; don't start frame\n"); + v4l2_dbg(1, debug, &video->v4l2_dev, "No signal; don't start frame\n"); return 0; } if (!(seq_ctrl & VE_SEQ_CTRL_COMP_BUSY) || !(seq_ctrl & VE_SEQ_CTRL_CAP_BUSY)) { - v4l2_warn(&video->v4l2_dev, "Engine busy; don't start frame\n"); + v4l2_dbg(1, debug, &video->v4l2_dev, "Engine busy; don't start frame\n"); return -EBUSY; } @@ -615,7 +615,7 @@ static int aspeed_video_start_frame(struct aspeed_video *video) struct aspeed_video_buffer, link); if (!buf) { spin_unlock_irqrestore(&video->lock, flags); - v4l2_warn(&video->v4l2_dev, "No buffers; don't start frame\n"); + v4l2_dbg(1, debug, &video->v4l2_dev, "No buffers; don't start frame\n"); return -EPROTO; } @@ -796,7 +796,7 @@ static irqreturn_t aspeed_video_irq(int irq, void *arg) if (video->format == VIDEO_FMT_STANDARD && list_is_last(&buf->link, &video->buffers)) { empty = false; - v4l2_warn(&video->v4l2_dev, "skip to keep last frame updated\n"); + v4l2_dbg(1, debug, &video->v4l2_dev, "skip to keep last frame updated\n"); } else { buf->vb.vb2_buf.timestamp = ktime_get_ns(); buf->vb.sequence = video->sequence++; @@ -1060,7 +1060,7 @@ static void aspeed_video_get_resolution(struct aspeed_video *video) res_check(video), MODE_DETECT_TIMEOUT); if (!rc) { - v4l2_warn(&video->v4l2_dev, "Timed out; first mode detect\n"); + v4l2_dbg(1, debug, &video->v4l2_dev, "Timed out; first mode detect\n"); clear_bit(VIDEO_RES_DETECT, &video->flags); return; } @@ -1081,7 +1081,7 @@ static void aspeed_video_get_resolution(struct aspeed_video *video) MODE_DETECT_TIMEOUT); clear_bit(VIDEO_RES_DETECT, &video->flags); if (!rc) { - v4l2_warn(&video->v4l2_dev, "Timed out; second mode detect\n"); + v4l2_dbg(1, debug, &video->v4l2_dev, "Timed out; second mode detect\n"); return; } @@ -1104,7 +1104,7 @@ static void aspeed_video_get_resolution(struct aspeed_video *video) } while (invalid_resolution && (tries++ < INVALID_RESOLUTION_RETRIES)); if (invalid_resolution) { - v4l2_warn(&video->v4l2_dev, "Invalid resolution detected\n"); + v4l2_dbg(1, debug, &video->v4l2_dev, "Invalid resolution detected\n"); return; } @@ -1855,7 +1855,7 @@ static void aspeed_video_stop_streaming(struct vb2_queue *q) !test_bit(VIDEO_FRAME_INPRG, &video->flags), STOP_TIMEOUT); if (!rc) { - v4l2_warn(&video->v4l2_dev, "Timed out when stopping streaming\n"); + v4l2_dbg(1, debug, &video->v4l2_dev, "Timed out when stopping streaming\n"); /* * Need to force stop any DMA and try and get HW into a good @@ -1961,19 +1961,7 @@ static int aspeed_video_debugfs_show(struct seq_file *s, void *data) return 0; } - -static int aspeed_video_proc_open(struct inode *inode, struct file *file) -{ - return single_open(file, aspeed_video_debugfs_show, inode->i_private); -} - -static const struct file_operations aspeed_video_debugfs_ops = { - .owner = THIS_MODULE, - .open = aspeed_video_proc_open, - .read = seq_read, - .llseek = seq_lseek, - .release = single_release, -}; +DEFINE_SHOW_ATTRIBUTE(aspeed_video_debugfs); static struct dentry *debugfs_entry; @@ -1987,7 +1975,7 @@ static int aspeed_video_debugfs_create(struct aspeed_video *video) { debugfs_entry = debugfs_create_file(DEVICE_NAME, 0444, NULL, video, - &aspeed_video_debugfs_ops); + &aspeed_video_debugfs_fops); if (!debugfs_entry) aspeed_video_debugfs_remove(video); diff --git a/drivers/media/platform/atmel/Kconfig b/drivers/media/platform/atmel/Kconfig index f399dba62e17..3866ccae07df 100644 --- a/drivers/media/platform/atmel/Kconfig +++ b/drivers/media/platform/atmel/Kconfig @@ -2,42 +2,6 @@ comment "Atmel media platform drivers" -config VIDEO_ATMEL_ISC - tristate "ATMEL Image Sensor Controller (ISC) support" - depends on V4L_PLATFORM_DRIVERS - depends on VIDEO_DEV && COMMON_CLK - depends on ARCH_AT91 || COMPILE_TEST - select MEDIA_CONTROLLER - select VIDEO_V4L2_SUBDEV_API - select VIDEOBUF2_DMA_CONTIG - select REGMAP_MMIO - select V4L2_FWNODE - select VIDEO_ATMEL_ISC_BASE - help - This module makes the ATMEL Image Sensor Controller available - as a v4l2 device. - -config VIDEO_ATMEL_XISC - tristate "ATMEL eXtended Image Sensor Controller (XISC) support" - depends on V4L_PLATFORM_DRIVERS - depends on VIDEO_DEV && COMMON_CLK - depends on ARCH_AT91 || COMPILE_TEST - select VIDEOBUF2_DMA_CONTIG - select REGMAP_MMIO - select V4L2_FWNODE - select VIDEO_ATMEL_ISC_BASE - select MEDIA_CONTROLLER - select VIDEO_V4L2_SUBDEV_API - help - This module makes the ATMEL eXtended Image Sensor Controller - available as a v4l2 device. - -config VIDEO_ATMEL_ISC_BASE - tristate - default n - help - ATMEL ISC and XISC common code base. - config VIDEO_ATMEL_ISI tristate "ATMEL Image Sensor Interface (ISI) support" depends on V4L_PLATFORM_DRIVERS @@ -49,18 +13,3 @@ config VIDEO_ATMEL_ISI This module makes the ATMEL Image Sensor Interface available as a v4l2 device. -config VIDEO_MICROCHIP_CSI2DC - tristate "Microchip CSI2 Demux Controller" - depends on V4L_PLATFORM_DRIVERS - depends on VIDEO_DEV && COMMON_CLK && OF - depends on ARCH_AT91 || COMPILE_TEST - select MEDIA_CONTROLLER - select VIDEO_V4L2_SUBDEV_API - select V4L2_FWNODE - help - CSI2 Demux Controller driver. CSI2DC is a helper chip - that converts IDI interface byte stream to a parallel pixel stream. - It supports various RAW formats as input. - - To compile this driver as a module, choose M here: the - module will be called microchip-csi2dc. diff --git a/drivers/media/platform/atmel/Makefile b/drivers/media/platform/atmel/Makefile index 794e8f739287..a14ac6b5211d 100644 --- a/drivers/media/platform/atmel/Makefile +++ b/drivers/media/platform/atmel/Makefile @@ -1,10 +1,3 @@ # SPDX-License-Identifier: GPL-2.0-only -atmel-isc-objs = atmel-sama5d2-isc.o -atmel-xisc-objs = atmel-sama7g5-isc.o -atmel-isc-common-objs = atmel-isc-base.o atmel-isc-clk.o obj-$(CONFIG_VIDEO_ATMEL_ISI) += atmel-isi.o -obj-$(CONFIG_VIDEO_ATMEL_ISC_BASE) += atmel-isc-common.o -obj-$(CONFIG_VIDEO_ATMEL_ISC) += atmel-isc.o -obj-$(CONFIG_VIDEO_ATMEL_XISC) += atmel-xisc.o -obj-$(CONFIG_VIDEO_MICROCHIP_CSI2DC) += microchip-csi2dc.o diff --git a/drivers/media/platform/chips-media/coda-jpeg.c b/drivers/media/platform/chips-media/coda-jpeg.c index 435e7030fc2a..ba8f41002917 100644 --- a/drivers/media/platform/chips-media/coda-jpeg.c +++ b/drivers/media/platform/chips-media/coda-jpeg.c @@ -1052,10 +1052,16 @@ static int coda9_jpeg_start_encoding(struct coda_ctx *ctx) v4l2_err(&dev->v4l2_dev, "error loading Huffman tables\n"); return ret; } - if (!ctx->params.jpeg_qmat_tab[0]) + if (!ctx->params.jpeg_qmat_tab[0]) { ctx->params.jpeg_qmat_tab[0] = kmalloc(64, GFP_KERNEL); - if (!ctx->params.jpeg_qmat_tab[1]) + if (!ctx->params.jpeg_qmat_tab[0]) + return -ENOMEM; + } + if (!ctx->params.jpeg_qmat_tab[1]) { ctx->params.jpeg_qmat_tab[1] = kmalloc(64, GFP_KERNEL); + if (!ctx->params.jpeg_qmat_tab[1]) + return -ENOMEM; + } coda_set_jpeg_compression_quality(ctx, ctx->params.jpeg_quality); return 0; diff --git a/drivers/media/platform/mediatek/jpeg/mtk_jpeg_dec_hw.c b/drivers/media/platform/mediatek/jpeg/mtk_jpeg_dec_hw.c index d98f4cdfeea9..8c07fa02fd9a 100644 --- a/drivers/media/platform/mediatek/jpeg/mtk_jpeg_dec_hw.c +++ b/drivers/media/platform/mediatek/jpeg/mtk_jpeg_dec_hw.c @@ -580,6 +580,11 @@ static int mtk_jpegdec_hw_init_irq(struct mtk_jpegdec_comp_dev *dev) return 0; } +static void mtk_jpegdec_destroy_workqueue(void *data) +{ + destroy_workqueue(data); +} + static int mtk_jpegdec_hw_probe(struct platform_device *pdev) { struct mtk_jpegdec_clk *jpegdec_clk; @@ -614,6 +619,11 @@ static int mtk_jpegdec_hw_probe(struct platform_device *pdev) | WQ_FREEZABLE); if (!master_dev->workqueue) return -EINVAL; + + ret = devm_add_action_or_reset(&pdev->dev, mtk_jpegdec_destroy_workqueue, + master_dev->workqueue); + if (ret) + return ret; } atomic_set(&master_dev->dechw_rdy, MTK_JPEGDEC_HW_MAX); diff --git a/drivers/media/platform/mediatek/mdp3/Kconfig b/drivers/media/platform/mediatek/mdp3/Kconfig index 50ae07b75b5f..846e759a8f6a 100644 --- a/drivers/media/platform/mediatek/mdp3/Kconfig +++ b/drivers/media/platform/mediatek/mdp3/Kconfig @@ -9,7 +9,6 @@ config VIDEO_MEDIATEK_MDP3 select VIDEOBUF2_DMA_CONTIG select V4L2_MEM2MEM_DEV select MTK_MMSYS - select VIDEO_MEDIATEK_VPU select MTK_CMDQ select MTK_SCP default n diff --git a/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc.c b/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc.c index d810a78dde51..d65800a3b89d 100644 --- a/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc.c +++ b/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc.c @@ -1397,7 +1397,10 @@ int mtk_vcodec_enc_ctrls_setup(struct mtk_vcodec_ctx *ctx) 0, V4L2_MPEG_VIDEO_HEADER_MODE_SEPARATE); v4l2_ctrl_new_std_menu(handler, ops, V4L2_CID_MPEG_VIDEO_H264_PROFILE, V4L2_MPEG_VIDEO_H264_PROFILE_HIGH, - 0, V4L2_MPEG_VIDEO_H264_PROFILE_HIGH); + ~((1 << V4L2_MPEG_VIDEO_H264_PROFILE_BASELINE) | + (1 << V4L2_MPEG_VIDEO_H264_PROFILE_MAIN) | + (1 << V4L2_MPEG_VIDEO_H264_PROFILE_HIGH)), + V4L2_MPEG_VIDEO_H264_PROFILE_HIGH); v4l2_ctrl_new_std_menu(handler, ops, V4L2_CID_MPEG_VIDEO_H264_LEVEL, h264_max_level, 0, V4L2_MPEG_VIDEO_H264_LEVEL_4_0); diff --git a/drivers/media/platform/mediatek/vcodec/vdec/vdec_h264_req_multi_if.c b/drivers/media/platform/mediatek/vcodec/vdec/vdec_h264_req_multi_if.c index 4cc92700692b..18e048755d11 100644 --- a/drivers/media/platform/mediatek/vcodec/vdec/vdec_h264_req_multi_if.c +++ b/drivers/media/platform/mediatek/vcodec/vdec/vdec_h264_req_multi_if.c @@ -539,6 +539,29 @@ vdec_dec_end: return 0; } +static void vdec_h264_insert_startcode(struct mtk_vcodec_dev *vcodec_dev, unsigned char *buf, + size_t *bs_size, struct mtk_h264_pps_param *pps) +{ + struct device *dev = &vcodec_dev->plat_dev->dev; + + /* Need to add pending data at the end of bitstream when bs_sz is small than + * 20 bytes for cavlc bitstream, or lat will decode fail. This pending data is + * useful for mt8192 and mt8195 platform. + * + * cavlc bitstream when entropy_coding_mode_flag is false. + */ + if (pps->entropy_coding_mode_flag || *bs_size > 20 || + !(of_device_is_compatible(dev->of_node, "mediatek,mt8192-vcodec-dec") || + of_device_is_compatible(dev->of_node, "mediatek,mt8195-vcodec-dec"))) + return; + + buf[*bs_size] = 0; + buf[*bs_size + 1] = 0; + buf[*bs_size + 2] = 1; + buf[*bs_size + 3] = 0xff; + (*bs_size) += 4; +} + static int vdec_h264_slice_lat_decode(void *h_vdec, struct mtk_vcodec_mem *bs, struct vdec_fb *fb, bool *res_chg) { @@ -582,9 +605,6 @@ static int vdec_h264_slice_lat_decode(void *h_vdec, struct mtk_vcodec_mem *bs, } inst->vsi->dec.nal_info = buf[nal_start_idx]; - inst->vsi->dec.bs_buf_addr = (u64)bs->dma_addr; - inst->vsi->dec.bs_buf_size = bs->size; - lat_buf->src_buf_req = src_buf_info->m2m_buf.vb.vb2_buf.req_obj.req; v4l2_m2m_buf_copy_metadata(&src_buf_info->m2m_buf.vb, &lat_buf->ts_info, true); @@ -592,6 +612,12 @@ static int vdec_h264_slice_lat_decode(void *h_vdec, struct mtk_vcodec_mem *bs, if (err) goto err_free_fb_out; + vdec_h264_insert_startcode(inst->ctx->dev, buf, &bs->size, + &share_info->h264_slice_params.pps); + + inst->vsi->dec.bs_buf_addr = (uint64_t)bs->dma_addr; + inst->vsi->dec.bs_buf_size = bs->size; + *res_chg = inst->resolution_changed; if (inst->resolution_changed) { mtk_vcodec_debug(inst, "- resolution changed -"); diff --git a/drivers/media/platform/microchip/Kconfig b/drivers/media/platform/microchip/Kconfig new file mode 100644 index 000000000000..4734ecced029 --- /dev/null +++ b/drivers/media/platform/microchip/Kconfig @@ -0,0 +1,61 @@ +# SPDX-License-Identifier: GPL-2.0-only + +comment "Microchip Technology, Inc. media platform drivers" + +config VIDEO_MICROCHIP_ISC + tristate "Microchip Image Sensor Controller (ISC) support" + depends on V4L_PLATFORM_DRIVERS + depends on VIDEO_DEV && COMMON_CLK + depends on ARCH_AT91 || COMPILE_TEST + select MEDIA_CONTROLLER + select VIDEO_V4L2_SUBDEV_API + select VIDEOBUF2_DMA_CONTIG + select REGMAP_MMIO + select V4L2_FWNODE + select VIDEO_MICROCHIP_ISC_BASE + help + This module makes the Microchip Image Sensor Controller available + as a v4l2 device. + + To compile this driver as a module, choose M here: the + module will be called microchip-isc. + +config VIDEO_MICROCHIP_XISC + tristate "Microchip eXtended Image Sensor Controller (XISC) support" + depends on V4L_PLATFORM_DRIVERS + depends on VIDEO_DEV && COMMON_CLK + depends on ARCH_AT91 || COMPILE_TEST + select VIDEOBUF2_DMA_CONTIG + select REGMAP_MMIO + select V4L2_FWNODE + select VIDEO_MICROCHIP_ISC_BASE + select MEDIA_CONTROLLER + select VIDEO_V4L2_SUBDEV_API + help + This module makes the Microchip eXtended Image Sensor Controller + available as a v4l2 device. + + To compile this driver as a module, choose M here: the + module will be called microchip-xisc. + +config VIDEO_MICROCHIP_ISC_BASE + tristate + default n + help + Microchip ISC and XISC common code base. + +config VIDEO_MICROCHIP_CSI2DC + tristate "Microchip CSI2 Demux Controller" + depends on V4L_PLATFORM_DRIVERS + depends on VIDEO_DEV && COMMON_CLK && OF + depends on ARCH_AT91 || COMPILE_TEST + select MEDIA_CONTROLLER + select VIDEO_V4L2_SUBDEV_API + select V4L2_FWNODE + help + CSI2 Demux Controller driver. CSI2DC is a helper chip + that converts IDI interface byte stream to a parallel pixel stream. + It supports various RAW formats as input. + + To compile this driver as a module, choose M here: the + module will be called microchip-csi2dc. diff --git a/drivers/media/platform/microchip/Makefile b/drivers/media/platform/microchip/Makefile new file mode 100644 index 000000000000..bd8d6e779c51 --- /dev/null +++ b/drivers/media/platform/microchip/Makefile @@ -0,0 +1,9 @@ +# SPDX-License-Identifier: GPL-2.0-only +microchip-isc-objs = microchip-sama5d2-isc.o +microchip-xisc-objs = microchip-sama7g5-isc.o +microchip-isc-common-objs = microchip-isc-base.o microchip-isc-clk.o microchip-isc-scaler.o + +obj-$(CONFIG_VIDEO_MICROCHIP_ISC_BASE) += microchip-isc-common.o +obj-$(CONFIG_VIDEO_MICROCHIP_ISC) += microchip-isc.o +obj-$(CONFIG_VIDEO_MICROCHIP_XISC) += microchip-xisc.o +obj-$(CONFIG_VIDEO_MICROCHIP_CSI2DC) += microchip-csi2dc.o diff --git a/drivers/media/platform/atmel/microchip-csi2dc.c b/drivers/media/platform/microchip/microchip-csi2dc.c index d5b359f607ae..d5b359f607ae 100644 --- a/drivers/media/platform/atmel/microchip-csi2dc.c +++ b/drivers/media/platform/microchip/microchip-csi2dc.c diff --git a/drivers/media/platform/microchip/microchip-isc-base.c b/drivers/media/platform/microchip/microchip-isc-base.c new file mode 100644 index 000000000000..e2994d48f10c --- /dev/null +++ b/drivers/media/platform/microchip/microchip-isc-base.c @@ -0,0 +1,2040 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Microchip Image Sensor Controller (ISC) common driver base + * + * Copyright (C) 2016-2019 Microchip Technology, Inc. + * + * Author: Songjun Wu + * Author: Eugen Hristev <[email protected]> + * + */ +#include <linux/delay.h> +#include <linux/interrupt.h> +#include <linux/math64.h> +#include <linux/module.h> +#include <linux/of.h> +#include <linux/of_graph.h> +#include <linux/platform_device.h> +#include <linux/pm_runtime.h> +#include <linux/regmap.h> +#include <linux/videodev2.h> +#include <linux/atmel-isc-media.h> + +#include <media/v4l2-ctrls.h> +#include <media/v4l2-device.h> +#include <media/v4l2-event.h> +#include <media/v4l2-image-sizes.h> +#include <media/v4l2-ioctl.h> +#include <media/v4l2-fwnode.h> +#include <media/v4l2-subdev.h> +#include <media/videobuf2-dma-contig.h> + +#include "microchip-isc-regs.h" +#include "microchip-isc.h" + +static unsigned int debug; +module_param(debug, int, 0644); +MODULE_PARM_DESC(debug, "debug level (0-2)"); + +#define ISC_IS_FORMAT_RAW(mbus_code) \ + (((mbus_code) & 0xf000) == 0x3000) + +#define ISC_IS_FORMAT_GREY(mbus_code) \ + (((mbus_code) == MEDIA_BUS_FMT_Y10_1X10) | \ + (((mbus_code) == MEDIA_BUS_FMT_Y8_1X8))) + +static inline void isc_update_v4l2_ctrls(struct isc_device *isc) +{ + struct isc_ctrls *ctrls = &isc->ctrls; + + /* In here we set the v4l2 controls w.r.t. our pipeline config */ + v4l2_ctrl_s_ctrl(isc->r_gain_ctrl, ctrls->gain[ISC_HIS_CFG_MODE_R]); + v4l2_ctrl_s_ctrl(isc->b_gain_ctrl, ctrls->gain[ISC_HIS_CFG_MODE_B]); + v4l2_ctrl_s_ctrl(isc->gr_gain_ctrl, ctrls->gain[ISC_HIS_CFG_MODE_GR]); + v4l2_ctrl_s_ctrl(isc->gb_gain_ctrl, ctrls->gain[ISC_HIS_CFG_MODE_GB]); + + v4l2_ctrl_s_ctrl(isc->r_off_ctrl, ctrls->offset[ISC_HIS_CFG_MODE_R]); + v4l2_ctrl_s_ctrl(isc->b_off_ctrl, ctrls->offset[ISC_HIS_CFG_MODE_B]); + v4l2_ctrl_s_ctrl(isc->gr_off_ctrl, ctrls->offset[ISC_HIS_CFG_MODE_GR]); + v4l2_ctrl_s_ctrl(isc->gb_off_ctrl, ctrls->offset[ISC_HIS_CFG_MODE_GB]); +} + +static inline void isc_update_awb_ctrls(struct isc_device *isc) +{ + struct isc_ctrls *ctrls = &isc->ctrls; + + /* In here we set our actual hw pipeline config */ + + regmap_write(isc->regmap, ISC_WB_O_RGR, + ((ctrls->offset[ISC_HIS_CFG_MODE_R])) | + ((ctrls->offset[ISC_HIS_CFG_MODE_GR]) << 16)); + regmap_write(isc->regmap, ISC_WB_O_BGB, + ((ctrls->offset[ISC_HIS_CFG_MODE_B])) | + ((ctrls->offset[ISC_HIS_CFG_MODE_GB]) << 16)); + regmap_write(isc->regmap, ISC_WB_G_RGR, + ctrls->gain[ISC_HIS_CFG_MODE_R] | + (ctrls->gain[ISC_HIS_CFG_MODE_GR] << 16)); + regmap_write(isc->regmap, ISC_WB_G_BGB, + ctrls->gain[ISC_HIS_CFG_MODE_B] | + (ctrls->gain[ISC_HIS_CFG_MODE_GB] << 16)); +} + +static inline void isc_reset_awb_ctrls(struct isc_device *isc) +{ + unsigned int c; + + for (c = ISC_HIS_CFG_MODE_GR; c <= ISC_HIS_CFG_MODE_B; c++) { + /* gains have a fixed point at 9 decimals */ + isc->ctrls.gain[c] = 1 << 9; + /* offsets are in 2's complements */ + isc->ctrls.offset[c] = 0; + } +} + +static int isc_queue_setup(struct vb2_queue *vq, + unsigned int *nbuffers, unsigned int *nplanes, + unsigned int sizes[], struct device *alloc_devs[]) +{ + struct isc_device *isc = vb2_get_drv_priv(vq); + unsigned int size = isc->fmt.fmt.pix.sizeimage; + + if (*nplanes) + return sizes[0] < size ? -EINVAL : 0; + + *nplanes = 1; + sizes[0] = size; + + return 0; +} + +static int isc_buffer_prepare(struct vb2_buffer *vb) +{ + struct vb2_v4l2_buffer *vbuf = to_vb2_v4l2_buffer(vb); + struct isc_device *isc = vb2_get_drv_priv(vb->vb2_queue); + unsigned long size = isc->fmt.fmt.pix.sizeimage; + + if (vb2_plane_size(vb, 0) < size) { + v4l2_err(&isc->v4l2_dev, "buffer too small (%lu < %lu)\n", + vb2_plane_size(vb, 0), size); + return -EINVAL; + } + + vb2_set_plane_payload(vb, 0, size); + + vbuf->field = isc->fmt.fmt.pix.field; + + return 0; +} + +static void isc_crop_pfe(struct isc_device *isc) +{ + struct regmap *regmap = isc->regmap; + u32 h, w; + + h = isc->fmt.fmt.pix.height; + w = isc->fmt.fmt.pix.width; + + /* + * In case the sensor is not RAW, it will output a pixel (12-16 bits) + * with two samples on the ISC Data bus (which is 8-12) + * ISC will count each sample, so, we need to multiply these values + * by two, to get the real number of samples for the required pixels. + */ + if (!ISC_IS_FORMAT_RAW(isc->config.sd_format->mbus_code)) { + h <<= 1; + w <<= 1; + } + + /* + * We limit the column/row count that the ISC will output according + * to the configured resolution that we want. + * This will avoid the situation where the sensor is misconfigured, + * sending more data, and the ISC will just take it and DMA to memory, + * causing corruption. + */ + regmap_write(regmap, ISC_PFE_CFG1, + (ISC_PFE_CFG1_COLMIN(0) & ISC_PFE_CFG1_COLMIN_MASK) | + (ISC_PFE_CFG1_COLMAX(w - 1) & ISC_PFE_CFG1_COLMAX_MASK)); + + regmap_write(regmap, ISC_PFE_CFG2, + (ISC_PFE_CFG2_ROWMIN(0) & ISC_PFE_CFG2_ROWMIN_MASK) | + (ISC_PFE_CFG2_ROWMAX(h - 1) & ISC_PFE_CFG2_ROWMAX_MASK)); + + regmap_update_bits(regmap, ISC_PFE_CFG0, + ISC_PFE_CFG0_COLEN | ISC_PFE_CFG0_ROWEN, + ISC_PFE_CFG0_COLEN | ISC_PFE_CFG0_ROWEN); +} + +static void isc_start_dma(struct isc_device *isc) +{ + struct regmap *regmap = isc->regmap; + u32 sizeimage = isc->fmt.fmt.pix.sizeimage; + u32 dctrl_dview; + dma_addr_t addr0; + + addr0 = vb2_dma_contig_plane_dma_addr(&isc->cur_frm->vb.vb2_buf, 0); + regmap_write(regmap, ISC_DAD0 + isc->offsets.dma, addr0); + + switch (isc->config.fourcc) { + case V4L2_PIX_FMT_YUV420: + regmap_write(regmap, ISC_DAD1 + isc->offsets.dma, + addr0 + (sizeimage * 2) / 3); + regmap_write(regmap, ISC_DAD2 + isc->offsets.dma, + addr0 + (sizeimage * 5) / 6); + break; + case V4L2_PIX_FMT_YUV422P: + regmap_write(regmap, ISC_DAD1 + isc->offsets.dma, + addr0 + sizeimage / 2); + regmap_write(regmap, ISC_DAD2 + isc->offsets.dma, + addr0 + (sizeimage * 3) / 4); + break; + default: + break; + } + + dctrl_dview = isc->config.dctrl_dview; + + regmap_write(regmap, ISC_DCTRL + isc->offsets.dma, + dctrl_dview | ISC_DCTRL_IE_IS); + spin_lock(&isc->awb_lock); + regmap_write(regmap, ISC_CTRLEN, ISC_CTRL_CAPTURE); + spin_unlock(&isc->awb_lock); +} + +static void isc_set_pipeline(struct isc_device *isc, u32 pipeline) +{ + struct regmap *regmap = isc->regmap; + struct isc_ctrls *ctrls = &isc->ctrls; + u32 val, bay_cfg; + const u32 *gamma; + unsigned int i; + + /* WB-->CFA-->CC-->GAM-->CSC-->CBC-->SUB422-->SUB420 */ + for (i = 0; i < ISC_PIPE_LINE_NODE_NUM; i++) { + val = pipeline & BIT(i) ? 1 : 0; + regmap_field_write(isc->pipeline[i], val); + } + + if (!pipeline) + return; + + bay_cfg = isc->config.sd_format->cfa_baycfg; + + regmap_write(regmap, ISC_WB_CFG, bay_cfg); + isc_update_awb_ctrls(isc); + isc_update_v4l2_ctrls(isc); + + regmap_write(regmap, ISC_CFA_CFG, bay_cfg | ISC_CFA_CFG_EITPOL); + + gamma = &isc->gamma_table[ctrls->gamma_index][0]; + regmap_bulk_write(regmap, ISC_GAM_BENTRY, gamma, GAMMA_ENTRIES); + regmap_bulk_write(regmap, ISC_GAM_GENTRY, gamma, GAMMA_ENTRIES); + regmap_bulk_write(regmap, ISC_GAM_RENTRY, gamma, GAMMA_ENTRIES); + + isc->config_dpc(isc); + isc->config_csc(isc); + isc->config_cbc(isc); + isc->config_cc(isc); + isc->config_gam(isc); +} + +static int isc_update_profile(struct isc_device *isc) +{ + struct regmap *regmap = isc->regmap; + u32 sr; + int counter = 100; + + regmap_write(regmap, ISC_CTRLEN, ISC_CTRL_UPPRO); + + regmap_read(regmap, ISC_CTRLSR, &sr); + while ((sr & ISC_CTRL_UPPRO) && counter--) { + usleep_range(1000, 2000); + regmap_read(regmap, ISC_CTRLSR, &sr); + } + + if (counter < 0) { + v4l2_warn(&isc->v4l2_dev, "Time out to update profile\n"); + return -ETIMEDOUT; + } + + return 0; +} + +static void isc_set_histogram(struct isc_device *isc, bool enable) +{ + struct regmap *regmap = isc->regmap; + struct isc_ctrls *ctrls = &isc->ctrls; + + if (enable) { + regmap_write(regmap, ISC_HIS_CFG + isc->offsets.his, + ISC_HIS_CFG_MODE_GR | + (isc->config.sd_format->cfa_baycfg + << ISC_HIS_CFG_BAYSEL_SHIFT) | + ISC_HIS_CFG_RAR); + regmap_write(regmap, ISC_HIS_CTRL + isc->offsets.his, + ISC_HIS_CTRL_EN); + regmap_write(regmap, ISC_INTEN, ISC_INT_HISDONE); + ctrls->hist_id = ISC_HIS_CFG_MODE_GR; + isc_update_profile(isc); + regmap_write(regmap, ISC_CTRLEN, ISC_CTRL_HISREQ); + + ctrls->hist_stat = HIST_ENABLED; + } else { + regmap_write(regmap, ISC_INTDIS, ISC_INT_HISDONE); + regmap_write(regmap, ISC_HIS_CTRL + isc->offsets.his, + ISC_HIS_CTRL_DIS); + + ctrls->hist_stat = HIST_DISABLED; + } +} + +static int isc_configure(struct isc_device *isc) +{ + struct regmap *regmap = isc->regmap; + u32 pfe_cfg0, dcfg, mask, pipeline; + struct isc_subdev_entity *subdev = isc->current_subdev; + + pfe_cfg0 = isc->config.sd_format->pfe_cfg0_bps; + pipeline = isc->config.bits_pipeline; + + dcfg = isc->config.dcfg_imode | isc->dcfg; + + pfe_cfg0 |= subdev->pfe_cfg0 | ISC_PFE_CFG0_MODE_PROGRESSIVE; + mask = ISC_PFE_CFG0_BPS_MASK | ISC_PFE_CFG0_HPOL_LOW | + ISC_PFE_CFG0_VPOL_LOW | ISC_PFE_CFG0_PPOL_LOW | + ISC_PFE_CFG0_MODE_MASK | ISC_PFE_CFG0_CCIR_CRC | + ISC_PFE_CFG0_CCIR656 | ISC_PFE_CFG0_MIPI; + + regmap_update_bits(regmap, ISC_PFE_CFG0, mask, pfe_cfg0); + + isc->config_rlp(isc); + + regmap_write(regmap, ISC_DCFG + isc->offsets.dma, dcfg); + + /* Set the pipeline */ + isc_set_pipeline(isc, pipeline); + + /* + * The current implemented histogram is available for RAW R, B, GB, GR + * channels. We need to check if sensor is outputting RAW BAYER + */ + if (isc->ctrls.awb && + ISC_IS_FORMAT_RAW(isc->config.sd_format->mbus_code)) + isc_set_histogram(isc, true); + else + isc_set_histogram(isc, false); + + /* Update profile */ + return isc_update_profile(isc); +} + +static int isc_prepare_streaming(struct vb2_queue *vq) +{ + struct isc_device *isc = vb2_get_drv_priv(vq); + + return media_pipeline_start(isc->video_dev.entity.pads, &isc->mpipe); +} + +static int isc_start_streaming(struct vb2_queue *vq, unsigned int count) +{ + struct isc_device *isc = vb2_get_drv_priv(vq); + struct regmap *regmap = isc->regmap; + struct isc_buffer *buf; + unsigned long flags; + int ret; + + /* Enable stream on the sub device */ + ret = v4l2_subdev_call(isc->current_subdev->sd, video, s_stream, 1); + if (ret && ret != -ENOIOCTLCMD) { + v4l2_err(&isc->v4l2_dev, "stream on failed in subdev %d\n", + ret); + goto err_start_stream; + } + + ret = pm_runtime_resume_and_get(isc->dev); + if (ret < 0) { + v4l2_err(&isc->v4l2_dev, "RPM resume failed in subdev %d\n", + ret); + goto err_pm_get; + } + + ret = isc_configure(isc); + if (unlikely(ret)) + goto err_configure; + + /* Enable DMA interrupt */ + regmap_write(regmap, ISC_INTEN, ISC_INT_DDONE); + + spin_lock_irqsave(&isc->dma_queue_lock, flags); + + isc->sequence = 0; + isc->stop = false; + reinit_completion(&isc->comp); + + isc->cur_frm = list_first_entry(&isc->dma_queue, + struct isc_buffer, list); + list_del(&isc->cur_frm->list); + + isc_crop_pfe(isc); + isc_start_dma(isc); + + spin_unlock_irqrestore(&isc->dma_queue_lock, flags); + + /* if we streaming from RAW, we can do one-shot white balance adj */ + if (ISC_IS_FORMAT_RAW(isc->config.sd_format->mbus_code)) + v4l2_ctrl_activate(isc->do_wb_ctrl, true); + + return 0; + +err_configure: + pm_runtime_put_sync(isc->dev); +err_pm_get: + v4l2_subdev_call(isc->current_subdev->sd, video, s_stream, 0); + +err_start_stream: + spin_lock_irqsave(&isc->dma_queue_lock, flags); + list_for_each_entry(buf, &isc->dma_queue, list) + vb2_buffer_done(&buf->vb.vb2_buf, VB2_BUF_STATE_QUEUED); + INIT_LIST_HEAD(&isc->dma_queue); + spin_unlock_irqrestore(&isc->dma_queue_lock, flags); + + return ret; +} + +static void isc_unprepare_streaming(struct vb2_queue *vq) +{ + struct isc_device *isc = vb2_get_drv_priv(vq); + + /* Stop media pipeline */ + media_pipeline_stop(isc->video_dev.entity.pads); +} + +static void isc_stop_streaming(struct vb2_queue *vq) +{ + struct isc_device *isc = vb2_get_drv_priv(vq); + unsigned long flags; + struct isc_buffer *buf; + int ret; + + mutex_lock(&isc->awb_mutex); + v4l2_ctrl_activate(isc->do_wb_ctrl, false); + + isc->stop = true; + + /* Wait until the end of the current frame */ + if (isc->cur_frm && !wait_for_completion_timeout(&isc->comp, 5 * HZ)) + v4l2_err(&isc->v4l2_dev, + "Timeout waiting for end of the capture\n"); + + mutex_unlock(&isc->awb_mutex); + + /* Disable DMA interrupt */ + regmap_write(isc->regmap, ISC_INTDIS, ISC_INT_DDONE); + + pm_runtime_put_sync(isc->dev); + + /* Disable stream on the sub device */ + ret = v4l2_subdev_call(isc->current_subdev->sd, video, s_stream, 0); + if (ret && ret != -ENOIOCTLCMD) + v4l2_err(&isc->v4l2_dev, "stream off failed in subdev\n"); + + /* Release all active buffers */ + spin_lock_irqsave(&isc->dma_queue_lock, flags); + if (unlikely(isc->cur_frm)) { + vb2_buffer_done(&isc->cur_frm->vb.vb2_buf, + VB2_BUF_STATE_ERROR); + isc->cur_frm = NULL; + } + list_for_each_entry(buf, &isc->dma_queue, list) + vb2_buffer_done(&buf->vb.vb2_buf, VB2_BUF_STATE_ERROR); + INIT_LIST_HEAD(&isc->dma_queue); + spin_unlock_irqrestore(&isc->dma_queue_lock, flags); +} + +static void isc_buffer_queue(struct vb2_buffer *vb) +{ + struct vb2_v4l2_buffer *vbuf = to_vb2_v4l2_buffer(vb); + struct isc_buffer *buf = container_of(vbuf, struct isc_buffer, vb); + struct isc_device *isc = vb2_get_drv_priv(vb->vb2_queue); + unsigned long flags; + + spin_lock_irqsave(&isc->dma_queue_lock, flags); + if (!isc->cur_frm && list_empty(&isc->dma_queue) && + vb2_start_streaming_called(vb->vb2_queue)) { + isc->cur_frm = buf; + isc_start_dma(isc); + } else { + list_add_tail(&buf->list, &isc->dma_queue); + } + spin_unlock_irqrestore(&isc->dma_queue_lock, flags); +} + +static const struct vb2_ops isc_vb2_ops = { + .queue_setup = isc_queue_setup, + .wait_prepare = vb2_ops_wait_prepare, + .wait_finish = vb2_ops_wait_finish, + .buf_prepare = isc_buffer_prepare, + .start_streaming = isc_start_streaming, + .stop_streaming = isc_stop_streaming, + .buf_queue = isc_buffer_queue, + .prepare_streaming = isc_prepare_streaming, + .unprepare_streaming = isc_unprepare_streaming, +}; + +static int isc_querycap(struct file *file, void *priv, + struct v4l2_capability *cap) +{ + struct isc_device *isc = video_drvdata(file); + + strscpy(cap->driver, "microchip-isc", sizeof(cap->driver)); + strscpy(cap->card, "Microchip Image Sensor Controller", sizeof(cap->card)); + snprintf(cap->bus_info, sizeof(cap->bus_info), + "platform:%s", isc->v4l2_dev.name); + + return 0; +} + +static int isc_enum_fmt_vid_cap(struct file *file, void *priv, + struct v4l2_fmtdesc *f) +{ + struct isc_device *isc = video_drvdata(file); + u32 index = f->index; + u32 i, supported_index = 0; + struct isc_format *fmt; + + /* + * If we are not asked a specific mbus_code, we have to report all + * the formats that we can output. + */ + if (!f->mbus_code) { + if (index >= isc->controller_formats_size) + return -EINVAL; + + f->pixelformat = isc->controller_formats[index].fourcc; + + return 0; + } + + /* + * If a specific mbus_code is requested, check if we support + * this mbus_code as input for the ISC. + * If it's supported, then we report the corresponding pixelformat + * as first possible option for the ISC. + * E.g. mbus MEDIA_BUS_FMT_YUYV8_2X8 and report + * 'YUYV' (YUYV 4:2:2) + */ + fmt = isc_find_format_by_code(isc, f->mbus_code, &i); + if (!fmt) + return -EINVAL; + + if (!index) { + f->pixelformat = fmt->fourcc; + + return 0; + } + + supported_index++; + + /* If the index is not raw, we don't have anymore formats to report */ + if (!ISC_IS_FORMAT_RAW(f->mbus_code)) + return -EINVAL; + + /* + * We are asked for a specific mbus code, which is raw. + * We have to search through the formats we can convert to. + * We have to skip the raw formats, we cannot convert to raw. + * E.g. 'AR12' (16-bit ARGB 4-4-4-4), 'AR15' (16-bit ARGB 1-5-5-5), etc. + */ + for (i = 0; i < isc->controller_formats_size; i++) { + if (isc->controller_formats[i].raw) + continue; + if (index == supported_index) { + f->pixelformat = isc->controller_formats[i].fourcc; + return 0; + } + supported_index++; + } + + return -EINVAL; +} + +static int isc_g_fmt_vid_cap(struct file *file, void *priv, + struct v4l2_format *fmt) +{ + struct isc_device *isc = video_drvdata(file); + + *fmt = isc->fmt; + + return 0; +} + +/* + * Checks the current configured format, if ISC can output it, + * considering which type of format the ISC receives from the sensor + */ +static int isc_try_validate_formats(struct isc_device *isc) +{ + int ret; + bool bayer = false, yuv = false, rgb = false, grey = false; + + /* all formats supported by the RLP module are OK */ + switch (isc->try_config.fourcc) { + case V4L2_PIX_FMT_SBGGR8: + case V4L2_PIX_FMT_SGBRG8: + case V4L2_PIX_FMT_SGRBG8: + case V4L2_PIX_FMT_SRGGB8: + case V4L2_PIX_FMT_SBGGR10: + case V4L2_PIX_FMT_SGBRG10: + case V4L2_PIX_FMT_SGRBG10: + case V4L2_PIX_FMT_SRGGB10: + case V4L2_PIX_FMT_SBGGR12: + case V4L2_PIX_FMT_SGBRG12: + case V4L2_PIX_FMT_SGRBG12: + case V4L2_PIX_FMT_SRGGB12: + ret = 0; + bayer = true; + break; + + case V4L2_PIX_FMT_YUV420: + case V4L2_PIX_FMT_YUV422P: + case V4L2_PIX_FMT_YUYV: + case V4L2_PIX_FMT_UYVY: + case V4L2_PIX_FMT_VYUY: + ret = 0; + yuv = true; + break; + + case V4L2_PIX_FMT_RGB565: + case V4L2_PIX_FMT_ABGR32: + case V4L2_PIX_FMT_XBGR32: + case V4L2_PIX_FMT_ARGB444: + case V4L2_PIX_FMT_ARGB555: + ret = 0; + rgb = true; + break; + case V4L2_PIX_FMT_GREY: + case V4L2_PIX_FMT_Y10: + case V4L2_PIX_FMT_Y16: + ret = 0; + grey = true; + break; + default: + /* any other different formats are not supported */ + v4l2_err(&isc->v4l2_dev, "Requested unsupported format.\n"); + ret = -EINVAL; + } + v4l2_dbg(1, debug, &isc->v4l2_dev, + "Format validation, requested rgb=%u, yuv=%u, grey=%u, bayer=%u\n", + rgb, yuv, grey, bayer); + + if (bayer && + !ISC_IS_FORMAT_RAW(isc->try_config.sd_format->mbus_code)) { + v4l2_err(&isc->v4l2_dev, "Cannot output RAW if we do not receive RAW.\n"); + return -EINVAL; + } + + if (grey && !ISC_IS_FORMAT_RAW(isc->try_config.sd_format->mbus_code) && + !ISC_IS_FORMAT_GREY(isc->try_config.sd_format->mbus_code)) { + v4l2_err(&isc->v4l2_dev, "Cannot output GREY if we do not receive RAW/GREY.\n"); + return -EINVAL; + } + + if ((rgb || bayer || yuv) && + ISC_IS_FORMAT_GREY(isc->try_config.sd_format->mbus_code)) { + v4l2_err(&isc->v4l2_dev, "Cannot convert GREY to another format.\n"); + return -EINVAL; + } + + return ret; +} + +/* + * Configures the RLP and DMA modules, depending on the output format + * configured for the ISC. + * If direct_dump == true, just dump raw data 8/16 bits depending on format. + */ +static int isc_try_configure_rlp_dma(struct isc_device *isc, bool direct_dump) +{ + isc->try_config.rlp_cfg_mode = 0; + + switch (isc->try_config.fourcc) { + case V4L2_PIX_FMT_SBGGR8: + case V4L2_PIX_FMT_SGBRG8: + case V4L2_PIX_FMT_SGRBG8: + case V4L2_PIX_FMT_SRGGB8: + isc->try_config.rlp_cfg_mode = ISC_RLP_CFG_MODE_DAT8; + isc->try_config.dcfg_imode = ISC_DCFG_IMODE_PACKED8; + isc->try_config.dctrl_dview = ISC_DCTRL_DVIEW_PACKED; + isc->try_config.bpp = 8; + isc->try_config.bpp_v4l2 = 8; + break; + case V4L2_PIX_FMT_SBGGR10: + case V4L2_PIX_FMT_SGBRG10: + case V4L2_PIX_FMT_SGRBG10: + case V4L2_PIX_FMT_SRGGB10: + isc->try_config.rlp_cfg_mode = ISC_RLP_CFG_MODE_DAT10; + isc->try_config.dcfg_imode = ISC_DCFG_IMODE_PACKED16; + isc->try_config.dctrl_dview = ISC_DCTRL_DVIEW_PACKED; + isc->try_config.bpp = 16; + isc->try_config.bpp_v4l2 = 16; + break; + case V4L2_PIX_FMT_SBGGR12: + case V4L2_PIX_FMT_SGBRG12: + case V4L2_PIX_FMT_SGRBG12: + case V4L2_PIX_FMT_SRGGB12: + isc->try_config.rlp_cfg_mode = ISC_RLP_CFG_MODE_DAT12; + isc->try_config.dcfg_imode = ISC_DCFG_IMODE_PACKED16; + isc->try_config.dctrl_dview = ISC_DCTRL_DVIEW_PACKED; + isc->try_config.bpp = 16; + isc->try_config.bpp_v4l2 = 16; + break; + case V4L2_PIX_FMT_RGB565: + isc->try_config.rlp_cfg_mode = ISC_RLP_CFG_MODE_RGB565; + isc->try_config.dcfg_imode = ISC_DCFG_IMODE_PACKED16; + isc->try_config.dctrl_dview = ISC_DCTRL_DVIEW_PACKED; + isc->try_config.bpp = 16; + isc->try_config.bpp_v4l2 = 16; + break; + case V4L2_PIX_FMT_ARGB444: + isc->try_config.rlp_cfg_mode = ISC_RLP_CFG_MODE_ARGB444; + isc->try_config.dcfg_imode = ISC_DCFG_IMODE_PACKED16; + isc->try_config.dctrl_dview = ISC_DCTRL_DVIEW_PACKED; + isc->try_config.bpp = 16; + isc->try_config.bpp_v4l2 = 16; + break; + case V4L2_PIX_FMT_ARGB555: + isc->try_config.rlp_cfg_mode = ISC_RLP_CFG_MODE_ARGB555; + isc->try_config.dcfg_imode = ISC_DCFG_IMODE_PACKED16; + isc->try_config.dctrl_dview = ISC_DCTRL_DVIEW_PACKED; + isc->try_config.bpp = 16; + isc->try_config.bpp_v4l2 = 16; + break; + case V4L2_PIX_FMT_ABGR32: + case V4L2_PIX_FMT_XBGR32: + isc->try_config.rlp_cfg_mode = ISC_RLP_CFG_MODE_ARGB32; + isc->try_config.dcfg_imode = ISC_DCFG_IMODE_PACKED32; + isc->try_config.dctrl_dview = ISC_DCTRL_DVIEW_PACKED; + isc->try_config.bpp = 32; + isc->try_config.bpp_v4l2 = 32; + break; + case V4L2_PIX_FMT_YUV420: + isc->try_config.rlp_cfg_mode = ISC_RLP_CFG_MODE_YYCC; + isc->try_config.dcfg_imode = ISC_DCFG_IMODE_YC420P; + isc->try_config.dctrl_dview = ISC_DCTRL_DVIEW_PLANAR; + isc->try_config.bpp = 12; + isc->try_config.bpp_v4l2 = 8; /* only first plane */ + break; + case V4L2_PIX_FMT_YUV422P: + isc->try_config.rlp_cfg_mode = ISC_RLP_CFG_MODE_YYCC; + isc->try_config.dcfg_imode = ISC_DCFG_IMODE_YC422P; + isc->try_config.dctrl_dview = ISC_DCTRL_DVIEW_PLANAR; + isc->try_config.bpp = 16; + isc->try_config.bpp_v4l2 = 8; /* only first plane */ + break; + case V4L2_PIX_FMT_YUYV: + isc->try_config.rlp_cfg_mode = ISC_RLP_CFG_MODE_YCYC | ISC_RLP_CFG_YMODE_YUYV; + isc->try_config.dcfg_imode = ISC_DCFG_IMODE_PACKED32; + isc->try_config.dctrl_dview = ISC_DCTRL_DVIEW_PACKED; + isc->try_config.bpp = 16; + isc->try_config.bpp_v4l2 = 16; + break; + case V4L2_PIX_FMT_UYVY: + isc->try_config.rlp_cfg_mode = ISC_RLP_CFG_MODE_YCYC | ISC_RLP_CFG_YMODE_UYVY; + isc->try_config.dcfg_imode = ISC_DCFG_IMODE_PACKED32; + isc->try_config.dctrl_dview = ISC_DCTRL_DVIEW_PACKED; + isc->try_config.bpp = 16; + isc->try_config.bpp_v4l2 = 16; + break; + case V4L2_PIX_FMT_VYUY: + isc->try_config.rlp_cfg_mode = ISC_RLP_CFG_MODE_YCYC | ISC_RLP_CFG_YMODE_VYUY; + isc->try_config.dcfg_imode = ISC_DCFG_IMODE_PACKED32; + isc->try_config.dctrl_dview = ISC_DCTRL_DVIEW_PACKED; + isc->try_config.bpp = 16; + isc->try_config.bpp_v4l2 = 16; + break; + case V4L2_PIX_FMT_GREY: + isc->try_config.rlp_cfg_mode = ISC_RLP_CFG_MODE_DATY8; + isc->try_config.dcfg_imode = ISC_DCFG_IMODE_PACKED8; + isc->try_config.dctrl_dview = ISC_DCTRL_DVIEW_PACKED; + isc->try_config.bpp = 8; + isc->try_config.bpp_v4l2 = 8; + break; + case V4L2_PIX_FMT_Y16: + isc->try_config.rlp_cfg_mode = ISC_RLP_CFG_MODE_DATY10 | ISC_RLP_CFG_LSH; + fallthrough; + case V4L2_PIX_FMT_Y10: + isc->try_config.rlp_cfg_mode |= ISC_RLP_CFG_MODE_DATY10; + isc->try_config.dcfg_imode = ISC_DCFG_IMODE_PACKED16; + isc->try_config.dctrl_dview = ISC_DCTRL_DVIEW_PACKED; + isc->try_config.bpp = 16; + isc->try_config.bpp_v4l2 = 16; + break; + default: + return -EINVAL; + } + + if (direct_dump) { + isc->try_config.rlp_cfg_mode = ISC_RLP_CFG_MODE_DAT8; + isc->try_config.dcfg_imode = ISC_DCFG_IMODE_PACKED8; + isc->try_config.dctrl_dview = ISC_DCTRL_DVIEW_PACKED; + return 0; + } + + return 0; +} + +/* + * Configuring pipeline modules, depending on which format the ISC outputs + * and considering which format it has as input from the sensor. + */ +static int isc_try_configure_pipeline(struct isc_device *isc) +{ + switch (isc->try_config.fourcc) { + case V4L2_PIX_FMT_RGB565: + case V4L2_PIX_FMT_ARGB555: + case V4L2_PIX_FMT_ARGB444: + case V4L2_PIX_FMT_ABGR32: + case V4L2_PIX_FMT_XBGR32: + /* if sensor format is RAW, we convert inside ISC */ + if (ISC_IS_FORMAT_RAW(isc->try_config.sd_format->mbus_code)) { + isc->try_config.bits_pipeline = CFA_ENABLE | + WB_ENABLE | GAM_ENABLES | DPC_BLCENABLE | + CC_ENABLE; + } else { + isc->try_config.bits_pipeline = 0x0; + } + break; + case V4L2_PIX_FMT_YUV420: + /* if sensor format is RAW, we convert inside ISC */ + if (ISC_IS_FORMAT_RAW(isc->try_config.sd_format->mbus_code)) { + isc->try_config.bits_pipeline = CFA_ENABLE | + CSC_ENABLE | GAM_ENABLES | WB_ENABLE | + SUB420_ENABLE | SUB422_ENABLE | CBC_ENABLE | + DPC_BLCENABLE; + } else { + isc->try_config.bits_pipeline = 0x0; + } + break; + case V4L2_PIX_FMT_YUV422P: + /* if sensor format is RAW, we convert inside ISC */ + if (ISC_IS_FORMAT_RAW(isc->try_config.sd_format->mbus_code)) { + isc->try_config.bits_pipeline = CFA_ENABLE | + CSC_ENABLE | WB_ENABLE | GAM_ENABLES | + SUB422_ENABLE | CBC_ENABLE | DPC_BLCENABLE; + } else { + isc->try_config.bits_pipeline = 0x0; + } + break; + case V4L2_PIX_FMT_YUYV: + case V4L2_PIX_FMT_UYVY: + case V4L2_PIX_FMT_VYUY: + /* if sensor format is RAW, we convert inside ISC */ + if (ISC_IS_FORMAT_RAW(isc->try_config.sd_format->mbus_code)) { + isc->try_config.bits_pipeline = CFA_ENABLE | + CSC_ENABLE | WB_ENABLE | GAM_ENABLES | + SUB422_ENABLE | CBC_ENABLE | DPC_BLCENABLE; + } else { + isc->try_config.bits_pipeline = 0x0; + } + break; + case V4L2_PIX_FMT_GREY: + case V4L2_PIX_FMT_Y16: + /* if sensor format is RAW, we convert inside ISC */ + if (ISC_IS_FORMAT_RAW(isc->try_config.sd_format->mbus_code)) { + isc->try_config.bits_pipeline = CFA_ENABLE | + CSC_ENABLE | WB_ENABLE | GAM_ENABLES | + CBC_ENABLE | DPC_BLCENABLE; + } else { + isc->try_config.bits_pipeline = 0x0; + } + break; + default: + if (ISC_IS_FORMAT_RAW(isc->try_config.sd_format->mbus_code)) + isc->try_config.bits_pipeline = WB_ENABLE | DPC_BLCENABLE; + else + isc->try_config.bits_pipeline = 0x0; + } + + /* Tune the pipeline to product specific */ + isc->adapt_pipeline(isc); + + return 0; +} + +static void isc_try_fse(struct isc_device *isc, + struct v4l2_subdev_state *sd_state) +{ + int ret; + struct v4l2_subdev_frame_size_enum fse = {}; + + /* + * If we do not know yet which format the subdev is using, we cannot + * do anything. + */ + if (!isc->config.sd_format) + return; + + fse.code = isc->try_config.sd_format->mbus_code; + fse.which = V4L2_SUBDEV_FORMAT_TRY; + + ret = v4l2_subdev_call(isc->current_subdev->sd, pad, enum_frame_size, + sd_state, &fse); + /* + * Attempt to obtain format size from subdev. If not available, + * just use the maximum ISC can receive. + */ + if (ret) { + sd_state->pads->try_crop.width = isc->max_width; + sd_state->pads->try_crop.height = isc->max_height; + } else { + sd_state->pads->try_crop.width = fse.max_width; + sd_state->pads->try_crop.height = fse.max_height; + } +} + +static int isc_try_fmt(struct isc_device *isc, struct v4l2_format *f) +{ + struct v4l2_pix_format *pixfmt = &f->fmt.pix; + unsigned int i; + + if (f->type != V4L2_BUF_TYPE_VIDEO_CAPTURE) + return -EINVAL; + + isc->try_config.fourcc = isc->controller_formats[0].fourcc; + + /* find if the format requested is supported */ + for (i = 0; i < isc->controller_formats_size; i++) + if (isc->controller_formats[i].fourcc == pixfmt->pixelformat) { + isc->try_config.fourcc = pixfmt->pixelformat; + break; + } + + isc_try_configure_rlp_dma(isc, false); + + /* Limit to Microchip ISC hardware capabilities */ + v4l_bound_align_image(&pixfmt->width, 16, isc->max_width, 0, + &pixfmt->height, 16, isc->max_height, 0, 0); + /* If we did not find the requested format, we will fallback here */ + pixfmt->pixelformat = isc->try_config.fourcc; + pixfmt->colorspace = V4L2_COLORSPACE_SRGB; + pixfmt->field = V4L2_FIELD_NONE; + + pixfmt->bytesperline = (pixfmt->width * isc->try_config.bpp_v4l2) >> 3; + pixfmt->sizeimage = ((pixfmt->width * isc->try_config.bpp) >> 3) * + pixfmt->height; + + isc->try_fmt = *f; + + return 0; +} + +static int isc_set_fmt(struct isc_device *isc, struct v4l2_format *f) +{ + isc_try_fmt(isc, f); + + /* make the try configuration active */ + isc->config = isc->try_config; + isc->fmt = isc->try_fmt; + + v4l2_dbg(1, debug, &isc->v4l2_dev, "ISC set_fmt to %.4s @%dx%d\n", + (char *)&f->fmt.pix.pixelformat, + f->fmt.pix.width, f->fmt.pix.height); + + return 0; +} + +static int isc_validate(struct isc_device *isc) +{ + int ret; + int i; + struct isc_format *sd_fmt = NULL; + struct v4l2_pix_format *pixfmt = &isc->fmt.fmt.pix; + struct v4l2_subdev_format format = { + .which = V4L2_SUBDEV_FORMAT_ACTIVE, + .pad = isc->remote_pad, + }; + struct v4l2_subdev_pad_config pad_cfg = {}; + struct v4l2_subdev_state pad_state = { + .pads = &pad_cfg, + }; + + /* Get current format from subdev */ + ret = v4l2_subdev_call(isc->current_subdev->sd, pad, get_fmt, NULL, + &format); + if (ret) + return ret; + + /* Identify the subdev's format configuration */ + for (i = 0; i < isc->formats_list_size; i++) + if (isc->formats_list[i].mbus_code == format.format.code) { + sd_fmt = &isc->formats_list[i]; + break; + } + + /* Check if the format is not supported */ + if (!sd_fmt) { + v4l2_err(&isc->v4l2_dev, + "Current subdevice is streaming a media bus code that is not supported 0x%x\n", + format.format.code); + return -EPIPE; + } + + /* At this moment we know which format the subdev will use */ + isc->try_config.sd_format = sd_fmt; + + /* If the sensor is not RAW, we can only do a direct dump */ + if (!ISC_IS_FORMAT_RAW(isc->try_config.sd_format->mbus_code)) + isc_try_configure_rlp_dma(isc, true); + + /* Limit to Microchip ISC hardware capabilities */ + v4l_bound_align_image(&format.format.width, 16, isc->max_width, 0, + &format.format.height, 16, isc->max_height, 0, 0); + + /* Check if the frame size is the same. Otherwise we may overflow */ + if (pixfmt->height != format.format.height || + pixfmt->width != format.format.width) { + v4l2_err(&isc->v4l2_dev, + "ISC not configured with the proper frame size: %dx%d\n", + format.format.width, format.format.height); + return -EPIPE; + } + + v4l2_dbg(1, debug, &isc->v4l2_dev, + "Identified subdev using format %.4s with %dx%d %d bpp\n", + (char *)&sd_fmt->fourcc, pixfmt->width, pixfmt->height, + isc->try_config.bpp); + + /* Reset and restart AWB if the subdevice changed the format */ + if (isc->try_config.sd_format && isc->config.sd_format && + isc->try_config.sd_format != isc->config.sd_format) { + isc->ctrls.hist_stat = HIST_INIT; + isc_reset_awb_ctrls(isc); + isc_update_v4l2_ctrls(isc); + } + + /* Validate formats */ + ret = isc_try_validate_formats(isc); + if (ret) + return ret; + + /* Obtain frame sizes if possible to have crop requirements ready */ + isc_try_fse(isc, &pad_state); + + /* Configure ISC pipeline for the config */ + ret = isc_try_configure_pipeline(isc); + if (ret) + return ret; + + isc->config = isc->try_config; + + v4l2_dbg(1, debug, &isc->v4l2_dev, "New ISC configuration in place\n"); + + return 0; +} + +static int isc_s_fmt_vid_cap(struct file *file, void *priv, + struct v4l2_format *f) +{ + struct isc_device *isc = video_drvdata(file); + + if (vb2_is_busy(&isc->vb2_vidq)) + return -EBUSY; + + return isc_set_fmt(isc, f); +} + +static int isc_try_fmt_vid_cap(struct file *file, void *priv, + struct v4l2_format *f) +{ + struct isc_device *isc = video_drvdata(file); + + return isc_try_fmt(isc, f); +} + +static int isc_enum_input(struct file *file, void *priv, + struct v4l2_input *inp) +{ + if (inp->index != 0) + return -EINVAL; + + inp->type = V4L2_INPUT_TYPE_CAMERA; + inp->std = 0; + strscpy(inp->name, "Camera", sizeof(inp->name)); + + return 0; +} + +static int isc_g_input(struct file *file, void *priv, unsigned int *i) +{ + *i = 0; + + return 0; +} + +static int isc_s_input(struct file *file, void *priv, unsigned int i) +{ + if (i > 0) + return -EINVAL; + + return 0; +} + +static int isc_g_parm(struct file *file, void *fh, struct v4l2_streamparm *a) +{ + struct isc_device *isc = video_drvdata(file); + + return v4l2_g_parm_cap(video_devdata(file), isc->current_subdev->sd, a); +} + +static int isc_s_parm(struct file *file, void *fh, struct v4l2_streamparm *a) +{ + struct isc_device *isc = video_drvdata(file); + + return v4l2_s_parm_cap(video_devdata(file), isc->current_subdev->sd, a); +} + +static int isc_enum_framesizes(struct file *file, void *fh, + struct v4l2_frmsizeenum *fsize) +{ + struct isc_device *isc = video_drvdata(file); + int ret = -EINVAL; + int i; + + if (fsize->index) + return -EINVAL; + + for (i = 0; i < isc->controller_formats_size; i++) + if (isc->controller_formats[i].fourcc == fsize->pixel_format) + ret = 0; + + if (ret) + return ret; + + fsize->type = V4L2_FRMSIZE_TYPE_CONTINUOUS; + + fsize->stepwise.min_width = 16; + fsize->stepwise.max_width = isc->max_width; + fsize->stepwise.min_height = 16; + fsize->stepwise.max_height = isc->max_height; + fsize->stepwise.step_width = 1; + fsize->stepwise.step_height = 1; + + return 0; +} + +static const struct v4l2_ioctl_ops isc_ioctl_ops = { + .vidioc_querycap = isc_querycap, + .vidioc_enum_fmt_vid_cap = isc_enum_fmt_vid_cap, + .vidioc_g_fmt_vid_cap = isc_g_fmt_vid_cap, + .vidioc_s_fmt_vid_cap = isc_s_fmt_vid_cap, + .vidioc_try_fmt_vid_cap = isc_try_fmt_vid_cap, + + .vidioc_enum_input = isc_enum_input, + .vidioc_g_input = isc_g_input, + .vidioc_s_input = isc_s_input, + + .vidioc_reqbufs = vb2_ioctl_reqbufs, + .vidioc_querybuf = vb2_ioctl_querybuf, + .vidioc_qbuf = vb2_ioctl_qbuf, + .vidioc_expbuf = vb2_ioctl_expbuf, + .vidioc_dqbuf = vb2_ioctl_dqbuf, + .vidioc_create_bufs = vb2_ioctl_create_bufs, + .vidioc_prepare_buf = vb2_ioctl_prepare_buf, + .vidioc_streamon = vb2_ioctl_streamon, + .vidioc_streamoff = vb2_ioctl_streamoff, + + .vidioc_g_parm = isc_g_parm, + .vidioc_s_parm = isc_s_parm, + .vidioc_enum_framesizes = isc_enum_framesizes, + + .vidioc_log_status = v4l2_ctrl_log_status, + .vidioc_subscribe_event = v4l2_ctrl_subscribe_event, + .vidioc_unsubscribe_event = v4l2_event_unsubscribe, +}; + +static int isc_open(struct file *file) +{ + struct isc_device *isc = video_drvdata(file); + struct v4l2_subdev *sd = isc->current_subdev->sd; + int ret; + + if (mutex_lock_interruptible(&isc->lock)) + return -ERESTARTSYS; + + ret = v4l2_fh_open(file); + if (ret < 0) + goto unlock; + + if (!v4l2_fh_is_singular_file(file)) + goto unlock; + + ret = v4l2_subdev_call(sd, core, s_power, 1); + if (ret < 0 && ret != -ENOIOCTLCMD) { + v4l2_fh_release(file); + goto unlock; + } + + ret = isc_set_fmt(isc, &isc->fmt); + if (ret) { + v4l2_subdev_call(sd, core, s_power, 0); + v4l2_fh_release(file); + } + +unlock: + mutex_unlock(&isc->lock); + return ret; +} + +static int isc_release(struct file *file) +{ + struct isc_device *isc = video_drvdata(file); + struct v4l2_subdev *sd = isc->current_subdev->sd; + bool fh_singular; + int ret; + + mutex_lock(&isc->lock); + + fh_singular = v4l2_fh_is_singular_file(file); + + ret = _vb2_fop_release(file, NULL); + + if (fh_singular) + v4l2_subdev_call(sd, core, s_power, 0); + + mutex_unlock(&isc->lock); + + return ret; +} + +static const struct v4l2_file_operations isc_fops = { + .owner = THIS_MODULE, + .open = isc_open, + .release = isc_release, + .unlocked_ioctl = video_ioctl2, + .read = vb2_fop_read, + .mmap = vb2_fop_mmap, + .poll = vb2_fop_poll, +}; + +irqreturn_t microchip_isc_interrupt(int irq, void *dev_id) +{ + struct isc_device *isc = (struct isc_device *)dev_id; + struct regmap *regmap = isc->regmap; + u32 isc_intsr, isc_intmask, pending; + irqreturn_t ret = IRQ_NONE; + + regmap_read(regmap, ISC_INTSR, &isc_intsr); + regmap_read(regmap, ISC_INTMASK, &isc_intmask); + + pending = isc_intsr & isc_intmask; + + if (likely(pending & ISC_INT_DDONE)) { + spin_lock(&isc->dma_queue_lock); + if (isc->cur_frm) { + struct vb2_v4l2_buffer *vbuf = &isc->cur_frm->vb; + struct vb2_buffer *vb = &vbuf->vb2_buf; + + vb->timestamp = ktime_get_ns(); + vbuf->sequence = isc->sequence++; + vb2_buffer_done(vb, VB2_BUF_STATE_DONE); + isc->cur_frm = NULL; + } + + if (!list_empty(&isc->dma_queue) && !isc->stop) { + isc->cur_frm = list_first_entry(&isc->dma_queue, + struct isc_buffer, list); + list_del(&isc->cur_frm->list); + + isc_start_dma(isc); + } + + if (isc->stop) + complete(&isc->comp); + + ret = IRQ_HANDLED; + spin_unlock(&isc->dma_queue_lock); + } + + if (pending & ISC_INT_HISDONE) { + schedule_work(&isc->awb_work); + ret = IRQ_HANDLED; + } + + return ret; +} +EXPORT_SYMBOL_GPL(microchip_isc_interrupt); + +static void isc_hist_count(struct isc_device *isc, u32 *min, u32 *max) +{ + struct regmap *regmap = isc->regmap; + struct isc_ctrls *ctrls = &isc->ctrls; + u32 *hist_count = &ctrls->hist_count[ctrls->hist_id]; + u32 *hist_entry = &ctrls->hist_entry[0]; + u32 i; + + *min = 0; + *max = HIST_ENTRIES; + + regmap_bulk_read(regmap, ISC_HIS_ENTRY + isc->offsets.his_entry, + hist_entry, HIST_ENTRIES); + + *hist_count = 0; + /* + * we deliberately ignore the end of the histogram, + * the most white pixels + */ + for (i = 1; i < HIST_ENTRIES; i++) { + if (*hist_entry && !*min) + *min = i; + if (*hist_entry) + *max = i; + *hist_count += i * (*hist_entry++); + } + + if (!*min) + *min = 1; + + v4l2_dbg(1, debug, &isc->v4l2_dev, + "isc wb: hist_id %u, hist_count %u", + ctrls->hist_id, *hist_count); +} + +static void isc_wb_update(struct isc_ctrls *ctrls) +{ + struct isc_device *isc = container_of(ctrls, struct isc_device, ctrls); + u32 *hist_count = &ctrls->hist_count[0]; + u32 c, offset[4]; + u64 avg = 0; + /* We compute two gains, stretch gain and grey world gain */ + u32 s_gain[4], gw_gain[4]; + + /* + * According to Grey World, we need to set gains for R/B to normalize + * them towards the green channel. + * Thus we want to keep Green as fixed and adjust only Red/Blue + * Compute the average of the both green channels first + */ + avg = (u64)hist_count[ISC_HIS_CFG_MODE_GR] + + (u64)hist_count[ISC_HIS_CFG_MODE_GB]; + avg >>= 1; + + v4l2_dbg(1, debug, &isc->v4l2_dev, + "isc wb: green components average %llu\n", avg); + + /* Green histogram is null, nothing to do */ + if (!avg) + return; + + for (c = ISC_HIS_CFG_MODE_GR; c <= ISC_HIS_CFG_MODE_B; c++) { + /* + * the color offset is the minimum value of the histogram. + * we stretch this color to the full range by substracting + * this value from the color component. + */ + offset[c] = ctrls->hist_minmax[c][HIST_MIN_INDEX]; + /* + * The offset is always at least 1. If the offset is 1, we do + * not need to adjust it, so our result must be zero. + * the offset is computed in a histogram on 9 bits (0..512) + * but the offset in register is based on + * 12 bits pipeline (0..4096). + * we need to shift with the 3 bits that the histogram is + * ignoring + */ + ctrls->offset[c] = (offset[c] - 1) << 3; + + /* + * the offset is then taken and converted to 2's complements, + * and must be negative, as we subtract this value from the + * color components + */ + ctrls->offset[c] = -ctrls->offset[c]; + + /* + * the stretch gain is the total number of histogram bins + * divided by the actual range of color component (Max - Min) + * If we compute gain like this, the actual color component + * will be stretched to the full histogram. + * We need to shift 9 bits for precision, we have 9 bits for + * decimals + */ + s_gain[c] = (HIST_ENTRIES << 9) / + (ctrls->hist_minmax[c][HIST_MAX_INDEX] - + ctrls->hist_minmax[c][HIST_MIN_INDEX] + 1); + + /* + * Now we have to compute the gain w.r.t. the average. + * Add/lose gain to the component towards the average. + * If it happens that the component is zero, use the + * fixed point value : 1.0 gain. + */ + if (hist_count[c]) + gw_gain[c] = div_u64(avg << 9, hist_count[c]); + else + gw_gain[c] = 1 << 9; + + v4l2_dbg(1, debug, &isc->v4l2_dev, + "isc wb: component %d, s_gain %u, gw_gain %u\n", + c, s_gain[c], gw_gain[c]); + /* multiply both gains and adjust for decimals */ + ctrls->gain[c] = s_gain[c] * gw_gain[c]; + ctrls->gain[c] >>= 9; + + /* make sure we are not out of range */ + ctrls->gain[c] = clamp_val(ctrls->gain[c], 0, GENMASK(12, 0)); + + v4l2_dbg(1, debug, &isc->v4l2_dev, + "isc wb: component %d, final gain %u\n", + c, ctrls->gain[c]); + } +} + +static void isc_awb_work(struct work_struct *w) +{ + struct isc_device *isc = + container_of(w, struct isc_device, awb_work); + struct regmap *regmap = isc->regmap; + struct isc_ctrls *ctrls = &isc->ctrls; + u32 hist_id = ctrls->hist_id; + u32 baysel; + unsigned long flags; + u32 min, max; + int ret; + + if (ctrls->hist_stat != HIST_ENABLED) + return; + + isc_hist_count(isc, &min, &max); + + v4l2_dbg(1, debug, &isc->v4l2_dev, + "isc wb mode %d: hist min %u , max %u\n", hist_id, min, max); + + ctrls->hist_minmax[hist_id][HIST_MIN_INDEX] = min; + ctrls->hist_minmax[hist_id][HIST_MAX_INDEX] = max; + + if (hist_id != ISC_HIS_CFG_MODE_B) { + hist_id++; + } else { + isc_wb_update(ctrls); + hist_id = ISC_HIS_CFG_MODE_GR; + } + + ctrls->hist_id = hist_id; + baysel = isc->config.sd_format->cfa_baycfg << ISC_HIS_CFG_BAYSEL_SHIFT; + + ret = pm_runtime_resume_and_get(isc->dev); + if (ret < 0) + return; + + /* + * only update if we have all the required histograms and controls + * if awb has been disabled, we need to reset registers as well. + */ + if (hist_id == ISC_HIS_CFG_MODE_GR || ctrls->awb == ISC_WB_NONE) { + /* + * It may happen that DMA Done IRQ will trigger while we are + * updating white balance registers here. + * In that case, only parts of the controls have been updated. + * We can avoid that by locking the section. + */ + spin_lock_irqsave(&isc->awb_lock, flags); + isc_update_awb_ctrls(isc); + spin_unlock_irqrestore(&isc->awb_lock, flags); + + /* + * if we are doing just the one time white balance adjustment, + * we are basically done. + */ + if (ctrls->awb == ISC_WB_ONETIME) { + v4l2_info(&isc->v4l2_dev, + "Completed one time white-balance adjustment.\n"); + /* update the v4l2 controls values */ + isc_update_v4l2_ctrls(isc); + ctrls->awb = ISC_WB_NONE; + } + } + regmap_write(regmap, ISC_HIS_CFG + isc->offsets.his, + hist_id | baysel | ISC_HIS_CFG_RAR); + + /* + * We have to make sure the streaming has not stopped meanwhile. + * ISC requires a frame to clock the internal profile update. + * To avoid issues, lock the sequence with a mutex + */ + mutex_lock(&isc->awb_mutex); + + /* streaming is not active anymore */ + if (isc->stop) { + mutex_unlock(&isc->awb_mutex); + return; + } + + isc_update_profile(isc); + + mutex_unlock(&isc->awb_mutex); + + /* if awb has been disabled, we don't need to start another histogram */ + if (ctrls->awb) + regmap_write(regmap, ISC_CTRLEN, ISC_CTRL_HISREQ); + + pm_runtime_put_sync(isc->dev); +} + +static int isc_s_ctrl(struct v4l2_ctrl *ctrl) +{ + struct isc_device *isc = container_of(ctrl->handler, + struct isc_device, ctrls.handler); + struct isc_ctrls *ctrls = &isc->ctrls; + + if (ctrl->flags & V4L2_CTRL_FLAG_INACTIVE) + return 0; + + switch (ctrl->id) { + case V4L2_CID_BRIGHTNESS: + ctrls->brightness = ctrl->val & ISC_CBC_BRIGHT_MASK; + break; + case V4L2_CID_CONTRAST: + ctrls->contrast = ctrl->val & ISC_CBC_CONTRAST_MASK; + break; + case V4L2_CID_GAMMA: + ctrls->gamma_index = ctrl->val; + break; + default: + return -EINVAL; + } + + return 0; +} + +static const struct v4l2_ctrl_ops isc_ctrl_ops = { + .s_ctrl = isc_s_ctrl, +}; + +static int isc_s_awb_ctrl(struct v4l2_ctrl *ctrl) +{ + struct isc_device *isc = container_of(ctrl->handler, + struct isc_device, ctrls.handler); + struct isc_ctrls *ctrls = &isc->ctrls; + + if (ctrl->flags & V4L2_CTRL_FLAG_INACTIVE) + return 0; + + switch (ctrl->id) { + case V4L2_CID_AUTO_WHITE_BALANCE: + if (ctrl->val == 1) + ctrls->awb = ISC_WB_AUTO; + else + ctrls->awb = ISC_WB_NONE; + + /* configure the controls with new values from v4l2 */ + if (ctrl->cluster[ISC_CTRL_R_GAIN]->is_new) + ctrls->gain[ISC_HIS_CFG_MODE_R] = isc->r_gain_ctrl->val; + if (ctrl->cluster[ISC_CTRL_B_GAIN]->is_new) + ctrls->gain[ISC_HIS_CFG_MODE_B] = isc->b_gain_ctrl->val; + if (ctrl->cluster[ISC_CTRL_GR_GAIN]->is_new) + ctrls->gain[ISC_HIS_CFG_MODE_GR] = isc->gr_gain_ctrl->val; + if (ctrl->cluster[ISC_CTRL_GB_GAIN]->is_new) + ctrls->gain[ISC_HIS_CFG_MODE_GB] = isc->gb_gain_ctrl->val; + + if (ctrl->cluster[ISC_CTRL_R_OFF]->is_new) + ctrls->offset[ISC_HIS_CFG_MODE_R] = isc->r_off_ctrl->val; + if (ctrl->cluster[ISC_CTRL_B_OFF]->is_new) + ctrls->offset[ISC_HIS_CFG_MODE_B] = isc->b_off_ctrl->val; + if (ctrl->cluster[ISC_CTRL_GR_OFF]->is_new) + ctrls->offset[ISC_HIS_CFG_MODE_GR] = isc->gr_off_ctrl->val; + if (ctrl->cluster[ISC_CTRL_GB_OFF]->is_new) + ctrls->offset[ISC_HIS_CFG_MODE_GB] = isc->gb_off_ctrl->val; + + isc_update_awb_ctrls(isc); + + mutex_lock(&isc->awb_mutex); + if (vb2_is_streaming(&isc->vb2_vidq)) { + /* + * If we are streaming, we can update profile to + * have the new settings in place. + */ + isc_update_profile(isc); + } else { + /* + * The auto cluster will activate automatically this + * control. This has to be deactivated when not + * streaming. + */ + v4l2_ctrl_activate(isc->do_wb_ctrl, false); + } + mutex_unlock(&isc->awb_mutex); + + /* if we have autowhitebalance on, start histogram procedure */ + if (ctrls->awb == ISC_WB_AUTO && + vb2_is_streaming(&isc->vb2_vidq) && + ISC_IS_FORMAT_RAW(isc->config.sd_format->mbus_code)) + isc_set_histogram(isc, true); + + /* + * for one time whitebalance adjustment, check the button, + * if it's pressed, perform the one time operation. + */ + if (ctrls->awb == ISC_WB_NONE && + ctrl->cluster[ISC_CTRL_DO_WB]->is_new && + !(ctrl->cluster[ISC_CTRL_DO_WB]->flags & + V4L2_CTRL_FLAG_INACTIVE)) { + ctrls->awb = ISC_WB_ONETIME; + isc_set_histogram(isc, true); + v4l2_dbg(1, debug, &isc->v4l2_dev, + "One time white-balance started.\n"); + } + return 0; + } + return 0; +} + +static int isc_g_volatile_awb_ctrl(struct v4l2_ctrl *ctrl) +{ + struct isc_device *isc = container_of(ctrl->handler, + struct isc_device, ctrls.handler); + struct isc_ctrls *ctrls = &isc->ctrls; + + switch (ctrl->id) { + /* being a cluster, this id will be called for every control */ + case V4L2_CID_AUTO_WHITE_BALANCE: + ctrl->cluster[ISC_CTRL_R_GAIN]->val = + ctrls->gain[ISC_HIS_CFG_MODE_R]; + ctrl->cluster[ISC_CTRL_B_GAIN]->val = + ctrls->gain[ISC_HIS_CFG_MODE_B]; + ctrl->cluster[ISC_CTRL_GR_GAIN]->val = + ctrls->gain[ISC_HIS_CFG_MODE_GR]; + ctrl->cluster[ISC_CTRL_GB_GAIN]->val = + ctrls->gain[ISC_HIS_CFG_MODE_GB]; + + ctrl->cluster[ISC_CTRL_R_OFF]->val = + ctrls->offset[ISC_HIS_CFG_MODE_R]; + ctrl->cluster[ISC_CTRL_B_OFF]->val = + ctrls->offset[ISC_HIS_CFG_MODE_B]; + ctrl->cluster[ISC_CTRL_GR_OFF]->val = + ctrls->offset[ISC_HIS_CFG_MODE_GR]; + ctrl->cluster[ISC_CTRL_GB_OFF]->val = + ctrls->offset[ISC_HIS_CFG_MODE_GB]; + break; + } + return 0; +} + +static const struct v4l2_ctrl_ops isc_awb_ops = { + .s_ctrl = isc_s_awb_ctrl, + .g_volatile_ctrl = isc_g_volatile_awb_ctrl, +}; + +#define ISC_CTRL_OFF(_name, _id, _name_str) \ + static const struct v4l2_ctrl_config _name = { \ + .ops = &isc_awb_ops, \ + .id = _id, \ + .name = _name_str, \ + .type = V4L2_CTRL_TYPE_INTEGER, \ + .flags = V4L2_CTRL_FLAG_SLIDER, \ + .min = -4095, \ + .max = 4095, \ + .step = 1, \ + .def = 0, \ + } + +ISC_CTRL_OFF(isc_r_off_ctrl, ISC_CID_R_OFFSET, "Red Component Offset"); +ISC_CTRL_OFF(isc_b_off_ctrl, ISC_CID_B_OFFSET, "Blue Component Offset"); +ISC_CTRL_OFF(isc_gr_off_ctrl, ISC_CID_GR_OFFSET, "Green Red Component Offset"); +ISC_CTRL_OFF(isc_gb_off_ctrl, ISC_CID_GB_OFFSET, "Green Blue Component Offset"); + +#define ISC_CTRL_GAIN(_name, _id, _name_str) \ + static const struct v4l2_ctrl_config _name = { \ + .ops = &isc_awb_ops, \ + .id = _id, \ + .name = _name_str, \ + .type = V4L2_CTRL_TYPE_INTEGER, \ + .flags = V4L2_CTRL_FLAG_SLIDER, \ + .min = 0, \ + .max = 8191, \ + .step = 1, \ + .def = 512, \ + } + +ISC_CTRL_GAIN(isc_r_gain_ctrl, ISC_CID_R_GAIN, "Red Component Gain"); +ISC_CTRL_GAIN(isc_b_gain_ctrl, ISC_CID_B_GAIN, "Blue Component Gain"); +ISC_CTRL_GAIN(isc_gr_gain_ctrl, ISC_CID_GR_GAIN, "Green Red Component Gain"); +ISC_CTRL_GAIN(isc_gb_gain_ctrl, ISC_CID_GB_GAIN, "Green Blue Component Gain"); + +static int isc_ctrl_init(struct isc_device *isc) +{ + const struct v4l2_ctrl_ops *ops = &isc_ctrl_ops; + struct isc_ctrls *ctrls = &isc->ctrls; + struct v4l2_ctrl_handler *hdl = &ctrls->handler; + int ret; + + ctrls->hist_stat = HIST_INIT; + isc_reset_awb_ctrls(isc); + + ret = v4l2_ctrl_handler_init(hdl, 13); + if (ret < 0) + return ret; + + /* Initialize product specific controls. For example, contrast */ + isc->config_ctrls(isc, ops); + + ctrls->brightness = 0; + + v4l2_ctrl_new_std(hdl, ops, V4L2_CID_BRIGHTNESS, -1024, 1023, 1, 0); + v4l2_ctrl_new_std(hdl, ops, V4L2_CID_GAMMA, 0, isc->gamma_max, 1, + isc->gamma_max); + isc->awb_ctrl = v4l2_ctrl_new_std(hdl, &isc_awb_ops, + V4L2_CID_AUTO_WHITE_BALANCE, + 0, 1, 1, 1); + + /* do_white_balance is a button, so min,max,step,default are ignored */ + isc->do_wb_ctrl = v4l2_ctrl_new_std(hdl, &isc_awb_ops, + V4L2_CID_DO_WHITE_BALANCE, + 0, 0, 0, 0); + + if (!isc->do_wb_ctrl) { + ret = hdl->error; + v4l2_ctrl_handler_free(hdl); + return ret; + } + + v4l2_ctrl_activate(isc->do_wb_ctrl, false); + + isc->r_gain_ctrl = v4l2_ctrl_new_custom(hdl, &isc_r_gain_ctrl, NULL); + isc->b_gain_ctrl = v4l2_ctrl_new_custom(hdl, &isc_b_gain_ctrl, NULL); + isc->gr_gain_ctrl = v4l2_ctrl_new_custom(hdl, &isc_gr_gain_ctrl, NULL); + isc->gb_gain_ctrl = v4l2_ctrl_new_custom(hdl, &isc_gb_gain_ctrl, NULL); + isc->r_off_ctrl = v4l2_ctrl_new_custom(hdl, &isc_r_off_ctrl, NULL); + isc->b_off_ctrl = v4l2_ctrl_new_custom(hdl, &isc_b_off_ctrl, NULL); + isc->gr_off_ctrl = v4l2_ctrl_new_custom(hdl, &isc_gr_off_ctrl, NULL); + isc->gb_off_ctrl = v4l2_ctrl_new_custom(hdl, &isc_gb_off_ctrl, NULL); + + /* + * The cluster is in auto mode with autowhitebalance enabled + * and manual mode otherwise. + */ + v4l2_ctrl_auto_cluster(10, &isc->awb_ctrl, 0, true); + + v4l2_ctrl_handler_setup(hdl); + + return 0; +} + +static int isc_async_bound(struct v4l2_async_notifier *notifier, + struct v4l2_subdev *subdev, + struct v4l2_async_subdev *asd) +{ + struct isc_device *isc = container_of(notifier->v4l2_dev, + struct isc_device, v4l2_dev); + struct isc_subdev_entity *subdev_entity = + container_of(notifier, struct isc_subdev_entity, notifier); + int pad; + + if (video_is_registered(&isc->video_dev)) { + v4l2_err(&isc->v4l2_dev, "only supports one sub-device.\n"); + return -EBUSY; + } + + subdev_entity->sd = subdev; + + pad = media_entity_get_fwnode_pad(&subdev->entity, asd->match.fwnode, + MEDIA_PAD_FL_SOURCE); + if (pad < 0) { + v4l2_err(&isc->v4l2_dev, "failed to find pad for %s\n", + subdev->name); + return pad; + } + + isc->remote_pad = pad; + + return 0; +} + +static void isc_async_unbind(struct v4l2_async_notifier *notifier, + struct v4l2_subdev *subdev, + struct v4l2_async_subdev *asd) +{ + struct isc_device *isc = container_of(notifier->v4l2_dev, + struct isc_device, v4l2_dev); + mutex_destroy(&isc->awb_mutex); + cancel_work_sync(&isc->awb_work); + video_unregister_device(&isc->video_dev); + v4l2_ctrl_handler_free(&isc->ctrls.handler); +} + +struct isc_format *isc_find_format_by_code(struct isc_device *isc, + unsigned int code, int *index) +{ + struct isc_format *fmt = &isc->formats_list[0]; + unsigned int i; + + for (i = 0; i < isc->formats_list_size; i++) { + if (fmt->mbus_code == code) { + *index = i; + return fmt; + } + + fmt++; + } + + return NULL; +} +EXPORT_SYMBOL_GPL(isc_find_format_by_code); + +static int isc_set_default_fmt(struct isc_device *isc) +{ + struct v4l2_format f = { + .type = V4L2_BUF_TYPE_VIDEO_CAPTURE, + .fmt.pix = { + .width = VGA_WIDTH, + .height = VGA_HEIGHT, + .field = V4L2_FIELD_NONE, + .pixelformat = isc->controller_formats[0].fourcc, + }, + }; + int ret; + + ret = isc_try_fmt(isc, &f); + if (ret) + return ret; + + isc->fmt = f; + return 0; +} + +static int isc_async_complete(struct v4l2_async_notifier *notifier) +{ + struct isc_device *isc = container_of(notifier->v4l2_dev, + struct isc_device, v4l2_dev); + struct video_device *vdev = &isc->video_dev; + struct vb2_queue *q = &isc->vb2_vidq; + int ret = 0; + + INIT_WORK(&isc->awb_work, isc_awb_work); + + ret = v4l2_device_register_subdev_nodes(&isc->v4l2_dev); + if (ret < 0) { + v4l2_err(&isc->v4l2_dev, "Failed to register subdev nodes\n"); + return ret; + } + + isc->current_subdev = container_of(notifier, + struct isc_subdev_entity, notifier); + mutex_init(&isc->lock); + mutex_init(&isc->awb_mutex); + + init_completion(&isc->comp); + + /* Initialize videobuf2 queue */ + q->type = V4L2_BUF_TYPE_VIDEO_CAPTURE; + q->io_modes = VB2_MMAP | VB2_DMABUF | VB2_READ; + q->drv_priv = isc; + q->buf_struct_size = sizeof(struct isc_buffer); + q->ops = &isc_vb2_ops; + q->mem_ops = &vb2_dma_contig_memops; + q->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC; + q->lock = &isc->lock; + q->min_buffers_needed = 1; + q->dev = isc->dev; + + ret = vb2_queue_init(q); + if (ret < 0) { + v4l2_err(&isc->v4l2_dev, + "vb2_queue_init() failed: %d\n", ret); + goto isc_async_complete_err; + } + + /* Init video dma queues */ + INIT_LIST_HEAD(&isc->dma_queue); + spin_lock_init(&isc->dma_queue_lock); + spin_lock_init(&isc->awb_lock); + + ret = isc_set_default_fmt(isc); + if (ret) { + v4l2_err(&isc->v4l2_dev, "Could not set default format\n"); + goto isc_async_complete_err; + } + + ret = isc_ctrl_init(isc); + if (ret) { + v4l2_err(&isc->v4l2_dev, "Init isc ctrols failed: %d\n", ret); + goto isc_async_complete_err; + } + + /* Register video device */ + strscpy(vdev->name, KBUILD_MODNAME, sizeof(vdev->name)); + vdev->release = video_device_release_empty; + vdev->fops = &isc_fops; + vdev->ioctl_ops = &isc_ioctl_ops; + vdev->v4l2_dev = &isc->v4l2_dev; + vdev->vfl_dir = VFL_DIR_RX; + vdev->queue = q; + vdev->lock = &isc->lock; + vdev->ctrl_handler = &isc->ctrls.handler; + vdev->device_caps = V4L2_CAP_STREAMING | V4L2_CAP_VIDEO_CAPTURE | + V4L2_CAP_IO_MC; + video_set_drvdata(vdev, isc); + + ret = video_register_device(vdev, VFL_TYPE_VIDEO, -1); + if (ret < 0) { + v4l2_err(&isc->v4l2_dev, + "video_register_device failed: %d\n", ret); + goto isc_async_complete_err; + } + + ret = isc_scaler_link(isc); + if (ret < 0) + goto isc_async_complete_unregister_device; + + ret = media_device_register(&isc->mdev); + if (ret < 0) + goto isc_async_complete_unregister_device; + + return 0; + +isc_async_complete_unregister_device: + video_unregister_device(vdev); + +isc_async_complete_err: + mutex_destroy(&isc->awb_mutex); + mutex_destroy(&isc->lock); + return ret; +} + +const struct v4l2_async_notifier_operations microchip_isc_async_ops = { + .bound = isc_async_bound, + .unbind = isc_async_unbind, + .complete = isc_async_complete, +}; +EXPORT_SYMBOL_GPL(microchip_isc_async_ops); + +void microchip_isc_subdev_cleanup(struct isc_device *isc) +{ + struct isc_subdev_entity *subdev_entity; + + list_for_each_entry(subdev_entity, &isc->subdev_entities, list) { + v4l2_async_nf_unregister(&subdev_entity->notifier); + v4l2_async_nf_cleanup(&subdev_entity->notifier); + } + + INIT_LIST_HEAD(&isc->subdev_entities); +} +EXPORT_SYMBOL_GPL(microchip_isc_subdev_cleanup); + +int microchip_isc_pipeline_init(struct isc_device *isc) +{ + struct device *dev = isc->dev; + struct regmap *regmap = isc->regmap; + struct regmap_field *regs; + unsigned int i; + + /* + * DPCEN-->GDCEN-->BLCEN-->WB-->CFA-->CC--> + * GAM-->VHXS-->CSC-->CBC-->SUB422-->SUB420 + */ + const struct reg_field regfields[ISC_PIPE_LINE_NODE_NUM] = { + REG_FIELD(ISC_DPC_CTRL, 0, 0), + REG_FIELD(ISC_DPC_CTRL, 1, 1), + REG_FIELD(ISC_DPC_CTRL, 2, 2), + REG_FIELD(ISC_WB_CTRL, 0, 0), + REG_FIELD(ISC_CFA_CTRL, 0, 0), + REG_FIELD(ISC_CC_CTRL, 0, 0), + REG_FIELD(ISC_GAM_CTRL, 0, 0), + REG_FIELD(ISC_GAM_CTRL, 1, 1), + REG_FIELD(ISC_GAM_CTRL, 2, 2), + REG_FIELD(ISC_GAM_CTRL, 3, 3), + REG_FIELD(ISC_VHXS_CTRL, 0, 0), + REG_FIELD(ISC_CSC_CTRL + isc->offsets.csc, 0, 0), + REG_FIELD(ISC_CBC_CTRL + isc->offsets.cbc, 0, 0), + REG_FIELD(ISC_SUB422_CTRL + isc->offsets.sub422, 0, 0), + REG_FIELD(ISC_SUB420_CTRL + isc->offsets.sub420, 0, 0), + }; + + for (i = 0; i < ISC_PIPE_LINE_NODE_NUM; i++) { + regs = devm_regmap_field_alloc(dev, regmap, regfields[i]); + if (IS_ERR(regs)) + return PTR_ERR(regs); + + isc->pipeline[i] = regs; + } + + return 0; +} +EXPORT_SYMBOL_GPL(microchip_isc_pipeline_init); + +static int isc_link_validate(struct media_link *link) +{ + struct video_device *vdev = + media_entity_to_video_device(link->sink->entity); + struct isc_device *isc = video_get_drvdata(vdev); + int ret; + + ret = v4l2_subdev_link_validate(link); + if (ret) + return ret; + + return isc_validate(isc); +} + +static const struct media_entity_operations isc_entity_operations = { + .link_validate = isc_link_validate, +}; + +int isc_mc_init(struct isc_device *isc, u32 ver) +{ + const struct of_device_id *match; + int ret; + + isc->video_dev.entity.function = MEDIA_ENT_F_IO_V4L; + isc->video_dev.entity.flags = MEDIA_ENT_FL_DEFAULT; + isc->video_dev.entity.ops = &isc_entity_operations; + + isc->pads[ISC_PAD_SINK].flags = MEDIA_PAD_FL_SINK; + + ret = media_entity_pads_init(&isc->video_dev.entity, ISC_PADS_NUM, + isc->pads); + if (ret < 0) { + dev_err(isc->dev, "media entity init failed\n"); + return ret; + } + + isc->mdev.dev = isc->dev; + + match = of_match_node(isc->dev->driver->of_match_table, + isc->dev->of_node); + + strscpy(isc->mdev.driver_name, KBUILD_MODNAME, + sizeof(isc->mdev.driver_name)); + strscpy(isc->mdev.model, match->compatible, sizeof(isc->mdev.model)); + snprintf(isc->mdev.bus_info, sizeof(isc->mdev.bus_info), "platform:%s", + isc->v4l2_dev.name); + isc->mdev.hw_revision = ver; + + media_device_init(&isc->mdev); + + isc->v4l2_dev.mdev = &isc->mdev; + + return isc_scaler_init(isc); +} +EXPORT_SYMBOL_GPL(isc_mc_init); + +void isc_mc_cleanup(struct isc_device *isc) +{ + media_entity_cleanup(&isc->video_dev.entity); + media_device_cleanup(&isc->mdev); +} +EXPORT_SYMBOL_GPL(isc_mc_cleanup); + +/* regmap configuration */ +#define MICROCHIP_ISC_REG_MAX 0xd5c +const struct regmap_config microchip_isc_regmap_config = { + .reg_bits = 32, + .reg_stride = 4, + .val_bits = 32, + .max_register = MICROCHIP_ISC_REG_MAX, +}; +EXPORT_SYMBOL_GPL(microchip_isc_regmap_config); + +MODULE_AUTHOR("Songjun Wu"); +MODULE_AUTHOR("Eugen Hristev"); +MODULE_DESCRIPTION("Microchip ISC common code base"); +MODULE_LICENSE("GPL v2"); diff --git a/drivers/media/platform/microchip/microchip-isc-clk.c b/drivers/media/platform/microchip/microchip-isc-clk.c new file mode 100644 index 000000000000..24358d804e75 --- /dev/null +++ b/drivers/media/platform/microchip/microchip-isc-clk.c @@ -0,0 +1,311 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Microchip Image Sensor Controller (ISC) common clock driver setup + * + * Copyright (C) 2016 Microchip Technology, Inc. + * + * Author: Songjun Wu + * Author: Eugen Hristev <[email protected]> + * + */ +#include <linux/clk.h> +#include <linux/clkdev.h> +#include <linux/clk-provider.h> +#include <linux/pm_runtime.h> +#include <linux/regmap.h> + +#include "microchip-isc-regs.h" +#include "microchip-isc.h" + +static int isc_wait_clk_stable(struct clk_hw *hw) +{ + struct isc_clk *isc_clk = to_isc_clk(hw); + struct regmap *regmap = isc_clk->regmap; + unsigned long timeout = jiffies + usecs_to_jiffies(1000); + unsigned int status; + + while (time_before(jiffies, timeout)) { + regmap_read(regmap, ISC_CLKSR, &status); + if (!(status & ISC_CLKSR_SIP)) + return 0; + + usleep_range(10, 250); + } + + return -ETIMEDOUT; +} + +static int isc_clk_prepare(struct clk_hw *hw) +{ + struct isc_clk *isc_clk = to_isc_clk(hw); + int ret; + + ret = pm_runtime_resume_and_get(isc_clk->dev); + if (ret < 0) + return ret; + + return isc_wait_clk_stable(hw); +} + +static void isc_clk_unprepare(struct clk_hw *hw) +{ + struct isc_clk *isc_clk = to_isc_clk(hw); + + isc_wait_clk_stable(hw); + + pm_runtime_put_sync(isc_clk->dev); +} + +static int isc_clk_enable(struct clk_hw *hw) +{ + struct isc_clk *isc_clk = to_isc_clk(hw); + u32 id = isc_clk->id; + struct regmap *regmap = isc_clk->regmap; + unsigned long flags; + unsigned int status; + + dev_dbg(isc_clk->dev, "ISC CLK: %s, id = %d, div = %d, parent id = %d\n", + __func__, id, isc_clk->div, isc_clk->parent_id); + + spin_lock_irqsave(&isc_clk->lock, flags); + regmap_update_bits(regmap, ISC_CLKCFG, + ISC_CLKCFG_DIV_MASK(id) | ISC_CLKCFG_SEL_MASK(id), + (isc_clk->div << ISC_CLKCFG_DIV_SHIFT(id)) | + (isc_clk->parent_id << ISC_CLKCFG_SEL_SHIFT(id))); + + regmap_write(regmap, ISC_CLKEN, ISC_CLK(id)); + spin_unlock_irqrestore(&isc_clk->lock, flags); + + regmap_read(regmap, ISC_CLKSR, &status); + if (status & ISC_CLK(id)) + return 0; + else + return -EINVAL; +} + +static void isc_clk_disable(struct clk_hw *hw) +{ + struct isc_clk *isc_clk = to_isc_clk(hw); + u32 id = isc_clk->id; + unsigned long flags; + + spin_lock_irqsave(&isc_clk->lock, flags); + regmap_write(isc_clk->regmap, ISC_CLKDIS, ISC_CLK(id)); + spin_unlock_irqrestore(&isc_clk->lock, flags); +} + +static int isc_clk_is_enabled(struct clk_hw *hw) +{ + struct isc_clk *isc_clk = to_isc_clk(hw); + u32 status; + int ret; + + ret = pm_runtime_resume_and_get(isc_clk->dev); + if (ret < 0) + return 0; + + regmap_read(isc_clk->regmap, ISC_CLKSR, &status); + + pm_runtime_put_sync(isc_clk->dev); + + return status & ISC_CLK(isc_clk->id) ? 1 : 0; +} + +static unsigned long +isc_clk_recalc_rate(struct clk_hw *hw, unsigned long parent_rate) +{ + struct isc_clk *isc_clk = to_isc_clk(hw); + + return DIV_ROUND_CLOSEST(parent_rate, isc_clk->div + 1); +} + +static int isc_clk_determine_rate(struct clk_hw *hw, + struct clk_rate_request *req) +{ + struct isc_clk *isc_clk = to_isc_clk(hw); + long best_rate = -EINVAL; + int best_diff = -1; + unsigned int i, div; + + for (i = 0; i < clk_hw_get_num_parents(hw); i++) { + struct clk_hw *parent; + unsigned long parent_rate; + + parent = clk_hw_get_parent_by_index(hw, i); + if (!parent) + continue; + + parent_rate = clk_hw_get_rate(parent); + if (!parent_rate) + continue; + + for (div = 1; div < ISC_CLK_MAX_DIV + 2; div++) { + unsigned long rate; + int diff; + + rate = DIV_ROUND_CLOSEST(parent_rate, div); + diff = abs(req->rate - rate); + + if (best_diff < 0 || best_diff > diff) { + best_rate = rate; + best_diff = diff; + req->best_parent_rate = parent_rate; + req->best_parent_hw = parent; + } + + if (!best_diff || rate < req->rate) + break; + } + + if (!best_diff) + break; + } + + dev_dbg(isc_clk->dev, + "ISC CLK: %s, best_rate = %ld, parent clk: %s @ %ld\n", + __func__, best_rate, + __clk_get_name((req->best_parent_hw)->clk), + req->best_parent_rate); + + if (best_rate < 0) + return best_rate; + + req->rate = best_rate; + + return 0; +} + +static int isc_clk_set_parent(struct clk_hw *hw, u8 index) +{ + struct isc_clk *isc_clk = to_isc_clk(hw); + + if (index >= clk_hw_get_num_parents(hw)) + return -EINVAL; + + isc_clk->parent_id = index; + + return 0; +} + +static u8 isc_clk_get_parent(struct clk_hw *hw) +{ + struct isc_clk *isc_clk = to_isc_clk(hw); + + return isc_clk->parent_id; +} + +static int isc_clk_set_rate(struct clk_hw *hw, + unsigned long rate, + unsigned long parent_rate) +{ + struct isc_clk *isc_clk = to_isc_clk(hw); + u32 div; + + if (!rate) + return -EINVAL; + + div = DIV_ROUND_CLOSEST(parent_rate, rate); + if (div > (ISC_CLK_MAX_DIV + 1) || !div) + return -EINVAL; + + isc_clk->div = div - 1; + + return 0; +} + +static const struct clk_ops isc_clk_ops = { + .prepare = isc_clk_prepare, + .unprepare = isc_clk_unprepare, + .enable = isc_clk_enable, + .disable = isc_clk_disable, + .is_enabled = isc_clk_is_enabled, + .recalc_rate = isc_clk_recalc_rate, + .determine_rate = isc_clk_determine_rate, + .set_parent = isc_clk_set_parent, + .get_parent = isc_clk_get_parent, + .set_rate = isc_clk_set_rate, +}; + +static int isc_clk_register(struct isc_device *isc, unsigned int id) +{ + struct regmap *regmap = isc->regmap; + struct device_node *np = isc->dev->of_node; + struct isc_clk *isc_clk; + struct clk_init_data init; + const char *clk_name = np->name; + const char *parent_names[3]; + int num_parents; + + if (id == ISC_ISPCK && !isc->ispck_required) + return 0; + + num_parents = of_clk_get_parent_count(np); + if (num_parents < 1 || num_parents > 3) + return -EINVAL; + + if (num_parents > 2 && id == ISC_ISPCK) + num_parents = 2; + + of_clk_parent_fill(np, parent_names, num_parents); + + if (id == ISC_MCK) + of_property_read_string(np, "clock-output-names", &clk_name); + else + clk_name = "isc-ispck"; + + init.parent_names = parent_names; + init.num_parents = num_parents; + init.name = clk_name; + init.ops = &isc_clk_ops; + init.flags = CLK_SET_RATE_GATE | CLK_SET_PARENT_GATE; + + isc_clk = &isc->isc_clks[id]; + isc_clk->hw.init = &init; + isc_clk->regmap = regmap; + isc_clk->id = id; + isc_clk->dev = isc->dev; + spin_lock_init(&isc_clk->lock); + + isc_clk->clk = clk_register(isc->dev, &isc_clk->hw); + if (IS_ERR(isc_clk->clk)) { + dev_err(isc->dev, "%s: clock register fail\n", clk_name); + return PTR_ERR(isc_clk->clk); + } else if (id == ISC_MCK) { + of_clk_add_provider(np, of_clk_src_simple_get, isc_clk->clk); + } + + return 0; +} + +int microchip_isc_clk_init(struct isc_device *isc) +{ + unsigned int i; + int ret; + + for (i = 0; i < ARRAY_SIZE(isc->isc_clks); i++) + isc->isc_clks[i].clk = ERR_PTR(-EINVAL); + + for (i = 0; i < ARRAY_SIZE(isc->isc_clks); i++) { + ret = isc_clk_register(isc, i); + if (ret) + return ret; + } + + return 0; +} +EXPORT_SYMBOL_GPL(microchip_isc_clk_init); + +void microchip_isc_clk_cleanup(struct isc_device *isc) +{ + unsigned int i; + + of_clk_del_provider(isc->dev->of_node); + + for (i = 0; i < ARRAY_SIZE(isc->isc_clks); i++) { + struct isc_clk *isc_clk = &isc->isc_clks[i]; + + if (!IS_ERR(isc_clk->clk)) + clk_unregister(isc_clk->clk); + } +} +EXPORT_SYMBOL_GPL(microchip_isc_clk_cleanup); diff --git a/drivers/media/platform/microchip/microchip-isc-regs.h b/drivers/media/platform/microchip/microchip-isc-regs.h new file mode 100644 index 000000000000..e77e1d9a1db8 --- /dev/null +++ b/drivers/media/platform/microchip/microchip-isc-regs.h @@ -0,0 +1,413 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#ifndef __MICROCHIP_ISC_REGS_H +#define __MICROCHIP_ISC_REGS_H + +#include <linux/bitops.h> + +/* ISC Control Enable Register 0 */ +#define ISC_CTRLEN 0x00000000 + +/* ISC Control Disable Register 0 */ +#define ISC_CTRLDIS 0x00000004 + +/* ISC Control Status Register 0 */ +#define ISC_CTRLSR 0x00000008 + +#define ISC_CTRL_CAPTURE BIT(0) +#define ISC_CTRL_UPPRO BIT(1) +#define ISC_CTRL_HISREQ BIT(2) +#define ISC_CTRL_HISCLR BIT(3) + +/* ISC Parallel Front End Configuration 0 Register */ +#define ISC_PFE_CFG0 0x0000000c + +#define ISC_PFE_CFG0_HPOL_LOW BIT(0) +#define ISC_PFE_CFG0_VPOL_LOW BIT(1) +#define ISC_PFE_CFG0_PPOL_LOW BIT(2) +#define ISC_PFE_CFG0_CCIR656 BIT(9) +#define ISC_PFE_CFG0_CCIR_CRC BIT(10) +#define ISC_PFE_CFG0_MIPI BIT(14) + +#define ISC_PFE_CFG0_MODE_PROGRESSIVE (0x0 << 4) +#define ISC_PFE_CFG0_MODE_MASK GENMASK(6, 4) + +#define ISC_PFE_CFG0_BPS_EIGHT (0x4 << 28) +#define ISC_PFG_CFG0_BPS_NINE (0x3 << 28) +#define ISC_PFG_CFG0_BPS_TEN (0x2 << 28) +#define ISC_PFG_CFG0_BPS_ELEVEN (0x1 << 28) +#define ISC_PFG_CFG0_BPS_TWELVE (0x0 << 28) +#define ISC_PFE_CFG0_BPS_MASK GENMASK(30, 28) + +#define ISC_PFE_CFG0_COLEN BIT(12) +#define ISC_PFE_CFG0_ROWEN BIT(13) + +/* ISC Parallel Front End Configuration 1 Register */ +#define ISC_PFE_CFG1 0x00000010 + +#define ISC_PFE_CFG1_COLMIN(v) ((v)) +#define ISC_PFE_CFG1_COLMIN_MASK GENMASK(15, 0) +#define ISC_PFE_CFG1_COLMAX(v) ((v) << 16) +#define ISC_PFE_CFG1_COLMAX_MASK GENMASK(31, 16) + +/* ISC Parallel Front End Configuration 2 Register */ +#define ISC_PFE_CFG2 0x00000014 + +#define ISC_PFE_CFG2_ROWMIN(v) ((v)) +#define ISC_PFE_CFG2_ROWMIN_MASK GENMASK(15, 0) +#define ISC_PFE_CFG2_ROWMAX(v) ((v) << 16) +#define ISC_PFE_CFG2_ROWMAX_MASK GENMASK(31, 16) + +/* ISC Clock Enable Register */ +#define ISC_CLKEN 0x00000018 + +/* ISC Clock Disable Register */ +#define ISC_CLKDIS 0x0000001c + +/* ISC Clock Status Register */ +#define ISC_CLKSR 0x00000020 +#define ISC_CLKSR_SIP BIT(31) + +#define ISC_CLK(n) BIT(n) + +/* ISC Clock Configuration Register */ +#define ISC_CLKCFG 0x00000024 +#define ISC_CLKCFG_DIV_SHIFT(n) ((n) * 16) +#define ISC_CLKCFG_DIV_MASK(n) GENMASK(((n) * 16 + 7), (n) * 16) +#define ISC_CLKCFG_SEL_SHIFT(n) ((n) * 16 + 8) +#define ISC_CLKCFG_SEL_MASK(n) GENMASK(((n) * 17 + 8), ((n) * 16 + 8)) + +/* ISC Interrupt Enable Register */ +#define ISC_INTEN 0x00000028 + +/* ISC Interrupt Disable Register */ +#define ISC_INTDIS 0x0000002c + +/* ISC Interrupt Mask Register */ +#define ISC_INTMASK 0x00000030 + +/* ISC Interrupt Status Register */ +#define ISC_INTSR 0x00000034 + +#define ISC_INT_DDONE BIT(8) +#define ISC_INT_HISDONE BIT(12) + +/* ISC DPC Control Register */ +#define ISC_DPC_CTRL 0x40 + +#define ISC_DPC_CTRL_DPCEN BIT(0) +#define ISC_DPC_CTRL_GDCEN BIT(1) +#define ISC_DPC_CTRL_BLCEN BIT(2) + +/* ISC DPC Config Register */ +#define ISC_DPC_CFG 0x44 + +#define ISC_DPC_CFG_BAYSEL_SHIFT 0 + +#define ISC_DPC_CFG_EITPOL BIT(4) + +#define ISC_DPC_CFG_TA_ENABLE BIT(14) +#define ISC_DPC_CFG_TC_ENABLE BIT(13) +#define ISC_DPC_CFG_TM_ENABLE BIT(12) + +#define ISC_DPC_CFG_RE_MODE BIT(17) + +#define ISC_DPC_CFG_GDCCLP_SHIFT 20 +#define ISC_DPC_CFG_GDCCLP_MASK GENMASK(22, 20) + +#define ISC_DPC_CFG_BLOFF_SHIFT 24 +#define ISC_DPC_CFG_BLOFF_MASK GENMASK(31, 24) + +#define ISC_DPC_CFG_BAYCFG_SHIFT 0 +#define ISC_DPC_CFG_BAYCFG_MASK GENMASK(1, 0) +/* ISC DPC Threshold Median Register */ +#define ISC_DPC_THRESHM 0x48 + +/* ISC DPC Threshold Closest Register */ +#define ISC_DPC_THRESHC 0x4C + +/* ISC DPC Threshold Average Register */ +#define ISC_DPC_THRESHA 0x50 + +/* ISC DPC STatus Register */ +#define ISC_DPC_SR 0x54 + +/* ISC White Balance Control Register */ +#define ISC_WB_CTRL 0x00000058 + +/* ISC White Balance Configuration Register */ +#define ISC_WB_CFG 0x0000005c + +/* ISC White Balance Offset for R, GR Register */ +#define ISC_WB_O_RGR 0x00000060 + +/* ISC White Balance Offset for B, GB Register */ +#define ISC_WB_O_BGB 0x00000064 + +/* ISC White Balance Gain for R, GR Register */ +#define ISC_WB_G_RGR 0x00000068 + +/* ISC White Balance Gain for B, GB Register */ +#define ISC_WB_G_BGB 0x0000006c + +/* ISC Color Filter Array Control Register */ +#define ISC_CFA_CTRL 0x00000070 + +/* ISC Color Filter Array Configuration Register */ +#define ISC_CFA_CFG 0x00000074 +#define ISC_CFA_CFG_EITPOL BIT(4) + +#define ISC_BAY_CFG_GRGR 0x0 +#define ISC_BAY_CFG_RGRG 0x1 +#define ISC_BAY_CFG_GBGB 0x2 +#define ISC_BAY_CFG_BGBG 0x3 + +/* ISC Color Correction Control Register */ +#define ISC_CC_CTRL 0x00000078 + +/* ISC Color Correction RR RG Register */ +#define ISC_CC_RR_RG 0x0000007c + +/* ISC Color Correction RB OR Register */ +#define ISC_CC_RB_OR 0x00000080 + +/* ISC Color Correction GR GG Register */ +#define ISC_CC_GR_GG 0x00000084 + +/* ISC Color Correction GB OG Register */ +#define ISC_CC_GB_OG 0x00000088 + +/* ISC Color Correction BR BG Register */ +#define ISC_CC_BR_BG 0x0000008c + +/* ISC Color Correction BB OB Register */ +#define ISC_CC_BB_OB 0x00000090 + +/* ISC Gamma Correction Control Register */ +#define ISC_GAM_CTRL 0x00000094 + +#define ISC_GAM_CTRL_BIPART BIT(4) + +/* ISC_Gamma Correction Blue Entry Register */ +#define ISC_GAM_BENTRY 0x00000098 + +/* ISC_Gamma Correction Green Entry Register */ +#define ISC_GAM_GENTRY 0x00000198 + +/* ISC_Gamma Correction Green Entry Register */ +#define ISC_GAM_RENTRY 0x00000298 + +/* ISC VHXS Control Register */ +#define ISC_VHXS_CTRL 0x398 + +/* ISC VHXS Source Size Register */ +#define ISC_VHXS_SS 0x39C + +/* ISC VHXS Destination Size Register */ +#define ISC_VHXS_DS 0x3A0 + +/* ISC Vertical Factor Register */ +#define ISC_VXS_FACT 0x3a4 + +/* ISC Horizontal Factor Register */ +#define ISC_HXS_FACT 0x3a8 + +/* ISC Vertical Config Register */ +#define ISC_VXS_CFG 0x3ac + +/* ISC Horizontal Config Register */ +#define ISC_HXS_CFG 0x3b0 + +/* ISC Vertical Tap Register */ +#define ISC_VXS_TAP 0x3b4 + +/* ISC Horizontal Tap Register */ +#define ISC_HXS_TAP 0x434 + +/* Offset for CSC register specific to sama5d2 product */ +#define ISC_SAMA5D2_CSC_OFFSET 0 +/* Offset for CSC register specific to sama7g5 product */ +#define ISC_SAMA7G5_CSC_OFFSET 0x11c + +/* Color Space Conversion Control Register */ +#define ISC_CSC_CTRL 0x00000398 + +/* Color Space Conversion YR YG Register */ +#define ISC_CSC_YR_YG 0x0000039c + +/* Color Space Conversion YB OY Register */ +#define ISC_CSC_YB_OY 0x000003a0 + +/* Color Space Conversion CBR CBG Register */ +#define ISC_CSC_CBR_CBG 0x000003a4 + +/* Color Space Conversion CBB OCB Register */ +#define ISC_CSC_CBB_OCB 0x000003a8 + +/* Color Space Conversion CRR CRG Register */ +#define ISC_CSC_CRR_CRG 0x000003ac + +/* Color Space Conversion CRB OCR Register */ +#define ISC_CSC_CRB_OCR 0x000003b0 + +/* Offset for CBC register specific to sama5d2 product */ +#define ISC_SAMA5D2_CBC_OFFSET 0 +/* Offset for CBC register specific to sama7g5 product */ +#define ISC_SAMA7G5_CBC_OFFSET 0x11c + +/* Contrast And Brightness Control Register */ +#define ISC_CBC_CTRL 0x000003b4 + +/* Contrast And Brightness Configuration Register */ +#define ISC_CBC_CFG 0x000003b8 + +/* Brightness Register */ +#define ISC_CBC_BRIGHT 0x000003bc +#define ISC_CBC_BRIGHT_MASK GENMASK(10, 0) + +/* Contrast Register */ +#define ISC_CBC_CONTRAST 0x000003c0 +#define ISC_CBC_CONTRAST_MASK GENMASK(11, 0) + +/* Hue Register */ +#define ISC_CBCHS_HUE 0x4e0 +/* Saturation Register */ +#define ISC_CBCHS_SAT 0x4e4 + +/* Offset for SUB422 register specific to sama5d2 product */ +#define ISC_SAMA5D2_SUB422_OFFSET 0 +/* Offset for SUB422 register specific to sama7g5 product */ +#define ISC_SAMA7G5_SUB422_OFFSET 0x124 + +/* Subsampling 4:4:4 to 4:2:2 Control Register */ +#define ISC_SUB422_CTRL 0x000003c4 + +/* Offset for SUB420 register specific to sama5d2 product */ +#define ISC_SAMA5D2_SUB420_OFFSET 0 +/* Offset for SUB420 register specific to sama7g5 product */ +#define ISC_SAMA7G5_SUB420_OFFSET 0x124 +/* Subsampling 4:2:2 to 4:2:0 Control Register */ +#define ISC_SUB420_CTRL 0x000003cc + +/* Offset for RLP register specific to sama5d2 product */ +#define ISC_SAMA5D2_RLP_OFFSET 0 +/* Offset for RLP register specific to sama7g5 product */ +#define ISC_SAMA7G5_RLP_OFFSET 0x124 +/* Rounding, Limiting and Packing Configuration Register */ +#define ISC_RLP_CFG 0x000003d0 + +#define ISC_RLP_CFG_MODE_DAT8 0x0 +#define ISC_RLP_CFG_MODE_DAT9 0x1 +#define ISC_RLP_CFG_MODE_DAT10 0x2 +#define ISC_RLP_CFG_MODE_DAT11 0x3 +#define ISC_RLP_CFG_MODE_DAT12 0x4 +#define ISC_RLP_CFG_MODE_DATY8 0x5 +#define ISC_RLP_CFG_MODE_DATY10 0x6 +#define ISC_RLP_CFG_MODE_ARGB444 0x7 +#define ISC_RLP_CFG_MODE_ARGB555 0x8 +#define ISC_RLP_CFG_MODE_RGB565 0x9 +#define ISC_RLP_CFG_MODE_ARGB32 0xa +#define ISC_RLP_CFG_MODE_YYCC 0xb +#define ISC_RLP_CFG_MODE_YYCC_LIMITED 0xc +#define ISC_RLP_CFG_MODE_YCYC 0xd +#define ISC_RLP_CFG_MODE_MASK GENMASK(3, 0) + +#define ISC_RLP_CFG_LSH BIT(5) + +#define ISC_RLP_CFG_YMODE_YUYV (3 << 6) +#define ISC_RLP_CFG_YMODE_YVYU (2 << 6) +#define ISC_RLP_CFG_YMODE_VYUY (0 << 6) +#define ISC_RLP_CFG_YMODE_UYVY (1 << 6) + +#define ISC_RLP_CFG_YMODE_MASK GENMASK(7, 6) + +/* Offset for HIS register specific to sama5d2 product */ +#define ISC_SAMA5D2_HIS_OFFSET 0 +/* Offset for HIS register specific to sama7g5 product */ +#define ISC_SAMA7G5_HIS_OFFSET 0x124 +/* Histogram Control Register */ +#define ISC_HIS_CTRL 0x000003d4 + +#define ISC_HIS_CTRL_EN BIT(0) +#define ISC_HIS_CTRL_DIS 0x0 + +/* Histogram Configuration Register */ +#define ISC_HIS_CFG 0x000003d8 + +#define ISC_HIS_CFG_MODE_GR 0x0 +#define ISC_HIS_CFG_MODE_R 0x1 +#define ISC_HIS_CFG_MODE_GB 0x2 +#define ISC_HIS_CFG_MODE_B 0x3 +#define ISC_HIS_CFG_MODE_Y 0x4 +#define ISC_HIS_CFG_MODE_RAW 0x5 +#define ISC_HIS_CFG_MODE_YCCIR656 0x6 + +#define ISC_HIS_CFG_BAYSEL_SHIFT 4 + +#define ISC_HIS_CFG_RAR BIT(8) + +/* Offset for DMA register specific to sama5d2 product */ +#define ISC_SAMA5D2_DMA_OFFSET 0 +/* Offset for DMA register specific to sama7g5 product */ +#define ISC_SAMA7G5_DMA_OFFSET 0x13c + +/* DMA Configuration Register */ +#define ISC_DCFG 0x000003e0 +#define ISC_DCFG_IMODE_PACKED8 0x0 +#define ISC_DCFG_IMODE_PACKED16 0x1 +#define ISC_DCFG_IMODE_PACKED32 0x2 +#define ISC_DCFG_IMODE_YC422SP 0x3 +#define ISC_DCFG_IMODE_YC422P 0x4 +#define ISC_DCFG_IMODE_YC420SP 0x5 +#define ISC_DCFG_IMODE_YC420P 0x6 +#define ISC_DCFG_IMODE_MASK GENMASK(2, 0) + +#define ISC_DCFG_YMBSIZE_SINGLE (0x0 << 4) +#define ISC_DCFG_YMBSIZE_BEATS4 (0x1 << 4) +#define ISC_DCFG_YMBSIZE_BEATS8 (0x2 << 4) +#define ISC_DCFG_YMBSIZE_BEATS16 (0x3 << 4) +#define ISC_DCFG_YMBSIZE_BEATS32 (0x4 << 4) +#define ISC_DCFG_YMBSIZE_MASK GENMASK(6, 4) + +#define ISC_DCFG_CMBSIZE_SINGLE (0x0 << 8) +#define ISC_DCFG_CMBSIZE_BEATS4 (0x1 << 8) +#define ISC_DCFG_CMBSIZE_BEATS8 (0x2 << 8) +#define ISC_DCFG_CMBSIZE_BEATS16 (0x3 << 8) +#define ISC_DCFG_CMBSIZE_BEATS32 (0x4 << 8) +#define ISC_DCFG_CMBSIZE_MASK GENMASK(10, 8) + +/* DMA Control Register */ +#define ISC_DCTRL 0x000003e4 + +#define ISC_DCTRL_DVIEW_PACKED (0x0 << 1) +#define ISC_DCTRL_DVIEW_SEMIPLANAR (0x1 << 1) +#define ISC_DCTRL_DVIEW_PLANAR (0x2 << 1) +#define ISC_DCTRL_DVIEW_MASK GENMASK(2, 1) + +#define ISC_DCTRL_IE_IS (0x0 << 4) + +/* DMA Descriptor Address Register */ +#define ISC_DNDA 0x000003e8 + +/* DMA Address 0 Register */ +#define ISC_DAD0 0x000003ec + +/* DMA Address 1 Register */ +#define ISC_DAD1 0x000003f4 + +/* DMA Address 2 Register */ +#define ISC_DAD2 0x000003fc + +/* Offset for version register specific to sama5d2 product */ +#define ISC_SAMA5D2_VERSION_OFFSET 0 +#define ISC_SAMA7G5_VERSION_OFFSET 0x13c +/* Version Register */ +#define ISC_VERSION 0x0000040c + +/* Offset for version register specific to sama5d2 product */ +#define ISC_SAMA5D2_HIS_ENTRY_OFFSET 0 +/* Offset for version register specific to sama7g5 product */ +#define ISC_SAMA7G5_HIS_ENTRY_OFFSET 0x14c +/* Histogram Entry */ +#define ISC_HIS_ENTRY 0x00000410 + +#endif diff --git a/drivers/media/platform/microchip/microchip-isc-scaler.c b/drivers/media/platform/microchip/microchip-isc-scaler.c new file mode 100644 index 000000000000..0f29a32d15ce --- /dev/null +++ b/drivers/media/platform/microchip/microchip-isc-scaler.c @@ -0,0 +1,267 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Microchip Image Sensor Controller (ISC) Scaler entity support + * + * Copyright (C) 2022 Microchip Technology, Inc. + * + * Author: Eugen Hristev <[email protected]> + * + */ + +#include <media/media-device.h> +#include <media/media-entity.h> +#include <media/v4l2-device.h> +#include <media/v4l2-subdev.h> + +#include "microchip-isc-regs.h" +#include "microchip-isc.h" + +static void isc_scaler_prepare_fmt(struct v4l2_mbus_framefmt *framefmt) +{ + framefmt->colorspace = V4L2_COLORSPACE_SRGB; + framefmt->field = V4L2_FIELD_NONE; + framefmt->ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT; + framefmt->quantization = V4L2_QUANTIZATION_DEFAULT; + framefmt->xfer_func = V4L2_XFER_FUNC_DEFAULT; +}; + +static int isc_scaler_get_fmt(struct v4l2_subdev *sd, + struct v4l2_subdev_state *sd_state, + struct v4l2_subdev_format *format) +{ + struct isc_device *isc = container_of(sd, struct isc_device, scaler_sd); + struct v4l2_mbus_framefmt *v4l2_try_fmt; + + if (format->which == V4L2_SUBDEV_FORMAT_TRY) { + v4l2_try_fmt = v4l2_subdev_get_try_format(sd, sd_state, + format->pad); + format->format = *v4l2_try_fmt; + + return 0; + } + + format->format = isc->scaler_format[format->pad]; + + return 0; +} + +static int isc_scaler_set_fmt(struct v4l2_subdev *sd, + struct v4l2_subdev_state *sd_state, + struct v4l2_subdev_format *req_fmt) +{ + struct isc_device *isc = container_of(sd, struct isc_device, scaler_sd); + struct v4l2_mbus_framefmt *v4l2_try_fmt; + struct isc_format *fmt; + unsigned int i; + + /* Source format is fixed, we cannot change it */ + if (req_fmt->pad == ISC_SCALER_PAD_SOURCE) { + req_fmt->format = isc->scaler_format[ISC_SCALER_PAD_SOURCE]; + return 0; + } + + /* There is no limit on the frame size on the sink pad */ + v4l_bound_align_image(&req_fmt->format.width, 16, UINT_MAX, 0, + &req_fmt->format.height, 16, UINT_MAX, 0, 0); + + isc_scaler_prepare_fmt(&req_fmt->format); + + fmt = isc_find_format_by_code(isc, req_fmt->format.code, &i); + + if (!fmt) + fmt = &isc->formats_list[0]; + + req_fmt->format.code = fmt->mbus_code; + + if (req_fmt->which == V4L2_SUBDEV_FORMAT_TRY) { + v4l2_try_fmt = v4l2_subdev_get_try_format(sd, sd_state, + req_fmt->pad); + *v4l2_try_fmt = req_fmt->format; + /* Trying on the sink pad makes the source pad change too */ + v4l2_try_fmt = v4l2_subdev_get_try_format(sd, sd_state, + ISC_SCALER_PAD_SOURCE); + *v4l2_try_fmt = req_fmt->format; + + v4l_bound_align_image(&v4l2_try_fmt->width, + 16, isc->max_width, 0, + &v4l2_try_fmt->height, + 16, isc->max_height, 0, 0); + /* if we are just trying, we are done */ + return 0; + } + + isc->scaler_format[ISC_SCALER_PAD_SINK] = req_fmt->format; + + /* The source pad is the same as the sink, but we have to crop it */ + isc->scaler_format[ISC_SCALER_PAD_SOURCE] = + isc->scaler_format[ISC_SCALER_PAD_SINK]; + v4l_bound_align_image + (&isc->scaler_format[ISC_SCALER_PAD_SOURCE].width, 16, + isc->max_width, 0, + &isc->scaler_format[ISC_SCALER_PAD_SOURCE].height, 16, + isc->max_height, 0, 0); + + return 0; +} + +static int isc_scaler_enum_mbus_code(struct v4l2_subdev *sd, + struct v4l2_subdev_state *sd_state, + struct v4l2_subdev_mbus_code_enum *code) +{ + struct isc_device *isc = container_of(sd, struct isc_device, scaler_sd); + + /* + * All formats supported by the ISC are supported by the scaler. + * Advertise the formats which the ISC can take as input, as the scaler + * entity cropping is part of the PFE module (parallel front end) + */ + if (code->index < isc->formats_list_size) { + code->code = isc->formats_list[code->index].mbus_code; + return 0; + } + + return -EINVAL; +} + +static int isc_scaler_g_sel(struct v4l2_subdev *sd, + struct v4l2_subdev_state *sd_state, + struct v4l2_subdev_selection *sel) +{ + struct isc_device *isc = container_of(sd, struct isc_device, scaler_sd); + + if (sel->pad == ISC_SCALER_PAD_SOURCE) + return -EINVAL; + + if (sel->target != V4L2_SEL_TGT_CROP_BOUNDS && + sel->target != V4L2_SEL_TGT_CROP) + return -EINVAL; + + sel->r.height = isc->scaler_format[ISC_SCALER_PAD_SOURCE].height; + sel->r.width = isc->scaler_format[ISC_SCALER_PAD_SOURCE].width; + + sel->r.left = 0; + sel->r.top = 0; + + return 0; +} + +static int isc_scaler_init_cfg(struct v4l2_subdev *sd, + struct v4l2_subdev_state *sd_state) +{ + struct v4l2_mbus_framefmt *v4l2_try_fmt = + v4l2_subdev_get_try_format(sd, sd_state, 0); + struct v4l2_rect *try_crop; + struct isc_device *isc = container_of(sd, struct isc_device, scaler_sd); + + *v4l2_try_fmt = isc->scaler_format[ISC_SCALER_PAD_SOURCE]; + + try_crop = v4l2_subdev_get_try_crop(sd, sd_state, 0); + + try_crop->top = 0; + try_crop->left = 0; + try_crop->width = v4l2_try_fmt->width; + try_crop->height = v4l2_try_fmt->height; + + return 0; +} + +static const struct v4l2_subdev_pad_ops isc_scaler_pad_ops = { + .enum_mbus_code = isc_scaler_enum_mbus_code, + .set_fmt = isc_scaler_set_fmt, + .get_fmt = isc_scaler_get_fmt, + .get_selection = isc_scaler_g_sel, + .init_cfg = isc_scaler_init_cfg, +}; + +static const struct media_entity_operations isc_scaler_entity_ops = { + .link_validate = v4l2_subdev_link_validate, +}; + +static const struct v4l2_subdev_ops xisc_scaler_subdev_ops = { + .pad = &isc_scaler_pad_ops, +}; + +int isc_scaler_init(struct isc_device *isc) +{ + int ret; + + v4l2_subdev_init(&isc->scaler_sd, &xisc_scaler_subdev_ops); + + isc->scaler_sd.owner = THIS_MODULE; + isc->scaler_sd.dev = isc->dev; + snprintf(isc->scaler_sd.name, sizeof(isc->scaler_sd.name), + "microchip_isc_scaler"); + + isc->scaler_sd.flags |= V4L2_SUBDEV_FL_HAS_DEVNODE; + isc->scaler_sd.entity.function = MEDIA_ENT_F_PROC_VIDEO_SCALER; + isc->scaler_sd.entity.ops = &isc_scaler_entity_ops; + isc->scaler_pads[ISC_SCALER_PAD_SINK].flags = MEDIA_PAD_FL_SINK; + isc->scaler_pads[ISC_SCALER_PAD_SOURCE].flags = MEDIA_PAD_FL_SOURCE; + + isc_scaler_prepare_fmt(&isc->scaler_format[ISC_SCALER_PAD_SOURCE]); + isc->scaler_format[ISC_SCALER_PAD_SOURCE].height = isc->max_height; + isc->scaler_format[ISC_SCALER_PAD_SOURCE].width = isc->max_width; + isc->scaler_format[ISC_SCALER_PAD_SOURCE].code = + isc->formats_list[0].mbus_code; + + isc->scaler_format[ISC_SCALER_PAD_SINK] = + isc->scaler_format[ISC_SCALER_PAD_SOURCE]; + + ret = media_entity_pads_init(&isc->scaler_sd.entity, + ISC_SCALER_PADS_NUM, + isc->scaler_pads); + if (ret < 0) { + dev_err(isc->dev, "scaler sd media entity init failed\n"); + return ret; + } + + ret = v4l2_device_register_subdev(&isc->v4l2_dev, &isc->scaler_sd); + if (ret < 0) { + dev_err(isc->dev, "scaler sd failed to register subdev\n"); + return ret; + } + + return ret; +} +EXPORT_SYMBOL_GPL(isc_scaler_init); + +int isc_scaler_link(struct isc_device *isc) +{ + int ret; + + ret = media_create_pad_link(&isc->current_subdev->sd->entity, + isc->remote_pad, &isc->scaler_sd.entity, + ISC_SCALER_PAD_SINK, + MEDIA_LNK_FL_ENABLED | + MEDIA_LNK_FL_IMMUTABLE); + + if (ret < 0) { + dev_err(isc->dev, "Failed to create pad link: %s to %s\n", + isc->current_subdev->sd->entity.name, + isc->scaler_sd.entity.name); + return ret; + } + + dev_dbg(isc->dev, "link with %s pad: %d\n", + isc->current_subdev->sd->name, isc->remote_pad); + + ret = media_create_pad_link(&isc->scaler_sd.entity, + ISC_SCALER_PAD_SOURCE, + &isc->video_dev.entity, ISC_PAD_SINK, + MEDIA_LNK_FL_ENABLED | + MEDIA_LNK_FL_IMMUTABLE); + + if (ret < 0) { + dev_err(isc->dev, "Failed to create pad link: %s to %s\n", + isc->scaler_sd.entity.name, + isc->video_dev.entity.name); + return ret; + } + + dev_dbg(isc->dev, "link with %s pad: %d\n", isc->scaler_sd.name, + ISC_SCALER_PAD_SOURCE); + + return ret; +} +EXPORT_SYMBOL_GPL(isc_scaler_link); + diff --git a/drivers/media/platform/microchip/microchip-isc.h b/drivers/media/platform/microchip/microchip-isc.h new file mode 100644 index 000000000000..e3a6c7367e70 --- /dev/null +++ b/drivers/media/platform/microchip/microchip-isc.h @@ -0,0 +1,400 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* + * Microchip Image Sensor Controller (ISC) driver header file + * + * Copyright (C) 2016-2019 Microchip Technology, Inc. + * + * Author: Songjun Wu + * Author: Eugen Hristev <[email protected]> + * + */ +#ifndef _MICROCHIP_ISC_H_ + +#include <linux/clk-provider.h> +#include <linux/platform_device.h> + +#include <media/v4l2-ctrls.h> +#include <media/v4l2-device.h> +#include <media/videobuf2-dma-contig.h> + +#define ISC_CLK_MAX_DIV 255 + +enum isc_clk_id { + ISC_ISPCK = 0, + ISC_MCK = 1, +}; + +struct isc_clk { + struct clk_hw hw; + struct clk *clk; + struct regmap *regmap; + spinlock_t lock; /* serialize access to clock registers */ + u8 id; + u8 parent_id; + u32 div; + struct device *dev; +}; + +#define to_isc_clk(v) container_of(v, struct isc_clk, hw) + +struct isc_buffer { + struct vb2_v4l2_buffer vb; + struct list_head list; +}; + +struct isc_subdev_entity { + struct v4l2_subdev *sd; + struct v4l2_async_subdev *asd; + struct device_node *epn; + struct v4l2_async_notifier notifier; + + u32 pfe_cfg0; + + struct list_head list; +}; + +/* + * struct isc_format - ISC media bus format information + This structure represents the interface between the ISC + and the sensor. It's the input format received by + the ISC. + * @fourcc: Fourcc code for this format + * @mbus_code: V4L2 media bus format code. + * @cfa_baycfg: If this format is RAW BAYER, indicate the type of bayer. + this is either BGBG, RGRG, etc. + * @pfe_cfg0_bps: Number of hardware data lines connected to the ISC + * @raw: If the format is raw bayer. + */ + +struct isc_format { + u32 fourcc; + u32 mbus_code; + u32 cfa_baycfg; + u32 pfe_cfg0_bps; + + bool raw; +}; + +/* Pipeline bitmap */ +#define DPC_DPCENABLE BIT(0) +#define DPC_GDCENABLE BIT(1) +#define DPC_BLCENABLE BIT(2) +#define WB_ENABLE BIT(3) +#define CFA_ENABLE BIT(4) +#define CC_ENABLE BIT(5) +#define GAM_ENABLE BIT(6) +#define GAM_BENABLE BIT(7) +#define GAM_GENABLE BIT(8) +#define GAM_RENABLE BIT(9) +#define VHXS_ENABLE BIT(10) +#define CSC_ENABLE BIT(11) +#define CBC_ENABLE BIT(12) +#define SUB422_ENABLE BIT(13) +#define SUB420_ENABLE BIT(14) + +#define GAM_ENABLES (GAM_RENABLE | GAM_GENABLE | GAM_BENABLE | GAM_ENABLE) + +/* + * struct fmt_config - ISC format configuration and internal pipeline + This structure represents the internal configuration + of the ISC. + It also holds the format that ISC will present to v4l2. + * @sd_format: Pointer to an isc_format struct that holds the sensor + configuration. + * @fourcc: Fourcc code for this format. + * @bpp: Bytes per pixel in the current format. + * @bpp_v4l2: Bytes per pixel in the current format, for v4l2. + This differs from 'bpp' in the sense that in planar + formats, it refers only to the first plane. + * @rlp_cfg_mode: Configuration of the RLP (rounding, limiting packaging) + * @dcfg_imode: Configuration of the input of the DMA module + * @dctrl_dview: Configuration of the output of the DMA module + * @bits_pipeline: Configuration of the pipeline, which modules are enabled + */ +struct fmt_config { + struct isc_format *sd_format; + + u32 fourcc; + u8 bpp; + u8 bpp_v4l2; + + u32 rlp_cfg_mode; + u32 dcfg_imode; + u32 dctrl_dview; + + u32 bits_pipeline; +}; + +#define HIST_ENTRIES 512 +#define HIST_BAYER (ISC_HIS_CFG_MODE_B + 1) + +enum{ + HIST_INIT = 0, + HIST_ENABLED, + HIST_DISABLED, +}; + +struct isc_ctrls { + struct v4l2_ctrl_handler handler; + + u32 brightness; + u32 contrast; + u8 gamma_index; +#define ISC_WB_NONE 0 +#define ISC_WB_AUTO 1 +#define ISC_WB_ONETIME 2 + u8 awb; + + /* one for each component : GR, R, GB, B */ + u32 gain[HIST_BAYER]; + s32 offset[HIST_BAYER]; + + u32 hist_entry[HIST_ENTRIES]; + u32 hist_count[HIST_BAYER]; + u8 hist_id; + u8 hist_stat; +#define HIST_MIN_INDEX 0 +#define HIST_MAX_INDEX 1 + u32 hist_minmax[HIST_BAYER][2]; +}; + +#define ISC_PIPE_LINE_NODE_NUM 15 + +/* + * struct isc_reg_offsets - ISC device register offsets + * @csc: Offset for the CSC register + * @cbc: Offset for the CBC register + * @sub422: Offset for the SUB422 register + * @sub420: Offset for the SUB420 register + * @rlp: Offset for the RLP register + * @his: Offset for the HIS related registers + * @dma: Offset for the DMA related registers + * @version: Offset for the version register + * @his_entry: Offset for the HIS entries registers + */ +struct isc_reg_offsets { + u32 csc; + u32 cbc; + u32 sub422; + u32 sub420; + u32 rlp; + u32 his; + u32 dma; + u32 version; + u32 his_entry; +}; + +enum isc_mc_pads { + ISC_PAD_SINK = 0, + ISC_PADS_NUM = 1, +}; + +enum isc_scaler_pads { + ISC_SCALER_PAD_SINK = 0, + ISC_SCALER_PAD_SOURCE = 1, + ISC_SCALER_PADS_NUM = 2, +}; + +/* + * struct isc_device - ISC device driver data/config struct + * @regmap: Register map + * @hclock: Hclock clock input (refer datasheet) + * @ispck: iscpck clock (refer datasheet) + * @isc_clks: ISC clocks + * @ispck_required: ISC requires ISP Clock initialization + * @dcfg: DMA master configuration, architecture dependent + * + * @dev: Registered device driver + * @v4l2_dev: v4l2 registered device + * @video_dev: registered video device + * + * @vb2_vidq: video buffer 2 video queue + * @dma_queue_lock: lock to serialize the dma buffer queue + * @dma_queue: the queue for dma buffers + * @cur_frm: current isc frame/buffer + * @sequence: current frame number + * @stop: true if isc is not streaming, false if streaming + * @comp: completion reference that signals frame completion + * + * @fmt: current v42l format + * @try_fmt: current v4l2 try format + * + * @config: current ISC format configuration + * @try_config: the current ISC try format , not yet activated + * + * @ctrls: holds information about ISC controls + * @do_wb_ctrl: control regarding the DO_WHITE_BALANCE button + * @awb_work: workqueue reference for autowhitebalance histogram + * analysis + * + * @lock: lock for serializing userspace file operations + * with ISC operations + * @awb_mutex: serialize access to streaming status from awb work queue + * @awb_lock: lock for serializing awb work queue operations + * with DMA/buffer operations + * + * @pipeline: configuration of the ISC pipeline + * + * @current_subdev: current subdevice: the sensor + * @subdev_entities: list of subdevice entitites + * + * @gamma_table: pointer to the table with gamma values, has + * gamma_max sets of GAMMA_ENTRIES entries each + * @gamma_max: maximum number of sets of inside the gamma_table + * + * @max_width: maximum frame width, dependent on the internal RAM + * @max_height: maximum frame height, dependent on the internal RAM + * + * @config_dpc: pointer to a function that initializes product + * specific DPC module + * @config_csc: pointer to a function that initializes product + * specific CSC module + * @config_cbc: pointer to a function that initializes product + * specific CBC module + * @config_cc: pointer to a function that initializes product + * specific CC module + * @config_gam: pointer to a function that initializes product + * specific GAMMA module + * @config_rlp: pointer to a function that initializes product + * specific RLP module + * @config_ctrls: pointer to a functoin that initializes product + * specific v4l2 controls. + * + * @adapt_pipeline: pointer to a function that adapts the pipeline bits + * to the product specific pipeline + * + * @offsets: struct holding the product specific register offsets + * @controller_formats: pointer to the array of possible formats that the + * controller can output + * @formats_list: pointer to the array of possible formats that can + * be used as an input to the controller + * @controller_formats_size: size of controller_formats array + * @formats_list_size: size of formats_list array + * @pads: media controller pads for isc video entity + * @mdev: media device that is registered by the isc + * @mpipe: media device pipeline used by the isc + * @remote_pad: remote pad on the connected subdevice + * @scaler_sd: subdevice for the scaler that isc registers + * @scaler_pads: media controller pads for the scaler subdevice + * @scaler_format: current format for the scaler subdevice + */ +struct isc_device { + struct regmap *regmap; + struct clk *hclock; + struct clk *ispck; + struct isc_clk isc_clks[2]; + bool ispck_required; + u32 dcfg; + + struct device *dev; + struct v4l2_device v4l2_dev; + struct video_device video_dev; + + struct vb2_queue vb2_vidq; + spinlock_t dma_queue_lock; + struct list_head dma_queue; + struct isc_buffer *cur_frm; + unsigned int sequence; + bool stop; + struct completion comp; + + struct v4l2_format fmt; + struct v4l2_format try_fmt; + + struct fmt_config config; + struct fmt_config try_config; + + struct isc_ctrls ctrls; + struct work_struct awb_work; + + struct mutex lock; + struct mutex awb_mutex; + spinlock_t awb_lock; + + struct regmap_field *pipeline[ISC_PIPE_LINE_NODE_NUM]; + + struct isc_subdev_entity *current_subdev; + struct list_head subdev_entities; + + struct { +#define ISC_CTRL_DO_WB 1 +#define ISC_CTRL_R_GAIN 2 +#define ISC_CTRL_B_GAIN 3 +#define ISC_CTRL_GR_GAIN 4 +#define ISC_CTRL_GB_GAIN 5 +#define ISC_CTRL_R_OFF 6 +#define ISC_CTRL_B_OFF 7 +#define ISC_CTRL_GR_OFF 8 +#define ISC_CTRL_GB_OFF 9 + struct v4l2_ctrl *awb_ctrl; + struct v4l2_ctrl *do_wb_ctrl; + struct v4l2_ctrl *r_gain_ctrl; + struct v4l2_ctrl *b_gain_ctrl; + struct v4l2_ctrl *gr_gain_ctrl; + struct v4l2_ctrl *gb_gain_ctrl; + struct v4l2_ctrl *r_off_ctrl; + struct v4l2_ctrl *b_off_ctrl; + struct v4l2_ctrl *gr_off_ctrl; + struct v4l2_ctrl *gb_off_ctrl; + }; + +#define GAMMA_ENTRIES 64 + /* pointer to the defined gamma table */ + const u32 (*gamma_table)[GAMMA_ENTRIES]; + u32 gamma_max; + + u32 max_width; + u32 max_height; + + struct { + void (*config_dpc)(struct isc_device *isc); + void (*config_csc)(struct isc_device *isc); + void (*config_cbc)(struct isc_device *isc); + void (*config_cc)(struct isc_device *isc); + void (*config_gam)(struct isc_device *isc); + void (*config_rlp)(struct isc_device *isc); + + void (*config_ctrls)(struct isc_device *isc, + const struct v4l2_ctrl_ops *ops); + + void (*adapt_pipeline)(struct isc_device *isc); + }; + + struct isc_reg_offsets offsets; + const struct isc_format *controller_formats; + struct isc_format *formats_list; + u32 controller_formats_size; + u32 formats_list_size; + + struct { + struct media_pad pads[ISC_PADS_NUM]; + struct media_device mdev; + struct media_pipeline mpipe; + + u32 remote_pad; + }; + + struct { + struct v4l2_subdev scaler_sd; + struct media_pad scaler_pads[ISC_SCALER_PADS_NUM]; + struct v4l2_mbus_framefmt scaler_format[ISC_SCALER_PADS_NUM]; + }; +}; + +extern const struct regmap_config microchip_isc_regmap_config; +extern const struct v4l2_async_notifier_operations microchip_isc_async_ops; + +irqreturn_t microchip_isc_interrupt(int irq, void *dev_id); +int microchip_isc_pipeline_init(struct isc_device *isc); +int microchip_isc_clk_init(struct isc_device *isc); +void microchip_isc_subdev_cleanup(struct isc_device *isc); +void microchip_isc_clk_cleanup(struct isc_device *isc); + +int isc_scaler_link(struct isc_device *isc); +int isc_scaler_init(struct isc_device *isc); +int isc_mc_init(struct isc_device *isc, u32 ver); +void isc_mc_cleanup(struct isc_device *isc); + +struct isc_format *isc_find_format_by_code(struct isc_device *isc, + unsigned int code, int *index); +#endif diff --git a/drivers/media/platform/microchip/microchip-sama5d2-isc.c b/drivers/media/platform/microchip/microchip-sama5d2-isc.c new file mode 100644 index 000000000000..ac4715d91de6 --- /dev/null +++ b/drivers/media/platform/microchip/microchip-sama5d2-isc.c @@ -0,0 +1,683 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Microchip Image Sensor Controller (ISC) driver + * + * Copyright (C) 2016-2019 Microchip Technology, Inc. + * + * Author: Songjun Wu + * Author: Eugen Hristev <[email protected]> + * + * + * Sensor-->PFE-->WB-->CFA-->CC-->GAM-->CSC-->CBC-->SUB-->RLP-->DMA + * + * ISC video pipeline integrates the following submodules: + * PFE: Parallel Front End to sample the camera sensor input stream + * WB: Programmable white balance in the Bayer domain + * CFA: Color filter array interpolation module + * CC: Programmable color correction + * GAM: Gamma correction + * CSC: Programmable color space conversion + * CBC: Contrast and Brightness control + * SUB: This module performs YCbCr444 to YCbCr420 chrominance subsampling + * RLP: This module performs rounding, range limiting + * and packing of the incoming data + */ + +#include <linux/clk.h> +#include <linux/clkdev.h> +#include <linux/clk-provider.h> +#include <linux/delay.h> +#include <linux/interrupt.h> +#include <linux/math64.h> +#include <linux/module.h> +#include <linux/of.h> +#include <linux/of_graph.h> +#include <linux/platform_device.h> +#include <linux/pm_runtime.h> +#include <linux/regmap.h> +#include <linux/videodev2.h> + +#include <media/v4l2-ctrls.h> +#include <media/v4l2-device.h> +#include <media/v4l2-event.h> +#include <media/v4l2-image-sizes.h> +#include <media/v4l2-ioctl.h> +#include <media/v4l2-fwnode.h> +#include <media/v4l2-subdev.h> +#include <media/videobuf2-dma-contig.h> + +#include "microchip-isc-regs.h" +#include "microchip-isc.h" + +#define ISC_SAMA5D2_MAX_SUPPORT_WIDTH 2592 +#define ISC_SAMA5D2_MAX_SUPPORT_HEIGHT 1944 + +#define ISC_SAMA5D2_PIPELINE \ + (WB_ENABLE | CFA_ENABLE | CC_ENABLE | GAM_ENABLES | CSC_ENABLE | \ + CBC_ENABLE | SUB422_ENABLE | SUB420_ENABLE) + +/* This is a list of the formats that the ISC can *output* */ +static const struct isc_format sama5d2_controller_formats[] = { + { + .fourcc = V4L2_PIX_FMT_ARGB444, + }, { + .fourcc = V4L2_PIX_FMT_ARGB555, + }, { + .fourcc = V4L2_PIX_FMT_RGB565, + }, { + .fourcc = V4L2_PIX_FMT_ABGR32, + }, { + .fourcc = V4L2_PIX_FMT_XBGR32, + }, { + .fourcc = V4L2_PIX_FMT_YUV420, + }, { + .fourcc = V4L2_PIX_FMT_YUYV, + }, { + .fourcc = V4L2_PIX_FMT_YUV422P, + }, { + .fourcc = V4L2_PIX_FMT_GREY, + }, { + .fourcc = V4L2_PIX_FMT_Y10, + }, { + .fourcc = V4L2_PIX_FMT_SBGGR8, + .raw = true, + }, { + .fourcc = V4L2_PIX_FMT_SGBRG8, + .raw = true, + }, { + .fourcc = V4L2_PIX_FMT_SGRBG8, + .raw = true, + }, { + .fourcc = V4L2_PIX_FMT_SRGGB8, + .raw = true, + }, { + .fourcc = V4L2_PIX_FMT_SBGGR10, + .raw = true, + }, { + .fourcc = V4L2_PIX_FMT_SGBRG10, + .raw = true, + }, { + .fourcc = V4L2_PIX_FMT_SGRBG10, + .raw = true, + }, { + .fourcc = V4L2_PIX_FMT_SRGGB10, + .raw = true, + }, { + .fourcc = V4L2_PIX_FMT_SBGGR12, + .raw = true, + }, { + .fourcc = V4L2_PIX_FMT_SGBRG12, + .raw = true, + }, { + .fourcc = V4L2_PIX_FMT_SGRBG12, + .raw = true, + }, { + .fourcc = V4L2_PIX_FMT_SRGGB12, + .raw = true, + }, +}; + +/* This is a list of formats that the ISC can receive as *input* */ +static struct isc_format sama5d2_formats_list[] = { + { + .fourcc = V4L2_PIX_FMT_SBGGR8, + .mbus_code = MEDIA_BUS_FMT_SBGGR8_1X8, + .pfe_cfg0_bps = ISC_PFE_CFG0_BPS_EIGHT, + .cfa_baycfg = ISC_BAY_CFG_BGBG, + }, + { + .fourcc = V4L2_PIX_FMT_SGBRG8, + .mbus_code = MEDIA_BUS_FMT_SGBRG8_1X8, + .pfe_cfg0_bps = ISC_PFE_CFG0_BPS_EIGHT, + .cfa_baycfg = ISC_BAY_CFG_GBGB, + }, + { + .fourcc = V4L2_PIX_FMT_SGRBG8, + .mbus_code = MEDIA_BUS_FMT_SGRBG8_1X8, + .pfe_cfg0_bps = ISC_PFE_CFG0_BPS_EIGHT, + .cfa_baycfg = ISC_BAY_CFG_GRGR, + }, + { + .fourcc = V4L2_PIX_FMT_SRGGB8, + .mbus_code = MEDIA_BUS_FMT_SRGGB8_1X8, + .pfe_cfg0_bps = ISC_PFE_CFG0_BPS_EIGHT, + .cfa_baycfg = ISC_BAY_CFG_RGRG, + }, + { + .fourcc = V4L2_PIX_FMT_SBGGR10, + .mbus_code = MEDIA_BUS_FMT_SBGGR10_1X10, + .pfe_cfg0_bps = ISC_PFG_CFG0_BPS_TEN, + .cfa_baycfg = ISC_BAY_CFG_RGRG, + }, + { + .fourcc = V4L2_PIX_FMT_SGBRG10, + .mbus_code = MEDIA_BUS_FMT_SGBRG10_1X10, + .pfe_cfg0_bps = ISC_PFG_CFG0_BPS_TEN, + .cfa_baycfg = ISC_BAY_CFG_GBGB, + }, + { + .fourcc = V4L2_PIX_FMT_SGRBG10, + .mbus_code = MEDIA_BUS_FMT_SGRBG10_1X10, + .pfe_cfg0_bps = ISC_PFG_CFG0_BPS_TEN, + .cfa_baycfg = ISC_BAY_CFG_GRGR, + }, + { + .fourcc = V4L2_PIX_FMT_SRGGB10, + .mbus_code = MEDIA_BUS_FMT_SRGGB10_1X10, + .pfe_cfg0_bps = ISC_PFG_CFG0_BPS_TEN, + .cfa_baycfg = ISC_BAY_CFG_RGRG, + }, + { + .fourcc = V4L2_PIX_FMT_SBGGR12, + .mbus_code = MEDIA_BUS_FMT_SBGGR12_1X12, + .pfe_cfg0_bps = ISC_PFG_CFG0_BPS_TWELVE, + .cfa_baycfg = ISC_BAY_CFG_BGBG, + }, + { + .fourcc = V4L2_PIX_FMT_SGBRG12, + .mbus_code = MEDIA_BUS_FMT_SGBRG12_1X12, + .pfe_cfg0_bps = ISC_PFG_CFG0_BPS_TWELVE, + .cfa_baycfg = ISC_BAY_CFG_GBGB, + }, + { + .fourcc = V4L2_PIX_FMT_SGRBG12, + .mbus_code = MEDIA_BUS_FMT_SGRBG12_1X12, + .pfe_cfg0_bps = ISC_PFG_CFG0_BPS_TWELVE, + .cfa_baycfg = ISC_BAY_CFG_GRGR, + }, + { + .fourcc = V4L2_PIX_FMT_SRGGB12, + .mbus_code = MEDIA_BUS_FMT_SRGGB12_1X12, + .pfe_cfg0_bps = ISC_PFG_CFG0_BPS_TWELVE, + .cfa_baycfg = ISC_BAY_CFG_RGRG, + }, + { + .fourcc = V4L2_PIX_FMT_GREY, + .mbus_code = MEDIA_BUS_FMT_Y8_1X8, + .pfe_cfg0_bps = ISC_PFE_CFG0_BPS_EIGHT, + }, + { + .fourcc = V4L2_PIX_FMT_YUYV, + .mbus_code = MEDIA_BUS_FMT_YUYV8_2X8, + .pfe_cfg0_bps = ISC_PFE_CFG0_BPS_EIGHT, + }, + { + .fourcc = V4L2_PIX_FMT_RGB565, + .mbus_code = MEDIA_BUS_FMT_RGB565_2X8_LE, + .pfe_cfg0_bps = ISC_PFE_CFG0_BPS_EIGHT, + }, + { + .fourcc = V4L2_PIX_FMT_Y10, + .mbus_code = MEDIA_BUS_FMT_Y10_1X10, + .pfe_cfg0_bps = ISC_PFG_CFG0_BPS_TEN, + }, + +}; + +static void isc_sama5d2_config_csc(struct isc_device *isc) +{ + struct regmap *regmap = isc->regmap; + + /* Convert RGB to YUV */ + regmap_write(regmap, ISC_CSC_YR_YG + isc->offsets.csc, + 0x42 | (0x81 << 16)); + regmap_write(regmap, ISC_CSC_YB_OY + isc->offsets.csc, + 0x19 | (0x10 << 16)); + regmap_write(regmap, ISC_CSC_CBR_CBG + isc->offsets.csc, + 0xFDA | (0xFB6 << 16)); + regmap_write(regmap, ISC_CSC_CBB_OCB + isc->offsets.csc, + 0x70 | (0x80 << 16)); + regmap_write(regmap, ISC_CSC_CRR_CRG + isc->offsets.csc, + 0x70 | (0xFA2 << 16)); + regmap_write(regmap, ISC_CSC_CRB_OCR + isc->offsets.csc, + 0xFEE | (0x80 << 16)); +} + +static void isc_sama5d2_config_cbc(struct isc_device *isc) +{ + struct regmap *regmap = isc->regmap; + + regmap_write(regmap, ISC_CBC_BRIGHT + isc->offsets.cbc, + isc->ctrls.brightness); + regmap_write(regmap, ISC_CBC_CONTRAST + isc->offsets.cbc, + isc->ctrls.contrast); +} + +static void isc_sama5d2_config_cc(struct isc_device *isc) +{ + struct regmap *regmap = isc->regmap; + + /* Configure each register at the neutral fixed point 1.0 or 0.0 */ + regmap_write(regmap, ISC_CC_RR_RG, (1 << 8)); + regmap_write(regmap, ISC_CC_RB_OR, 0); + regmap_write(regmap, ISC_CC_GR_GG, (1 << 8) << 16); + regmap_write(regmap, ISC_CC_GB_OG, 0); + regmap_write(regmap, ISC_CC_BR_BG, 0); + regmap_write(regmap, ISC_CC_BB_OB, (1 << 8)); +} + +static void isc_sama5d2_config_ctrls(struct isc_device *isc, + const struct v4l2_ctrl_ops *ops) +{ + struct isc_ctrls *ctrls = &isc->ctrls; + struct v4l2_ctrl_handler *hdl = &ctrls->handler; + + ctrls->contrast = 256; + + v4l2_ctrl_new_std(hdl, ops, V4L2_CID_CONTRAST, -2048, 2047, 1, 256); +} + +static void isc_sama5d2_config_dpc(struct isc_device *isc) +{ + /* This module is not present on sama5d2 pipeline */ +} + +static void isc_sama5d2_config_gam(struct isc_device *isc) +{ + /* No specific gamma configuration */ +} + +static void isc_sama5d2_config_rlp(struct isc_device *isc) +{ + struct regmap *regmap = isc->regmap; + u32 rlp_mode = isc->config.rlp_cfg_mode; + + /* + * In sama5d2, the YUV planar modes and the YUYV modes are treated + * in the same way in RLP register. + * Normally, YYCC mode should be Luma(n) - Color B(n) - Color R (n) + * and YCYC should be Luma(n + 1) - Color B (n) - Luma (n) - Color R (n) + * but in sama5d2, the YCYC mode does not exist, and YYCC must be + * selected for both planar and interleaved modes, as in fact + * both modes are supported. + * + * Thus, if the YCYC mode is selected, replace it with the + * sama5d2-compliant mode which is YYCC . + */ + if ((rlp_mode & ISC_RLP_CFG_MODE_MASK) == ISC_RLP_CFG_MODE_YCYC) { + rlp_mode &= ~ISC_RLP_CFG_MODE_MASK; + rlp_mode |= ISC_RLP_CFG_MODE_YYCC; + } + + regmap_update_bits(regmap, ISC_RLP_CFG + isc->offsets.rlp, + ISC_RLP_CFG_MODE_MASK, rlp_mode); +} + +static void isc_sama5d2_adapt_pipeline(struct isc_device *isc) +{ + isc->try_config.bits_pipeline &= ISC_SAMA5D2_PIPELINE; +} + +/* Gamma table with gamma 1/2.2 */ +static const u32 isc_sama5d2_gamma_table[][GAMMA_ENTRIES] = { + /* 0 --> gamma 1/1.8 */ + { 0x65, 0x66002F, 0x950025, 0xBB0020, 0xDB001D, 0xF8001A, + 0x1130018, 0x12B0017, 0x1420016, 0x1580014, 0x16D0013, 0x1810012, + 0x1940012, 0x1A60012, 0x1B80011, 0x1C90010, 0x1DA0010, 0x1EA000F, + 0x1FA000F, 0x209000F, 0x218000F, 0x227000E, 0x235000E, 0x243000E, + 0x251000E, 0x25F000D, 0x26C000D, 0x279000D, 0x286000D, 0x293000C, + 0x2A0000C, 0x2AC000C, 0x2B8000C, 0x2C4000C, 0x2D0000B, 0x2DC000B, + 0x2E7000B, 0x2F3000B, 0x2FE000B, 0x309000B, 0x314000B, 0x31F000A, + 0x32A000A, 0x334000B, 0x33F000A, 0x349000A, 0x354000A, 0x35E000A, + 0x368000A, 0x372000A, 0x37C000A, 0x386000A, 0x3900009, 0x399000A, + 0x3A30009, 0x3AD0009, 0x3B60009, 0x3BF000A, 0x3C90009, 0x3D20009, + 0x3DB0009, 0x3E40009, 0x3ED0009, 0x3F60009 }, + + /* 1 --> gamma 1/2 */ + { 0x7F, 0x800034, 0xB50028, 0xDE0021, 0x100001E, 0x11E001B, + 0x1390019, 0x1520017, 0x16A0015, 0x1800014, 0x1940014, 0x1A80013, + 0x1BB0012, 0x1CD0011, 0x1DF0010, 0x1EF0010, 0x200000F, 0x20F000F, + 0x21F000E, 0x22D000F, 0x23C000E, 0x24A000E, 0x258000D, 0x265000D, + 0x273000C, 0x27F000D, 0x28C000C, 0x299000C, 0x2A5000C, 0x2B1000B, + 0x2BC000C, 0x2C8000B, 0x2D3000C, 0x2DF000B, 0x2EA000A, 0x2F5000A, + 0x2FF000B, 0x30A000A, 0x314000B, 0x31F000A, 0x329000A, 0x333000A, + 0x33D0009, 0x3470009, 0x350000A, 0x35A0009, 0x363000A, 0x36D0009, + 0x3760009, 0x37F0009, 0x3880009, 0x3910009, 0x39A0009, 0x3A30009, + 0x3AC0008, 0x3B40009, 0x3BD0008, 0x3C60008, 0x3CE0008, 0x3D60009, + 0x3DF0008, 0x3E70008, 0x3EF0008, 0x3F70008 }, + + /* 2 --> gamma 1/2.2 */ + { 0x99, 0x9B0038, 0xD4002A, 0xFF0023, 0x122001F, 0x141001B, + 0x15D0019, 0x1760017, 0x18E0015, 0x1A30015, 0x1B80013, 0x1CC0012, + 0x1DE0011, 0x1F00010, 0x2010010, 0x2110010, 0x221000F, 0x230000F, + 0x23F000E, 0x24D000E, 0x25B000D, 0x269000C, 0x276000C, 0x283000C, + 0x28F000C, 0x29B000C, 0x2A7000C, 0x2B3000B, 0x2BF000B, 0x2CA000B, + 0x2D5000B, 0x2E0000A, 0x2EB000A, 0x2F5000A, 0x2FF000A, 0x30A000A, + 0x3140009, 0x31E0009, 0x327000A, 0x3310009, 0x33A0009, 0x3440009, + 0x34D0009, 0x3560009, 0x35F0009, 0x3680008, 0x3710008, 0x3790009, + 0x3820008, 0x38A0008, 0x3930008, 0x39B0008, 0x3A30008, 0x3AB0008, + 0x3B30008, 0x3BB0008, 0x3C30008, 0x3CB0007, 0x3D20008, 0x3DA0007, + 0x3E20007, 0x3E90007, 0x3F00008, 0x3F80007 }, +}; + +static int isc_parse_dt(struct device *dev, struct isc_device *isc) +{ + struct device_node *np = dev->of_node; + struct device_node *epn = NULL; + struct isc_subdev_entity *subdev_entity; + unsigned int flags; + int ret; + + INIT_LIST_HEAD(&isc->subdev_entities); + + while (1) { + struct v4l2_fwnode_endpoint v4l2_epn = { .bus_type = 0 }; + + epn = of_graph_get_next_endpoint(np, epn); + if (!epn) + return 0; + + ret = v4l2_fwnode_endpoint_parse(of_fwnode_handle(epn), + &v4l2_epn); + if (ret) { + ret = -EINVAL; + dev_err(dev, "Could not parse the endpoint\n"); + break; + } + + subdev_entity = devm_kzalloc(dev, sizeof(*subdev_entity), + GFP_KERNEL); + if (!subdev_entity) { + ret = -ENOMEM; + break; + } + subdev_entity->epn = epn; + + flags = v4l2_epn.bus.parallel.flags; + + if (flags & V4L2_MBUS_HSYNC_ACTIVE_LOW) + subdev_entity->pfe_cfg0 = ISC_PFE_CFG0_HPOL_LOW; + + if (flags & V4L2_MBUS_VSYNC_ACTIVE_LOW) + subdev_entity->pfe_cfg0 |= ISC_PFE_CFG0_VPOL_LOW; + + if (flags & V4L2_MBUS_PCLK_SAMPLE_FALLING) + subdev_entity->pfe_cfg0 |= ISC_PFE_CFG0_PPOL_LOW; + + if (v4l2_epn.bus_type == V4L2_MBUS_BT656) + subdev_entity->pfe_cfg0 |= ISC_PFE_CFG0_CCIR_CRC | + ISC_PFE_CFG0_CCIR656; + + list_add_tail(&subdev_entity->list, &isc->subdev_entities); + } + of_node_put(epn); + + return ret; +} + +static int microchip_isc_probe(struct platform_device *pdev) +{ + struct device *dev = &pdev->dev; + struct isc_device *isc; + struct resource *res; + void __iomem *io_base; + struct isc_subdev_entity *subdev_entity; + int irq; + int ret; + u32 ver; + + isc = devm_kzalloc(dev, sizeof(*isc), GFP_KERNEL); + if (!isc) + return -ENOMEM; + + platform_set_drvdata(pdev, isc); + isc->dev = dev; + + res = platform_get_resource(pdev, IORESOURCE_MEM, 0); + io_base = devm_ioremap_resource(dev, res); + if (IS_ERR(io_base)) + return PTR_ERR(io_base); + + isc->regmap = devm_regmap_init_mmio(dev, io_base, µchip_isc_regmap_config); + if (IS_ERR(isc->regmap)) { + ret = PTR_ERR(isc->regmap); + dev_err(dev, "failed to init register map: %d\n", ret); + return ret; + } + + irq = platform_get_irq(pdev, 0); + if (irq < 0) + return irq; + + ret = devm_request_irq(dev, irq, microchip_isc_interrupt, 0, + "microchip-sama5d2-isc", isc); + if (ret < 0) { + dev_err(dev, "can't register ISR for IRQ %u (ret=%i)\n", + irq, ret); + return ret; + } + + isc->gamma_table = isc_sama5d2_gamma_table; + isc->gamma_max = 2; + + isc->max_width = ISC_SAMA5D2_MAX_SUPPORT_WIDTH; + isc->max_height = ISC_SAMA5D2_MAX_SUPPORT_HEIGHT; + + isc->config_dpc = isc_sama5d2_config_dpc; + isc->config_csc = isc_sama5d2_config_csc; + isc->config_cbc = isc_sama5d2_config_cbc; + isc->config_cc = isc_sama5d2_config_cc; + isc->config_gam = isc_sama5d2_config_gam; + isc->config_rlp = isc_sama5d2_config_rlp; + isc->config_ctrls = isc_sama5d2_config_ctrls; + + isc->adapt_pipeline = isc_sama5d2_adapt_pipeline; + + isc->offsets.csc = ISC_SAMA5D2_CSC_OFFSET; + isc->offsets.cbc = ISC_SAMA5D2_CBC_OFFSET; + isc->offsets.sub422 = ISC_SAMA5D2_SUB422_OFFSET; + isc->offsets.sub420 = ISC_SAMA5D2_SUB420_OFFSET; + isc->offsets.rlp = ISC_SAMA5D2_RLP_OFFSET; + isc->offsets.his = ISC_SAMA5D2_HIS_OFFSET; + isc->offsets.dma = ISC_SAMA5D2_DMA_OFFSET; + isc->offsets.version = ISC_SAMA5D2_VERSION_OFFSET; + isc->offsets.his_entry = ISC_SAMA5D2_HIS_ENTRY_OFFSET; + + isc->controller_formats = sama5d2_controller_formats; + isc->controller_formats_size = ARRAY_SIZE(sama5d2_controller_formats); + isc->formats_list = sama5d2_formats_list; + isc->formats_list_size = ARRAY_SIZE(sama5d2_formats_list); + + /* sama5d2-isc - 8 bits per beat */ + isc->dcfg = ISC_DCFG_YMBSIZE_BEATS8 | ISC_DCFG_CMBSIZE_BEATS8; + + /* sama5d2-isc : ISPCK is required and mandatory */ + isc->ispck_required = true; + + ret = microchip_isc_pipeline_init(isc); + if (ret) + return ret; + + isc->hclock = devm_clk_get(dev, "hclock"); + if (IS_ERR(isc->hclock)) { + ret = PTR_ERR(isc->hclock); + dev_err(dev, "failed to get hclock: %d\n", ret); + return ret; + } + + ret = clk_prepare_enable(isc->hclock); + if (ret) { + dev_err(dev, "failed to enable hclock: %d\n", ret); + return ret; + } + + ret = microchip_isc_clk_init(isc); + if (ret) { + dev_err(dev, "failed to init isc clock: %d\n", ret); + goto unprepare_hclk; + } + ret = v4l2_device_register(dev, &isc->v4l2_dev); + if (ret) { + dev_err(dev, "unable to register v4l2 device.\n"); + goto unprepare_clk; + } + + ret = isc_parse_dt(dev, isc); + if (ret) { + dev_err(dev, "fail to parse device tree\n"); + goto unregister_v4l2_device; + } + + if (list_empty(&isc->subdev_entities)) { + dev_err(dev, "no subdev found\n"); + ret = -ENODEV; + goto unregister_v4l2_device; + } + + list_for_each_entry(subdev_entity, &isc->subdev_entities, list) { + struct v4l2_async_subdev *asd; + struct fwnode_handle *fwnode = + of_fwnode_handle(subdev_entity->epn); + + v4l2_async_nf_init(&subdev_entity->notifier); + + asd = v4l2_async_nf_add_fwnode_remote(&subdev_entity->notifier, + fwnode, + struct v4l2_async_subdev); + + of_node_put(subdev_entity->epn); + subdev_entity->epn = NULL; + + if (IS_ERR(asd)) { + ret = PTR_ERR(asd); + goto cleanup_subdev; + } + + subdev_entity->notifier.ops = µchip_isc_async_ops; + + ret = v4l2_async_nf_register(&isc->v4l2_dev, + &subdev_entity->notifier); + if (ret) { + dev_err(dev, "fail to register async notifier\n"); + goto cleanup_subdev; + } + + if (video_is_registered(&isc->video_dev)) + break; + } + + regmap_read(isc->regmap, ISC_VERSION + isc->offsets.version, &ver); + + ret = isc_mc_init(isc, ver); + if (ret < 0) + goto isc_probe_mc_init_err; + + pm_runtime_set_active(dev); + pm_runtime_enable(dev); + pm_request_idle(dev); + + isc->ispck = isc->isc_clks[ISC_ISPCK].clk; + + ret = clk_prepare_enable(isc->ispck); + if (ret) { + dev_err(dev, "failed to enable ispck: %d\n", ret); + goto disable_pm; + } + + /* ispck should be greater or equal to hclock */ + ret = clk_set_rate(isc->ispck, clk_get_rate(isc->hclock)); + if (ret) { + dev_err(dev, "failed to set ispck rate: %d\n", ret); + goto unprepare_clk; + } + + dev_info(dev, "Microchip ISC version %x\n", ver); + + return 0; + +unprepare_clk: + clk_disable_unprepare(isc->ispck); + +disable_pm: + pm_runtime_disable(dev); + +isc_probe_mc_init_err: + isc_mc_cleanup(isc); + +cleanup_subdev: + microchip_isc_subdev_cleanup(isc); + +unregister_v4l2_device: + v4l2_device_unregister(&isc->v4l2_dev); + +unprepare_hclk: + clk_disable_unprepare(isc->hclock); + + microchip_isc_clk_cleanup(isc); + + return ret; +} + +static int microchip_isc_remove(struct platform_device *pdev) +{ + struct isc_device *isc = platform_get_drvdata(pdev); + + pm_runtime_disable(&pdev->dev); + + isc_mc_cleanup(isc); + + microchip_isc_subdev_cleanup(isc); + + v4l2_device_unregister(&isc->v4l2_dev); + + clk_disable_unprepare(isc->ispck); + clk_disable_unprepare(isc->hclock); + + microchip_isc_clk_cleanup(isc); + + return 0; +} + +static int __maybe_unused isc_runtime_suspend(struct device *dev) +{ + struct isc_device *isc = dev_get_drvdata(dev); + + clk_disable_unprepare(isc->ispck); + clk_disable_unprepare(isc->hclock); + + return 0; +} + +static int __maybe_unused isc_runtime_resume(struct device *dev) +{ + struct isc_device *isc = dev_get_drvdata(dev); + int ret; + + ret = clk_prepare_enable(isc->hclock); + if (ret) + return ret; + + ret = clk_prepare_enable(isc->ispck); + if (ret) + clk_disable_unprepare(isc->hclock); + + return ret; +} + +static const struct dev_pm_ops microchip_isc_dev_pm_ops = { + SET_RUNTIME_PM_OPS(isc_runtime_suspend, isc_runtime_resume, NULL) +}; + +#if IS_ENABLED(CONFIG_OF) +static const struct of_device_id microchip_isc_of_match[] = { + { .compatible = "atmel,sama5d2-isc" }, + { } +}; +MODULE_DEVICE_TABLE(of, microchip_isc_of_match); +#endif + +static struct platform_driver microchip_isc_driver = { + .probe = microchip_isc_probe, + .remove = microchip_isc_remove, + .driver = { + .name = "microchip-sama5d2-isc", + .pm = µchip_isc_dev_pm_ops, + .of_match_table = of_match_ptr(microchip_isc_of_match), + }, +}; + +module_platform_driver(microchip_isc_driver); + +MODULE_AUTHOR("Songjun Wu"); +MODULE_DESCRIPTION("The V4L2 driver for Microchip-ISC"); +MODULE_LICENSE("GPL v2"); diff --git a/drivers/media/platform/microchip/microchip-sama7g5-isc.c b/drivers/media/platform/microchip/microchip-sama7g5-isc.c new file mode 100644 index 000000000000..d583eafe5cc1 --- /dev/null +++ b/drivers/media/platform/microchip/microchip-sama7g5-isc.c @@ -0,0 +1,646 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Microchip eXtended Image Sensor Controller (XISC) driver + * + * Copyright (C) 2019-2021 Microchip Technology, Inc. and its subsidiaries + * + * Author: Eugen Hristev <[email protected]> + * + * Sensor-->PFE-->DPC-->WB-->CFA-->CC-->GAM-->VHXS-->CSC-->CBHS-->SUB-->RLP-->DMA-->HIS + * + * ISC video pipeline integrates the following submodules: + * PFE: Parallel Front End to sample the camera sensor input stream + * DPC: Defective Pixel Correction with black offset correction, green disparity + * correction and defective pixel correction (3 modules total) + * WB: Programmable white balance in the Bayer domain + * CFA: Color filter array interpolation module + * CC: Programmable color correction + * GAM: Gamma correction + *VHXS: Vertical and Horizontal Scaler + * CSC: Programmable color space conversion + *CBHS: Contrast Brightness Hue and Saturation control + * SUB: This module performs YCbCr444 to YCbCr420 chrominance subsampling + * RLP: This module performs rounding, range limiting + * and packing of the incoming data + * DMA: This module performs DMA master accesses to write frames to external RAM + * HIS: Histogram module performs statistic counters on the frames + */ + +#include <linux/clk.h> +#include <linux/clkdev.h> +#include <linux/clk-provider.h> +#include <linux/delay.h> +#include <linux/interrupt.h> +#include <linux/math64.h> +#include <linux/module.h> +#include <linux/of.h> +#include <linux/of_graph.h> +#include <linux/platform_device.h> +#include <linux/pm_runtime.h> +#include <linux/regmap.h> +#include <linux/videodev2.h> + +#include <media/v4l2-ctrls.h> +#include <media/v4l2-device.h> +#include <media/v4l2-event.h> +#include <media/v4l2-image-sizes.h> +#include <media/v4l2-ioctl.h> +#include <media/v4l2-fwnode.h> +#include <media/v4l2-subdev.h> +#include <media/videobuf2-dma-contig.h> + +#include "microchip-isc-regs.h" +#include "microchip-isc.h" + +#define ISC_SAMA7G5_MAX_SUPPORT_WIDTH 3264 +#define ISC_SAMA7G5_MAX_SUPPORT_HEIGHT 2464 + +#define ISC_SAMA7G5_PIPELINE \ + (WB_ENABLE | CFA_ENABLE | CC_ENABLE | GAM_ENABLES | CSC_ENABLE | \ + CBC_ENABLE | SUB422_ENABLE | SUB420_ENABLE) + +/* This is a list of the formats that the ISC can *output* */ +static const struct isc_format sama7g5_controller_formats[] = { + { + .fourcc = V4L2_PIX_FMT_ARGB444, + }, { + .fourcc = V4L2_PIX_FMT_ARGB555, + }, { + .fourcc = V4L2_PIX_FMT_RGB565, + }, { + .fourcc = V4L2_PIX_FMT_ABGR32, + }, { + .fourcc = V4L2_PIX_FMT_XBGR32, + }, { + .fourcc = V4L2_PIX_FMT_YUV420, + }, { + .fourcc = V4L2_PIX_FMT_UYVY, + }, { + .fourcc = V4L2_PIX_FMT_VYUY, + }, { + .fourcc = V4L2_PIX_FMT_YUYV, + }, { + .fourcc = V4L2_PIX_FMT_YUV422P, + }, { + .fourcc = V4L2_PIX_FMT_GREY, + }, { + .fourcc = V4L2_PIX_FMT_Y10, + }, { + .fourcc = V4L2_PIX_FMT_Y16, + }, { + .fourcc = V4L2_PIX_FMT_SBGGR8, + .raw = true, + }, { + .fourcc = V4L2_PIX_FMT_SGBRG8, + .raw = true, + }, { + .fourcc = V4L2_PIX_FMT_SGRBG8, + .raw = true, + }, { + .fourcc = V4L2_PIX_FMT_SRGGB8, + .raw = true, + }, { + .fourcc = V4L2_PIX_FMT_SBGGR10, + .raw = true, + }, { + .fourcc = V4L2_PIX_FMT_SGBRG10, + .raw = true, + }, { + .fourcc = V4L2_PIX_FMT_SGRBG10, + .raw = true, + }, { + .fourcc = V4L2_PIX_FMT_SRGGB10, + .raw = true, + }, { + .fourcc = V4L2_PIX_FMT_SBGGR12, + .raw = true, + }, { + .fourcc = V4L2_PIX_FMT_SGBRG12, + .raw = true, + }, { + .fourcc = V4L2_PIX_FMT_SGRBG12, + .raw = true, + }, { + .fourcc = V4L2_PIX_FMT_SRGGB12, + .raw = true, + }, +}; + +/* This is a list of formats that the ISC can receive as *input* */ +static struct isc_format sama7g5_formats_list[] = { + { + .fourcc = V4L2_PIX_FMT_SBGGR8, + .mbus_code = MEDIA_BUS_FMT_SBGGR8_1X8, + .pfe_cfg0_bps = ISC_PFE_CFG0_BPS_EIGHT, + .cfa_baycfg = ISC_BAY_CFG_BGBG, + }, + { + .fourcc = V4L2_PIX_FMT_SGBRG8, + .mbus_code = MEDIA_BUS_FMT_SGBRG8_1X8, + .pfe_cfg0_bps = ISC_PFE_CFG0_BPS_EIGHT, + .cfa_baycfg = ISC_BAY_CFG_GBGB, + }, + { + .fourcc = V4L2_PIX_FMT_SGRBG8, + .mbus_code = MEDIA_BUS_FMT_SGRBG8_1X8, + .pfe_cfg0_bps = ISC_PFE_CFG0_BPS_EIGHT, + .cfa_baycfg = ISC_BAY_CFG_GRGR, + }, + { + .fourcc = V4L2_PIX_FMT_SRGGB8, + .mbus_code = MEDIA_BUS_FMT_SRGGB8_1X8, + .pfe_cfg0_bps = ISC_PFE_CFG0_BPS_EIGHT, + .cfa_baycfg = ISC_BAY_CFG_RGRG, + }, + { + .fourcc = V4L2_PIX_FMT_SBGGR10, + .mbus_code = MEDIA_BUS_FMT_SBGGR10_1X10, + .pfe_cfg0_bps = ISC_PFG_CFG0_BPS_TEN, + .cfa_baycfg = ISC_BAY_CFG_RGRG, + }, + { + .fourcc = V4L2_PIX_FMT_SGBRG10, + .mbus_code = MEDIA_BUS_FMT_SGBRG10_1X10, + .pfe_cfg0_bps = ISC_PFG_CFG0_BPS_TEN, + .cfa_baycfg = ISC_BAY_CFG_GBGB, + }, + { + .fourcc = V4L2_PIX_FMT_SGRBG10, + .mbus_code = MEDIA_BUS_FMT_SGRBG10_1X10, + .pfe_cfg0_bps = ISC_PFG_CFG0_BPS_TEN, + .cfa_baycfg = ISC_BAY_CFG_GRGR, + }, + { + .fourcc = V4L2_PIX_FMT_SRGGB10, + .mbus_code = MEDIA_BUS_FMT_SRGGB10_1X10, + .pfe_cfg0_bps = ISC_PFG_CFG0_BPS_TEN, + .cfa_baycfg = ISC_BAY_CFG_RGRG, + }, + { + .fourcc = V4L2_PIX_FMT_SBGGR12, + .mbus_code = MEDIA_BUS_FMT_SBGGR12_1X12, + .pfe_cfg0_bps = ISC_PFG_CFG0_BPS_TWELVE, + .cfa_baycfg = ISC_BAY_CFG_BGBG, + }, + { + .fourcc = V4L2_PIX_FMT_SGBRG12, + .mbus_code = MEDIA_BUS_FMT_SGBRG12_1X12, + .pfe_cfg0_bps = ISC_PFG_CFG0_BPS_TWELVE, + .cfa_baycfg = ISC_BAY_CFG_GBGB, + }, + { + .fourcc = V4L2_PIX_FMT_SGRBG12, + .mbus_code = MEDIA_BUS_FMT_SGRBG12_1X12, + .pfe_cfg0_bps = ISC_PFG_CFG0_BPS_TWELVE, + .cfa_baycfg = ISC_BAY_CFG_GRGR, + }, + { + .fourcc = V4L2_PIX_FMT_SRGGB12, + .mbus_code = MEDIA_BUS_FMT_SRGGB12_1X12, + .pfe_cfg0_bps = ISC_PFG_CFG0_BPS_TWELVE, + .cfa_baycfg = ISC_BAY_CFG_RGRG, + }, + { + .fourcc = V4L2_PIX_FMT_GREY, + .mbus_code = MEDIA_BUS_FMT_Y8_1X8, + .pfe_cfg0_bps = ISC_PFE_CFG0_BPS_EIGHT, + }, + { + .fourcc = V4L2_PIX_FMT_YUYV, + .mbus_code = MEDIA_BUS_FMT_YUYV8_2X8, + .pfe_cfg0_bps = ISC_PFE_CFG0_BPS_EIGHT, + }, + { + .fourcc = V4L2_PIX_FMT_UYVY, + .mbus_code = MEDIA_BUS_FMT_UYVY8_2X8, + .pfe_cfg0_bps = ISC_PFE_CFG0_BPS_EIGHT, + }, + { + .fourcc = V4L2_PIX_FMT_RGB565, + .mbus_code = MEDIA_BUS_FMT_RGB565_2X8_LE, + .pfe_cfg0_bps = ISC_PFE_CFG0_BPS_EIGHT, + }, + { + .fourcc = V4L2_PIX_FMT_Y10, + .mbus_code = MEDIA_BUS_FMT_Y10_1X10, + .pfe_cfg0_bps = ISC_PFG_CFG0_BPS_TEN, + }, +}; + +static void isc_sama7g5_config_csc(struct isc_device *isc) +{ + struct regmap *regmap = isc->regmap; + + /* Convert RGB to YUV */ + regmap_write(regmap, ISC_CSC_YR_YG + isc->offsets.csc, + 0x42 | (0x81 << 16)); + regmap_write(regmap, ISC_CSC_YB_OY + isc->offsets.csc, + 0x19 | (0x10 << 16)); + regmap_write(regmap, ISC_CSC_CBR_CBG + isc->offsets.csc, + 0xFDA | (0xFB6 << 16)); + regmap_write(regmap, ISC_CSC_CBB_OCB + isc->offsets.csc, + 0x70 | (0x80 << 16)); + regmap_write(regmap, ISC_CSC_CRR_CRG + isc->offsets.csc, + 0x70 | (0xFA2 << 16)); + regmap_write(regmap, ISC_CSC_CRB_OCR + isc->offsets.csc, + 0xFEE | (0x80 << 16)); +} + +static void isc_sama7g5_config_cbc(struct isc_device *isc) +{ + struct regmap *regmap = isc->regmap; + + /* Configure what is set via v4l2 ctrls */ + regmap_write(regmap, ISC_CBC_BRIGHT + isc->offsets.cbc, isc->ctrls.brightness); + regmap_write(regmap, ISC_CBC_CONTRAST + isc->offsets.cbc, isc->ctrls.contrast); + /* Configure Hue and Saturation as neutral midpoint */ + regmap_write(regmap, ISC_CBCHS_HUE, 0); + regmap_write(regmap, ISC_CBCHS_SAT, (1 << 4)); +} + +static void isc_sama7g5_config_cc(struct isc_device *isc) +{ + struct regmap *regmap = isc->regmap; + + /* Configure each register at the neutral fixed point 1.0 or 0.0 */ + regmap_write(regmap, ISC_CC_RR_RG, (1 << 8)); + regmap_write(regmap, ISC_CC_RB_OR, 0); + regmap_write(regmap, ISC_CC_GR_GG, (1 << 8) << 16); + regmap_write(regmap, ISC_CC_GB_OG, 0); + regmap_write(regmap, ISC_CC_BR_BG, 0); + regmap_write(regmap, ISC_CC_BB_OB, (1 << 8)); +} + +static void isc_sama7g5_config_ctrls(struct isc_device *isc, + const struct v4l2_ctrl_ops *ops) +{ + struct isc_ctrls *ctrls = &isc->ctrls; + struct v4l2_ctrl_handler *hdl = &ctrls->handler; + + ctrls->contrast = 16; + + v4l2_ctrl_new_std(hdl, ops, V4L2_CID_CONTRAST, -2048, 2047, 1, 16); +} + +static void isc_sama7g5_config_dpc(struct isc_device *isc) +{ + u32 bay_cfg = isc->config.sd_format->cfa_baycfg; + struct regmap *regmap = isc->regmap; + + regmap_update_bits(regmap, ISC_DPC_CFG, ISC_DPC_CFG_BLOFF_MASK, + (64 << ISC_DPC_CFG_BLOFF_SHIFT)); + regmap_update_bits(regmap, ISC_DPC_CFG, ISC_DPC_CFG_BAYCFG_MASK, + (bay_cfg << ISC_DPC_CFG_BAYCFG_SHIFT)); +} + +static void isc_sama7g5_config_gam(struct isc_device *isc) +{ + struct regmap *regmap = isc->regmap; + + regmap_update_bits(regmap, ISC_GAM_CTRL, ISC_GAM_CTRL_BIPART, + ISC_GAM_CTRL_BIPART); +} + +static void isc_sama7g5_config_rlp(struct isc_device *isc) +{ + struct regmap *regmap = isc->regmap; + u32 rlp_mode = isc->config.rlp_cfg_mode; + + regmap_update_bits(regmap, ISC_RLP_CFG + isc->offsets.rlp, + ISC_RLP_CFG_MODE_MASK | ISC_RLP_CFG_LSH | + ISC_RLP_CFG_YMODE_MASK, rlp_mode); +} + +static void isc_sama7g5_adapt_pipeline(struct isc_device *isc) +{ + isc->try_config.bits_pipeline &= ISC_SAMA7G5_PIPELINE; +} + +/* Gamma table with gamma 1/2.2 */ +static const u32 isc_sama7g5_gamma_table[][GAMMA_ENTRIES] = { + /* index 0 --> gamma bipartite */ + { + 0x980, 0x4c0320, 0x650260, 0x7801e0, 0x8701a0, 0x940180, + 0xa00160, 0xab0120, 0xb40120, 0xbd0120, 0xc60100, 0xce0100, + 0xd600e0, 0xdd00e0, 0xe400e0, 0xeb00c0, 0xf100c0, 0xf700c0, + 0xfd00c0, 0x10300a0, 0x10800c0, 0x10e00a0, 0x11300a0, 0x11800a0, + 0x11d00a0, 0x12200a0, 0x12700a0, 0x12c0080, 0x13000a0, 0x1350080, + 0x13900a0, 0x13e0080, 0x1420076, 0x17d0062, 0x1ae0054, 0x1d8004a, + 0x1fd0044, 0x21f003e, 0x23e003a, 0x25b0036, 0x2760032, 0x28f0030, + 0x2a7002e, 0x2be002c, 0x2d4002c, 0x2ea0028, 0x2fe0028, 0x3120026, + 0x3250024, 0x3370024, 0x3490022, 0x35a0022, 0x36b0020, 0x37b0020, + 0x38b0020, 0x39b001e, 0x3aa001e, 0x3b9001c, 0x3c7001c, 0x3d5001c, + 0x3e3001c, 0x3f1001c, 0x3ff001a, 0x40c001a }, +}; + +static int xisc_parse_dt(struct device *dev, struct isc_device *isc) +{ + struct device_node *np = dev->of_node; + struct device_node *epn = NULL; + struct isc_subdev_entity *subdev_entity; + unsigned int flags; + int ret; + bool mipi_mode; + + INIT_LIST_HEAD(&isc->subdev_entities); + + mipi_mode = of_property_read_bool(np, "microchip,mipi-mode"); + + while (1) { + struct v4l2_fwnode_endpoint v4l2_epn = { .bus_type = 0 }; + + epn = of_graph_get_next_endpoint(np, epn); + if (!epn) + return 0; + + ret = v4l2_fwnode_endpoint_parse(of_fwnode_handle(epn), + &v4l2_epn); + if (ret) { + ret = -EINVAL; + dev_err(dev, "Could not parse the endpoint\n"); + break; + } + + subdev_entity = devm_kzalloc(dev, sizeof(*subdev_entity), + GFP_KERNEL); + if (!subdev_entity) { + ret = -ENOMEM; + break; + } + subdev_entity->epn = epn; + + flags = v4l2_epn.bus.parallel.flags; + + if (flags & V4L2_MBUS_HSYNC_ACTIVE_LOW) + subdev_entity->pfe_cfg0 = ISC_PFE_CFG0_HPOL_LOW; + + if (flags & V4L2_MBUS_VSYNC_ACTIVE_LOW) + subdev_entity->pfe_cfg0 |= ISC_PFE_CFG0_VPOL_LOW; + + if (flags & V4L2_MBUS_PCLK_SAMPLE_FALLING) + subdev_entity->pfe_cfg0 |= ISC_PFE_CFG0_PPOL_LOW; + + if (v4l2_epn.bus_type == V4L2_MBUS_BT656) + subdev_entity->pfe_cfg0 |= ISC_PFE_CFG0_CCIR_CRC | + ISC_PFE_CFG0_CCIR656; + + if (mipi_mode) + subdev_entity->pfe_cfg0 |= ISC_PFE_CFG0_MIPI; + + list_add_tail(&subdev_entity->list, &isc->subdev_entities); + } + of_node_put(epn); + + return ret; +} + +static int microchip_xisc_probe(struct platform_device *pdev) +{ + struct device *dev = &pdev->dev; + struct isc_device *isc; + struct resource *res; + void __iomem *io_base; + struct isc_subdev_entity *subdev_entity; + int irq; + int ret; + u32 ver; + + isc = devm_kzalloc(dev, sizeof(*isc), GFP_KERNEL); + if (!isc) + return -ENOMEM; + + platform_set_drvdata(pdev, isc); + isc->dev = dev; + + res = platform_get_resource(pdev, IORESOURCE_MEM, 0); + io_base = devm_ioremap_resource(dev, res); + if (IS_ERR(io_base)) + return PTR_ERR(io_base); + + isc->regmap = devm_regmap_init_mmio(dev, io_base, µchip_isc_regmap_config); + if (IS_ERR(isc->regmap)) { + ret = PTR_ERR(isc->regmap); + dev_err(dev, "failed to init register map: %d\n", ret); + return ret; + } + + irq = platform_get_irq(pdev, 0); + if (irq < 0) + return irq; + + ret = devm_request_irq(dev, irq, microchip_isc_interrupt, 0, + "microchip-sama7g5-xisc", isc); + if (ret < 0) { + dev_err(dev, "can't register ISR for IRQ %u (ret=%i)\n", + irq, ret); + return ret; + } + + isc->gamma_table = isc_sama7g5_gamma_table; + isc->gamma_max = 0; + + isc->max_width = ISC_SAMA7G5_MAX_SUPPORT_WIDTH; + isc->max_height = ISC_SAMA7G5_MAX_SUPPORT_HEIGHT; + + isc->config_dpc = isc_sama7g5_config_dpc; + isc->config_csc = isc_sama7g5_config_csc; + isc->config_cbc = isc_sama7g5_config_cbc; + isc->config_cc = isc_sama7g5_config_cc; + isc->config_gam = isc_sama7g5_config_gam; + isc->config_rlp = isc_sama7g5_config_rlp; + isc->config_ctrls = isc_sama7g5_config_ctrls; + + isc->adapt_pipeline = isc_sama7g5_adapt_pipeline; + + isc->offsets.csc = ISC_SAMA7G5_CSC_OFFSET; + isc->offsets.cbc = ISC_SAMA7G5_CBC_OFFSET; + isc->offsets.sub422 = ISC_SAMA7G5_SUB422_OFFSET; + isc->offsets.sub420 = ISC_SAMA7G5_SUB420_OFFSET; + isc->offsets.rlp = ISC_SAMA7G5_RLP_OFFSET; + isc->offsets.his = ISC_SAMA7G5_HIS_OFFSET; + isc->offsets.dma = ISC_SAMA7G5_DMA_OFFSET; + isc->offsets.version = ISC_SAMA7G5_VERSION_OFFSET; + isc->offsets.his_entry = ISC_SAMA7G5_HIS_ENTRY_OFFSET; + + isc->controller_formats = sama7g5_controller_formats; + isc->controller_formats_size = ARRAY_SIZE(sama7g5_controller_formats); + isc->formats_list = sama7g5_formats_list; + isc->formats_list_size = ARRAY_SIZE(sama7g5_formats_list); + + /* sama7g5-isc RAM access port is full AXI4 - 32 bits per beat */ + isc->dcfg = ISC_DCFG_YMBSIZE_BEATS32 | ISC_DCFG_CMBSIZE_BEATS32; + + /* sama7g5-isc : ISPCK does not exist, ISC is clocked by MCK */ + isc->ispck_required = false; + + ret = microchip_isc_pipeline_init(isc); + if (ret) + return ret; + + isc->hclock = devm_clk_get(dev, "hclock"); + if (IS_ERR(isc->hclock)) { + ret = PTR_ERR(isc->hclock); + dev_err(dev, "failed to get hclock: %d\n", ret); + return ret; + } + + ret = clk_prepare_enable(isc->hclock); + if (ret) { + dev_err(dev, "failed to enable hclock: %d\n", ret); + return ret; + } + + ret = microchip_isc_clk_init(isc); + if (ret) { + dev_err(dev, "failed to init isc clock: %d\n", ret); + goto unprepare_hclk; + } + + ret = v4l2_device_register(dev, &isc->v4l2_dev); + if (ret) { + dev_err(dev, "unable to register v4l2 device.\n"); + goto unprepare_hclk; + } + + ret = xisc_parse_dt(dev, isc); + if (ret) { + dev_err(dev, "fail to parse device tree\n"); + goto unregister_v4l2_device; + } + + if (list_empty(&isc->subdev_entities)) { + dev_err(dev, "no subdev found\n"); + ret = -ENODEV; + goto unregister_v4l2_device; + } + + list_for_each_entry(subdev_entity, &isc->subdev_entities, list) { + struct v4l2_async_subdev *asd; + struct fwnode_handle *fwnode = + of_fwnode_handle(subdev_entity->epn); + + v4l2_async_nf_init(&subdev_entity->notifier); + + asd = v4l2_async_nf_add_fwnode_remote(&subdev_entity->notifier, + fwnode, + struct v4l2_async_subdev); + + of_node_put(subdev_entity->epn); + subdev_entity->epn = NULL; + + if (IS_ERR(asd)) { + ret = PTR_ERR(asd); + goto cleanup_subdev; + } + + subdev_entity->notifier.ops = µchip_isc_async_ops; + + ret = v4l2_async_nf_register(&isc->v4l2_dev, + &subdev_entity->notifier); + if (ret) { + dev_err(dev, "fail to register async notifier\n"); + goto cleanup_subdev; + } + + if (video_is_registered(&isc->video_dev)) + break; + } + + regmap_read(isc->regmap, ISC_VERSION + isc->offsets.version, &ver); + + ret = isc_mc_init(isc, ver); + if (ret < 0) + goto isc_probe_mc_init_err; + + pm_runtime_set_active(dev); + pm_runtime_enable(dev); + pm_request_idle(dev); + + dev_info(dev, "Microchip XISC version %x\n", ver); + + return 0; + +isc_probe_mc_init_err: + isc_mc_cleanup(isc); + +cleanup_subdev: + microchip_isc_subdev_cleanup(isc); + +unregister_v4l2_device: + v4l2_device_unregister(&isc->v4l2_dev); + +unprepare_hclk: + clk_disable_unprepare(isc->hclock); + + microchip_isc_clk_cleanup(isc); + + return ret; +} + +static int microchip_xisc_remove(struct platform_device *pdev) +{ + struct isc_device *isc = platform_get_drvdata(pdev); + + pm_runtime_disable(&pdev->dev); + + isc_mc_cleanup(isc); + + microchip_isc_subdev_cleanup(isc); + + v4l2_device_unregister(&isc->v4l2_dev); + + clk_disable_unprepare(isc->hclock); + + microchip_isc_clk_cleanup(isc); + + return 0; +} + +static int __maybe_unused xisc_runtime_suspend(struct device *dev) +{ + struct isc_device *isc = dev_get_drvdata(dev); + + clk_disable_unprepare(isc->hclock); + + return 0; +} + +static int __maybe_unused xisc_runtime_resume(struct device *dev) +{ + struct isc_device *isc = dev_get_drvdata(dev); + int ret; + + ret = clk_prepare_enable(isc->hclock); + if (ret) + return ret; + + return ret; +} + +static const struct dev_pm_ops microchip_xisc_dev_pm_ops = { + SET_RUNTIME_PM_OPS(xisc_runtime_suspend, xisc_runtime_resume, NULL) +}; + +#if IS_ENABLED(CONFIG_OF) +static const struct of_device_id microchip_xisc_of_match[] = { + { .compatible = "microchip,sama7g5-isc" }, + { } +}; +MODULE_DEVICE_TABLE(of, microchip_xisc_of_match); +#endif + +static struct platform_driver microchip_xisc_driver = { + .probe = microchip_xisc_probe, + .remove = microchip_xisc_remove, + .driver = { + .name = "microchip-sama7g5-xisc", + .pm = µchip_xisc_dev_pm_ops, + .of_match_table = of_match_ptr(microchip_xisc_of_match), + }, +}; + +module_platform_driver(microchip_xisc_driver); + +MODULE_AUTHOR("Eugen Hristev <[email protected]>"); +MODULE_DESCRIPTION("The V4L2 driver for Microchip-XISC"); +MODULE_LICENSE("GPL v2"); diff --git a/drivers/media/platform/nxp/imx-jpeg/mxc-jpeg.c b/drivers/media/platform/nxp/imx-jpeg/mxc-jpeg.c index bb2bc28b55ed..6cd015a35f7c 100644 --- a/drivers/media/platform/nxp/imx-jpeg/mxc-jpeg.c +++ b/drivers/media/platform/nxp/imx-jpeg/mxc-jpeg.c @@ -2431,6 +2431,8 @@ static int mxc_jpeg_probe(struct platform_device *pdev) unsigned int slot; of_id = of_match_node(mxc_jpeg_match, dev->of_node); + if (!of_id) + return -ENODEV; mode = *(const int *)of_id->data; jpeg = devm_kzalloc(dev, sizeof(struct mxc_jpeg_dev), GFP_KERNEL); diff --git a/drivers/media/platform/samsung/exynos4-is/fimc-core.c b/drivers/media/platform/samsung/exynos4-is/fimc-core.c index 91cc8d58a663..1791100b6935 100644 --- a/drivers/media/platform/samsung/exynos4-is/fimc-core.c +++ b/drivers/media/platform/samsung/exynos4-is/fimc-core.c @@ -1173,7 +1173,7 @@ int __init fimc_register_driver(void) return platform_driver_register(&fimc_driver); } -void __exit fimc_unregister_driver(void) +void fimc_unregister_driver(void) { platform_driver_unregister(&fimc_driver); } diff --git a/drivers/media/platform/samsung/exynos4-is/media-dev.c b/drivers/media/platform/samsung/exynos4-is/media-dev.c index 383a1e0ab912..2f3071acb9c9 100644 --- a/drivers/media/platform/samsung/exynos4-is/media-dev.c +++ b/drivers/media/platform/samsung/exynos4-is/media-dev.c @@ -1584,7 +1584,11 @@ static int __init fimc_md_init(void) if (ret) return ret; - return platform_driver_register(&fimc_md_driver); + ret = platform_driver_register(&fimc_md_driver); + if (ret) + fimc_unregister_driver(); + + return ret; } static void __exit fimc_md_exit(void) diff --git a/drivers/media/platform/samsung/s5p-mfc/s5p_mfc.c b/drivers/media/platform/samsung/s5p-mfc/s5p_mfc.c index fca5c6405eec..8b08306dabbf 100644 --- a/drivers/media/platform/samsung/s5p-mfc/s5p_mfc.c +++ b/drivers/media/platform/samsung/s5p-mfc/s5p_mfc.c @@ -36,7 +36,7 @@ #define S5P_MFC_ENC_NAME "s5p-mfc-enc" int mfc_debug_level; -module_param_named(debug, mfc_debug_level, int, S_IRUGO | S_IWUSR); +module_param_named(debug, mfc_debug_level, int, 0644); MODULE_PARM_DESC(debug, "Debug level - higher value produces more verbose messages"); static char *mfc_mem_size; @@ -148,11 +148,13 @@ static void s5p_mfc_watchdog(struct timer_list *t) if (test_bit(0, &dev->hw_lock)) atomic_inc(&dev->watchdog_cnt); if (atomic_read(&dev->watchdog_cnt) >= MFC_WATCHDOG_CNT) { - /* This means that hw is busy and no interrupts were + /* + * This means that hw is busy and no interrupts were * generated by hw for the Nth time of running this * watchdog timer. This usually means a serious hw * error. Now it is time to kill all instances and - * reset the MFC. */ + * reset the MFC. + */ mfc_err("Time out during waiting for HW\n"); schedule_work(&dev->watchdog_work); } @@ -172,8 +174,10 @@ static void s5p_mfc_watchdog_worker(struct work_struct *work) dev = container_of(work, struct s5p_mfc_dev, watchdog_work); mfc_err("Driver timeout error handling\n"); - /* Lock the mutex that protects open and release. - * This is necessary as they may load and unload firmware. */ + /* + * Lock the mutex that protects open and release. + * This is necessary as they may load and unload firmware. + */ mutex_locked = mutex_trylock(&dev->mfc_mutex); if (!mutex_locked) mfc_err("Error: some instance may be closing/opening\n"); @@ -197,8 +201,10 @@ static void s5p_mfc_watchdog_worker(struct work_struct *work) /* De-init MFC */ s5p_mfc_deinit_hw(dev); - /* Double check if there is at least one instance running. - * If no instance is in memory than no firmware should be present */ + /* + * Double check if there is at least one instance running. + * If no instance is in memory than no firmware should be present + */ if (dev->num_inst > 0) { ret = s5p_mfc_load_firmware(dev); if (ret) { @@ -260,8 +266,10 @@ static void s5p_mfc_handle_frame_copy_time(struct s5p_mfc_ctx *ctx) return; dec_y_addr = (u32)s5p_mfc_hw_call(dev->mfc_ops, get_dec_y_adr, dev); - /* Copy timestamp / timecode from decoded src to dst and set - appropriate flags. */ + /* + * Copy timestamp / timecode from decoded src to dst and set + * appropriate flags. + */ src_buf = list_entry(ctx->src_queue.next, struct s5p_mfc_buf, list); list_for_each_entry(dst_buf, &ctx->dst_queue, list) { u32 addr = (u32)vb2_dma_contig_plane_dma_addr(&dst_buf->b->vb2_buf, 0); @@ -289,8 +297,10 @@ static void s5p_mfc_handle_frame_copy_time(struct s5p_mfc_ctx *ctx) V4L2_BUF_FLAG_BFRAME; break; default: - /* Don't know how to handle - S5P_FIMV_DECODE_FRAME_OTHER_FRAME. */ + /* + * Don't know how to handle + * S5P_FIMV_DECODE_FRAME_OTHER_FRAME. + */ mfc_debug(2, "Unexpected frame type: %d\n", frame_type); } @@ -322,8 +332,10 @@ static void s5p_mfc_handle_frame_new(struct s5p_mfc_ctx *ctx, unsigned int err) return; } ctx->sequence++; - /* The MFC returns address of the buffer, now we have to - * check which vb2_buffer does it correspond to */ + /* + * The MFC returns address of the buffer, now we have to + * check which vb2_buffer does it correspond to + */ list_for_each_entry(dst_buf, &ctx->dst_queue, list) { u32 addr = (u32)vb2_dma_contig_plane_dma_addr(&dst_buf->b->vb2_buf, 0); @@ -476,8 +488,10 @@ static void s5p_mfc_handle_error(struct s5p_mfc_dev *dev, case MFCINST_FINISHING: case MFCINST_FINISHED: case MFCINST_RUNNING: - /* It is highly probable that an error occurred - * while decoding a frame */ + /* + * It is highly probable that an error occurred + * while decoding a frame + */ clear_work_bit(ctx); ctx->state = MFCINST_ERROR; /* Mark all dst buffers as having an error */ @@ -535,6 +549,7 @@ static void s5p_mfc_handle_seq_done(struct s5p_mfc_ctx *ctx, ctx->codec_mode == S5P_MFC_CODEC_H264_MVC_DEC) && !list_empty(&ctx->src_queue)) { struct s5p_mfc_buf *src_buf; + src_buf = list_entry(ctx->src_queue.next, struct s5p_mfc_buf, list); if (s5p_mfc_hw_call(dev->mfc_ops, get_consumed_stream, @@ -951,7 +966,7 @@ static int s5p_mfc_release(struct file *file) /* * If instance was initialised and not yet freed, * return instance and free resources - */ + */ if (ctx->state != MFCINST_FREE && ctx->state != MFCINST_INIT) { mfc_debug(2, "Has to free instance\n"); s5p_mfc_close_mfc_inst(dev, ctx); @@ -1047,10 +1062,10 @@ static int s5p_mfc_mmap(struct file *file, struct vm_area_struct *vma) int ret; if (offset < DST_QUEUE_OFF_BASE) { - mfc_debug(2, "mmaping source\n"); + mfc_debug(2, "mmapping source\n"); ret = vb2_mmap(&ctx->vq_src, vma); } else { /* capture */ - mfc_debug(2, "mmaping destination\n"); + mfc_debug(2, "mmapping destination\n"); vma->vm_pgoff -= (DST_QUEUE_OFF_BASE >> PAGE_SHIFT); ret = vb2_mmap(&ctx->vq_dst, vma); } @@ -1149,7 +1164,6 @@ static int s5p_mfc_configure_2port_memory(struct s5p_mfc_dev *mfc_dev) bank2_virt = dma_alloc_coherent(mfc_dev->mem_dev[BANK_R_CTX], align_size, &bank2_dma_addr, GFP_KERNEL); if (!bank2_virt) { - mfc_err("Allocating bank2 base failed\n"); s5p_mfc_release_firmware(mfc_dev); device_unregister(mfc_dev->mem_dev[BANK_R_CTX]); device_unregister(mfc_dev->mem_dev[BANK_L_CTX]); @@ -1318,7 +1332,7 @@ static int s5p_mfc_probe(struct platform_device *pdev) /* * Load fails if fs isn't mounted. Try loading anyway. - * _open() will load it, it it fails now. Ignore failure. + * _open() will load it, it fails now. Ignore failure. */ s5p_mfc_load_firmware(dev); @@ -1429,7 +1443,7 @@ static int s5p_mfc_remove(struct platform_device *pdev) * Clear ctx dev pointer to avoid races between s5p_mfc_remove() * and s5p_mfc_release() and s5p_mfc_release() accessing ctx->dev * after s5p_mfc_remove() is run during unbind. - */ + */ mutex_lock(&dev->mfc_mutex); for (i = 0; i < MFC_NUM_CONTEXTS; i++) { ctx = dev->ctx[i]; diff --git a/drivers/media/platform/st/sti/c8sectpfe/c8sectpfe-core.c b/drivers/media/platform/st/sti/c8sectpfe/c8sectpfe-core.c index cefe6b7bfdc4..4c5027a0480d 100644 --- a/drivers/media/platform/st/sti/c8sectpfe/c8sectpfe-core.c +++ b/drivers/media/platform/st/sti/c8sectpfe/c8sectpfe-core.c @@ -24,16 +24,18 @@ #include <linux/module.h> #include <linux/of_gpio.h> #include <linux/of_platform.h> +#include <linux/pinctrl/consumer.h> +#include <linux/pinctrl/pinctrl.h> #include <linux/platform_device.h> -#include <linux/usb.h> #include <linux/slab.h> #include <linux/time.h> +#include <linux/usb.h> #include <linux/wait.h> -#include <linux/pinctrl/pinctrl.h> -#include "c8sectpfe-core.h" #include "c8sectpfe-common.h" +#include "c8sectpfe-core.h" #include "c8sectpfe-debugfs.h" + #include <media/dmxdev.h> #include <media/dvb_demux.h> #include <media/dvb_frontend.h> diff --git a/drivers/media/platform/st/stm32/stm32-dcmi.c b/drivers/media/platform/st/stm32/stm32-dcmi.c index 37458d4d9564..7d393f696bff 100644 --- a/drivers/media/platform/st/stm32/stm32-dcmi.c +++ b/drivers/media/platform/st/stm32/stm32-dcmi.c @@ -1997,10 +1997,8 @@ static int dcmi_probe(struct platform_device *pdev) } dcmi->regs = devm_ioremap_resource(&pdev->dev, dcmi->res); - if (IS_ERR(dcmi->regs)) { - dev_err(&pdev->dev, "Could not map registers\n"); + if (IS_ERR(dcmi->regs)) return PTR_ERR(dcmi->regs); - } mclk = devm_clk_get(&pdev->dev, "mclk"); if (IS_ERR(mclk)) { diff --git a/drivers/media/platform/sunxi/sun6i-csi/Makefile b/drivers/media/platform/sunxi/sun6i-csi/Makefile index e7e315347804..87e7a715140a 100644 --- a/drivers/media/platform/sunxi/sun6i-csi/Makefile +++ b/drivers/media/platform/sunxi/sun6i-csi/Makefile @@ -1,4 +1,4 @@ # SPDX-License-Identifier: GPL-2.0-only -sun6i-csi-y += sun6i_video.o sun6i_csi.o +sun6i-csi-y += sun6i_csi.o sun6i_csi_bridge.o sun6i_csi_capture.o obj-$(CONFIG_VIDEO_SUN6I_CSI) += sun6i-csi.o diff --git a/drivers/media/platform/sunxi/sun6i-csi/sun6i_csi.c b/drivers/media/platform/sunxi/sun6i-csi/sun6i_csi.c index 9119f5e0e05e..e3e6650181c8 100644 --- a/drivers/media/platform/sunxi/sun6i-csi/sun6i_csi.c +++ b/drivers/media/platform/sunxi/sun6i-csi/sun6i_csi.c @@ -1,18 +1,14 @@ // SPDX-License-Identifier: GPL-2.0+ /* * Copyright (c) 2011-2018 Magewell Electronics Co., Ltd. (Nanjing) - * All rights reserved. * Author: Yong Deng <[email protected]> + * Copyright 2021-2022 Bootlin + * Author: Paul Kocialkowski <[email protected]> */ #include <linux/clk.h> -#include <linux/delay.h> -#include <linux/dma-mapping.h> #include <linux/err.h> -#include <linux/fs.h> #include <linux/interrupt.h> -#include <linux/io.h> -#include <linux/ioctl.h> #include <linux/module.h> #include <linux/of.h> #include <linux/of_device.h> @@ -20,561 +16,56 @@ #include <linux/pm_runtime.h> #include <linux/regmap.h> #include <linux/reset.h> -#include <linux/sched.h> -#include <linux/sizes.h> -#include <linux/slab.h> +#include <media/v4l2-device.h> #include <media/v4l2-mc.h> #include "sun6i_csi.h" +#include "sun6i_csi_bridge.h" +#include "sun6i_csi_capture.h" #include "sun6i_csi_reg.h" -/* Helpers */ +/* ISP */ -/* TODO add 10&12 bit YUV, RGB support */ -bool sun6i_csi_is_format_supported(struct sun6i_csi_device *csi_dev, - u32 pixformat, u32 mbus_code) +int sun6i_csi_isp_complete(struct sun6i_csi_device *csi_dev, + struct v4l2_device *v4l2_dev) { - struct sun6i_csi_v4l2 *v4l2 = &csi_dev->v4l2; - - /* - * Some video receivers have the ability to be compatible with - * 8bit and 16bit bus width. - * Identify the media bus format from device tree. - */ - if ((v4l2->v4l2_ep.bus_type == V4L2_MBUS_PARALLEL - || v4l2->v4l2_ep.bus_type == V4L2_MBUS_BT656) - && v4l2->v4l2_ep.bus.parallel.bus_width == 16) { - switch (pixformat) { - case V4L2_PIX_FMT_NV12_16L16: - case V4L2_PIX_FMT_NV12: - case V4L2_PIX_FMT_NV21: - case V4L2_PIX_FMT_NV16: - case V4L2_PIX_FMT_NV61: - case V4L2_PIX_FMT_YUV420: - case V4L2_PIX_FMT_YVU420: - case V4L2_PIX_FMT_YUV422P: - switch (mbus_code) { - case MEDIA_BUS_FMT_UYVY8_1X16: - case MEDIA_BUS_FMT_VYUY8_1X16: - case MEDIA_BUS_FMT_YUYV8_1X16: - case MEDIA_BUS_FMT_YVYU8_1X16: - return true; - default: - dev_dbg(csi_dev->dev, - "Unsupported mbus code: 0x%x\n", - mbus_code); - break; - } - break; - default: - dev_dbg(csi_dev->dev, "Unsupported pixformat: 0x%x\n", - pixformat); - break; - } - return false; - } + if (csi_dev->v4l2_dev && csi_dev->v4l2_dev != v4l2_dev) + return -EINVAL; - switch (pixformat) { - case V4L2_PIX_FMT_SBGGR8: - return (mbus_code == MEDIA_BUS_FMT_SBGGR8_1X8); - case V4L2_PIX_FMT_SGBRG8: - return (mbus_code == MEDIA_BUS_FMT_SGBRG8_1X8); - case V4L2_PIX_FMT_SGRBG8: - return (mbus_code == MEDIA_BUS_FMT_SGRBG8_1X8); - case V4L2_PIX_FMT_SRGGB8: - return (mbus_code == MEDIA_BUS_FMT_SRGGB8_1X8); - case V4L2_PIX_FMT_SBGGR10: - return (mbus_code == MEDIA_BUS_FMT_SBGGR10_1X10); - case V4L2_PIX_FMT_SGBRG10: - return (mbus_code == MEDIA_BUS_FMT_SGBRG10_1X10); - case V4L2_PIX_FMT_SGRBG10: - return (mbus_code == MEDIA_BUS_FMT_SGRBG10_1X10); - case V4L2_PIX_FMT_SRGGB10: - return (mbus_code == MEDIA_BUS_FMT_SRGGB10_1X10); - case V4L2_PIX_FMT_SBGGR12: - return (mbus_code == MEDIA_BUS_FMT_SBGGR12_1X12); - case V4L2_PIX_FMT_SGBRG12: - return (mbus_code == MEDIA_BUS_FMT_SGBRG12_1X12); - case V4L2_PIX_FMT_SGRBG12: - return (mbus_code == MEDIA_BUS_FMT_SGRBG12_1X12); - case V4L2_PIX_FMT_SRGGB12: - return (mbus_code == MEDIA_BUS_FMT_SRGGB12_1X12); - - case V4L2_PIX_FMT_YUYV: - return (mbus_code == MEDIA_BUS_FMT_YUYV8_2X8); - case V4L2_PIX_FMT_YVYU: - return (mbus_code == MEDIA_BUS_FMT_YVYU8_2X8); - case V4L2_PIX_FMT_UYVY: - return (mbus_code == MEDIA_BUS_FMT_UYVY8_2X8); - case V4L2_PIX_FMT_VYUY: - return (mbus_code == MEDIA_BUS_FMT_VYUY8_2X8); - - case V4L2_PIX_FMT_NV12_16L16: - case V4L2_PIX_FMT_NV12: - case V4L2_PIX_FMT_NV21: - case V4L2_PIX_FMT_NV16: - case V4L2_PIX_FMT_NV61: - case V4L2_PIX_FMT_YUV420: - case V4L2_PIX_FMT_YVU420: - case V4L2_PIX_FMT_YUV422P: - switch (mbus_code) { - case MEDIA_BUS_FMT_UYVY8_2X8: - case MEDIA_BUS_FMT_VYUY8_2X8: - case MEDIA_BUS_FMT_YUYV8_2X8: - case MEDIA_BUS_FMT_YVYU8_2X8: - return true; - default: - dev_dbg(csi_dev->dev, "Unsupported mbus code: 0x%x\n", - mbus_code); - break; - } - break; - - case V4L2_PIX_FMT_RGB565: - return (mbus_code == MEDIA_BUS_FMT_RGB565_2X8_LE); - case V4L2_PIX_FMT_RGB565X: - return (mbus_code == MEDIA_BUS_FMT_RGB565_2X8_BE); - - case V4L2_PIX_FMT_JPEG: - return (mbus_code == MEDIA_BUS_FMT_JPEG_1X8); - - default: - dev_dbg(csi_dev->dev, "Unsupported pixformat: 0x%x\n", - pixformat); - break; - } + csi_dev->v4l2_dev = v4l2_dev; + csi_dev->media_dev = v4l2_dev->mdev; - return false; + return sun6i_csi_capture_setup(csi_dev); } -int sun6i_csi_set_power(struct sun6i_csi_device *csi_dev, bool enable) +static int sun6i_csi_isp_detect(struct sun6i_csi_device *csi_dev) { struct device *dev = csi_dev->dev; - struct regmap *regmap = csi_dev->regmap; - int ret; - - if (!enable) { - regmap_update_bits(regmap, CSI_EN_REG, CSI_EN_CSI_EN, 0); - pm_runtime_put(dev); + struct fwnode_handle *handle; + /* + * ISP is not available if not connected via fwnode graph. + * This will also check that the remote parent node is available. + */ + handle = fwnode_graph_get_endpoint_by_id(dev_fwnode(dev), + SUN6I_CSI_PORT_ISP, 0, + FWNODE_GRAPH_ENDPOINT_NEXT); + if (!handle) return 0; - } - - ret = pm_runtime_resume_and_get(dev); - if (ret < 0) - return ret; - - regmap_update_bits(regmap, CSI_EN_REG, CSI_EN_CSI_EN, CSI_EN_CSI_EN); - - return 0; -} - -static enum csi_input_fmt get_csi_input_format(struct sun6i_csi_device *csi_dev, - u32 mbus_code, u32 pixformat) -{ - /* non-YUV */ - if ((mbus_code & 0xF000) != 0x2000) - return CSI_INPUT_FORMAT_RAW; - - switch (pixformat) { - case V4L2_PIX_FMT_YUYV: - case V4L2_PIX_FMT_YVYU: - case V4L2_PIX_FMT_UYVY: - case V4L2_PIX_FMT_VYUY: - return CSI_INPUT_FORMAT_RAW; - default: - break; - } - /* not support YUV420 input format yet */ - dev_dbg(csi_dev->dev, "Select YUV422 as default input format of CSI.\n"); - return CSI_INPUT_FORMAT_YUV422; -} - -static enum csi_output_fmt -get_csi_output_format(struct sun6i_csi_device *csi_dev, u32 pixformat, - u32 field) -{ - bool buf_interlaced = false; - - if (field == V4L2_FIELD_INTERLACED - || field == V4L2_FIELD_INTERLACED_TB - || field == V4L2_FIELD_INTERLACED_BT) - buf_interlaced = true; - - switch (pixformat) { - case V4L2_PIX_FMT_SBGGR8: - case V4L2_PIX_FMT_SGBRG8: - case V4L2_PIX_FMT_SGRBG8: - case V4L2_PIX_FMT_SRGGB8: - return buf_interlaced ? CSI_FRAME_RAW_8 : CSI_FIELD_RAW_8; - case V4L2_PIX_FMT_SBGGR10: - case V4L2_PIX_FMT_SGBRG10: - case V4L2_PIX_FMT_SGRBG10: - case V4L2_PIX_FMT_SRGGB10: - return buf_interlaced ? CSI_FRAME_RAW_10 : CSI_FIELD_RAW_10; - case V4L2_PIX_FMT_SBGGR12: - case V4L2_PIX_FMT_SGBRG12: - case V4L2_PIX_FMT_SGRBG12: - case V4L2_PIX_FMT_SRGGB12: - return buf_interlaced ? CSI_FRAME_RAW_12 : CSI_FIELD_RAW_12; - - case V4L2_PIX_FMT_YUYV: - case V4L2_PIX_FMT_YVYU: - case V4L2_PIX_FMT_UYVY: - case V4L2_PIX_FMT_VYUY: - return buf_interlaced ? CSI_FRAME_RAW_8 : CSI_FIELD_RAW_8; - - case V4L2_PIX_FMT_NV12_16L16: - return buf_interlaced ? CSI_FRAME_MB_YUV420 : - CSI_FIELD_MB_YUV420; - case V4L2_PIX_FMT_NV12: - case V4L2_PIX_FMT_NV21: - return buf_interlaced ? CSI_FRAME_UV_CB_YUV420 : - CSI_FIELD_UV_CB_YUV420; - case V4L2_PIX_FMT_YUV420: - case V4L2_PIX_FMT_YVU420: - return buf_interlaced ? CSI_FRAME_PLANAR_YUV420 : - CSI_FIELD_PLANAR_YUV420; - case V4L2_PIX_FMT_NV16: - case V4L2_PIX_FMT_NV61: - return buf_interlaced ? CSI_FRAME_UV_CB_YUV422 : - CSI_FIELD_UV_CB_YUV422; - case V4L2_PIX_FMT_YUV422P: - return buf_interlaced ? CSI_FRAME_PLANAR_YUV422 : - CSI_FIELD_PLANAR_YUV422; - - case V4L2_PIX_FMT_RGB565: - case V4L2_PIX_FMT_RGB565X: - return buf_interlaced ? CSI_FRAME_RGB565 : CSI_FIELD_RGB565; - - case V4L2_PIX_FMT_JPEG: - return buf_interlaced ? CSI_FRAME_RAW_8 : CSI_FIELD_RAW_8; - - default: - dev_warn(csi_dev->dev, "Unsupported pixformat: 0x%x\n", pixformat); - break; - } + fwnode_handle_put(handle); - return CSI_FIELD_RAW_8; -} - -static enum csi_input_seq get_csi_input_seq(struct sun6i_csi_device *csi_dev, - u32 mbus_code, u32 pixformat) -{ - /* Input sequence does not apply to non-YUV formats */ - if ((mbus_code & 0xF000) != 0x2000) + if (!IS_ENABLED(CONFIG_VIDEO_SUN6I_ISP)) { + dev_warn(dev, + "ISP link is detected but not enabled in kernel config!"); return 0; - - switch (pixformat) { - case V4L2_PIX_FMT_NV12_16L16: - case V4L2_PIX_FMT_NV12: - case V4L2_PIX_FMT_NV16: - case V4L2_PIX_FMT_YUV420: - case V4L2_PIX_FMT_YUV422P: - switch (mbus_code) { - case MEDIA_BUS_FMT_UYVY8_2X8: - case MEDIA_BUS_FMT_UYVY8_1X16: - return CSI_INPUT_SEQ_UYVY; - case MEDIA_BUS_FMT_VYUY8_2X8: - case MEDIA_BUS_FMT_VYUY8_1X16: - return CSI_INPUT_SEQ_VYUY; - case MEDIA_BUS_FMT_YUYV8_2X8: - case MEDIA_BUS_FMT_YUYV8_1X16: - return CSI_INPUT_SEQ_YUYV; - case MEDIA_BUS_FMT_YVYU8_1X16: - case MEDIA_BUS_FMT_YVYU8_2X8: - return CSI_INPUT_SEQ_YVYU; - default: - dev_warn(csi_dev->dev, "Unsupported mbus code: 0x%x\n", - mbus_code); - break; - } - break; - case V4L2_PIX_FMT_NV21: - case V4L2_PIX_FMT_NV61: - case V4L2_PIX_FMT_YVU420: - switch (mbus_code) { - case MEDIA_BUS_FMT_UYVY8_2X8: - case MEDIA_BUS_FMT_UYVY8_1X16: - return CSI_INPUT_SEQ_VYUY; - case MEDIA_BUS_FMT_VYUY8_2X8: - case MEDIA_BUS_FMT_VYUY8_1X16: - return CSI_INPUT_SEQ_UYVY; - case MEDIA_BUS_FMT_YUYV8_2X8: - case MEDIA_BUS_FMT_YUYV8_1X16: - return CSI_INPUT_SEQ_YVYU; - case MEDIA_BUS_FMT_YVYU8_1X16: - case MEDIA_BUS_FMT_YVYU8_2X8: - return CSI_INPUT_SEQ_YUYV; - default: - dev_warn(csi_dev->dev, "Unsupported mbus code: 0x%x\n", - mbus_code); - break; - } - break; - - case V4L2_PIX_FMT_YUYV: - return CSI_INPUT_SEQ_YUYV; - - default: - dev_warn(csi_dev->dev, "Unsupported pixformat: 0x%x, defaulting to YUYV\n", - pixformat); - break; - } - - return CSI_INPUT_SEQ_YUYV; -} - -static void sun6i_csi_setup_bus(struct sun6i_csi_device *csi_dev) -{ - struct v4l2_fwnode_endpoint *endpoint = &csi_dev->v4l2.v4l2_ep; - struct sun6i_csi_config *config = &csi_dev->config; - unsigned char bus_width; - u32 flags; - u32 cfg; - bool input_interlaced = false; - - if (config->field == V4L2_FIELD_INTERLACED - || config->field == V4L2_FIELD_INTERLACED_TB - || config->field == V4L2_FIELD_INTERLACED_BT) - input_interlaced = true; - - bus_width = endpoint->bus.parallel.bus_width; - - regmap_read(csi_dev->regmap, CSI_IF_CFG_REG, &cfg); - - cfg &= ~(CSI_IF_CFG_CSI_IF_MASK | CSI_IF_CFG_MIPI_IF_MASK | - CSI_IF_CFG_IF_DATA_WIDTH_MASK | - CSI_IF_CFG_CLK_POL_MASK | CSI_IF_CFG_VREF_POL_MASK | - CSI_IF_CFG_HREF_POL_MASK | CSI_IF_CFG_FIELD_MASK | - CSI_IF_CFG_SRC_TYPE_MASK); - - if (input_interlaced) - cfg |= CSI_IF_CFG_SRC_TYPE_INTERLACED; - else - cfg |= CSI_IF_CFG_SRC_TYPE_PROGRESSED; - - switch (endpoint->bus_type) { - case V4L2_MBUS_PARALLEL: - cfg |= CSI_IF_CFG_MIPI_IF_CSI; - - flags = endpoint->bus.parallel.flags; - - cfg |= (bus_width == 16) ? CSI_IF_CFG_CSI_IF_YUV422_16BIT : - CSI_IF_CFG_CSI_IF_YUV422_INTLV; - - if (flags & V4L2_MBUS_FIELD_EVEN_LOW) - cfg |= CSI_IF_CFG_FIELD_POSITIVE; - - if (flags & V4L2_MBUS_VSYNC_ACTIVE_LOW) - cfg |= CSI_IF_CFG_VREF_POL_POSITIVE; - if (flags & V4L2_MBUS_HSYNC_ACTIVE_LOW) - cfg |= CSI_IF_CFG_HREF_POL_POSITIVE; - - if (flags & V4L2_MBUS_PCLK_SAMPLE_RISING) - cfg |= CSI_IF_CFG_CLK_POL_FALLING_EDGE; - break; - case V4L2_MBUS_BT656: - cfg |= CSI_IF_CFG_MIPI_IF_CSI; - - flags = endpoint->bus.parallel.flags; - - cfg |= (bus_width == 16) ? CSI_IF_CFG_CSI_IF_BT1120 : - CSI_IF_CFG_CSI_IF_BT656; - - if (flags & V4L2_MBUS_FIELD_EVEN_LOW) - cfg |= CSI_IF_CFG_FIELD_POSITIVE; - - if (flags & V4L2_MBUS_PCLK_SAMPLE_FALLING) - cfg |= CSI_IF_CFG_CLK_POL_FALLING_EDGE; - break; - default: - dev_warn(csi_dev->dev, "Unsupported bus type: %d\n", - endpoint->bus_type); - break; - } - - switch (bus_width) { - case 8: - cfg |= CSI_IF_CFG_IF_DATA_WIDTH_8BIT; - break; - case 10: - cfg |= CSI_IF_CFG_IF_DATA_WIDTH_10BIT; - break; - case 12: - cfg |= CSI_IF_CFG_IF_DATA_WIDTH_12BIT; - break; - case 16: /* No need to configure DATA_WIDTH for 16bit */ - break; - default: - dev_warn(csi_dev->dev, "Unsupported bus width: %u\n", bus_width); - break; - } - - regmap_write(csi_dev->regmap, CSI_IF_CFG_REG, cfg); -} - -static void sun6i_csi_set_format(struct sun6i_csi_device *csi_dev) -{ - struct sun6i_csi_config *config = &csi_dev->config; - u32 cfg; - u32 val; - - regmap_read(csi_dev->regmap, CSI_CH_CFG_REG, &cfg); - - cfg &= ~(CSI_CH_CFG_INPUT_FMT_MASK | - CSI_CH_CFG_OUTPUT_FMT_MASK | CSI_CH_CFG_VFLIP_EN | - CSI_CH_CFG_HFLIP_EN | CSI_CH_CFG_FIELD_SEL_MASK | - CSI_CH_CFG_INPUT_SEQ_MASK); - - val = get_csi_input_format(csi_dev, config->code, - config->pixelformat); - cfg |= CSI_CH_CFG_INPUT_FMT(val); - - val = get_csi_output_format(csi_dev, config->pixelformat, - config->field); - cfg |= CSI_CH_CFG_OUTPUT_FMT(val); - - val = get_csi_input_seq(csi_dev, config->code, - config->pixelformat); - cfg |= CSI_CH_CFG_INPUT_SEQ(val); - - if (config->field == V4L2_FIELD_TOP) - cfg |= CSI_CH_CFG_FIELD_SEL_FIELD0; - else if (config->field == V4L2_FIELD_BOTTOM) - cfg |= CSI_CH_CFG_FIELD_SEL_FIELD1; - else - cfg |= CSI_CH_CFG_FIELD_SEL_BOTH; - - regmap_write(csi_dev->regmap, CSI_CH_CFG_REG, cfg); -} - -static void sun6i_csi_set_window(struct sun6i_csi_device *csi_dev) -{ - struct sun6i_csi_config *config = &csi_dev->config; - u32 bytesperline_y; - u32 bytesperline_c; - int *planar_offset = csi_dev->planar_offset; - u32 width = config->width; - u32 height = config->height; - u32 hor_len = width; - - switch (config->pixelformat) { - case V4L2_PIX_FMT_YUYV: - case V4L2_PIX_FMT_YVYU: - case V4L2_PIX_FMT_UYVY: - case V4L2_PIX_FMT_VYUY: - dev_dbg(csi_dev->dev, - "Horizontal length should be 2 times of width for packed YUV formats!\n"); - hor_len = width * 2; - break; - default: - break; - } - - regmap_write(csi_dev->regmap, CSI_CH_HSIZE_REG, - CSI_CH_HSIZE_HOR_LEN(hor_len) | - CSI_CH_HSIZE_HOR_START(0)); - regmap_write(csi_dev->regmap, CSI_CH_VSIZE_REG, - CSI_CH_VSIZE_VER_LEN(height) | - CSI_CH_VSIZE_VER_START(0)); - - planar_offset[0] = 0; - switch (config->pixelformat) { - case V4L2_PIX_FMT_NV12_16L16: - case V4L2_PIX_FMT_NV12: - case V4L2_PIX_FMT_NV21: - case V4L2_PIX_FMT_NV16: - case V4L2_PIX_FMT_NV61: - bytesperline_y = width; - bytesperline_c = width; - planar_offset[1] = bytesperline_y * height; - planar_offset[2] = -1; - break; - case V4L2_PIX_FMT_YUV420: - case V4L2_PIX_FMT_YVU420: - bytesperline_y = width; - bytesperline_c = width / 2; - planar_offset[1] = bytesperline_y * height; - planar_offset[2] = planar_offset[1] + - bytesperline_c * height / 2; - break; - case V4L2_PIX_FMT_YUV422P: - bytesperline_y = width; - bytesperline_c = width / 2; - planar_offset[1] = bytesperline_y * height; - planar_offset[2] = planar_offset[1] + - bytesperline_c * height; - break; - default: /* raw */ - dev_dbg(csi_dev->dev, - "Calculating pixelformat(0x%x)'s bytesperline as a packed format\n", - config->pixelformat); - bytesperline_y = (sun6i_csi_get_bpp(config->pixelformat) * - config->width) / 8; - bytesperline_c = 0; - planar_offset[1] = -1; - planar_offset[2] = -1; - break; } - regmap_write(csi_dev->regmap, CSI_CH_BUF_LEN_REG, - CSI_CH_BUF_LEN_BUF_LEN_C(bytesperline_c) | - CSI_CH_BUF_LEN_BUF_LEN_Y(bytesperline_y)); -} - -int sun6i_csi_update_config(struct sun6i_csi_device *csi_dev, - struct sun6i_csi_config *config) -{ - if (!config) - return -EINVAL; - - memcpy(&csi_dev->config, config, sizeof(csi_dev->config)); - - sun6i_csi_setup_bus(csi_dev); - sun6i_csi_set_format(csi_dev); - sun6i_csi_set_window(csi_dev); + csi_dev->isp_available = true; return 0; } -void sun6i_csi_update_buf_addr(struct sun6i_csi_device *csi_dev, - dma_addr_t addr) -{ - regmap_write(csi_dev->regmap, CSI_CH_F0_BUFA_REG, - (addr + csi_dev->planar_offset[0]) >> 2); - if (csi_dev->planar_offset[1] != -1) - regmap_write(csi_dev->regmap, CSI_CH_F1_BUFA_REG, - (addr + csi_dev->planar_offset[1]) >> 2); - if (csi_dev->planar_offset[2] != -1) - regmap_write(csi_dev->regmap, CSI_CH_F2_BUFA_REG, - (addr + csi_dev->planar_offset[2]) >> 2); -} - -void sun6i_csi_set_stream(struct sun6i_csi_device *csi_dev, bool enable) -{ - struct regmap *regmap = csi_dev->regmap; - - if (!enable) { - regmap_update_bits(regmap, CSI_CAP_REG, CSI_CAP_CH0_VCAP_ON, 0); - regmap_write(regmap, CSI_CH_INT_EN_REG, 0); - return; - } - - regmap_write(regmap, CSI_CH_INT_STA_REG, 0xFF); - regmap_write(regmap, CSI_CH_INT_EN_REG, - CSI_CH_INT_EN_HB_OF_INT_EN | - CSI_CH_INT_EN_FIFO2_OF_INT_EN | - CSI_CH_INT_EN_FIFO1_OF_INT_EN | - CSI_CH_INT_EN_FIFO0_OF_INT_EN | - CSI_CH_INT_EN_FD_INT_EN | - CSI_CH_INT_EN_CD_INT_EN); - - regmap_update_bits(regmap, CSI_CAP_REG, CSI_CAP_CH0_VCAP_ON, - CSI_CAP_CH0_VCAP_ON); -} - /* Media */ static const struct media_device_ops sun6i_csi_media_ops = { @@ -583,103 +74,11 @@ static const struct media_device_ops sun6i_csi_media_ops = { /* V4L2 */ -static int sun6i_csi_link_entity(struct sun6i_csi_device *csi_dev, - struct media_entity *entity, - struct fwnode_handle *fwnode) -{ - struct media_entity *sink; - struct media_pad *sink_pad; - int src_pad_index; - int ret; - - ret = media_entity_get_fwnode_pad(entity, fwnode, MEDIA_PAD_FL_SOURCE); - if (ret < 0) { - dev_err(csi_dev->dev, - "%s: no source pad in external entity %s\n", __func__, - entity->name); - return -EINVAL; - } - - src_pad_index = ret; - - sink = &csi_dev->video.video_dev.entity; - sink_pad = &csi_dev->video.pad; - - dev_dbg(csi_dev->dev, "creating %s:%u -> %s:%u link\n", - entity->name, src_pad_index, sink->name, sink_pad->index); - ret = media_create_pad_link(entity, src_pad_index, sink, - sink_pad->index, - MEDIA_LNK_FL_ENABLED | - MEDIA_LNK_FL_IMMUTABLE); - if (ret < 0) { - dev_err(csi_dev->dev, "failed to create %s:%u -> %s:%u link\n", - entity->name, src_pad_index, - sink->name, sink_pad->index); - return ret; - } - - return 0; -} - -static int sun6i_subdev_notify_complete(struct v4l2_async_notifier *notifier) -{ - struct sun6i_csi_device *csi_dev = - container_of(notifier, struct sun6i_csi_device, - v4l2.notifier); - struct sun6i_csi_v4l2 *v4l2 = &csi_dev->v4l2; - struct v4l2_device *v4l2_dev = &v4l2->v4l2_dev; - struct v4l2_subdev *sd; - int ret; - - dev_dbg(csi_dev->dev, "notify complete, all subdevs registered\n"); - - sd = list_first_entry(&v4l2_dev->subdevs, struct v4l2_subdev, list); - if (!sd) - return -EINVAL; - - ret = sun6i_csi_link_entity(csi_dev, &sd->entity, sd->fwnode); - if (ret < 0) - return ret; - - ret = v4l2_device_register_subdev_nodes(v4l2_dev); - if (ret < 0) - return ret; - - return 0; -} - -static const struct v4l2_async_notifier_operations sun6i_csi_async_ops = { - .complete = sun6i_subdev_notify_complete, -}; - -static int sun6i_csi_fwnode_parse(struct device *dev, - struct v4l2_fwnode_endpoint *vep, - struct v4l2_async_subdev *asd) -{ - struct sun6i_csi_device *csi_dev = dev_get_drvdata(dev); - - if (vep->base.port || vep->base.id) { - dev_warn(dev, "Only support a single port with one endpoint\n"); - return -ENOTCONN; - } - - switch (vep->bus_type) { - case V4L2_MBUS_PARALLEL: - case V4L2_MBUS_BT656: - csi_dev->v4l2.v4l2_ep = *vep; - return 0; - default: - dev_err(dev, "Unsupported media bus type\n"); - return -ENOTCONN; - } -} - static int sun6i_csi_v4l2_setup(struct sun6i_csi_device *csi_dev) { struct sun6i_csi_v4l2 *v4l2 = &csi_dev->v4l2; struct media_device *media_dev = &v4l2->media_dev; struct v4l2_device *v4l2_dev = &v4l2->v4l2_dev; - struct v4l2_async_notifier *notifier = &v4l2->notifier; struct device *dev = csi_dev->dev; int ret; @@ -709,42 +108,11 @@ static int sun6i_csi_v4l2_setup(struct sun6i_csi_device *csi_dev) goto error_media; } - /* Video */ - - ret = sun6i_video_setup(csi_dev); - if (ret) - goto error_v4l2_device; - - /* V4L2 Async */ - - v4l2_async_nf_init(notifier); - notifier->ops = &sun6i_csi_async_ops; - - ret = v4l2_async_nf_parse_fwnode_endpoints(dev, notifier, - sizeof(struct - v4l2_async_subdev), - sun6i_csi_fwnode_parse); - if (ret) - goto error_video; - - ret = v4l2_async_nf_register(v4l2_dev, notifier); - if (ret) { - dev_err(dev, "failed to register v4l2 async notifier: %d\n", - ret); - goto error_v4l2_async_notifier; - } + csi_dev->v4l2_dev = v4l2_dev; + csi_dev->media_dev = media_dev; return 0; -error_v4l2_async_notifier: - v4l2_async_nf_cleanup(notifier); - -error_video: - sun6i_video_cleanup(csi_dev); - -error_v4l2_device: - v4l2_device_unregister(&v4l2->v4l2_dev); - error_media: media_device_unregister(media_dev); media_device_cleanup(media_dev); @@ -757,9 +125,6 @@ static void sun6i_csi_v4l2_cleanup(struct sun6i_csi_device *csi_dev) struct sun6i_csi_v4l2 *v4l2 = &csi_dev->v4l2; media_device_unregister(&v4l2->media_dev); - v4l2_async_nf_unregister(&v4l2->notifier); - v4l2_async_nf_cleanup(&v4l2->notifier); - sun6i_video_cleanup(csi_dev); v4l2_device_unregister(&v4l2->v4l2_dev); media_device_cleanup(&v4l2->media_dev); } @@ -769,29 +134,39 @@ static void sun6i_csi_v4l2_cleanup(struct sun6i_csi_device *csi_dev) static irqreturn_t sun6i_csi_interrupt(int irq, void *private) { struct sun6i_csi_device *csi_dev = private; + bool capture_streaming = csi_dev->capture.state.streaming; struct regmap *regmap = csi_dev->regmap; - u32 status; + u32 status = 0, enable = 0; - regmap_read(regmap, CSI_CH_INT_STA_REG, &status); + regmap_read(regmap, SUN6I_CSI_CH_INT_STA_REG, &status); + regmap_read(regmap, SUN6I_CSI_CH_INT_EN_REG, &enable); - if (!(status & 0xFF)) + if (!status) return IRQ_NONE; - - if ((status & CSI_CH_INT_STA_FIFO0_OF_PD) || - (status & CSI_CH_INT_STA_FIFO1_OF_PD) || - (status & CSI_CH_INT_STA_FIFO2_OF_PD) || - (status & CSI_CH_INT_STA_HB_OF_PD)) { - regmap_write(regmap, CSI_CH_INT_STA_REG, status); - regmap_update_bits(regmap, CSI_EN_REG, CSI_EN_CSI_EN, 0); - regmap_update_bits(regmap, CSI_EN_REG, CSI_EN_CSI_EN, - CSI_EN_CSI_EN); + else if (!(status & enable) || !capture_streaming) + goto complete; + + if ((status & SUN6I_CSI_CH_INT_STA_FIFO0_OF) || + (status & SUN6I_CSI_CH_INT_STA_FIFO1_OF) || + (status & SUN6I_CSI_CH_INT_STA_FIFO2_OF) || + (status & SUN6I_CSI_CH_INT_STA_HB_OF)) { + regmap_write(regmap, SUN6I_CSI_CH_INT_STA_REG, status); + + regmap_update_bits(regmap, SUN6I_CSI_EN_REG, + SUN6I_CSI_EN_CSI_EN, 0); + regmap_update_bits(regmap, SUN6I_CSI_EN_REG, + SUN6I_CSI_EN_CSI_EN, SUN6I_CSI_EN_CSI_EN); return IRQ_HANDLED; } - if (status & CSI_CH_INT_STA_FD_PD) - sun6i_video_frame_done(csi_dev); + if (status & SUN6I_CSI_CH_INT_STA_FD) + sun6i_csi_capture_frame_done(csi_dev); + + if (status & SUN6I_CSI_CH_INT_STA_VS) + sun6i_csi_capture_sync(csi_dev); - regmap_write(regmap, CSI_CH_INT_STA_REG, status); +complete: + regmap_write(regmap, SUN6I_CSI_CH_INT_STA_REG, status); return IRQ_HANDLED; } @@ -917,8 +292,8 @@ static int sun6i_csi_resources_setup(struct sun6i_csi_device *csi_dev, goto error_clock_rate_exclusive; } - ret = devm_request_irq(dev, irq, sun6i_csi_interrupt, 0, SUN6I_CSI_NAME, - csi_dev); + ret = devm_request_irq(dev, irq, sun6i_csi_interrupt, IRQF_SHARED, + SUN6I_CSI_NAME, csi_dev); if (ret) { dev_err(dev, "failed to request interrupt\n"); goto error_clock_rate_exclusive; @@ -959,12 +334,41 @@ static int sun6i_csi_probe(struct platform_device *platform_dev) if (ret) return ret; - ret = sun6i_csi_v4l2_setup(csi_dev); + ret = sun6i_csi_isp_detect(csi_dev); if (ret) goto error_resources; + /* + * Register our own v4l2 and media devices when there is no ISP around. + * Otherwise the ISP will use async subdev registration with our bridge, + * which will provide v4l2 and media devices that are used to register + * the video interface. + */ + if (!csi_dev->isp_available) { + ret = sun6i_csi_v4l2_setup(csi_dev); + if (ret) + goto error_resources; + } + + ret = sun6i_csi_bridge_setup(csi_dev); + if (ret) + goto error_v4l2; + + if (!csi_dev->isp_available) { + ret = sun6i_csi_capture_setup(csi_dev); + if (ret) + goto error_bridge; + } + return 0; +error_bridge: + sun6i_csi_bridge_cleanup(csi_dev); + +error_v4l2: + if (!csi_dev->isp_available) + sun6i_csi_v4l2_cleanup(csi_dev); + error_resources: sun6i_csi_resources_cleanup(csi_dev); @@ -975,7 +379,12 @@ static int sun6i_csi_remove(struct platform_device *pdev) { struct sun6i_csi_device *csi_dev = platform_get_drvdata(pdev); - sun6i_csi_v4l2_cleanup(csi_dev); + sun6i_csi_capture_cleanup(csi_dev); + sun6i_csi_bridge_cleanup(csi_dev); + + if (!csi_dev->isp_available) + sun6i_csi_v4l2_cleanup(csi_dev); + sun6i_csi_resources_cleanup(csi_dev); return 0; @@ -1029,4 +438,5 @@ module_platform_driver(sun6i_csi_platform_driver); MODULE_DESCRIPTION("Allwinner A31 Camera Sensor Interface driver"); MODULE_AUTHOR("Yong Deng <[email protected]>"); +MODULE_AUTHOR("Paul Kocialkowski <[email protected]>"); MODULE_LICENSE("GPL"); diff --git a/drivers/media/platform/sunxi/sun6i-csi/sun6i_csi.h b/drivers/media/platform/sunxi/sun6i-csi/sun6i_csi.h index bab705678280..bc3f0dae35df 100644 --- a/drivers/media/platform/sunxi/sun6i-csi/sun6i_csi.h +++ b/drivers/media/platform/sunxi/sun6i-csi/sun6i_csi.h @@ -1,166 +1,63 @@ /* SPDX-License-Identifier: GPL-2.0+ */ /* * Copyright (c) 2011-2018 Magewell Electronics Co., Ltd. (Nanjing) - * All rights reserved. * Author: Yong Deng <[email protected]> + * Copyright 2021-2022 Bootlin + * Author: Paul Kocialkowski <[email protected]> */ -#ifndef __SUN6I_CSI_H__ -#define __SUN6I_CSI_H__ +#ifndef _SUN6I_CSI_H_ +#define _SUN6I_CSI_H_ #include <media/v4l2-device.h> -#include <media/v4l2-fwnode.h> #include <media/videobuf2-v4l2.h> -#include "sun6i_video.h" +#include "sun6i_csi_bridge.h" +#include "sun6i_csi_capture.h" #define SUN6I_CSI_NAME "sun6i-csi" #define SUN6I_CSI_DESCRIPTION "Allwinner A31 CSI Device" +enum sun6i_csi_port { + SUN6I_CSI_PORT_PARALLEL = 0, + SUN6I_CSI_PORT_MIPI_CSI2 = 1, + SUN6I_CSI_PORT_ISP = 2, +}; + struct sun6i_csi_buffer { struct vb2_v4l2_buffer v4l2_buffer; struct list_head list; - - dma_addr_t dma_addr; - bool queued_to_csi; -}; - -/** - * struct sun6i_csi_config - configs for sun6i csi - * @pixelformat: v4l2 pixel format (V4L2_PIX_FMT_*) - * @code: media bus format code (MEDIA_BUS_FMT_*) - * @field: used interlacing type (enum v4l2_field) - * @width: frame width - * @height: frame height - */ -struct sun6i_csi_config { - u32 pixelformat; - u32 code; - u32 field; - u32 width; - u32 height; }; struct sun6i_csi_v4l2 { struct v4l2_device v4l2_dev; struct media_device media_dev; - - struct v4l2_async_notifier notifier; - /* video port settings */ - struct v4l2_fwnode_endpoint v4l2_ep; }; struct sun6i_csi_device { struct device *dev; + struct v4l2_device *v4l2_dev; + struct media_device *media_dev; - struct sun6i_csi_config config; struct sun6i_csi_v4l2 v4l2; - struct sun6i_video video; + struct sun6i_csi_bridge bridge; + struct sun6i_csi_capture capture; struct regmap *regmap; struct clk *clock_mod; struct clk *clock_ram; struct reset_control *reset; - int planar_offset[3]; + bool isp_available; }; struct sun6i_csi_variant { unsigned long clock_mod_rate; }; -/** - * sun6i_csi_is_format_supported() - check if the format supported by csi - * @csi_dev: pointer to the csi device - * @pixformat: v4l2 pixel format (V4L2_PIX_FMT_*) - * @mbus_code: media bus format code (MEDIA_BUS_FMT_*) - * - * Return: true if format is supported, false otherwise. - */ -bool sun6i_csi_is_format_supported(struct sun6i_csi_device *csi_dev, - u32 pixformat, u32 mbus_code); - -/** - * sun6i_csi_set_power() - power on/off the csi - * @csi_dev: pointer to the csi device - * @enable: on/off - * - * Return: 0 if successful, error code otherwise. - */ -int sun6i_csi_set_power(struct sun6i_csi_device *csi_dev, bool enable); - -/** - * sun6i_csi_update_config() - update the csi register settings - * @csi_dev: pointer to the csi device - * @config: see struct sun6i_csi_config - * - * Return: 0 if successful, error code otherwise. - */ -int sun6i_csi_update_config(struct sun6i_csi_device *csi_dev, - struct sun6i_csi_config *config); - -/** - * sun6i_csi_update_buf_addr() - update the csi frame buffer address - * @csi_dev: pointer to the csi device - * @addr: frame buffer's physical address - */ -void sun6i_csi_update_buf_addr(struct sun6i_csi_device *csi_dev, - dma_addr_t addr); - -/** - * sun6i_csi_set_stream() - start/stop csi streaming - * @csi_dev: pointer to the csi device - * @enable: start/stop - */ -void sun6i_csi_set_stream(struct sun6i_csi_device *csi_dev, bool enable); - -/* get bpp form v4l2 pixformat */ -static inline int sun6i_csi_get_bpp(unsigned int pixformat) -{ - switch (pixformat) { - case V4L2_PIX_FMT_SBGGR8: - case V4L2_PIX_FMT_SGBRG8: - case V4L2_PIX_FMT_SGRBG8: - case V4L2_PIX_FMT_SRGGB8: - case V4L2_PIX_FMT_JPEG: - return 8; - case V4L2_PIX_FMT_SBGGR10: - case V4L2_PIX_FMT_SGBRG10: - case V4L2_PIX_FMT_SGRBG10: - case V4L2_PIX_FMT_SRGGB10: - return 10; - case V4L2_PIX_FMT_SBGGR12: - case V4L2_PIX_FMT_SGBRG12: - case V4L2_PIX_FMT_SGRBG12: - case V4L2_PIX_FMT_SRGGB12: - case V4L2_PIX_FMT_NV12_16L16: - case V4L2_PIX_FMT_NV12: - case V4L2_PIX_FMT_NV21: - case V4L2_PIX_FMT_YUV420: - case V4L2_PIX_FMT_YVU420: - return 12; - case V4L2_PIX_FMT_YUYV: - case V4L2_PIX_FMT_YVYU: - case V4L2_PIX_FMT_UYVY: - case V4L2_PIX_FMT_VYUY: - case V4L2_PIX_FMT_NV16: - case V4L2_PIX_FMT_NV61: - case V4L2_PIX_FMT_YUV422P: - case V4L2_PIX_FMT_RGB565: - case V4L2_PIX_FMT_RGB565X: - return 16; - case V4L2_PIX_FMT_RGB24: - case V4L2_PIX_FMT_BGR24: - return 24; - case V4L2_PIX_FMT_RGB32: - case V4L2_PIX_FMT_BGR32: - return 32; - default: - WARN(1, "Unsupported pixformat: 0x%x\n", pixformat); - break; - } +/* ISP */ - return 0; -} +int sun6i_csi_isp_complete(struct sun6i_csi_device *csi_dev, + struct v4l2_device *v4l2_dev); -#endif /* __SUN6I_CSI_H__ */ +#endif diff --git a/drivers/media/platform/sunxi/sun6i-csi/sun6i_csi_bridge.c b/drivers/media/platform/sunxi/sun6i-csi/sun6i_csi_bridge.c new file mode 100644 index 000000000000..86d20c1c35ed --- /dev/null +++ b/drivers/media/platform/sunxi/sun6i-csi/sun6i_csi_bridge.c @@ -0,0 +1,868 @@ +// SPDX-License-Identifier: GPL-2.0+ +/* + * Copyright 2021-2022 Bootlin + * Author: Paul Kocialkowski <[email protected]> + */ + +#include <linux/pm_runtime.h> +#include <linux/regmap.h> +#include <media/v4l2-device.h> +#include <media/v4l2-fwnode.h> + +#include "sun6i_csi.h" +#include "sun6i_csi_bridge.h" +#include "sun6i_csi_reg.h" + +/* Helpers */ + +void sun6i_csi_bridge_dimensions(struct sun6i_csi_device *csi_dev, + unsigned int *width, unsigned int *height) +{ + if (width) + *width = csi_dev->bridge.mbus_format.width; + if (height) + *height = csi_dev->bridge.mbus_format.height; +} + +void sun6i_csi_bridge_format(struct sun6i_csi_device *csi_dev, + u32 *mbus_code, u32 *field) +{ + if (mbus_code) + *mbus_code = csi_dev->bridge.mbus_format.code; + if (field) + *field = csi_dev->bridge.mbus_format.field; +} + +/* Format */ + +static const struct sun6i_csi_bridge_format sun6i_csi_bridge_formats[] = { + /* Bayer */ + { + .mbus_code = MEDIA_BUS_FMT_SBGGR8_1X8, + .input_format = SUN6I_CSI_INPUT_FMT_RAW, + }, + { + .mbus_code = MEDIA_BUS_FMT_SGBRG8_1X8, + .input_format = SUN6I_CSI_INPUT_FMT_RAW, + }, + { + .mbus_code = MEDIA_BUS_FMT_SGRBG8_1X8, + .input_format = SUN6I_CSI_INPUT_FMT_RAW, + }, + { + .mbus_code = MEDIA_BUS_FMT_SRGGB8_1X8, + .input_format = SUN6I_CSI_INPUT_FMT_RAW, + }, + { + .mbus_code = MEDIA_BUS_FMT_SBGGR10_1X10, + .input_format = SUN6I_CSI_INPUT_FMT_RAW, + }, + { + .mbus_code = MEDIA_BUS_FMT_SGBRG10_1X10, + .input_format = SUN6I_CSI_INPUT_FMT_RAW, + }, + { + .mbus_code = MEDIA_BUS_FMT_SGRBG10_1X10, + .input_format = SUN6I_CSI_INPUT_FMT_RAW, + }, + { + .mbus_code = MEDIA_BUS_FMT_SRGGB10_1X10, + .input_format = SUN6I_CSI_INPUT_FMT_RAW, + }, + { + .mbus_code = MEDIA_BUS_FMT_SBGGR12_1X12, + .input_format = SUN6I_CSI_INPUT_FMT_RAW, + }, + { + .mbus_code = MEDIA_BUS_FMT_SGBRG12_1X12, + .input_format = SUN6I_CSI_INPUT_FMT_RAW, + }, + { + .mbus_code = MEDIA_BUS_FMT_SGRBG12_1X12, + .input_format = SUN6I_CSI_INPUT_FMT_RAW, + }, + { + .mbus_code = MEDIA_BUS_FMT_SRGGB12_1X12, + .input_format = SUN6I_CSI_INPUT_FMT_RAW, + }, + /* RGB */ + { + .mbus_code = MEDIA_BUS_FMT_RGB565_2X8_LE, + .input_format = SUN6I_CSI_INPUT_FMT_RAW, + }, + { + .mbus_code = MEDIA_BUS_FMT_RGB565_2X8_BE, + .input_format = SUN6I_CSI_INPUT_FMT_RAW, + }, + /* YUV422 */ + { + .mbus_code = MEDIA_BUS_FMT_YUYV8_2X8, + .input_format = SUN6I_CSI_INPUT_FMT_YUV422, + .input_yuv_seq = SUN6I_CSI_INPUT_YUV_SEQ_YUYV, + .input_yuv_seq_invert = SUN6I_CSI_INPUT_YUV_SEQ_YVYU, + }, + { + .mbus_code = MEDIA_BUS_FMT_UYVY8_2X8, + .input_format = SUN6I_CSI_INPUT_FMT_YUV422, + .input_yuv_seq = SUN6I_CSI_INPUT_YUV_SEQ_UYVY, + .input_yuv_seq_invert = SUN6I_CSI_INPUT_YUV_SEQ_VYUY, + }, + { + .mbus_code = MEDIA_BUS_FMT_YVYU8_2X8, + .input_format = SUN6I_CSI_INPUT_FMT_YUV422, + .input_yuv_seq = SUN6I_CSI_INPUT_YUV_SEQ_YVYU, + .input_yuv_seq_invert = SUN6I_CSI_INPUT_YUV_SEQ_YUYV, + }, + { + .mbus_code = MEDIA_BUS_FMT_UYVY8_2X8, + .input_format = SUN6I_CSI_INPUT_FMT_YUV422, + .input_yuv_seq = SUN6I_CSI_INPUT_YUV_SEQ_UYVY, + .input_yuv_seq_invert = SUN6I_CSI_INPUT_YUV_SEQ_VYUY, + }, + { + .mbus_code = MEDIA_BUS_FMT_VYUY8_2X8, + .input_format = SUN6I_CSI_INPUT_FMT_YUV422, + .input_yuv_seq = SUN6I_CSI_INPUT_YUV_SEQ_VYUY, + .input_yuv_seq_invert = SUN6I_CSI_INPUT_YUV_SEQ_UYVY, + }, + { + .mbus_code = MEDIA_BUS_FMT_YUYV8_1X16, + .input_format = SUN6I_CSI_INPUT_FMT_YUV422, + .input_yuv_seq = SUN6I_CSI_INPUT_YUV_SEQ_YUYV, + .input_yuv_seq_invert = SUN6I_CSI_INPUT_YUV_SEQ_YVYU, + }, + { + .mbus_code = MEDIA_BUS_FMT_UYVY8_1X16, + .input_format = SUN6I_CSI_INPUT_FMT_YUV422, + .input_yuv_seq = SUN6I_CSI_INPUT_YUV_SEQ_UYVY, + .input_yuv_seq_invert = SUN6I_CSI_INPUT_YUV_SEQ_VYUY, + }, + { + .mbus_code = MEDIA_BUS_FMT_YVYU8_1X16, + .input_format = SUN6I_CSI_INPUT_FMT_YUV422, + .input_yuv_seq = SUN6I_CSI_INPUT_YUV_SEQ_YVYU, + .input_yuv_seq_invert = SUN6I_CSI_INPUT_YUV_SEQ_YUYV, + }, + { + .mbus_code = MEDIA_BUS_FMT_UYVY8_1X16, + .input_format = SUN6I_CSI_INPUT_FMT_YUV422, + .input_yuv_seq = SUN6I_CSI_INPUT_YUV_SEQ_UYVY, + .input_yuv_seq_invert = SUN6I_CSI_INPUT_YUV_SEQ_VYUY, + }, + { + .mbus_code = MEDIA_BUS_FMT_VYUY8_1X16, + .input_format = SUN6I_CSI_INPUT_FMT_YUV422, + .input_yuv_seq = SUN6I_CSI_INPUT_YUV_SEQ_VYUY, + .input_yuv_seq_invert = SUN6I_CSI_INPUT_YUV_SEQ_UYVY, + }, + /* Compressed */ + { + .mbus_code = MEDIA_BUS_FMT_JPEG_1X8, + .input_format = SUN6I_CSI_INPUT_FMT_RAW, + }, +}; + +const struct sun6i_csi_bridge_format * +sun6i_csi_bridge_format_find(u32 mbus_code) +{ + unsigned int i; + + for (i = 0; i < ARRAY_SIZE(sun6i_csi_bridge_formats); i++) + if (sun6i_csi_bridge_formats[i].mbus_code == mbus_code) + return &sun6i_csi_bridge_formats[i]; + + return NULL; +} + +/* Bridge */ + +static void sun6i_csi_bridge_irq_enable(struct sun6i_csi_device *csi_dev) +{ + struct regmap *regmap = csi_dev->regmap; + + regmap_write(regmap, SUN6I_CSI_CH_INT_EN_REG, + SUN6I_CSI_CH_INT_EN_VS | + SUN6I_CSI_CH_INT_EN_HB_OF | + SUN6I_CSI_CH_INT_EN_FIFO2_OF | + SUN6I_CSI_CH_INT_EN_FIFO1_OF | + SUN6I_CSI_CH_INT_EN_FIFO0_OF | + SUN6I_CSI_CH_INT_EN_FD | + SUN6I_CSI_CH_INT_EN_CD); +} + +static void sun6i_csi_bridge_irq_disable(struct sun6i_csi_device *csi_dev) +{ + struct regmap *regmap = csi_dev->regmap; + + regmap_write(regmap, SUN6I_CSI_CH_INT_EN_REG, 0); +} + +static void sun6i_csi_bridge_irq_clear(struct sun6i_csi_device *csi_dev) +{ + struct regmap *regmap = csi_dev->regmap; + + regmap_write(regmap, SUN6I_CSI_CH_INT_EN_REG, 0); + regmap_write(regmap, SUN6I_CSI_CH_INT_STA_REG, + SUN6I_CSI_CH_INT_STA_CLEAR); +} + +static void sun6i_csi_bridge_enable(struct sun6i_csi_device *csi_dev) +{ + struct regmap *regmap = csi_dev->regmap; + + regmap_update_bits(regmap, SUN6I_CSI_EN_REG, SUN6I_CSI_EN_CSI_EN, + SUN6I_CSI_EN_CSI_EN); + + regmap_update_bits(regmap, SUN6I_CSI_CAP_REG, SUN6I_CSI_CAP_VCAP_ON, + SUN6I_CSI_CAP_VCAP_ON); +} + +static void sun6i_csi_bridge_disable(struct sun6i_csi_device *csi_dev) +{ + struct regmap *regmap = csi_dev->regmap; + + regmap_update_bits(regmap, SUN6I_CSI_CAP_REG, SUN6I_CSI_CAP_VCAP_ON, 0); + regmap_update_bits(regmap, SUN6I_CSI_EN_REG, SUN6I_CSI_EN_CSI_EN, 0); +} + +static void +sun6i_csi_bridge_configure_parallel(struct sun6i_csi_device *csi_dev) +{ + struct device *dev = csi_dev->dev; + struct regmap *regmap = csi_dev->regmap; + struct v4l2_fwnode_endpoint *endpoint = + &csi_dev->bridge.source_parallel.endpoint; + unsigned char bus_width = endpoint->bus.parallel.bus_width; + unsigned int flags = endpoint->bus.parallel.flags; + u32 field; + u32 value = SUN6I_CSI_IF_CFG_IF_CSI; + + sun6i_csi_bridge_format(csi_dev, NULL, &field); + + if (field == V4L2_FIELD_INTERLACED || + field == V4L2_FIELD_INTERLACED_TB || + field == V4L2_FIELD_INTERLACED_BT) + value |= SUN6I_CSI_IF_CFG_SRC_TYPE_INTERLACED | + SUN6I_CSI_IF_CFG_FIELD_DT_PCLK_SHIFT(1) | + SUN6I_CSI_IF_CFG_FIELD_DT_FIELD_VSYNC; + else + value |= SUN6I_CSI_IF_CFG_SRC_TYPE_PROGRESSIVE; + + switch (endpoint->bus_type) { + case V4L2_MBUS_PARALLEL: + if (bus_width == 16) + value |= SUN6I_CSI_IF_CFG_IF_CSI_YUV_COMBINED; + else + value |= SUN6I_CSI_IF_CFG_IF_CSI_YUV_RAW; + + if (flags & V4L2_MBUS_FIELD_EVEN_LOW) + value |= SUN6I_CSI_IF_CFG_FIELD_NEGATIVE; + else + value |= SUN6I_CSI_IF_CFG_FIELD_POSITIVE; + + if (flags & V4L2_MBUS_VSYNC_ACTIVE_LOW) + value |= SUN6I_CSI_IF_CFG_VREF_POL_NEGATIVE; + else + value |= SUN6I_CSI_IF_CFG_VREF_POL_POSITIVE; + + if (flags & V4L2_MBUS_HSYNC_ACTIVE_LOW) + value |= SUN6I_CSI_IF_CFG_HREF_POL_NEGATIVE; + else + value |= SUN6I_CSI_IF_CFG_HREF_POL_POSITIVE; + + if (flags & V4L2_MBUS_PCLK_SAMPLE_RISING) + value |= SUN6I_CSI_IF_CFG_CLK_POL_RISING; + else + value |= SUN6I_CSI_IF_CFG_CLK_POL_FALLING; + break; + case V4L2_MBUS_BT656: + if (bus_width == 16) + value |= SUN6I_CSI_IF_CFG_IF_CSI_BT1120; + else + value |= SUN6I_CSI_IF_CFG_IF_CSI_BT656; + + if (flags & V4L2_MBUS_FIELD_EVEN_LOW) + value |= SUN6I_CSI_IF_CFG_FIELD_NEGATIVE; + else + value |= SUN6I_CSI_IF_CFG_FIELD_POSITIVE; + + if (flags & V4L2_MBUS_PCLK_SAMPLE_FALLING) + value |= SUN6I_CSI_IF_CFG_CLK_POL_RISING; + else + value |= SUN6I_CSI_IF_CFG_CLK_POL_FALLING; + break; + default: + dev_warn(dev, "unsupported bus type: %d\n", endpoint->bus_type); + break; + } + + switch (bus_width) { + case 8: + /* 16-bit YUV formats use a doubled width in 8-bit mode. */ + case 16: + value |= SUN6I_CSI_IF_CFG_DATA_WIDTH_8; + break; + case 10: + value |= SUN6I_CSI_IF_CFG_DATA_WIDTH_10; + break; + case 12: + value |= SUN6I_CSI_IF_CFG_DATA_WIDTH_12; + break; + default: + dev_warn(dev, "unsupported bus width: %u\n", bus_width); + break; + } + + regmap_write(regmap, SUN6I_CSI_IF_CFG_REG, value); +} + +static void +sun6i_csi_bridge_configure_mipi_csi2(struct sun6i_csi_device *csi_dev) +{ + struct regmap *regmap = csi_dev->regmap; + u32 value = SUN6I_CSI_IF_CFG_IF_MIPI; + u32 field; + + sun6i_csi_bridge_format(csi_dev, NULL, &field); + + if (field == V4L2_FIELD_INTERLACED || + field == V4L2_FIELD_INTERLACED_TB || + field == V4L2_FIELD_INTERLACED_BT) + value |= SUN6I_CSI_IF_CFG_SRC_TYPE_INTERLACED; + else + value |= SUN6I_CSI_IF_CFG_SRC_TYPE_PROGRESSIVE; + + regmap_write(regmap, SUN6I_CSI_IF_CFG_REG, value); +} + +static void sun6i_csi_bridge_configure_format(struct sun6i_csi_device *csi_dev) +{ + struct regmap *regmap = csi_dev->regmap; + bool capture_streaming = csi_dev->capture.state.streaming; + const struct sun6i_csi_bridge_format *bridge_format; + const struct sun6i_csi_capture_format *capture_format; + u32 mbus_code, field, pixelformat; + u8 input_format, input_yuv_seq, output_format; + u32 value = 0; + + sun6i_csi_bridge_format(csi_dev, &mbus_code, &field); + + bridge_format = sun6i_csi_bridge_format_find(mbus_code); + if (WARN_ON(!bridge_format)) + return; + + input_format = bridge_format->input_format; + input_yuv_seq = bridge_format->input_yuv_seq; + + if (capture_streaming) { + sun6i_csi_capture_format(csi_dev, &pixelformat, NULL); + + capture_format = sun6i_csi_capture_format_find(pixelformat); + if (WARN_ON(!capture_format)) + return; + + if (capture_format->input_format_raw) + input_format = SUN6I_CSI_INPUT_FMT_RAW; + + if (capture_format->input_yuv_seq_invert) + input_yuv_seq = bridge_format->input_yuv_seq_invert; + + if (field == V4L2_FIELD_INTERLACED || + field == V4L2_FIELD_INTERLACED_TB || + field == V4L2_FIELD_INTERLACED_BT) + output_format = capture_format->output_format_field; + else + output_format = capture_format->output_format_frame; + + value |= SUN6I_CSI_CH_CFG_OUTPUT_FMT(output_format); + } + + value |= SUN6I_CSI_CH_CFG_INPUT_FMT(input_format); + value |= SUN6I_CSI_CH_CFG_INPUT_YUV_SEQ(input_yuv_seq); + + if (field == V4L2_FIELD_TOP) + value |= SUN6I_CSI_CH_CFG_FIELD_SEL_FIELD0; + else if (field == V4L2_FIELD_BOTTOM) + value |= SUN6I_CSI_CH_CFG_FIELD_SEL_FIELD1; + else + value |= SUN6I_CSI_CH_CFG_FIELD_SEL_EITHER; + + regmap_write(regmap, SUN6I_CSI_CH_CFG_REG, value); +} + +static void sun6i_csi_bridge_configure(struct sun6i_csi_device *csi_dev, + struct sun6i_csi_bridge_source *source) +{ + struct sun6i_csi_bridge *bridge = &csi_dev->bridge; + + if (source == &bridge->source_parallel) + sun6i_csi_bridge_configure_parallel(csi_dev); + else + sun6i_csi_bridge_configure_mipi_csi2(csi_dev); + + sun6i_csi_bridge_configure_format(csi_dev); +} + +/* V4L2 Subdev */ + +static int sun6i_csi_bridge_s_stream(struct v4l2_subdev *subdev, int on) +{ + struct sun6i_csi_device *csi_dev = v4l2_get_subdevdata(subdev); + struct sun6i_csi_bridge *bridge = &csi_dev->bridge; + struct media_pad *local_pad = &bridge->pads[SUN6I_CSI_BRIDGE_PAD_SINK]; + bool capture_streaming = csi_dev->capture.state.streaming; + struct device *dev = csi_dev->dev; + struct sun6i_csi_bridge_source *source; + struct v4l2_subdev *source_subdev; + struct media_pad *remote_pad; + /* Initialize to 0 to use both in disable label (ret != 0) and off. */ + int ret = 0; + + /* Source */ + + remote_pad = media_pad_remote_pad_unique(local_pad); + if (IS_ERR(remote_pad)) { + dev_err(dev, + "zero or more than a single source connected to the bridge\n"); + return PTR_ERR(remote_pad); + } + + source_subdev = media_entity_to_v4l2_subdev(remote_pad->entity); + + if (source_subdev == bridge->source_parallel.subdev) + source = &bridge->source_parallel; + else + source = &bridge->source_mipi_csi2; + + if (!on) { + v4l2_subdev_call(source_subdev, video, s_stream, 0); + goto disable; + } + + /* PM */ + + ret = pm_runtime_resume_and_get(dev); + if (ret < 0) + return ret; + + /* Clear */ + + sun6i_csi_bridge_irq_clear(csi_dev); + + /* Configure */ + + sun6i_csi_bridge_configure(csi_dev, source); + + if (capture_streaming) + sun6i_csi_capture_configure(csi_dev); + + /* State Update */ + + if (capture_streaming) + sun6i_csi_capture_state_update(csi_dev); + + /* Enable */ + + if (capture_streaming) + sun6i_csi_bridge_irq_enable(csi_dev); + + sun6i_csi_bridge_enable(csi_dev); + + ret = v4l2_subdev_call(source_subdev, video, s_stream, 1); + if (ret && ret != -ENOIOCTLCMD) + goto disable; + + return 0; + +disable: + if (capture_streaming) + sun6i_csi_bridge_irq_disable(csi_dev); + + sun6i_csi_bridge_disable(csi_dev); + + pm_runtime_put(dev); + + return ret; +} + +static const struct v4l2_subdev_video_ops sun6i_csi_bridge_video_ops = { + .s_stream = sun6i_csi_bridge_s_stream, +}; + +static void +sun6i_csi_bridge_mbus_format_prepare(struct v4l2_mbus_framefmt *mbus_format) +{ + if (!sun6i_csi_bridge_format_find(mbus_format->code)) + mbus_format->code = sun6i_csi_bridge_formats[0].mbus_code; + + mbus_format->field = V4L2_FIELD_NONE; + mbus_format->colorspace = V4L2_COLORSPACE_RAW; + mbus_format->quantization = V4L2_QUANTIZATION_DEFAULT; + mbus_format->xfer_func = V4L2_XFER_FUNC_DEFAULT; +} + +static int sun6i_csi_bridge_init_cfg(struct v4l2_subdev *subdev, + struct v4l2_subdev_state *state) +{ + struct sun6i_csi_device *csi_dev = v4l2_get_subdevdata(subdev); + unsigned int pad = SUN6I_CSI_BRIDGE_PAD_SINK; + struct v4l2_mbus_framefmt *mbus_format = + v4l2_subdev_get_try_format(subdev, state, pad); + struct mutex *lock = &csi_dev->bridge.lock; + + mutex_lock(lock); + + mbus_format->code = sun6i_csi_bridge_formats[0].mbus_code; + mbus_format->width = 1280; + mbus_format->height = 720; + + sun6i_csi_bridge_mbus_format_prepare(mbus_format); + + mutex_unlock(lock); + + return 0; +} + +static int +sun6i_csi_bridge_enum_mbus_code(struct v4l2_subdev *subdev, + struct v4l2_subdev_state *state, + struct v4l2_subdev_mbus_code_enum *code_enum) +{ + if (code_enum->index >= ARRAY_SIZE(sun6i_csi_bridge_formats)) + return -EINVAL; + + code_enum->code = sun6i_csi_bridge_formats[code_enum->index].mbus_code; + + return 0; +} + +static int sun6i_csi_bridge_get_fmt(struct v4l2_subdev *subdev, + struct v4l2_subdev_state *state, + struct v4l2_subdev_format *format) +{ + struct sun6i_csi_device *csi_dev = v4l2_get_subdevdata(subdev); + struct v4l2_mbus_framefmt *mbus_format = &format->format; + struct mutex *lock = &csi_dev->bridge.lock; + + mutex_lock(lock); + + if (format->which == V4L2_SUBDEV_FORMAT_TRY) + *mbus_format = *v4l2_subdev_get_try_format(subdev, state, + format->pad); + else + *mbus_format = csi_dev->bridge.mbus_format; + + mutex_unlock(lock); + + return 0; +} + +static int sun6i_csi_bridge_set_fmt(struct v4l2_subdev *subdev, + struct v4l2_subdev_state *state, + struct v4l2_subdev_format *format) +{ + struct sun6i_csi_device *csi_dev = v4l2_get_subdevdata(subdev); + struct v4l2_mbus_framefmt *mbus_format = &format->format; + struct mutex *lock = &csi_dev->bridge.lock; + + mutex_lock(lock); + + sun6i_csi_bridge_mbus_format_prepare(mbus_format); + + if (format->which == V4L2_SUBDEV_FORMAT_TRY) + *v4l2_subdev_get_try_format(subdev, state, format->pad) = + *mbus_format; + else + csi_dev->bridge.mbus_format = *mbus_format; + + mutex_unlock(lock); + + return 0; +} + +static const struct v4l2_subdev_pad_ops sun6i_csi_bridge_pad_ops = { + .init_cfg = sun6i_csi_bridge_init_cfg, + .enum_mbus_code = sun6i_csi_bridge_enum_mbus_code, + .get_fmt = sun6i_csi_bridge_get_fmt, + .set_fmt = sun6i_csi_bridge_set_fmt, +}; + +const struct v4l2_subdev_ops sun6i_csi_bridge_subdev_ops = { + .video = &sun6i_csi_bridge_video_ops, + .pad = &sun6i_csi_bridge_pad_ops, +}; + +/* Media Entity */ + +static const struct media_entity_operations sun6i_csi_bridge_entity_ops = { + .link_validate = v4l2_subdev_link_validate, +}; + +/* V4L2 Async */ + +static int sun6i_csi_bridge_link(struct sun6i_csi_device *csi_dev, + int sink_pad_index, + struct v4l2_subdev *remote_subdev, + bool enabled) +{ + struct device *dev = csi_dev->dev; + struct v4l2_subdev *subdev = &csi_dev->bridge.subdev; + struct media_entity *sink_entity = &subdev->entity; + struct media_entity *source_entity = &remote_subdev->entity; + int source_pad_index; + int ret; + + /* Get the first remote source pad. */ + ret = media_entity_get_fwnode_pad(source_entity, remote_subdev->fwnode, + MEDIA_PAD_FL_SOURCE); + if (ret < 0) { + dev_err(dev, "missing source pad in external entity %s\n", + source_entity->name); + return -EINVAL; + } + + source_pad_index = ret; + + dev_dbg(dev, "creating %s:%u -> %s:%u link\n", source_entity->name, + source_pad_index, sink_entity->name, sink_pad_index); + + ret = media_create_pad_link(source_entity, source_pad_index, + sink_entity, sink_pad_index, + enabled ? MEDIA_LNK_FL_ENABLED : 0); + if (ret < 0) { + dev_err(dev, "failed to create %s:%u -> %s:%u link\n", + source_entity->name, source_pad_index, + sink_entity->name, sink_pad_index); + return ret; + } + + return 0; +} + +static int +sun6i_csi_bridge_notifier_bound(struct v4l2_async_notifier *notifier, + struct v4l2_subdev *remote_subdev, + struct v4l2_async_subdev *async_subdev) +{ + struct sun6i_csi_device *csi_dev = + container_of(notifier, struct sun6i_csi_device, + bridge.notifier); + struct sun6i_csi_bridge_async_subdev *bridge_async_subdev = + container_of(async_subdev, struct sun6i_csi_bridge_async_subdev, + async_subdev); + struct sun6i_csi_bridge *bridge = &csi_dev->bridge; + struct sun6i_csi_bridge_source *source = bridge_async_subdev->source; + bool enabled; + int ret; + + switch (source->endpoint.base.port) { + case SUN6I_CSI_PORT_PARALLEL: + enabled = true; + break; + case SUN6I_CSI_PORT_MIPI_CSI2: + enabled = !bridge->source_parallel.expected; + break; + default: + break; + } + + source->subdev = remote_subdev; + + if (csi_dev->isp_available) { + /* + * Hook to the first available remote subdev to get v4l2 and + * media devices and register the capture device then. + */ + ret = sun6i_csi_isp_complete(csi_dev, remote_subdev->v4l2_dev); + if (ret) + return ret; + } + + return sun6i_csi_bridge_link(csi_dev, SUN6I_CSI_BRIDGE_PAD_SINK, + remote_subdev, enabled); +} + +static int +sun6i_csi_bridge_notifier_complete(struct v4l2_async_notifier *notifier) +{ + struct sun6i_csi_device *csi_dev = + container_of(notifier, struct sun6i_csi_device, + bridge.notifier); + struct v4l2_device *v4l2_dev = &csi_dev->v4l2.v4l2_dev; + + if (csi_dev->isp_available) + return 0; + + return v4l2_device_register_subdev_nodes(v4l2_dev); +} + +static const struct v4l2_async_notifier_operations +sun6i_csi_bridge_notifier_ops = { + .bound = sun6i_csi_bridge_notifier_bound, + .complete = sun6i_csi_bridge_notifier_complete, +}; + +/* Bridge */ + +static int sun6i_csi_bridge_source_setup(struct sun6i_csi_device *csi_dev, + struct sun6i_csi_bridge_source *source, + u32 port, + enum v4l2_mbus_type *bus_types) +{ + struct device *dev = csi_dev->dev; + struct v4l2_async_notifier *notifier = &csi_dev->bridge.notifier; + struct v4l2_fwnode_endpoint *endpoint = &source->endpoint; + struct sun6i_csi_bridge_async_subdev *bridge_async_subdev; + struct fwnode_handle *handle; + int ret; + + handle = fwnode_graph_get_endpoint_by_id(dev_fwnode(dev), port, 0, 0); + if (!handle) + return -ENODEV; + + ret = v4l2_fwnode_endpoint_parse(handle, endpoint); + if (ret) + goto complete; + + if (bus_types) { + bool valid = false; + unsigned int i; + + for (i = 0; bus_types[i] != V4L2_MBUS_INVALID; i++) { + if (endpoint->bus_type == bus_types[i]) { + valid = true; + break; + } + } + + if (!valid) { + dev_err(dev, "unsupported bus type for port %d\n", + port); + ret = -EINVAL; + goto complete; + } + } + + bridge_async_subdev = + v4l2_async_nf_add_fwnode_remote(notifier, handle, + struct + sun6i_csi_bridge_async_subdev); + if (IS_ERR(bridge_async_subdev)) { + ret = PTR_ERR(bridge_async_subdev); + goto complete; + } + + bridge_async_subdev->source = source; + + source->expected = true; + +complete: + fwnode_handle_put(handle); + + return ret; +} + +int sun6i_csi_bridge_setup(struct sun6i_csi_device *csi_dev) +{ + struct device *dev = csi_dev->dev; + struct sun6i_csi_bridge *bridge = &csi_dev->bridge; + struct v4l2_device *v4l2_dev = csi_dev->v4l2_dev; + struct v4l2_subdev *subdev = &bridge->subdev; + struct v4l2_async_notifier *notifier = &bridge->notifier; + struct media_pad *pads = bridge->pads; + enum v4l2_mbus_type parallel_mbus_types[] = { + V4L2_MBUS_PARALLEL, + V4L2_MBUS_BT656, + V4L2_MBUS_INVALID + }; + int ret; + + mutex_init(&bridge->lock); + + /* V4L2 Subdev */ + + v4l2_subdev_init(subdev, &sun6i_csi_bridge_subdev_ops); + strscpy(subdev->name, SUN6I_CSI_BRIDGE_NAME, sizeof(subdev->name)); + subdev->flags |= V4L2_SUBDEV_FL_HAS_DEVNODE; + subdev->owner = THIS_MODULE; + subdev->dev = dev; + + v4l2_set_subdevdata(subdev, csi_dev); + + /* Media Entity */ + + subdev->entity.function = MEDIA_ENT_F_VID_IF_BRIDGE; + subdev->entity.ops = &sun6i_csi_bridge_entity_ops; + + /* Media Pads */ + + pads[SUN6I_CSI_BRIDGE_PAD_SINK].flags = MEDIA_PAD_FL_SINK; + pads[SUN6I_CSI_BRIDGE_PAD_SOURCE].flags = MEDIA_PAD_FL_SOURCE | + MEDIA_PAD_FL_MUST_CONNECT; + + ret = media_entity_pads_init(&subdev->entity, + SUN6I_CSI_BRIDGE_PAD_COUNT, pads); + if (ret < 0) + return ret; + + /* V4L2 Subdev */ + + if (csi_dev->isp_available) + ret = v4l2_async_register_subdev(subdev); + else + ret = v4l2_device_register_subdev(v4l2_dev, subdev); + + if (ret) { + dev_err(dev, "failed to register v4l2 subdev: %d\n", ret); + goto error_media_entity; + } + + /* V4L2 Async */ + + v4l2_async_nf_init(notifier); + notifier->ops = &sun6i_csi_bridge_notifier_ops; + + sun6i_csi_bridge_source_setup(csi_dev, &bridge->source_parallel, + SUN6I_CSI_PORT_PARALLEL, + parallel_mbus_types); + sun6i_csi_bridge_source_setup(csi_dev, &bridge->source_mipi_csi2, + SUN6I_CSI_PORT_MIPI_CSI2, NULL); + + if (csi_dev->isp_available) + ret = v4l2_async_subdev_nf_register(subdev, notifier); + else + ret = v4l2_async_nf_register(v4l2_dev, notifier); + if (ret) { + dev_err(dev, "failed to register v4l2 async notifier: %d\n", + ret); + goto error_v4l2_async_notifier; + } + + return 0; + +error_v4l2_async_notifier: + v4l2_async_nf_cleanup(notifier); + + if (csi_dev->isp_available) + v4l2_async_unregister_subdev(subdev); + else + v4l2_device_unregister_subdev(subdev); + +error_media_entity: + media_entity_cleanup(&subdev->entity); + + return ret; +} + +void sun6i_csi_bridge_cleanup(struct sun6i_csi_device *csi_dev) +{ + struct v4l2_subdev *subdev = &csi_dev->bridge.subdev; + struct v4l2_async_notifier *notifier = &csi_dev->bridge.notifier; + + v4l2_async_nf_unregister(notifier); + v4l2_async_nf_cleanup(notifier); + + v4l2_device_unregister_subdev(subdev); + + media_entity_cleanup(&subdev->entity); +} diff --git a/drivers/media/platform/sunxi/sun6i-csi/sun6i_csi_bridge.h b/drivers/media/platform/sunxi/sun6i-csi/sun6i_csi_bridge.h new file mode 100644 index 000000000000..ee592a14b9c5 --- /dev/null +++ b/drivers/media/platform/sunxi/sun6i-csi/sun6i_csi_bridge.h @@ -0,0 +1,69 @@ +/* SPDX-License-Identifier: GPL-2.0+ */ +/* + * Copyright 2021-2022 Bootlin + * Author: Paul Kocialkowski <[email protected]> + */ + +#ifndef _SUN6I_CSI_BRIDGE_H_ +#define _SUN6I_CSI_BRIDGE_H_ + +#include <media/v4l2-device.h> +#include <media/v4l2-fwnode.h> + +#define SUN6I_CSI_BRIDGE_NAME "sun6i-csi-bridge" + +enum sun6i_csi_bridge_pad { + SUN6I_CSI_BRIDGE_PAD_SINK = 0, + SUN6I_CSI_BRIDGE_PAD_SOURCE = 1, + SUN6I_CSI_BRIDGE_PAD_COUNT = 2, +}; + +struct sun6i_csi_device; + +struct sun6i_csi_bridge_format { + u32 mbus_code; + u8 input_format; + u8 input_yuv_seq; + u8 input_yuv_seq_invert; +}; + +struct sun6i_csi_bridge_source { + struct v4l2_subdev *subdev; + struct v4l2_fwnode_endpoint endpoint; + bool expected; +}; + +struct sun6i_csi_bridge_async_subdev { + struct v4l2_async_subdev async_subdev; + struct sun6i_csi_bridge_source *source; +}; + +struct sun6i_csi_bridge { + struct v4l2_subdev subdev; + struct v4l2_async_notifier notifier; + struct media_pad pads[2]; + struct v4l2_mbus_framefmt mbus_format; + struct mutex lock; /* Mbus format lock. */ + + struct sun6i_csi_bridge_source source_parallel; + struct sun6i_csi_bridge_source source_mipi_csi2; +}; + +/* Helpers */ + +void sun6i_csi_bridge_dimensions(struct sun6i_csi_device *csi_dev, + unsigned int *width, unsigned int *height); +void sun6i_csi_bridge_format(struct sun6i_csi_device *csi_dev, + u32 *mbus_code, u32 *field); + +/* Format */ + +const struct sun6i_csi_bridge_format * +sun6i_csi_bridge_format_find(u32 mbus_code); + +/* Bridge */ + +int sun6i_csi_bridge_setup(struct sun6i_csi_device *csi_dev); +void sun6i_csi_bridge_cleanup(struct sun6i_csi_device *csi_dev); + +#endif diff --git a/drivers/media/platform/sunxi/sun6i-csi/sun6i_csi_capture.c b/drivers/media/platform/sunxi/sun6i-csi/sun6i_csi_capture.c new file mode 100644 index 000000000000..6d34f5c0768f --- /dev/null +++ b/drivers/media/platform/sunxi/sun6i-csi/sun6i_csi_capture.c @@ -0,0 +1,1102 @@ +// SPDX-License-Identifier: GPL-2.0+ +/* + * Copyright (c) 2011-2018 Magewell Electronics Co., Ltd. (Nanjing) + * Author: Yong Deng <[email protected]> + * Copyright 2021-2022 Bootlin + * Author: Paul Kocialkowski <[email protected]> + */ + +#include <linux/of.h> +#include <linux/regmap.h> +#include <media/v4l2-device.h> +#include <media/v4l2-event.h> +#include <media/v4l2-ioctl.h> +#include <media/v4l2-mc.h> +#include <media/videobuf2-dma-contig.h> +#include <media/videobuf2-v4l2.h> + +#include "sun6i_csi.h" +#include "sun6i_csi_bridge.h" +#include "sun6i_csi_capture.h" +#include "sun6i_csi_reg.h" + +/* Helpers */ + +void sun6i_csi_capture_dimensions(struct sun6i_csi_device *csi_dev, + unsigned int *width, unsigned int *height) +{ + if (width) + *width = csi_dev->capture.format.fmt.pix.width; + if (height) + *height = csi_dev->capture.format.fmt.pix.height; +} + +void sun6i_csi_capture_format(struct sun6i_csi_device *csi_dev, + u32 *pixelformat, u32 *field) +{ + if (pixelformat) + *pixelformat = csi_dev->capture.format.fmt.pix.pixelformat; + + if (field) + *field = csi_dev->capture.format.fmt.pix.field; +} + +/* Format */ + +static const struct sun6i_csi_capture_format sun6i_csi_capture_formats[] = { + /* Bayer */ + { + .pixelformat = V4L2_PIX_FMT_SBGGR8, + .output_format_frame = SUN6I_CSI_OUTPUT_FMT_FRAME_RAW_8, + .output_format_field = SUN6I_CSI_OUTPUT_FMT_FIELD_RAW_8, + }, + { + .pixelformat = V4L2_PIX_FMT_SGBRG8, + .output_format_frame = SUN6I_CSI_OUTPUT_FMT_FRAME_RAW_8, + .output_format_field = SUN6I_CSI_OUTPUT_FMT_FIELD_RAW_8, + }, + { + .pixelformat = V4L2_PIX_FMT_SGRBG8, + .output_format_frame = SUN6I_CSI_OUTPUT_FMT_FRAME_RAW_8, + .output_format_field = SUN6I_CSI_OUTPUT_FMT_FIELD_RAW_8, + }, + { + .pixelformat = V4L2_PIX_FMT_SRGGB8, + .output_format_frame = SUN6I_CSI_OUTPUT_FMT_FRAME_RAW_8, + .output_format_field = SUN6I_CSI_OUTPUT_FMT_FIELD_RAW_8, + }, + { + .pixelformat = V4L2_PIX_FMT_SBGGR10, + .output_format_frame = SUN6I_CSI_OUTPUT_FMT_FRAME_RAW_10, + .output_format_field = SUN6I_CSI_OUTPUT_FMT_FIELD_RAW_10, + }, + { + .pixelformat = V4L2_PIX_FMT_SGBRG10, + .output_format_frame = SUN6I_CSI_OUTPUT_FMT_FRAME_RAW_10, + .output_format_field = SUN6I_CSI_OUTPUT_FMT_FIELD_RAW_10, + }, + { + .pixelformat = V4L2_PIX_FMT_SGRBG10, + .output_format_frame = SUN6I_CSI_OUTPUT_FMT_FRAME_RAW_10, + .output_format_field = SUN6I_CSI_OUTPUT_FMT_FIELD_RAW_10, + }, + { + .pixelformat = V4L2_PIX_FMT_SRGGB10, + .output_format_frame = SUN6I_CSI_OUTPUT_FMT_FRAME_RAW_10, + .output_format_field = SUN6I_CSI_OUTPUT_FMT_FIELD_RAW_10, + }, + { + .pixelformat = V4L2_PIX_FMT_SBGGR12, + .output_format_frame = SUN6I_CSI_OUTPUT_FMT_FRAME_RAW_12, + .output_format_field = SUN6I_CSI_OUTPUT_FMT_FIELD_RAW_12, + }, + { + .pixelformat = V4L2_PIX_FMT_SGBRG12, + .output_format_frame = SUN6I_CSI_OUTPUT_FMT_FRAME_RAW_12, + .output_format_field = SUN6I_CSI_OUTPUT_FMT_FIELD_RAW_12, + }, + { + .pixelformat = V4L2_PIX_FMT_SGRBG12, + .output_format_frame = SUN6I_CSI_OUTPUT_FMT_FRAME_RAW_12, + .output_format_field = SUN6I_CSI_OUTPUT_FMT_FIELD_RAW_12, + }, + { + .pixelformat = V4L2_PIX_FMT_SRGGB12, + .output_format_frame = SUN6I_CSI_OUTPUT_FMT_FRAME_RAW_12, + .output_format_field = SUN6I_CSI_OUTPUT_FMT_FIELD_RAW_12, + }, + /* RGB */ + { + .pixelformat = V4L2_PIX_FMT_RGB565, + .output_format_frame = SUN6I_CSI_OUTPUT_FMT_FRAME_RGB565, + .output_format_field = SUN6I_CSI_OUTPUT_FMT_FIELD_RGB565, + }, + { + .pixelformat = V4L2_PIX_FMT_RGB565X, + .output_format_frame = SUN6I_CSI_OUTPUT_FMT_FRAME_RGB565, + .output_format_field = SUN6I_CSI_OUTPUT_FMT_FIELD_RGB565, + }, + /* YUV422 */ + { + .pixelformat = V4L2_PIX_FMT_YUYV, + .output_format_frame = SUN6I_CSI_OUTPUT_FMT_FRAME_RAW_8, + .output_format_field = SUN6I_CSI_OUTPUT_FMT_FIELD_RAW_8, + .input_format_raw = true, + .hsize_len_factor = 2, + }, + { + .pixelformat = V4L2_PIX_FMT_YVYU, + .output_format_frame = SUN6I_CSI_OUTPUT_FMT_FRAME_RAW_8, + .output_format_field = SUN6I_CSI_OUTPUT_FMT_FIELD_RAW_8, + .input_format_raw = true, + .hsize_len_factor = 2, + }, + { + .pixelformat = V4L2_PIX_FMT_UYVY, + .output_format_frame = SUN6I_CSI_OUTPUT_FMT_FRAME_RAW_8, + .output_format_field = SUN6I_CSI_OUTPUT_FMT_FIELD_RAW_8, + .input_format_raw = true, + .hsize_len_factor = 2, + }, + { + .pixelformat = V4L2_PIX_FMT_VYUY, + .output_format_frame = SUN6I_CSI_OUTPUT_FMT_FRAME_RAW_8, + .output_format_field = SUN6I_CSI_OUTPUT_FMT_FIELD_RAW_8, + .input_format_raw = true, + .hsize_len_factor = 2, + }, + { + .pixelformat = V4L2_PIX_FMT_NV16, + .output_format_frame = SUN6I_CSI_OUTPUT_FMT_FRAME_YUV422SP, + .output_format_field = SUN6I_CSI_OUTPUT_FMT_FIELD_YUV422SP, + }, + { + .pixelformat = V4L2_PIX_FMT_NV61, + .output_format_frame = SUN6I_CSI_OUTPUT_FMT_FRAME_YUV422SP, + .output_format_field = SUN6I_CSI_OUTPUT_FMT_FIELD_YUV422SP, + .input_yuv_seq_invert = true, + }, + { + .pixelformat = V4L2_PIX_FMT_YUV422P, + .output_format_frame = SUN6I_CSI_OUTPUT_FMT_FRAME_YUV422P, + .output_format_field = SUN6I_CSI_OUTPUT_FMT_FIELD_YUV422P, + }, + /* YUV420 */ + { + .pixelformat = V4L2_PIX_FMT_NV12_16L16, + .output_format_frame = SUN6I_CSI_OUTPUT_FMT_FRAME_YUV420MB, + .output_format_field = SUN6I_CSI_OUTPUT_FMT_FIELD_YUV420MB, + }, + { + .pixelformat = V4L2_PIX_FMT_NV12, + .output_format_frame = SUN6I_CSI_OUTPUT_FMT_FRAME_YUV420SP, + .output_format_field = SUN6I_CSI_OUTPUT_FMT_FIELD_YUV420SP, + }, + { + .pixelformat = V4L2_PIX_FMT_NV21, + .output_format_frame = SUN6I_CSI_OUTPUT_FMT_FRAME_YUV420SP, + .output_format_field = SUN6I_CSI_OUTPUT_FMT_FIELD_YUV420SP, + .input_yuv_seq_invert = true, + }, + + { + .pixelformat = V4L2_PIX_FMT_YUV420, + .output_format_frame = SUN6I_CSI_OUTPUT_FMT_FRAME_YUV420P, + .output_format_field = SUN6I_CSI_OUTPUT_FMT_FIELD_YUV420P, + }, + { + .pixelformat = V4L2_PIX_FMT_YVU420, + .output_format_frame = SUN6I_CSI_OUTPUT_FMT_FRAME_YUV420P, + .output_format_field = SUN6I_CSI_OUTPUT_FMT_FIELD_YUV420P, + .input_yuv_seq_invert = true, + }, + /* Compressed */ + { + .pixelformat = V4L2_PIX_FMT_JPEG, + .output_format_frame = SUN6I_CSI_OUTPUT_FMT_FRAME_RAW_8, + .output_format_field = SUN6I_CSI_OUTPUT_FMT_FIELD_RAW_8, + }, +}; + +const +struct sun6i_csi_capture_format *sun6i_csi_capture_format_find(u32 pixelformat) +{ + unsigned int i; + + for (i = 0; i < ARRAY_SIZE(sun6i_csi_capture_formats); i++) + if (sun6i_csi_capture_formats[i].pixelformat == pixelformat) + return &sun6i_csi_capture_formats[i]; + + return NULL; +} + +/* RAW formats need an exact match between pixel and mbus formats. */ +static const +struct sun6i_csi_capture_format_match sun6i_csi_capture_format_matches[] = { + /* YUV420 */ + { + .pixelformat = V4L2_PIX_FMT_YUYV, + .mbus_code = MEDIA_BUS_FMT_YUYV8_2X8, + }, + { + .pixelformat = V4L2_PIX_FMT_YUYV, + .mbus_code = MEDIA_BUS_FMT_YUYV8_1X16, + }, + { + .pixelformat = V4L2_PIX_FMT_YVYU, + .mbus_code = MEDIA_BUS_FMT_YVYU8_2X8, + }, + { + .pixelformat = V4L2_PIX_FMT_YVYU, + .mbus_code = MEDIA_BUS_FMT_YVYU8_1X16, + }, + { + .pixelformat = V4L2_PIX_FMT_UYVY, + .mbus_code = MEDIA_BUS_FMT_UYVY8_2X8, + }, + { + .pixelformat = V4L2_PIX_FMT_UYVY, + .mbus_code = MEDIA_BUS_FMT_UYVY8_1X16, + }, + { + .pixelformat = V4L2_PIX_FMT_VYUY, + .mbus_code = MEDIA_BUS_FMT_VYUY8_2X8, + }, + { + .pixelformat = V4L2_PIX_FMT_VYUY, + .mbus_code = MEDIA_BUS_FMT_VYUY8_1X16, + }, + /* RGB */ + { + .pixelformat = V4L2_PIX_FMT_RGB565, + .mbus_code = MEDIA_BUS_FMT_RGB565_2X8_LE, + }, + { + .pixelformat = V4L2_PIX_FMT_RGB565X, + .mbus_code = MEDIA_BUS_FMT_RGB565_2X8_BE, + }, + /* Bayer */ + { + .pixelformat = V4L2_PIX_FMT_SBGGR8, + .mbus_code = MEDIA_BUS_FMT_SBGGR8_1X8, + }, + { + .pixelformat = V4L2_PIX_FMT_SGBRG8, + .mbus_code = MEDIA_BUS_FMT_SGBRG8_1X8, + }, + { + .pixelformat = V4L2_PIX_FMT_SGRBG8, + .mbus_code = MEDIA_BUS_FMT_SGRBG8_1X8, + }, + { + .pixelformat = V4L2_PIX_FMT_SRGGB8, + .mbus_code = MEDIA_BUS_FMT_SRGGB8_1X8, + }, + { + .pixelformat = V4L2_PIX_FMT_SBGGR10, + .mbus_code = MEDIA_BUS_FMT_SBGGR10_1X10, + }, + { + .pixelformat = V4L2_PIX_FMT_SGBRG10, + .mbus_code = MEDIA_BUS_FMT_SGBRG10_1X10, + }, + { + .pixelformat = V4L2_PIX_FMT_SGRBG10, + .mbus_code = MEDIA_BUS_FMT_SGRBG10_1X10, + }, + { + .pixelformat = V4L2_PIX_FMT_SRGGB10, + .mbus_code = MEDIA_BUS_FMT_SRGGB10_1X10, + }, + { + .pixelformat = V4L2_PIX_FMT_SBGGR12, + .mbus_code = MEDIA_BUS_FMT_SBGGR12_1X12, + }, + { + .pixelformat = V4L2_PIX_FMT_SGBRG12, + .mbus_code = MEDIA_BUS_FMT_SGBRG12_1X12, + }, + { + .pixelformat = V4L2_PIX_FMT_SGRBG12, + .mbus_code = MEDIA_BUS_FMT_SGRBG12_1X12, + }, + { + .pixelformat = V4L2_PIX_FMT_SRGGB12, + .mbus_code = MEDIA_BUS_FMT_SRGGB12_1X12, + }, + /* Compressed */ + { + .pixelformat = V4L2_PIX_FMT_JPEG, + .mbus_code = MEDIA_BUS_FMT_JPEG_1X8, + }, +}; + +static bool sun6i_csi_capture_format_match(u32 pixelformat, u32 mbus_code) +{ + unsigned int i; + + for (i = 0; i < ARRAY_SIZE(sun6i_csi_capture_format_matches); i++) { + const struct sun6i_csi_capture_format_match *match = + &sun6i_csi_capture_format_matches[i]; + + if (match->pixelformat == pixelformat && + match->mbus_code == mbus_code) + return true; + } + + return false; +} + +/* Capture */ + +static void +sun6i_csi_capture_buffer_configure(struct sun6i_csi_device *csi_dev, + struct sun6i_csi_buffer *csi_buffer) +{ + struct regmap *regmap = csi_dev->regmap; + const struct v4l2_format_info *info; + struct vb2_buffer *vb2_buffer; + unsigned int width, height; + dma_addr_t address; + u32 pixelformat; + + vb2_buffer = &csi_buffer->v4l2_buffer.vb2_buf; + address = vb2_dma_contig_plane_dma_addr(vb2_buffer, 0); + + regmap_write(regmap, SUN6I_CSI_CH_FIFO0_ADDR_REG, + SUN6I_CSI_ADDR_VALUE(address)); + + sun6i_csi_capture_dimensions(csi_dev, &width, &height); + sun6i_csi_capture_format(csi_dev, &pixelformat, NULL); + + info = v4l2_format_info(pixelformat); + /* Unsupported formats are single-plane, so we can stop here. */ + if (!info) + return; + + if (info->comp_planes > 1) { + address += info->bpp[0] * width * height; + + regmap_write(regmap, SUN6I_CSI_CH_FIFO1_ADDR_REG, + SUN6I_CSI_ADDR_VALUE(address)); + } + + if (info->comp_planes > 2) { + address += info->bpp[1] * DIV_ROUND_UP(width, info->hdiv) * + DIV_ROUND_UP(height, info->vdiv); + + regmap_write(regmap, SUN6I_CSI_CH_FIFO2_ADDR_REG, + SUN6I_CSI_ADDR_VALUE(address)); + } +} + +void sun6i_csi_capture_configure(struct sun6i_csi_device *csi_dev) +{ + struct regmap *regmap = csi_dev->regmap; + const struct sun6i_csi_capture_format *format; + const struct v4l2_format_info *info; + u32 hsize_len, vsize_len; + u32 luma_line, chroma_line = 0; + u32 pixelformat, field; + u32 width, height; + + sun6i_csi_capture_dimensions(csi_dev, &width, &height); + sun6i_csi_capture_format(csi_dev, &pixelformat, &field); + + format = sun6i_csi_capture_format_find(pixelformat); + if (WARN_ON(!format)) + return; + + hsize_len = width; + vsize_len = height; + + /* + * When using 8-bit raw input/output (for packed YUV), we need to adapt + * the width to account for the difference in bpp when it's not 8-bit. + */ + if (format->hsize_len_factor) + hsize_len *= format->hsize_len_factor; + + regmap_write(regmap, SUN6I_CSI_CH_HSIZE_REG, + SUN6I_CSI_CH_HSIZE_LEN(hsize_len) | + SUN6I_CSI_CH_HSIZE_START(0)); + + regmap_write(regmap, SUN6I_CSI_CH_VSIZE_REG, + SUN6I_CSI_CH_VSIZE_LEN(vsize_len) | + SUN6I_CSI_CH_VSIZE_START(0)); + + switch (pixelformat) { + case V4L2_PIX_FMT_RGB565X: + luma_line = width * 2; + break; + case V4L2_PIX_FMT_NV12_16L16: + luma_line = width; + chroma_line = width; + break; + case V4L2_PIX_FMT_JPEG: + luma_line = width; + break; + default: + info = v4l2_format_info(pixelformat); + if (WARN_ON(!info)) + return; + + luma_line = width * info->bpp[0]; + + if (info->comp_planes > 1) + chroma_line = width * info->bpp[1] / info->hdiv; + break; + } + + regmap_write(regmap, SUN6I_CSI_CH_BUF_LEN_REG, + SUN6I_CSI_CH_BUF_LEN_CHROMA_LINE(chroma_line) | + SUN6I_CSI_CH_BUF_LEN_LUMA_LINE(luma_line)); +} + +/* State */ + +static void sun6i_csi_capture_state_cleanup(struct sun6i_csi_device *csi_dev, + bool error) +{ + struct sun6i_csi_capture_state *state = &csi_dev->capture.state; + struct sun6i_csi_buffer **csi_buffer_states[] = { + &state->pending, &state->current, &state->complete, + }; + struct sun6i_csi_buffer *csi_buffer; + struct vb2_buffer *vb2_buffer; + unsigned long flags; + unsigned int i; + + spin_lock_irqsave(&state->lock, flags); + + for (i = 0; i < ARRAY_SIZE(csi_buffer_states); i++) { + csi_buffer = *csi_buffer_states[i]; + if (!csi_buffer) + continue; + + vb2_buffer = &csi_buffer->v4l2_buffer.vb2_buf; + vb2_buffer_done(vb2_buffer, error ? VB2_BUF_STATE_ERROR : + VB2_BUF_STATE_QUEUED); + + *csi_buffer_states[i] = NULL; + } + + list_for_each_entry(csi_buffer, &state->queue, list) { + vb2_buffer = &csi_buffer->v4l2_buffer.vb2_buf; + vb2_buffer_done(vb2_buffer, error ? VB2_BUF_STATE_ERROR : + VB2_BUF_STATE_QUEUED); + } + + INIT_LIST_HEAD(&state->queue); + + spin_unlock_irqrestore(&state->lock, flags); +} + +void sun6i_csi_capture_state_update(struct sun6i_csi_device *csi_dev) +{ + struct sun6i_csi_capture_state *state = &csi_dev->capture.state; + struct sun6i_csi_buffer *csi_buffer; + unsigned long flags; + + spin_lock_irqsave(&state->lock, flags); + + if (list_empty(&state->queue)) + goto complete; + + if (state->pending) + goto complete; + + csi_buffer = list_first_entry(&state->queue, struct sun6i_csi_buffer, + list); + + sun6i_csi_capture_buffer_configure(csi_dev, csi_buffer); + + list_del(&csi_buffer->list); + + state->pending = csi_buffer; + +complete: + spin_unlock_irqrestore(&state->lock, flags); +} + +static void sun6i_csi_capture_state_complete(struct sun6i_csi_device *csi_dev) +{ + struct sun6i_csi_capture_state *state = &csi_dev->capture.state; + unsigned long flags; + + spin_lock_irqsave(&state->lock, flags); + + if (!state->pending) + goto complete; + + state->complete = state->current; + state->current = state->pending; + state->pending = NULL; + + if (state->complete) { + struct sun6i_csi_buffer *csi_buffer = state->complete; + struct vb2_buffer *vb2_buffer = + &csi_buffer->v4l2_buffer.vb2_buf; + + vb2_buffer->timestamp = ktime_get_ns(); + csi_buffer->v4l2_buffer.sequence = state->sequence; + + vb2_buffer_done(vb2_buffer, VB2_BUF_STATE_DONE); + + state->complete = NULL; + } + +complete: + spin_unlock_irqrestore(&state->lock, flags); +} + +void sun6i_csi_capture_frame_done(struct sun6i_csi_device *csi_dev) +{ + struct sun6i_csi_capture_state *state = &csi_dev->capture.state; + unsigned long flags; + + spin_lock_irqsave(&state->lock, flags); + state->sequence++; + spin_unlock_irqrestore(&state->lock, flags); +} + +void sun6i_csi_capture_sync(struct sun6i_csi_device *csi_dev) +{ + sun6i_csi_capture_state_complete(csi_dev); + sun6i_csi_capture_state_update(csi_dev); +} + +/* Queue */ + +static int sun6i_csi_capture_queue_setup(struct vb2_queue *queue, + unsigned int *buffers_count, + unsigned int *planes_count, + unsigned int sizes[], + struct device *alloc_devs[]) +{ + struct sun6i_csi_device *csi_dev = vb2_get_drv_priv(queue); + unsigned int size = csi_dev->capture.format.fmt.pix.sizeimage; + + if (*planes_count) + return sizes[0] < size ? -EINVAL : 0; + + *planes_count = 1; + sizes[0] = size; + + return 0; +} + +static int sun6i_csi_capture_buffer_prepare(struct vb2_buffer *buffer) +{ + struct sun6i_csi_device *csi_dev = vb2_get_drv_priv(buffer->vb2_queue); + struct sun6i_csi_capture *capture = &csi_dev->capture; + struct v4l2_device *v4l2_dev = csi_dev->v4l2_dev; + struct vb2_v4l2_buffer *v4l2_buffer = to_vb2_v4l2_buffer(buffer); + unsigned long size = capture->format.fmt.pix.sizeimage; + + if (vb2_plane_size(buffer, 0) < size) { + v4l2_err(v4l2_dev, "buffer too small (%lu < %lu)\n", + vb2_plane_size(buffer, 0), size); + return -EINVAL; + } + + vb2_set_plane_payload(buffer, 0, size); + + v4l2_buffer->field = capture->format.fmt.pix.field; + + return 0; +} + +static void sun6i_csi_capture_buffer_queue(struct vb2_buffer *buffer) +{ + struct sun6i_csi_device *csi_dev = vb2_get_drv_priv(buffer->vb2_queue); + struct sun6i_csi_capture_state *state = &csi_dev->capture.state; + struct vb2_v4l2_buffer *v4l2_buffer = to_vb2_v4l2_buffer(buffer); + struct sun6i_csi_buffer *csi_buffer = + container_of(v4l2_buffer, struct sun6i_csi_buffer, v4l2_buffer); + unsigned long flags; + + spin_lock_irqsave(&state->lock, flags); + list_add_tail(&csi_buffer->list, &state->queue); + spin_unlock_irqrestore(&state->lock, flags); +} + +static int sun6i_csi_capture_start_streaming(struct vb2_queue *queue, + unsigned int count) +{ + struct sun6i_csi_device *csi_dev = vb2_get_drv_priv(queue); + struct sun6i_csi_capture_state *state = &csi_dev->capture.state; + struct video_device *video_dev = &csi_dev->capture.video_dev; + struct v4l2_subdev *subdev = &csi_dev->bridge.subdev; + int ret; + + state->sequence = 0; + + ret = video_device_pipeline_alloc_start(video_dev); + if (ret < 0) + goto error_state; + + state->streaming = true; + + ret = v4l2_subdev_call(subdev, video, s_stream, 1); + if (ret && ret != -ENOIOCTLCMD) + goto error_streaming; + + return 0; + +error_streaming: + state->streaming = false; + + video_device_pipeline_stop(video_dev); + +error_state: + sun6i_csi_capture_state_cleanup(csi_dev, false); + + return ret; +} + +static void sun6i_csi_capture_stop_streaming(struct vb2_queue *queue) +{ + struct sun6i_csi_device *csi_dev = vb2_get_drv_priv(queue); + struct sun6i_csi_capture_state *state = &csi_dev->capture.state; + struct video_device *video_dev = &csi_dev->capture.video_dev; + struct v4l2_subdev *subdev = &csi_dev->bridge.subdev; + + v4l2_subdev_call(subdev, video, s_stream, 0); + + state->streaming = false; + + video_device_pipeline_stop(video_dev); + + sun6i_csi_capture_state_cleanup(csi_dev, true); +} + +static const struct vb2_ops sun6i_csi_capture_queue_ops = { + .queue_setup = sun6i_csi_capture_queue_setup, + .buf_prepare = sun6i_csi_capture_buffer_prepare, + .buf_queue = sun6i_csi_capture_buffer_queue, + .start_streaming = sun6i_csi_capture_start_streaming, + .stop_streaming = sun6i_csi_capture_stop_streaming, + .wait_prepare = vb2_ops_wait_prepare, + .wait_finish = vb2_ops_wait_finish, +}; + +/* V4L2 Device */ + +static void sun6i_csi_capture_format_prepare(struct v4l2_format *format) +{ + struct v4l2_pix_format *pix_format = &format->fmt.pix; + const struct v4l2_format_info *info; + unsigned int width, height; + + v4l_bound_align_image(&pix_format->width, SUN6I_CSI_CAPTURE_WIDTH_MIN, + SUN6I_CSI_CAPTURE_WIDTH_MAX, 1, + &pix_format->height, SUN6I_CSI_CAPTURE_HEIGHT_MIN, + SUN6I_CSI_CAPTURE_HEIGHT_MAX, 1, 0); + + if (!sun6i_csi_capture_format_find(pix_format->pixelformat)) + pix_format->pixelformat = + sun6i_csi_capture_formats[0].pixelformat; + + width = pix_format->width; + height = pix_format->height; + + info = v4l2_format_info(pix_format->pixelformat); + + switch (pix_format->pixelformat) { + case V4L2_PIX_FMT_NV12_16L16: + pix_format->bytesperline = width * 12 / 8; + pix_format->sizeimage = pix_format->bytesperline * height; + break; + case V4L2_PIX_FMT_JPEG: + pix_format->bytesperline = width; + pix_format->sizeimage = pix_format->bytesperline * height; + break; + default: + v4l2_fill_pixfmt(pix_format, pix_format->pixelformat, + width, height); + break; + } + + if (pix_format->field == V4L2_FIELD_ANY) + pix_format->field = V4L2_FIELD_NONE; + + if (pix_format->pixelformat == V4L2_PIX_FMT_JPEG) + pix_format->colorspace = V4L2_COLORSPACE_JPEG; + else if (info && info->pixel_enc == V4L2_PIXEL_ENC_BAYER) + pix_format->colorspace = V4L2_COLORSPACE_RAW; + else + pix_format->colorspace = V4L2_COLORSPACE_SRGB; + + pix_format->ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT; + pix_format->quantization = V4L2_QUANTIZATION_DEFAULT; + pix_format->xfer_func = V4L2_XFER_FUNC_DEFAULT; +} + +static int sun6i_csi_capture_querycap(struct file *file, void *private, + struct v4l2_capability *capability) +{ + struct sun6i_csi_device *csi_dev = video_drvdata(file); + struct video_device *video_dev = &csi_dev->capture.video_dev; + + strscpy(capability->driver, SUN6I_CSI_NAME, sizeof(capability->driver)); + strscpy(capability->card, video_dev->name, sizeof(capability->card)); + snprintf(capability->bus_info, sizeof(capability->bus_info), + "platform:%s", dev_name(csi_dev->dev)); + + return 0; +} + +static int sun6i_csi_capture_enum_fmt(struct file *file, void *private, + struct v4l2_fmtdesc *fmtdesc) +{ + u32 index = fmtdesc->index; + + if (index >= ARRAY_SIZE(sun6i_csi_capture_formats)) + return -EINVAL; + + fmtdesc->pixelformat = sun6i_csi_capture_formats[index].pixelformat; + + return 0; +} + +static int sun6i_csi_capture_g_fmt(struct file *file, void *private, + struct v4l2_format *format) +{ + struct sun6i_csi_device *csi_dev = video_drvdata(file); + + *format = csi_dev->capture.format; + + return 0; +} + +static int sun6i_csi_capture_s_fmt(struct file *file, void *private, + struct v4l2_format *format) +{ + struct sun6i_csi_device *csi_dev = video_drvdata(file); + struct sun6i_csi_capture *capture = &csi_dev->capture; + + if (vb2_is_busy(&capture->queue)) + return -EBUSY; + + sun6i_csi_capture_format_prepare(format); + + csi_dev->capture.format = *format; + + return 0; +} + +static int sun6i_csi_capture_try_fmt(struct file *file, void *private, + struct v4l2_format *format) +{ + sun6i_csi_capture_format_prepare(format); + + return 0; +} + +static int sun6i_csi_capture_enum_input(struct file *file, void *private, + struct v4l2_input *input) +{ + if (input->index != 0) + return -EINVAL; + + input->type = V4L2_INPUT_TYPE_CAMERA; + strscpy(input->name, "Camera", sizeof(input->name)); + + return 0; +} + +static int sun6i_csi_capture_g_input(struct file *file, void *private, + unsigned int *index) +{ + *index = 0; + + return 0; +} + +static int sun6i_csi_capture_s_input(struct file *file, void *private, + unsigned int index) +{ + if (index != 0) + return -EINVAL; + + return 0; +} + +static const struct v4l2_ioctl_ops sun6i_csi_capture_ioctl_ops = { + .vidioc_querycap = sun6i_csi_capture_querycap, + + .vidioc_enum_fmt_vid_cap = sun6i_csi_capture_enum_fmt, + .vidioc_g_fmt_vid_cap = sun6i_csi_capture_g_fmt, + .vidioc_s_fmt_vid_cap = sun6i_csi_capture_s_fmt, + .vidioc_try_fmt_vid_cap = sun6i_csi_capture_try_fmt, + + .vidioc_enum_input = sun6i_csi_capture_enum_input, + .vidioc_g_input = sun6i_csi_capture_g_input, + .vidioc_s_input = sun6i_csi_capture_s_input, + + .vidioc_create_bufs = vb2_ioctl_create_bufs, + .vidioc_prepare_buf = vb2_ioctl_prepare_buf, + .vidioc_reqbufs = vb2_ioctl_reqbufs, + .vidioc_querybuf = vb2_ioctl_querybuf, + .vidioc_expbuf = vb2_ioctl_expbuf, + .vidioc_qbuf = vb2_ioctl_qbuf, + .vidioc_dqbuf = vb2_ioctl_dqbuf, + .vidioc_streamon = vb2_ioctl_streamon, + .vidioc_streamoff = vb2_ioctl_streamoff, +}; + +/* V4L2 File */ + +static int sun6i_csi_capture_open(struct file *file) +{ + struct sun6i_csi_device *csi_dev = video_drvdata(file); + struct sun6i_csi_capture *capture = &csi_dev->capture; + int ret = 0; + + if (mutex_lock_interruptible(&capture->lock)) + return -ERESTARTSYS; + + ret = v4l2_pipeline_pm_get(&capture->video_dev.entity); + if (ret < 0) + goto error_lock; + + ret = v4l2_fh_open(file); + if (ret < 0) + goto error_pipeline; + + mutex_unlock(&capture->lock); + + return 0; + +error_pipeline: + v4l2_pipeline_pm_put(&capture->video_dev.entity); + +error_lock: + mutex_unlock(&capture->lock); + + return ret; +} + +static int sun6i_csi_capture_close(struct file *file) +{ + struct sun6i_csi_device *csi_dev = video_drvdata(file); + struct sun6i_csi_capture *capture = &csi_dev->capture; + + mutex_lock(&capture->lock); + + _vb2_fop_release(file, NULL); + v4l2_pipeline_pm_put(&capture->video_dev.entity); + + mutex_unlock(&capture->lock); + + return 0; +} + +static const struct v4l2_file_operations sun6i_csi_capture_fops = { + .owner = THIS_MODULE, + .open = sun6i_csi_capture_open, + .release = sun6i_csi_capture_close, + .unlocked_ioctl = video_ioctl2, + .mmap = vb2_fop_mmap, + .poll = vb2_fop_poll +}; + +/* Media Entity */ + +static int sun6i_csi_capture_link_validate(struct media_link *link) +{ + struct video_device *video_dev = + media_entity_to_video_device(link->sink->entity); + struct sun6i_csi_device *csi_dev = video_get_drvdata(video_dev); + struct v4l2_device *v4l2_dev = csi_dev->v4l2_dev; + const struct sun6i_csi_capture_format *capture_format; + const struct sun6i_csi_bridge_format *bridge_format; + unsigned int capture_width, capture_height; + unsigned int bridge_width, bridge_height; + const struct v4l2_format_info *format_info; + u32 pixelformat, capture_field; + u32 mbus_code, bridge_field; + bool match; + + sun6i_csi_capture_dimensions(csi_dev, &capture_width, &capture_height); + + sun6i_csi_capture_format(csi_dev, &pixelformat, &capture_field); + capture_format = sun6i_csi_capture_format_find(pixelformat); + if (WARN_ON(!capture_format)) + return -EINVAL; + + sun6i_csi_bridge_dimensions(csi_dev, &bridge_width, &bridge_height); + + sun6i_csi_bridge_format(csi_dev, &mbus_code, &bridge_field); + bridge_format = sun6i_csi_bridge_format_find(mbus_code); + if (WARN_ON(!bridge_format)) + return -EINVAL; + + /* No cropping/scaling is supported. */ + if (capture_width != bridge_width || capture_height != bridge_height) { + v4l2_err(v4l2_dev, + "invalid input/output dimensions: %ux%u/%ux%u\n", + bridge_width, bridge_height, capture_width, + capture_height); + return -EINVAL; + } + + format_info = v4l2_format_info(pixelformat); + /* Some formats are not listed. */ + if (!format_info) + return 0; + + if (format_info->pixel_enc == V4L2_PIXEL_ENC_BAYER && + bridge_format->input_format != SUN6I_CSI_INPUT_FMT_RAW) + goto invalid; + + if (format_info->pixel_enc == V4L2_PIXEL_ENC_RGB && + bridge_format->input_format != SUN6I_CSI_INPUT_FMT_RAW) + goto invalid; + + if (format_info->pixel_enc == V4L2_PIXEL_ENC_YUV) { + if (bridge_format->input_format != SUN6I_CSI_INPUT_FMT_YUV420 && + bridge_format->input_format != SUN6I_CSI_INPUT_FMT_YUV422) + goto invalid; + + /* YUV420 input can't produce YUV422 output. */ + if (bridge_format->input_format == SUN6I_CSI_INPUT_FMT_YUV420 && + format_info->vdiv == 1) + goto invalid; + } + + /* With raw input mode, we need a 1:1 match between input and output. */ + if (bridge_format->input_format == SUN6I_CSI_INPUT_FMT_RAW || + capture_format->input_format_raw) { + match = sun6i_csi_capture_format_match(pixelformat, mbus_code); + if (!match) + goto invalid; + } + + return 0; + +invalid: + v4l2_err(v4l2_dev, "invalid input/output format combination\n"); + return -EINVAL; +} + +static const struct media_entity_operations sun6i_csi_capture_media_ops = { + .link_validate = sun6i_csi_capture_link_validate +}; + +/* Capture */ + +int sun6i_csi_capture_setup(struct sun6i_csi_device *csi_dev) +{ + struct sun6i_csi_capture *capture = &csi_dev->capture; + struct sun6i_csi_capture_state *state = &capture->state; + struct v4l2_device *v4l2_dev = csi_dev->v4l2_dev; + struct v4l2_subdev *bridge_subdev = &csi_dev->bridge.subdev; + struct video_device *video_dev = &capture->video_dev; + struct vb2_queue *queue = &capture->queue; + struct media_pad *pad = &capture->pad; + struct v4l2_format *format = &csi_dev->capture.format; + struct v4l2_pix_format *pix_format = &format->fmt.pix; + int ret; + + /* This may happen with multiple bridge notifier bound calls. */ + if (state->setup) + return 0; + + /* State */ + + INIT_LIST_HEAD(&state->queue); + spin_lock_init(&state->lock); + + /* Media Entity */ + + video_dev->entity.ops = &sun6i_csi_capture_media_ops; + + /* Media Pad */ + + pad->flags = MEDIA_PAD_FL_SINK | MEDIA_PAD_FL_MUST_CONNECT; + + ret = media_entity_pads_init(&video_dev->entity, 1, pad); + if (ret < 0) + return ret; + + /* Queue */ + + mutex_init(&capture->lock); + + queue->type = V4L2_BUF_TYPE_VIDEO_CAPTURE; + queue->io_modes = VB2_MMAP | VB2_DMABUF; + queue->buf_struct_size = sizeof(struct sun6i_csi_buffer); + queue->ops = &sun6i_csi_capture_queue_ops; + queue->mem_ops = &vb2_dma_contig_memops; + queue->min_buffers_needed = 2; + queue->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC; + queue->lock = &capture->lock; + queue->dev = csi_dev->dev; + queue->drv_priv = csi_dev; + + ret = vb2_queue_init(queue); + if (ret) { + v4l2_err(v4l2_dev, "failed to initialize vb2 queue: %d\n", ret); + goto error_media_entity; + } + + /* V4L2 Format */ + + format->type = queue->type; + pix_format->pixelformat = sun6i_csi_capture_formats[0].pixelformat; + pix_format->width = 1280; + pix_format->height = 720; + pix_format->field = V4L2_FIELD_NONE; + + sun6i_csi_capture_format_prepare(format); + + /* Video Device */ + + strscpy(video_dev->name, SUN6I_CSI_CAPTURE_NAME, + sizeof(video_dev->name)); + video_dev->device_caps = V4L2_CAP_VIDEO_CAPTURE | V4L2_CAP_STREAMING; + video_dev->vfl_dir = VFL_DIR_RX; + video_dev->release = video_device_release_empty; + video_dev->fops = &sun6i_csi_capture_fops; + video_dev->ioctl_ops = &sun6i_csi_capture_ioctl_ops; + video_dev->v4l2_dev = v4l2_dev; + video_dev->queue = queue; + video_dev->lock = &capture->lock; + + video_set_drvdata(video_dev, csi_dev); + + ret = video_register_device(video_dev, VFL_TYPE_VIDEO, -1); + if (ret < 0) { + v4l2_err(v4l2_dev, "failed to register video device: %d\n", + ret); + goto error_media_entity; + } + + /* Media Pad Link */ + + ret = media_create_pad_link(&bridge_subdev->entity, + SUN6I_CSI_BRIDGE_PAD_SOURCE, + &video_dev->entity, 0, + csi_dev->isp_available ? 0 : + MEDIA_LNK_FL_ENABLED | + MEDIA_LNK_FL_IMMUTABLE); + if (ret < 0) { + v4l2_err(v4l2_dev, "failed to create %s:%u -> %s:%u link\n", + bridge_subdev->entity.name, + SUN6I_CSI_BRIDGE_PAD_SOURCE, + video_dev->entity.name, 0); + goto error_video_device; + } + + state->setup = true; + + return 0; + +error_video_device: + vb2_video_unregister_device(video_dev); + +error_media_entity: + media_entity_cleanup(&video_dev->entity); + + mutex_destroy(&capture->lock); + + return ret; +} + +void sun6i_csi_capture_cleanup(struct sun6i_csi_device *csi_dev) +{ + struct sun6i_csi_capture *capture = &csi_dev->capture; + struct video_device *video_dev = &capture->video_dev; + + /* This may happen if async registration failed to complete. */ + if (!capture->state.setup) + return; + + vb2_video_unregister_device(video_dev); + media_entity_cleanup(&video_dev->entity); + mutex_destroy(&capture->lock); + + capture->state.setup = false; +} diff --git a/drivers/media/platform/sunxi/sun6i-csi/sun6i_csi_capture.h b/drivers/media/platform/sunxi/sun6i-csi/sun6i_csi_capture.h new file mode 100644 index 000000000000..3ee5ccefbd10 --- /dev/null +++ b/drivers/media/platform/sunxi/sun6i-csi/sun6i_csi_capture.h @@ -0,0 +1,89 @@ +/* SPDX-License-Identifier: GPL-2.0+ */ +/* + * Copyright (c) 2011-2018 Magewell Electronics Co., Ltd. (Nanjing) + * Author: Yong Deng <[email protected]> + * Copyright 2021-2022 Bootlin + * Author: Paul Kocialkowski <[email protected]> + */ + +#ifndef _SUN6I_CAPTURE_H_ +#define _SUN6I_CAPTURE_H_ + +#include <media/v4l2-device.h> + +#define SUN6I_CSI_CAPTURE_NAME "sun6i-csi-capture" + +#define SUN6I_CSI_CAPTURE_WIDTH_MIN 32 +#define SUN6I_CSI_CAPTURE_WIDTH_MAX 4800 +#define SUN6I_CSI_CAPTURE_HEIGHT_MIN 32 +#define SUN6I_CSI_CAPTURE_HEIGHT_MAX 4800 + +struct sun6i_csi_device; + +struct sun6i_csi_capture_format { + u32 pixelformat; + u8 output_format_field; + u8 output_format_frame; + bool input_yuv_seq_invert; + bool input_format_raw; + u32 hsize_len_factor; +}; + +struct sun6i_csi_capture_format_match { + u32 pixelformat; + u32 mbus_code; +}; + +#undef current +struct sun6i_csi_capture_state { + struct list_head queue; + spinlock_t lock; /* Queue and buffers lock. */ + + struct sun6i_csi_buffer *pending; + struct sun6i_csi_buffer *current; + struct sun6i_csi_buffer *complete; + + unsigned int sequence; + bool streaming; + bool setup; +}; + +struct sun6i_csi_capture { + struct sun6i_csi_capture_state state; + + struct video_device video_dev; + struct vb2_queue queue; + struct mutex lock; /* Queue lock. */ + struct media_pad pad; + + struct v4l2_format format; +}; + +/* Helpers */ + +void sun6i_csi_capture_dimensions(struct sun6i_csi_device *csi_dev, + unsigned int *width, unsigned int *height); +void sun6i_csi_capture_format(struct sun6i_csi_device *csi_dev, + u32 *pixelformat, u32 *field); + +/* Format */ + +const +struct sun6i_csi_capture_format *sun6i_csi_capture_format_find(u32 pixelformat); + +/* Capture */ + +void sun6i_csi_capture_configure(struct sun6i_csi_device *csi_dev); +void sun6i_csi_capture_state_update(struct sun6i_csi_device *csi_dev); + +/* State */ + +void sun6i_csi_capture_sync(struct sun6i_csi_device *csi_dev); +void sun6i_csi_capture_frame_done(struct sun6i_csi_device *csi_dev); + +/* Capture */ + +int sun6i_csi_capture_setup(struct sun6i_csi_device *csi_dev); +void sun6i_csi_capture_cleanup(struct sun6i_csi_device *csi_dev); + +#endif diff --git a/drivers/media/platform/sunxi/sun6i-csi/sun6i_csi_reg.h b/drivers/media/platform/sunxi/sun6i-csi/sun6i_csi_reg.h index 703fa14bb313..e01c5b9c2d60 100644 --- a/drivers/media/platform/sunxi/sun6i-csi/sun6i_csi_reg.h +++ b/drivers/media/platform/sunxi/sun6i-csi/sun6i_csi_reg.h @@ -1,196 +1,184 @@ /* SPDX-License-Identifier: GPL-2.0+ */ /* * Copyright (c) 2011-2018 Magewell Electronics Co., Ltd. (Nanjing) - * All rights reserved. * Author: Yong Deng <[email protected]> + * Copyright 2021-2022 Bootlin + * Author: Paul Kocialkowski <[email protected]> */ -#ifndef __SUN6I_CSI_REG_H__ -#define __SUN6I_CSI_REG_H__ +#ifndef _SUN6I_CSI_REG_H_ +#define _SUN6I_CSI_REG_H_ #include <linux/kernel.h> -#define CSI_EN_REG 0x0 -#define CSI_EN_VER_EN BIT(30) -#define CSI_EN_CSI_EN BIT(0) - -#define CSI_IF_CFG_REG 0x4 -#define CSI_IF_CFG_SRC_TYPE_MASK BIT(21) -#define CSI_IF_CFG_SRC_TYPE_PROGRESSED ((0 << 21) & CSI_IF_CFG_SRC_TYPE_MASK) -#define CSI_IF_CFG_SRC_TYPE_INTERLACED ((1 << 21) & CSI_IF_CFG_SRC_TYPE_MASK) -#define CSI_IF_CFG_FPS_DS_EN BIT(20) -#define CSI_IF_CFG_FIELD_MASK BIT(19) -#define CSI_IF_CFG_FIELD_NEGATIVE ((0 << 19) & CSI_IF_CFG_FIELD_MASK) -#define CSI_IF_CFG_FIELD_POSITIVE ((1 << 19) & CSI_IF_CFG_FIELD_MASK) -#define CSI_IF_CFG_VREF_POL_MASK BIT(18) -#define CSI_IF_CFG_VREF_POL_NEGATIVE ((0 << 18) & CSI_IF_CFG_VREF_POL_MASK) -#define CSI_IF_CFG_VREF_POL_POSITIVE ((1 << 18) & CSI_IF_CFG_VREF_POL_MASK) -#define CSI_IF_CFG_HREF_POL_MASK BIT(17) -#define CSI_IF_CFG_HREF_POL_NEGATIVE ((0 << 17) & CSI_IF_CFG_HREF_POL_MASK) -#define CSI_IF_CFG_HREF_POL_POSITIVE ((1 << 17) & CSI_IF_CFG_HREF_POL_MASK) -#define CSI_IF_CFG_CLK_POL_MASK BIT(16) -#define CSI_IF_CFG_CLK_POL_RISING_EDGE ((0 << 16) & CSI_IF_CFG_CLK_POL_MASK) -#define CSI_IF_CFG_CLK_POL_FALLING_EDGE ((1 << 16) & CSI_IF_CFG_CLK_POL_MASK) -#define CSI_IF_CFG_IF_DATA_WIDTH_MASK GENMASK(10, 8) -#define CSI_IF_CFG_IF_DATA_WIDTH_8BIT ((0 << 8) & CSI_IF_CFG_IF_DATA_WIDTH_MASK) -#define CSI_IF_CFG_IF_DATA_WIDTH_10BIT ((1 << 8) & CSI_IF_CFG_IF_DATA_WIDTH_MASK) -#define CSI_IF_CFG_IF_DATA_WIDTH_12BIT ((2 << 8) & CSI_IF_CFG_IF_DATA_WIDTH_MASK) -#define CSI_IF_CFG_MIPI_IF_MASK BIT(7) -#define CSI_IF_CFG_MIPI_IF_CSI (0 << 7) -#define CSI_IF_CFG_MIPI_IF_MIPI BIT(7) -#define CSI_IF_CFG_CSI_IF_MASK GENMASK(4, 0) -#define CSI_IF_CFG_CSI_IF_YUV422_INTLV ((0 << 0) & CSI_IF_CFG_CSI_IF_MASK) -#define CSI_IF_CFG_CSI_IF_YUV422_16BIT ((1 << 0) & CSI_IF_CFG_CSI_IF_MASK) -#define CSI_IF_CFG_CSI_IF_BT656 ((4 << 0) & CSI_IF_CFG_CSI_IF_MASK) -#define CSI_IF_CFG_CSI_IF_BT1120 ((5 << 0) & CSI_IF_CFG_CSI_IF_MASK) - -#define CSI_CAP_REG 0x8 -#define CSI_CAP_CH0_CAP_MASK_MASK GENMASK(5, 2) -#define CSI_CAP_CH0_CAP_MASK(count) (((count) << 2) & CSI_CAP_CH0_CAP_MASK_MASK) -#define CSI_CAP_CH0_VCAP_ON BIT(1) -#define CSI_CAP_CH0_SCAP_ON BIT(0) - -#define CSI_SYNC_CNT_REG 0xc -#define CSI_FIFO_THRS_REG 0x10 -#define CSI_BT656_HEAD_CFG_REG 0x14 -#define CSI_PTN_LEN_REG 0x30 -#define CSI_PTN_ADDR_REG 0x34 -#define CSI_VER_REG 0x3c - -#define CSI_CH_CFG_REG 0x44 -#define CSI_CH_CFG_INPUT_FMT_MASK GENMASK(23, 20) -#define CSI_CH_CFG_INPUT_FMT(fmt) (((fmt) << 20) & CSI_CH_CFG_INPUT_FMT_MASK) -#define CSI_CH_CFG_OUTPUT_FMT_MASK GENMASK(19, 16) -#define CSI_CH_CFG_OUTPUT_FMT(fmt) (((fmt) << 16) & CSI_CH_CFG_OUTPUT_FMT_MASK) -#define CSI_CH_CFG_VFLIP_EN BIT(13) -#define CSI_CH_CFG_HFLIP_EN BIT(12) -#define CSI_CH_CFG_FIELD_SEL_MASK GENMASK(11, 10) -#define CSI_CH_CFG_FIELD_SEL_FIELD0 ((0 << 10) & CSI_CH_CFG_FIELD_SEL_MASK) -#define CSI_CH_CFG_FIELD_SEL_FIELD1 ((1 << 10) & CSI_CH_CFG_FIELD_SEL_MASK) -#define CSI_CH_CFG_FIELD_SEL_BOTH ((2 << 10) & CSI_CH_CFG_FIELD_SEL_MASK) -#define CSI_CH_CFG_INPUT_SEQ_MASK GENMASK(9, 8) -#define CSI_CH_CFG_INPUT_SEQ(seq) (((seq) << 8) & CSI_CH_CFG_INPUT_SEQ_MASK) - -#define CSI_CH_SCALE_REG 0x4c -#define CSI_CH_SCALE_QUART_EN BIT(0) - -#define CSI_CH_F0_BUFA_REG 0x50 - -#define CSI_CH_F1_BUFA_REG 0x58 - -#define CSI_CH_F2_BUFA_REG 0x60 - -#define CSI_CH_STA_REG 0x6c -#define CSI_CH_STA_FIELD_STA_MASK BIT(2) -#define CSI_CH_STA_FIELD_STA_FIELD0 ((0 << 2) & CSI_CH_STA_FIELD_STA_MASK) -#define CSI_CH_STA_FIELD_STA_FIELD1 ((1 << 2) & CSI_CH_STA_FIELD_STA_MASK) -#define CSI_CH_STA_VCAP_STA BIT(1) -#define CSI_CH_STA_SCAP_STA BIT(0) - -#define CSI_CH_INT_EN_REG 0x70 -#define CSI_CH_INT_EN_VS_INT_EN BIT(7) -#define CSI_CH_INT_EN_HB_OF_INT_EN BIT(6) -#define CSI_CH_INT_EN_MUL_ERR_INT_EN BIT(5) -#define CSI_CH_INT_EN_FIFO2_OF_INT_EN BIT(4) -#define CSI_CH_INT_EN_FIFO1_OF_INT_EN BIT(3) -#define CSI_CH_INT_EN_FIFO0_OF_INT_EN BIT(2) -#define CSI_CH_INT_EN_FD_INT_EN BIT(1) -#define CSI_CH_INT_EN_CD_INT_EN BIT(0) - -#define CSI_CH_INT_STA_REG 0x74 -#define CSI_CH_INT_STA_VS_PD BIT(7) -#define CSI_CH_INT_STA_HB_OF_PD BIT(6) -#define CSI_CH_INT_STA_MUL_ERR_PD BIT(5) -#define CSI_CH_INT_STA_FIFO2_OF_PD BIT(4) -#define CSI_CH_INT_STA_FIFO1_OF_PD BIT(3) -#define CSI_CH_INT_STA_FIFO0_OF_PD BIT(2) -#define CSI_CH_INT_STA_FD_PD BIT(1) -#define CSI_CH_INT_STA_CD_PD BIT(0) - -#define CSI_CH_FLD1_VSIZE_REG 0x78 - -#define CSI_CH_HSIZE_REG 0x80 -#define CSI_CH_HSIZE_HOR_LEN_MASK GENMASK(28, 16) -#define CSI_CH_HSIZE_HOR_LEN(len) (((len) << 16) & CSI_CH_HSIZE_HOR_LEN_MASK) -#define CSI_CH_HSIZE_HOR_START_MASK GENMASK(12, 0) -#define CSI_CH_HSIZE_HOR_START(start) (((start) << 0) & CSI_CH_HSIZE_HOR_START_MASK) - -#define CSI_CH_VSIZE_REG 0x84 -#define CSI_CH_VSIZE_VER_LEN_MASK GENMASK(28, 16) -#define CSI_CH_VSIZE_VER_LEN(len) (((len) << 16) & CSI_CH_VSIZE_VER_LEN_MASK) -#define CSI_CH_VSIZE_VER_START_MASK GENMASK(12, 0) -#define CSI_CH_VSIZE_VER_START(start) (((start) << 0) & CSI_CH_VSIZE_VER_START_MASK) - -#define CSI_CH_BUF_LEN_REG 0x88 -#define CSI_CH_BUF_LEN_BUF_LEN_C_MASK GENMASK(29, 16) -#define CSI_CH_BUF_LEN_BUF_LEN_C(len) (((len) << 16) & CSI_CH_BUF_LEN_BUF_LEN_C_MASK) -#define CSI_CH_BUF_LEN_BUF_LEN_Y_MASK GENMASK(13, 0) -#define CSI_CH_BUF_LEN_BUF_LEN_Y(len) (((len) << 0) & CSI_CH_BUF_LEN_BUF_LEN_Y_MASK) - -#define CSI_CH_FLIP_SIZE_REG 0x8c -#define CSI_CH_FLIP_SIZE_VER_LEN_MASK GENMASK(28, 16) -#define CSI_CH_FLIP_SIZE_VER_LEN(len) (((len) << 16) & CSI_CH_FLIP_SIZE_VER_LEN_MASK) -#define CSI_CH_FLIP_SIZE_VALID_LEN_MASK GENMASK(12, 0) -#define CSI_CH_FLIP_SIZE_VALID_LEN(len) (((len) << 0) & CSI_CH_FLIP_SIZE_VALID_LEN_MASK) - -#define CSI_CH_FRM_CLK_CNT_REG 0x90 -#define CSI_CH_ACC_ITNL_CLK_CNT_REG 0x94 -#define CSI_CH_FIFO_STAT_REG 0x98 -#define CSI_CH_PCLK_STAT_REG 0x9c - -/* - * csi input data format - */ -enum csi_input_fmt { - CSI_INPUT_FORMAT_RAW = 0, - CSI_INPUT_FORMAT_YUV422 = 3, - CSI_INPUT_FORMAT_YUV420 = 4, -}; - -/* - * csi output data format - */ -enum csi_output_fmt { - /* only when input format is RAW */ - CSI_FIELD_RAW_8 = 0, - CSI_FIELD_RAW_10 = 1, - CSI_FIELD_RAW_12 = 2, - CSI_FIELD_RGB565 = 4, - CSI_FIELD_RGB888 = 5, - CSI_FIELD_PRGB888 = 6, - CSI_FRAME_RAW_8 = 8, - CSI_FRAME_RAW_10 = 9, - CSI_FRAME_RAW_12 = 10, - CSI_FRAME_RGB565 = 12, - CSI_FRAME_RGB888 = 13, - CSI_FRAME_PRGB888 = 14, - - /* only when input format is YUV422 */ - CSI_FIELD_PLANAR_YUV422 = 0, - CSI_FIELD_PLANAR_YUV420 = 1, - CSI_FRAME_PLANAR_YUV420 = 2, - CSI_FRAME_PLANAR_YUV422 = 3, - CSI_FIELD_UV_CB_YUV422 = 4, - CSI_FIELD_UV_CB_YUV420 = 5, - CSI_FRAME_UV_CB_YUV420 = 6, - CSI_FRAME_UV_CB_YUV422 = 7, - CSI_FIELD_MB_YUV422 = 8, - CSI_FIELD_MB_YUV420 = 9, - CSI_FRAME_MB_YUV420 = 10, - CSI_FRAME_MB_YUV422 = 11, - CSI_FIELD_UV_CB_YUV422_10 = 12, - CSI_FIELD_UV_CB_YUV420_10 = 13, -}; - -/* - * csi YUV input data sequence - */ -enum csi_input_seq { - /* only when input format is YUV422 */ - CSI_INPUT_SEQ_YUYV = 0, - CSI_INPUT_SEQ_YVYU, - CSI_INPUT_SEQ_UYVY, - CSI_INPUT_SEQ_VYUY, -}; - -#endif /* __SUN6I_CSI_REG_H__ */ +#define SUN6I_CSI_ADDR_VALUE(a) ((a) >> 2) + +#define SUN6I_CSI_EN_REG 0x0 +#define SUN6I_CSI_EN_VER_EN BIT(30) +#define SUN6I_CSI_EN_PTN_CYCLE(v) (((v) << 16) & GENMASK(23, 16)) +#define SUN6I_CSI_EN_SRAM_PWDN BIT(8) +#define SUN6I_CSI_EN_PTN_START BIT(4) +#define SUN6I_CSI_EN_CLK_CNT_SPL_VSYNC BIT(3) +#define SUN6I_CSI_EN_CLK_CNT_EN BIT(2) +#define SUN6I_CSI_EN_PTN_GEN_EN BIT(1) +#define SUN6I_CSI_EN_CSI_EN BIT(0) + +/* Note that Allwinner manuals and code invert positive/negative definitions. */ + +#define SUN6I_CSI_IF_CFG_REG 0x4 +#define SUN6I_CSI_IF_CFG_FIELD_DT_PCLK_SHIFT(v) (((v) << 24) & GENMASK(27, 24)) +#define SUN6I_CSI_IF_CFG_SRC_TYPE_PROGRESSIVE (0 << 21) +#define SUN6I_CSI_IF_CFG_SRC_TYPE_INTERLACED (1 << 21) +#define SUN6I_CSI_IF_CFG_FPS_DS BIT(20) +#define SUN6I_CSI_IF_CFG_FIELD_POSITIVE (0 << 19) +#define SUN6I_CSI_IF_CFG_FIELD_NEGATIVE (1 << 19) +#define SUN6I_CSI_IF_CFG_VREF_POL_POSITIVE (0 << 18) +#define SUN6I_CSI_IF_CFG_VREF_POL_NEGATIVE (1 << 18) +#define SUN6I_CSI_IF_CFG_HREF_POL_POSITIVE (0 << 17) +#define SUN6I_CSI_IF_CFG_HREF_POL_NEGATIVE (1 << 17) +#define SUN6I_CSI_IF_CFG_CLK_POL_FALLING (0 << 16) +#define SUN6I_CSI_IF_CFG_CLK_POL_RISING (1 << 16) +#define SUN6I_CSI_IF_CFG_FIELD_DT_FIELD_VSYNC (0 << 14) +#define SUN6I_CSI_IF_CFG_FIELD_DT_FIELD (1 << 14) +#define SUN6I_CSI_IF_CFG_FIELD_DT_VSYNC (2 << 14) +#define SUN6I_CSI_IF_CFG_DATA_WIDTH_8 (0 << 8) +#define SUN6I_CSI_IF_CFG_DATA_WIDTH_10 (1 << 8) +#define SUN6I_CSI_IF_CFG_DATA_WIDTH_12 (2 << 8) +#define SUN6I_CSI_IF_CFG_DATA_WIDTH_8_PLUS_2 (3 << 8) +#define SUN6I_CSI_IF_CFG_DATA_WIDTH_2_TIMES_8 (4 << 8) +#define SUN6I_CSI_IF_CFG_IF_CSI (0 << 7) +#define SUN6I_CSI_IF_CFG_IF_MIPI (1 << 7) +#define SUN6I_CSI_IF_CFG_IF_CSI_YUV_RAW (0 << 0) +#define SUN6I_CSI_IF_CFG_IF_CSI_YUV_COMBINED (1 << 0) +#define SUN6I_CSI_IF_CFG_IF_CSI_BT656 (4 << 0) +#define SUN6I_CSI_IF_CFG_IF_CSI_BT1120 (5 << 0) + +#define SUN6I_CSI_CAP_REG 0x8 +#define SUN6I_CSI_CAP_MASK(v) (((v) << 2) & GENMASK(5, 2)) +#define SUN6I_CSI_CAP_VCAP_ON BIT(1) +#define SUN6I_CSI_CAP_SCAP_ON BIT(0) + +#define SUN6I_CSI_SYNC_CNT_REG 0xc +#define SUN6I_CSI_FIFO_THRS_REG 0x10 +#define SUN6I_CSI_BT656_HEAD_CFG_REG 0x14 + +#define SUN6I_CSI_PTN_LEN_REG 0x30 +#define SUN6I_CSI_PTN_ADDR_REG 0x34 +#define SUN6I_CSI_VER_REG 0x3c + +#define SUN6I_CSI_CH_CFG_REG 0x44 +#define SUN6I_CSI_CH_CFG_PAD_VAL(v) (((v) << 24) & GENMASK(31, 24)) +#define SUN6I_CSI_CH_CFG_INPUT_FMT(v) (((v) << 20) & GENMASK(23, 20)) +#define SUN6I_CSI_CH_CFG_OUTPUT_FMT(v) (((v) << 16) & GENMASK(19, 16)) +#define SUN6I_CSI_CH_CFG_VFLIP_EN BIT(13) +#define SUN6I_CSI_CH_CFG_HFLIP_EN BIT(12) +#define SUN6I_CSI_CH_CFG_FIELD_SEL_FIELD0 (0 << 10) +#define SUN6I_CSI_CH_CFG_FIELD_SEL_FIELD1 (1 << 10) +#define SUN6I_CSI_CH_CFG_FIELD_SEL_EITHER (2 << 10) +#define SUN6I_CSI_CH_CFG_INPUT_YUV_SEQ(v) (((v) << 8) & GENMASK(9, 8)) + +#define SUN6I_CSI_INPUT_FMT_RAW 0 +#define SUN6I_CSI_INPUT_FMT_YUV422 3 +#define SUN6I_CSI_INPUT_FMT_YUV420 4 + +/* Note that Allwinner manuals and code invert frame/field definitions. */ + +/* RAW */ +#define SUN6I_CSI_OUTPUT_FMT_FRAME_RAW_8 0 +#define SUN6I_CSI_OUTPUT_FMT_FRAME_RAW_10 1 +#define SUN6I_CSI_OUTPUT_FMT_FRAME_RAW_12 2 +#define SUN6I_CSI_OUTPUT_FMT_FRAME_RGB565 4 +#define SUN6I_CSI_OUTPUT_FMT_FRAME_RGB888 5 +#define SUN6I_CSI_OUTPUT_FMT_FRAME_PRGB888 6 +#define SUN6I_CSI_OUTPUT_FMT_FIELD_RAW_8 8 +#define SUN6I_CSI_OUTPUT_FMT_FIELD_RAW_10 9 +#define SUN6I_CSI_OUTPUT_FMT_FIELD_RAW_12 10 +#define SUN6I_CSI_OUTPUT_FMT_FIELD_RGB565 12 +#define SUN6I_CSI_OUTPUT_FMT_FIELD_RGB888 13 +#define SUN6I_CSI_OUTPUT_FMT_FIELD_PRGB888 14 + +/* YUV */ +#define SUN6I_CSI_OUTPUT_FMT_FRAME_YUV422P 0 +#define SUN6I_CSI_OUTPUT_FMT_FRAME_YUV420P 1 +#define SUN6I_CSI_OUTPUT_FMT_FIELD_YUV420P 2 +#define SUN6I_CSI_OUTPUT_FMT_FIELD_YUV422P 3 +#define SUN6I_CSI_OUTPUT_FMT_FRAME_YUV422SP 4 +#define SUN6I_CSI_OUTPUT_FMT_FRAME_YUV420SP 5 +#define SUN6I_CSI_OUTPUT_FMT_FIELD_YUV420SP 6 +#define SUN6I_CSI_OUTPUT_FMT_FIELD_YUV422SP 7 +#define SUN6I_CSI_OUTPUT_FMT_FRAME_YUV422MB 8 +#define SUN6I_CSI_OUTPUT_FMT_FRAME_YUV420MB 9 +#define SUN6I_CSI_OUTPUT_FMT_FIELD_YUV420MB 10 +#define SUN6I_CSI_OUTPUT_FMT_FIELD_YUV422MB 11 +#define SUN6I_CSI_OUTPUT_FMT_FRAME_YUV422SP_10 12 +#define SUN6I_CSI_OUTPUT_FMT_FRAME_YUV420SP_10 13 + +/* YUV Planar */ +#define SUN6I_CSI_INPUT_YUV_SEQ_YUYV 0 +#define SUN6I_CSI_INPUT_YUV_SEQ_YVYU 1 +#define SUN6I_CSI_INPUT_YUV_SEQ_UYVY 2 +#define SUN6I_CSI_INPUT_YUV_SEQ_VYUY 3 + +/* YUV Semi-planar */ +#define SUN6I_CSI_INPUT_YUV_SEQ_UV 0 +#define SUN6I_CSI_INPUT_YUV_SEQ_VU 1 + +#define SUN6I_CSI_CH_SCALE_REG 0x4c +#define SUN6I_CSI_CH_SCALE_QUART_EN BIT(0) + +#define SUN6I_CSI_CH_FIFO0_ADDR_REG 0x50 +#define SUN6I_CSI_CH_FIFO1_ADDR_REG 0x58 +#define SUN6I_CSI_CH_FIFO2_ADDR_REG 0x60 + +#define SUN6I_CSI_CH_STA_REG 0x6c +#define SUN6I_CSI_CH_STA_FIELD BIT(2) +#define SUN6I_CSI_CH_STA_VCAP BIT(1) +#define SUN6I_CSI_CH_STA_SCAP BIT(0) + +#define SUN6I_CSI_CH_INT_EN_REG 0x70 +#define SUN6I_CSI_CH_INT_EN_VS BIT(7) +#define SUN6I_CSI_CH_INT_EN_HB_OF BIT(6) +#define SUN6I_CSI_CH_INT_EN_MUL_ERR BIT(5) +#define SUN6I_CSI_CH_INT_EN_FIFO2_OF BIT(4) +#define SUN6I_CSI_CH_INT_EN_FIFO1_OF BIT(3) +#define SUN6I_CSI_CH_INT_EN_FIFO0_OF BIT(2) +#define SUN6I_CSI_CH_INT_EN_FD BIT(1) +#define SUN6I_CSI_CH_INT_EN_CD BIT(0) + +#define SUN6I_CSI_CH_INT_STA_REG 0x74 +#define SUN6I_CSI_CH_INT_STA_CLEAR 0xff +#define SUN6I_CSI_CH_INT_STA_VS BIT(7) +#define SUN6I_CSI_CH_INT_STA_HB_OF BIT(6) +#define SUN6I_CSI_CH_INT_STA_MUL_ERR BIT(5) +#define SUN6I_CSI_CH_INT_STA_FIFO2_OF BIT(4) +#define SUN6I_CSI_CH_INT_STA_FIFO1_OF BIT(3) +#define SUN6I_CSI_CH_INT_STA_FIFO0_OF BIT(2) +#define SUN6I_CSI_CH_INT_STA_FD BIT(1) +#define SUN6I_CSI_CH_INT_STA_CD BIT(0) + +#define SUN6I_CSI_CH_FLD1_VSIZE_REG 0x78 +#define SUN6I_CSI_CH_FLD1_VSIZE_VER_LEN(v) (((v) << 16) & GENMASK(28, 16)) +#define SUN6I_CSI_CH_FLD1_VSIZE_VER_START(v) ((v) & GENMASK(12, 0)) + +#define SUN6I_CSI_CH_HSIZE_REG 0x80 +#define SUN6I_CSI_CH_HSIZE_LEN(v) (((v) << 16) & GENMASK(28, 16)) +#define SUN6I_CSI_CH_HSIZE_START(v) ((v) & GENMASK(12, 0)) + +#define SUN6I_CSI_CH_VSIZE_REG 0x84 +#define SUN6I_CSI_CH_VSIZE_LEN(v) (((v) << 16) & GENMASK(28, 16)) +#define SUN6I_CSI_CH_VSIZE_START(v) ((v) & GENMASK(12, 0)) + +#define SUN6I_CSI_CH_BUF_LEN_REG 0x88 +#define SUN6I_CSI_CH_BUF_LEN_CHROMA_LINE(v) (((v) << 16) & GENMASK(29, 16)) +#define SUN6I_CSI_CH_BUF_LEN_LUMA_LINE(v) ((v) & GENMASK(13, 0)) + +#define SUN6I_CSI_CH_FLIP_SIZE_REG 0x8c +#define SUN6I_CSI_CH_FLIP_SIZE_VER_LEN(v) (((v) << 16) & GENMASK(28, 16)) +#define SUN6I_CSI_CH_FLIP_SIZE_VALID_LEN(v) ((v) & GENMASK(12, 0)) + +#define SUN6I_CSI_CH_FRM_CLK_CNT_REG 0x90 +#define SUN6I_CSI_CH_ACC_ITNL_CLK_CNT_REG 0x94 +#define SUN6I_CSI_CH_FIFO_STAT_REG 0x98 +#define SUN6I_CSI_CH_PCLK_STAT_REG 0x9c + +#endif diff --git a/drivers/media/platform/sunxi/sun6i-csi/sun6i_video.c b/drivers/media/platform/sunxi/sun6i-csi/sun6i_video.c deleted file mode 100644 index 791583d23a65..000000000000 --- a/drivers/media/platform/sunxi/sun6i-csi/sun6i_video.c +++ /dev/null @@ -1,733 +0,0 @@ -// SPDX-License-Identifier: GPL-2.0+ -/* - * Copyright (c) 2011-2018 Magewell Electronics Co., Ltd. (Nanjing) - * All rights reserved. - * Author: Yong Deng <[email protected]> - */ - -#include <linux/of.h> - -#include <media/v4l2-device.h> -#include <media/v4l2-event.h> -#include <media/v4l2-ioctl.h> -#include <media/v4l2-mc.h> -#include <media/videobuf2-dma-contig.h> -#include <media/videobuf2-v4l2.h> - -#include "sun6i_csi.h" -#include "sun6i_video.h" - -/* This is got from BSP sources. */ -#define MIN_WIDTH (32) -#define MIN_HEIGHT (32) -#define MAX_WIDTH (4800) -#define MAX_HEIGHT (4800) - -/* Helpers */ - -static struct v4l2_subdev * -sun6i_video_remote_subdev(struct sun6i_video *video, u32 *pad) -{ - struct media_pad *remote; - - remote = media_pad_remote_pad_first(&video->pad); - - if (!remote || !is_media_entity_v4l2_subdev(remote->entity)) - return NULL; - - if (pad) - *pad = remote->index; - - return media_entity_to_v4l2_subdev(remote->entity); -} - -/* Format */ - -static const u32 sun6i_video_formats[] = { - V4L2_PIX_FMT_SBGGR8, - V4L2_PIX_FMT_SGBRG8, - V4L2_PIX_FMT_SGRBG8, - V4L2_PIX_FMT_SRGGB8, - V4L2_PIX_FMT_SBGGR10, - V4L2_PIX_FMT_SGBRG10, - V4L2_PIX_FMT_SGRBG10, - V4L2_PIX_FMT_SRGGB10, - V4L2_PIX_FMT_SBGGR12, - V4L2_PIX_FMT_SGBRG12, - V4L2_PIX_FMT_SGRBG12, - V4L2_PIX_FMT_SRGGB12, - V4L2_PIX_FMT_YUYV, - V4L2_PIX_FMT_YVYU, - V4L2_PIX_FMT_UYVY, - V4L2_PIX_FMT_VYUY, - V4L2_PIX_FMT_NV12_16L16, - V4L2_PIX_FMT_NV12, - V4L2_PIX_FMT_NV21, - V4L2_PIX_FMT_YUV420, - V4L2_PIX_FMT_YVU420, - V4L2_PIX_FMT_NV16, - V4L2_PIX_FMT_NV61, - V4L2_PIX_FMT_YUV422P, - V4L2_PIX_FMT_RGB565, - V4L2_PIX_FMT_RGB565X, - V4L2_PIX_FMT_JPEG, -}; - -static bool sun6i_video_format_check(u32 format) -{ - unsigned int i; - - for (i = 0; i < ARRAY_SIZE(sun6i_video_formats); i++) - if (sun6i_video_formats[i] == format) - return true; - - return false; -} - -/* Video */ - -static void sun6i_video_buffer_configure(struct sun6i_csi_device *csi_dev, - struct sun6i_csi_buffer *csi_buffer) -{ - csi_buffer->queued_to_csi = true; - sun6i_csi_update_buf_addr(csi_dev, csi_buffer->dma_addr); -} - -static void sun6i_video_configure(struct sun6i_csi_device *csi_dev) -{ - struct sun6i_video *video = &csi_dev->video; - struct sun6i_csi_config config = { 0 }; - - config.pixelformat = video->format.fmt.pix.pixelformat; - config.code = video->mbus_code; - config.field = video->format.fmt.pix.field; - config.width = video->format.fmt.pix.width; - config.height = video->format.fmt.pix.height; - - sun6i_csi_update_config(csi_dev, &config); -} - -/* Queue */ - -static int sun6i_video_queue_setup(struct vb2_queue *queue, - unsigned int *buffers_count, - unsigned int *planes_count, - unsigned int sizes[], - struct device *alloc_devs[]) -{ - struct sun6i_csi_device *csi_dev = vb2_get_drv_priv(queue); - struct sun6i_video *video = &csi_dev->video; - unsigned int size = video->format.fmt.pix.sizeimage; - - if (*planes_count) - return sizes[0] < size ? -EINVAL : 0; - - *planes_count = 1; - sizes[0] = size; - - return 0; -} - -static int sun6i_video_buffer_prepare(struct vb2_buffer *buffer) -{ - struct sun6i_csi_device *csi_dev = vb2_get_drv_priv(buffer->vb2_queue); - struct sun6i_video *video = &csi_dev->video; - struct v4l2_device *v4l2_dev = &csi_dev->v4l2.v4l2_dev; - struct vb2_v4l2_buffer *v4l2_buffer = to_vb2_v4l2_buffer(buffer); - struct sun6i_csi_buffer *csi_buffer = - container_of(v4l2_buffer, struct sun6i_csi_buffer, v4l2_buffer); - unsigned long size = video->format.fmt.pix.sizeimage; - - if (vb2_plane_size(buffer, 0) < size) { - v4l2_err(v4l2_dev, "buffer too small (%lu < %lu)\n", - vb2_plane_size(buffer, 0), size); - return -EINVAL; - } - - vb2_set_plane_payload(buffer, 0, size); - - csi_buffer->dma_addr = vb2_dma_contig_plane_dma_addr(buffer, 0); - v4l2_buffer->field = video->format.fmt.pix.field; - - return 0; -} - -static void sun6i_video_buffer_queue(struct vb2_buffer *buffer) -{ - struct sun6i_csi_device *csi_dev = vb2_get_drv_priv(buffer->vb2_queue); - struct sun6i_video *video = &csi_dev->video; - struct vb2_v4l2_buffer *v4l2_buffer = to_vb2_v4l2_buffer(buffer); - struct sun6i_csi_buffer *csi_buffer = - container_of(v4l2_buffer, struct sun6i_csi_buffer, v4l2_buffer); - unsigned long flags; - - spin_lock_irqsave(&video->dma_queue_lock, flags); - csi_buffer->queued_to_csi = false; - list_add_tail(&csi_buffer->list, &video->dma_queue); - spin_unlock_irqrestore(&video->dma_queue_lock, flags); -} - -static int sun6i_video_start_streaming(struct vb2_queue *queue, - unsigned int count) -{ - struct sun6i_csi_device *csi_dev = vb2_get_drv_priv(queue); - struct sun6i_video *video = &csi_dev->video; - struct video_device *video_dev = &video->video_dev; - struct sun6i_csi_buffer *buf; - struct sun6i_csi_buffer *next_buf; - struct v4l2_subdev *subdev; - unsigned long flags; - int ret; - - video->sequence = 0; - - ret = video_device_pipeline_alloc_start(video_dev); - if (ret < 0) - goto error_dma_queue_flush; - - if (video->mbus_code == 0) { - ret = -EINVAL; - goto error_media_pipeline; - } - - subdev = sun6i_video_remote_subdev(video, NULL); - if (!subdev) { - ret = -EINVAL; - goto error_media_pipeline; - } - - sun6i_video_configure(csi_dev); - - spin_lock_irqsave(&video->dma_queue_lock, flags); - - buf = list_first_entry(&video->dma_queue, - struct sun6i_csi_buffer, list); - sun6i_video_buffer_configure(csi_dev, buf); - - sun6i_csi_set_stream(csi_dev, true); - - /* - * CSI will lookup the next dma buffer for next frame before the - * current frame done IRQ triggered. This is not documented - * but reported by Ondřej Jirman. - * The BSP code has workaround for this too. It skip to mark the - * first buffer as frame done for VB2 and pass the second buffer - * to CSI in the first frame done ISR call. Then in second frame - * done ISR call, it mark the first buffer as frame done for VB2 - * and pass the third buffer to CSI. And so on. The bad thing is - * that the first buffer will be written twice and the first frame - * is dropped even the queued buffer is sufficient. - * So, I make some improvement here. Pass the next buffer to CSI - * just follow starting the CSI. In this case, the first frame - * will be stored in first buffer, second frame in second buffer. - * This method is used to avoid dropping the first frame, it - * would also drop frame when lacking of queued buffer. - */ - next_buf = list_next_entry(buf, list); - sun6i_video_buffer_configure(csi_dev, next_buf); - - spin_unlock_irqrestore(&video->dma_queue_lock, flags); - - ret = v4l2_subdev_call(subdev, video, s_stream, 1); - if (ret && ret != -ENOIOCTLCMD) - goto error_stream; - - return 0; - -error_stream: - sun6i_csi_set_stream(csi_dev, false); - -error_media_pipeline: - video_device_pipeline_stop(video_dev); - -error_dma_queue_flush: - spin_lock_irqsave(&video->dma_queue_lock, flags); - list_for_each_entry(buf, &video->dma_queue, list) - vb2_buffer_done(&buf->v4l2_buffer.vb2_buf, - VB2_BUF_STATE_QUEUED); - INIT_LIST_HEAD(&video->dma_queue); - spin_unlock_irqrestore(&video->dma_queue_lock, flags); - - return ret; -} - -static void sun6i_video_stop_streaming(struct vb2_queue *queue) -{ - struct sun6i_csi_device *csi_dev = vb2_get_drv_priv(queue); - struct sun6i_video *video = &csi_dev->video; - struct v4l2_subdev *subdev; - unsigned long flags; - struct sun6i_csi_buffer *buf; - - subdev = sun6i_video_remote_subdev(video, NULL); - if (subdev) - v4l2_subdev_call(subdev, video, s_stream, 0); - - sun6i_csi_set_stream(csi_dev, false); - - video_device_pipeline_stop(&video->video_dev); - - /* Release all active buffers */ - spin_lock_irqsave(&video->dma_queue_lock, flags); - list_for_each_entry(buf, &video->dma_queue, list) - vb2_buffer_done(&buf->v4l2_buffer.vb2_buf, VB2_BUF_STATE_ERROR); - INIT_LIST_HEAD(&video->dma_queue); - spin_unlock_irqrestore(&video->dma_queue_lock, flags); -} - -void sun6i_video_frame_done(struct sun6i_csi_device *csi_dev) -{ - struct sun6i_video *video = &csi_dev->video; - struct sun6i_csi_buffer *buf; - struct sun6i_csi_buffer *next_buf; - struct vb2_v4l2_buffer *v4l2_buffer; - - spin_lock(&video->dma_queue_lock); - - buf = list_first_entry(&video->dma_queue, - struct sun6i_csi_buffer, list); - if (list_is_last(&buf->list, &video->dma_queue)) { - dev_dbg(csi_dev->dev, "Frame dropped!\n"); - goto complete; - } - - next_buf = list_next_entry(buf, list); - /* If a new buffer (#next_buf) had not been queued to CSI, the old - * buffer (#buf) is still holding by CSI for storing the next - * frame. So, we queue a new buffer (#next_buf) to CSI then wait - * for next ISR call. - */ - if (!next_buf->queued_to_csi) { - sun6i_video_buffer_configure(csi_dev, next_buf); - dev_dbg(csi_dev->dev, "Frame dropped!\n"); - goto complete; - } - - list_del(&buf->list); - v4l2_buffer = &buf->v4l2_buffer; - v4l2_buffer->vb2_buf.timestamp = ktime_get_ns(); - v4l2_buffer->sequence = video->sequence; - vb2_buffer_done(&v4l2_buffer->vb2_buf, VB2_BUF_STATE_DONE); - - /* Prepare buffer for next frame but one. */ - if (!list_is_last(&next_buf->list, &video->dma_queue)) { - next_buf = list_next_entry(next_buf, list); - sun6i_video_buffer_configure(csi_dev, next_buf); - } else { - dev_dbg(csi_dev->dev, "Next frame will be dropped!\n"); - } - -complete: - video->sequence++; - spin_unlock(&video->dma_queue_lock); -} - -static const struct vb2_ops sun6i_video_queue_ops = { - .queue_setup = sun6i_video_queue_setup, - .buf_prepare = sun6i_video_buffer_prepare, - .buf_queue = sun6i_video_buffer_queue, - .start_streaming = sun6i_video_start_streaming, - .stop_streaming = sun6i_video_stop_streaming, - .wait_prepare = vb2_ops_wait_prepare, - .wait_finish = vb2_ops_wait_finish, -}; - -/* V4L2 Device */ - -static int sun6i_video_querycap(struct file *file, void *private, - struct v4l2_capability *capability) -{ - struct sun6i_csi_device *csi_dev = video_drvdata(file); - struct video_device *video_dev = &csi_dev->video.video_dev; - - strscpy(capability->driver, SUN6I_CSI_NAME, sizeof(capability->driver)); - strscpy(capability->card, video_dev->name, sizeof(capability->card)); - snprintf(capability->bus_info, sizeof(capability->bus_info), - "platform:%s", dev_name(csi_dev->dev)); - - return 0; -} - -static int sun6i_video_enum_fmt(struct file *file, void *private, - struct v4l2_fmtdesc *fmtdesc) -{ - u32 index = fmtdesc->index; - - if (index >= ARRAY_SIZE(sun6i_video_formats)) - return -EINVAL; - - fmtdesc->pixelformat = sun6i_video_formats[index]; - - return 0; -} - -static int sun6i_video_g_fmt(struct file *file, void *private, - struct v4l2_format *format) -{ - struct sun6i_csi_device *csi_dev = video_drvdata(file); - struct sun6i_video *video = &csi_dev->video; - - *format = video->format; - - return 0; -} - -static int sun6i_video_format_try(struct sun6i_video *video, - struct v4l2_format *format) -{ - struct v4l2_pix_format *pix_format = &format->fmt.pix; - int bpp; - - if (!sun6i_video_format_check(pix_format->pixelformat)) - pix_format->pixelformat = sun6i_video_formats[0]; - - v4l_bound_align_image(&pix_format->width, MIN_WIDTH, MAX_WIDTH, 1, - &pix_format->height, MIN_HEIGHT, MAX_WIDTH, 1, 1); - - bpp = sun6i_csi_get_bpp(pix_format->pixelformat); - pix_format->bytesperline = (pix_format->width * bpp) >> 3; - pix_format->sizeimage = pix_format->bytesperline * pix_format->height; - - if (pix_format->field == V4L2_FIELD_ANY) - pix_format->field = V4L2_FIELD_NONE; - - if (pix_format->pixelformat == V4L2_PIX_FMT_JPEG) - pix_format->colorspace = V4L2_COLORSPACE_JPEG; - else - pix_format->colorspace = V4L2_COLORSPACE_SRGB; - - pix_format->ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT; - pix_format->quantization = V4L2_QUANTIZATION_DEFAULT; - pix_format->xfer_func = V4L2_XFER_FUNC_DEFAULT; - - return 0; -} - -static int sun6i_video_format_set(struct sun6i_video *video, - struct v4l2_format *format) -{ - int ret; - - ret = sun6i_video_format_try(video, format); - if (ret) - return ret; - - video->format = *format; - - return 0; -} - -static int sun6i_video_s_fmt(struct file *file, void *private, - struct v4l2_format *format) -{ - struct sun6i_csi_device *csi_dev = video_drvdata(file); - struct sun6i_video *video = &csi_dev->video; - - if (vb2_is_busy(&video->queue)) - return -EBUSY; - - return sun6i_video_format_set(video, format); -} - -static int sun6i_video_try_fmt(struct file *file, void *private, - struct v4l2_format *format) -{ - struct sun6i_csi_device *csi_dev = video_drvdata(file); - struct sun6i_video *video = &csi_dev->video; - - return sun6i_video_format_try(video, format); -} - -static int sun6i_video_enum_input(struct file *file, void *private, - struct v4l2_input *input) -{ - if (input->index != 0) - return -EINVAL; - - input->type = V4L2_INPUT_TYPE_CAMERA; - strscpy(input->name, "Camera", sizeof(input->name)); - - return 0; -} - -static int sun6i_video_g_input(struct file *file, void *private, - unsigned int *index) -{ - *index = 0; - - return 0; -} - -static int sun6i_video_s_input(struct file *file, void *private, - unsigned int index) -{ - if (index != 0) - return -EINVAL; - - return 0; -} - -static const struct v4l2_ioctl_ops sun6i_video_ioctl_ops = { - .vidioc_querycap = sun6i_video_querycap, - - .vidioc_enum_fmt_vid_cap = sun6i_video_enum_fmt, - .vidioc_g_fmt_vid_cap = sun6i_video_g_fmt, - .vidioc_s_fmt_vid_cap = sun6i_video_s_fmt, - .vidioc_try_fmt_vid_cap = sun6i_video_try_fmt, - - .vidioc_enum_input = sun6i_video_enum_input, - .vidioc_g_input = sun6i_video_g_input, - .vidioc_s_input = sun6i_video_s_input, - - .vidioc_create_bufs = vb2_ioctl_create_bufs, - .vidioc_prepare_buf = vb2_ioctl_prepare_buf, - .vidioc_reqbufs = vb2_ioctl_reqbufs, - .vidioc_querybuf = vb2_ioctl_querybuf, - .vidioc_expbuf = vb2_ioctl_expbuf, - .vidioc_qbuf = vb2_ioctl_qbuf, - .vidioc_dqbuf = vb2_ioctl_dqbuf, - .vidioc_streamon = vb2_ioctl_streamon, - .vidioc_streamoff = vb2_ioctl_streamoff, -}; - -/* V4L2 File */ - -static int sun6i_video_open(struct file *file) -{ - struct sun6i_csi_device *csi_dev = video_drvdata(file); - struct sun6i_video *video = &csi_dev->video; - int ret = 0; - - if (mutex_lock_interruptible(&video->lock)) - return -ERESTARTSYS; - - ret = v4l2_fh_open(file); - if (ret < 0) - goto error_lock; - - ret = v4l2_pipeline_pm_get(&video->video_dev.entity); - if (ret < 0) - goto error_v4l2_fh; - - /* Power on at first open. */ - if (v4l2_fh_is_singular_file(file)) { - ret = sun6i_csi_set_power(csi_dev, true); - if (ret < 0) - goto error_v4l2_fh; - } - - mutex_unlock(&video->lock); - - return 0; - -error_v4l2_fh: - v4l2_fh_release(file); - -error_lock: - mutex_unlock(&video->lock); - - return ret; -} - -static int sun6i_video_close(struct file *file) -{ - struct sun6i_csi_device *csi_dev = video_drvdata(file); - struct sun6i_video *video = &csi_dev->video; - bool last_close; - - mutex_lock(&video->lock); - - last_close = v4l2_fh_is_singular_file(file); - - _vb2_fop_release(file, NULL); - v4l2_pipeline_pm_put(&video->video_dev.entity); - - /* Power off at last close. */ - if (last_close) - sun6i_csi_set_power(csi_dev, false); - - mutex_unlock(&video->lock); - - return 0; -} - -static const struct v4l2_file_operations sun6i_video_fops = { - .owner = THIS_MODULE, - .open = sun6i_video_open, - .release = sun6i_video_close, - .unlocked_ioctl = video_ioctl2, - .mmap = vb2_fop_mmap, - .poll = vb2_fop_poll -}; - -/* Media Entity */ - -static int sun6i_video_link_validate_get_format(struct media_pad *pad, - struct v4l2_subdev_format *fmt) -{ - if (is_media_entity_v4l2_subdev(pad->entity)) { - struct v4l2_subdev *sd = - media_entity_to_v4l2_subdev(pad->entity); - - fmt->which = V4L2_SUBDEV_FORMAT_ACTIVE; - fmt->pad = pad->index; - return v4l2_subdev_call(sd, pad, get_fmt, NULL, fmt); - } - - return -EINVAL; -} - -static int sun6i_video_link_validate(struct media_link *link) -{ - struct video_device *vdev = container_of(link->sink->entity, - struct video_device, entity); - struct sun6i_csi_device *csi_dev = video_get_drvdata(vdev); - struct sun6i_video *video = &csi_dev->video; - struct v4l2_subdev_format source_fmt; - int ret; - - video->mbus_code = 0; - - if (!media_pad_remote_pad_first(link->sink->entity->pads)) { - dev_info(csi_dev->dev, "video node %s pad not connected\n", - vdev->name); - return -ENOLINK; - } - - ret = sun6i_video_link_validate_get_format(link->source, &source_fmt); - if (ret < 0) - return ret; - - if (!sun6i_csi_is_format_supported(csi_dev, - video->format.fmt.pix.pixelformat, - source_fmt.format.code)) { - dev_err(csi_dev->dev, - "Unsupported pixformat: 0x%x with mbus code: 0x%x!\n", - video->format.fmt.pix.pixelformat, - source_fmt.format.code); - return -EPIPE; - } - - if (source_fmt.format.width != video->format.fmt.pix.width || - source_fmt.format.height != video->format.fmt.pix.height) { - dev_err(csi_dev->dev, - "Wrong width or height %ux%u (%ux%u expected)\n", - video->format.fmt.pix.width, video->format.fmt.pix.height, - source_fmt.format.width, source_fmt.format.height); - return -EPIPE; - } - - video->mbus_code = source_fmt.format.code; - - return 0; -} - -static const struct media_entity_operations sun6i_video_media_ops = { - .link_validate = sun6i_video_link_validate -}; - -/* Video */ - -int sun6i_video_setup(struct sun6i_csi_device *csi_dev) -{ - struct sun6i_video *video = &csi_dev->video; - struct v4l2_device *v4l2_dev = &csi_dev->v4l2.v4l2_dev; - struct video_device *video_dev = &video->video_dev; - struct vb2_queue *queue = &video->queue; - struct media_pad *pad = &video->pad; - struct v4l2_format format = { 0 }; - struct v4l2_pix_format *pix_format = &format.fmt.pix; - int ret; - - /* Media Entity */ - - video_dev->entity.ops = &sun6i_video_media_ops; - - /* Media Pad */ - - pad->flags = MEDIA_PAD_FL_SINK | MEDIA_PAD_FL_MUST_CONNECT; - - ret = media_entity_pads_init(&video_dev->entity, 1, pad); - if (ret < 0) - return ret; - - /* DMA queue */ - - INIT_LIST_HEAD(&video->dma_queue); - spin_lock_init(&video->dma_queue_lock); - - video->sequence = 0; - - /* Queue */ - - mutex_init(&video->lock); - - queue->type = V4L2_BUF_TYPE_VIDEO_CAPTURE; - queue->io_modes = VB2_MMAP | VB2_DMABUF; - queue->buf_struct_size = sizeof(struct sun6i_csi_buffer); - queue->ops = &sun6i_video_queue_ops; - queue->mem_ops = &vb2_dma_contig_memops; - queue->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC; - queue->lock = &video->lock; - queue->dev = csi_dev->dev; - queue->drv_priv = csi_dev; - - /* Make sure non-dropped frame. */ - queue->min_buffers_needed = 3; - - ret = vb2_queue_init(queue); - if (ret) { - v4l2_err(v4l2_dev, "failed to initialize vb2 queue: %d\n", ret); - goto error_media_entity; - } - - /* V4L2 Format */ - - format.type = queue->type; - pix_format->pixelformat = sun6i_video_formats[0]; - pix_format->width = 1280; - pix_format->height = 720; - pix_format->field = V4L2_FIELD_NONE; - - sun6i_video_format_set(video, &format); - - /* Video Device */ - - strscpy(video_dev->name, SUN6I_CSI_NAME, sizeof(video_dev->name)); - video_dev->device_caps = V4L2_CAP_VIDEO_CAPTURE | V4L2_CAP_STREAMING; - video_dev->vfl_dir = VFL_DIR_RX; - video_dev->release = video_device_release_empty; - video_dev->fops = &sun6i_video_fops; - video_dev->ioctl_ops = &sun6i_video_ioctl_ops; - video_dev->v4l2_dev = v4l2_dev; - video_dev->queue = queue; - video_dev->lock = &video->lock; - - video_set_drvdata(video_dev, csi_dev); - - ret = video_register_device(video_dev, VFL_TYPE_VIDEO, -1); - if (ret < 0) { - v4l2_err(v4l2_dev, "failed to register video device: %d\n", - ret); - goto error_media_entity; - } - - return 0; - -error_media_entity: - media_entity_cleanup(&video_dev->entity); - - mutex_destroy(&video->lock); - - return ret; -} - -void sun6i_video_cleanup(struct sun6i_csi_device *csi_dev) -{ - struct sun6i_video *video = &csi_dev->video; - struct video_device *video_dev = &video->video_dev; - - vb2_video_unregister_device(video_dev); - media_entity_cleanup(&video_dev->entity); - mutex_destroy(&video->lock); -} diff --git a/drivers/media/platform/sunxi/sun6i-csi/sun6i_video.h b/drivers/media/platform/sunxi/sun6i-csi/sun6i_video.h deleted file mode 100644 index a917d2da6deb..000000000000 --- a/drivers/media/platform/sunxi/sun6i-csi/sun6i_video.h +++ /dev/null @@ -1,35 +0,0 @@ -/* SPDX-License-Identifier: GPL-2.0+ */ -/* - * Copyright (c) 2011-2018 Magewell Electronics Co., Ltd. (Nanjing) - * All rights reserved. - * Author: Yong Deng <[email protected]> - */ - -#ifndef __SUN6I_VIDEO_H__ -#define __SUN6I_VIDEO_H__ - -#include <media/v4l2-dev.h> -#include <media/videobuf2-core.h> - -struct sun6i_csi_device; - -struct sun6i_video { - struct video_device video_dev; - struct vb2_queue queue; - struct mutex lock; /* Queue lock. */ - struct media_pad pad; - - struct list_head dma_queue; - spinlock_t dma_queue_lock; /* DMA queue lock. */ - - struct v4l2_format format; - u32 mbus_code; - unsigned int sequence; -}; - -int sun6i_video_setup(struct sun6i_csi_device *csi_dev); -void sun6i_video_cleanup(struct sun6i_csi_device *csi_dev); - -void sun6i_video_frame_done(struct sun6i_csi_device *csi_dev); - -#endif /* __SUN6I_VIDEO_H__ */ diff --git a/drivers/media/radio/radio-terratec.c b/drivers/media/radio/radio-terratec.c index 8b8ce2b46a55..621bb8523271 100644 --- a/drivers/media/radio/radio-terratec.c +++ b/drivers/media/radio/radio-terratec.c @@ -82,7 +82,6 @@ static int terratec_s_mute_volume(struct radio_isa_card *isa, bool mute, int vol static int terratec_s_frequency(struct radio_isa_card *isa, u32 freq) { int i; - int p; int temp; long rest; unsigned char buffer[25]; /* we have to bit shift 25 registers */ @@ -93,7 +92,6 @@ static int terratec_s_frequency(struct radio_isa_card *isa, u32 freq) rest = freq * 10 + 10700; /* I once had understood what is going on here */ /* maybe some wise guy (friedhelm?) can comment this stuff */ i = 13; - p = 10; temp = 102400; while (rest != 0) { if (rest % temp == rest) @@ -103,7 +101,6 @@ static int terratec_s_frequency(struct radio_isa_card *isa, u32 freq) rest = rest - temp; } i--; - p--; temp = temp / 2; } diff --git a/drivers/media/test-drivers/Kconfig b/drivers/media/test-drivers/Kconfig index 51cf27834df0..459b433e9fae 100644 --- a/drivers/media/test-drivers/Kconfig +++ b/drivers/media/test-drivers/Kconfig @@ -20,6 +20,7 @@ config VIDEO_VIM2M source "drivers/media/test-drivers/vicodec/Kconfig" source "drivers/media/test-drivers/vimc/Kconfig" source "drivers/media/test-drivers/vivid/Kconfig" +source "drivers/media/test-drivers/visl/Kconfig" endif #V4L_TEST_DRIVERS diff --git a/drivers/media/test-drivers/Makefile b/drivers/media/test-drivers/Makefile index ff390b687189..740714a4584d 100644 --- a/drivers/media/test-drivers/Makefile +++ b/drivers/media/test-drivers/Makefile @@ -12,3 +12,4 @@ obj-$(CONFIG_VIDEO_VICODEC) += vicodec/ obj-$(CONFIG_VIDEO_VIM2M) += vim2m.o obj-$(CONFIG_VIDEO_VIMC) += vimc/ obj-$(CONFIG_VIDEO_VIVID) += vivid/ +obj-$(CONFIG_VIDEO_VISL) += visl/ diff --git a/drivers/media/test-drivers/vidtv/vidtv_bridge.c b/drivers/media/test-drivers/vidtv/vidtv_bridge.c index 82620613d56b..dff7265a42ca 100644 --- a/drivers/media/test-drivers/vidtv/vidtv_bridge.c +++ b/drivers/media/test-drivers/vidtv/vidtv_bridge.c @@ -459,26 +459,20 @@ fail_dmx_conn: for (j = j - 1; j >= 0; --j) dvb->demux.dmx.remove_frontend(&dvb->demux.dmx, &dvb->dmx_fe[j]); -fail_dmx_dev: dvb_dmxdev_release(&dvb->dmx_dev); -fail_dmx: +fail_dmx_dev: dvb_dmx_release(&dvb->demux); +fail_dmx: +fail_demod_probe: + for (i = i - 1; i >= 0; --i) { + dvb_unregister_frontend(dvb->fe[i]); fail_fe: - for (j = i; j >= 0; --j) - dvb_unregister_frontend(dvb->fe[j]); + dvb_module_release(dvb->i2c_client_tuner[i]); fail_tuner_probe: - for (j = i; j >= 0; --j) - if (dvb->i2c_client_tuner[j]) - dvb_module_release(dvb->i2c_client_tuner[j]); - -fail_demod_probe: - for (j = i; j >= 0; --j) - if (dvb->i2c_client_demod[j]) - dvb_module_release(dvb->i2c_client_demod[j]); - + dvb_module_release(dvb->i2c_client_demod[i]); + } fail_adapter: dvb_unregister_adapter(&dvb->adapter); - fail_i2c: i2c_del_adapter(&dvb->i2c_adapter); diff --git a/drivers/media/test-drivers/vimc/vimc-core.c b/drivers/media/test-drivers/vimc/vimc-core.c index 2ae7a0f11ebf..e82cfa5ffbf4 100644 --- a/drivers/media/test-drivers/vimc/vimc-core.c +++ b/drivers/media/test-drivers/vimc/vimc-core.c @@ -433,7 +433,7 @@ static int __init vimc_init(void) if (ret) { dev_err(&vimc_pdev.dev, "platform driver registration failed (err=%d)\n", ret); - platform_driver_unregister(&vimc_pdrv); + platform_device_unregister(&vimc_pdev); return ret; } diff --git a/drivers/media/test-drivers/visl/Kconfig b/drivers/media/test-drivers/visl/Kconfig new file mode 100644 index 000000000000..7508b904f196 --- /dev/null +++ b/drivers/media/test-drivers/visl/Kconfig @@ -0,0 +1,29 @@ +# SPDX-License-Identifier: GPL-2.0+ +config VIDEO_VISL + tristate "Virtual Stateless Decoder Driver (visl)" + depends on VIDEO_DEV + select FONT_SUPPORT + select FONT_8x16 + select VIDEOBUF2_VMALLOC + select V4L2_MEM2MEM_DEV + select MEDIA_CONTROLLER + select MEDIA_CONTROLLER_REQUEST_API + select VIDEO_V4L2_TPG + help + + A virtual stateless decoder device for uAPI development purposes. + + A userspace implementation can use visl to run a decoding loop even + when no hardware is available or when the kernel uAPI for the codec + has not been upstreamed yet. This can reveal bugs at an early stage. + + When in doubt, say N. + +config VISL_DEBUGFS + bool "Enable debugfs for visl" + depends on VIDEO_VISL + depends on DEBUG_FS + + help + Choose Y to dump the bitstream buffers through debugfs. + When in doubt, say N. diff --git a/drivers/media/test-drivers/visl/Makefile b/drivers/media/test-drivers/visl/Makefile new file mode 100644 index 000000000000..fb4d5ae1b17f --- /dev/null +++ b/drivers/media/test-drivers/visl/Makefile @@ -0,0 +1,8 @@ +# SPDX-License-Identifier: GPL-2.0+ +visl-y := visl-core.o visl-video.o visl-dec.o visl-trace-points.o + +ifeq ($(CONFIG_VISL_DEBUGFS),y) + visl-y += visl-debugfs.o +endif + +obj-$(CONFIG_VIDEO_VISL) += visl.o diff --git a/drivers/media/test-drivers/visl/visl-core.c b/drivers/media/test-drivers/visl/visl-core.c new file mode 100644 index 000000000000..9cb60ab653bf --- /dev/null +++ b/drivers/media/test-drivers/visl/visl-core.c @@ -0,0 +1,541 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * A virtual stateless decoder device for stateless uAPI development purposes. + * + * This tool's objective is to help the development and testing of userspace + * applications that use the V4L2 stateless API to decode media. + * + * A userspace implementation can use visl to run a decoding loop even when no + * hardware is available or when the kernel uAPI for the codec has not been + * upstreamed yet. This can reveal bugs at an early stage. + * + * This driver can also trace the contents of the V4L2 controls submitted to it. + * It can also dump the contents of the vb2 buffers through a debugfs + * interface. This is in many ways similar to the tracing infrastructure + * available for other popular encode/decode APIs out there and can help develop + * a userspace application by using another (working) one as a reference. + * + * Note that no actual decoding of video frames is performed by visl. The V4L2 + * test pattern generator is used to write various debug information to the + * capture buffers instead. + * + * Copyright (C) 2022 Collabora, Ltd. + * + * Based on the vim2m driver, that is: + * + * Copyright (c) 2009-2010 Samsung Electronics Co., Ltd. + * Pawel Osciak, <[email protected]> + * Marek Szyprowski, <[email protected]> + * + * Based on the vicodec driver, that is: + * + * Copyright 2018 Cisco Systems, Inc. and/or its affiliates. All rights reserved. + * + * Based on the Cedrus VPU driver, that is: + * + * Copyright (C) 2016 Florent Revest <[email protected]> + * Copyright (C) 2018 Paul Kocialkowski <[email protected]> + * Copyright (C) 2018 Bootlin + */ + +#include <linux/debugfs.h> +#include <linux/module.h> +#include <linux/platform_device.h> +#include <media/v4l2-ctrls.h> +#include <media/v4l2-device.h> +#include <media/v4l2-ioctl.h> +#include <media/v4l2-mem2mem.h> + +#include "visl.h" +#include "visl-dec.h" +#include "visl-debugfs.h" +#include "visl-video.h" + +unsigned int visl_debug; +module_param(visl_debug, uint, 0644); +MODULE_PARM_DESC(visl_debug, " activates debug info"); + +unsigned int visl_transtime_ms; +module_param(visl_transtime_ms, uint, 0644); +MODULE_PARM_DESC(visl_transtime_ms, " simulated process time in milliseconds."); + +/* + * dprintk can be slow through serial. This lets one limit the tracing to a + * particular number of frames + */ +int visl_dprintk_frame_start = -1; +module_param(visl_dprintk_frame_start, int, 0); +MODULE_PARM_DESC(visl_dprintk_frame_start, + " a frame number to start tracing with dprintk"); + +unsigned int visl_dprintk_nframes; +module_param(visl_dprintk_nframes, uint, 0); +MODULE_PARM_DESC(visl_dprintk_nframes, + " the number of frames to trace with dprintk"); + +bool keep_bitstream_buffers; +module_param(keep_bitstream_buffers, bool, false); +MODULE_PARM_DESC(keep_bitstream_buffers, + " keep bitstream buffers in debugfs after streaming is stopped"); + +int bitstream_trace_frame_start = -1; +module_param(bitstream_trace_frame_start, int, 0); +MODULE_PARM_DESC(bitstream_trace_frame_start, + " a frame number to start dumping the bitstream through debugfs"); + +unsigned int bitstream_trace_nframes; +module_param(bitstream_trace_nframes, uint, 0); +MODULE_PARM_DESC(bitstream_trace_nframes, + " the number of frames to dump the bitstream through debugfs"); + +static const struct visl_ctrl_desc visl_fwht_ctrl_descs[] = { + { + .cfg.id = V4L2_CID_STATELESS_FWHT_PARAMS, + }, +}; + +const struct visl_ctrls visl_fwht_ctrls = { + .ctrls = visl_fwht_ctrl_descs, + .num_ctrls = ARRAY_SIZE(visl_fwht_ctrl_descs) +}; + +static const struct visl_ctrl_desc visl_mpeg2_ctrl_descs[] = { + { + .cfg.id = V4L2_CID_STATELESS_MPEG2_SEQUENCE, + }, + { + .cfg.id = V4L2_CID_STATELESS_MPEG2_PICTURE, + }, + { + .cfg.id = V4L2_CID_STATELESS_MPEG2_QUANTISATION, + }, +}; + +const struct visl_ctrls visl_mpeg2_ctrls = { + .ctrls = visl_mpeg2_ctrl_descs, + .num_ctrls = ARRAY_SIZE(visl_mpeg2_ctrl_descs), +}; + +static const struct visl_ctrl_desc visl_vp8_ctrl_descs[] = { + { + .cfg.id = V4L2_CID_STATELESS_VP8_FRAME, + }, +}; + +const struct visl_ctrls visl_vp8_ctrls = { + .ctrls = visl_vp8_ctrl_descs, + .num_ctrls = ARRAY_SIZE(visl_vp8_ctrl_descs), +}; + +static const struct visl_ctrl_desc visl_vp9_ctrl_descs[] = { + { + .cfg.id = V4L2_CID_STATELESS_VP9_FRAME, + }, + { + .cfg.id = V4L2_CID_STATELESS_VP9_COMPRESSED_HDR, + }, +}; + +const struct visl_ctrls visl_vp9_ctrls = { + .ctrls = visl_vp9_ctrl_descs, + .num_ctrls = ARRAY_SIZE(visl_vp9_ctrl_descs), +}; + +static const struct visl_ctrl_desc visl_h264_ctrl_descs[] = { + { + .cfg.id = V4L2_CID_STATELESS_H264_DECODE_PARAMS, + }, + { + .cfg.id = V4L2_CID_STATELESS_H264_SPS, + }, + { + .cfg.id = V4L2_CID_STATELESS_H264_PPS, + }, + { + .cfg.id = V4L2_CID_STATELESS_H264_SCALING_MATRIX, + }, + { + .cfg.id = V4L2_CID_STATELESS_H264_DECODE_MODE, + }, + { + .cfg.id = V4L2_CID_STATELESS_H264_START_CODE, + }, + { + .cfg.id = V4L2_CID_STATELESS_H264_SLICE_PARAMS, + }, + { + .cfg.id = V4L2_CID_STATELESS_H264_PRED_WEIGHTS, + }, +}; + +const struct visl_ctrls visl_h264_ctrls = { + .ctrls = visl_h264_ctrl_descs, + .num_ctrls = ARRAY_SIZE(visl_h264_ctrl_descs), +}; + +static const struct visl_ctrl_desc visl_hevc_ctrl_descs[] = { + { + .cfg.id = V4L2_CID_STATELESS_HEVC_SPS, + }, + { + .cfg.id = V4L2_CID_STATELESS_HEVC_PPS, + }, + { + .cfg.id = V4L2_CID_STATELESS_HEVC_SLICE_PARAMS, + /* The absolute maximum for level > 6 */ + .cfg.dims = { 600 }, + }, + { + .cfg.id = V4L2_CID_STATELESS_HEVC_SCALING_MATRIX, + }, + { + .cfg.id = V4L2_CID_STATELESS_HEVC_DECODE_PARAMS, + }, + { + .cfg.id = V4L2_CID_STATELESS_HEVC_DECODE_MODE, + }, + { + .cfg.id = V4L2_CID_STATELESS_HEVC_START_CODE, + }, + { + .cfg.id = V4L2_CID_STATELESS_HEVC_ENTRY_POINT_OFFSETS, + .cfg.dims = { 256 }, + .cfg.max = 0xffffffff, + .cfg.step = 1, + }, + +}; + +const struct visl_ctrls visl_hevc_ctrls = { + .ctrls = visl_hevc_ctrl_descs, + .num_ctrls = ARRAY_SIZE(visl_hevc_ctrl_descs), +}; + +struct v4l2_ctrl *visl_find_control(struct visl_ctx *ctx, u32 id) +{ + struct v4l2_ctrl_handler *hdl = &ctx->hdl; + + return v4l2_ctrl_find(hdl, id); +} + +void *visl_find_control_data(struct visl_ctx *ctx, u32 id) +{ + struct v4l2_ctrl *ctrl; + + ctrl = visl_find_control(ctx, id); + if (ctrl) + return ctrl->p_cur.p; + + return NULL; +} + +u32 visl_control_num_elems(struct visl_ctx *ctx, u32 id) +{ + struct v4l2_ctrl *ctrl; + + ctrl = visl_find_control(ctx, id); + if (ctrl) + return ctrl->elems; + + return 0; +} + +static void visl_device_release(struct video_device *vdev) +{ + struct visl_dev *dev = container_of(vdev, struct visl_dev, vfd); + + v4l2_device_unregister(&dev->v4l2_dev); + v4l2_m2m_release(dev->m2m_dev); + media_device_cleanup(&dev->mdev); + visl_debugfs_deinit(dev); + kfree(dev); +} + +#define VISL_CONTROLS_COUNT ARRAY_SIZE(visl_controls) + +static int visl_init_ctrls(struct visl_ctx *ctx) +{ + struct visl_dev *dev = ctx->dev; + struct v4l2_ctrl_handler *hdl = &ctx->hdl; + unsigned int ctrl_cnt = 0; + unsigned int i; + unsigned int j; + const struct visl_ctrls *ctrls; + + for (i = 0; i < num_coded_fmts; i++) + ctrl_cnt += visl_coded_fmts[i].ctrls->num_ctrls; + + v4l2_ctrl_handler_init(hdl, ctrl_cnt); + + for (i = 0; i < num_coded_fmts; i++) { + ctrls = visl_coded_fmts[i].ctrls; + for (j = 0; j < ctrls->num_ctrls; j++) + v4l2_ctrl_new_custom(hdl, &ctrls->ctrls[j].cfg, NULL); + } + + if (hdl->error) { + v4l2_err(&dev->v4l2_dev, + "Failed to initialize control handler\n"); + v4l2_ctrl_handler_free(hdl); + return hdl->error; + } + + ctx->fh.ctrl_handler = hdl; + v4l2_ctrl_handler_setup(hdl); + + return 0; +} + +static int visl_open(struct file *file) +{ + struct visl_dev *dev = video_drvdata(file); + struct visl_ctx *ctx = NULL; + int rc = 0; + + if (mutex_lock_interruptible(&dev->dev_mutex)) + return -ERESTARTSYS; + ctx = kzalloc(sizeof(*ctx), GFP_KERNEL); + if (!ctx) { + rc = -ENOMEM; + goto unlock; + } + + ctx->tpg_str_buf = kzalloc(TPG_STR_BUF_SZ, GFP_KERNEL); + + v4l2_fh_init(&ctx->fh, video_devdata(file)); + file->private_data = &ctx->fh; + ctx->dev = dev; + + rc = visl_init_ctrls(ctx); + if (rc) + goto free_ctx; + + ctx->fh.m2m_ctx = v4l2_m2m_ctx_init(dev->m2m_dev, ctx, &visl_queue_init); + + mutex_init(&ctx->vb_mutex); + + if (IS_ERR(ctx->fh.m2m_ctx)) { + rc = PTR_ERR(ctx->fh.m2m_ctx); + goto free_hdl; + } + + rc = visl_set_default_format(ctx); + if (rc) + goto free_m2m_ctx; + + v4l2_fh_add(&ctx->fh); + + dprintk(dev, "Created instance: %p, m2m_ctx: %p\n", + ctx, ctx->fh.m2m_ctx); + + mutex_unlock(&dev->dev_mutex); + return rc; + +free_m2m_ctx: + v4l2_m2m_ctx_release(ctx->fh.m2m_ctx); +free_hdl: + v4l2_ctrl_handler_free(&ctx->hdl); + v4l2_fh_exit(&ctx->fh); +free_ctx: + kfree(ctx->tpg_str_buf); + kfree(ctx); +unlock: + mutex_unlock(&dev->dev_mutex); + return rc; +} + +static int visl_release(struct file *file) +{ + struct visl_dev *dev = video_drvdata(file); + struct visl_ctx *ctx = visl_file_to_ctx(file); + + dprintk(dev, "Releasing instance %p\n", ctx); + + tpg_free(&ctx->tpg); + v4l2_fh_del(&ctx->fh); + v4l2_fh_exit(&ctx->fh); + v4l2_ctrl_handler_free(&ctx->hdl); + mutex_lock(&dev->dev_mutex); + v4l2_m2m_ctx_release(ctx->fh.m2m_ctx); + mutex_unlock(&dev->dev_mutex); + + kfree(ctx->tpg_str_buf); + kfree(ctx); + + return 0; +} + +static const struct v4l2_file_operations visl_fops = { + .owner = THIS_MODULE, + .open = visl_open, + .release = visl_release, + .poll = v4l2_m2m_fop_poll, + .unlocked_ioctl = video_ioctl2, + .mmap = v4l2_m2m_fop_mmap, +}; + +static const struct video_device visl_videodev = { + .name = VISL_NAME, + .vfl_dir = VFL_DIR_M2M, + .fops = &visl_fops, + .ioctl_ops = &visl_ioctl_ops, + .minor = -1, + .release = visl_device_release, + .device_caps = V4L2_CAP_VIDEO_M2M_MPLANE | V4L2_CAP_STREAMING, +}; + +static const struct v4l2_m2m_ops visl_m2m_ops = { + .device_run = visl_device_run, +}; + +static const struct media_device_ops visl_m2m_media_ops = { + .req_validate = visl_request_validate, + .req_queue = v4l2_m2m_request_queue, +}; + +static int visl_probe(struct platform_device *pdev) +{ + struct visl_dev *dev; + struct video_device *vfd; + int ret; + int rc; + + dev = kzalloc(sizeof(*dev), GFP_KERNEL); + if (!dev) + return -ENOMEM; + + ret = v4l2_device_register(&pdev->dev, &dev->v4l2_dev); + if (ret) + goto error_visl_dev; + + mutex_init(&dev->dev_mutex); + + dev->vfd = visl_videodev; + vfd = &dev->vfd; + vfd->lock = &dev->dev_mutex; + vfd->v4l2_dev = &dev->v4l2_dev; + + video_set_drvdata(vfd, dev); + + platform_set_drvdata(pdev, dev); + + dev->m2m_dev = v4l2_m2m_init(&visl_m2m_ops); + if (IS_ERR(dev->m2m_dev)) { + v4l2_err(&dev->v4l2_dev, "Failed to init mem2mem device\n"); + ret = PTR_ERR(dev->m2m_dev); + dev->m2m_dev = NULL; + goto error_dev; + } + + dev->mdev.dev = &pdev->dev; + strscpy(dev->mdev.model, "visl", sizeof(dev->mdev.model)); + strscpy(dev->mdev.bus_info, "platform:visl", + sizeof(dev->mdev.bus_info)); + media_device_init(&dev->mdev); + dev->mdev.ops = &visl_m2m_media_ops; + dev->v4l2_dev.mdev = &dev->mdev; + + ret = video_register_device(vfd, VFL_TYPE_VIDEO, -1); + if (ret) { + v4l2_err(&dev->v4l2_dev, "Failed to register video device\n"); + goto error_m2m; + } + + v4l2_info(&dev->v4l2_dev, + "Device registered as /dev/video%d\n", vfd->num); + + ret = v4l2_m2m_register_media_controller(dev->m2m_dev, vfd, + MEDIA_ENT_F_PROC_VIDEO_DECODER); + if (ret) { + v4l2_err(&dev->v4l2_dev, "Failed to init mem2mem media controller\n"); + goto error_v4l2; + } + + ret = media_device_register(&dev->mdev); + if (ret) { + v4l2_err(&dev->v4l2_dev, "Failed to register mem2mem media device\n"); + goto error_m2m_mc; + } + + rc = visl_debugfs_init(dev); + if (rc) + dprintk(dev, "visl_debugfs_init failed: %d\n" + "Continuing without debugfs support\n", rc); + + return 0; + +error_m2m_mc: + v4l2_m2m_unregister_media_controller(dev->m2m_dev); +error_v4l2: + video_unregister_device(&dev->vfd); + /* visl_device_release called by video_unregister_device to release various objects */ + return ret; +error_m2m: + v4l2_m2m_release(dev->m2m_dev); +error_dev: + v4l2_device_unregister(&dev->v4l2_dev); +error_visl_dev: + kfree(dev); + + return ret; +} + +static int visl_remove(struct platform_device *pdev) +{ + struct visl_dev *dev = platform_get_drvdata(pdev); + + v4l2_info(&dev->v4l2_dev, "Removing " VISL_NAME); + +#ifdef CONFIG_MEDIA_CONTROLLER + if (media_devnode_is_registered(dev->mdev.devnode)) { + media_device_unregister(&dev->mdev); + v4l2_m2m_unregister_media_controller(dev->m2m_dev); + } +#endif + video_unregister_device(&dev->vfd); + + return 0; +} + +static struct platform_driver visl_pdrv = { + .probe = visl_probe, + .remove = visl_remove, + .driver = { + .name = VISL_NAME, + }, +}; + +static void visl_dev_release(struct device *dev) {} + +static struct platform_device visl_pdev = { + .name = VISL_NAME, + .dev.release = visl_dev_release, +}; + +static void __exit visl_exit(void) +{ + platform_driver_unregister(&visl_pdrv); + platform_device_unregister(&visl_pdev); +} + +static int __init visl_init(void) +{ + int ret; + + ret = platform_device_register(&visl_pdev); + if (ret) + return ret; + + ret = platform_driver_register(&visl_pdrv); + if (ret) + platform_device_unregister(&visl_pdev); + + return ret; +} + +MODULE_DESCRIPTION("Virtual stateless decoder device"); +MODULE_AUTHOR("Daniel Almeida <[email protected]>"); +MODULE_LICENSE("GPL"); + +module_init(visl_init); +module_exit(visl_exit); diff --git a/drivers/media/test-drivers/visl/visl-debugfs.c b/drivers/media/test-drivers/visl/visl-debugfs.c new file mode 100644 index 000000000000..45f2a8268014 --- /dev/null +++ b/drivers/media/test-drivers/visl/visl-debugfs.c @@ -0,0 +1,112 @@ +// SPDX-License-Identifier: GPL-2.0+ +/* + * Debugfs tracing for bitstream buffers. This is similar to VA-API's + * LIBVA_TRACE_BUFDATA in that the raw bitstream can be dumped as a debugging + * aid. + * + * Produces one file per OUTPUT buffer. Files are automatically cleared on + * STREAMOFF unless the module parameter "keep_bitstream_buffers" is set. + */ + +#include <linux/debugfs.h> +#include <linux/list.h> +#include <linux/mutex.h> +#include <media/v4l2-mem2mem.h> + +#include "visl-debugfs.h" + +int visl_debugfs_init(struct visl_dev *dev) +{ + dev->debugfs_root = debugfs_create_dir("visl", NULL); + INIT_LIST_HEAD(&dev->bitstream_blobs); + mutex_init(&dev->bitstream_lock); + + if (IS_ERR(dev->debugfs_root)) + return PTR_ERR(dev->debugfs_root); + + return visl_debugfs_bitstream_init(dev); +} + +int visl_debugfs_bitstream_init(struct visl_dev *dev) +{ + dev->bitstream_debugfs = debugfs_create_dir("bitstream", + dev->debugfs_root); + if (IS_ERR(dev->bitstream_debugfs)) + return PTR_ERR(dev->bitstream_debugfs); + + return 0; +} + +void visl_trace_bitstream(struct visl_ctx *ctx, struct visl_run *run) +{ + u8 *vaddr = vb2_plane_vaddr(&run->src->vb2_buf, 0); + struct visl_blob *blob; + size_t data_sz = vb2_get_plane_payload(&run->src->vb2_buf, 0); + struct dentry *dentry; + char name[32]; + + blob = kzalloc(sizeof(*blob), GFP_KERNEL); + if (!blob) + return; + + blob->blob.data = vzalloc(data_sz); + if (!blob->blob.data) + goto err_vmalloc; + + blob->blob.size = data_sz; + snprintf(name, 32, "bitstream%d", run->src->sequence); + + memcpy(blob->blob.data, vaddr, data_sz); + + dentry = debugfs_create_blob(name, 0444, ctx->dev->bitstream_debugfs, + &blob->blob); + if (IS_ERR(dentry)) + goto err_debugfs; + + blob->dentry = dentry; + + mutex_lock(&ctx->dev->bitstream_lock); + list_add_tail(&blob->list, &ctx->dev->bitstream_blobs); + mutex_unlock(&ctx->dev->bitstream_lock); + + return; + +err_debugfs: + vfree(blob->blob.data); +err_vmalloc: + kfree(blob); +} + +void visl_debugfs_clear_bitstream(struct visl_dev *dev) +{ + struct visl_blob *blob; + struct visl_blob *tmp; + + mutex_lock(&dev->bitstream_lock); + if (list_empty(&dev->bitstream_blobs)) + goto unlock; + + list_for_each_entry_safe(blob, tmp, &dev->bitstream_blobs, list) { + list_del(&blob->list); + debugfs_remove(blob->dentry); + vfree(blob->blob.data); + kfree(blob); + } + +unlock: + mutex_unlock(&dev->bitstream_lock); +} + +void visl_debugfs_bitstream_deinit(struct visl_dev *dev) +{ + visl_debugfs_clear_bitstream(dev); + debugfs_remove_recursive(dev->bitstream_debugfs); + dev->bitstream_debugfs = NULL; +} + +void visl_debugfs_deinit(struct visl_dev *dev) +{ + visl_debugfs_bitstream_deinit(dev); + debugfs_remove_recursive(dev->debugfs_root); + dev->debugfs_root = NULL; +} diff --git a/drivers/media/test-drivers/visl/visl-debugfs.h b/drivers/media/test-drivers/visl/visl-debugfs.h new file mode 100644 index 000000000000..81508f611918 --- /dev/null +++ b/drivers/media/test-drivers/visl/visl-debugfs.h @@ -0,0 +1,40 @@ +/* SPDX-License-Identifier: GPL-2.0+ */ +/* + * Debugfs tracing for bitstream buffers. This is similar to VA-API's + * LIBVA_TRACE_BUFDATA in that the raw bitstream can be dumped as a debugging + * aid. + * + * Produces one file per OUTPUT buffer. Files are automatically cleared on + * STREAMOFF unless the module parameter "keep_bitstream_buffers" is set. + */ + +#include "visl.h" +#include "visl-dec.h" + +#ifdef CONFIG_VISL_DEBUGFS + +int visl_debugfs_init(struct visl_dev *dev); +int visl_debugfs_bitstream_init(struct visl_dev *dev); +void visl_trace_bitstream(struct visl_ctx *ctx, struct visl_run *run); +void visl_debugfs_clear_bitstream(struct visl_dev *dev); +void visl_debugfs_bitstream_deinit(struct visl_dev *dev); +void visl_debugfs_deinit(struct visl_dev *dev); + +#else + +static inline int visl_debugfs_init(struct visl_dev *dev) +{ + return 0; +} + +static inline int visl_debugfs_bitstream_init(struct visl_dev *dev) +{ + return 0; +} + +static inline void visl_trace_bitstream(struct visl_ctx *ctx, struct visl_run *run) {} +static inline void visl_debugfs_clear_bitstream(struct visl_dev *dev) {} +static inline void visl_debugfs_bitstream_deinit(struct visl_dev *dev) {} +static inline void visl_debugfs_deinit(struct visl_dev *dev) {} + +#endif diff --git a/drivers/media/test-drivers/visl/visl-dec.c b/drivers/media/test-drivers/visl/visl-dec.c new file mode 100644 index 000000000000..318d675e5668 --- /dev/null +++ b/drivers/media/test-drivers/visl/visl-dec.c @@ -0,0 +1,499 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Contains the virtual decoder logic. The functions here control the + * tracing/TPG on a per-frame basis + */ + +#include "visl.h" +#include "visl-debugfs.h" +#include "visl-dec.h" +#include "visl-trace-fwht.h" +#include "visl-trace-mpeg2.h" +#include "visl-trace-vp8.h" +#include "visl-trace-vp9.h" +#include "visl-trace-h264.h" +#include "visl-trace-hevc.h" + +#include <linux/delay.h> +#include <linux/workqueue.h> +#include <media/v4l2-mem2mem.h> +#include <media/tpg/v4l2-tpg.h> + +static void *plane_vaddr(struct tpg_data *tpg, struct vb2_buffer *buf, + u32 p, u32 bpl[TPG_MAX_PLANES], u32 h) +{ + u32 i; + void *vbuf; + + if (p == 0 || tpg_g_buffers(tpg) > 1) + return vb2_plane_vaddr(buf, p); + vbuf = vb2_plane_vaddr(buf, 0); + for (i = 0; i < p; i++) + vbuf += bpl[i] * h / tpg->vdownsampling[i]; + return vbuf; +} + +static void visl_get_ref_frames(struct visl_ctx *ctx, u8 *buf, + __kernel_size_t buflen, struct visl_run *run) +{ + struct vb2_queue *cap_q = &ctx->fh.m2m_ctx->cap_q_ctx.q; + char header[] = "Reference frames:\n"; + u32 i; + u32 len; + + len = scnprintf(buf, buflen, header); + buf += len; + buflen -= len; + + switch (ctx->current_codec) { + case VISL_CODEC_NONE: + break; + + case VISL_CODEC_FWHT: { + struct vb2_buffer *vb2_buf; + + vb2_buf = vb2_find_buffer(cap_q, run->fwht.params->backward_ref_ts); + + scnprintf(buf, buflen, "backwards_ref_ts: %lld, vb2_idx: %d", + run->fwht.params->backward_ref_ts, + vb2_buf ? vb2_buf->index : -1); + break; + } + + case VISL_CODEC_MPEG2: { + struct vb2_buffer *b_ref; + struct vb2_buffer *f_ref; + + b_ref = vb2_find_buffer(cap_q, run->mpeg2.pic->backward_ref_ts); + f_ref = vb2_find_buffer(cap_q, run->mpeg2.pic->forward_ref_ts); + + scnprintf(buf, buflen, + "backward_ref_ts: %llu, vb2_idx: %d\n" + "forward_ref_ts: %llu, vb2_idx: %d\n", + run->mpeg2.pic->backward_ref_ts, + b_ref ? b_ref->index : -1, + run->mpeg2.pic->forward_ref_ts, + f_ref ? f_ref->index : -1); + break; + } + + case VISL_CODEC_VP8: { + struct vb2_buffer *last; + struct vb2_buffer *golden; + struct vb2_buffer *alt; + + last = vb2_find_buffer(cap_q, run->vp8.frame->last_frame_ts); + golden = vb2_find_buffer(cap_q, run->vp8.frame->golden_frame_ts); + alt = vb2_find_buffer(cap_q, run->vp8.frame->alt_frame_ts); + + scnprintf(buf, buflen, + "last_ref_ts: %llu, vb2_idx: %d\n" + "golden_ref_ts: %llu, vb2_idx: %d\n" + "alt_ref_ts: %llu, vb2_idx: %d\n", + run->vp8.frame->last_frame_ts, + last ? last->index : -1, + run->vp8.frame->golden_frame_ts, + golden ? golden->index : -1, + run->vp8.frame->alt_frame_ts, + alt ? alt->index : -1); + break; + } + + case VISL_CODEC_VP9: { + struct vb2_buffer *last; + struct vb2_buffer *golden; + struct vb2_buffer *alt; + + last = vb2_find_buffer(cap_q, run->vp9.frame->last_frame_ts); + golden = vb2_find_buffer(cap_q, run->vp9.frame->golden_frame_ts); + alt = vb2_find_buffer(cap_q, run->vp9.frame->alt_frame_ts); + + scnprintf(buf, buflen, + "last_ref_ts: %llu, vb2_idx: %d\n" + "golden_ref_ts: %llu, vb2_idx: %d\n" + "alt_ref_ts: %llu, vb2_idx: %d\n", + run->vp9.frame->last_frame_ts, + last ? last->index : -1, + run->vp9.frame->golden_frame_ts, + golden ? golden->index : -1, + run->vp9.frame->alt_frame_ts, + alt ? alt->index : -1); + break; + } + + case VISL_CODEC_H264: { + char entry[] = "dpb[%d]:%u, vb2_index: %d\n"; + struct vb2_buffer *vb2_buf; + + for (i = 0; i < ARRAY_SIZE(run->h264.dpram->dpb); i++) { + vb2_buf = vb2_find_buffer(cap_q, run->h264.dpram->dpb[i].reference_ts); + len = scnprintf(buf, buflen, entry, i, + run->h264.dpram->dpb[i].reference_ts, + vb2_buf ? vb2_buf->index : -1); + buf += len; + buflen -= len; + } + + break; + } + + case VISL_CODEC_HEVC: { + char entry[] = "dpb[%d]:%u, vb2_index: %d\n"; + struct vb2_buffer *vb2_buf; + + for (i = 0; i < ARRAY_SIZE(run->hevc.dpram->dpb); i++) { + vb2_buf = vb2_find_buffer(cap_q, run->hevc.dpram->dpb[i].timestamp); + len = scnprintf(buf, buflen, entry, i, + run->hevc.dpram->dpb[i].timestamp, + vb2_buf ? vb2_buf->index : -1); + buf += len; + buflen -= len; + } + + break; + } + } +} + +static char *visl_get_vb2_state(enum vb2_buffer_state state) +{ + switch (state) { + case VB2_BUF_STATE_DEQUEUED: + return "Dequeued"; + case VB2_BUF_STATE_IN_REQUEST: + return "In request"; + case VB2_BUF_STATE_PREPARING: + return "Preparing"; + case VB2_BUF_STATE_QUEUED: + return "Queued"; + case VB2_BUF_STATE_ACTIVE: + return "Active"; + case VB2_BUF_STATE_DONE: + return "Done"; + case VB2_BUF_STATE_ERROR: + return "Error"; + default: + return ""; + } +} + +static int visl_fill_bytesused(struct vb2_v4l2_buffer *v4l2_vb2_buf, char *buf, size_t bufsz) +{ + int len = 0; + u32 i; + + for (i = 0; i < v4l2_vb2_buf->vb2_buf.num_planes; i++) + len += scnprintf(buf, bufsz, + "bytesused[%u]: %u length[%u]: %u data_offset[%u]: %u", + i, v4l2_vb2_buf->planes[i].bytesused, + i, v4l2_vb2_buf->planes[i].length, + i, v4l2_vb2_buf->planes[i].data_offset); + + return len; +} + +static void visl_tpg_fill_sequence(struct visl_ctx *ctx, + struct visl_run *run, char buf[], size_t bufsz) +{ + u32 stream_ms; + + stream_ms = jiffies_to_msecs(get_jiffies_64() - ctx->capture_streamon_jiffies); + + scnprintf(buf, bufsz, + "stream time: %02d:%02d:%02d:%03d sequence:%u timestamp:%lld field:%s", + (stream_ms / (60 * 60 * 1000)) % 24, + (stream_ms / (60 * 1000)) % 60, + (stream_ms / 1000) % 60, + stream_ms % 1000, + run->dst->sequence, + run->dst->vb2_buf.timestamp, + (run->dst->field == V4L2_FIELD_ALTERNATE) ? + (run->dst->field == V4L2_FIELD_TOP ? + " top" : " bottom") : "none"); +} + +static void visl_tpg_fill(struct visl_ctx *ctx, struct visl_run *run) +{ + u8 *basep[TPG_MAX_PLANES][2]; + char *buf = ctx->tpg_str_buf; + char *tmp = buf; + char *line_str; + u32 line = 1; + const u32 line_height = 16; + u32 len; + struct vb2_queue *out_q = &ctx->fh.m2m_ctx->out_q_ctx.q; + struct vb2_queue *cap_q = &ctx->fh.m2m_ctx->cap_q_ctx.q; + struct v4l2_pix_format_mplane *coded_fmt = &ctx->coded_fmt.fmt.pix_mp; + struct v4l2_pix_format_mplane *decoded_fmt = &ctx->decoded_fmt.fmt.pix_mp; + u32 p; + u32 i; + + for (p = 0; p < tpg_g_planes(&ctx->tpg); p++) { + void *vbuf = plane_vaddr(&ctx->tpg, + &run->dst->vb2_buf, p, + ctx->tpg.bytesperline, + ctx->tpg.buf_height); + + tpg_calc_text_basep(&ctx->tpg, basep, p, vbuf); + tpg_fill_plane_buffer(&ctx->tpg, 0, p, vbuf); + } + + visl_tpg_fill_sequence(ctx, run, buf, TPG_STR_BUF_SZ); + tpg_gen_text(&ctx->tpg, basep, line++ * line_height, 16, buf); + frame_dprintk(ctx->dev, run->dst->sequence, "%s\n", buf); + frame_dprintk(ctx->dev, run->dst->sequence, ""); + line++; + + visl_get_ref_frames(ctx, buf, TPG_STR_BUF_SZ, run); + + while ((line_str = strsep(&tmp, "\n")) && strlen(line_str)) { + tpg_gen_text(&ctx->tpg, basep, line++ * line_height, 16, line_str); + frame_dprintk(ctx->dev, run->dst->sequence, "%s\n", line_str); + } + + frame_dprintk(ctx->dev, run->dst->sequence, ""); + line++; + + scnprintf(buf, + TPG_STR_BUF_SZ, + "OUTPUT pixelformat: %c%c%c%c, resolution: %dx%d, num_planes: %d", + coded_fmt->pixelformat, + (coded_fmt->pixelformat >> 8) & 0xff, + (coded_fmt->pixelformat >> 16) & 0xff, + (coded_fmt->pixelformat >> 24) & 0xff, + coded_fmt->width, + coded_fmt->height, + coded_fmt->num_planes); + + tpg_gen_text(&ctx->tpg, basep, line++ * line_height, 16, buf); + frame_dprintk(ctx->dev, run->dst->sequence, "%s\n", buf); + + for (i = 0; i < coded_fmt->num_planes; i++) { + scnprintf(buf, + TPG_STR_BUF_SZ, + "plane[%d]: bytesperline: %d, sizeimage: %d", + i, + coded_fmt->plane_fmt[i].bytesperline, + coded_fmt->plane_fmt[i].sizeimage); + + tpg_gen_text(&ctx->tpg, basep, line++ * line_height, 16, buf); + frame_dprintk(ctx->dev, run->dst->sequence, "%s\n", buf); + } + + line++; + frame_dprintk(ctx->dev, run->dst->sequence, ""); + scnprintf(buf, TPG_STR_BUF_SZ, "Output queue status:"); + tpg_gen_text(&ctx->tpg, basep, line++ * line_height, 16, buf); + frame_dprintk(ctx->dev, run->dst->sequence, "%s\n", buf); + + len = 0; + for (i = 0; i < out_q->num_buffers; i++) { + char entry[] = "index: %u, state: %s, request_fd: %d, "; + u32 old_len = len; + char *q_status = visl_get_vb2_state(out_q->bufs[i]->state); + + len += scnprintf(&buf[len], TPG_STR_BUF_SZ - len, + entry, i, q_status, + to_vb2_v4l2_buffer(out_q->bufs[i])->request_fd); + + len += visl_fill_bytesused(to_vb2_v4l2_buffer(out_q->bufs[i]), + &buf[len], + TPG_STR_BUF_SZ - len); + + tpg_gen_text(&ctx->tpg, basep, line++ * line_height, 16, &buf[old_len]); + frame_dprintk(ctx->dev, run->dst->sequence, "%s", &buf[old_len]); + } + + line++; + frame_dprintk(ctx->dev, run->dst->sequence, ""); + + scnprintf(buf, + TPG_STR_BUF_SZ, + "CAPTURE pixelformat: %c%c%c%c, resolution: %dx%d, num_planes: %d", + decoded_fmt->pixelformat, + (decoded_fmt->pixelformat >> 8) & 0xff, + (decoded_fmt->pixelformat >> 16) & 0xff, + (decoded_fmt->pixelformat >> 24) & 0xff, + decoded_fmt->width, + decoded_fmt->height, + decoded_fmt->num_planes); + + tpg_gen_text(&ctx->tpg, basep, line++ * line_height, 16, buf); + frame_dprintk(ctx->dev, run->dst->sequence, "%s\n", buf); + + for (i = 0; i < decoded_fmt->num_planes; i++) { + scnprintf(buf, + TPG_STR_BUF_SZ, + "plane[%d]: bytesperline: %d, sizeimage: %d", + i, + decoded_fmt->plane_fmt[i].bytesperline, + decoded_fmt->plane_fmt[i].sizeimage); + + tpg_gen_text(&ctx->tpg, basep, line++ * line_height, 16, buf); + frame_dprintk(ctx->dev, run->dst->sequence, "%s\n", buf); + } + + line++; + frame_dprintk(ctx->dev, run->dst->sequence, ""); + scnprintf(buf, TPG_STR_BUF_SZ, "Capture queue status:"); + tpg_gen_text(&ctx->tpg, basep, line++ * line_height, 16, buf); + frame_dprintk(ctx->dev, run->dst->sequence, "%s\n", buf); + + len = 0; + for (i = 0; i < cap_q->num_buffers; i++) { + u32 old_len = len; + char *q_status = visl_get_vb2_state(cap_q->bufs[i]->state); + + len += scnprintf(&buf[len], TPG_STR_BUF_SZ - len, + "index: %u, status: %s, timestamp: %llu, is_held: %d", + cap_q->bufs[i]->index, q_status, + cap_q->bufs[i]->timestamp, + to_vb2_v4l2_buffer(cap_q->bufs[i])->is_held); + + tpg_gen_text(&ctx->tpg, basep, line++ * line_height, 16, &buf[old_len]); + frame_dprintk(ctx->dev, run->dst->sequence, "%s", &buf[old_len]); + } +} + +static void visl_trace_ctrls(struct visl_ctx *ctx, struct visl_run *run) +{ + int i; + + switch (ctx->current_codec) { + default: + case VISL_CODEC_NONE: + break; + case VISL_CODEC_FWHT: + trace_v4l2_ctrl_fwht_params(run->fwht.params); + break; + case VISL_CODEC_MPEG2: + trace_v4l2_ctrl_mpeg2_sequence(run->mpeg2.seq); + trace_v4l2_ctrl_mpeg2_picture(run->mpeg2.pic); + trace_v4l2_ctrl_mpeg2_quantisation(run->mpeg2.quant); + break; + case VISL_CODEC_VP8: + trace_v4l2_ctrl_vp8_frame(run->vp8.frame); + trace_v4l2_ctrl_vp8_entropy(run->vp8.frame); + break; + case VISL_CODEC_VP9: + trace_v4l2_ctrl_vp9_frame(run->vp9.frame); + trace_v4l2_ctrl_vp9_compressed_hdr(run->vp9.probs); + trace_v4l2_ctrl_vp9_compressed_coeff(run->vp9.probs); + trace_v4l2_vp9_mv_probs(&run->vp9.probs->mv); + break; + case VISL_CODEC_H264: + trace_v4l2_ctrl_h264_sps(run->h264.sps); + trace_v4l2_ctrl_h264_pps(run->h264.pps); + trace_v4l2_ctrl_h264_scaling_matrix(run->h264.sm); + trace_v4l2_ctrl_h264_slice_params(run->h264.spram); + + for (i = 0; i < ARRAY_SIZE(run->h264.spram->ref_pic_list0); i++) + trace_v4l2_h264_ref_pic_list0(&run->h264.spram->ref_pic_list0[i], i); + for (i = 0; i < ARRAY_SIZE(run->h264.spram->ref_pic_list0); i++) + trace_v4l2_h264_ref_pic_list1(&run->h264.spram->ref_pic_list1[i], i); + + trace_v4l2_ctrl_h264_decode_params(run->h264.dpram); + + for (i = 0; i < ARRAY_SIZE(run->h264.dpram->dpb); i++) + trace_v4l2_h264_dpb_entry(&run->h264.dpram->dpb[i], i); + + trace_v4l2_ctrl_h264_pred_weights(run->h264.pwht); + break; + case VISL_CODEC_HEVC: + trace_v4l2_ctrl_hevc_sps(run->hevc.sps); + trace_v4l2_ctrl_hevc_pps(run->hevc.pps); + trace_v4l2_ctrl_hevc_slice_params(run->hevc.spram); + trace_v4l2_ctrl_hevc_scaling_matrix(run->hevc.sm); + trace_v4l2_ctrl_hevc_decode_params(run->hevc.dpram); + + for (i = 0; i < ARRAY_SIZE(run->hevc.dpram->dpb); i++) + trace_v4l2_hevc_dpb_entry(&run->hevc.dpram->dpb[i]); + + trace_v4l2_hevc_pred_weight_table(&run->hevc.spram->pred_weight_table); + break; + } +} + +void visl_device_run(void *priv) +{ + struct visl_ctx *ctx = priv; + struct visl_run run = {}; + struct media_request *src_req; + + run.src = v4l2_m2m_next_src_buf(ctx->fh.m2m_ctx); + run.dst = v4l2_m2m_next_dst_buf(ctx->fh.m2m_ctx); + + /* Apply request(s) controls if needed. */ + src_req = run.src->vb2_buf.req_obj.req; + + if (src_req) + v4l2_ctrl_request_setup(src_req, &ctx->hdl); + + v4l2_m2m_buf_copy_metadata(run.src, run.dst, true); + run.dst->sequence = ctx->q_data[V4L2_M2M_DST].sequence++; + run.src->sequence = ctx->q_data[V4L2_M2M_SRC].sequence++; + run.dst->field = ctx->decoded_fmt.fmt.pix.field; + + switch (ctx->current_codec) { + default: + case VISL_CODEC_NONE: + break; + case VISL_CODEC_FWHT: + run.fwht.params = visl_find_control_data(ctx, V4L2_CID_STATELESS_FWHT_PARAMS); + break; + case VISL_CODEC_MPEG2: + run.mpeg2.seq = visl_find_control_data(ctx, V4L2_CID_STATELESS_MPEG2_SEQUENCE); + run.mpeg2.pic = visl_find_control_data(ctx, V4L2_CID_STATELESS_MPEG2_PICTURE); + run.mpeg2.quant = visl_find_control_data(ctx, + V4L2_CID_STATELESS_MPEG2_QUANTISATION); + break; + case VISL_CODEC_VP8: + run.vp8.frame = visl_find_control_data(ctx, V4L2_CID_STATELESS_VP8_FRAME); + break; + case VISL_CODEC_VP9: + run.vp9.frame = visl_find_control_data(ctx, V4L2_CID_STATELESS_VP9_FRAME); + run.vp9.probs = visl_find_control_data(ctx, V4L2_CID_STATELESS_VP9_COMPRESSED_HDR); + break; + case VISL_CODEC_H264: + run.h264.sps = visl_find_control_data(ctx, V4L2_CID_STATELESS_H264_SPS); + run.h264.pps = visl_find_control_data(ctx, V4L2_CID_STATELESS_H264_PPS); + run.h264.sm = visl_find_control_data(ctx, V4L2_CID_STATELESS_H264_SCALING_MATRIX); + run.h264.spram = visl_find_control_data(ctx, V4L2_CID_STATELESS_H264_SLICE_PARAMS); + run.h264.dpram = visl_find_control_data(ctx, V4L2_CID_STATELESS_H264_DECODE_PARAMS); + run.h264.pwht = visl_find_control_data(ctx, V4L2_CID_STATELESS_H264_PRED_WEIGHTS); + break; + case VISL_CODEC_HEVC: + run.hevc.sps = visl_find_control_data(ctx, V4L2_CID_STATELESS_HEVC_SPS); + run.hevc.pps = visl_find_control_data(ctx, V4L2_CID_STATELESS_HEVC_PPS); + run.hevc.spram = visl_find_control_data(ctx, V4L2_CID_STATELESS_HEVC_SLICE_PARAMS); + run.hevc.sm = visl_find_control_data(ctx, V4L2_CID_STATELESS_HEVC_SCALING_MATRIX); + run.hevc.dpram = visl_find_control_data(ctx, V4L2_CID_STATELESS_HEVC_DECODE_PARAMS); + break; + } + + frame_dprintk(ctx->dev, run.dst->sequence, + "Got OUTPUT buffer sequence %d, timestamp %llu\n", + run.src->sequence, run.src->vb2_buf.timestamp); + + frame_dprintk(ctx->dev, run.dst->sequence, + "Got CAPTURE buffer sequence %d, timestamp %llu\n", + run.dst->sequence, run.dst->vb2_buf.timestamp); + + visl_tpg_fill(ctx, &run); + visl_trace_ctrls(ctx, &run); + + if (bitstream_trace_frame_start > -1 && + run.dst->sequence >= bitstream_trace_frame_start && + run.dst->sequence < bitstream_trace_frame_start + bitstream_trace_nframes) + visl_trace_bitstream(ctx, &run); + + /* Complete request(s) controls if needed. */ + if (src_req) + v4l2_ctrl_request_complete(src_req, &ctx->hdl); + + if (visl_transtime_ms) + usleep_range(visl_transtime_ms * 1000, 2 * visl_transtime_ms * 1000); + + v4l2_m2m_buf_done_and_job_finish(ctx->dev->m2m_dev, + ctx->fh.m2m_ctx, VB2_BUF_STATE_DONE); +} diff --git a/drivers/media/test-drivers/visl/visl-dec.h b/drivers/media/test-drivers/visl/visl-dec.h new file mode 100644 index 000000000000..4a706a9de02e --- /dev/null +++ b/drivers/media/test-drivers/visl/visl-dec.h @@ -0,0 +1,67 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* + * Contains the virtual decoder logic. The functions here control the + * tracing/TPG on a per-frame basis + */ + +#ifndef _VISL_DEC_H_ +#define _VISL_DEC_H_ + +#include "visl.h" + +struct visl_fwht_run { + const struct v4l2_ctrl_fwht_params *params; +}; + +struct visl_mpeg2_run { + const struct v4l2_ctrl_mpeg2_sequence *seq; + const struct v4l2_ctrl_mpeg2_picture *pic; + const struct v4l2_ctrl_mpeg2_quantisation *quant; +}; + +struct visl_vp8_run { + const struct v4l2_ctrl_vp8_frame *frame; +}; + +struct visl_vp9_run { + const struct v4l2_ctrl_vp9_frame *frame; + const struct v4l2_ctrl_vp9_compressed_hdr *probs; +}; + +struct visl_h264_run { + const struct v4l2_ctrl_h264_sps *sps; + const struct v4l2_ctrl_h264_pps *pps; + const struct v4l2_ctrl_h264_scaling_matrix *sm; + const struct v4l2_ctrl_h264_slice_params *spram; + const struct v4l2_ctrl_h264_decode_params *dpram; + const struct v4l2_ctrl_h264_pred_weights *pwht; +}; + +struct visl_hevc_run { + const struct v4l2_ctrl_hevc_sps *sps; + const struct v4l2_ctrl_hevc_pps *pps; + const struct v4l2_ctrl_hevc_slice_params *spram; + const struct v4l2_ctrl_hevc_scaling_matrix *sm; + const struct v4l2_ctrl_hevc_decode_params *dpram; +}; + +struct visl_run { + struct vb2_v4l2_buffer *src; + struct vb2_v4l2_buffer *dst; + + union { + struct visl_fwht_run fwht; + struct visl_mpeg2_run mpeg2; + struct visl_vp8_run vp8; + struct visl_vp9_run vp9; + struct visl_h264_run h264; + struct visl_hevc_run hevc; + }; +}; + +int visl_dec_start(struct visl_ctx *ctx); +int visl_dec_stop(struct visl_ctx *ctx); +int visl_job_ready(void *priv); +void visl_device_run(void *priv); + +#endif /* _VISL_DEC_H_ */ diff --git a/drivers/media/test-drivers/visl/visl-trace-fwht.h b/drivers/media/test-drivers/visl/visl-trace-fwht.h new file mode 100644 index 000000000000..54b119359ff5 --- /dev/null +++ b/drivers/media/test-drivers/visl/visl-trace-fwht.h @@ -0,0 +1,66 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#if !defined(_VISL_TRACE_FWHT_H_) || defined(TRACE_HEADER_MULTI_READ) +#define _VISL_TRACE_FWHT_H_ + +#include <linux/tracepoint.h> +#include "visl.h" + +#undef TRACE_SYSTEM +#define TRACE_SYSTEM visl_fwht_controls + +DECLARE_EVENT_CLASS(v4l2_ctrl_fwht_params_tmpl, + TP_PROTO(const struct v4l2_ctrl_fwht_params *p), + TP_ARGS(p), + TP_STRUCT__entry( + __field(u64, backward_ref_ts) + __field(u32, version) + __field(u32, width) + __field(u32, height) + __field(u32, flags) + __field(u32, colorspace) + __field(u32, xfer_func) + __field(u32, ycbcr_enc) + __field(u32, quantization) + ), + TP_fast_assign( + __entry->backward_ref_ts = p->backward_ref_ts; + __entry->version = p->version; + __entry->width = p->width; + __entry->height = p->height; + __entry->flags = p->flags; + __entry->colorspace = p->colorspace; + __entry->xfer_func = p->xfer_func; + __entry->ycbcr_enc = p->ycbcr_enc; + __entry->quantization = p->quantization; + ), + TP_printk("backward_ref_ts %llu version %u width %u height %u flags %s colorspace %u xfer_func %u ycbcr_enc %u quantization %u", + __entry->backward_ref_ts, __entry->version, __entry->width, __entry->height, + __print_flags(__entry->flags, "|", + {V4L2_FWHT_FL_IS_INTERLACED, "IS_INTERLACED"}, + {V4L2_FWHT_FL_IS_BOTTOM_FIRST, "IS_BOTTOM_FIRST"}, + {V4L2_FWHT_FL_IS_ALTERNATE, "IS_ALTERNATE"}, + {V4L2_FWHT_FL_IS_BOTTOM_FIELD, "IS_BOTTOM_FIELD"}, + {V4L2_FWHT_FL_LUMA_IS_UNCOMPRESSED, "LUMA_IS_UNCOMPRESSED"}, + {V4L2_FWHT_FL_CB_IS_UNCOMPRESSED, "CB_IS_UNCOMPRESSED"}, + {V4L2_FWHT_FL_CR_IS_UNCOMPRESSED, "CR_IS_UNCOMPRESSED"}, + {V4L2_FWHT_FL_ALPHA_IS_UNCOMPRESSED, "ALPHA_IS_UNCOMPRESSED"}, + {V4L2_FWHT_FL_I_FRAME, "I_FRAME"}, + {V4L2_FWHT_FL_PIXENC_HSV, "PIXENC_HSV"}, + {V4L2_FWHT_FL_PIXENC_RGB, "PIXENC_RGB"}, + {V4L2_FWHT_FL_PIXENC_YUV, "PIXENC_YUV"}), + __entry->colorspace, __entry->xfer_func, __entry->ycbcr_enc, + __entry->quantization) +); + +DEFINE_EVENT(v4l2_ctrl_fwht_params_tmpl, v4l2_ctrl_fwht_params, + TP_PROTO(const struct v4l2_ctrl_fwht_params *p), + TP_ARGS(p) +); + +#endif + +#undef TRACE_INCLUDE_PATH +#undef TRACE_INCLUDE_FILE +#define TRACE_INCLUDE_PATH ../../drivers/media/test-drivers/visl +#define TRACE_INCLUDE_FILE visl-trace-fwht +#include <trace/define_trace.h> diff --git a/drivers/media/test-drivers/visl/visl-trace-h264.h b/drivers/media/test-drivers/visl/visl-trace-h264.h new file mode 100644 index 000000000000..d84296a01deb --- /dev/null +++ b/drivers/media/test-drivers/visl/visl-trace-h264.h @@ -0,0 +1,349 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#if !defined(_VISL_TRACE_H264_H_) || defined(TRACE_HEADER_MULTI_READ) +#define _VISL_TRACE_H264_H_ + +#include <linux/tracepoint.h> +#include "visl.h" + +#undef TRACE_SYSTEM +#define TRACE_SYSTEM visl_h264_controls + +DECLARE_EVENT_CLASS(v4l2_ctrl_h264_sps_tmpl, + TP_PROTO(const struct v4l2_ctrl_h264_sps *s), + TP_ARGS(s), + TP_STRUCT__entry(__field_struct(struct v4l2_ctrl_h264_sps, s)), + TP_fast_assign(__entry->s = *s), + TP_printk("\nprofile_idc %u\n" + "constraint_set_flags %s\n" + "level_idc %u\n" + "seq_parameter_set_id %u\n" + "chroma_format_idc %u\n" + "bit_depth_luma_minus8 %u\n" + "bit_depth_chroma_minus8 %u\n" + "log2_max_frame_num_minus4 %u\n" + "pic_order_cnt_type %u\n" + "log2_max_pic_order_cnt_lsb_minus4 %u\n" + "max_num_ref_frames %u\n" + "num_ref_frames_in_pic_order_cnt_cycle %u\n" + "offset_for_ref_frame %s\n" + "offset_for_non_ref_pic %d\n" + "offset_for_top_to_bottom_field %d\n" + "pic_width_in_mbs_minus1 %u\n" + "pic_height_in_map_units_minus1 %u\n" + "flags %s", + __entry->s.profile_idc, + __print_flags(__entry->s.constraint_set_flags, "|", + {V4L2_H264_SPS_CONSTRAINT_SET0_FLAG, "CONSTRAINT_SET0_FLAG"}, + {V4L2_H264_SPS_CONSTRAINT_SET1_FLAG, "CONSTRAINT_SET1_FLAG"}, + {V4L2_H264_SPS_CONSTRAINT_SET2_FLAG, "CONSTRAINT_SET2_FLAG"}, + {V4L2_H264_SPS_CONSTRAINT_SET3_FLAG, "CONSTRAINT_SET3_FLAG"}, + {V4L2_H264_SPS_CONSTRAINT_SET4_FLAG, "CONSTRAINT_SET4_FLAG"}, + {V4L2_H264_SPS_CONSTRAINT_SET5_FLAG, "CONSTRAINT_SET5_FLAG"}), + __entry->s.level_idc, + __entry->s.seq_parameter_set_id, + __entry->s.chroma_format_idc, + __entry->s.bit_depth_luma_minus8, + __entry->s.bit_depth_chroma_minus8, + __entry->s.log2_max_frame_num_minus4, + __entry->s.pic_order_cnt_type, + __entry->s.log2_max_pic_order_cnt_lsb_minus4, + __entry->s.max_num_ref_frames, + __entry->s.num_ref_frames_in_pic_order_cnt_cycle, + __print_array(__entry->s.offset_for_ref_frame, + ARRAY_SIZE(__entry->s.offset_for_ref_frame), + sizeof(__entry->s.offset_for_ref_frame[0])), + __entry->s.offset_for_non_ref_pic, + __entry->s.offset_for_top_to_bottom_field, + __entry->s.pic_width_in_mbs_minus1, + __entry->s.pic_height_in_map_units_minus1, + __print_flags(__entry->s.flags, "|", + {V4L2_H264_SPS_FLAG_SEPARATE_COLOUR_PLANE, "SEPARATE_COLOUR_PLANE"}, + {V4L2_H264_SPS_FLAG_QPPRIME_Y_ZERO_TRANSFORM_BYPASS, "QPPRIME_Y_ZERO_TRANSFORM_BYPASS"}, + {V4L2_H264_SPS_FLAG_DELTA_PIC_ORDER_ALWAYS_ZERO, "DELTA_PIC_ORDER_ALWAYS_ZERO"}, + {V4L2_H264_SPS_FLAG_GAPS_IN_FRAME_NUM_VALUE_ALLOWED, "GAPS_IN_FRAME_NUM_VALUE_ALLOWED"}, + {V4L2_H264_SPS_FLAG_FRAME_MBS_ONLY, "FRAME_MBS_ONLY"}, + {V4L2_H264_SPS_FLAG_MB_ADAPTIVE_FRAME_FIELD, "MB_ADAPTIVE_FRAME_FIELD"}, + {V4L2_H264_SPS_FLAG_DIRECT_8X8_INFERENCE, "DIRECT_8X8_INFERENCE"} + )) +); + +DECLARE_EVENT_CLASS(v4l2_ctrl_h264_pps_tmpl, + TP_PROTO(const struct v4l2_ctrl_h264_pps *p), + TP_ARGS(p), + TP_STRUCT__entry(__field_struct(struct v4l2_ctrl_h264_pps, p)), + TP_fast_assign(__entry->p = *p), + TP_printk("\npic_parameter_set_id %u\n" + "seq_parameter_set_id %u\n" + "num_slice_groups_minus1 %u\n" + "num_ref_idx_l0_default_active_minus1 %u\n" + "num_ref_idx_l1_default_active_minus1 %u\n" + "weighted_bipred_idc %u\n" + "pic_init_qp_minus26 %d\n" + "pic_init_qs_minus26 %d\n" + "chroma_qp_index_offset %d\n" + "second_chroma_qp_index_offset %d\n" + "flags %s", + __entry->p.pic_parameter_set_id, + __entry->p.seq_parameter_set_id, + __entry->p.num_slice_groups_minus1, + __entry->p.num_ref_idx_l0_default_active_minus1, + __entry->p.num_ref_idx_l1_default_active_minus1, + __entry->p.weighted_bipred_idc, + __entry->p.pic_init_qp_minus26, + __entry->p.pic_init_qs_minus26, + __entry->p.chroma_qp_index_offset, + __entry->p.second_chroma_qp_index_offset, + __print_flags(__entry->p.flags, "|", + {V4L2_H264_PPS_FLAG_ENTROPY_CODING_MODE, "ENTROPY_CODING_MODE"}, + {V4L2_H264_PPS_FLAG_BOTTOM_FIELD_PIC_ORDER_IN_FRAME_PRESENT, "BOTTOM_FIELD_PIC_ORDER_IN_FRAME_PRESENT"}, + {V4L2_H264_PPS_FLAG_WEIGHTED_PRED, "WEIGHTED_PRED"}, + {V4L2_H264_PPS_FLAG_DEBLOCKING_FILTER_CONTROL_PRESENT, "DEBLOCKING_FILTER_CONTROL_PRESENT"}, + {V4L2_H264_PPS_FLAG_CONSTRAINED_INTRA_PRED, "CONSTRAINED_INTRA_PRED"}, + {V4L2_H264_PPS_FLAG_REDUNDANT_PIC_CNT_PRESENT, "REDUNDANT_PIC_CNT_PRESENT"}, + {V4L2_H264_PPS_FLAG_TRANSFORM_8X8_MODE, "TRANSFORM_8X8_MODE"}, + {V4L2_H264_PPS_FLAG_SCALING_MATRIX_PRESENT, "SCALING_MATRIX_PRESENT"} + )) +); + +DECLARE_EVENT_CLASS(v4l2_ctrl_h264_scaling_matrix_tmpl, + TP_PROTO(const struct v4l2_ctrl_h264_scaling_matrix *s), + TP_ARGS(s), + TP_STRUCT__entry(__field_struct(struct v4l2_ctrl_h264_scaling_matrix, s)), + TP_fast_assign(__entry->s = *s), + TP_printk("\nscaling_list_4x4 {%s}\nscaling_list_8x8 {%s}", + __print_hex_dump("", DUMP_PREFIX_NONE, 32, 1, + __entry->s.scaling_list_4x4, + sizeof(__entry->s.scaling_list_4x4), + false), + __print_hex_dump("", DUMP_PREFIX_NONE, 32, 1, + __entry->s.scaling_list_8x8, + sizeof(__entry->s.scaling_list_8x8), + false) + ) +); + +DECLARE_EVENT_CLASS(v4l2_ctrl_h264_pred_weights_tmpl, + TP_PROTO(const struct v4l2_ctrl_h264_pred_weights *p), + TP_ARGS(p), + TP_STRUCT__entry(__field_struct(struct v4l2_ctrl_h264_pred_weights, p)), + TP_fast_assign(__entry->p = *p), + TP_printk("\nluma_log2_weight_denom %u\n" + "chroma_log2_weight_denom %u\n" + "weight_factor[0].luma_weight %s\n" + "weight_factor[0].luma_offset %s\n" + "weight_factor[0].chroma_weight {%s}\n" + "weight_factor[0].chroma_offset {%s}\n" + "weight_factor[1].luma_weight %s\n" + "weight_factor[1].luma_offset %s\n" + "weight_factor[1].chroma_weight {%s}\n" + "weight_factor[1].chroma_offset {%s}\n", + __entry->p.luma_log2_weight_denom, + __entry->p.chroma_log2_weight_denom, + __print_array(__entry->p.weight_factors[0].luma_weight, + ARRAY_SIZE(__entry->p.weight_factors[0].luma_weight), + sizeof(__entry->p.weight_factors[0].luma_weight[0])), + __print_array(__entry->p.weight_factors[0].luma_offset, + ARRAY_SIZE(__entry->p.weight_factors[0].luma_offset), + sizeof(__entry->p.weight_factors[0].luma_offset[0])), + __print_hex_dump("", DUMP_PREFIX_NONE, 32, 1, + __entry->p.weight_factors[0].chroma_weight, + sizeof(__entry->p.weight_factors[0].chroma_weight), + false), + __print_hex_dump("", DUMP_PREFIX_NONE, 32, 1, + __entry->p.weight_factors[0].chroma_offset, + sizeof(__entry->p.weight_factors[0].chroma_offset), + false), + __print_array(__entry->p.weight_factors[1].luma_weight, + ARRAY_SIZE(__entry->p.weight_factors[1].luma_weight), + sizeof(__entry->p.weight_factors[1].luma_weight[0])), + __print_array(__entry->p.weight_factors[1].luma_offset, + ARRAY_SIZE(__entry->p.weight_factors[1].luma_offset), + sizeof(__entry->p.weight_factors[1].luma_offset[0])), + __print_hex_dump("", DUMP_PREFIX_NONE, 32, 1, + __entry->p.weight_factors[1].chroma_weight, + sizeof(__entry->p.weight_factors[1].chroma_weight), + false), + __print_hex_dump("", DUMP_PREFIX_NONE, 32, 1, + __entry->p.weight_factors[1].chroma_offset, + sizeof(__entry->p.weight_factors[1].chroma_offset), + false) + ) +); + +DECLARE_EVENT_CLASS(v4l2_ctrl_h264_slice_params_tmpl, + TP_PROTO(const struct v4l2_ctrl_h264_slice_params *s), + TP_ARGS(s), + TP_STRUCT__entry(__field_struct(struct v4l2_ctrl_h264_slice_params, s)), + TP_fast_assign(__entry->s = *s), + TP_printk("\nheader_bit_size %u\n" + "first_mb_in_slice %u\n" + "slice_type %s\n" + "colour_plane_id %u\n" + "redundant_pic_cnt %u\n" + "cabac_init_idc %u\n" + "slice_qp_delta %d\n" + "slice_qs_delta %d\n" + "disable_deblocking_filter_idc %u\n" + "slice_alpha_c0_offset_div2 %u\n" + "slice_beta_offset_div2 %u\n" + "num_ref_idx_l0_active_minus1 %u\n" + "num_ref_idx_l1_active_minus1 %u\n" + "flags %s", + __entry->s.header_bit_size, + __entry->s.first_mb_in_slice, + __print_symbolic(__entry->s.slice_type, + {V4L2_H264_SLICE_TYPE_P, "P"}, + {V4L2_H264_SLICE_TYPE_B, "B"}, + {V4L2_H264_SLICE_TYPE_I, "I"}, + {V4L2_H264_SLICE_TYPE_SP, "SP"}, + {V4L2_H264_SLICE_TYPE_SI, "SI"}), + __entry->s.colour_plane_id, + __entry->s.redundant_pic_cnt, + __entry->s.cabac_init_idc, + __entry->s.slice_qp_delta, + __entry->s.slice_qs_delta, + __entry->s.disable_deblocking_filter_idc, + __entry->s.slice_alpha_c0_offset_div2, + __entry->s.slice_beta_offset_div2, + __entry->s.num_ref_idx_l0_active_minus1, + __entry->s.num_ref_idx_l1_active_minus1, + __print_flags(__entry->s.flags, "|", + {V4L2_H264_SLICE_FLAG_DIRECT_SPATIAL_MV_PRED, "DIRECT_SPATIAL_MV_PRED"}, + {V4L2_H264_SLICE_FLAG_SP_FOR_SWITCH, "SP_FOR_SWITCH"}) + ) +); + +DECLARE_EVENT_CLASS(v4l2_h264_reference_tmpl, + TP_PROTO(const struct v4l2_h264_reference *r, int i), + TP_ARGS(r, i), + TP_STRUCT__entry(__field_struct(struct v4l2_h264_reference, r) + __field(int, i)), + TP_fast_assign(__entry->r = *r; __entry->i = i;), + TP_printk("[%d]: fields %s index %u", + __entry->i, + __print_flags(__entry->r.fields, "|", + {V4L2_H264_TOP_FIELD_REF, "TOP_FIELD_REF"}, + {V4L2_H264_BOTTOM_FIELD_REF, "BOTTOM_FIELD_REF"}, + {V4L2_H264_FRAME_REF, "FRAME_REF"}), + __entry->r.index + ) +); + +DECLARE_EVENT_CLASS(v4l2_ctrl_h264_decode_params_tmpl, + TP_PROTO(const struct v4l2_ctrl_h264_decode_params *d), + TP_ARGS(d), + TP_STRUCT__entry(__field_struct(struct v4l2_ctrl_h264_decode_params, d)), + TP_fast_assign(__entry->d = *d), + TP_printk("\nnal_ref_idc %u\n" + "frame_num %u\n" + "top_field_order_cnt %d\n" + "bottom_field_order_cnt %d\n" + "idr_pic_id %u\n" + "pic_order_cnt_lsb %u\n" + "delta_pic_order_cnt_bottom %d\n" + "delta_pic_order_cnt0 %d\n" + "delta_pic_order_cnt1 %d\n" + "dec_ref_pic_marking_bit_size %u\n" + "pic_order_cnt_bit_size %u\n" + "slice_group_change_cycle %u\n" + "flags %s\n", + __entry->d.nal_ref_idc, + __entry->d.frame_num, + __entry->d.top_field_order_cnt, + __entry->d.bottom_field_order_cnt, + __entry->d.idr_pic_id, + __entry->d.pic_order_cnt_lsb, + __entry->d.delta_pic_order_cnt_bottom, + __entry->d.delta_pic_order_cnt0, + __entry->d.delta_pic_order_cnt1, + __entry->d.dec_ref_pic_marking_bit_size, + __entry->d.pic_order_cnt_bit_size, + __entry->d.slice_group_change_cycle, + __print_flags(__entry->d.flags, "|", + {V4L2_H264_DECODE_PARAM_FLAG_IDR_PIC, "IDR_PIC"}, + {V4L2_H264_DECODE_PARAM_FLAG_FIELD_PIC, "FIELD_PIC"}, + {V4L2_H264_DECODE_PARAM_FLAG_BOTTOM_FIELD, "BOTTOM_FIELD"}, + {V4L2_H264_DECODE_PARAM_FLAG_PFRAME, "PFRAME"}, + {V4L2_H264_DECODE_PARAM_FLAG_BFRAME, "BFRAME"}) + ) +); + +DECLARE_EVENT_CLASS(v4l2_h264_dpb_entry_tmpl, + TP_PROTO(const struct v4l2_h264_dpb_entry *e, int i), + TP_ARGS(e, i), + TP_STRUCT__entry(__field_struct(struct v4l2_h264_dpb_entry, e) + __field(int, i)), + TP_fast_assign(__entry->e = *e; __entry->i = i;), + TP_printk("[%d]: reference_ts %llu, pic_num %u frame_num %u fields %s " + "top_field_order_cnt %d bottom_field_order_cnt %d flags %s", + __entry->i, + __entry->e.reference_ts, + __entry->e.pic_num, + __entry->e.frame_num, + __print_flags(__entry->e.fields, "|", + {V4L2_H264_TOP_FIELD_REF, "TOP_FIELD_REF"}, + {V4L2_H264_BOTTOM_FIELD_REF, "BOTTOM_FIELD_REF"}, + {V4L2_H264_FRAME_REF, "FRAME_REF"}), + __entry->e.top_field_order_cnt, + __entry->e.bottom_field_order_cnt, + __print_flags(__entry->e.flags, "|", + {V4L2_H264_DPB_ENTRY_FLAG_VALID, "VALID"}, + {V4L2_H264_DPB_ENTRY_FLAG_ACTIVE, "ACTIVE"}, + {V4L2_H264_DPB_ENTRY_FLAG_LONG_TERM, "LONG_TERM"}, + {V4L2_H264_DPB_ENTRY_FLAG_FIELD, "FIELD"}) + + ) +); + +DEFINE_EVENT(v4l2_ctrl_h264_sps_tmpl, v4l2_ctrl_h264_sps, + TP_PROTO(const struct v4l2_ctrl_h264_sps *s), + TP_ARGS(s) +); + +DEFINE_EVENT(v4l2_ctrl_h264_pps_tmpl, v4l2_ctrl_h264_pps, + TP_PROTO(const struct v4l2_ctrl_h264_pps *p), + TP_ARGS(p) +); + +DEFINE_EVENT(v4l2_ctrl_h264_scaling_matrix_tmpl, v4l2_ctrl_h264_scaling_matrix, + TP_PROTO(const struct v4l2_ctrl_h264_scaling_matrix *s), + TP_ARGS(s) +); + +DEFINE_EVENT(v4l2_ctrl_h264_pred_weights_tmpl, v4l2_ctrl_h264_pred_weights, + TP_PROTO(const struct v4l2_ctrl_h264_pred_weights *p), + TP_ARGS(p) +); + +DEFINE_EVENT(v4l2_ctrl_h264_slice_params_tmpl, v4l2_ctrl_h264_slice_params, + TP_PROTO(const struct v4l2_ctrl_h264_slice_params *s), + TP_ARGS(s) +); + +DEFINE_EVENT(v4l2_h264_reference_tmpl, v4l2_h264_ref_pic_list0, + TP_PROTO(const struct v4l2_h264_reference *r, int i), + TP_ARGS(r, i) +); + +DEFINE_EVENT(v4l2_h264_reference_tmpl, v4l2_h264_ref_pic_list1, + TP_PROTO(const struct v4l2_h264_reference *r, int i), + TP_ARGS(r, i) +); + +DEFINE_EVENT(v4l2_ctrl_h264_decode_params_tmpl, v4l2_ctrl_h264_decode_params, + TP_PROTO(const struct v4l2_ctrl_h264_decode_params *d), + TP_ARGS(d) +); + +DEFINE_EVENT(v4l2_h264_dpb_entry_tmpl, v4l2_h264_dpb_entry, + TP_PROTO(const struct v4l2_h264_dpb_entry *e, int i), + TP_ARGS(e, i) +); + +#endif + +#undef TRACE_INCLUDE_PATH +#undef TRACE_INCLUDE_FILE +#define TRACE_INCLUDE_PATH ../../drivers/media/test-drivers/visl +#define TRACE_INCLUDE_FILE visl-trace-h264 +#include <trace/define_trace.h> diff --git a/drivers/media/test-drivers/visl/visl-trace-hevc.h b/drivers/media/test-drivers/visl/visl-trace-hevc.h new file mode 100644 index 000000000000..837b8ec12e97 --- /dev/null +++ b/drivers/media/test-drivers/visl/visl-trace-hevc.h @@ -0,0 +1,405 @@ +/* SPDX-License-Identifier: GPL-2.0+ */ +#if !defined(_VISL_TRACE_HEVC_H_) || defined(TRACE_HEADER_MULTI_READ) +#define _VISL_TRACE_HEVC_H_ + +#include <linux/tracepoint.h> +#include "visl.h" + +#undef TRACE_SYSTEM +#define TRACE_SYSTEM visl_hevc_controls + +DECLARE_EVENT_CLASS(v4l2_ctrl_hevc_sps_tmpl, + TP_PROTO(const struct v4l2_ctrl_hevc_sps *s), + TP_ARGS(s), + TP_STRUCT__entry(__field_struct(struct v4l2_ctrl_hevc_sps, s)), + TP_fast_assign(__entry->s = *s), + TP_printk("\nvideo_parameter_set_id %u\n" + "seq_parameter_set_id %u\n" + "pic_width_in_luma_samples %u\n" + "pic_height_in_luma_samples %u\n" + "bit_depth_luma_minus8 %u\n" + "bit_depth_chroma_minus8 %u\n" + "log2_max_pic_order_cnt_lsb_minus4 %u\n" + "sps_max_dec_pic_buffering_minus1 %u\n" + "sps_max_num_reorder_pics %u\n" + "sps_max_latency_increase_plus1 %u\n" + "log2_min_luma_coding_block_size_minus3 %u\n" + "log2_diff_max_min_luma_coding_block_size %u\n" + "log2_min_luma_transform_block_size_minus2 %u\n" + "log2_diff_max_min_luma_transform_block_size %u\n" + "max_transform_hierarchy_depth_inter %u\n" + "max_transform_hierarchy_depth_intra %u\n" + "pcm_sample_bit_depth_luma_minus1 %u\n" + "pcm_sample_bit_depth_chroma_minus1 %u\n" + "log2_min_pcm_luma_coding_block_size_minus3 %u\n" + "log2_diff_max_min_pcm_luma_coding_block_size %u\n" + "num_short_term_ref_pic_sets %u\n" + "num_long_term_ref_pics_sps %u\n" + "chroma_format_idc %u\n" + "sps_max_sub_layers_minus1 %u\n" + "flags %s", + __entry->s.video_parameter_set_id, + __entry->s.seq_parameter_set_id, + __entry->s.pic_width_in_luma_samples, + __entry->s.pic_height_in_luma_samples, + __entry->s.bit_depth_luma_minus8, + __entry->s.bit_depth_chroma_minus8, + __entry->s.log2_max_pic_order_cnt_lsb_minus4, + __entry->s.sps_max_dec_pic_buffering_minus1, + __entry->s.sps_max_num_reorder_pics, + __entry->s.sps_max_latency_increase_plus1, + __entry->s.log2_min_luma_coding_block_size_minus3, + __entry->s.log2_diff_max_min_luma_coding_block_size, + __entry->s.log2_min_luma_transform_block_size_minus2, + __entry->s.log2_diff_max_min_luma_transform_block_size, + __entry->s.max_transform_hierarchy_depth_inter, + __entry->s.max_transform_hierarchy_depth_intra, + __entry->s.pcm_sample_bit_depth_luma_minus1, + __entry->s.pcm_sample_bit_depth_chroma_minus1, + __entry->s.log2_min_pcm_luma_coding_block_size_minus3, + __entry->s.log2_diff_max_min_pcm_luma_coding_block_size, + __entry->s.num_short_term_ref_pic_sets, + __entry->s.num_long_term_ref_pics_sps, + __entry->s.chroma_format_idc, + __entry->s.sps_max_sub_layers_minus1, + __print_flags(__entry->s.flags, "|", + {V4L2_HEVC_SPS_FLAG_SEPARATE_COLOUR_PLANE, "SEPARATE_COLOUR_PLANE"}, + {V4L2_HEVC_SPS_FLAG_SCALING_LIST_ENABLED, "SCALING_LIST_ENABLED"}, + {V4L2_HEVC_SPS_FLAG_AMP_ENABLED, "AMP_ENABLED"}, + {V4L2_HEVC_SPS_FLAG_SAMPLE_ADAPTIVE_OFFSET, "SAMPLE_ADAPTIVE_OFFSET"}, + {V4L2_HEVC_SPS_FLAG_PCM_ENABLED, "PCM_ENABLED"}, + {V4L2_HEVC_SPS_FLAG_PCM_LOOP_FILTER_DISABLED, "V4L2_HEVC_SPS_FLAG_PCM_LOOP_FILTER_DISABLED"}, + {V4L2_HEVC_SPS_FLAG_LONG_TERM_REF_PICS_PRESENT, "LONG_TERM_REF_PICS_PRESENT"}, + {V4L2_HEVC_SPS_FLAG_SPS_TEMPORAL_MVP_ENABLED, "TEMPORAL_MVP_ENABLED"}, + {V4L2_HEVC_SPS_FLAG_STRONG_INTRA_SMOOTHING_ENABLED, "STRONG_INTRA_SMOOTHING_ENABLED"} + )) + +); + + +DECLARE_EVENT_CLASS(v4l2_ctrl_hevc_pps_tmpl, + TP_PROTO(const struct v4l2_ctrl_hevc_pps *p), + TP_ARGS(p), + TP_STRUCT__entry(__field_struct(struct v4l2_ctrl_hevc_pps, p)), + TP_fast_assign(__entry->p = *p), + TP_printk("\npic_parameter_set_id %u\n" + "num_extra_slice_header_bits %u\n" + "num_ref_idx_l0_default_active_minus1 %u\n" + "num_ref_idx_l1_default_active_minus1 %u\n" + "init_qp_minus26 %d\n" + "diff_cu_qp_delta_depth %u\n" + "pps_cb_qp_offset %d\n" + "pps_cr_qp_offset %d\n" + "num_tile_columns_minus1 %d\n" + "num_tile_rows_minus1 %d\n" + "column_width_minus1 %s\n" + "row_height_minus1 %s\n" + "pps_beta_offset_div2 %d\n" + "pps_tc_offset_div2 %d\n" + "log2_parallel_merge_level_minus2 %u\n" + "flags %s", + __entry->p.pic_parameter_set_id, + __entry->p.num_extra_slice_header_bits, + __entry->p.num_ref_idx_l0_default_active_minus1, + __entry->p.num_ref_idx_l1_default_active_minus1, + __entry->p.init_qp_minus26, + __entry->p.diff_cu_qp_delta_depth, + __entry->p.pps_cb_qp_offset, + __entry->p.pps_cr_qp_offset, + __entry->p.num_tile_columns_minus1, + __entry->p.num_tile_rows_minus1, + __print_array(__entry->p.column_width_minus1, + ARRAY_SIZE(__entry->p.column_width_minus1), + sizeof(__entry->p.column_width_minus1[0])), + __print_array(__entry->p.row_height_minus1, + ARRAY_SIZE(__entry->p.row_height_minus1), + sizeof(__entry->p.row_height_minus1[0])), + __entry->p.pps_beta_offset_div2, + __entry->p.pps_tc_offset_div2, + __entry->p.log2_parallel_merge_level_minus2, + __print_flags(__entry->p.flags, "|", + {V4L2_HEVC_PPS_FLAG_DEPENDENT_SLICE_SEGMENT_ENABLED, "DEPENDENT_SLICE_SEGMENT_ENABLED"}, + {V4L2_HEVC_PPS_FLAG_OUTPUT_FLAG_PRESENT, "OUTPUT_FLAG_PRESENT"}, + {V4L2_HEVC_PPS_FLAG_SIGN_DATA_HIDING_ENABLED, "SIGN_DATA_HIDING_ENABLED"}, + {V4L2_HEVC_PPS_FLAG_CABAC_INIT_PRESENT, "CABAC_INIT_PRESENT"}, + {V4L2_HEVC_PPS_FLAG_CONSTRAINED_INTRA_PRED, "CONSTRAINED_INTRA_PRED"}, + {V4L2_HEVC_PPS_FLAG_CU_QP_DELTA_ENABLED, "CU_QP_DELTA_ENABLED"}, + {V4L2_HEVC_PPS_FLAG_PPS_SLICE_CHROMA_QP_OFFSETS_PRESENT, "PPS_SLICE_CHROMA_QP_OFFSETS_PRESENT"}, + {V4L2_HEVC_PPS_FLAG_WEIGHTED_PRED, "WEIGHTED_PRED"}, + {V4L2_HEVC_PPS_FLAG_WEIGHTED_BIPRED, "WEIGHTED_BIPRED"}, + {V4L2_HEVC_PPS_FLAG_TRANSQUANT_BYPASS_ENABLED, "TRANSQUANT_BYPASS_ENABLED"}, + {V4L2_HEVC_PPS_FLAG_TILES_ENABLED, "TILES_ENABLED"}, + {V4L2_HEVC_PPS_FLAG_ENTROPY_CODING_SYNC_ENABLED, "ENTROPY_CODING_SYNC_ENABLED"}, + {V4L2_HEVC_PPS_FLAG_LOOP_FILTER_ACROSS_TILES_ENABLED, "LOOP_FILTER_ACROSS_TILES_ENABLED"}, + {V4L2_HEVC_PPS_FLAG_PPS_LOOP_FILTER_ACROSS_SLICES_ENABLED, "PPS_LOOP_FILTER_ACROSS_SLICES_ENABLED"}, + {V4L2_HEVC_PPS_FLAG_DEBLOCKING_FILTER_OVERRIDE_ENABLED, "DEBLOCKING_FILTER_OVERRIDE_ENABLED"}, + {V4L2_HEVC_PPS_FLAG_PPS_DISABLE_DEBLOCKING_FILTER, "DISABLE_DEBLOCKING_FILTER"}, + {V4L2_HEVC_PPS_FLAG_LISTS_MODIFICATION_PRESENT, "LISTS_MODIFICATION_PRESENT"}, + {V4L2_HEVC_PPS_FLAG_SLICE_SEGMENT_HEADER_EXTENSION_PRESENT, "SLICE_SEGMENT_HEADER_EXTENSION_PRESENT"}, + {V4L2_HEVC_PPS_FLAG_DEBLOCKING_FILTER_CONTROL_PRESENT, "DEBLOCKING_FILTER_CONTROL_PRESENT"}, + {V4L2_HEVC_PPS_FLAG_UNIFORM_SPACING, "UNIFORM_SPACING"} + )) + +); + + + +DECLARE_EVENT_CLASS(v4l2_ctrl_hevc_slice_params_tmpl, + TP_PROTO(const struct v4l2_ctrl_hevc_slice_params *s), + TP_ARGS(s), + TP_STRUCT__entry(__field_struct(struct v4l2_ctrl_hevc_slice_params, s)), + TP_fast_assign(__entry->s = *s), + TP_printk("\nbit_size %u\n" + "data_byte_offset %u\n" + "num_entry_point_offsets %u\n" + "nal_unit_type %u\n" + "nuh_temporal_id_plus1 %u\n" + "slice_type %u\n" + "colour_plane_id %u\n" + "slice_pic_order_cnt %d\n" + "num_ref_idx_l0_active_minus1 %u\n" + "num_ref_idx_l1_active_minus1 %u\n" + "collocated_ref_idx %u\n" + "five_minus_max_num_merge_cand %u\n" + "slice_qp_delta %d\n" + "slice_cb_qp_offset %d\n" + "slice_cr_qp_offset %d\n" + "slice_act_y_qp_offset %d\n" + "slice_act_cb_qp_offset %d\n" + "slice_act_cr_qp_offset %d\n" + "slice_beta_offset_div2 %d\n" + "slice_tc_offset_div2 %d\n" + "pic_struct %u\n" + "slice_segment_addr %u\n" + "ref_idx_l0 %s\n" + "ref_idx_l1 %s\n" + "short_term_ref_pic_set_size %u\n" + "long_term_ref_pic_set_size %u\n" + "flags %s", + __entry->s.bit_size, + __entry->s.data_byte_offset, + __entry->s.num_entry_point_offsets, + __entry->s.nal_unit_type, + __entry->s.nuh_temporal_id_plus1, + __entry->s.slice_type, + __entry->s.colour_plane_id, + __entry->s.slice_pic_order_cnt, + __entry->s.num_ref_idx_l0_active_minus1, + __entry->s.num_ref_idx_l1_active_minus1, + __entry->s.collocated_ref_idx, + __entry->s.five_minus_max_num_merge_cand, + __entry->s.slice_qp_delta, + __entry->s.slice_cb_qp_offset, + __entry->s.slice_cr_qp_offset, + __entry->s.slice_act_y_qp_offset, + __entry->s.slice_act_cb_qp_offset, + __entry->s.slice_act_cr_qp_offset, + __entry->s.slice_beta_offset_div2, + __entry->s.slice_tc_offset_div2, + __entry->s.pic_struct, + __entry->s.slice_segment_addr, + __print_array(__entry->s.ref_idx_l0, + ARRAY_SIZE(__entry->s.ref_idx_l0), + sizeof(__entry->s.ref_idx_l0[0])), + __print_array(__entry->s.ref_idx_l1, + ARRAY_SIZE(__entry->s.ref_idx_l1), + sizeof(__entry->s.ref_idx_l1[0])), + __entry->s.short_term_ref_pic_set_size, + __entry->s.long_term_ref_pic_set_size, + __print_flags(__entry->s.flags, "|", + {V4L2_HEVC_SLICE_PARAMS_FLAG_SLICE_SAO_LUMA, "SLICE_SAO_LUMA"}, + {V4L2_HEVC_SLICE_PARAMS_FLAG_SLICE_SAO_CHROMA, "SLICE_SAO_CHROMA"}, + {V4L2_HEVC_SLICE_PARAMS_FLAG_SLICE_TEMPORAL_MVP_ENABLED, "SLICE_TEMPORAL_MVP_ENABLED"}, + {V4L2_HEVC_SLICE_PARAMS_FLAG_MVD_L1_ZERO, "MVD_L1_ZERO"}, + {V4L2_HEVC_SLICE_PARAMS_FLAG_CABAC_INIT, "CABAC_INIT"}, + {V4L2_HEVC_SLICE_PARAMS_FLAG_COLLOCATED_FROM_L0, "COLLOCATED_FROM_L0"}, + {V4L2_HEVC_SLICE_PARAMS_FLAG_USE_INTEGER_MV, "USE_INTEGER_MV"}, + {V4L2_HEVC_SLICE_PARAMS_FLAG_SLICE_DEBLOCKING_FILTER_DISABLED, "SLICE_DEBLOCKING_FILTER_DISABLED"}, + {V4L2_HEVC_SLICE_PARAMS_FLAG_SLICE_LOOP_FILTER_ACROSS_SLICES_ENABLED, "SLICE_LOOP_FILTER_ACROSS_SLICES_ENABLED"}, + {V4L2_HEVC_SLICE_PARAMS_FLAG_DEPENDENT_SLICE_SEGMENT, "DEPENDENT_SLICE_SEGMENT"} + + )) +); + +DECLARE_EVENT_CLASS(v4l2_hevc_pred_weight_table_tmpl, + TP_PROTO(const struct v4l2_hevc_pred_weight_table *p), + TP_ARGS(p), + TP_STRUCT__entry(__field_struct(struct v4l2_hevc_pred_weight_table, p)), + TP_fast_assign(__entry->p = *p), + TP_printk("\ndelta_luma_weight_l0 %s\n" + "luma_offset_l0 %s\n" + "delta_chroma_weight_l0 {%s}\n" + "chroma_offset_l0 {%s}\n" + "delta_luma_weight_l1 %s\n" + "luma_offset_l1 %s\n" + "delta_chroma_weight_l1 {%s}\n" + "chroma_offset_l1 {%s}\n" + "luma_log2_weight_denom %d\n" + "delta_chroma_log2_weight_denom %d\n", + __print_array(__entry->p.delta_luma_weight_l0, + ARRAY_SIZE(__entry->p.delta_luma_weight_l0), + sizeof(__entry->p.delta_luma_weight_l0[0])), + __print_array(__entry->p.luma_offset_l0, + ARRAY_SIZE(__entry->p.luma_offset_l0), + sizeof(__entry->p.luma_offset_l0[0])), + __print_hex_dump("", DUMP_PREFIX_NONE, 32, 1, + __entry->p.delta_chroma_weight_l0, + sizeof(__entry->p.delta_chroma_weight_l0), + false), + __print_hex_dump("", DUMP_PREFIX_NONE, 32, 1, + __entry->p.chroma_offset_l0, + sizeof(__entry->p.chroma_offset_l0), + false), + __print_array(__entry->p.delta_luma_weight_l1, + ARRAY_SIZE(__entry->p.delta_luma_weight_l1), + sizeof(__entry->p.delta_luma_weight_l1[0])), + __print_array(__entry->p.luma_offset_l1, + ARRAY_SIZE(__entry->p.luma_offset_l1), + sizeof(__entry->p.luma_offset_l1[0])), + __print_hex_dump("", DUMP_PREFIX_NONE, 32, 1, + __entry->p.delta_chroma_weight_l1, + sizeof(__entry->p.delta_chroma_weight_l1), + false), + __print_hex_dump("", DUMP_PREFIX_NONE, 32, 1, + __entry->p.chroma_offset_l1, + sizeof(__entry->p.chroma_offset_l1), + false), + __entry->p.luma_log2_weight_denom, + __entry->p.delta_chroma_log2_weight_denom + + )) + +DECLARE_EVENT_CLASS(v4l2_ctrl_hevc_scaling_matrix_tmpl, + TP_PROTO(const struct v4l2_ctrl_hevc_scaling_matrix *s), + TP_ARGS(s), + TP_STRUCT__entry(__field_struct(struct v4l2_ctrl_hevc_scaling_matrix, s)), + TP_fast_assign(__entry->s = *s), + TP_printk("\nscaling_list_4x4 {%s}\n" + "scaling_list_8x8 {%s}\n" + "scaling_list_16x16 {%s}\n" + "scaling_list_32x32 {%s}\n" + "scaling_list_dc_coef_16x16 %s\n" + "scaling_list_dc_coef_32x32 %s\n", + __print_hex_dump("", DUMP_PREFIX_NONE, 32, 1, + __entry->s.scaling_list_4x4, + sizeof(__entry->s.scaling_list_4x4), + false), + __print_hex_dump("", DUMP_PREFIX_NONE, 32, 1, + __entry->s.scaling_list_8x8, + sizeof(__entry->s.scaling_list_8x8), + false), + __print_hex_dump("", DUMP_PREFIX_NONE, 32, 1, + __entry->s.scaling_list_16x16, + sizeof(__entry->s.scaling_list_16x16), + false), + __print_hex_dump("", DUMP_PREFIX_NONE, 32, 1, + __entry->s.scaling_list_32x32, + sizeof(__entry->s.scaling_list_32x32), + false), + __print_array(__entry->s.scaling_list_dc_coef_16x16, + ARRAY_SIZE(__entry->s.scaling_list_dc_coef_16x16), + sizeof(__entry->s.scaling_list_dc_coef_16x16[0])), + __print_array(__entry->s.scaling_list_dc_coef_32x32, + ARRAY_SIZE(__entry->s.scaling_list_dc_coef_32x32), + sizeof(__entry->s.scaling_list_dc_coef_32x32[0])) + )) + +DECLARE_EVENT_CLASS(v4l2_ctrl_hevc_decode_params_tmpl, + TP_PROTO(const struct v4l2_ctrl_hevc_decode_params *d), + TP_ARGS(d), + TP_STRUCT__entry(__field_struct(struct v4l2_ctrl_hevc_decode_params, d)), + TP_fast_assign(__entry->d = *d), + TP_printk("\npic_order_cnt_val %d\n" + "short_term_ref_pic_set_size %u\n" + "long_term_ref_pic_set_size %u\n" + "num_active_dpb_entries %u\n" + "num_poc_st_curr_before %u\n" + "num_poc_st_curr_after %u\n" + "num_poc_lt_curr %u\n" + "poc_st_curr_before %s\n" + "poc_st_curr_after %s\n" + "poc_lt_curr %s\n" + "flags %s", + __entry->d.pic_order_cnt_val, + __entry->d.short_term_ref_pic_set_size, + __entry->d.long_term_ref_pic_set_size, + __entry->d.num_active_dpb_entries, + __entry->d.num_poc_st_curr_before, + __entry->d.num_poc_st_curr_after, + __entry->d.num_poc_lt_curr, + __print_array(__entry->d.poc_st_curr_before, + ARRAY_SIZE(__entry->d.poc_st_curr_before), + sizeof(__entry->d.poc_st_curr_before[0])), + __print_array(__entry->d.poc_st_curr_after, + ARRAY_SIZE(__entry->d.poc_st_curr_after), + sizeof(__entry->d.poc_st_curr_after[0])), + __print_array(__entry->d.poc_lt_curr, + ARRAY_SIZE(__entry->d.poc_lt_curr), + sizeof(__entry->d.poc_lt_curr[0])), + __print_flags(__entry->d.flags, "|", + {V4L2_HEVC_DECODE_PARAM_FLAG_IRAP_PIC, "IRAP_PIC"}, + {V4L2_HEVC_DECODE_PARAM_FLAG_IDR_PIC, "IDR_PIC"}, + {V4L2_HEVC_DECODE_PARAM_FLAG_NO_OUTPUT_OF_PRIOR, "NO_OUTPUT_OF_PRIOR"} + )) +); + + +DECLARE_EVENT_CLASS(v4l2_hevc_dpb_entry_tmpl, + TP_PROTO(const struct v4l2_hevc_dpb_entry *e), + TP_ARGS(e), + TP_STRUCT__entry(__field_struct(struct v4l2_hevc_dpb_entry, e)), + TP_fast_assign(__entry->e = *e), + TP_printk("\ntimestamp %llu\n" + "flags %s\n" + "field_pic %u\n" + "pic_order_cnt_val %d\n", + __entry->e.timestamp, + __print_flags(__entry->e.flags, "|", + {V4L2_HEVC_DPB_ENTRY_LONG_TERM_REFERENCE, "LONG_TERM_REFERENCE"} + ), + __entry->e.field_pic, + __entry->e.pic_order_cnt_val + )) + +DEFINE_EVENT(v4l2_ctrl_hevc_sps_tmpl, v4l2_ctrl_hevc_sps, + TP_PROTO(const struct v4l2_ctrl_hevc_sps *s), + TP_ARGS(s) +); + +DEFINE_EVENT(v4l2_ctrl_hevc_pps_tmpl, v4l2_ctrl_hevc_pps, + TP_PROTO(const struct v4l2_ctrl_hevc_pps *p), + TP_ARGS(p) +); + +DEFINE_EVENT(v4l2_ctrl_hevc_slice_params_tmpl, v4l2_ctrl_hevc_slice_params, + TP_PROTO(const struct v4l2_ctrl_hevc_slice_params *s), + TP_ARGS(s) +); + +DEFINE_EVENT(v4l2_hevc_pred_weight_table_tmpl, v4l2_hevc_pred_weight_table, + TP_PROTO(const struct v4l2_hevc_pred_weight_table *p), + TP_ARGS(p) +); + +DEFINE_EVENT(v4l2_ctrl_hevc_scaling_matrix_tmpl, v4l2_ctrl_hevc_scaling_matrix, + TP_PROTO(const struct v4l2_ctrl_hevc_scaling_matrix *s), + TP_ARGS(s) +); + +DEFINE_EVENT(v4l2_ctrl_hevc_decode_params_tmpl, v4l2_ctrl_hevc_decode_params, + TP_PROTO(const struct v4l2_ctrl_hevc_decode_params *d), + TP_ARGS(d) +); + +DEFINE_EVENT(v4l2_hevc_dpb_entry_tmpl, v4l2_hevc_dpb_entry, + TP_PROTO(const struct v4l2_hevc_dpb_entry *e), + TP_ARGS(e) +); + +#endif + +#undef TRACE_INCLUDE_PATH +#undef TRACE_INCLUDE_FILE +#define TRACE_INCLUDE_PATH ../../drivers/media/test-drivers/visl +#define TRACE_INCLUDE_FILE visl-trace-hevc +#include <trace/define_trace.h> diff --git a/drivers/media/test-drivers/visl/visl-trace-mpeg2.h b/drivers/media/test-drivers/visl/visl-trace-mpeg2.h new file mode 100644 index 000000000000..ba6c65481194 --- /dev/null +++ b/drivers/media/test-drivers/visl/visl-trace-mpeg2.h @@ -0,0 +1,99 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#if !defined(_VISL_TRACE_MPEG2_H_) || defined(TRACE_HEADER_MULTI_READ) +#define _VISL_TRACE_MPEG2_H_ + +#include <linux/tracepoint.h> +#include "visl.h" + +#undef TRACE_SYSTEM +#define TRACE_SYSTEM visl_mpeg2_controls + +DECLARE_EVENT_CLASS(v4l2_ctrl_mpeg2_seq_tmpl, + TP_PROTO(const struct v4l2_ctrl_mpeg2_sequence *s), + TP_ARGS(s), + TP_STRUCT__entry(__field_struct(struct v4l2_ctrl_mpeg2_sequence, s)), + TP_fast_assign(__entry->s = *s;), + TP_printk("\nhorizontal_size %u\nvertical_size %u\nvbv_buffer_size %u\n" + "profile_and_level_indication %u\nchroma_format %u\nflags %s\n", + __entry->s.horizontal_size, + __entry->s.vertical_size, + __entry->s.vbv_buffer_size, + __entry->s.profile_and_level_indication, + __entry->s.chroma_format, + __print_flags(__entry->s.flags, "|", + {V4L2_MPEG2_SEQ_FLAG_PROGRESSIVE, "PROGRESSIVE"}) + ) +); + +DECLARE_EVENT_CLASS(v4l2_ctrl_mpeg2_pic_tmpl, + TP_PROTO(const struct v4l2_ctrl_mpeg2_picture *p), + TP_ARGS(p), + TP_STRUCT__entry(__field_struct(struct v4l2_ctrl_mpeg2_picture, p)), + TP_fast_assign(__entry->p = *p;), + TP_printk("\nbackward_ref_ts %llu\nforward_ref_ts %llu\nflags %s\nf_code {%s}\n" + "picture_coding_type: %u\npicture_structure %u\nintra_dc_precision %u\n", + __entry->p.backward_ref_ts, + __entry->p.forward_ref_ts, + __print_flags(__entry->p.flags, "|", + {V4L2_MPEG2_PIC_FLAG_TOP_FIELD_FIRST, "TOP_FIELD_FIRST"}, + {V4L2_MPEG2_PIC_FLAG_FRAME_PRED_DCT, "FRAME_PRED_DCT"}, + {V4L2_MPEG2_PIC_FLAG_CONCEALMENT_MV, "CONCEALMENT_MV"}, + {V4L2_MPEG2_PIC_FLAG_Q_SCALE_TYPE, "Q_SCALE_TYPE"}, + {V4L2_MPEG2_PIC_FLAG_INTRA_VLC, "INTA_VLC"}, + {V4L2_MPEG2_PIC_FLAG_ALT_SCAN, "ALT_SCAN"}, + {V4L2_MPEG2_PIC_FLAG_REPEAT_FIRST, "REPEAT_FIRST"}, + {V4L2_MPEG2_PIC_FLAG_PROGRESSIVE, "PROGRESSIVE"}), + __print_hex_dump("", DUMP_PREFIX_NONE, 32, 1, + __entry->p.f_code, + sizeof(__entry->p.f_code), + false), + __entry->p.picture_coding_type, + __entry->p.picture_structure, + __entry->p.intra_dc_precision + ) +); + +DECLARE_EVENT_CLASS(v4l2_ctrl_mpeg2_quant_tmpl, + TP_PROTO(const struct v4l2_ctrl_mpeg2_quantisation *q), + TP_ARGS(q), + TP_STRUCT__entry(__field_struct(struct v4l2_ctrl_mpeg2_quantisation, q)), + TP_fast_assign(__entry->q = *q;), + TP_printk("\nintra_quantiser_matrix %s\nnon_intra_quantiser_matrix %s\n" + "chroma_intra_quantiser_matrix %s\nchroma_non_intra_quantiser_matrix %s\n", + __print_array(__entry->q.intra_quantiser_matrix, + ARRAY_SIZE(__entry->q.intra_quantiser_matrix), + sizeof(__entry->q.intra_quantiser_matrix[0])), + __print_array(__entry->q.non_intra_quantiser_matrix, + ARRAY_SIZE(__entry->q.non_intra_quantiser_matrix), + sizeof(__entry->q.non_intra_quantiser_matrix[0])), + __print_array(__entry->q.chroma_intra_quantiser_matrix, + ARRAY_SIZE(__entry->q.chroma_intra_quantiser_matrix), + sizeof(__entry->q.chroma_intra_quantiser_matrix[0])), + __print_array(__entry->q.chroma_non_intra_quantiser_matrix, + ARRAY_SIZE(__entry->q.chroma_non_intra_quantiser_matrix), + sizeof(__entry->q.chroma_non_intra_quantiser_matrix[0])) + ) +) + +DEFINE_EVENT(v4l2_ctrl_mpeg2_seq_tmpl, v4l2_ctrl_mpeg2_sequence, + TP_PROTO(const struct v4l2_ctrl_mpeg2_sequence *s), + TP_ARGS(s) +); + +DEFINE_EVENT(v4l2_ctrl_mpeg2_pic_tmpl, v4l2_ctrl_mpeg2_picture, + TP_PROTO(const struct v4l2_ctrl_mpeg2_picture *p), + TP_ARGS(p) +); + +DEFINE_EVENT(v4l2_ctrl_mpeg2_quant_tmpl, v4l2_ctrl_mpeg2_quantisation, + TP_PROTO(const struct v4l2_ctrl_mpeg2_quantisation *q), + TP_ARGS(q) +); + +#endif + +#undef TRACE_INCLUDE_PATH +#undef TRACE_INCLUDE_FILE +#define TRACE_INCLUDE_PATH ../../drivers/media/test-drivers/visl +#define TRACE_INCLUDE_FILE visl-trace-mpeg2 +#include <trace/define_trace.h> diff --git a/drivers/media/test-drivers/visl/visl-trace-points.c b/drivers/media/test-drivers/visl/visl-trace-points.c new file mode 100644 index 000000000000..f7b866534f1e --- /dev/null +++ b/drivers/media/test-drivers/visl/visl-trace-points.c @@ -0,0 +1,10 @@ +// SPDX-License-Identifier: GPL-2.0 +#include "visl.h" + +#define CREATE_TRACE_POINTS +#include "visl-trace-fwht.h" +#include "visl-trace-mpeg2.h" +#include "visl-trace-vp8.h" +#include "visl-trace-vp9.h" +#include "visl-trace-h264.h" +#include "visl-trace-hevc.h" diff --git a/drivers/media/test-drivers/visl/visl-trace-vp8.h b/drivers/media/test-drivers/visl/visl-trace-vp8.h new file mode 100644 index 000000000000..dabe17d69ddc --- /dev/null +++ b/drivers/media/test-drivers/visl/visl-trace-vp8.h @@ -0,0 +1,156 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#if !defined(_VISL_TRACE_VP8_H_) || defined(TRACE_HEADER_MULTI_READ) +#define _VISL_TRACE_VP8_H_ + +#include <linux/tracepoint.h> +#include "visl.h" + +#undef TRACE_SYSTEM +#define TRACE_SYSTEM visl_vp8_controls + +DECLARE_EVENT_CLASS(v4l2_ctrl_vp8_entropy_tmpl, + TP_PROTO(const struct v4l2_ctrl_vp8_frame *f), + TP_ARGS(f), + TP_STRUCT__entry(__field_struct(struct v4l2_ctrl_vp8_frame, f)), + TP_fast_assign(__entry->f = *f;), + TP_printk("\nentropy.coeff_probs {%s}\n" + "entropy.y_mode_probs %s\n" + "entropy.uv_mode_probs %s\n" + "entropy.mv_probs {%s}", + __print_hex_dump("", DUMP_PREFIX_NONE, 32, 1, + __entry->f.entropy.coeff_probs, + sizeof(__entry->f.entropy.coeff_probs), + false), + __print_array(__entry->f.entropy.y_mode_probs, + ARRAY_SIZE(__entry->f.entropy.y_mode_probs), + sizeof(__entry->f.entropy.y_mode_probs[0])), + __print_array(__entry->f.entropy.uv_mode_probs, + ARRAY_SIZE(__entry->f.entropy.uv_mode_probs), + sizeof(__entry->f.entropy.uv_mode_probs[0])), + __print_hex_dump("", DUMP_PREFIX_NONE, 32, 1, + __entry->f.entropy.mv_probs, + sizeof(__entry->f.entropy.mv_probs), + false) + ) +) + +DECLARE_EVENT_CLASS(v4l2_ctrl_vp8_frame_tmpl, + TP_PROTO(const struct v4l2_ctrl_vp8_frame *f), + TP_ARGS(f), + TP_STRUCT__entry(__field_struct(struct v4l2_ctrl_vp8_frame, f)), + TP_fast_assign(__entry->f = *f;), + TP_printk("\nsegment.quant_update %s\n" + "segment.lf_update %s\n" + "segment.segment_probs %s\n" + "segment.flags %s\n" + "lf.ref_frm_delta %s\n" + "lf.mb_mode_delta %s\n" + "lf.sharpness_level %u\n" + "lf.level %u\n" + "lf.flags %s\n" + "quant.y_ac_qi %u\n" + "quant.y_dc_delta %d\n" + "quant.y2_dc_delta %d\n" + "quant.y2_ac_delta %d\n" + "quant.uv_dc_delta %d\n" + "quant.uv_ac_delta %d\n" + "coder_state.range %u\n" + "coder_state.value %u\n" + "coder_state.bit_count %u\n" + "width %u\n" + "height %u\n" + "horizontal_scale %u\n" + "vertical_scale %u\n" + "version %u\n" + "prob_skip_false %u\n" + "prob_intra %u\n" + "prob_last %u\n" + "prob_gf %u\n" + "num_dct_parts %u\n" + "first_part_size %u\n" + "first_part_header_bits %u\n" + "dct_part_sizes %s\n" + "last_frame_ts %llu\n" + "golden_frame_ts %llu\n" + "alt_frame_ts %llu\n" + "flags %s", + __print_array(__entry->f.segment.quant_update, + ARRAY_SIZE(__entry->f.segment.quant_update), + sizeof(__entry->f.segment.quant_update[0])), + __print_array(__entry->f.segment.lf_update, + ARRAY_SIZE(__entry->f.segment.lf_update), + sizeof(__entry->f.segment.lf_update[0])), + __print_array(__entry->f.segment.segment_probs, + ARRAY_SIZE(__entry->f.segment.segment_probs), + sizeof(__entry->f.segment.segment_probs[0])), + __print_flags(__entry->f.segment.flags, "|", + {V4L2_VP8_SEGMENT_FLAG_ENABLED, "SEGMENT_ENABLED"}, + {V4L2_VP8_SEGMENT_FLAG_UPDATE_MAP, "SEGMENT_UPDATE_MAP"}, + {V4L2_VP8_SEGMENT_FLAG_UPDATE_FEATURE_DATA, "SEGMENT_UPDATE_FEATURE_DATA"}, + {V4L2_VP8_SEGMENT_FLAG_DELTA_VALUE_MODE, "SEGMENT_DELTA_VALUE_MODE"}), + __print_array(__entry->f.lf.ref_frm_delta, + ARRAY_SIZE(__entry->f.lf.ref_frm_delta), + sizeof(__entry->f.lf.ref_frm_delta[0])), + __print_array(__entry->f.lf.mb_mode_delta, + ARRAY_SIZE(__entry->f.lf.mb_mode_delta), + sizeof(__entry->f.lf.mb_mode_delta[0])), + __entry->f.lf.sharpness_level, + __entry->f.lf.level, + __print_flags(__entry->f.lf.flags, "|", + {V4L2_VP8_LF_ADJ_ENABLE, "LF_ADJ_ENABLED"}, + {V4L2_VP8_LF_DELTA_UPDATE, "LF_DELTA_UPDATE"}, + {V4L2_VP8_LF_FILTER_TYPE_SIMPLE, "LF_FILTER_TYPE_SIMPLE"}), + __entry->f.quant.y_ac_qi, + __entry->f.quant.y_dc_delta, + __entry->f.quant.y2_dc_delta, + __entry->f.quant.y2_ac_delta, + __entry->f.quant.uv_dc_delta, + __entry->f.quant.uv_ac_delta, + __entry->f.coder_state.range, + __entry->f.coder_state.value, + __entry->f.coder_state.bit_count, + __entry->f.width, + __entry->f.height, + __entry->f.horizontal_scale, + __entry->f.vertical_scale, + __entry->f.version, + __entry->f.prob_skip_false, + __entry->f.prob_intra, + __entry->f.prob_last, + __entry->f.prob_gf, + __entry->f.num_dct_parts, + __entry->f.first_part_size, + __entry->f.first_part_header_bits, + __print_array(__entry->f.dct_part_sizes, + ARRAY_SIZE(__entry->f.dct_part_sizes), + sizeof(__entry->f.dct_part_sizes[0])), + __entry->f.last_frame_ts, + __entry->f.golden_frame_ts, + __entry->f.alt_frame_ts, + __print_flags(__entry->f.flags, "|", + {V4L2_VP8_FRAME_FLAG_KEY_FRAME, "KEY_FRAME"}, + {V4L2_VP8_FRAME_FLAG_EXPERIMENTAL, "EXPERIMENTAL"}, + {V4L2_VP8_FRAME_FLAG_SHOW_FRAME, "SHOW_FRAME"}, + {V4L2_VP8_FRAME_FLAG_MB_NO_SKIP_COEFF, "MB_NO_SKIP_COEFF"}, + {V4L2_VP8_FRAME_FLAG_SIGN_BIAS_GOLDEN, "SIGN_BIAS_GOLDEN"}, + {V4L2_VP8_FRAME_FLAG_SIGN_BIAS_ALT, "SIGN_BIAS_ALT"}) + ) +); + +DEFINE_EVENT(v4l2_ctrl_vp8_frame_tmpl, v4l2_ctrl_vp8_frame, + TP_PROTO(const struct v4l2_ctrl_vp8_frame *f), + TP_ARGS(f) +); + +DEFINE_EVENT(v4l2_ctrl_vp8_entropy_tmpl, v4l2_ctrl_vp8_entropy, + TP_PROTO(const struct v4l2_ctrl_vp8_frame *f), + TP_ARGS(f) +); + +#endif + +#undef TRACE_INCLUDE_PATH +#undef TRACE_INCLUDE_FILE +#define TRACE_INCLUDE_PATH ../../drivers/media/test-drivers/visl +#define TRACE_INCLUDE_FILE visl-trace-vp8 +#include <trace/define_trace.h> diff --git a/drivers/media/test-drivers/visl/visl-trace-vp9.h b/drivers/media/test-drivers/visl/visl-trace-vp9.h new file mode 100644 index 000000000000..362b92b07f93 --- /dev/null +++ b/drivers/media/test-drivers/visl/visl-trace-vp9.h @@ -0,0 +1,292 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#if !defined(_VISL_TRACE_VP9_H_) || defined(TRACE_HEADER_MULTI_READ) +#define _VISL_TRACE_VP9_H_ + +#include <linux/tracepoint.h> +#include "visl.h" + +#undef TRACE_SYSTEM +#define TRACE_SYSTEM visl_vp9_controls + +DECLARE_EVENT_CLASS(v4l2_ctrl_vp9_frame_tmpl, + TP_PROTO(const struct v4l2_ctrl_vp9_frame *f), + TP_ARGS(f), + TP_STRUCT__entry(__field_struct(struct v4l2_ctrl_vp9_frame, f)), + TP_fast_assign(__entry->f = *f;), + TP_printk("\nlf.ref_deltas %s\n" + "lf.mode_deltas %s\n" + "lf.level %u\n" + "lf.sharpness %u\n" + "lf.flags %s\n" + "quant.base_q_idx %u\n" + "quant.delta_q_y_dc %d\n" + "quant.delta_q_uv_dc %d\n" + "quant.delta_q_uv_ac %d\n" + "seg.feature_data {%s}\n" + "seg.feature_enabled %s\n" + "seg.tree_probs %s\n" + "seg.pred_probs %s\n" + "seg.flags %s\n" + "flags %s\n" + "compressed_header_size %u\n" + "uncompressed_header_size %u\n" + "frame_width_minus_1 %u\n" + "frame_height_minus_1 %u\n" + "render_width_minus_1 %u\n" + "render_height_minus_1 %u\n" + "last_frame_ts %llu\n" + "golden_frame_ts %llu\n" + "alt_frame_ts %llu\n" + "ref_frame_sign_bias %s\n" + "reset_frame_context %s\n" + "frame_context_idx %u\n" + "profile %u\n" + "bit_depth %u\n" + "interpolation_filter %s\n" + "tile_cols_log2 %u\n" + "tile_rows_log_2 %u\n" + "reference_mode %s\n", + __print_array(__entry->f.lf.ref_deltas, + ARRAY_SIZE(__entry->f.lf.ref_deltas), + sizeof(__entry->f.lf.ref_deltas[0])), + __print_array(__entry->f.lf.mode_deltas, + ARRAY_SIZE(__entry->f.lf.mode_deltas), + sizeof(__entry->f.lf.mode_deltas[0])), + __entry->f.lf.level, + __entry->f.lf.sharpness, + __print_flags(__entry->f.lf.flags, "|", + {V4L2_VP9_LOOP_FILTER_FLAG_DELTA_ENABLED, "DELTA_ENABLED"}, + {V4L2_VP9_LOOP_FILTER_FLAG_DELTA_UPDATE, "DELTA_UPDATE"}), + __entry->f.quant.base_q_idx, + __entry->f.quant.delta_q_y_dc, + __entry->f.quant.delta_q_uv_dc, + __entry->f.quant.delta_q_uv_ac, + __print_hex_dump("", DUMP_PREFIX_NONE, 32, 1, + __entry->f.seg.feature_data, + sizeof(__entry->f.seg.feature_data), + false), + __print_array(__entry->f.seg.feature_enabled, + ARRAY_SIZE(__entry->f.seg.feature_enabled), + sizeof(__entry->f.seg.feature_enabled[0])), + __print_array(__entry->f.seg.tree_probs, + ARRAY_SIZE(__entry->f.seg.tree_probs), + sizeof(__entry->f.seg.tree_probs[0])), + __print_array(__entry->f.seg.pred_probs, + ARRAY_SIZE(__entry->f.seg.pred_probs), + sizeof(__entry->f.seg.pred_probs[0])), + __print_flags(__entry->f.seg.flags, "|", + {V4L2_VP9_SEGMENTATION_FLAG_ENABLED, "ENABLED"}, + {V4L2_VP9_SEGMENTATION_FLAG_UPDATE_MAP, "UPDATE_MAP"}, + {V4L2_VP9_SEGMENTATION_FLAG_TEMPORAL_UPDATE, "TEMPORAL_UPDATE"}, + {V4L2_VP9_SEGMENTATION_FLAG_UPDATE_DATA, "UPDATE_DATA"}, + {V4L2_VP9_SEGMENTATION_FLAG_ABS_OR_DELTA_UPDATE, "ABS_OR_DELTA_UPDATE"}), + __print_flags(__entry->f.flags, "|", + {V4L2_VP9_FRAME_FLAG_KEY_FRAME, "KEY_FRAME"}, + {V4L2_VP9_FRAME_FLAG_SHOW_FRAME, "SHOW_FRAME"}, + {V4L2_VP9_FRAME_FLAG_ERROR_RESILIENT, "ERROR_RESILIENT"}, + {V4L2_VP9_FRAME_FLAG_INTRA_ONLY, "INTRA_ONLY"}, + {V4L2_VP9_FRAME_FLAG_ALLOW_HIGH_PREC_MV, "ALLOW_HIGH_PREC_MV"}, + {V4L2_VP9_FRAME_FLAG_REFRESH_FRAME_CTX, "REFRESH_FRAME_CTX"}, + {V4L2_VP9_FRAME_FLAG_PARALLEL_DEC_MODE, "PARALLEL_DEC_MODE"}, + {V4L2_VP9_FRAME_FLAG_X_SUBSAMPLING, "X_SUBSAMPLING"}, + {V4L2_VP9_FRAME_FLAG_Y_SUBSAMPLING, "Y_SUBSAMPLING"}, + {V4L2_VP9_FRAME_FLAG_COLOR_RANGE_FULL_SWING, "COLOR_RANGE_FULL_SWING"}), + __entry->f.compressed_header_size, + __entry->f.uncompressed_header_size, + __entry->f.frame_width_minus_1, + __entry->f.frame_height_minus_1, + __entry->f.render_width_minus_1, + __entry->f.render_height_minus_1, + __entry->f.last_frame_ts, + __entry->f.golden_frame_ts, + __entry->f.alt_frame_ts, + __print_symbolic(__entry->f.ref_frame_sign_bias, + {V4L2_VP9_SIGN_BIAS_LAST, "SIGN_BIAS_LAST"}, + {V4L2_VP9_SIGN_BIAS_GOLDEN, "SIGN_BIAS_GOLDEN"}, + {V4L2_VP9_SIGN_BIAS_ALT, "SIGN_BIAS_ALT"}), + __print_symbolic(__entry->f.reset_frame_context, + {V4L2_VP9_RESET_FRAME_CTX_NONE, "RESET_FRAME_CTX_NONE"}, + {V4L2_VP9_RESET_FRAME_CTX_SPEC, "RESET_FRAME_CTX_SPEC"}, + {V4L2_VP9_RESET_FRAME_CTX_ALL, "RESET_FRAME_CTX_ALL"}), + __entry->f.frame_context_idx, + __entry->f.profile, + __entry->f.bit_depth, + __print_symbolic(__entry->f.interpolation_filter, + {V4L2_VP9_INTERP_FILTER_EIGHTTAP, "INTERP_FILTER_EIGHTTAP"}, + {V4L2_VP9_INTERP_FILTER_EIGHTTAP_SMOOTH, "INTERP_FILTER_EIGHTTAP_SMOOTH"}, + {V4L2_VP9_INTERP_FILTER_EIGHTTAP_SHARP, "INTERP_FILTER_EIGHTTAP_SHARP"}, + {V4L2_VP9_INTERP_FILTER_BILINEAR, "INTERP_FILTER_BILINEAR"}, + {V4L2_VP9_INTERP_FILTER_SWITCHABLE, "INTERP_FILTER_SWITCHABLE"}), + __entry->f.tile_cols_log2, + __entry->f.tile_rows_log2, + __print_symbolic(__entry->f.reference_mode, + {V4L2_VP9_REFERENCE_MODE_SINGLE_REFERENCE, "REFERENCE_MODE_SINGLE_REFERENCE"}, + {V4L2_VP9_REFERENCE_MODE_COMPOUND_REFERENCE, "REFERENCE_MODE_COMPOUND_REFERENCE"}, + {V4L2_VP9_REFERENCE_MODE_SELECT, "REFERENCE_MODE_SELECT"})) +); + +DECLARE_EVENT_CLASS(v4l2_ctrl_vp9_compressed_hdr_tmpl, + TP_PROTO(const struct v4l2_ctrl_vp9_compressed_hdr *h), + TP_ARGS(h), + TP_STRUCT__entry(__field_struct(struct v4l2_ctrl_vp9_compressed_hdr, h)), + TP_fast_assign(__entry->h = *h;), + TP_printk("\ntx_mode %s\n" + "tx8 {%s}\n" + "tx16 {%s}\n" + "tx32 {%s}\n" + "skip %s\n" + "inter_mode {%s}\n" + "interp_filter {%s}\n" + "is_inter %s\n" + "comp_mode %s\n" + "single_ref {%s}\n" + "comp_ref %s\n" + "y_mode {%s}\n" + "uv_mode {%s}\n" + "partition {%s}\n", + __print_symbolic(__entry->h.tx_mode, + {V4L2_VP9_TX_MODE_ONLY_4X4, "TX_MODE_ONLY_4X4"}, + {V4L2_VP9_TX_MODE_ALLOW_8X8, "TX_MODE_ALLOW_8X8"}, + {V4L2_VP9_TX_MODE_ALLOW_16X16, "TX_MODE_ALLOW_16X16"}, + {V4L2_VP9_TX_MODE_ALLOW_32X32, "TX_MODE_ALLOW_32X32"}, + {V4L2_VP9_TX_MODE_SELECT, "TX_MODE_SELECT"}), + __print_hex_dump("", DUMP_PREFIX_NONE, 32, 1, + __entry->h.tx8, + sizeof(__entry->h.tx8), + false), + __print_hex_dump("", DUMP_PREFIX_NONE, 32, 1, + __entry->h.tx16, + sizeof(__entry->h.tx16), + false), + __print_hex_dump("", DUMP_PREFIX_NONE, 32, 1, + __entry->h.tx32, + sizeof(__entry->h.tx32), + false), + __print_array(__entry->h.skip, + ARRAY_SIZE(__entry->h.skip), + sizeof(__entry->h.skip[0])), + __print_hex_dump("", DUMP_PREFIX_NONE, 32, 1, + __entry->h.inter_mode, + sizeof(__entry->h.inter_mode), + false), + __print_hex_dump("", DUMP_PREFIX_NONE, 32, 1, + __entry->h.interp_filter, + sizeof(__entry->h.interp_filter), + false), + __print_array(__entry->h.is_inter, + ARRAY_SIZE(__entry->h.is_inter), + sizeof(__entry->h.is_inter[0])), + __print_array(__entry->h.comp_mode, + ARRAY_SIZE(__entry->h.comp_mode), + sizeof(__entry->h.comp_mode[0])), + __print_hex_dump("", DUMP_PREFIX_NONE, 32, 1, + __entry->h.single_ref, + sizeof(__entry->h.single_ref), + false), + __print_array(__entry->h.comp_ref, + ARRAY_SIZE(__entry->h.comp_ref), + sizeof(__entry->h.comp_ref[0])), + __print_hex_dump("", DUMP_PREFIX_NONE, 32, 1, + __entry->h.y_mode, + sizeof(__entry->h.y_mode), + false), + __print_hex_dump("", DUMP_PREFIX_NONE, 32, 1, + __entry->h.uv_mode, + sizeof(__entry->h.uv_mode), + false), + __print_hex_dump("", DUMP_PREFIX_NONE, 32, 1, + __entry->h.partition, + sizeof(__entry->h.partition), + false) + ) +); + +DECLARE_EVENT_CLASS(v4l2_ctrl_vp9_compressed_coef_tmpl, + TP_PROTO(const struct v4l2_ctrl_vp9_compressed_hdr *h), + TP_ARGS(h), + TP_STRUCT__entry(__field_struct(struct v4l2_ctrl_vp9_compressed_hdr, h)), + TP_fast_assign(__entry->h = *h;), + TP_printk("\n coef {%s}", + __print_hex_dump("", DUMP_PREFIX_NONE, 32, 1, + __entry->h.coef, + sizeof(__entry->h.coef), + false) + ) +); + +DECLARE_EVENT_CLASS(v4l2_vp9_mv_probs_tmpl, + TP_PROTO(const struct v4l2_vp9_mv_probs *p), + TP_ARGS(p), + TP_STRUCT__entry(__field_struct(struct v4l2_vp9_mv_probs, p)), + TP_fast_assign(__entry->p = *p;), + TP_printk("\n joint %s\n" + "sign %s\n" + "classes {%s}\n" + "class0_bit %s\n" + "bits {%s}\n" + "class0_fr {%s}\n" + "fr {%s}\n" + "class0_hp %s\n" + "hp %s\n", + __print_array(__entry->p.joint, + ARRAY_SIZE(__entry->p.joint), + sizeof(__entry->p.joint[0])), + __print_array(__entry->p.sign, + ARRAY_SIZE(__entry->p.sign), + sizeof(__entry->p.sign[0])), + __print_hex_dump("", DUMP_PREFIX_NONE, 32, 1, + __entry->p.classes, + sizeof(__entry->p.classes), + false), + __print_array(__entry->p.class0_bit, + ARRAY_SIZE(__entry->p.class0_bit), + sizeof(__entry->p.class0_bit[0])), + __print_hex_dump("", DUMP_PREFIX_NONE, 32, 1, + __entry->p.bits, + sizeof(__entry->p.bits), + false), + __print_hex_dump("", DUMP_PREFIX_NONE, 32, 1, + __entry->p.class0_fr, + sizeof(__entry->p.class0_fr), + false), + __print_hex_dump("", DUMP_PREFIX_NONE, 32, 1, + __entry->p.fr, + sizeof(__entry->p.fr), + false), + __print_array(__entry->p.class0_hp, + ARRAY_SIZE(__entry->p.class0_hp), + sizeof(__entry->p.class0_hp[0])), + __print_array(__entry->p.hp, + ARRAY_SIZE(__entry->p.hp), + sizeof(__entry->p.hp[0])) + ) +); + +DEFINE_EVENT(v4l2_ctrl_vp9_frame_tmpl, v4l2_ctrl_vp9_frame, + TP_PROTO(const struct v4l2_ctrl_vp9_frame *f), + TP_ARGS(f) +); + +DEFINE_EVENT(v4l2_ctrl_vp9_compressed_hdr_tmpl, v4l2_ctrl_vp9_compressed_hdr, + TP_PROTO(const struct v4l2_ctrl_vp9_compressed_hdr *h), + TP_ARGS(h) +); + +DEFINE_EVENT(v4l2_ctrl_vp9_compressed_coef_tmpl, v4l2_ctrl_vp9_compressed_coeff, + TP_PROTO(const struct v4l2_ctrl_vp9_compressed_hdr *h), + TP_ARGS(h) +); + + +DEFINE_EVENT(v4l2_vp9_mv_probs_tmpl, v4l2_vp9_mv_probs, + TP_PROTO(const struct v4l2_vp9_mv_probs *p), + TP_ARGS(p) +); + +#endif + +#undef TRACE_INCLUDE_PATH +#undef TRACE_INCLUDE_FILE +#define TRACE_INCLUDE_PATH ../../drivers/media/test-drivers/visl +#define TRACE_INCLUDE_FILE visl-trace-vp9 +#include <trace/define_trace.h> diff --git a/drivers/media/test-drivers/visl/visl-video.c b/drivers/media/test-drivers/visl/visl-video.c new file mode 100644 index 000000000000..b08664dfbe5f --- /dev/null +++ b/drivers/media/test-drivers/visl/visl-video.c @@ -0,0 +1,767 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Contains the driver implementation for the V4L2 stateless interface. + */ + +#include <linux/debugfs.h> +#include <linux/font.h> +#include <media/v4l2-event.h> +#include <media/v4l2-ioctl.h> +#include <media/videobuf2-vmalloc.h> +#include <media/videobuf2-v4l2.h> + +#include "visl-video.h" + +#include "visl.h" +#include "visl-debugfs.h" + +#define MIN_CODED_SZ (1024U * 256U) + +static void visl_set_current_codec(struct visl_ctx *ctx) +{ + u32 fourcc = ctx->coded_fmt.fmt.pix_mp.pixelformat; + + switch (fourcc) { + case V4L2_PIX_FMT_FWHT_STATELESS: + ctx->current_codec = VISL_CODEC_FWHT; + break; + case V4L2_PIX_FMT_MPEG2_SLICE: + ctx->current_codec = VISL_CODEC_MPEG2; + break; + case V4L2_PIX_FMT_VP8_FRAME: + ctx->current_codec = VISL_CODEC_VP8; + break; + case V4L2_PIX_FMT_VP9_FRAME: + ctx->current_codec = VISL_CODEC_VP9; + break; + case V4L2_PIX_FMT_H264_SLICE: + ctx->current_codec = VISL_CODEC_H264; + break; + case V4L2_PIX_FMT_HEVC_SLICE: + ctx->current_codec = VISL_CODEC_HEVC; + break; + default: + dprintk(ctx->dev, "Warning: unsupported fourcc: %d\n", fourcc); + ctx->current_codec = VISL_CODEC_NONE; + break; + } +} + +static void visl_print_fmt(struct visl_ctx *ctx, const struct v4l2_format *f) +{ + const struct v4l2_pix_format_mplane *pix_mp = &f->fmt.pix_mp; + u32 i; + + dprintk(ctx->dev, "width: %d\n", pix_mp->width); + dprintk(ctx->dev, "height: %d\n", pix_mp->height); + dprintk(ctx->dev, "pixelformat: %c%c%c%c\n", + pix_mp->pixelformat, + (pix_mp->pixelformat >> 8) & 0xff, + (pix_mp->pixelformat >> 16) & 0xff, + (pix_mp->pixelformat >> 24) & 0xff); + + dprintk(ctx->dev, "field: %d\n", pix_mp->field); + dprintk(ctx->dev, "colorspace: %d\n", pix_mp->colorspace); + dprintk(ctx->dev, "num_planes: %d\n", pix_mp->num_planes); + dprintk(ctx->dev, "flags: %d\n", pix_mp->flags); + dprintk(ctx->dev, "quantization: %d\n", pix_mp->quantization); + dprintk(ctx->dev, "xfer_func: %d\n", pix_mp->xfer_func); + + for (i = 0; i < pix_mp->num_planes; i++) { + dprintk(ctx->dev, + "plane[%d]: sizeimage: %d\n", i, pix_mp->plane_fmt[i].sizeimage); + dprintk(ctx->dev, + "plane[%d]: bytesperline: %d\n", i, pix_mp->plane_fmt[i].bytesperline); + } +} + +static int visl_tpg_init(struct visl_ctx *ctx) +{ + const struct font_desc *font; + const char *font_name = "VGA8x16"; + int ret; + u32 width = ctx->decoded_fmt.fmt.pix_mp.width; + u32 height = ctx->decoded_fmt.fmt.pix_mp.height; + struct v4l2_pix_format_mplane *f = &ctx->decoded_fmt.fmt.pix_mp; + + tpg_free(&ctx->tpg); + + font = find_font(font_name); + if (font) { + tpg_init(&ctx->tpg, width, height); + + ret = tpg_alloc(&ctx->tpg, width); + if (ret) + goto err_alloc; + + tpg_set_font(font->data); + ret = tpg_s_fourcc(&ctx->tpg, + f->pixelformat); + + if (!ret) + goto err_fourcc; + + tpg_reset_source(&ctx->tpg, width, height, f->field); + + tpg_s_pattern(&ctx->tpg, TPG_PAT_75_COLORBAR); + + tpg_s_field(&ctx->tpg, f->field, false); + tpg_s_colorspace(&ctx->tpg, f->colorspace); + tpg_s_ycbcr_enc(&ctx->tpg, f->ycbcr_enc); + tpg_s_quantization(&ctx->tpg, f->quantization); + tpg_s_xfer_func(&ctx->tpg, f->xfer_func); + } else { + v4l2_err(&ctx->dev->v4l2_dev, + "Font %s not found\n", font_name); + + return -EINVAL; + } + + dprintk(ctx->dev, "Initialized the V4L2 test pattern generator, w=%d, h=%d, max_w=%d\n", + width, height, width); + + return 0; +err_alloc: + return ret; +err_fourcc: + tpg_free(&ctx->tpg); + return ret; +} + +static const u32 visl_decoded_fmts[] = { + V4L2_PIX_FMT_NV12, + V4L2_PIX_FMT_YUV420, +}; + +const struct visl_coded_format_desc visl_coded_fmts[] = { + { + .pixelformat = V4L2_PIX_FMT_FWHT_STATELESS, + .frmsize = { + .min_width = 640, + .max_width = 4096, + .step_width = 1, + .min_height = 360, + .max_height = 2160, + .step_height = 1, + }, + .ctrls = &visl_fwht_ctrls, + .num_decoded_fmts = ARRAY_SIZE(visl_decoded_fmts), + .decoded_fmts = visl_decoded_fmts, + }, + { + .pixelformat = V4L2_PIX_FMT_MPEG2_SLICE, + .frmsize = { + .min_width = 16, + .max_width = 1920, + .step_width = 1, + .min_height = 16, + .max_height = 1152, + .step_height = 1, + }, + .ctrls = &visl_mpeg2_ctrls, + .num_decoded_fmts = ARRAY_SIZE(visl_decoded_fmts), + .decoded_fmts = visl_decoded_fmts, + }, + { + .pixelformat = V4L2_PIX_FMT_VP8_FRAME, + .frmsize = { + .min_width = 64, + .max_width = 16383, + .step_width = 1, + .min_height = 64, + .max_height = 16383, + .step_height = 1, + }, + .ctrls = &visl_vp8_ctrls, + .num_decoded_fmts = ARRAY_SIZE(visl_decoded_fmts), + .decoded_fmts = visl_decoded_fmts, + }, + { + .pixelformat = V4L2_PIX_FMT_VP9_FRAME, + .frmsize = { + .min_width = 64, + .max_width = 8192, + .step_width = 1, + .min_height = 64, + .max_height = 4352, + .step_height = 1, + }, + .ctrls = &visl_vp9_ctrls, + .num_decoded_fmts = ARRAY_SIZE(visl_decoded_fmts), + .decoded_fmts = visl_decoded_fmts, + }, + { + .pixelformat = V4L2_PIX_FMT_H264_SLICE, + .frmsize = { + .min_width = 64, + .max_width = 4096, + .step_width = 1, + .min_height = 64, + .max_height = 2304, + .step_height = 1, + }, + .ctrls = &visl_h264_ctrls, + .num_decoded_fmts = ARRAY_SIZE(visl_decoded_fmts), + .decoded_fmts = visl_decoded_fmts, + }, + { + .pixelformat = V4L2_PIX_FMT_HEVC_SLICE, + .frmsize = { + .min_width = 64, + .max_width = 4096, + .step_width = 1, + .min_height = 64, + .max_height = 2304, + .step_height = 1, + }, + .ctrls = &visl_hevc_ctrls, + .num_decoded_fmts = ARRAY_SIZE(visl_decoded_fmts), + .decoded_fmts = visl_decoded_fmts, + }, +}; + +const size_t num_coded_fmts = ARRAY_SIZE(visl_coded_fmts); + +static const struct visl_coded_format_desc* +visl_find_coded_fmt_desc(u32 fourcc) +{ + unsigned int i; + + for (i = 0; i < ARRAY_SIZE(visl_coded_fmts); i++) { + if (visl_coded_fmts[i].pixelformat == fourcc) + return &visl_coded_fmts[i]; + } + + return NULL; +} + +static void visl_init_fmt(struct v4l2_format *f, u32 fourcc) +{ memset(f, 0, sizeof(*f)); + f->fmt.pix_mp.pixelformat = fourcc; + f->fmt.pix_mp.field = V4L2_FIELD_NONE; + f->fmt.pix_mp.colorspace = V4L2_COLORSPACE_REC709; + f->fmt.pix_mp.ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT; + f->fmt.pix_mp.quantization = V4L2_QUANTIZATION_DEFAULT; + f->fmt.pix_mp.xfer_func = V4L2_XFER_FUNC_DEFAULT; +} + +static void visl_reset_coded_fmt(struct visl_ctx *ctx) +{ + struct v4l2_format *f = &ctx->coded_fmt; + struct v4l2_pix_format_mplane *pix_mp = &f->fmt.pix_mp; + + ctx->coded_format_desc = &visl_coded_fmts[0]; + visl_init_fmt(f, ctx->coded_format_desc->pixelformat); + + f->type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE; + f->fmt.pix_mp.width = ctx->coded_format_desc->frmsize.min_width; + f->fmt.pix_mp.height = ctx->coded_format_desc->frmsize.min_height; + + pix_mp->num_planes = 1; + pix_mp->plane_fmt[0].sizeimage = pix_mp->width * pix_mp->height * 8; + + dprintk(ctx->dev, "OUTPUT format was set to:\n"); + visl_print_fmt(ctx, &ctx->coded_fmt); + + visl_set_current_codec(ctx); +} + +static int visl_reset_decoded_fmt(struct visl_ctx *ctx) +{ + struct v4l2_format *f = &ctx->decoded_fmt; + u32 decoded_fmt = ctx->coded_format_desc[0].decoded_fmts[0]; + + visl_init_fmt(f, decoded_fmt); + + f->type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE; + + v4l2_fill_pixfmt_mp(&f->fmt.pix_mp, + ctx->coded_format_desc->decoded_fmts[0], + ctx->coded_fmt.fmt.pix_mp.width, + ctx->coded_fmt.fmt.pix_mp.height); + + dprintk(ctx->dev, "CAPTURE format was set to:\n"); + visl_print_fmt(ctx, &ctx->decoded_fmt); + + return visl_tpg_init(ctx); +} + +int visl_set_default_format(struct visl_ctx *ctx) +{ + visl_reset_coded_fmt(ctx); + return visl_reset_decoded_fmt(ctx); +} + +static struct visl_q_data *get_q_data(struct visl_ctx *ctx, + enum v4l2_buf_type type) +{ + switch (type) { + case V4L2_BUF_TYPE_VIDEO_OUTPUT: + case V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE: + return &ctx->q_data[V4L2_M2M_SRC]; + case V4L2_BUF_TYPE_VIDEO_CAPTURE: + case V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE: + return &ctx->q_data[V4L2_M2M_DST]; + default: + break; + } + return NULL; +} + +static int visl_querycap(struct file *file, void *priv, + struct v4l2_capability *cap) +{ + strscpy(cap->driver, VISL_NAME, sizeof(cap->driver)); + strscpy(cap->card, VISL_NAME, sizeof(cap->card)); + snprintf(cap->bus_info, sizeof(cap->bus_info), + "platform:%s", VISL_NAME); + + return 0; +} + +static int visl_enum_fmt_vid_cap(struct file *file, void *priv, + struct v4l2_fmtdesc *f) +{ + struct visl_ctx *ctx = visl_file_to_ctx(file); + + if (f->index >= ctx->coded_format_desc->num_decoded_fmts) + return -EINVAL; + + f->pixelformat = ctx->coded_format_desc->decoded_fmts[f->index]; + return 0; +} + +static int visl_enum_fmt_vid_out(struct file *file, void *priv, + struct v4l2_fmtdesc *f) +{ + if (f->index >= ARRAY_SIZE(visl_coded_fmts)) + return -EINVAL; + + f->pixelformat = visl_coded_fmts[f->index].pixelformat; + return 0; +} + +static int visl_g_fmt_vid_cap(struct file *file, void *priv, + struct v4l2_format *f) +{ + struct visl_ctx *ctx = visl_file_to_ctx(file); + *f = ctx->decoded_fmt; + + return 0; +} + +static int visl_g_fmt_vid_out(struct file *file, void *priv, + struct v4l2_format *f) +{ + struct visl_ctx *ctx = visl_file_to_ctx(file); + + *f = ctx->coded_fmt; + return 0; +} + +static int visl_try_fmt_vid_cap(struct file *file, void *priv, + struct v4l2_format *f) +{ + struct v4l2_pix_format_mplane *pix_mp = &f->fmt.pix_mp; + struct visl_ctx *ctx = visl_file_to_ctx(file); + const struct visl_coded_format_desc *coded_desc; + unsigned int i; + + coded_desc = ctx->coded_format_desc; + + for (i = 0; i < coded_desc->num_decoded_fmts; i++) { + if (coded_desc->decoded_fmts[i] == pix_mp->pixelformat) + break; + } + + if (i == coded_desc->num_decoded_fmts) + pix_mp->pixelformat = coded_desc->decoded_fmts[0]; + + v4l2_apply_frmsize_constraints(&pix_mp->width, + &pix_mp->height, + &coded_desc->frmsize); + + v4l2_fill_pixfmt_mp(pix_mp, pix_mp->pixelformat, + pix_mp->width, pix_mp->height); + + pix_mp->field = V4L2_FIELD_NONE; + + return 0; +} + +static int visl_try_fmt_vid_out(struct file *file, void *priv, + struct v4l2_format *f) +{ + struct v4l2_pix_format_mplane *pix_mp = &f->fmt.pix_mp; + const struct visl_coded_format_desc *coded_desc; + + coded_desc = visl_find_coded_fmt_desc(pix_mp->pixelformat); + if (!coded_desc) { + pix_mp->pixelformat = visl_coded_fmts[0].pixelformat; + coded_desc = &visl_coded_fmts[0]; + } + + v4l2_apply_frmsize_constraints(&pix_mp->width, + &pix_mp->height, + &coded_desc->frmsize); + + pix_mp->field = V4L2_FIELD_NONE; + pix_mp->num_planes = 1; + + if (pix_mp->plane_fmt[0].sizeimage == 0) + pix_mp->plane_fmt[0].sizeimage = max(MIN_CODED_SZ, + pix_mp->width * pix_mp->height * 3); + + return 0; +} + +static int visl_s_fmt_vid_out(struct file *file, void *priv, + struct v4l2_format *f) +{ + struct visl_ctx *ctx = visl_file_to_ctx(file); + struct v4l2_m2m_ctx *m2m_ctx = ctx->fh.m2m_ctx; + const struct visl_coded_format_desc *desc; + struct vb2_queue *peer_vq; + int ret; + + peer_vq = v4l2_m2m_get_vq(m2m_ctx, V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE); + if (vb2_is_busy(peer_vq)) + return -EBUSY; + + dprintk(ctx->dev, "Trying to set the OUTPUT format to:\n"); + visl_print_fmt(ctx, f); + + ret = visl_try_fmt_vid_out(file, priv, f); + if (ret) + return ret; + + desc = visl_find_coded_fmt_desc(f->fmt.pix_mp.pixelformat); + ctx->coded_format_desc = desc; + ctx->coded_fmt = *f; + + ret = visl_reset_decoded_fmt(ctx); + if (ret) + return ret; + + ctx->decoded_fmt.fmt.pix_mp.colorspace = f->fmt.pix_mp.colorspace; + ctx->decoded_fmt.fmt.pix_mp.xfer_func = f->fmt.pix_mp.xfer_func; + ctx->decoded_fmt.fmt.pix_mp.ycbcr_enc = f->fmt.pix_mp.ycbcr_enc; + ctx->decoded_fmt.fmt.pix_mp.quantization = f->fmt.pix_mp.quantization; + + dprintk(ctx->dev, "OUTPUT format was set to:\n"); + visl_print_fmt(ctx, &ctx->coded_fmt); + + visl_set_current_codec(ctx); + return 0; +} + +static int visl_s_fmt_vid_cap(struct file *file, void *priv, + struct v4l2_format *f) +{ + struct visl_ctx *ctx = visl_file_to_ctx(file); + int ret; + + dprintk(ctx->dev, "Trying to set the CAPTURE format to:\n"); + visl_print_fmt(ctx, f); + + ret = visl_try_fmt_vid_cap(file, priv, f); + if (ret) + return ret; + + ctx->decoded_fmt = *f; + + dprintk(ctx->dev, "CAPTURE format was set to:\n"); + visl_print_fmt(ctx, &ctx->decoded_fmt); + + visl_tpg_init(ctx); + return 0; +} + +static int visl_enum_framesizes(struct file *file, void *priv, + struct v4l2_frmsizeenum *fsize) +{ + const struct visl_coded_format_desc *fmt; + struct visl_ctx *ctx = visl_file_to_ctx(file); + + if (fsize->index != 0) + return -EINVAL; + + fmt = visl_find_coded_fmt_desc(fsize->pixel_format); + if (!fmt) { + dprintk(ctx->dev, + "Unsupported format for the OUTPUT queue: %d\n", + fsize->pixel_format); + + return -EINVAL; + } + + fsize->type = V4L2_FRMSIZE_TYPE_STEPWISE; + fsize->stepwise = fmt->frmsize; + return 0; +} + +const struct v4l2_ioctl_ops visl_ioctl_ops = { + .vidioc_querycap = visl_querycap, + .vidioc_enum_framesizes = visl_enum_framesizes, + + .vidioc_enum_fmt_vid_cap = visl_enum_fmt_vid_cap, + .vidioc_g_fmt_vid_cap_mplane = visl_g_fmt_vid_cap, + .vidioc_try_fmt_vid_cap_mplane = visl_try_fmt_vid_cap, + .vidioc_s_fmt_vid_cap_mplane = visl_s_fmt_vid_cap, + + .vidioc_enum_fmt_vid_out = visl_enum_fmt_vid_out, + .vidioc_g_fmt_vid_out_mplane = visl_g_fmt_vid_out, + .vidioc_try_fmt_vid_out_mplane = visl_try_fmt_vid_out, + .vidioc_s_fmt_vid_out_mplane = visl_s_fmt_vid_out, + + .vidioc_reqbufs = v4l2_m2m_ioctl_reqbufs, + .vidioc_querybuf = v4l2_m2m_ioctl_querybuf, + .vidioc_qbuf = v4l2_m2m_ioctl_qbuf, + .vidioc_dqbuf = v4l2_m2m_ioctl_dqbuf, + .vidioc_prepare_buf = v4l2_m2m_ioctl_prepare_buf, + .vidioc_create_bufs = v4l2_m2m_ioctl_create_bufs, + .vidioc_expbuf = v4l2_m2m_ioctl_expbuf, + + .vidioc_streamon = v4l2_m2m_ioctl_streamon, + .vidioc_streamoff = v4l2_m2m_ioctl_streamoff, + + .vidioc_subscribe_event = v4l2_ctrl_subscribe_event, + .vidioc_unsubscribe_event = v4l2_event_unsubscribe, +}; + +static int visl_queue_setup(struct vb2_queue *vq, + unsigned int *nbuffers, + unsigned int *num_planes, + unsigned int sizes[], + struct device *alloc_devs[]) +{ + struct visl_ctx *ctx = vb2_get_drv_priv(vq); + struct v4l2_format *f; + u32 i; + char *qname; + + if (V4L2_TYPE_IS_OUTPUT(vq->type)) { + f = &ctx->coded_fmt; + qname = "Output"; + } else { + f = &ctx->decoded_fmt; + qname = "Capture"; + } + + if (*num_planes) { + if (*num_planes != f->fmt.pix_mp.num_planes) + return -EINVAL; + + for (i = 0; i < f->fmt.pix_mp.num_planes; i++) { + if (sizes[i] < f->fmt.pix_mp.plane_fmt[i].sizeimage) + return -EINVAL; + } + } else { + *num_planes = f->fmt.pix_mp.num_planes; + for (i = 0; i < f->fmt.pix_mp.num_planes; i++) + sizes[i] = f->fmt.pix_mp.plane_fmt[i].sizeimage; + } + + dprintk(ctx->dev, "%s: %d buffer(s) requested, num_planes=%d.\n", + qname, *nbuffers, *num_planes); + + for (i = 0; i < f->fmt.pix_mp.num_planes; i++) + dprintk(ctx->dev, "plane[%d].sizeimage=%d\n", + i, f->fmt.pix_mp.plane_fmt[i].sizeimage); + + return 0; +} + +static void visl_queue_cleanup(struct vb2_queue *vq, u32 state) +{ + struct visl_ctx *ctx = vb2_get_drv_priv(vq); + struct vb2_v4l2_buffer *vbuf; + + dprintk(ctx->dev, "Cleaning up queues\n"); + for (;;) { + if (V4L2_TYPE_IS_OUTPUT(vq->type)) + vbuf = v4l2_m2m_src_buf_remove(ctx->fh.m2m_ctx); + else + vbuf = v4l2_m2m_dst_buf_remove(ctx->fh.m2m_ctx); + + if (!vbuf) + break; + + v4l2_ctrl_request_complete(vbuf->vb2_buf.req_obj.req, + &ctx->hdl); + dprintk(ctx->dev, "Marked request %p as complete\n", + vbuf->vb2_buf.req_obj.req); + + v4l2_m2m_buf_done(vbuf, state); + dprintk(ctx->dev, + "Marked buffer %llu as done, state is %d\n", + vbuf->vb2_buf.timestamp, + state); + } +} + +static int visl_buf_out_validate(struct vb2_buffer *vb) +{ + struct vb2_v4l2_buffer *vbuf = to_vb2_v4l2_buffer(vb); + + vbuf->field = V4L2_FIELD_NONE; + return 0; +} + +static int visl_buf_prepare(struct vb2_buffer *vb) +{ + struct vb2_queue *vq = vb->vb2_queue; + struct visl_ctx *ctx = vb2_get_drv_priv(vq); + u32 plane_sz = vb2_plane_size(vb, 0); + struct v4l2_pix_format *pix_fmt; + + if (V4L2_TYPE_IS_OUTPUT(vq->type)) { + pix_fmt = &ctx->coded_fmt.fmt.pix; + } else { + pix_fmt = &ctx->decoded_fmt.fmt.pix; + vb2_set_plane_payload(vb, 0, pix_fmt->sizeimage); + } + + if (plane_sz < pix_fmt->sizeimage) { + v4l2_err(&ctx->dev->v4l2_dev, "plane[0] size is %d, sizeimage is %d\n", + plane_sz, pix_fmt->sizeimage); + return -EINVAL; + } + + return 0; +} + +static int visl_start_streaming(struct vb2_queue *vq, unsigned int count) +{ + struct visl_ctx *ctx = vb2_get_drv_priv(vq); + struct visl_q_data *q_data = get_q_data(ctx, vq->type); + int rc = 0; + + if (!q_data) { + rc = -EINVAL; + goto err; + } + + q_data->sequence = 0; + + if (V4L2_TYPE_IS_CAPTURE(vq->type)) { + ctx->capture_streamon_jiffies = get_jiffies_64(); + return 0; + } + + if (WARN_ON(!ctx->coded_format_desc)) { + rc = -EINVAL; + goto err; + } + + return 0; + +err: + visl_queue_cleanup(vq, VB2_BUF_STATE_QUEUED); + return rc; +} + +static void visl_stop_streaming(struct vb2_queue *vq) +{ + struct visl_ctx *ctx = vb2_get_drv_priv(vq); + + dprintk(ctx->dev, "Stop streaming\n"); + visl_queue_cleanup(vq, VB2_BUF_STATE_ERROR); + + if (!keep_bitstream_buffers) + visl_debugfs_clear_bitstream(ctx->dev); +} + +static void visl_buf_queue(struct vb2_buffer *vb) +{ + struct vb2_v4l2_buffer *vbuf = to_vb2_v4l2_buffer(vb); + struct visl_ctx *ctx = vb2_get_drv_priv(vb->vb2_queue); + + v4l2_m2m_buf_queue(ctx->fh.m2m_ctx, vbuf); +} + +static void visl_buf_request_complete(struct vb2_buffer *vb) +{ + struct visl_ctx *ctx = vb2_get_drv_priv(vb->vb2_queue); + + v4l2_ctrl_request_complete(vb->req_obj.req, &ctx->hdl); +} + +const struct vb2_ops visl_qops = { + .queue_setup = visl_queue_setup, + .buf_out_validate = visl_buf_out_validate, + .buf_prepare = visl_buf_prepare, + .buf_queue = visl_buf_queue, + .start_streaming = visl_start_streaming, + .stop_streaming = visl_stop_streaming, + .wait_prepare = vb2_ops_wait_prepare, + .wait_finish = vb2_ops_wait_finish, + .buf_request_complete = visl_buf_request_complete, +}; + +int visl_queue_init(void *priv, struct vb2_queue *src_vq, + struct vb2_queue *dst_vq) +{ + struct visl_ctx *ctx = priv; + int ret; + + src_vq->type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE; + src_vq->io_modes = VB2_MMAP | VB2_DMABUF; + src_vq->drv_priv = ctx; + src_vq->buf_struct_size = sizeof(struct v4l2_m2m_buffer); + src_vq->ops = &visl_qops; + src_vq->mem_ops = &vb2_vmalloc_memops; + src_vq->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_COPY; + src_vq->lock = &ctx->vb_mutex; + src_vq->supports_requests = true; + src_vq->subsystem_flags |= VB2_V4L2_FL_SUPPORTS_M2M_HOLD_CAPTURE_BUF; + + ret = vb2_queue_init(src_vq); + if (ret) + return ret; + + dst_vq->type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE; + dst_vq->io_modes = VB2_MMAP | VB2_DMABUF; + dst_vq->drv_priv = ctx; + dst_vq->buf_struct_size = sizeof(struct v4l2_m2m_buffer); + dst_vq->ops = &visl_qops; + dst_vq->mem_ops = &vb2_vmalloc_memops; + dst_vq->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_COPY; + dst_vq->lock = &ctx->vb_mutex; + + return vb2_queue_init(dst_vq); +} + +int visl_request_validate(struct media_request *req) +{ + struct media_request_object *obj; + struct visl_ctx *ctx = NULL; + unsigned int count; + + list_for_each_entry(obj, &req->objects, list) { + struct vb2_buffer *vb; + + if (vb2_request_object_is_buffer(obj)) { + vb = container_of(obj, struct vb2_buffer, req_obj); + ctx = vb2_get_drv_priv(vb->vb2_queue); + + break; + } + } + + if (!ctx) + return -ENOENT; + + count = vb2_request_buffer_cnt(req); + if (!count) { + v4l2_err(&ctx->dev->v4l2_dev, + "No buffer was provided with the request\n"); + return -ENOENT; + } else if (count > 1) { + v4l2_err(&ctx->dev->v4l2_dev, + "More than one buffer was provided with the request\n"); + return -EINVAL; + } + + return vb2_request_validate(req); +} diff --git a/drivers/media/test-drivers/visl/visl-video.h b/drivers/media/test-drivers/visl/visl-video.h new file mode 100644 index 000000000000..27ad70a558db --- /dev/null +++ b/drivers/media/test-drivers/visl/visl-video.h @@ -0,0 +1,27 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* + * Contains the driver implementation for the V4L2 stateless interface. + */ + +#ifndef _VISL_VIDEO_H_ +#define _VISL_VIDEO_H_ +#include <media/v4l2-mem2mem.h> + +#include "visl.h" + +extern const struct v4l2_ioctl_ops visl_ioctl_ops; + +extern const struct visl_ctrls visl_fwht_ctrls; +extern const struct visl_ctrls visl_mpeg2_ctrls; +extern const struct visl_ctrls visl_vp8_ctrls; +extern const struct visl_ctrls visl_vp9_ctrls; +extern const struct visl_ctrls visl_h264_ctrls; +extern const struct visl_ctrls visl_hevc_ctrls; + +int visl_queue_init(void *priv, struct vb2_queue *src_vq, + struct vb2_queue *dst_vq); + +int visl_set_default_format(struct visl_ctx *ctx); +int visl_request_validate(struct media_request *req); + +#endif /* _VISL_VIDEO_H_ */ diff --git a/drivers/media/test-drivers/visl/visl.h b/drivers/media/test-drivers/visl/visl.h new file mode 100644 index 000000000000..31639f2e593d --- /dev/null +++ b/drivers/media/test-drivers/visl/visl.h @@ -0,0 +1,176 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* + * A virtual stateless device for stateless uAPI development purposes. + * + * This tool's objective is to help the development and testing of userspace + * applications that use the V4L2 stateless API to decode media. + * + * A userspace implementation can use visl to run a decoding loop even when no + * hardware is available or when the kernel uAPI for the codec has not been + * upstreamed yet. This can reveal bugs at an early stage. + * + * This driver can also trace the contents of the V4L2 controls submitted to it. + * It can also dump the contents of the vb2 buffers through a debugfs + * interface. This is in many ways similar to the tracing infrastructure + * available for other popular encode/decode APIs out there and can help develop + * a userspace application by using another (working) one as a reference. + * + * Note that no actual decoding of video frames is performed by visl. The V4L2 + * test pattern generator is used to write various debug information to the + * capture buffers instead. + * + * Copyright (C) 2022 Collabora, Ltd. + * + * Based on the vim2m driver, that is: + * + * Copyright (c) 2009-2010 Samsung Electronics Co., Ltd. + * Pawel Osciak, <[email protected]> + * Marek Szyprowski, <[email protected]> + * + * Based on the vicodec driver, that is: + * + * Copyright 2018 Cisco Systems, Inc. and/or its affiliates. All rights reserved. + * + * Based on the Cedrus VPU driver, that is: + * + * Copyright (C) 2016 Florent Revest <[email protected]> + * Copyright (C) 2018 Paul Kocialkowski <[email protected]> + * Copyright (C) 2018 Bootlin + */ + +#ifndef _VISL_H_ +#define _VISL_H_ + +#include <linux/debugfs.h> +#include <linux/list.h> + +#include <media/v4l2-ctrls.h> +#include <media/v4l2-device.h> +#include <media/tpg/v4l2-tpg.h> + +#define VISL_NAME "visl" +#define VISL_M2M_NQUEUES 2 + +#define TPG_STR_BUF_SZ 2048 + +extern unsigned int visl_transtime_ms; + +struct visl_ctrls { + const struct visl_ctrl_desc *ctrls; + unsigned int num_ctrls; +}; + +struct visl_coded_format_desc { + u32 pixelformat; + struct v4l2_frmsize_stepwise frmsize; + const struct visl_ctrls *ctrls; + unsigned int num_decoded_fmts; + const u32 *decoded_fmts; +}; + +extern const struct visl_coded_format_desc visl_coded_fmts[]; +extern const size_t num_coded_fmts; + +enum { + V4L2_M2M_SRC = 0, + V4L2_M2M_DST = 1, +}; + +extern unsigned int visl_debug; +#define dprintk(dev, fmt, arg...) \ + v4l2_dbg(1, visl_debug, &(dev)->v4l2_dev, "%s: " fmt, __func__, ## arg) + +extern int visl_dprintk_frame_start; +extern unsigned int visl_dprintk_nframes; +extern bool keep_bitstream_buffers; +extern int bitstream_trace_frame_start; +extern unsigned int bitstream_trace_nframes; + +#define frame_dprintk(dev, current, fmt, arg...) \ + do { \ + if (visl_dprintk_frame_start > -1 && \ + (current) >= visl_dprintk_frame_start && \ + (current) < visl_dprintk_frame_start + visl_dprintk_nframes) \ + dprintk(dev, fmt, ## arg); \ + } while (0) \ + +struct visl_q_data { + unsigned int sequence; +}; + +struct visl_dev { + struct v4l2_device v4l2_dev; + struct video_device vfd; +#ifdef CONFIG_MEDIA_CONTROLLER + struct media_device mdev; +#endif + + struct mutex dev_mutex; + + struct v4l2_m2m_dev *m2m_dev; + +#ifdef CONFIG_VISL_DEBUGFS + struct dentry *debugfs_root; + struct dentry *bitstream_debugfs; + struct list_head bitstream_blobs; + + /* Protects the "blob" list */ + struct mutex bitstream_lock; +#endif +}; + +enum visl_codec { + VISL_CODEC_NONE, + VISL_CODEC_FWHT, + VISL_CODEC_MPEG2, + VISL_CODEC_VP8, + VISL_CODEC_VP9, + VISL_CODEC_H264, + VISL_CODEC_HEVC, +}; + +struct visl_blob { + struct list_head list; + struct dentry *dentry; + struct debugfs_blob_wrapper blob; +}; + +struct visl_ctx { + struct v4l2_fh fh; + struct visl_dev *dev; + struct v4l2_ctrl_handler hdl; + + struct mutex vb_mutex; + + struct visl_q_data q_data[VISL_M2M_NQUEUES]; + enum visl_codec current_codec; + + const struct visl_coded_format_desc *coded_format_desc; + + struct v4l2_format coded_fmt; + struct v4l2_format decoded_fmt; + + struct tpg_data tpg; + u64 capture_streamon_jiffies; + char *tpg_str_buf; +}; + +struct visl_ctrl_desc { + struct v4l2_ctrl_config cfg; +}; + +static inline struct visl_ctx *visl_file_to_ctx(struct file *file) +{ + return container_of(file->private_data, struct visl_ctx, fh); +} + +static inline struct visl_ctx *visl_v4l2fh_to_ctx(struct v4l2_fh *v4l2_fh) +{ + return container_of(v4l2_fh, struct visl_ctx, fh); +} + +void *visl_find_control_data(struct visl_ctx *ctx, u32 id); +struct v4l2_ctrl *visl_find_control(struct visl_ctx *ctx, u32 id); +u32 visl_control_num_elems(struct visl_ctx *ctx, u32 id); + +#endif /* _VISL_H_ */ diff --git a/drivers/media/test-drivers/vivid/vivid-ctrls.c b/drivers/media/test-drivers/vivid/vivid-ctrls.c index 92b1a7598470..f2b20e25a7a4 100644 --- a/drivers/media/test-drivers/vivid/vivid-ctrls.c +++ b/drivers/media/test-drivers/vivid/vivid-ctrls.c @@ -36,6 +36,8 @@ #define VIVID_CID_RO_INTEGER (VIVID_CID_CUSTOM_BASE + 12) #define VIVID_CID_U32_DYN_ARRAY (VIVID_CID_CUSTOM_BASE + 13) #define VIVID_CID_U8_PIXEL_ARRAY (VIVID_CID_CUSTOM_BASE + 14) +#define VIVID_CID_S32_ARRAY (VIVID_CID_CUSTOM_BASE + 15) +#define VIVID_CID_S64_ARRAY (VIVID_CID_CUSTOM_BASE + 16) #define VIVID_CID_VIVID_BASE (0x00f00000 | 0xf000) #define VIVID_CID_VIVID_CLASS (0x00f00000 | 1) @@ -241,6 +243,30 @@ static const struct v4l2_ctrl_config vivid_ctrl_u8_pixel_array = { .dims = { 640 / PIXEL_ARRAY_DIV, 360 / PIXEL_ARRAY_DIV }, }; +static const struct v4l2_ctrl_config vivid_ctrl_s32_array = { + .ops = &vivid_user_gen_ctrl_ops, + .id = VIVID_CID_S32_ARRAY, + .name = "S32 2 Element Array", + .type = V4L2_CTRL_TYPE_INTEGER, + .def = 2, + .min = -10, + .max = 10, + .step = 1, + .dims = { 2 }, +}; + +static const struct v4l2_ctrl_config vivid_ctrl_s64_array = { + .ops = &vivid_user_gen_ctrl_ops, + .id = VIVID_CID_S64_ARRAY, + .name = "S64 5 Element Array", + .type = V4L2_CTRL_TYPE_INTEGER64, + .def = 4, + .min = -10, + .max = 10, + .step = 1, + .dims = { 5 }, +}; + static const char * const vivid_ctrl_menu_strings[] = { "Menu Item 0 (Skipped)", "Menu Item 1", @@ -1656,6 +1682,8 @@ int vivid_create_controls(struct vivid_dev *dev, bool show_ccs_cap, v4l2_ctrl_new_custom(hdl_user_gen, &vivid_ctrl_u16_matrix, NULL); v4l2_ctrl_new_custom(hdl_user_gen, &vivid_ctrl_u8_4d_array, NULL); dev->pixel_array = v4l2_ctrl_new_custom(hdl_user_gen, &vivid_ctrl_u8_pixel_array, NULL); + v4l2_ctrl_new_custom(hdl_user_gen, &vivid_ctrl_s32_array, NULL); + v4l2_ctrl_new_custom(hdl_user_gen, &vivid_ctrl_s64_array, NULL); if (dev->has_vid_cap) { /* Image Processing Controls */ diff --git a/drivers/media/test-drivers/vivid/vivid-vbi-gen.c b/drivers/media/test-drivers/vivid/vivid-vbi-gen.c index a141369a7a63..70a4024d461e 100644 --- a/drivers/media/test-drivers/vivid/vivid-vbi-gen.c +++ b/drivers/media/test-drivers/vivid/vivid-vbi-gen.c @@ -194,7 +194,6 @@ static void vivid_vbi_gen_set_time_of_day(u8 *packet) for (checksum = i = 0; i <= 8; i++) checksum += packet[i] & 0x7f; packet[9] = calc_parity(0x100 - checksum); - checksum = 0; packet[10] = calc_parity(0x07); packet[11] = calc_parity(0x04); if (sys_tz.tz_minuteswest >= 0) diff --git a/drivers/media/tuners/mxl5005s.c b/drivers/media/tuners/mxl5005s.c index ab4c43df9d18..3a509038c8df 100644 --- a/drivers/media/tuners/mxl5005s.c +++ b/drivers/media/tuners/mxl5005s.c @@ -3637,7 +3637,7 @@ static u16 MXL_GetCHRegister_ZeroIF(struct dvb_frontend *fe, u8 *RegNum, u16 status = 0; int i; - u8 RegAddr[] = {43, 136}; + static const u8 RegAddr[] = {43, 136}; *count = ARRAY_SIZE(RegAddr); diff --git a/drivers/media/usb/au0828/au0828-vbi.c b/drivers/media/usb/au0828/au0828-vbi.c index 97f5e8733c2a..b0333637b747 100644 --- a/drivers/media/usb/au0828/au0828-vbi.c +++ b/drivers/media/usb/au0828/au0828-vbi.c @@ -14,6 +14,7 @@ #include <linux/module.h> #include <linux/init.h> #include <linux/slab.h> +#include <media/v4l2-mc.h> /* ------------------------------------------------------------------ */ @@ -70,6 +71,7 @@ const struct vb2_ops au0828_vbi_qops = { .queue_setup = vbi_queue_setup, .buf_prepare = vbi_buffer_prepare, .buf_queue = vbi_buffer_queue, + .prepare_streaming = v4l_vb2q_enable_media_source, .start_streaming = au0828_start_analog_streaming, .stop_streaming = au0828_stop_vbi_streaming, .wait_prepare = vb2_ops_wait_prepare, diff --git a/drivers/media/usb/au0828/au0828-video.c b/drivers/media/usb/au0828/au0828-video.c index eb303e94cceb..fd9fc43d47e0 100644 --- a/drivers/media/usb/au0828/au0828-video.c +++ b/drivers/media/usb/au0828/au0828-video.c @@ -915,6 +915,7 @@ static const struct vb2_ops au0828_video_qops = { .queue_setup = queue_setup, .buf_prepare = buffer_prepare, .buf_queue = buffer_queue, + .prepare_streaming = v4l_vb2q_enable_media_source, .start_streaming = au0828_start_analog_streaming, .stop_streaming = au0828_stop_streaming, .wait_prepare = vb2_ops_wait_prepare, diff --git a/drivers/media/v4l2-core/v4l2-ctrls-core.c b/drivers/media/v4l2-core/v4l2-ctrls-core.c index 0dab1d7b90f0..29169170880a 100644 --- a/drivers/media/v4l2-core/v4l2-ctrls-core.c +++ b/drivers/media/v4l2-core/v4l2-ctrls-core.c @@ -1827,7 +1827,7 @@ struct v4l2_ctrl *v4l2_ctrl_new_std_menu(struct v4l2_ctrl_handler *hdl, else if (type == V4L2_CTRL_TYPE_INTEGER_MENU) qmenu_int = v4l2_ctrl_get_int_menu(id, &qmenu_int_len); - if ((!qmenu && !qmenu_int) || (qmenu_int && max > qmenu_int_len)) { + if ((!qmenu && !qmenu_int) || (qmenu_int && max >= qmenu_int_len)) { handler_set_err(hdl, -EINVAL); return NULL; } diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c b/drivers/media/v4l2-core/v4l2-ioctl.c index 8cb4b976064e..de9efa94fec9 100644 --- a/drivers/media/v4l2-core/v4l2-ioctl.c +++ b/drivers/media/v4l2-core/v4l2-ioctl.c @@ -1347,23 +1347,23 @@ static void v4l_fill_fmtdesc(struct v4l2_fmtdesc *fmt) case V4L2_PIX_FMT_YUV420: descr = "Planar YUV 4:2:0"; break; case V4L2_PIX_FMT_HI240: descr = "8-bit Dithered RGB (BTTV)"; break; case V4L2_PIX_FMT_M420: descr = "YUV 4:2:0 (M420)"; break; - case V4L2_PIX_FMT_NV12: descr = "Y/CbCr 4:2:0"; break; - case V4L2_PIX_FMT_NV21: descr = "Y/CrCb 4:2:0"; break; - case V4L2_PIX_FMT_NV16: descr = "Y/CbCr 4:2:2"; break; - case V4L2_PIX_FMT_NV61: descr = "Y/CrCb 4:2:2"; break; - case V4L2_PIX_FMT_NV24: descr = "Y/CbCr 4:4:4"; break; - case V4L2_PIX_FMT_NV42: descr = "Y/CrCb 4:4:4"; break; - case V4L2_PIX_FMT_P010: descr = "10-bit Y/CbCr 4:2:0"; break; - case V4L2_PIX_FMT_NV12_4L4: descr = "Y/CbCr 4:2:0 (4x4 Linear)"; break; - case V4L2_PIX_FMT_NV12_16L16: descr = "Y/CbCr 4:2:0 (16x16 Linear)"; break; - case V4L2_PIX_FMT_NV12_32L32: descr = "Y/CbCr 4:2:0 (32x32 Linear)"; break; - case V4L2_PIX_FMT_P010_4L4: descr = "10-bit Y/CbCr 4:2:0 (4x4 Linear)"; break; - case V4L2_PIX_FMT_NV12M: descr = "Y/CbCr 4:2:0 (N-C)"; break; - case V4L2_PIX_FMT_NV21M: descr = "Y/CrCb 4:2:0 (N-C)"; break; - case V4L2_PIX_FMT_NV16M: descr = "Y/CbCr 4:2:2 (N-C)"; break; - case V4L2_PIX_FMT_NV61M: descr = "Y/CrCb 4:2:2 (N-C)"; break; - case V4L2_PIX_FMT_NV12MT: descr = "Y/CbCr 4:2:0 (64x32 MB, N-C)"; break; - case V4L2_PIX_FMT_NV12MT_16X16: descr = "Y/CbCr 4:2:0 (16x16 MB, N-C)"; break; + case V4L2_PIX_FMT_NV12: descr = "Y/UV 4:2:0"; break; + case V4L2_PIX_FMT_NV21: descr = "Y/VU 4:2:0"; break; + case V4L2_PIX_FMT_NV16: descr = "Y/UV 4:2:2"; break; + case V4L2_PIX_FMT_NV61: descr = "Y/VU 4:2:2"; break; + case V4L2_PIX_FMT_NV24: descr = "Y/UV 4:4:4"; break; + case V4L2_PIX_FMT_NV42: descr = "Y/VU 4:4:4"; break; + case V4L2_PIX_FMT_P010: descr = "10-bit Y/UV 4:2:0"; break; + case V4L2_PIX_FMT_NV12_4L4: descr = "Y/UV 4:2:0 (4x4 Linear)"; break; + case V4L2_PIX_FMT_NV12_16L16: descr = "Y/UV 4:2:0 (16x16 Linear)"; break; + case V4L2_PIX_FMT_NV12_32L32: descr = "Y/UV 4:2:0 (32x32 Linear)"; break; + case V4L2_PIX_FMT_P010_4L4: descr = "10-bit Y/UV 4:2:0 (4x4 Linear)"; break; + case V4L2_PIX_FMT_NV12M: descr = "Y/UV 4:2:0 (N-C)"; break; + case V4L2_PIX_FMT_NV21M: descr = "Y/VU 4:2:0 (N-C)"; break; + case V4L2_PIX_FMT_NV16M: descr = "Y/UV 4:2:2 (N-C)"; break; + case V4L2_PIX_FMT_NV61M: descr = "Y/VU 4:2:2 (N-C)"; break; + case V4L2_PIX_FMT_NV12MT: descr = "Y/UV 4:2:0 (64x32 MB, N-C)"; break; + case V4L2_PIX_FMT_NV12MT_16X16: descr = "Y/UV 4:2:0 (16x16 MB, N-C)"; break; case V4L2_PIX_FMT_YUV420M: descr = "Planar YUV 4:2:0 (N-C)"; break; case V4L2_PIX_FMT_YVU420M: descr = "Planar YVU 4:2:0 (N-C)"; break; case V4L2_PIX_FMT_YUV422M: descr = "Planar YUV 4:2:2 (N-C)"; break; diff --git a/drivers/media/v4l2-core/v4l2-subdev.c b/drivers/media/v4l2-core/v4l2-subdev.c index 8a4ca2bd1584..4988a25bd8f4 100644 --- a/drivers/media/v4l2-core/v4l2-subdev.c +++ b/drivers/media/v4l2-core/v4l2-subdev.c @@ -860,7 +860,7 @@ int v4l2_subdev_get_fwnode_pad_1_to_1(struct media_entity *entity, fwnode = fwnode_graph_get_port_parent(endpoint->local_fwnode); fwnode_handle_put(fwnode); - if (dev_fwnode(sd->dev) == fwnode) + if (device_match_fwnode(sd->dev, fwnode)) return endpoint->port; return -ENXIO; diff --git a/drivers/staging/media/Kconfig b/drivers/staging/media/Kconfig index d4f03b203ae5..b79f93684c4f 100644 --- a/drivers/staging/media/Kconfig +++ b/drivers/staging/media/Kconfig @@ -51,6 +51,7 @@ menuconfig STAGING_MEDIA_DEPRECATED If in doubt, say N here. if STAGING_MEDIA_DEPRECATED +source "drivers/staging/media/deprecated/atmel/Kconfig" source "drivers/staging/media/deprecated/cpia2/Kconfig" source "drivers/staging/media/deprecated/fsl-viu/Kconfig" source "drivers/staging/media/deprecated/meye/Kconfig" diff --git a/drivers/staging/media/Makefile b/drivers/staging/media/Makefile index a387692b84f2..54bbdd4b0d08 100644 --- a/drivers/staging/media/Makefile +++ b/drivers/staging/media/Makefile @@ -1,4 +1,5 @@ # SPDX-License-Identifier: GPL-2.0 +obj-$(CONFIG_VIDEO_ATMEL_ISC_BASE) += deprecated/atmel/ obj-$(CONFIG_INTEL_ATOMISP) += atomisp/ obj-$(CONFIG_VIDEO_CPIA2) += deprecated/cpia2/ obj-$(CONFIG_VIDEO_IMX_MEDIA) += imx/ diff --git a/drivers/staging/media/atomisp/pci/atomisp_cmd.c b/drivers/staging/media/atomisp/pci/atomisp_cmd.c index c72d0e344671..eeb66b3b79ab 100644 --- a/drivers/staging/media/atomisp/pci/atomisp_cmd.c +++ b/drivers/staging/media/atomisp/pci/atomisp_cmd.c @@ -3288,7 +3288,7 @@ int atomisp_css_cp_dvs2_coefs(struct atomisp_sub_device *asd, if (!IS_ISP2401) { if (sizeof(*cur) != sizeof(coefs->grid) || memcmp(&coefs->grid, cur, sizeof(coefs->grid))) { - dev_err(asd->isp->dev, "dvs grid mis-match!\n"); + dev_err(asd->isp->dev, "dvs grid mismatch!\n"); /* If the grid info in the argument differs from the current grid info, we tell the caller to reset the grid size and try again. */ @@ -3344,7 +3344,7 @@ int atomisp_css_cp_dvs2_coefs(struct atomisp_sub_device *asd, if (sizeof(*cur) != sizeof(dvs2_coefs.grid) || memcmp(&dvs2_coefs.grid, cur, sizeof(dvs2_coefs.grid))) { - dev_err(asd->isp->dev, "dvs grid mis-match!\n"); + dev_err(asd->isp->dev, "dvs grid mismatch!\n"); /* If the grid info in the argument differs from the current grid info, we tell the caller to reset the grid size and try again. */ @@ -3692,14 +3692,14 @@ void atomisp_handle_parameter_and_buffer(struct atomisp_video_pipe *pipe) unsigned long irqflags; bool need_to_enqueue_buffer = false; - lockdep_assert_held(&asd->isp->mutex); - if (!asd) { dev_err(pipe->isp->dev, "%s(): asd is NULL, device is %s\n", __func__, pipe->vdev.name); return; } + lockdep_assert_held(&asd->isp->mutex); + if (atomisp_is_vf_pipe(pipe)) return; @@ -3774,14 +3774,14 @@ int atomisp_set_parameters(struct video_device *vdev, struct atomisp_css_params *css_param = &asd->params.css_param; int ret; - lockdep_assert_held(&asd->isp->mutex); - if (!asd) { dev_err(pipe->isp->dev, "%s(): asd is NULL, device is %s\n", __func__, vdev->name); return -EINVAL; } + lockdep_assert_held(&asd->isp->mutex); + if (!asd->stream_env[ATOMISP_INPUT_STREAM_GENERAL].stream) { dev_err(asd->isp->dev, "%s: internal error!\n", __func__); return -EINVAL; diff --git a/drivers/staging/media/atomisp/pci/atomisp_compat_css20.c b/drivers/staging/media/atomisp/pci/atomisp_compat_css20.c index fdc05548d972..b36cbde7036a 100644 --- a/drivers/staging/media/atomisp/pci/atomisp_compat_css20.c +++ b/drivers/staging/media/atomisp/pci/atomisp_compat_css20.c @@ -3180,7 +3180,7 @@ static int atomisp_compare_dvs_grid(struct atomisp_sub_device *asd, } if (sizeof(*cur) != sizeof(*atomgrid)) { - dev_err(asd->isp->dev, "dvs grid mis-match!\n"); + dev_err(asd->isp->dev, "dvs grid mismatch!\n"); return -EINVAL; } diff --git a/drivers/staging/media/atomisp/pci/css_2401_system/host/pixelgen_private.h b/drivers/staging/media/atomisp/pci/css_2401_system/host/pixelgen_private.h index 1c7938d8ccb5..8f79424bedb2 100644 --- a/drivers/staging/media/atomisp/pci/css_2401_system/host/pixelgen_private.h +++ b/drivers/staging/media/atomisp/pci/css_2401_system/host/pixelgen_private.h @@ -161,7 +161,7 @@ STORAGE_CLASS_PIXELGEN_C void pixelgen_ctrl_dump_state( state->syng_stat_fcnt); ia_css_print("Pixel Generator ID %d syng stat done 0x%x\n", ID, state->syng_stat_done); - ia_css_print("Pixel Generator ID %d tpg modee 0x%x\n", ID, state->tpg_mode); + ia_css_print("Pixel Generator ID %d tpg mode 0x%x\n", ID, state->tpg_mode); ia_css_print("Pixel Generator ID %d tpg hcnt mask 0x%x\n", ID, state->tpg_hcnt_mask); ia_css_print("Pixel Generator ID %d tpg hcnt mask 0x%x\n", ID, diff --git a/drivers/staging/media/deprecated/atmel/Kconfig b/drivers/staging/media/deprecated/atmel/Kconfig new file mode 100644 index 000000000000..418841ea5a0d --- /dev/null +++ b/drivers/staging/media/deprecated/atmel/Kconfig @@ -0,0 +1,47 @@ +# SPDX-License-Identifier: GPL-2.0-only + +comment "Atmel media platform drivers" + +config VIDEO_ATMEL_ISC + tristate "ATMEL Image Sensor Controller (ISC) support (DEPRECATED)" + depends on V4L_PLATFORM_DRIVERS + depends on VIDEO_DEV && COMMON_CLK + depends on ARCH_AT91 || COMPILE_TEST + depends on !VIDEO_MICROCHIP_ISC_BASE || COMPILE_TEST + select MEDIA_CONTROLLER + select VIDEO_V4L2_SUBDEV_API + select VIDEOBUF2_DMA_CONTIG + select REGMAP_MMIO + select V4L2_FWNODE + select VIDEO_ATMEL_ISC_BASE + help + This module makes the ATMEL Image Sensor Controller available + as a v4l2 device. + + This driver is deprecated and is scheduled for removal by + the beginning of 2026. See the TODO file for more information. + +config VIDEO_ATMEL_XISC + tristate "ATMEL eXtended Image Sensor Controller (XISC) support (DEPRECATED)" + depends on V4L_PLATFORM_DRIVERS + depends on VIDEO_DEV && COMMON_CLK + depends on ARCH_AT91 || COMPILE_TEST + depends on !VIDEO_MICROCHIP_ISC_BASE || COMPILE_TEST + select VIDEOBUF2_DMA_CONTIG + select REGMAP_MMIO + select V4L2_FWNODE + select VIDEO_ATMEL_ISC_BASE + select MEDIA_CONTROLLER + select VIDEO_V4L2_SUBDEV_API + help + This module makes the ATMEL eXtended Image Sensor Controller + available as a v4l2 device. + + This driver is deprecated and is scheduled for removal by + the beginning of 2026. See the TODO file for more information. + +config VIDEO_ATMEL_ISC_BASE + tristate + default n + help + ATMEL ISC and XISC common code base. diff --git a/drivers/staging/media/deprecated/atmel/Makefile b/drivers/staging/media/deprecated/atmel/Makefile new file mode 100644 index 000000000000..34eaeeac5bba --- /dev/null +++ b/drivers/staging/media/deprecated/atmel/Makefile @@ -0,0 +1,8 @@ +# SPDX-License-Identifier: GPL-2.0-only +atmel-isc-objs = atmel-sama5d2-isc.o +atmel-xisc-objs = atmel-sama7g5-isc.o +atmel-isc-common-objs = atmel-isc-base.o atmel-isc-clk.o + +obj-$(CONFIG_VIDEO_ATMEL_ISC_BASE) += atmel-isc-common.o +obj-$(CONFIG_VIDEO_ATMEL_ISC) += atmel-isc.o +obj-$(CONFIG_VIDEO_ATMEL_XISC) += atmel-xisc.o diff --git a/drivers/staging/media/deprecated/atmel/TODO b/drivers/staging/media/deprecated/atmel/TODO new file mode 100644 index 000000000000..71691df07a80 --- /dev/null +++ b/drivers/staging/media/deprecated/atmel/TODO @@ -0,0 +1,34 @@ +The Atmel ISC driver is not compliant with media controller specification. +In order to evolve this driver, it has to move to media controller, to +support enhanced features and future products which embed it. +The move to media controller involves several changes which are +not backwards compatible with the current usability of the driver. + +The best example is the way the format is propagated from the top video +driver /dev/videoX down to the sensor. + +In a simple configuration sensor ==> isc , the isc just calls subdev s_fmt +and controls the sensor directly. This is achieved by having a lot of code +inside the driver that will query the subdev at probe time and make a list +of formats which are usable. +Basically the user has nothing to configure, as the isc will handle +everything at the top level. This is an easy way to capture, but also comes +with the drawback of lack of flexibility. +In a more complicated pipeline +sensor ==> controller 1 ==> controller 2 ==> isc +this will not be achievable, as controller 1 and controller 2 might be +media-controller configurable, and will not propagate the formats down to +the sensor. + +After discussions with the media maintainers, the decision is to move +Atmel ISC to staging as-is, to keep the Kconfig symbols and the users +to the driver in staging. Thus, all the existing users of the non +media-controller paradigm will continue to be happy and use the old config +way. + +The new driver was added in the media subsystem with a different +symbol, with the conversion to media controller done, and new users +of the driver will be able to use all the new features. + +The replacement driver is named VIDEO_MICROCHIP_ISC or +VIDEO_MICROCHIP_XISC depending on the product flavor. diff --git a/drivers/media/platform/atmel/atmel-isc-base.c b/drivers/staging/media/deprecated/atmel/atmel-isc-base.c index 9e5317a7d516..99e61bbfc9bc 100644 --- a/drivers/media/platform/atmel/atmel-isc-base.c +++ b/drivers/staging/media/deprecated/atmel/atmel-isc-base.c @@ -1221,7 +1221,7 @@ static const struct v4l2_file_operations isc_fops = { .poll = vb2_fop_poll, }; -irqreturn_t isc_interrupt(int irq, void *dev_id) +irqreturn_t atmel_isc_interrupt(int irq, void *dev_id) { struct isc_device *isc = (struct isc_device *)dev_id; struct regmap *regmap = isc->regmap; @@ -1267,7 +1267,7 @@ irqreturn_t isc_interrupt(int irq, void *dev_id) return ret; } -EXPORT_SYMBOL_GPL(isc_interrupt); +EXPORT_SYMBOL_GPL(atmel_isc_interrupt); static void isc_hist_count(struct isc_device *isc, u32 *min, u32 *max) { @@ -1934,14 +1934,14 @@ isc_async_complete_err: return ret; } -const struct v4l2_async_notifier_operations isc_async_ops = { +const struct v4l2_async_notifier_operations atmel_isc_async_ops = { .bound = isc_async_bound, .unbind = isc_async_unbind, .complete = isc_async_complete, }; -EXPORT_SYMBOL_GPL(isc_async_ops); +EXPORT_SYMBOL_GPL(atmel_isc_async_ops); -void isc_subdev_cleanup(struct isc_device *isc) +void atmel_isc_subdev_cleanup(struct isc_device *isc) { struct isc_subdev_entity *subdev_entity; @@ -1952,9 +1952,9 @@ void isc_subdev_cleanup(struct isc_device *isc) INIT_LIST_HEAD(&isc->subdev_entities); } -EXPORT_SYMBOL_GPL(isc_subdev_cleanup); +EXPORT_SYMBOL_GPL(atmel_isc_subdev_cleanup); -int isc_pipeline_init(struct isc_device *isc) +int atmel_isc_pipeline_init(struct isc_device *isc) { struct device *dev = isc->dev; struct regmap *regmap = isc->regmap; @@ -1993,17 +1993,17 @@ int isc_pipeline_init(struct isc_device *isc) return 0; } -EXPORT_SYMBOL_GPL(isc_pipeline_init); +EXPORT_SYMBOL_GPL(atmel_isc_pipeline_init); /* regmap configuration */ #define ATMEL_ISC_REG_MAX 0xd5c -const struct regmap_config isc_regmap_config = { +const struct regmap_config atmel_isc_regmap_config = { .reg_bits = 32, .reg_stride = 4, .val_bits = 32, .max_register = ATMEL_ISC_REG_MAX, }; -EXPORT_SYMBOL_GPL(isc_regmap_config); +EXPORT_SYMBOL_GPL(atmel_isc_regmap_config); MODULE_AUTHOR("Songjun Wu"); MODULE_AUTHOR("Eugen Hristev"); diff --git a/drivers/media/platform/atmel/atmel-isc-clk.c b/drivers/staging/media/deprecated/atmel/atmel-isc-clk.c index 2059fe376b00..d442b5f4c931 100644 --- a/drivers/media/platform/atmel/atmel-isc-clk.c +++ b/drivers/staging/media/deprecated/atmel/atmel-isc-clk.c @@ -277,7 +277,7 @@ static int isc_clk_register(struct isc_device *isc, unsigned int id) return 0; } -int isc_clk_init(struct isc_device *isc) +int atmel_isc_clk_init(struct isc_device *isc) { unsigned int i; int ret; @@ -293,9 +293,9 @@ int isc_clk_init(struct isc_device *isc) return 0; } -EXPORT_SYMBOL_GPL(isc_clk_init); +EXPORT_SYMBOL_GPL(atmel_isc_clk_init); -void isc_clk_cleanup(struct isc_device *isc) +void atmel_isc_clk_cleanup(struct isc_device *isc) { unsigned int i; @@ -308,4 +308,4 @@ void isc_clk_cleanup(struct isc_device *isc) clk_unregister(isc_clk->clk); } } -EXPORT_SYMBOL_GPL(isc_clk_cleanup); +EXPORT_SYMBOL_GPL(atmel_isc_clk_cleanup); diff --git a/drivers/media/platform/atmel/atmel-isc-regs.h b/drivers/staging/media/deprecated/atmel/atmel-isc-regs.h index d06b72228d4f..d06b72228d4f 100644 --- a/drivers/media/platform/atmel/atmel-isc-regs.h +++ b/drivers/staging/media/deprecated/atmel/atmel-isc-regs.h diff --git a/drivers/media/platform/atmel/atmel-isc.h b/drivers/staging/media/deprecated/atmel/atmel-isc.h index ff60ba020cb9..dfc030b5a08f 100644 --- a/drivers/media/platform/atmel/atmel-isc.h +++ b/drivers/staging/media/deprecated/atmel/atmel-isc.h @@ -350,13 +350,13 @@ struct isc_device { u32 formats_list_size; }; -extern const struct regmap_config isc_regmap_config; -extern const struct v4l2_async_notifier_operations isc_async_ops; - -irqreturn_t isc_interrupt(int irq, void *dev_id); -int isc_pipeline_init(struct isc_device *isc); -int isc_clk_init(struct isc_device *isc); -void isc_subdev_cleanup(struct isc_device *isc); -void isc_clk_cleanup(struct isc_device *isc); +extern const struct regmap_config atmel_isc_regmap_config; +extern const struct v4l2_async_notifier_operations atmel_isc_async_ops; + +irqreturn_t atmel_isc_interrupt(int irq, void *dev_id); +int atmel_isc_pipeline_init(struct isc_device *isc); +int atmel_isc_clk_init(struct isc_device *isc); +void atmel_isc_subdev_cleanup(struct isc_device *isc); +void atmel_isc_clk_cleanup(struct isc_device *isc); #endif diff --git a/drivers/media/platform/atmel/atmel-sama5d2-isc.c b/drivers/staging/media/deprecated/atmel/atmel-sama5d2-isc.c index 9881d89a645b..ba0614f981a2 100644 --- a/drivers/media/platform/atmel/atmel-sama5d2-isc.c +++ b/drivers/staging/media/deprecated/atmel/atmel-sama5d2-isc.c @@ -408,7 +408,7 @@ static int atmel_isc_probe(struct platform_device *pdev) if (IS_ERR(io_base)) return PTR_ERR(io_base); - isc->regmap = devm_regmap_init_mmio(dev, io_base, &isc_regmap_config); + isc->regmap = devm_regmap_init_mmio(dev, io_base, &atmel_isc_regmap_config); if (IS_ERR(isc->regmap)) { ret = PTR_ERR(isc->regmap); dev_err(dev, "failed to init register map: %d\n", ret); @@ -419,7 +419,7 @@ static int atmel_isc_probe(struct platform_device *pdev) if (irq < 0) return irq; - ret = devm_request_irq(dev, irq, isc_interrupt, 0, + ret = devm_request_irq(dev, irq, atmel_isc_interrupt, 0, "atmel-sama5d2-isc", isc); if (ret < 0) { dev_err(dev, "can't register ISR for IRQ %u (ret=%i)\n", @@ -464,7 +464,7 @@ static int atmel_isc_probe(struct platform_device *pdev) /* sama5d2-isc : ISPCK is required and mandatory */ isc->ispck_required = true; - ret = isc_pipeline_init(isc); + ret = atmel_isc_pipeline_init(isc); if (ret) return ret; @@ -481,7 +481,7 @@ static int atmel_isc_probe(struct platform_device *pdev) return ret; } - ret = isc_clk_init(isc); + ret = atmel_isc_clk_init(isc); if (ret) { dev_err(dev, "failed to init isc clock: %d\n", ret); goto unprepare_hclk; @@ -523,7 +523,7 @@ static int atmel_isc_probe(struct platform_device *pdev) goto cleanup_subdev; } - subdev_entity->notifier.ops = &isc_async_ops; + subdev_entity->notifier.ops = &atmel_isc_async_ops; ret = v4l2_async_nf_register(&isc->v4l2_dev, &subdev_entity->notifier); @@ -567,7 +567,7 @@ disable_pm: pm_runtime_disable(dev); cleanup_subdev: - isc_subdev_cleanup(isc); + atmel_isc_subdev_cleanup(isc); unregister_v4l2_device: v4l2_device_unregister(&isc->v4l2_dev); @@ -575,7 +575,7 @@ unregister_v4l2_device: unprepare_hclk: clk_disable_unprepare(isc->hclock); - isc_clk_cleanup(isc); + atmel_isc_clk_cleanup(isc); return ret; } @@ -586,14 +586,14 @@ static int atmel_isc_remove(struct platform_device *pdev) pm_runtime_disable(&pdev->dev); - isc_subdev_cleanup(isc); + atmel_isc_subdev_cleanup(isc); v4l2_device_unregister(&isc->v4l2_dev); clk_disable_unprepare(isc->ispck); clk_disable_unprepare(isc->hclock); - isc_clk_cleanup(isc); + atmel_isc_clk_cleanup(isc); return 0; } diff --git a/drivers/media/platform/atmel/atmel-sama7g5-isc.c b/drivers/staging/media/deprecated/atmel/atmel-sama7g5-isc.c index 8b11aa8340d7..01ababdfcbd9 100644 --- a/drivers/media/platform/atmel/atmel-sama7g5-isc.c +++ b/drivers/staging/media/deprecated/atmel/atmel-sama7g5-isc.c @@ -397,7 +397,7 @@ static int microchip_xisc_probe(struct platform_device *pdev) if (IS_ERR(io_base)) return PTR_ERR(io_base); - isc->regmap = devm_regmap_init_mmio(dev, io_base, &isc_regmap_config); + isc->regmap = devm_regmap_init_mmio(dev, io_base, &atmel_isc_regmap_config); if (IS_ERR(isc->regmap)) { ret = PTR_ERR(isc->regmap); dev_err(dev, "failed to init register map: %d\n", ret); @@ -408,7 +408,7 @@ static int microchip_xisc_probe(struct platform_device *pdev) if (irq < 0) return irq; - ret = devm_request_irq(dev, irq, isc_interrupt, 0, + ret = devm_request_irq(dev, irq, atmel_isc_interrupt, 0, "microchip-sama7g5-xisc", isc); if (ret < 0) { dev_err(dev, "can't register ISR for IRQ %u (ret=%i)\n", @@ -453,7 +453,7 @@ static int microchip_xisc_probe(struct platform_device *pdev) /* sama7g5-isc : ISPCK does not exist, ISC is clocked by MCK */ isc->ispck_required = false; - ret = isc_pipeline_init(isc); + ret = atmel_isc_pipeline_init(isc); if (ret) return ret; @@ -470,7 +470,7 @@ static int microchip_xisc_probe(struct platform_device *pdev) return ret; } - ret = isc_clk_init(isc); + ret = atmel_isc_clk_init(isc); if (ret) { dev_err(dev, "failed to init isc clock: %d\n", ret); goto unprepare_hclk; @@ -513,7 +513,7 @@ static int microchip_xisc_probe(struct platform_device *pdev) goto cleanup_subdev; } - subdev_entity->notifier.ops = &isc_async_ops; + subdev_entity->notifier.ops = &atmel_isc_async_ops; ret = v4l2_async_nf_register(&isc->v4l2_dev, &subdev_entity->notifier); @@ -536,7 +536,7 @@ static int microchip_xisc_probe(struct platform_device *pdev) return 0; cleanup_subdev: - isc_subdev_cleanup(isc); + atmel_isc_subdev_cleanup(isc); unregister_v4l2_device: v4l2_device_unregister(&isc->v4l2_dev); @@ -544,7 +544,7 @@ unregister_v4l2_device: unprepare_hclk: clk_disable_unprepare(isc->hclock); - isc_clk_cleanup(isc); + atmel_isc_clk_cleanup(isc); return ret; } @@ -555,13 +555,13 @@ static int microchip_xisc_remove(struct platform_device *pdev) pm_runtime_disable(&pdev->dev); - isc_subdev_cleanup(isc); + atmel_isc_subdev_cleanup(isc); v4l2_device_unregister(&isc->v4l2_dev); clk_disable_unprepare(isc->hclock); - isc_clk_cleanup(isc); + atmel_isc_clk_cleanup(isc); return 0; } diff --git a/drivers/staging/media/meson/vdec/codec_vp9.c b/drivers/staging/media/meson/vdec/codec_vp9.c index 897f5d7a6aad..394df5761556 100644 --- a/drivers/staging/media/meson/vdec/codec_vp9.c +++ b/drivers/staging/media/meson/vdec/codec_vp9.c @@ -1459,7 +1459,7 @@ static void vp9_tree_merge_probs(unsigned int *prev_prob, if (den == 0) { new_prob = pre_prob; } else { - m_count = den < MODE_MV_COUNT_SAT ? den : MODE_MV_COUNT_SAT; + m_count = min(den, MODE_MV_COUNT_SAT); get_prob = clip_prob(div_r32(((int64_t)tree_left * 256 + (den >> 1)), @@ -1513,7 +1513,7 @@ static void adapt_coef_probs_cxt(unsigned int *prev_prob, /* get binary prob */ num = branch_ct[node][0]; den = branch_ct[node][0] + branch_ct[node][1]; - m_count = den < count_sat ? den : count_sat; + m_count = min(den, count_sat); get_prob = (den == 0) ? 128u : @@ -1649,8 +1649,7 @@ static void adapt_coef_probs(int prev_kf, int cur_kf, int pre_fc, else if (coef_count_node_start == VP9_MV_BITS_1_COUNT_START) coef_node_start = VP9_MV_BITS_1_START; - else if (coef_count_node_start == - VP9_MV_CLASS0_HP_0_COUNT_START) + else /* node_start == VP9_MV_CLASS0_HP_0_COUNT_START */ coef_node_start = VP9_MV_CLASS0_HP_0_START; den = count[coef_count_node_start] + @@ -1664,8 +1663,7 @@ static void adapt_coef_probs(int prev_kf, int cur_kf, int pre_fc, if (den == 0) { new_prob = pre_prob; } else { - m_count = den < MODE_MV_COUNT_SAT ? - den : MODE_MV_COUNT_SAT; + m_count = min(den, MODE_MV_COUNT_SAT); get_prob = clip_prob(div_r32(((int64_t) count[coef_count_node_start] * 256 + diff --git a/drivers/staging/media/sunxi/Kconfig b/drivers/staging/media/sunxi/Kconfig index 4549a135741f..62a486aba88b 100644 --- a/drivers/staging/media/sunxi/Kconfig +++ b/drivers/staging/media/sunxi/Kconfig @@ -12,5 +12,6 @@ config VIDEO_SUNXI if VIDEO_SUNXI source "drivers/staging/media/sunxi/cedrus/Kconfig" +source "drivers/staging/media/sunxi/sun6i-isp/Kconfig" endif diff --git a/drivers/staging/media/sunxi/Makefile b/drivers/staging/media/sunxi/Makefile index b87140b0e15f..3d20b2f0e644 100644 --- a/drivers/staging/media/sunxi/Makefile +++ b/drivers/staging/media/sunxi/Makefile @@ -1,2 +1,3 @@ # SPDX-License-Identifier: GPL-2.0 obj-$(CONFIG_VIDEO_SUNXI_CEDRUS) += cedrus/ +obj-$(CONFIG_VIDEO_SUN6I_ISP) += sun6i-isp/ diff --git a/drivers/staging/media/sunxi/cedrus/cedrus.c b/drivers/staging/media/sunxi/cedrus/cedrus.c index 6a2c08928613..a43d5ff66716 100644 --- a/drivers/staging/media/sunxi/cedrus/cedrus.c +++ b/drivers/staging/media/sunxi/cedrus/cedrus.c @@ -45,23 +45,37 @@ static int cedrus_try_ctrl(struct v4l2_ctrl *ctrl) } else if (ctrl->id == V4L2_CID_STATELESS_HEVC_SPS) { const struct v4l2_ctrl_hevc_sps *sps = ctrl->p_new.p_hevc_sps; struct cedrus_ctx *ctx = container_of(ctrl->handler, struct cedrus_ctx, hdl); + unsigned int bit_depth, max_depth; + struct vb2_queue *vq; if (sps->chroma_format_idc != 1) /* Only 4:2:0 is supported */ return -EINVAL; - if (sps->bit_depth_luma_minus8 != sps->bit_depth_chroma_minus8) - /* Luma and chroma bit depth mismatch */ + bit_depth = max(sps->bit_depth_luma_minus8, + sps->bit_depth_chroma_minus8) + 8; + + if (cedrus_is_capable(ctx, CEDRUS_CAPABILITY_H265_10_DEC)) + max_depth = 10; + else + max_depth = 8; + + if (bit_depth > max_depth) return -EINVAL; - if (ctx->dev->capabilities & CEDRUS_CAPABILITY_H265_10_DEC) { - if (sps->bit_depth_luma_minus8 != 0 && sps->bit_depth_luma_minus8 != 2) - /* Only 8-bit and 10-bit are supported */ + vq = v4l2_m2m_get_vq(ctx->fh.m2m_ctx, + V4L2_BUF_TYPE_VIDEO_CAPTURE); + + /* + * Bit depth can't be higher than currently set once + * buffers are allocated. + */ + if (vb2_is_busy(vq)) { + if (ctx->bit_depth < bit_depth) return -EINVAL; } else { - if (sps->bit_depth_luma_minus8 != 0) - /* Only 8-bit is supported */ - return -EINVAL; + ctx->bit_depth = bit_depth; + cedrus_reset_cap_format(ctx); } } @@ -354,6 +368,7 @@ static int cedrus_open(struct file *file) v4l2_fh_init(&ctx->fh, video_devdata(file)); file->private_data = &ctx->fh; ctx->dev = dev; + ctx->bit_depth = 8; ctx->fh.m2m_ctx = v4l2_m2m_ctx_init(dev->m2m_dev, ctx, &cedrus_queue_init); diff --git a/drivers/staging/media/sunxi/cedrus/cedrus.h b/drivers/staging/media/sunxi/cedrus/cedrus.h index a6a649bacc95..522c184e2afc 100644 --- a/drivers/staging/media/sunxi/cedrus/cedrus.h +++ b/drivers/staging/media/sunxi/cedrus/cedrus.h @@ -100,7 +100,15 @@ struct cedrus_buffer { struct { unsigned int position; enum cedrus_h264_pic_type pic_type; + void *mv_col_buf; + dma_addr_t mv_col_buf_dma; + ssize_t mv_col_buf_size; } h264; + struct { + void *mv_col_buf; + dma_addr_t mv_col_buf_dma; + ssize_t mv_col_buf_size; + } h265; } codec; }; @@ -111,16 +119,13 @@ struct cedrus_ctx { struct v4l2_pix_format src_fmt; struct v4l2_pix_format dst_fmt; struct cedrus_dec_ops *current_codec; + unsigned int bit_depth; struct v4l2_ctrl_handler hdl; struct v4l2_ctrl **ctrls; union { struct { - void *mv_col_buf; - dma_addr_t mv_col_buf_dma; - ssize_t mv_col_buf_field_size; - ssize_t mv_col_buf_size; void *pic_info_buf; dma_addr_t pic_info_buf_dma; ssize_t pic_info_buf_size; @@ -134,10 +139,6 @@ struct cedrus_ctx { ssize_t intra_pred_buf_size; } h264; struct { - void *mv_col_buf; - dma_addr_t mv_col_buf_addr; - ssize_t mv_col_buf_size; - ssize_t mv_col_buf_unit_size; void *neighbor_info_buf; dma_addr_t neighbor_info_buf_addr; void *entry_points_buf; @@ -162,6 +163,8 @@ struct cedrus_dec_ops { int (*start)(struct cedrus_ctx *ctx); void (*stop)(struct cedrus_ctx *ctx); void (*trigger)(struct cedrus_ctx *ctx); + unsigned int (*extra_cap_size)(struct cedrus_ctx *ctx, + struct v4l2_pix_format *pix_fmt); }; struct cedrus_variant { diff --git a/drivers/staging/media/sunxi/cedrus/cedrus_h264.c b/drivers/staging/media/sunxi/cedrus/cedrus_h264.c index c139a37a567c..dfb401df138a 100644 --- a/drivers/staging/media/sunxi/cedrus/cedrus_h264.c +++ b/drivers/staging/media/sunxi/cedrus/cedrus_h264.c @@ -54,17 +54,13 @@ static void cedrus_h264_write_sram(struct cedrus_dev *dev, cedrus_write(dev, VE_AVC_SRAM_PORT_DATA, *buffer++); } -static dma_addr_t cedrus_h264_mv_col_buf_addr(struct cedrus_ctx *ctx, - unsigned int position, +static dma_addr_t cedrus_h264_mv_col_buf_addr(struct cedrus_buffer *buf, unsigned int field) { - dma_addr_t addr = ctx->codec.h264.mv_col_buf_dma; - - /* Adjust for the position */ - addr += position * ctx->codec.h264.mv_col_buf_field_size * 2; + dma_addr_t addr = buf->codec.h264.mv_col_buf_dma; /* Adjust for the field */ - addr += field * ctx->codec.h264.mv_col_buf_field_size; + addr += field * buf->codec.h264.mv_col_buf_size / 2; return addr; } @@ -76,7 +72,6 @@ static void cedrus_fill_ref_pic(struct cedrus_ctx *ctx, struct cedrus_h264_sram_ref_pic *pic) { struct vb2_buffer *vbuf = &buf->m2m_buf.vb.vb2_buf; - unsigned int position = buf->codec.h264.position; pic->top_field_order_cnt = cpu_to_le32(top_field_order_cnt); pic->bottom_field_order_cnt = cpu_to_le32(bottom_field_order_cnt); @@ -84,14 +79,12 @@ static void cedrus_fill_ref_pic(struct cedrus_ctx *ctx, pic->luma_ptr = cpu_to_le32(cedrus_buf_addr(vbuf, &ctx->dst_fmt, 0)); pic->chroma_ptr = cpu_to_le32(cedrus_buf_addr(vbuf, &ctx->dst_fmt, 1)); - pic->mv_col_top_ptr = - cpu_to_le32(cedrus_h264_mv_col_buf_addr(ctx, position, 0)); - pic->mv_col_bot_ptr = - cpu_to_le32(cedrus_h264_mv_col_buf_addr(ctx, position, 1)); + pic->mv_col_top_ptr = cpu_to_le32(cedrus_h264_mv_col_buf_addr(buf, 0)); + pic->mv_col_bot_ptr = cpu_to_le32(cedrus_h264_mv_col_buf_addr(buf, 1)); } -static void cedrus_write_frame_list(struct cedrus_ctx *ctx, - struct cedrus_run *run) +static int cedrus_write_frame_list(struct cedrus_ctx *ctx, + struct cedrus_run *run) { struct cedrus_h264_sram_ref_pic pic_list[CEDRUS_H264_FRAME_NUM]; const struct v4l2_ctrl_h264_decode_params *decode = run->h264.decode_params; @@ -146,6 +139,31 @@ static void cedrus_write_frame_list(struct cedrus_ctx *ctx, output_buf = vb2_to_cedrus_buffer(&run->dst->vb2_buf); output_buf->codec.h264.position = position; + if (!output_buf->codec.h264.mv_col_buf_size) { + const struct v4l2_ctrl_h264_sps *sps = run->h264.sps; + unsigned int field_size; + + field_size = DIV_ROUND_UP(ctx->src_fmt.width, 16) * + DIV_ROUND_UP(ctx->src_fmt.height, 16) * 16; + if (!(sps->flags & V4L2_H264_SPS_FLAG_DIRECT_8X8_INFERENCE)) + field_size = field_size * 2; + if (!(sps->flags & V4L2_H264_SPS_FLAG_FRAME_MBS_ONLY)) + field_size = field_size * 2; + + output_buf->codec.h264.mv_col_buf_size = field_size * 2; + /* Buffer is never accessed by CPU, so we can skip kernel mapping. */ + output_buf->codec.h264.mv_col_buf = + dma_alloc_attrs(dev->dev, + output_buf->codec.h264.mv_col_buf_size, + &output_buf->codec.h264.mv_col_buf_dma, + GFP_KERNEL, DMA_ATTR_NO_KERNEL_MAPPING); + + if (!output_buf->codec.h264.mv_col_buf) { + output_buf->codec.h264.mv_col_buf_size = 0; + return -ENOMEM; + } + } + if (decode->flags & V4L2_H264_DECODE_PARAM_FLAG_FIELD_PIC) output_buf->codec.h264.pic_type = CEDRUS_H264_PIC_TYPE_FIELD; else if (sps->flags & V4L2_H264_SPS_FLAG_MB_ADAPTIVE_FRAME_FIELD) @@ -162,6 +180,8 @@ static void cedrus_write_frame_list(struct cedrus_ctx *ctx, pic_list, sizeof(pic_list)); cedrus_write(dev, VE_H264_OUTPUT_FRAME_IDX, position); + + return 0; } #define CEDRUS_MAX_REF_IDX 32 @@ -496,6 +516,7 @@ static void cedrus_h264_irq_disable(struct cedrus_ctx *ctx) static int cedrus_h264_setup(struct cedrus_ctx *ctx, struct cedrus_run *run) { struct cedrus_dev *dev = ctx->dev; + int ret; cedrus_engine_enable(ctx); @@ -506,7 +527,9 @@ static int cedrus_h264_setup(struct cedrus_ctx *ctx, struct cedrus_run *run) ctx->codec.h264.neighbor_info_buf_dma); cedrus_write_scaling_lists(ctx, run); - cedrus_write_frame_list(ctx, run); + ret = cedrus_write_frame_list(ctx, run); + if (ret) + return ret; cedrus_set_params(ctx, run); @@ -517,8 +540,6 @@ static int cedrus_h264_start(struct cedrus_ctx *ctx) { struct cedrus_dev *dev = ctx->dev; unsigned int pic_info_size; - unsigned int field_size; - unsigned int mv_col_size; int ret; /* @@ -566,38 +587,6 @@ static int cedrus_h264_start(struct cedrus_ctx *ctx) goto err_pic_buf; } - field_size = DIV_ROUND_UP(ctx->src_fmt.width, 16) * - DIV_ROUND_UP(ctx->src_fmt.height, 16) * 16; - - /* - * FIXME: This is actually conditional to - * V4L2_H264_SPS_FLAG_DIRECT_8X8_INFERENCE not being set, we - * might have to rework this if memory efficiency ever is - * something we need to work on. - */ - field_size = field_size * 2; - - /* - * FIXME: This is actually conditional to - * V4L2_H264_SPS_FLAG_FRAME_MBS_ONLY not being set, we might - * have to rework this if memory efficiency ever is something - * we need to work on. - */ - field_size = field_size * 2; - ctx->codec.h264.mv_col_buf_field_size = field_size; - - mv_col_size = field_size * 2 * CEDRUS_H264_FRAME_NUM; - ctx->codec.h264.mv_col_buf_size = mv_col_size; - ctx->codec.h264.mv_col_buf = - dma_alloc_attrs(dev->dev, - ctx->codec.h264.mv_col_buf_size, - &ctx->codec.h264.mv_col_buf_dma, - GFP_KERNEL, DMA_ATTR_NO_KERNEL_MAPPING); - if (!ctx->codec.h264.mv_col_buf) { - ret = -ENOMEM; - goto err_neighbor_buf; - } - if (ctx->src_fmt.width > 2048) { /* * Formulas for deblock and intra prediction buffer sizes @@ -613,7 +602,7 @@ static int cedrus_h264_start(struct cedrus_ctx *ctx) GFP_KERNEL, DMA_ATTR_NO_KERNEL_MAPPING); if (!ctx->codec.h264.deblk_buf) { ret = -ENOMEM; - goto err_mv_col_buf; + goto err_neighbor_buf; } /* @@ -641,12 +630,6 @@ err_deblk_buf: ctx->codec.h264.deblk_buf_dma, DMA_ATTR_NO_KERNEL_MAPPING); -err_mv_col_buf: - dma_free_attrs(dev->dev, ctx->codec.h264.mv_col_buf_size, - ctx->codec.h264.mv_col_buf, - ctx->codec.h264.mv_col_buf_dma, - DMA_ATTR_NO_KERNEL_MAPPING); - err_neighbor_buf: dma_free_attrs(dev->dev, CEDRUS_NEIGHBOR_INFO_BUF_SIZE, ctx->codec.h264.neighbor_info_buf, @@ -664,11 +647,26 @@ err_pic_buf: static void cedrus_h264_stop(struct cedrus_ctx *ctx) { struct cedrus_dev *dev = ctx->dev; + struct cedrus_buffer *buf; + struct vb2_queue *vq; + unsigned int i; + + vq = v4l2_m2m_get_vq(ctx->fh.m2m_ctx, V4L2_BUF_TYPE_VIDEO_CAPTURE); + + for (i = 0; i < vq->num_buffers; i++) { + buf = vb2_to_cedrus_buffer(vb2_get_buffer(vq, i)); + + if (buf->codec.h264.mv_col_buf_size > 0) { + dma_free_attrs(dev->dev, + buf->codec.h264.mv_col_buf_size, + buf->codec.h264.mv_col_buf, + buf->codec.h264.mv_col_buf_dma, + DMA_ATTR_NO_KERNEL_MAPPING); + + buf->codec.h264.mv_col_buf_size = 0; + } + } - dma_free_attrs(dev->dev, ctx->codec.h264.mv_col_buf_size, - ctx->codec.h264.mv_col_buf, - ctx->codec.h264.mv_col_buf_dma, - DMA_ATTR_NO_KERNEL_MAPPING); dma_free_attrs(dev->dev, CEDRUS_NEIGHBOR_INFO_BUF_SIZE, ctx->codec.h264.neighbor_info_buf, ctx->codec.h264.neighbor_info_buf_dma, diff --git a/drivers/staging/media/sunxi/cedrus/cedrus_h265.c b/drivers/staging/media/sunxi/cedrus/cedrus_h265.c index 3f2946c8f174..fc9297232456 100644 --- a/drivers/staging/media/sunxi/cedrus/cedrus_h265.c +++ b/drivers/staging/media/sunxi/cedrus/cedrus_h265.c @@ -41,6 +41,19 @@ struct cedrus_h265_sram_pred_weight { __s8 offset; } __packed; +static unsigned int cedrus_h265_2bit_size(unsigned int width, + unsigned int height) +{ + /* + * Vendor library additionally aligns width and height to 16, + * but all capture formats are already aligned to that anyway, + * so we can skip that here. All formats are also one form of + * YUV 4:2:0 or another, so we can safely assume multiplication + * factor of 1.5. + */ + return ALIGN(width / 4, 32) * height * 3 / 2; +} + static enum cedrus_irq_status cedrus_h265_irq_status(struct cedrus_ctx *ctx) { struct cedrus_dev *dev = ctx->dev; @@ -90,12 +103,13 @@ static void cedrus_h265_sram_write_data(struct cedrus_dev *dev, void *data, } static inline dma_addr_t -cedrus_h265_frame_info_mv_col_buf_addr(struct cedrus_ctx *ctx, - unsigned int index, unsigned int field) +cedrus_h265_frame_info_mv_col_buf_addr(struct vb2_buffer *buf, + unsigned int field) { - return ctx->codec.h265.mv_col_buf_addr + index * - ctx->codec.h265.mv_col_buf_unit_size + - field * ctx->codec.h265.mv_col_buf_unit_size / 2; + struct cedrus_buffer *cedrus_buf = vb2_to_cedrus_buffer(buf); + + return cedrus_buf->codec.h265.mv_col_buf_dma + + field * cedrus_buf->codec.h265.mv_col_buf_size / 2; } static void cedrus_h265_frame_info_write_single(struct cedrus_ctx *ctx, @@ -108,9 +122,8 @@ static void cedrus_h265_frame_info_write_single(struct cedrus_ctx *ctx, dma_addr_t dst_luma_addr = cedrus_dst_buf_addr(ctx, buf, 0); dma_addr_t dst_chroma_addr = cedrus_dst_buf_addr(ctx, buf, 1); dma_addr_t mv_col_buf_addr[2] = { - cedrus_h265_frame_info_mv_col_buf_addr(ctx, buf->index, 0), - cedrus_h265_frame_info_mv_col_buf_addr(ctx, buf->index, - field_pic ? 1 : 0) + cedrus_h265_frame_info_mv_col_buf_addr(buf, 0), + cedrus_h265_frame_info_mv_col_buf_addr(buf, field_pic ? 1 : 0) }; u32 offset = VE_DEC_H265_SRAM_OFFSET_FRAME_INFO + VE_DEC_H265_SRAM_OFFSET_FRAME_INFO_UNIT * index; @@ -242,6 +255,18 @@ static void cedrus_h265_skip_bits(struct cedrus_dev *dev, int num) } } +static u32 cedrus_h265_show_bits(struct cedrus_dev *dev, int num) +{ + cedrus_write(dev, VE_DEC_H265_TRIGGER, + VE_DEC_H265_TRIGGER_SHOW_BITS | + VE_DEC_H265_TRIGGER_TYPE_N_BITS(num)); + + cedrus_wait_for(dev, VE_DEC_H265_STATUS, + VE_DEC_H265_STATUS_VLD_BUSY); + + return cedrus_read(dev, VE_DEC_H265_BITS_READ); +} + static void cedrus_h265_write_scaling_list(struct cedrus_ctx *ctx, struct cedrus_run *run) { @@ -400,13 +425,14 @@ static int cedrus_h265_setup(struct cedrus_ctx *ctx, struct cedrus_run *run) unsigned int width_in_ctb_luma, ctb_size_luma; unsigned int log2_max_luma_coding_block_size; unsigned int ctb_addr_x, ctb_addr_y; + struct cedrus_buffer *cedrus_buf; dma_addr_t src_buf_addr; dma_addr_t src_buf_end_addr; u32 chroma_log2_weight_denom; u32 num_entry_point_offsets; u32 output_pic_list_index; u32 pic_order_cnt[2]; - u8 *padding; + u8 padding; int count; u32 reg; @@ -416,6 +442,7 @@ static int cedrus_h265_setup(struct cedrus_ctx *ctx, struct cedrus_run *run) decode_params = run->h265.decode_params; pred_weight_table = &slice_params->pred_weight_table; num_entry_point_offsets = slice_params->num_entry_point_offsets; + cedrus_buf = vb2_to_cedrus_buffer(&run->dst->vb2_buf); /* * If entry points offsets are present, we should get them @@ -433,31 +460,25 @@ static int cedrus_h265_setup(struct cedrus_ctx *ctx, struct cedrus_run *run) DIV_ROUND_UP(sps->pic_width_in_luma_samples, ctb_size_luma); /* MV column buffer size and allocation. */ - if (!ctx->codec.h265.mv_col_buf_size) { - unsigned int num_buffers = - run->dst->vb2_buf.vb2_queue->num_buffers; - + if (!cedrus_buf->codec.h265.mv_col_buf_size) { /* * Each CTB requires a MV col buffer with a specific unit size. * Since the address is given with missing lsb bits, 1 KiB is * added to each buffer to ensure proper alignment. */ - ctx->codec.h265.mv_col_buf_unit_size = + cedrus_buf->codec.h265.mv_col_buf_size = DIV_ROUND_UP(ctx->src_fmt.width, ctb_size_luma) * DIV_ROUND_UP(ctx->src_fmt.height, ctb_size_luma) * CEDRUS_H265_MV_COL_BUF_UNIT_CTB_SIZE + SZ_1K; - ctx->codec.h265.mv_col_buf_size = num_buffers * - ctx->codec.h265.mv_col_buf_unit_size; - /* Buffer is never accessed by CPU, so we can skip kernel mapping. */ - ctx->codec.h265.mv_col_buf = + cedrus_buf->codec.h265.mv_col_buf = dma_alloc_attrs(dev->dev, - ctx->codec.h265.mv_col_buf_size, - &ctx->codec.h265.mv_col_buf_addr, + cedrus_buf->codec.h265.mv_col_buf_size, + &cedrus_buf->codec.h265.mv_col_buf_dma, GFP_KERNEL, DMA_ATTR_NO_KERNEL_MAPPING); - if (!ctx->codec.h265.mv_col_buf) { - ctx->codec.h265.mv_col_buf_size = 0; + if (!cedrus_buf->codec.h265.mv_col_buf) { + cedrus_buf->codec.h265.mv_col_buf_size = 0; return -ENOMEM; } } @@ -520,21 +541,22 @@ static int cedrus_h265_setup(struct cedrus_ctx *ctx, struct cedrus_run *run) if (slice_params->data_byte_offset == 0) return -EOPNOTSUPP; - padding = (u8 *)vb2_plane_vaddr(&run->src->vb2_buf, 0) + - slice_params->data_byte_offset - 1; + cedrus_h265_skip_bits(dev, (slice_params->data_byte_offset - 1) * 8); + + padding = cedrus_h265_show_bits(dev, 8); /* at least one bit must be set in that byte */ - if (*padding == 0) + if (padding == 0) return -EINVAL; for (count = 0; count < 8; count++) - if (*padding & (1 << count)) + if (padding & (1 << count)) break; /* Include the one bit. */ count++; - cedrus_h265_skip_bits(dev, slice_params->data_byte_offset * 8 - count); + cedrus_h265_skip_bits(dev, 8 - count); /* Bitstream parameters. */ @@ -793,6 +815,18 @@ static int cedrus_h265_setup(struct cedrus_ctx *ctx, struct cedrus_run *run) VE_DEC_H265_SRAM_OFFSET_PRED_WEIGHT_CHROMA_L1); } + if (ctx->bit_depth > 8) { + unsigned int stride = ALIGN(ctx->dst_fmt.width / 4, 32); + + reg = ctx->dst_fmt.sizeimage - + cedrus_h265_2bit_size(ctx->dst_fmt.width, + ctx->dst_fmt.height); + cedrus_write(dev, VE_DEC_H265_OFFSET_ADDR_FIRST_OUT, reg); + + reg = VE_DEC_H265_10BIT_CONFIGURE_FIRST_2BIT_STRIDE(stride); + cedrus_write(dev, VE_DEC_H265_10BIT_CONFIGURE, reg); + } + /* Enable appropriate interruptions. */ cedrus_write(dev, VE_DEC_H265_CTRL, VE_DEC_H265_CTRL_IRQ_MASK); @@ -803,9 +837,6 @@ static int cedrus_h265_start(struct cedrus_ctx *ctx) { struct cedrus_dev *dev = ctx->dev; - /* The buffer size is calculated at setup time. */ - ctx->codec.h265.mv_col_buf_size = 0; - /* Buffer is never accessed by CPU, so we can skip kernel mapping. */ ctx->codec.h265.neighbor_info_buf = dma_alloc_attrs(dev->dev, CEDRUS_H265_NEIGHBOR_INFO_BUF_SIZE, @@ -832,14 +863,24 @@ static int cedrus_h265_start(struct cedrus_ctx *ctx) static void cedrus_h265_stop(struct cedrus_ctx *ctx) { struct cedrus_dev *dev = ctx->dev; + struct cedrus_buffer *buf; + struct vb2_queue *vq; + unsigned int i; - if (ctx->codec.h265.mv_col_buf_size > 0) { - dma_free_attrs(dev->dev, ctx->codec.h265.mv_col_buf_size, - ctx->codec.h265.mv_col_buf, - ctx->codec.h265.mv_col_buf_addr, - DMA_ATTR_NO_KERNEL_MAPPING); + vq = v4l2_m2m_get_vq(ctx->fh.m2m_ctx, V4L2_BUF_TYPE_VIDEO_CAPTURE); + + for (i = 0; i < vq->num_buffers; i++) { + buf = vb2_to_cedrus_buffer(vb2_get_buffer(vq, i)); - ctx->codec.h265.mv_col_buf_size = 0; + if (buf->codec.h265.mv_col_buf_size > 0) { + dma_free_attrs(dev->dev, + buf->codec.h265.mv_col_buf_size, + buf->codec.h265.mv_col_buf, + buf->codec.h265.mv_col_buf_dma, + DMA_ATTR_NO_KERNEL_MAPPING); + + buf->codec.h265.mv_col_buf_size = 0; + } } dma_free_attrs(dev->dev, CEDRUS_H265_NEIGHBOR_INFO_BUF_SIZE, @@ -858,6 +899,15 @@ static void cedrus_h265_trigger(struct cedrus_ctx *ctx) cedrus_write(dev, VE_DEC_H265_TRIGGER, VE_DEC_H265_TRIGGER_DEC_SLICE); } +static unsigned int cedrus_h265_extra_cap_size(struct cedrus_ctx *ctx, + struct v4l2_pix_format *pix_fmt) +{ + if (ctx->bit_depth > 8) + return cedrus_h265_2bit_size(pix_fmt->width, pix_fmt->height); + + return 0; +} + struct cedrus_dec_ops cedrus_dec_ops_h265 = { .irq_clear = cedrus_h265_irq_clear, .irq_disable = cedrus_h265_irq_disable, @@ -866,4 +916,5 @@ struct cedrus_dec_ops cedrus_dec_ops_h265 = { .start = cedrus_h265_start, .stop = cedrus_h265_stop, .trigger = cedrus_h265_trigger, + .extra_cap_size = cedrus_h265_extra_cap_size, }; diff --git a/drivers/staging/media/sunxi/cedrus/cedrus_regs.h b/drivers/staging/media/sunxi/cedrus/cedrus_regs.h index d81f7513ade0..05e6cbc548ab 100644 --- a/drivers/staging/media/sunxi/cedrus/cedrus_regs.h +++ b/drivers/staging/media/sunxi/cedrus/cedrus_regs.h @@ -498,6 +498,22 @@ #define VE_DEC_H265_LOW_ADDR (VE_ENGINE_DEC_H265 + 0x80) +#define VE_DEC_H265_OFFSET_ADDR_FIRST_OUT (VE_ENGINE_DEC_H265 + 0x84) +#define VE_DEC_H265_OFFSET_ADDR_SECOND_OUT (VE_ENGINE_DEC_H265 + 0x88) + +#define VE_DEC_H265_SECOND_OUT_FMT_8BIT_PLUS_2BIT 0 +#define VE_DEC_H265_SECOND_OUT_FMT_P010 1 +#define VE_DEC_H265_SECOND_OUT_FMT_10BIT_4x4_TILED 2 + +#define VE_DEC_H265_10BIT_CONFIGURE_SECOND_OUT_FMT(v) \ + SHIFT_AND_MASK_BITS(v, 24, 23) +#define VE_DEC_H265_10BIT_CONFIGURE_SECOND_2BIT_ENABLE BIT(22) +#define VE_DEC_H265_10BIT_CONFIGURE_SECOND_2BIT_STRIDE(v) \ + SHIFT_AND_MASK_BITS(v, 21, 11) +#define VE_DEC_H265_10BIT_CONFIGURE_FIRST_2BIT_STRIDE(v) \ + SHIFT_AND_MASK_BITS(v, 10, 0) +#define VE_DEC_H265_10BIT_CONFIGURE (VE_ENGINE_DEC_H265 + 0x8c) + #define VE_DEC_H265_LOW_ADDR_PRIMARY_CHROMA(a) \ SHIFT_AND_MASK_BITS(a, 31, 24) #define VE_DEC_H265_LOW_ADDR_SECONDARY_CHROMA(a) \ @@ -505,6 +521,8 @@ #define VE_DEC_H265_LOW_ADDR_ENTRY_POINTS_BUF(a) \ SHIFT_AND_MASK_BITS(a, 7, 0) +#define VE_DEC_H265_BITS_READ (VE_ENGINE_DEC_H265 + 0xdc) + #define VE_DEC_H265_SRAM_OFFSET (VE_ENGINE_DEC_H265 + 0xe0) #define VE_DEC_H265_SRAM_OFFSET_PRED_WEIGHT_LUMA_L0 0x00 diff --git a/drivers/staging/media/sunxi/cedrus/cedrus_video.c b/drivers/staging/media/sunxi/cedrus/cedrus_video.c index e6909be282d3..b00feaf4072c 100644 --- a/drivers/staging/media/sunxi/cedrus/cedrus_video.c +++ b/drivers/staging/media/sunxi/cedrus/cedrus_video.c @@ -250,6 +250,10 @@ static int cedrus_try_fmt_vid_cap_p(struct cedrus_ctx *ctx, pix_fmt->height = ctx->src_fmt.height; cedrus_prepare_format(pix_fmt); + if (ctx->current_codec->extra_cap_size) + pix_fmt->sizeimage += + ctx->current_codec->extra_cap_size(ctx, pix_fmt); + return 0; } @@ -558,7 +562,7 @@ static void cedrus_buf_request_complete(struct vb2_buffer *vb) v4l2_ctrl_request_complete(vb->req_obj.req, &ctx->hdl); } -static struct vb2_ops cedrus_qops = { +static const struct vb2_ops cedrus_qops = { .queue_setup = cedrus_queue_setup, .buf_prepare = cedrus_buf_prepare, .buf_queue = cedrus_buf_queue, diff --git a/drivers/staging/media/sunxi/sun6i-isp/Kconfig b/drivers/staging/media/sunxi/sun6i-isp/Kconfig new file mode 100644 index 000000000000..68dcae9cd7d7 --- /dev/null +++ b/drivers/staging/media/sunxi/sun6i-isp/Kconfig @@ -0,0 +1,15 @@ +# SPDX-License-Identifier: GPL-2.0-only +config VIDEO_SUN6I_ISP + tristate "Allwinner A31 Image Signal Processor (ISP) Driver" + depends on V4L_PLATFORM_DRIVERS && VIDEO_DEV + depends on ARCH_SUNXI || COMPILE_TEST + depends on PM && COMMON_CLK && HAS_DMA + select MEDIA_CONTROLLER + select VIDEO_V4L2_SUBDEV_API + select VIDEOBUF2_DMA_CONTIG + select VIDEOBUF2_VMALLOC + select V4L2_FWNODE + select REGMAP_MMIO + help + Support for the Allwinner A31 Image Signal Processor (ISP), also + found on other platforms such as the A80, A83T or V3/V3s. diff --git a/drivers/staging/media/sunxi/sun6i-isp/Makefile b/drivers/staging/media/sunxi/sun6i-isp/Makefile new file mode 100644 index 000000000000..da1034785144 --- /dev/null +++ b/drivers/staging/media/sunxi/sun6i-isp/Makefile @@ -0,0 +1,4 @@ +# SPDX-License-Identifier: GPL-2.0-only +sun6i-isp-y += sun6i_isp.o sun6i_isp_proc.o sun6i_isp_capture.o sun6i_isp_params.o + +obj-$(CONFIG_VIDEO_SUN6I_ISP) += sun6i-isp.o diff --git a/drivers/staging/media/sunxi/sun6i-isp/TODO.txt b/drivers/staging/media/sunxi/sun6i-isp/TODO.txt new file mode 100644 index 000000000000..1e3236edc1ab --- /dev/null +++ b/drivers/staging/media/sunxi/sun6i-isp/TODO.txt @@ -0,0 +1,6 @@ +Unstaging requirements: +- Add uAPI support and documentation for the configuration of all the hardware + modules and description of the statistics data structures; +- Add support for statistics reporting; +- Add userspace support in libcamera which demonstrates the ability to receive + statistics and adapt hardware modules configuration accordingly; diff --git a/drivers/staging/media/sunxi/sun6i-isp/sun6i_isp.c b/drivers/staging/media/sunxi/sun6i-isp/sun6i_isp.c new file mode 100644 index 000000000000..7b7947509b69 --- /dev/null +++ b/drivers/staging/media/sunxi/sun6i-isp/sun6i_isp.c @@ -0,0 +1,555 @@ +// SPDX-License-Identifier: GPL-2.0+ +/* + * Copyright 2021-2022 Bootlin + * Author: Paul Kocialkowski <[email protected]> + */ + +#include <linux/clk.h> +#include <linux/dma-mapping.h> +#include <linux/err.h> +#include <linux/interrupt.h> +#include <linux/module.h> +#include <linux/of.h> +#include <linux/of_device.h> +#include <linux/platform_device.h> +#include <linux/pm_runtime.h> +#include <linux/regmap.h> +#include <linux/reset.h> +#include <media/v4l2-device.h> +#include <media/v4l2-mc.h> + +#include "sun6i_isp.h" +#include "sun6i_isp_capture.h" +#include "sun6i_isp_params.h" +#include "sun6i_isp_proc.h" +#include "sun6i_isp_reg.h" + +/* Helpers */ + +u32 sun6i_isp_load_read(struct sun6i_isp_device *isp_dev, u32 offset) +{ + u32 *data = (u32 *)(isp_dev->tables.load.data + offset); + + return *data; +} + +void sun6i_isp_load_write(struct sun6i_isp_device *isp_dev, u32 offset, + u32 value) +{ + u32 *data = (u32 *)(isp_dev->tables.load.data + offset); + + *data = value; +} + +/* State */ + +/* + * The ISP works with a load buffer, which gets copied to the actual registers + * by the hardware before processing a frame when a specific flag is set. + * This is represented by tracking the ISP state in the different parts of + * the code with explicit sync points: + * - state update: to update the load buffer for the next frame if necessary; + * - state complete: to indicate that the state update was applied. + */ + +static void sun6i_isp_state_ready(struct sun6i_isp_device *isp_dev) +{ + struct regmap *regmap = isp_dev->regmap; + u32 value; + + regmap_read(regmap, SUN6I_ISP_FE_CTRL_REG, &value); + value |= SUN6I_ISP_FE_CTRL_PARA_READY; + regmap_write(regmap, SUN6I_ISP_FE_CTRL_REG, value); +} + +static void sun6i_isp_state_complete(struct sun6i_isp_device *isp_dev) +{ + unsigned long flags; + + spin_lock_irqsave(&isp_dev->state_lock, flags); + + sun6i_isp_capture_state_complete(isp_dev); + sun6i_isp_params_state_complete(isp_dev); + + spin_unlock_irqrestore(&isp_dev->state_lock, flags); +} + +void sun6i_isp_state_update(struct sun6i_isp_device *isp_dev, bool ready_hold) +{ + bool update = false; + unsigned long flags; + + spin_lock_irqsave(&isp_dev->state_lock, flags); + + sun6i_isp_capture_state_update(isp_dev, &update); + sun6i_isp_params_state_update(isp_dev, &update); + + if (update && !ready_hold) + sun6i_isp_state_ready(isp_dev); + + spin_unlock_irqrestore(&isp_dev->state_lock, flags); +} + +/* Tables */ + +static int sun6i_isp_table_setup(struct sun6i_isp_device *isp_dev, + struct sun6i_isp_table *table) +{ + table->data = dma_alloc_coherent(isp_dev->dev, table->size, + &table->address, GFP_KERNEL); + if (!table->data) + return -ENOMEM; + + return 0; +} + +static void sun6i_isp_table_cleanup(struct sun6i_isp_device *isp_dev, + struct sun6i_isp_table *table) +{ + dma_free_coherent(isp_dev->dev, table->size, table->data, + table->address); +} + +void sun6i_isp_tables_configure(struct sun6i_isp_device *isp_dev) +{ + struct regmap *regmap = isp_dev->regmap; + + regmap_write(regmap, SUN6I_ISP_REG_LOAD_ADDR_REG, + SUN6I_ISP_ADDR_VALUE(isp_dev->tables.load.address)); + + regmap_write(regmap, SUN6I_ISP_REG_SAVE_ADDR_REG, + SUN6I_ISP_ADDR_VALUE(isp_dev->tables.save.address)); + + regmap_write(regmap, SUN6I_ISP_LUT_TABLE_ADDR_REG, + SUN6I_ISP_ADDR_VALUE(isp_dev->tables.lut.address)); + + regmap_write(regmap, SUN6I_ISP_DRC_TABLE_ADDR_REG, + SUN6I_ISP_ADDR_VALUE(isp_dev->tables.drc.address)); + + regmap_write(regmap, SUN6I_ISP_STATS_ADDR_REG, + SUN6I_ISP_ADDR_VALUE(isp_dev->tables.stats.address)); +} + +static int sun6i_isp_tables_setup(struct sun6i_isp_device *isp_dev, + const struct sun6i_isp_variant *variant) +{ + struct sun6i_isp_tables *tables = &isp_dev->tables; + int ret; + + tables->load.size = variant->table_load_save_size; + ret = sun6i_isp_table_setup(isp_dev, &tables->load); + if (ret) + return ret; + + tables->save.size = variant->table_load_save_size; + ret = sun6i_isp_table_setup(isp_dev, &tables->save); + if (ret) + return ret; + + tables->lut.size = variant->table_lut_size; + ret = sun6i_isp_table_setup(isp_dev, &tables->lut); + if (ret) + return ret; + + tables->drc.size = variant->table_drc_size; + ret = sun6i_isp_table_setup(isp_dev, &tables->drc); + if (ret) + return ret; + + tables->stats.size = variant->table_stats_size; + ret = sun6i_isp_table_setup(isp_dev, &tables->stats); + if (ret) + return ret; + + return 0; +} + +static void sun6i_isp_tables_cleanup(struct sun6i_isp_device *isp_dev) +{ + struct sun6i_isp_tables *tables = &isp_dev->tables; + + sun6i_isp_table_cleanup(isp_dev, &tables->stats); + sun6i_isp_table_cleanup(isp_dev, &tables->drc); + sun6i_isp_table_cleanup(isp_dev, &tables->lut); + sun6i_isp_table_cleanup(isp_dev, &tables->save); + sun6i_isp_table_cleanup(isp_dev, &tables->load); +} + +/* Media */ + +static const struct media_device_ops sun6i_isp_media_ops = { + .link_notify = v4l2_pipeline_link_notify, +}; + +/* V4L2 */ + +static int sun6i_isp_v4l2_setup(struct sun6i_isp_device *isp_dev) +{ + struct sun6i_isp_v4l2 *v4l2 = &isp_dev->v4l2; + struct v4l2_device *v4l2_dev = &v4l2->v4l2_dev; + struct media_device *media_dev = &v4l2->media_dev; + struct device *dev = isp_dev->dev; + int ret; + + /* Media Device */ + + strscpy(media_dev->model, SUN6I_ISP_DESCRIPTION, + sizeof(media_dev->model)); + media_dev->ops = &sun6i_isp_media_ops; + media_dev->hw_revision = 0; + media_dev->dev = dev; + + media_device_init(media_dev); + + ret = media_device_register(media_dev); + if (ret) { + dev_err(dev, "failed to register media device\n"); + return ret; + } + + /* V4L2 Device */ + + v4l2_dev->mdev = media_dev; + + ret = v4l2_device_register(dev, v4l2_dev); + if (ret) { + dev_err(dev, "failed to register v4l2 device\n"); + goto error_media; + } + + return 0; + +error_media: + media_device_unregister(media_dev); + media_device_cleanup(media_dev); + + return ret; +} + +static void sun6i_isp_v4l2_cleanup(struct sun6i_isp_device *isp_dev) +{ + struct sun6i_isp_v4l2 *v4l2 = &isp_dev->v4l2; + + media_device_unregister(&v4l2->media_dev); + v4l2_device_unregister(&v4l2->v4l2_dev); + media_device_cleanup(&v4l2->media_dev); +} + +/* Platform */ + +static irqreturn_t sun6i_isp_interrupt(int irq, void *private) +{ + struct sun6i_isp_device *isp_dev = private; + struct regmap *regmap = isp_dev->regmap; + u32 status = 0, enable = 0; + + regmap_read(regmap, SUN6I_ISP_FE_INT_STA_REG, &status); + regmap_read(regmap, SUN6I_ISP_FE_INT_EN_REG, &enable); + + if (!status) + return IRQ_NONE; + else if (!(status & enable)) + goto complete; + + /* + * The ISP working cycle starts with a params-load, which makes the + * state from the load buffer active. Then it starts processing the + * frame and gives a finish interrupt. Soon after that, the next state + * coming from the load buffer will be applied for the next frame, + * giving a params-load as well. + * + * Because both frame finish and params-load are received almost + * at the same time (one ISR call), handle them in chronology order. + */ + + if (status & SUN6I_ISP_FE_INT_STA_FINISH) + sun6i_isp_capture_finish(isp_dev); + + if (status & SUN6I_ISP_FE_INT_STA_PARA_LOAD) { + sun6i_isp_state_complete(isp_dev); + sun6i_isp_state_update(isp_dev, false); + } + +complete: + regmap_write(regmap, SUN6I_ISP_FE_INT_STA_REG, status); + + return IRQ_HANDLED; +} + +static int sun6i_isp_suspend(struct device *dev) +{ + struct sun6i_isp_device *isp_dev = dev_get_drvdata(dev); + + reset_control_assert(isp_dev->reset); + clk_disable_unprepare(isp_dev->clock_ram); + clk_disable_unprepare(isp_dev->clock_mod); + + return 0; +} + +static int sun6i_isp_resume(struct device *dev) +{ + struct sun6i_isp_device *isp_dev = dev_get_drvdata(dev); + int ret; + + ret = reset_control_deassert(isp_dev->reset); + if (ret) { + dev_err(dev, "failed to deassert reset\n"); + return ret; + } + + ret = clk_prepare_enable(isp_dev->clock_mod); + if (ret) { + dev_err(dev, "failed to enable module clock\n"); + goto error_reset; + } + + ret = clk_prepare_enable(isp_dev->clock_ram); + if (ret) { + dev_err(dev, "failed to enable ram clock\n"); + goto error_clock_mod; + } + + return 0; + +error_clock_mod: + clk_disable_unprepare(isp_dev->clock_mod); + +error_reset: + reset_control_assert(isp_dev->reset); + + return ret; +} + +static const struct dev_pm_ops sun6i_isp_pm_ops = { + .runtime_suspend = sun6i_isp_suspend, + .runtime_resume = sun6i_isp_resume, +}; + +static const struct regmap_config sun6i_isp_regmap_config = { + .reg_bits = 32, + .reg_stride = 4, + .val_bits = 32, + .max_register = 0x400, +}; + +static int sun6i_isp_resources_setup(struct sun6i_isp_device *isp_dev, + struct platform_device *platform_dev) +{ + struct device *dev = isp_dev->dev; + void __iomem *io_base; + int irq; + int ret; + + /* Registers */ + + io_base = devm_platform_ioremap_resource(platform_dev, 0); + if (IS_ERR(io_base)) + return PTR_ERR(io_base); + + isp_dev->regmap = devm_regmap_init_mmio_clk(dev, "bus", io_base, + &sun6i_isp_regmap_config); + if (IS_ERR(isp_dev->regmap)) { + dev_err(dev, "failed to init register map\n"); + return PTR_ERR(isp_dev->regmap); + } + + /* Clocks */ + + isp_dev->clock_mod = devm_clk_get(dev, "mod"); + if (IS_ERR(isp_dev->clock_mod)) { + dev_err(dev, "failed to acquire module clock\n"); + return PTR_ERR(isp_dev->clock_mod); + } + + isp_dev->clock_ram = devm_clk_get(dev, "ram"); + if (IS_ERR(isp_dev->clock_ram)) { + dev_err(dev, "failed to acquire ram clock\n"); + return PTR_ERR(isp_dev->clock_ram); + } + + ret = clk_set_rate_exclusive(isp_dev->clock_mod, 297000000); + if (ret) { + dev_err(dev, "failed to set mod clock rate\n"); + return ret; + } + + /* Reset */ + + isp_dev->reset = devm_reset_control_get_shared(dev, NULL); + if (IS_ERR(isp_dev->reset)) { + dev_err(dev, "failed to acquire reset\n"); + ret = PTR_ERR(isp_dev->reset); + goto error_clock_rate_exclusive; + } + + /* Interrupt */ + + irq = platform_get_irq(platform_dev, 0); + if (irq < 0) { + dev_err(dev, "failed to get interrupt\n"); + ret = -ENXIO; + goto error_clock_rate_exclusive; + } + + ret = devm_request_irq(dev, irq, sun6i_isp_interrupt, IRQF_SHARED, + SUN6I_ISP_NAME, isp_dev); + if (ret) { + dev_err(dev, "failed to request interrupt\n"); + goto error_clock_rate_exclusive; + } + + /* Runtime PM */ + + pm_runtime_enable(dev); + + return 0; + +error_clock_rate_exclusive: + clk_rate_exclusive_put(isp_dev->clock_mod); + + return ret; +} + +static void sun6i_isp_resources_cleanup(struct sun6i_isp_device *isp_dev) +{ + struct device *dev = isp_dev->dev; + + pm_runtime_disable(dev); + clk_rate_exclusive_put(isp_dev->clock_mod); +} + +static int sun6i_isp_probe(struct platform_device *platform_dev) +{ + struct sun6i_isp_device *isp_dev; + struct device *dev = &platform_dev->dev; + const struct sun6i_isp_variant *variant; + int ret; + + variant = of_device_get_match_data(dev); + if (!variant) + return -EINVAL; + + isp_dev = devm_kzalloc(dev, sizeof(*isp_dev), GFP_KERNEL); + if (!isp_dev) + return -ENOMEM; + + isp_dev->dev = dev; + platform_set_drvdata(platform_dev, isp_dev); + + spin_lock_init(&isp_dev->state_lock); + + ret = sun6i_isp_resources_setup(isp_dev, platform_dev); + if (ret) + return ret; + + ret = sun6i_isp_tables_setup(isp_dev, variant); + if (ret) { + dev_err(dev, "failed to setup tables\n"); + goto error_resources; + } + + ret = sun6i_isp_v4l2_setup(isp_dev); + if (ret) { + dev_err(dev, "failed to setup v4l2\n"); + goto error_tables; + } + + ret = sun6i_isp_proc_setup(isp_dev); + if (ret) { + dev_err(dev, "failed to setup proc\n"); + goto error_v4l2; + } + + ret = sun6i_isp_capture_setup(isp_dev); + if (ret) { + dev_err(dev, "failed to setup capture\n"); + goto error_proc; + } + + ret = sun6i_isp_params_setup(isp_dev); + if (ret) { + dev_err(dev, "failed to setup params\n"); + goto error_capture; + } + + return 0; + +error_capture: + sun6i_isp_capture_cleanup(isp_dev); + +error_proc: + sun6i_isp_proc_cleanup(isp_dev); + +error_v4l2: + sun6i_isp_v4l2_cleanup(isp_dev); + +error_tables: + sun6i_isp_tables_cleanup(isp_dev); + +error_resources: + sun6i_isp_resources_cleanup(isp_dev); + + return ret; +} + +static int sun6i_isp_remove(struct platform_device *platform_dev) +{ + struct sun6i_isp_device *isp_dev = platform_get_drvdata(platform_dev); + + sun6i_isp_params_cleanup(isp_dev); + sun6i_isp_capture_cleanup(isp_dev); + sun6i_isp_proc_cleanup(isp_dev); + sun6i_isp_v4l2_cleanup(isp_dev); + sun6i_isp_tables_cleanup(isp_dev); + sun6i_isp_resources_cleanup(isp_dev); + + return 0; +} + +/* + * History of sun6i-isp: + * - sun4i-a10-isp: initial ISP tied to the CSI0 controller, + * apparently unused in software implementations; + * - sun6i-a31-isp: separate ISP loosely based on sun4i-a10-isp, + * adding extra modules and features; + * - sun9i-a80-isp: based on sun6i-a31-isp with some register offset changes + * and new modules like saturation and cnr; + * - sun8i-a23-isp/sun8i-h3-isp: based on sun9i-a80-isp with most modules + * related to raw removed; + * - sun8i-a83t-isp: based on sun9i-a80-isp with some register offset changes + * - sun8i-v3s-isp: based on sun8i-a83t-isp with a new disc module; + */ + +static const struct sun6i_isp_variant sun8i_v3s_isp_variant = { + .table_load_save_size = 0x1000, + .table_lut_size = 0xe00, + .table_drc_size = 0x600, + .table_stats_size = 0x2100, +}; + +static const struct of_device_id sun6i_isp_of_match[] = { + { + .compatible = "allwinner,sun8i-v3s-isp", + .data = &sun8i_v3s_isp_variant, + }, + {}, +}; + +MODULE_DEVICE_TABLE(of, sun6i_isp_of_match); + +static struct platform_driver sun6i_isp_platform_driver = { + .probe = sun6i_isp_probe, + .remove = sun6i_isp_remove, + .driver = { + .name = SUN6I_ISP_NAME, + .of_match_table = of_match_ptr(sun6i_isp_of_match), + .pm = &sun6i_isp_pm_ops, + }, +}; + +module_platform_driver(sun6i_isp_platform_driver); + +MODULE_DESCRIPTION("Allwinner A31 Image Signal Processor driver"); +MODULE_AUTHOR("Paul Kocialkowski <[email protected]>"); +MODULE_LICENSE("GPL"); diff --git a/drivers/staging/media/sunxi/sun6i-isp/sun6i_isp.h b/drivers/staging/media/sunxi/sun6i-isp/sun6i_isp.h new file mode 100644 index 000000000000..0e5f188319ff --- /dev/null +++ b/drivers/staging/media/sunxi/sun6i-isp/sun6i_isp.h @@ -0,0 +1,90 @@ +/* SPDX-License-Identifier: GPL-2.0+ */ +/* + * Copyright 2021-2022 Bootlin + * Author: Paul Kocialkowski <[email protected]> + */ + +#ifndef _SUN6I_ISP_H_ +#define _SUN6I_ISP_H_ + +#include <media/v4l2-device.h> +#include <media/videobuf2-v4l2.h> + +#include "sun6i_isp_capture.h" +#include "sun6i_isp_params.h" +#include "sun6i_isp_proc.h" + +#define SUN6I_ISP_NAME "sun6i-isp" +#define SUN6I_ISP_DESCRIPTION "Allwinner A31 ISP Device" + +enum sun6i_isp_port { + SUN6I_ISP_PORT_CSI0 = 0, + SUN6I_ISP_PORT_CSI1 = 1, +}; + +struct sun6i_isp_buffer { + struct vb2_v4l2_buffer v4l2_buffer; + struct list_head list; +}; + +struct sun6i_isp_v4l2 { + struct v4l2_device v4l2_dev; + struct media_device media_dev; +}; + +struct sun6i_isp_table { + void *data; + dma_addr_t address; + unsigned int size; +}; + +struct sun6i_isp_tables { + struct sun6i_isp_table load; + struct sun6i_isp_table save; + + struct sun6i_isp_table lut; + struct sun6i_isp_table drc; + struct sun6i_isp_table stats; +}; + +struct sun6i_isp_device { + struct device *dev; + + struct sun6i_isp_tables tables; + + struct sun6i_isp_v4l2 v4l2; + struct sun6i_isp_proc proc; + struct sun6i_isp_capture capture; + struct sun6i_isp_params params; + + struct regmap *regmap; + struct clk *clock_mod; + struct clk *clock_ram; + struct reset_control *reset; + + spinlock_t state_lock; /* State helpers lock. */ +}; + +struct sun6i_isp_variant { + unsigned int table_load_save_size; + unsigned int table_lut_size; + unsigned int table_drc_size; + unsigned int table_stats_size; +}; + +/* Helpers */ + +u32 sun6i_isp_load_read(struct sun6i_isp_device *isp_dev, u32 offset); +void sun6i_isp_load_write(struct sun6i_isp_device *isp_dev, u32 offset, + u32 value); +u32 sun6i_isp_address_value(dma_addr_t address); + +/* State */ + +void sun6i_isp_state_update(struct sun6i_isp_device *isp_dev, bool ready_hold); + +/* Tables */ + +void sun6i_isp_tables_configure(struct sun6i_isp_device *isp_dev); + +#endif diff --git a/drivers/staging/media/sunxi/sun6i-isp/sun6i_isp_capture.c b/drivers/staging/media/sunxi/sun6i-isp/sun6i_isp_capture.c new file mode 100644 index 000000000000..4b592820845a --- /dev/null +++ b/drivers/staging/media/sunxi/sun6i-isp/sun6i_isp_capture.c @@ -0,0 +1,742 @@ +// SPDX-License-Identifier: GPL-2.0+ +/* + * Copyright 2021-2022 Bootlin + * Author: Paul Kocialkowski <[email protected]> + */ + +#include <media/v4l2-device.h> +#include <media/v4l2-event.h> +#include <media/v4l2-ioctl.h> +#include <media/v4l2-mc.h> +#include <media/videobuf2-dma-contig.h> +#include <media/videobuf2-v4l2.h> + +#include "sun6i_isp.h" +#include "sun6i_isp_capture.h" +#include "sun6i_isp_proc.h" +#include "sun6i_isp_reg.h" + +/* Helpers */ + +void sun6i_isp_capture_dimensions(struct sun6i_isp_device *isp_dev, + unsigned int *width, unsigned int *height) +{ + if (width) + *width = isp_dev->capture.format.fmt.pix.width; + if (height) + *height = isp_dev->capture.format.fmt.pix.height; +} + +void sun6i_isp_capture_format(struct sun6i_isp_device *isp_dev, + u32 *pixelformat) +{ + if (pixelformat) + *pixelformat = isp_dev->capture.format.fmt.pix.pixelformat; +} + +/* Format */ + +static const struct sun6i_isp_capture_format sun6i_isp_capture_formats[] = { + { + .pixelformat = V4L2_PIX_FMT_NV12, + .output_format = SUN6I_ISP_OUTPUT_FMT_YUV420SP, + }, + { + .pixelformat = V4L2_PIX_FMT_NV21, + .output_format = SUN6I_ISP_OUTPUT_FMT_YVU420SP, + }, +}; + +const struct sun6i_isp_capture_format * +sun6i_isp_capture_format_find(u32 pixelformat) +{ + unsigned int i; + + for (i = 0; i < ARRAY_SIZE(sun6i_isp_capture_formats); i++) + if (sun6i_isp_capture_formats[i].pixelformat == pixelformat) + return &sun6i_isp_capture_formats[i]; + + return NULL; +} + +/* Capture */ + +static void +sun6i_isp_capture_buffer_configure(struct sun6i_isp_device *isp_dev, + struct sun6i_isp_buffer *isp_buffer) +{ + const struct v4l2_format_info *info; + struct vb2_buffer *vb2_buffer; + unsigned int width, height; + unsigned int width_aligned; + dma_addr_t address; + u32 pixelformat; + + vb2_buffer = &isp_buffer->v4l2_buffer.vb2_buf; + address = vb2_dma_contig_plane_dma_addr(vb2_buffer, 0); + + sun6i_isp_load_write(isp_dev, SUN6I_ISP_MCH_Y_ADDR0_REG, + SUN6I_ISP_ADDR_VALUE(address)); + + sun6i_isp_capture_dimensions(isp_dev, &width, &height); + sun6i_isp_capture_format(isp_dev, &pixelformat); + + info = v4l2_format_info(pixelformat); + if (WARN_ON(!info)) + return; + + /* Stride needs to be aligned to 4. */ + width_aligned = ALIGN(width, 2); + + if (info->comp_planes > 1) { + address += info->bpp[0] * width_aligned * height; + + sun6i_isp_load_write(isp_dev, SUN6I_ISP_MCH_U_ADDR0_REG, + SUN6I_ISP_ADDR_VALUE(address)); + } + + if (info->comp_planes > 2) { + address += info->bpp[1] * + DIV_ROUND_UP(width_aligned, info->hdiv) * + DIV_ROUND_UP(height, info->vdiv); + + sun6i_isp_load_write(isp_dev, SUN6I_ISP_MCH_V_ADDR0_REG, + SUN6I_ISP_ADDR_VALUE(address)); + } +} + +void sun6i_isp_capture_configure(struct sun6i_isp_device *isp_dev) +{ + unsigned int width, height; + unsigned int stride_luma, stride_chroma = 0; + unsigned int stride_luma_div4, stride_chroma_div4; + const struct sun6i_isp_capture_format *format; + const struct v4l2_format_info *info; + u32 pixelformat; + + sun6i_isp_capture_dimensions(isp_dev, &width, &height); + sun6i_isp_capture_format(isp_dev, &pixelformat); + + format = sun6i_isp_capture_format_find(pixelformat); + if (WARN_ON(!format)) + return; + + sun6i_isp_load_write(isp_dev, SUN6I_ISP_MCH_SIZE_CFG_REG, + SUN6I_ISP_MCH_SIZE_CFG_WIDTH(width) | + SUN6I_ISP_MCH_SIZE_CFG_HEIGHT(height)); + + info = v4l2_format_info(pixelformat); + if (WARN_ON(!info)) + return; + + stride_luma = width * info->bpp[0]; + stride_luma_div4 = DIV_ROUND_UP(stride_luma, 4); + + if (info->comp_planes > 1) { + stride_chroma = width * info->bpp[1] / info->hdiv; + stride_chroma_div4 = DIV_ROUND_UP(stride_chroma, 4); + } + + sun6i_isp_load_write(isp_dev, SUN6I_ISP_MCH_CFG_REG, + SUN6I_ISP_MCH_CFG_EN | + SUN6I_ISP_MCH_CFG_OUTPUT_FMT(format->output_format) | + SUN6I_ISP_MCH_CFG_STRIDE_Y_DIV4(stride_luma_div4) | + SUN6I_ISP_MCH_CFG_STRIDE_UV_DIV4(stride_chroma_div4)); +} + +/* State */ + +static void sun6i_isp_capture_state_cleanup(struct sun6i_isp_device *isp_dev, + bool error) +{ + struct sun6i_isp_capture_state *state = &isp_dev->capture.state; + struct sun6i_isp_buffer **isp_buffer_states[] = { + &state->pending, &state->current, &state->complete, + }; + struct sun6i_isp_buffer *isp_buffer; + struct vb2_buffer *vb2_buffer; + unsigned long flags; + unsigned int i; + + spin_lock_irqsave(&state->lock, flags); + + for (i = 0; i < ARRAY_SIZE(isp_buffer_states); i++) { + isp_buffer = *isp_buffer_states[i]; + if (!isp_buffer) + continue; + + vb2_buffer = &isp_buffer->v4l2_buffer.vb2_buf; + vb2_buffer_done(vb2_buffer, error ? VB2_BUF_STATE_ERROR : + VB2_BUF_STATE_QUEUED); + + *isp_buffer_states[i] = NULL; + } + + list_for_each_entry(isp_buffer, &state->queue, list) { + vb2_buffer = &isp_buffer->v4l2_buffer.vb2_buf; + vb2_buffer_done(vb2_buffer, error ? VB2_BUF_STATE_ERROR : + VB2_BUF_STATE_QUEUED); + } + + INIT_LIST_HEAD(&state->queue); + + spin_unlock_irqrestore(&state->lock, flags); +} + +void sun6i_isp_capture_state_update(struct sun6i_isp_device *isp_dev, + bool *update) +{ + struct sun6i_isp_capture_state *state = &isp_dev->capture.state; + struct sun6i_isp_buffer *isp_buffer; + unsigned long flags; + + spin_lock_irqsave(&state->lock, flags); + + if (list_empty(&state->queue)) + goto complete; + + if (state->pending) + goto complete; + + isp_buffer = list_first_entry(&state->queue, struct sun6i_isp_buffer, + list); + + sun6i_isp_capture_buffer_configure(isp_dev, isp_buffer); + + list_del(&isp_buffer->list); + + state->pending = isp_buffer; + + if (update) + *update = true; + +complete: + spin_unlock_irqrestore(&state->lock, flags); +} + +void sun6i_isp_capture_state_complete(struct sun6i_isp_device *isp_dev) +{ + struct sun6i_isp_capture_state *state = &isp_dev->capture.state; + unsigned long flags; + + spin_lock_irqsave(&state->lock, flags); + + if (!state->pending) + goto complete; + + state->complete = state->current; + state->current = state->pending; + state->pending = NULL; + + if (state->complete) { + struct sun6i_isp_buffer *isp_buffer = state->complete; + struct vb2_buffer *vb2_buffer = + &isp_buffer->v4l2_buffer.vb2_buf; + + vb2_buffer->timestamp = ktime_get_ns(); + isp_buffer->v4l2_buffer.sequence = state->sequence; + + vb2_buffer_done(vb2_buffer, VB2_BUF_STATE_DONE); + + state->complete = NULL; + } + +complete: + spin_unlock_irqrestore(&state->lock, flags); +} + +void sun6i_isp_capture_finish(struct sun6i_isp_device *isp_dev) +{ + struct sun6i_isp_capture_state *state = &isp_dev->capture.state; + unsigned long flags; + + spin_lock_irqsave(&state->lock, flags); + state->sequence++; + spin_unlock_irqrestore(&state->lock, flags); +} + +/* Queue */ + +static int sun6i_isp_capture_queue_setup(struct vb2_queue *queue, + unsigned int *buffers_count, + unsigned int *planes_count, + unsigned int sizes[], + struct device *alloc_devs[]) +{ + struct sun6i_isp_device *isp_dev = vb2_get_drv_priv(queue); + unsigned int size = isp_dev->capture.format.fmt.pix.sizeimage; + + if (*planes_count) + return sizes[0] < size ? -EINVAL : 0; + + *planes_count = 1; + sizes[0] = size; + + return 0; +} + +static int sun6i_isp_capture_buffer_prepare(struct vb2_buffer *vb2_buffer) +{ + struct sun6i_isp_device *isp_dev = + vb2_get_drv_priv(vb2_buffer->vb2_queue); + struct v4l2_device *v4l2_dev = &isp_dev->v4l2.v4l2_dev; + unsigned int size = isp_dev->capture.format.fmt.pix.sizeimage; + + if (vb2_plane_size(vb2_buffer, 0) < size) { + v4l2_err(v4l2_dev, "buffer too small (%lu < %u)\n", + vb2_plane_size(vb2_buffer, 0), size); + return -EINVAL; + } + + vb2_set_plane_payload(vb2_buffer, 0, size); + + return 0; +} + +static void sun6i_isp_capture_buffer_queue(struct vb2_buffer *vb2_buffer) +{ + struct sun6i_isp_device *isp_dev = + vb2_get_drv_priv(vb2_buffer->vb2_queue); + struct sun6i_isp_capture_state *state = &isp_dev->capture.state; + struct vb2_v4l2_buffer *v4l2_buffer = to_vb2_v4l2_buffer(vb2_buffer); + struct sun6i_isp_buffer *isp_buffer = + container_of(v4l2_buffer, struct sun6i_isp_buffer, v4l2_buffer); + unsigned long flags; + + spin_lock_irqsave(&state->lock, flags); + list_add_tail(&isp_buffer->list, &state->queue); + spin_unlock_irqrestore(&state->lock, flags); + + /* Update the state to schedule our buffer as soon as possible. */ + if (state->streaming) + sun6i_isp_state_update(isp_dev, false); +} + +static int sun6i_isp_capture_start_streaming(struct vb2_queue *queue, + unsigned int count) +{ + struct sun6i_isp_device *isp_dev = vb2_get_drv_priv(queue); + struct sun6i_isp_capture_state *state = &isp_dev->capture.state; + struct video_device *video_dev = &isp_dev->capture.video_dev; + struct v4l2_subdev *subdev = &isp_dev->proc.subdev; + int ret; + + state->sequence = 0; + + ret = video_device_pipeline_alloc_start(video_dev); + if (ret < 0) + goto error_state; + + state->streaming = true; + + ret = v4l2_subdev_call(subdev, video, s_stream, 1); + if (ret && ret != -ENOIOCTLCMD) + goto error_streaming; + + return 0; + +error_streaming: + state->streaming = false; + + video_device_pipeline_stop(video_dev); + +error_state: + sun6i_isp_capture_state_cleanup(isp_dev, false); + + return ret; +} + +static void sun6i_isp_capture_stop_streaming(struct vb2_queue *queue) +{ + struct sun6i_isp_device *isp_dev = vb2_get_drv_priv(queue); + struct sun6i_isp_capture_state *state = &isp_dev->capture.state; + struct video_device *video_dev = &isp_dev->capture.video_dev; + struct v4l2_subdev *subdev = &isp_dev->proc.subdev; + + v4l2_subdev_call(subdev, video, s_stream, 0); + + state->streaming = false; + + video_device_pipeline_stop(video_dev); + + sun6i_isp_capture_state_cleanup(isp_dev, true); +} + +static const struct vb2_ops sun6i_isp_capture_queue_ops = { + .queue_setup = sun6i_isp_capture_queue_setup, + .buf_prepare = sun6i_isp_capture_buffer_prepare, + .buf_queue = sun6i_isp_capture_buffer_queue, + .start_streaming = sun6i_isp_capture_start_streaming, + .stop_streaming = sun6i_isp_capture_stop_streaming, + .wait_prepare = vb2_ops_wait_prepare, + .wait_finish = vb2_ops_wait_finish, +}; + +/* Video Device */ + +static void sun6i_isp_capture_format_prepare(struct v4l2_format *format) +{ + struct v4l2_pix_format *pix_format = &format->fmt.pix; + const struct v4l2_format_info *info; + unsigned int width, height; + unsigned int width_aligned; + unsigned int i; + + v4l_bound_align_image(&pix_format->width, SUN6I_ISP_CAPTURE_WIDTH_MIN, + SUN6I_ISP_CAPTURE_WIDTH_MAX, 1, + &pix_format->height, SUN6I_ISP_CAPTURE_HEIGHT_MIN, + SUN6I_ISP_CAPTURE_HEIGHT_MAX, 1, 0); + + if (!sun6i_isp_capture_format_find(pix_format->pixelformat)) + pix_format->pixelformat = + sun6i_isp_capture_formats[0].pixelformat; + + info = v4l2_format_info(pix_format->pixelformat); + if (WARN_ON(!info)) + return; + + width = pix_format->width; + height = pix_format->height; + + /* Stride needs to be aligned to 4. */ + width_aligned = ALIGN(width, 2); + + pix_format->bytesperline = width_aligned * info->bpp[0]; + pix_format->sizeimage = 0; + + for (i = 0; i < info->comp_planes; i++) { + unsigned int hdiv = (i == 0) ? 1 : info->hdiv; + unsigned int vdiv = (i == 0) ? 1 : info->vdiv; + + pix_format->sizeimage += info->bpp[i] * + DIV_ROUND_UP(width_aligned, hdiv) * + DIV_ROUND_UP(height, vdiv); + } + + pix_format->field = V4L2_FIELD_NONE; + + pix_format->colorspace = V4L2_COLORSPACE_RAW; + pix_format->ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT; + pix_format->quantization = V4L2_QUANTIZATION_DEFAULT; + pix_format->xfer_func = V4L2_XFER_FUNC_DEFAULT; +} + +static int sun6i_isp_capture_querycap(struct file *file, void *private, + struct v4l2_capability *capability) +{ + struct sun6i_isp_device *isp_dev = video_drvdata(file); + struct video_device *video_dev = &isp_dev->capture.video_dev; + + strscpy(capability->driver, SUN6I_ISP_NAME, sizeof(capability->driver)); + strscpy(capability->card, video_dev->name, sizeof(capability->card)); + snprintf(capability->bus_info, sizeof(capability->bus_info), + "platform:%s", dev_name(isp_dev->dev)); + + return 0; +} + +static int sun6i_isp_capture_enum_fmt(struct file *file, void *private, + struct v4l2_fmtdesc *fmtdesc) +{ + u32 index = fmtdesc->index; + + if (index >= ARRAY_SIZE(sun6i_isp_capture_formats)) + return -EINVAL; + + fmtdesc->pixelformat = sun6i_isp_capture_formats[index].pixelformat; + + return 0; +} + +static int sun6i_isp_capture_g_fmt(struct file *file, void *private, + struct v4l2_format *format) +{ + struct sun6i_isp_device *isp_dev = video_drvdata(file); + + *format = isp_dev->capture.format; + + return 0; +} + +static int sun6i_isp_capture_s_fmt(struct file *file, void *private, + struct v4l2_format *format) +{ + struct sun6i_isp_device *isp_dev = video_drvdata(file); + + if (vb2_is_busy(&isp_dev->capture.queue)) + return -EBUSY; + + sun6i_isp_capture_format_prepare(format); + + isp_dev->capture.format = *format; + + return 0; +} + +static int sun6i_isp_capture_try_fmt(struct file *file, void *private, + struct v4l2_format *format) +{ + sun6i_isp_capture_format_prepare(format); + + return 0; +} + +static int sun6i_isp_capture_enum_input(struct file *file, void *private, + struct v4l2_input *input) +{ + if (input->index != 0) + return -EINVAL; + + input->type = V4L2_INPUT_TYPE_CAMERA; + strscpy(input->name, "Camera", sizeof(input->name)); + + return 0; +} + +static int sun6i_isp_capture_g_input(struct file *file, void *private, + unsigned int *index) +{ + *index = 0; + + return 0; +} + +static int sun6i_isp_capture_s_input(struct file *file, void *private, + unsigned int index) +{ + if (index != 0) + return -EINVAL; + + return 0; +} + +static const struct v4l2_ioctl_ops sun6i_isp_capture_ioctl_ops = { + .vidioc_querycap = sun6i_isp_capture_querycap, + + .vidioc_enum_fmt_vid_cap = sun6i_isp_capture_enum_fmt, + .vidioc_g_fmt_vid_cap = sun6i_isp_capture_g_fmt, + .vidioc_s_fmt_vid_cap = sun6i_isp_capture_s_fmt, + .vidioc_try_fmt_vid_cap = sun6i_isp_capture_try_fmt, + + .vidioc_enum_input = sun6i_isp_capture_enum_input, + .vidioc_g_input = sun6i_isp_capture_g_input, + .vidioc_s_input = sun6i_isp_capture_s_input, + + .vidioc_create_bufs = vb2_ioctl_create_bufs, + .vidioc_prepare_buf = vb2_ioctl_prepare_buf, + .vidioc_reqbufs = vb2_ioctl_reqbufs, + .vidioc_querybuf = vb2_ioctl_querybuf, + .vidioc_expbuf = vb2_ioctl_expbuf, + .vidioc_qbuf = vb2_ioctl_qbuf, + .vidioc_dqbuf = vb2_ioctl_dqbuf, + .vidioc_streamon = vb2_ioctl_streamon, + .vidioc_streamoff = vb2_ioctl_streamoff, +}; + +static int sun6i_isp_capture_open(struct file *file) +{ + struct sun6i_isp_device *isp_dev = video_drvdata(file); + struct video_device *video_dev = &isp_dev->capture.video_dev; + struct mutex *lock = &isp_dev->capture.lock; + int ret; + + if (mutex_lock_interruptible(lock)) + return -ERESTARTSYS; + + ret = v4l2_pipeline_pm_get(&video_dev->entity); + if (ret) + goto error_mutex; + + ret = v4l2_fh_open(file); + if (ret) + goto error_pipeline; + + mutex_unlock(lock); + + return 0; + +error_pipeline: + v4l2_pipeline_pm_put(&video_dev->entity); + +error_mutex: + mutex_unlock(lock); + + return ret; +} + +static int sun6i_isp_capture_release(struct file *file) +{ + struct sun6i_isp_device *isp_dev = video_drvdata(file); + struct video_device *video_dev = &isp_dev->capture.video_dev; + struct mutex *lock = &isp_dev->capture.lock; + + mutex_lock(lock); + + _vb2_fop_release(file, NULL); + v4l2_pipeline_pm_put(&video_dev->entity); + + mutex_unlock(lock); + + return 0; +} + +static const struct v4l2_file_operations sun6i_isp_capture_fops = { + .owner = THIS_MODULE, + .open = sun6i_isp_capture_open, + .release = sun6i_isp_capture_release, + .unlocked_ioctl = video_ioctl2, + .poll = vb2_fop_poll, + .mmap = vb2_fop_mmap, +}; + +/* Media Entity */ + +static int sun6i_isp_capture_link_validate(struct media_link *link) +{ + struct video_device *video_dev = + media_entity_to_video_device(link->sink->entity); + struct sun6i_isp_device *isp_dev = video_get_drvdata(video_dev); + struct v4l2_device *v4l2_dev = &isp_dev->v4l2.v4l2_dev; + unsigned int capture_width, capture_height; + unsigned int proc_width, proc_height; + + sun6i_isp_capture_dimensions(isp_dev, &capture_width, &capture_height); + sun6i_isp_proc_dimensions(isp_dev, &proc_width, &proc_height); + + /* No cropping/scaling is supported (yet). */ + if (capture_width != proc_width || capture_height != proc_height) { + v4l2_err(v4l2_dev, + "invalid input/output dimensions: %ux%u/%ux%u\n", + proc_width, proc_height, capture_width, + capture_height); + return -EINVAL; + } + + return 0; +} + +static const struct media_entity_operations sun6i_isp_capture_entity_ops = { + .link_validate = sun6i_isp_capture_link_validate, +}; + +/* Capture */ + +int sun6i_isp_capture_setup(struct sun6i_isp_device *isp_dev) +{ + struct sun6i_isp_capture *capture = &isp_dev->capture; + struct sun6i_isp_capture_state *state = &capture->state; + struct v4l2_device *v4l2_dev = &isp_dev->v4l2.v4l2_dev; + struct v4l2_subdev *proc_subdev = &isp_dev->proc.subdev; + struct video_device *video_dev = &capture->video_dev; + struct vb2_queue *queue = &capture->queue; + struct media_pad *pad = &capture->pad; + struct v4l2_format *format = &capture->format; + struct v4l2_pix_format *pix_format = &format->fmt.pix; + int ret; + + /* State */ + + INIT_LIST_HEAD(&state->queue); + spin_lock_init(&state->lock); + + /* Media Entity */ + + video_dev->entity.ops = &sun6i_isp_capture_entity_ops; + + /* Media Pads */ + + pad->flags = MEDIA_PAD_FL_SINK | MEDIA_PAD_FL_MUST_CONNECT; + + ret = media_entity_pads_init(&video_dev->entity, 1, pad); + if (ret) + goto error_mutex; + + /* Queue */ + + mutex_init(&capture->lock); + + queue->type = V4L2_BUF_TYPE_VIDEO_CAPTURE; + queue->io_modes = VB2_MMAP | VB2_DMABUF; + queue->buf_struct_size = sizeof(struct sun6i_isp_buffer); + queue->ops = &sun6i_isp_capture_queue_ops; + queue->mem_ops = &vb2_dma_contig_memops; + queue->min_buffers_needed = 2; + queue->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC; + queue->lock = &capture->lock; + queue->dev = isp_dev->dev; + queue->drv_priv = isp_dev; + + ret = vb2_queue_init(queue); + if (ret) { + v4l2_err(v4l2_dev, "failed to initialize vb2 queue: %d\n", ret); + goto error_media_entity; + } + + /* V4L2 Format */ + + format->type = queue->type; + pix_format->pixelformat = sun6i_isp_capture_formats[0].pixelformat; + pix_format->width = 1280; + pix_format->height = 720; + + sun6i_isp_capture_format_prepare(format); + + /* Video Device */ + + strscpy(video_dev->name, SUN6I_ISP_CAPTURE_NAME, + sizeof(video_dev->name)); + video_dev->device_caps = V4L2_CAP_VIDEO_CAPTURE | V4L2_CAP_STREAMING; + video_dev->vfl_dir = VFL_DIR_RX; + video_dev->release = video_device_release_empty; + video_dev->fops = &sun6i_isp_capture_fops; + video_dev->ioctl_ops = &sun6i_isp_capture_ioctl_ops; + video_dev->v4l2_dev = v4l2_dev; + video_dev->queue = queue; + video_dev->lock = &capture->lock; + + video_set_drvdata(video_dev, isp_dev); + + ret = video_register_device(video_dev, VFL_TYPE_VIDEO, -1); + if (ret) { + v4l2_err(v4l2_dev, "failed to register video device: %d\n", + ret); + goto error_media_entity; + } + + /* Media Pad Link */ + + ret = media_create_pad_link(&proc_subdev->entity, + SUN6I_ISP_PROC_PAD_SOURCE, + &video_dev->entity, 0, + MEDIA_LNK_FL_ENABLED | + MEDIA_LNK_FL_IMMUTABLE); + if (ret < 0) { + v4l2_err(v4l2_dev, "failed to create %s:%u -> %s:%u link\n", + proc_subdev->entity.name, SUN6I_ISP_PROC_PAD_SOURCE, + video_dev->entity.name, 0); + goto error_video_device; + } + + return 0; + +error_video_device: + vb2_video_unregister_device(video_dev); + +error_media_entity: + media_entity_cleanup(&video_dev->entity); + +error_mutex: + mutex_destroy(&capture->lock); + + return ret; +} + +void sun6i_isp_capture_cleanup(struct sun6i_isp_device *isp_dev) +{ + struct sun6i_isp_capture *capture = &isp_dev->capture; + struct video_device *video_dev = &capture->video_dev; + + vb2_video_unregister_device(video_dev); + media_entity_cleanup(&video_dev->entity); + mutex_destroy(&capture->lock); +} diff --git a/drivers/staging/media/sunxi/sun6i-isp/sun6i_isp_capture.h b/drivers/staging/media/sunxi/sun6i-isp/sun6i_isp_capture.h new file mode 100644 index 000000000000..0e3e4fa7a0f4 --- /dev/null +++ b/drivers/staging/media/sunxi/sun6i-isp/sun6i_isp_capture.h @@ -0,0 +1,78 @@ +/* SPDX-License-Identifier: GPL-2.0+ */ +/* + * Copyright 2021-2022 Bootlin + * Author: Paul Kocialkowski <[email protected]> + */ + +#ifndef _SUN6I_ISP_CAPTURE_H_ +#define _SUN6I_ISP_CAPTURE_H_ + +#include <media/v4l2-device.h> + +#define SUN6I_ISP_CAPTURE_NAME "sun6i-isp-capture" + +#define SUN6I_ISP_CAPTURE_WIDTH_MIN 16 +#define SUN6I_ISP_CAPTURE_WIDTH_MAX 3264 +#define SUN6I_ISP_CAPTURE_HEIGHT_MIN 16 +#define SUN6I_ISP_CAPTURE_HEIGHT_MAX 2448 + +struct sun6i_isp_device; + +struct sun6i_isp_capture_format { + u32 pixelformat; + u8 output_format; +}; + +#undef current +struct sun6i_isp_capture_state { + struct list_head queue; + spinlock_t lock; /* Queue and buffers lock. */ + + struct sun6i_isp_buffer *pending; + struct sun6i_isp_buffer *current; + struct sun6i_isp_buffer *complete; + + unsigned int sequence; + bool streaming; +}; + +struct sun6i_isp_capture { + struct sun6i_isp_capture_state state; + + struct video_device video_dev; + struct vb2_queue queue; + struct mutex lock; /* Queue lock. */ + struct media_pad pad; + + struct v4l2_format format; +}; + +/* Helpers */ + +void sun6i_isp_capture_dimensions(struct sun6i_isp_device *isp_dev, + unsigned int *width, unsigned int *height); +void sun6i_isp_capture_format(struct sun6i_isp_device *isp_dev, + u32 *pixelformat); + +/* Format */ + +const struct sun6i_isp_capture_format * +sun6i_isp_capture_format_find(u32 pixelformat); + +/* Capture */ + +void sun6i_isp_capture_configure(struct sun6i_isp_device *isp_dev); + +/* State */ + +void sun6i_isp_capture_state_update(struct sun6i_isp_device *isp_dev, + bool *update); +void sun6i_isp_capture_state_complete(struct sun6i_isp_device *isp_dev); +void sun6i_isp_capture_finish(struct sun6i_isp_device *isp_dev); + +/* Capture */ + +int sun6i_isp_capture_setup(struct sun6i_isp_device *isp_dev); +void sun6i_isp_capture_cleanup(struct sun6i_isp_device *isp_dev); + +#endif diff --git a/drivers/staging/media/sunxi/sun6i-isp/sun6i_isp_params.c b/drivers/staging/media/sunxi/sun6i-isp/sun6i_isp_params.c new file mode 100644 index 000000000000..8039e311cb1c --- /dev/null +++ b/drivers/staging/media/sunxi/sun6i-isp/sun6i_isp_params.c @@ -0,0 +1,566 @@ +// SPDX-License-Identifier: GPL-2.0+ +/* + * Copyright 2021-2022 Bootlin + * Author: Paul Kocialkowski <[email protected]> + */ + +#include <media/v4l2-device.h> +#include <media/v4l2-event.h> +#include <media/v4l2-ioctl.h> +#include <media/v4l2-mc.h> +#include <media/videobuf2-vmalloc.h> +#include <media/videobuf2-v4l2.h> + +#include "sun6i_isp.h" +#include "sun6i_isp_params.h" +#include "sun6i_isp_reg.h" +#include "uapi/sun6i-isp-config.h" + +/* Params */ + +static const struct sun6i_isp_params_config sun6i_isp_params_config_default = { + .modules_used = SUN6I_ISP_MODULE_BAYER, + + .bayer = { + .offset_r = 32, + .offset_gr = 32, + .offset_gb = 32, + .offset_b = 32, + + .gain_r = 256, + .gain_gr = 256, + .gain_gb = 256, + .gain_b = 256, + + }, + + .bdnf = { + .in_dis_min = 8, + .in_dis_max = 16, + + .coefficients_g = { 15, 4, 1 }, + .coefficients_rb = { 15, 4 }, + }, +}; + +static void sun6i_isp_params_configure_ob(struct sun6i_isp_device *isp_dev) +{ + unsigned int width, height; + + sun6i_isp_proc_dimensions(isp_dev, &width, &height); + + sun6i_isp_load_write(isp_dev, SUN6I_ISP_OB_SIZE_REG, + SUN6I_ISP_OB_SIZE_WIDTH(width) | + SUN6I_ISP_OB_SIZE_HEIGHT(height)); + + sun6i_isp_load_write(isp_dev, SUN6I_ISP_OB_VALID_REG, + SUN6I_ISP_OB_VALID_WIDTH(width) | + SUN6I_ISP_OB_VALID_HEIGHT(height)); + + sun6i_isp_load_write(isp_dev, SUN6I_ISP_OB_SRC0_VALID_START_REG, + SUN6I_ISP_OB_SRC0_VALID_START_HORZ(0) | + SUN6I_ISP_OB_SRC0_VALID_START_VERT(0)); +} + +static void sun6i_isp_params_configure_ae(struct sun6i_isp_device *isp_dev) +{ + /* These are default values that need to be set to get an output. */ + + sun6i_isp_load_write(isp_dev, SUN6I_ISP_AE_CFG_REG, + SUN6I_ISP_AE_CFG_LOW_BRI_TH(0xff) | + SUN6I_ISP_AE_CFG_HORZ_NUM(8) | + SUN6I_ISP_AE_CFG_HIGH_BRI_TH(0xf00) | + SUN6I_ISP_AE_CFG_VERT_NUM(8)); +} + +static void +sun6i_isp_params_configure_bayer(struct sun6i_isp_device *isp_dev, + const struct sun6i_isp_params_config *config) +{ + const struct sun6i_isp_params_config_bayer *bayer = &config->bayer; + + sun6i_isp_load_write(isp_dev, SUN6I_ISP_BAYER_OFFSET0_REG, + SUN6I_ISP_BAYER_OFFSET0_R(bayer->offset_r) | + SUN6I_ISP_BAYER_OFFSET0_GR(bayer->offset_gr)); + + sun6i_isp_load_write(isp_dev, SUN6I_ISP_BAYER_OFFSET1_REG, + SUN6I_ISP_BAYER_OFFSET1_GB(bayer->offset_gb) | + SUN6I_ISP_BAYER_OFFSET1_B(bayer->offset_b)); + + sun6i_isp_load_write(isp_dev, SUN6I_ISP_BAYER_GAIN0_REG, + SUN6I_ISP_BAYER_GAIN0_R(bayer->gain_r) | + SUN6I_ISP_BAYER_GAIN0_GR(bayer->gain_gr)); + + sun6i_isp_load_write(isp_dev, SUN6I_ISP_BAYER_GAIN1_REG, + SUN6I_ISP_BAYER_GAIN1_GB(bayer->gain_gb) | + SUN6I_ISP_BAYER_GAIN1_B(bayer->gain_b)); +} + +static void sun6i_isp_params_configure_wb(struct sun6i_isp_device *isp_dev) +{ + /* These are default values that need to be set to get an output. */ + + sun6i_isp_load_write(isp_dev, SUN6I_ISP_WB_GAIN0_REG, + SUN6I_ISP_WB_GAIN0_R(256) | + SUN6I_ISP_WB_GAIN0_GR(256)); + + sun6i_isp_load_write(isp_dev, SUN6I_ISP_WB_GAIN1_REG, + SUN6I_ISP_WB_GAIN1_GB(256) | + SUN6I_ISP_WB_GAIN1_B(256)); + + sun6i_isp_load_write(isp_dev, SUN6I_ISP_WB_CFG_REG, + SUN6I_ISP_WB_CFG_CLIP(0xfff)); +} + +static void sun6i_isp_params_configure_base(struct sun6i_isp_device *isp_dev) +{ + sun6i_isp_params_configure_ae(isp_dev); + sun6i_isp_params_configure_ob(isp_dev); + sun6i_isp_params_configure_wb(isp_dev); +} + +static void +sun6i_isp_params_configure_bdnf(struct sun6i_isp_device *isp_dev, + const struct sun6i_isp_params_config *config) +{ + const struct sun6i_isp_params_config_bdnf *bdnf = &config->bdnf; + + sun6i_isp_load_write(isp_dev, SUN6I_ISP_BDNF_CFG_REG, + SUN6I_ISP_BDNF_CFG_IN_DIS_MIN(bdnf->in_dis_min) | + SUN6I_ISP_BDNF_CFG_IN_DIS_MAX(bdnf->in_dis_max)); + + sun6i_isp_load_write(isp_dev, SUN6I_ISP_BDNF_COEF_RB_REG, + SUN6I_ISP_BDNF_COEF_RB(0, bdnf->coefficients_rb[0]) | + SUN6I_ISP_BDNF_COEF_RB(1, bdnf->coefficients_rb[1]) | + SUN6I_ISP_BDNF_COEF_RB(2, bdnf->coefficients_rb[2]) | + SUN6I_ISP_BDNF_COEF_RB(3, bdnf->coefficients_rb[3]) | + SUN6I_ISP_BDNF_COEF_RB(4, bdnf->coefficients_rb[4])); + + sun6i_isp_load_write(isp_dev, SUN6I_ISP_BDNF_COEF_G_REG, + SUN6I_ISP_BDNF_COEF_G(0, bdnf->coefficients_g[0]) | + SUN6I_ISP_BDNF_COEF_G(1, bdnf->coefficients_g[1]) | + SUN6I_ISP_BDNF_COEF_G(2, bdnf->coefficients_g[2]) | + SUN6I_ISP_BDNF_COEF_G(3, bdnf->coefficients_g[3]) | + SUN6I_ISP_BDNF_COEF_G(4, bdnf->coefficients_g[4]) | + SUN6I_ISP_BDNF_COEF_G(5, bdnf->coefficients_g[5]) | + SUN6I_ISP_BDNF_COEF_G(6, bdnf->coefficients_g[6])); +} + +static void +sun6i_isp_params_configure_modules(struct sun6i_isp_device *isp_dev, + const struct sun6i_isp_params_config *config) +{ + u32 value; + + if (config->modules_used & SUN6I_ISP_MODULE_BDNF) + sun6i_isp_params_configure_bdnf(isp_dev, config); + + if (config->modules_used & SUN6I_ISP_MODULE_BAYER) + sun6i_isp_params_configure_bayer(isp_dev, config); + + value = sun6i_isp_load_read(isp_dev, SUN6I_ISP_MODULE_EN_REG); + /* Clear all modules but keep input configuration. */ + value &= SUN6I_ISP_MODULE_EN_SRC0 | SUN6I_ISP_MODULE_EN_SRC1; + + if (config->modules_used & SUN6I_ISP_MODULE_BDNF) + value |= SUN6I_ISP_MODULE_EN_BDNF; + + /* Bayer stage is always enabled. */ + + sun6i_isp_load_write(isp_dev, SUN6I_ISP_MODULE_EN_REG, value); +} + +void sun6i_isp_params_configure(struct sun6i_isp_device *isp_dev) +{ + struct sun6i_isp_params_state *state = &isp_dev->params.state; + unsigned long flags; + + spin_lock_irqsave(&state->lock, flags); + + sun6i_isp_params_configure_base(isp_dev); + + /* Default config is only applied at the very first stream start. */ + if (state->configured) + goto complete; + + sun6i_isp_params_configure_modules(isp_dev, + &sun6i_isp_params_config_default); + + state->configured = true; + +complete: + spin_unlock_irqrestore(&state->lock, flags); +} + +/* State */ + +static void sun6i_isp_params_state_cleanup(struct sun6i_isp_device *isp_dev, + bool error) +{ + struct sun6i_isp_params_state *state = &isp_dev->params.state; + struct sun6i_isp_buffer *isp_buffer; + struct vb2_buffer *vb2_buffer; + unsigned long flags; + + spin_lock_irqsave(&state->lock, flags); + + if (state->pending) { + vb2_buffer = &state->pending->v4l2_buffer.vb2_buf; + vb2_buffer_done(vb2_buffer, error ? VB2_BUF_STATE_ERROR : + VB2_BUF_STATE_QUEUED); + } + + list_for_each_entry(isp_buffer, &state->queue, list) { + vb2_buffer = &isp_buffer->v4l2_buffer.vb2_buf; + vb2_buffer_done(vb2_buffer, error ? VB2_BUF_STATE_ERROR : + VB2_BUF_STATE_QUEUED); + } + + INIT_LIST_HEAD(&state->queue); + + spin_unlock_irqrestore(&state->lock, flags); +} + +void sun6i_isp_params_state_update(struct sun6i_isp_device *isp_dev, + bool *update) +{ + struct sun6i_isp_params_state *state = &isp_dev->params.state; + struct sun6i_isp_buffer *isp_buffer; + struct vb2_buffer *vb2_buffer; + const struct sun6i_isp_params_config *config; + unsigned long flags; + + spin_lock_irqsave(&state->lock, flags); + + if (list_empty(&state->queue)) + goto complete; + + if (state->pending) + goto complete; + + isp_buffer = list_first_entry(&state->queue, struct sun6i_isp_buffer, + list); + + vb2_buffer = &isp_buffer->v4l2_buffer.vb2_buf; + config = vb2_plane_vaddr(vb2_buffer, 0); + + sun6i_isp_params_configure_modules(isp_dev, config); + + list_del(&isp_buffer->list); + + state->pending = isp_buffer; + + if (update) + *update = true; + +complete: + spin_unlock_irqrestore(&state->lock, flags); +} + +void sun6i_isp_params_state_complete(struct sun6i_isp_device *isp_dev) +{ + struct sun6i_isp_params_state *state = &isp_dev->params.state; + struct sun6i_isp_buffer *isp_buffer; + struct vb2_buffer *vb2_buffer; + unsigned long flags; + + spin_lock_irqsave(&state->lock, flags); + + if (!state->pending) + goto complete; + + isp_buffer = state->pending; + vb2_buffer = &isp_buffer->v4l2_buffer.vb2_buf; + + vb2_buffer->timestamp = ktime_get_ns(); + + /* Parameters will be applied starting from the next frame. */ + isp_buffer->v4l2_buffer.sequence = isp_dev->capture.state.sequence + 1; + + vb2_buffer_done(vb2_buffer, VB2_BUF_STATE_DONE); + + state->pending = NULL; + +complete: + spin_unlock_irqrestore(&state->lock, flags); +} + +/* Queue */ + +static int sun6i_isp_params_queue_setup(struct vb2_queue *queue, + unsigned int *buffers_count, + unsigned int *planes_count, + unsigned int sizes[], + struct device *alloc_devs[]) +{ + struct sun6i_isp_device *isp_dev = vb2_get_drv_priv(queue); + unsigned int size = isp_dev->params.format.fmt.meta.buffersize; + + if (*planes_count) + return sizes[0] < size ? -EINVAL : 0; + + *planes_count = 1; + sizes[0] = size; + + return 0; +} + +static int sun6i_isp_params_buffer_prepare(struct vb2_buffer *vb2_buffer) +{ + struct sun6i_isp_device *isp_dev = + vb2_get_drv_priv(vb2_buffer->vb2_queue); + struct v4l2_device *v4l2_dev = &isp_dev->v4l2.v4l2_dev; + unsigned int size = isp_dev->params.format.fmt.meta.buffersize; + + if (vb2_plane_size(vb2_buffer, 0) < size) { + v4l2_err(v4l2_dev, "buffer too small (%lu < %u)\n", + vb2_plane_size(vb2_buffer, 0), size); + return -EINVAL; + } + + vb2_set_plane_payload(vb2_buffer, 0, size); + + return 0; +} + +static void sun6i_isp_params_buffer_queue(struct vb2_buffer *vb2_buffer) +{ + struct sun6i_isp_device *isp_dev = + vb2_get_drv_priv(vb2_buffer->vb2_queue); + struct sun6i_isp_params_state *state = &isp_dev->params.state; + struct vb2_v4l2_buffer *v4l2_buffer = to_vb2_v4l2_buffer(vb2_buffer); + struct sun6i_isp_buffer *isp_buffer = + container_of(v4l2_buffer, struct sun6i_isp_buffer, v4l2_buffer); + bool capture_streaming = isp_dev->capture.state.streaming; + unsigned long flags; + + spin_lock_irqsave(&state->lock, flags); + list_add_tail(&isp_buffer->list, &state->queue); + spin_unlock_irqrestore(&state->lock, flags); + + if (state->streaming && capture_streaming) + sun6i_isp_state_update(isp_dev, false); +} + +static int sun6i_isp_params_start_streaming(struct vb2_queue *queue, + unsigned int count) +{ + struct sun6i_isp_device *isp_dev = vb2_get_drv_priv(queue); + struct sun6i_isp_params_state *state = &isp_dev->params.state; + bool capture_streaming = isp_dev->capture.state.streaming; + + state->streaming = true; + + /* + * Update the state as soon as possible if capture is streaming, + * otherwise it will be applied when capture starts streaming. + */ + + if (capture_streaming) + sun6i_isp_state_update(isp_dev, false); + + return 0; +} + +static void sun6i_isp_params_stop_streaming(struct vb2_queue *queue) +{ + struct sun6i_isp_device *isp_dev = vb2_get_drv_priv(queue); + struct sun6i_isp_params_state *state = &isp_dev->params.state; + + state->streaming = false; + sun6i_isp_params_state_cleanup(isp_dev, true); +} + +static const struct vb2_ops sun6i_isp_params_queue_ops = { + .queue_setup = sun6i_isp_params_queue_setup, + .buf_prepare = sun6i_isp_params_buffer_prepare, + .buf_queue = sun6i_isp_params_buffer_queue, + .start_streaming = sun6i_isp_params_start_streaming, + .stop_streaming = sun6i_isp_params_stop_streaming, + .wait_prepare = vb2_ops_wait_prepare, + .wait_finish = vb2_ops_wait_finish, +}; + +/* Video Device */ + +static int sun6i_isp_params_querycap(struct file *file, void *private, + struct v4l2_capability *capability) +{ + struct sun6i_isp_device *isp_dev = video_drvdata(file); + struct video_device *video_dev = &isp_dev->params.video_dev; + + strscpy(capability->driver, SUN6I_ISP_NAME, sizeof(capability->driver)); + strscpy(capability->card, video_dev->name, sizeof(capability->card)); + snprintf(capability->bus_info, sizeof(capability->bus_info), + "platform:%s", dev_name(isp_dev->dev)); + + return 0; +} + +static int sun6i_isp_params_enum_fmt(struct file *file, void *private, + struct v4l2_fmtdesc *fmtdesc) +{ + struct sun6i_isp_device *isp_dev = video_drvdata(file); + struct v4l2_meta_format *params_format = + &isp_dev->params.format.fmt.meta; + + if (fmtdesc->index > 0) + return -EINVAL; + + fmtdesc->pixelformat = params_format->dataformat; + + return 0; +} + +static int sun6i_isp_params_g_fmt(struct file *file, void *private, + struct v4l2_format *format) +{ + struct sun6i_isp_device *isp_dev = video_drvdata(file); + + *format = isp_dev->params.format; + + return 0; +} + +static const struct v4l2_ioctl_ops sun6i_isp_params_ioctl_ops = { + .vidioc_querycap = sun6i_isp_params_querycap, + + .vidioc_enum_fmt_meta_out = sun6i_isp_params_enum_fmt, + .vidioc_g_fmt_meta_out = sun6i_isp_params_g_fmt, + .vidioc_s_fmt_meta_out = sun6i_isp_params_g_fmt, + .vidioc_try_fmt_meta_out = sun6i_isp_params_g_fmt, + + .vidioc_create_bufs = vb2_ioctl_create_bufs, + .vidioc_prepare_buf = vb2_ioctl_prepare_buf, + .vidioc_reqbufs = vb2_ioctl_reqbufs, + .vidioc_querybuf = vb2_ioctl_querybuf, + .vidioc_expbuf = vb2_ioctl_expbuf, + .vidioc_qbuf = vb2_ioctl_qbuf, + .vidioc_dqbuf = vb2_ioctl_dqbuf, + .vidioc_streamon = vb2_ioctl_streamon, + .vidioc_streamoff = vb2_ioctl_streamoff, +}; + +static const struct v4l2_file_operations sun6i_isp_params_fops = { + .owner = THIS_MODULE, + .unlocked_ioctl = video_ioctl2, + .open = v4l2_fh_open, + .release = vb2_fop_release, + .mmap = vb2_fop_mmap, + .poll = vb2_fop_poll, +}; + +/* Params */ + +int sun6i_isp_params_setup(struct sun6i_isp_device *isp_dev) +{ + struct sun6i_isp_params *params = &isp_dev->params; + struct sun6i_isp_params_state *state = ¶ms->state; + struct v4l2_device *v4l2_dev = &isp_dev->v4l2.v4l2_dev; + struct v4l2_subdev *proc_subdev = &isp_dev->proc.subdev; + struct video_device *video_dev = ¶ms->video_dev; + struct vb2_queue *queue = &isp_dev->params.queue; + struct media_pad *pad = &isp_dev->params.pad; + struct v4l2_format *format = &isp_dev->params.format; + struct v4l2_meta_format *params_format = &format->fmt.meta; + int ret; + + /* State */ + + INIT_LIST_HEAD(&state->queue); + spin_lock_init(&state->lock); + + /* Media Pads */ + + pad->flags = MEDIA_PAD_FL_SOURCE | MEDIA_PAD_FL_MUST_CONNECT; + + ret = media_entity_pads_init(&video_dev->entity, 1, pad); + if (ret) + goto error_mutex; + + /* Queue */ + + mutex_init(¶ms->lock); + + queue->type = V4L2_BUF_TYPE_META_OUTPUT; + queue->io_modes = VB2_MMAP | VB2_USERPTR | VB2_DMABUF; + queue->buf_struct_size = sizeof(struct sun6i_isp_buffer); + queue->ops = &sun6i_isp_params_queue_ops; + queue->mem_ops = &vb2_vmalloc_memops; + queue->min_buffers_needed = 1; + queue->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC; + queue->lock = ¶ms->lock; + queue->dev = isp_dev->dev; + queue->drv_priv = isp_dev; + + ret = vb2_queue_init(queue); + if (ret) { + v4l2_err(v4l2_dev, "failed to initialize vb2 queue: %d\n", ret); + goto error_media_entity; + } + + /* V4L2 Format */ + + format->type = queue->type; + params_format->dataformat = V4L2_META_FMT_SUN6I_ISP_PARAMS; + params_format->buffersize = sizeof(struct sun6i_isp_params_config); + + /* Video Device */ + + strscpy(video_dev->name, SUN6I_ISP_PARAMS_NAME, + sizeof(video_dev->name)); + video_dev->device_caps = V4L2_CAP_META_OUTPUT | V4L2_CAP_STREAMING; + video_dev->vfl_dir = VFL_DIR_TX; + video_dev->release = video_device_release_empty; + video_dev->fops = &sun6i_isp_params_fops; + video_dev->ioctl_ops = &sun6i_isp_params_ioctl_ops; + video_dev->v4l2_dev = v4l2_dev; + video_dev->queue = queue; + video_dev->lock = ¶ms->lock; + + video_set_drvdata(video_dev, isp_dev); + + ret = video_register_device(video_dev, VFL_TYPE_VIDEO, -1); + if (ret) { + v4l2_err(v4l2_dev, "failed to register video device: %d\n", + ret); + goto error_media_entity; + } + + /* Media Pad Link */ + + ret = media_create_pad_link(&video_dev->entity, 0, + &proc_subdev->entity, + SUN6I_ISP_PROC_PAD_SINK_PARAMS, + MEDIA_LNK_FL_ENABLED | + MEDIA_LNK_FL_IMMUTABLE); + if (ret < 0) { + v4l2_err(v4l2_dev, "failed to create %s:%u -> %s:%u link\n", + video_dev->entity.name, 0, proc_subdev->entity.name, + SUN6I_ISP_PROC_PAD_SINK_PARAMS); + goto error_video_device; + } + + return 0; + +error_video_device: + vb2_video_unregister_device(video_dev); + +error_media_entity: + media_entity_cleanup(&video_dev->entity); + +error_mutex: + mutex_destroy(¶ms->lock); + + return ret; +} + +void sun6i_isp_params_cleanup(struct sun6i_isp_device *isp_dev) +{ + struct sun6i_isp_params *params = &isp_dev->params; + struct video_device *video_dev = ¶ms->video_dev; + + vb2_video_unregister_device(video_dev); + media_entity_cleanup(&video_dev->entity); + mutex_destroy(¶ms->lock); +} diff --git a/drivers/staging/media/sunxi/sun6i-isp/sun6i_isp_params.h b/drivers/staging/media/sunxi/sun6i-isp/sun6i_isp_params.h new file mode 100644 index 000000000000..50f10f879c42 --- /dev/null +++ b/drivers/staging/media/sunxi/sun6i-isp/sun6i_isp_params.h @@ -0,0 +1,52 @@ +/* SPDX-License-Identifier: GPL-2.0+ */ +/* + * Copyright 2021-2022 Bootlin + * Author: Paul Kocialkowski <[email protected]> + */ + +#ifndef _SUN6I_ISP_PARAMS_H_ +#define _SUN6I_ISP_PARAMS_H_ + +#include <media/v4l2-device.h> + +#define SUN6I_ISP_PARAMS_NAME "sun6i-isp-params" + +struct sun6i_isp_device; + +struct sun6i_isp_params_state { + struct list_head queue; /* Queue and buffers lock. */ + spinlock_t lock; + + struct sun6i_isp_buffer *pending; + + bool configured; + bool streaming; +}; + +struct sun6i_isp_params { + struct sun6i_isp_params_state state; + + struct video_device video_dev; + struct vb2_queue queue; + struct mutex lock; /* Queue lock. */ + struct media_pad pad; + + struct v4l2_format format; +}; + +/* Params */ + +void sun6i_isp_params_configure(struct sun6i_isp_device *isp_dev); + +/* State */ + +void sun6i_isp_params_state_update(struct sun6i_isp_device *isp_dev, + bool *update); +void sun6i_isp_params_state_complete(struct sun6i_isp_device *isp_dev); + +/* Params */ + +int sun6i_isp_params_setup(struct sun6i_isp_device *isp_dev); +void sun6i_isp_params_cleanup(struct sun6i_isp_device *isp_dev); + +#endif diff --git a/drivers/staging/media/sunxi/sun6i-isp/sun6i_isp_proc.c b/drivers/staging/media/sunxi/sun6i-isp/sun6i_isp_proc.c new file mode 100644 index 000000000000..d69d2be0add2 --- /dev/null +++ b/drivers/staging/media/sunxi/sun6i-isp/sun6i_isp_proc.c @@ -0,0 +1,577 @@ +// SPDX-License-Identifier: GPL-2.0+ +/* + * Copyright 2021-2022 Bootlin + * Author: Paul Kocialkowski <[email protected]> + */ + +#include <linux/pm_runtime.h> +#include <linux/regmap.h> +#include <media/v4l2-device.h> +#include <media/v4l2-fwnode.h> + +#include "sun6i_isp.h" +#include "sun6i_isp_capture.h" +#include "sun6i_isp_params.h" +#include "sun6i_isp_proc.h" +#include "sun6i_isp_reg.h" + +/* Helpers */ + +void sun6i_isp_proc_dimensions(struct sun6i_isp_device *isp_dev, + unsigned int *width, unsigned int *height) +{ + if (width) + *width = isp_dev->proc.mbus_format.width; + if (height) + *height = isp_dev->proc.mbus_format.height; +} + +/* Format */ + +static const struct sun6i_isp_proc_format sun6i_isp_proc_formats[] = { + { + .mbus_code = MEDIA_BUS_FMT_SBGGR8_1X8, + .input_format = SUN6I_ISP_INPUT_FMT_RAW_BGGR, + }, + { + .mbus_code = MEDIA_BUS_FMT_SGBRG8_1X8, + .input_format = SUN6I_ISP_INPUT_FMT_RAW_GBRG, + }, + { + .mbus_code = MEDIA_BUS_FMT_SGRBG8_1X8, + .input_format = SUN6I_ISP_INPUT_FMT_RAW_GRBG, + }, + { + .mbus_code = MEDIA_BUS_FMT_SRGGB8_1X8, + .input_format = SUN6I_ISP_INPUT_FMT_RAW_RGGB, + }, + + { + .mbus_code = MEDIA_BUS_FMT_SBGGR10_1X10, + .input_format = SUN6I_ISP_INPUT_FMT_RAW_BGGR, + }, + { + .mbus_code = MEDIA_BUS_FMT_SGBRG10_1X10, + .input_format = SUN6I_ISP_INPUT_FMT_RAW_GBRG, + }, + { + .mbus_code = MEDIA_BUS_FMT_SGRBG10_1X10, + .input_format = SUN6I_ISP_INPUT_FMT_RAW_GRBG, + }, + { + .mbus_code = MEDIA_BUS_FMT_SRGGB10_1X10, + .input_format = SUN6I_ISP_INPUT_FMT_RAW_RGGB, + }, +}; + +const struct sun6i_isp_proc_format *sun6i_isp_proc_format_find(u32 mbus_code) +{ + unsigned int i; + + for (i = 0; i < ARRAY_SIZE(sun6i_isp_proc_formats); i++) + if (sun6i_isp_proc_formats[i].mbus_code == mbus_code) + return &sun6i_isp_proc_formats[i]; + + return NULL; +} + +/* Processor */ + +static void sun6i_isp_proc_irq_enable(struct sun6i_isp_device *isp_dev) +{ + struct regmap *regmap = isp_dev->regmap; + + regmap_write(regmap, SUN6I_ISP_FE_INT_EN_REG, + SUN6I_ISP_FE_INT_EN_FINISH | + SUN6I_ISP_FE_INT_EN_START | + SUN6I_ISP_FE_INT_EN_PARA_SAVE | + SUN6I_ISP_FE_INT_EN_PARA_LOAD | + SUN6I_ISP_FE_INT_EN_SRC0_FIFO | + SUN6I_ISP_FE_INT_EN_ROT_FINISH); +} + +static void sun6i_isp_proc_irq_disable(struct sun6i_isp_device *isp_dev) +{ + struct regmap *regmap = isp_dev->regmap; + + regmap_write(regmap, SUN6I_ISP_FE_INT_EN_REG, 0); +} + +static void sun6i_isp_proc_irq_clear(struct sun6i_isp_device *isp_dev) +{ + struct regmap *regmap = isp_dev->regmap; + + regmap_write(regmap, SUN6I_ISP_FE_INT_EN_REG, 0); + regmap_write(regmap, SUN6I_ISP_FE_INT_STA_REG, + SUN6I_ISP_FE_INT_STA_CLEAR); +} + +static void sun6i_isp_proc_enable(struct sun6i_isp_device *isp_dev, + struct sun6i_isp_proc_source *source) +{ + struct sun6i_isp_proc *proc = &isp_dev->proc; + struct regmap *regmap = isp_dev->regmap; + u8 mode; + + /* Frontend */ + + if (source == &proc->source_csi0) + mode = SUN6I_ISP_SRC_MODE_CSI(0); + else + mode = SUN6I_ISP_SRC_MODE_CSI(1); + + regmap_write(regmap, SUN6I_ISP_FE_CFG_REG, + SUN6I_ISP_FE_CFG_EN | SUN6I_ISP_FE_CFG_SRC0_MODE(mode)); + + regmap_write(regmap, SUN6I_ISP_FE_CTRL_REG, + SUN6I_ISP_FE_CTRL_VCAP_EN | SUN6I_ISP_FE_CTRL_PARA_READY); +} + +static void sun6i_isp_proc_disable(struct sun6i_isp_device *isp_dev) +{ + struct regmap *regmap = isp_dev->regmap; + + /* Frontend */ + + regmap_write(regmap, SUN6I_ISP_FE_CTRL_REG, 0); + regmap_write(regmap, SUN6I_ISP_FE_CFG_REG, 0); +} + +static void sun6i_isp_proc_configure(struct sun6i_isp_device *isp_dev) +{ + struct v4l2_mbus_framefmt *mbus_format = &isp_dev->proc.mbus_format; + const struct sun6i_isp_proc_format *format; + u32 value; + + /* Module */ + + value = sun6i_isp_load_read(isp_dev, SUN6I_ISP_MODULE_EN_REG); + value |= SUN6I_ISP_MODULE_EN_SRC0; + sun6i_isp_load_write(isp_dev, SUN6I_ISP_MODULE_EN_REG, value); + + /* Input */ + + format = sun6i_isp_proc_format_find(mbus_format->code); + if (WARN_ON(!format)) + return; + + sun6i_isp_load_write(isp_dev, SUN6I_ISP_MODE_REG, + SUN6I_ISP_MODE_INPUT_FMT(format->input_format) | + SUN6I_ISP_MODE_INPUT_YUV_SEQ(format->input_yuv_seq) | + SUN6I_ISP_MODE_SHARP(1) | + SUN6I_ISP_MODE_HIST(2)); +} + +/* V4L2 Subdev */ + +static int sun6i_isp_proc_s_stream(struct v4l2_subdev *subdev, int on) +{ + struct sun6i_isp_device *isp_dev = v4l2_get_subdevdata(subdev); + struct sun6i_isp_proc *proc = &isp_dev->proc; + struct media_pad *local_pad = &proc->pads[SUN6I_ISP_PROC_PAD_SINK_CSI]; + struct device *dev = isp_dev->dev; + struct sun6i_isp_proc_source *source; + struct v4l2_subdev *source_subdev; + struct media_pad *remote_pad; + /* Initialize to 0 to use both in disable label (ret != 0) and off. */ + int ret = 0; + + /* Source */ + + remote_pad = media_pad_remote_pad_unique(local_pad); + if (IS_ERR(remote_pad)) { + dev_err(dev, + "zero or more than a single source connected to the bridge\n"); + return PTR_ERR(remote_pad); + } + + source_subdev = media_entity_to_v4l2_subdev(remote_pad->entity); + + if (source_subdev == proc->source_csi0.subdev) + source = &proc->source_csi0; + else + source = &proc->source_csi1; + + if (!on) { + sun6i_isp_proc_irq_disable(isp_dev); + v4l2_subdev_call(source_subdev, video, s_stream, 0); + goto disable; + } + + /* PM */ + + ret = pm_runtime_resume_and_get(dev); + if (ret < 0) + return ret; + + /* Clear */ + + sun6i_isp_proc_irq_clear(isp_dev); + + /* Configure */ + + sun6i_isp_tables_configure(isp_dev); + sun6i_isp_params_configure(isp_dev); + sun6i_isp_proc_configure(isp_dev); + sun6i_isp_capture_configure(isp_dev); + + /* State Update */ + + sun6i_isp_state_update(isp_dev, true); + + /* Enable */ + + sun6i_isp_proc_irq_enable(isp_dev); + sun6i_isp_proc_enable(isp_dev, source); + + ret = v4l2_subdev_call(source_subdev, video, s_stream, 1); + if (ret && ret != -ENOIOCTLCMD) { + sun6i_isp_proc_irq_disable(isp_dev); + goto disable; + } + + return 0; + +disable: + sun6i_isp_proc_disable(isp_dev); + + pm_runtime_put(dev); + + return ret; +} + +static const struct v4l2_subdev_video_ops sun6i_isp_proc_video_ops = { + .s_stream = sun6i_isp_proc_s_stream, +}; + +static void +sun6i_isp_proc_mbus_format_prepare(struct v4l2_mbus_framefmt *mbus_format) +{ + if (!sun6i_isp_proc_format_find(mbus_format->code)) + mbus_format->code = sun6i_isp_proc_formats[0].mbus_code; + + mbus_format->field = V4L2_FIELD_NONE; + mbus_format->colorspace = V4L2_COLORSPACE_RAW; + mbus_format->quantization = V4L2_QUANTIZATION_DEFAULT; + mbus_format->xfer_func = V4L2_XFER_FUNC_DEFAULT; +} + +static int sun6i_isp_proc_init_cfg(struct v4l2_subdev *subdev, + struct v4l2_subdev_state *state) +{ + struct sun6i_isp_device *isp_dev = v4l2_get_subdevdata(subdev); + unsigned int pad = SUN6I_ISP_PROC_PAD_SINK_CSI; + struct v4l2_mbus_framefmt *mbus_format = + v4l2_subdev_get_try_format(subdev, state, pad); + struct mutex *lock = &isp_dev->proc.lock; + + mutex_lock(lock); + + mbus_format->code = sun6i_isp_proc_formats[0].mbus_code; + mbus_format->width = 1280; + mbus_format->height = 720; + + sun6i_isp_proc_mbus_format_prepare(mbus_format); + + mutex_unlock(lock); + + return 0; +} + +static int +sun6i_isp_proc_enum_mbus_code(struct v4l2_subdev *subdev, + struct v4l2_subdev_state *state, + struct v4l2_subdev_mbus_code_enum *code_enum) +{ + if (code_enum->index >= ARRAY_SIZE(sun6i_isp_proc_formats)) + return -EINVAL; + + code_enum->code = sun6i_isp_proc_formats[code_enum->index].mbus_code; + + return 0; +} + +static int sun6i_isp_proc_get_fmt(struct v4l2_subdev *subdev, + struct v4l2_subdev_state *state, + struct v4l2_subdev_format *format) +{ + struct sun6i_isp_device *isp_dev = v4l2_get_subdevdata(subdev); + struct v4l2_mbus_framefmt *mbus_format = &format->format; + struct mutex *lock = &isp_dev->proc.lock; + + mutex_lock(lock); + + if (format->which == V4L2_SUBDEV_FORMAT_TRY) + *mbus_format = *v4l2_subdev_get_try_format(subdev, state, + format->pad); + else + *mbus_format = isp_dev->proc.mbus_format; + + mutex_unlock(lock); + + return 0; +} + +static int sun6i_isp_proc_set_fmt(struct v4l2_subdev *subdev, + struct v4l2_subdev_state *state, + struct v4l2_subdev_format *format) +{ + struct sun6i_isp_device *isp_dev = v4l2_get_subdevdata(subdev); + struct v4l2_mbus_framefmt *mbus_format = &format->format; + struct mutex *lock = &isp_dev->proc.lock; + + mutex_lock(lock); + + sun6i_isp_proc_mbus_format_prepare(mbus_format); + + if (format->which == V4L2_SUBDEV_FORMAT_TRY) + *v4l2_subdev_get_try_format(subdev, state, format->pad) = + *mbus_format; + else + isp_dev->proc.mbus_format = *mbus_format; + + mutex_unlock(lock); + + return 0; +} + +static const struct v4l2_subdev_pad_ops sun6i_isp_proc_pad_ops = { + .init_cfg = sun6i_isp_proc_init_cfg, + .enum_mbus_code = sun6i_isp_proc_enum_mbus_code, + .get_fmt = sun6i_isp_proc_get_fmt, + .set_fmt = sun6i_isp_proc_set_fmt, +}; + +const struct v4l2_subdev_ops sun6i_isp_proc_subdev_ops = { + .video = &sun6i_isp_proc_video_ops, + .pad = &sun6i_isp_proc_pad_ops, +}; + +/* Media Entity */ + +static const struct media_entity_operations sun6i_isp_proc_entity_ops = { + .link_validate = v4l2_subdev_link_validate, +}; + +/* V4L2 Async */ + +static int sun6i_isp_proc_link(struct sun6i_isp_device *isp_dev, + int sink_pad_index, + struct v4l2_subdev *remote_subdev, bool enabled) +{ + struct device *dev = isp_dev->dev; + struct v4l2_subdev *subdev = &isp_dev->proc.subdev; + struct media_entity *sink_entity = &subdev->entity; + struct media_entity *source_entity = &remote_subdev->entity; + int source_pad_index; + int ret; + + /* Get the first remote source pad. */ + ret = media_entity_get_fwnode_pad(source_entity, remote_subdev->fwnode, + MEDIA_PAD_FL_SOURCE); + if (ret < 0) { + dev_err(dev, "missing source pad in external entity %s\n", + source_entity->name); + return -EINVAL; + } + + source_pad_index = ret; + + dev_dbg(dev, "creating %s:%u -> %s:%u link\n", source_entity->name, + source_pad_index, sink_entity->name, sink_pad_index); + + ret = media_create_pad_link(source_entity, source_pad_index, + sink_entity, sink_pad_index, + enabled ? MEDIA_LNK_FL_ENABLED : 0); + if (ret < 0) { + dev_err(dev, "failed to create %s:%u -> %s:%u link\n", + source_entity->name, source_pad_index, + sink_entity->name, sink_pad_index); + return ret; + } + + return 0; +} + +static int sun6i_isp_proc_notifier_bound(struct v4l2_async_notifier *notifier, + struct v4l2_subdev *remote_subdev, + struct v4l2_async_subdev *async_subdev) +{ + struct sun6i_isp_device *isp_dev = + container_of(notifier, struct sun6i_isp_device, proc.notifier); + struct sun6i_isp_proc_async_subdev *proc_async_subdev = + container_of(async_subdev, struct sun6i_isp_proc_async_subdev, + async_subdev); + struct sun6i_isp_proc *proc = &isp_dev->proc; + struct sun6i_isp_proc_source *source = proc_async_subdev->source; + bool enabled; + + switch (source->endpoint.base.port) { + case SUN6I_ISP_PORT_CSI0: + source = &proc->source_csi0; + enabled = true; + break; + case SUN6I_ISP_PORT_CSI1: + source = &proc->source_csi1; + enabled = !proc->source_csi0.expected; + break; + default: + break; + } + + source->subdev = remote_subdev; + + return sun6i_isp_proc_link(isp_dev, SUN6I_ISP_PROC_PAD_SINK_CSI, + remote_subdev, enabled); +} + +static int +sun6i_isp_proc_notifier_complete(struct v4l2_async_notifier *notifier) +{ + struct sun6i_isp_device *isp_dev = + container_of(notifier, struct sun6i_isp_device, proc.notifier); + struct v4l2_device *v4l2_dev = &isp_dev->v4l2.v4l2_dev; + int ret; + + ret = v4l2_device_register_subdev_nodes(v4l2_dev); + if (ret) + return ret; + + return 0; +} + +static const struct v4l2_async_notifier_operations +sun6i_isp_proc_notifier_ops = { + .bound = sun6i_isp_proc_notifier_bound, + .complete = sun6i_isp_proc_notifier_complete, +}; + +/* Processor */ + +static int sun6i_isp_proc_source_setup(struct sun6i_isp_device *isp_dev, + struct sun6i_isp_proc_source *source, + u32 port) +{ + struct device *dev = isp_dev->dev; + struct v4l2_async_notifier *notifier = &isp_dev->proc.notifier; + struct v4l2_fwnode_endpoint *endpoint = &source->endpoint; + struct sun6i_isp_proc_async_subdev *proc_async_subdev; + struct fwnode_handle *handle = NULL; + int ret; + + handle = fwnode_graph_get_endpoint_by_id(dev_fwnode(dev), port, 0, 0); + if (!handle) + return -ENODEV; + + ret = v4l2_fwnode_endpoint_parse(handle, endpoint); + if (ret) + goto complete; + + proc_async_subdev = + v4l2_async_nf_add_fwnode_remote(notifier, handle, + struct + sun6i_isp_proc_async_subdev); + if (IS_ERR(proc_async_subdev)) { + ret = PTR_ERR(proc_async_subdev); + goto complete; + } + + proc_async_subdev->source = source; + + source->expected = true; + +complete: + fwnode_handle_put(handle); + + return ret; +} + +int sun6i_isp_proc_setup(struct sun6i_isp_device *isp_dev) +{ + struct device *dev = isp_dev->dev; + struct sun6i_isp_proc *proc = &isp_dev->proc; + struct v4l2_device *v4l2_dev = &isp_dev->v4l2.v4l2_dev; + struct v4l2_async_notifier *notifier = &proc->notifier; + struct v4l2_subdev *subdev = &proc->subdev; + struct media_pad *pads = proc->pads; + int ret; + + mutex_init(&proc->lock); + + /* V4L2 Subdev */ + + v4l2_subdev_init(subdev, &sun6i_isp_proc_subdev_ops); + strscpy(subdev->name, SUN6I_ISP_PROC_NAME, sizeof(subdev->name)); + subdev->flags |= V4L2_SUBDEV_FL_HAS_DEVNODE; + subdev->owner = THIS_MODULE; + subdev->dev = dev; + + v4l2_set_subdevdata(subdev, isp_dev); + + /* Media Entity */ + + subdev->entity.function = MEDIA_ENT_F_PROC_VIDEO_ISP; + subdev->entity.ops = &sun6i_isp_proc_entity_ops; + + /* Media Pads */ + + pads[SUN6I_ISP_PROC_PAD_SINK_CSI].flags = MEDIA_PAD_FL_SINK | + MEDIA_PAD_FL_MUST_CONNECT; + pads[SUN6I_ISP_PROC_PAD_SINK_PARAMS].flags = MEDIA_PAD_FL_SINK | + MEDIA_PAD_FL_MUST_CONNECT; + pads[SUN6I_ISP_PROC_PAD_SOURCE].flags = MEDIA_PAD_FL_SOURCE; + + ret = media_entity_pads_init(&subdev->entity, SUN6I_ISP_PROC_PAD_COUNT, + pads); + if (ret) + return ret; + + /* V4L2 Subdev */ + + ret = v4l2_device_register_subdev(v4l2_dev, subdev); + if (ret < 0) { + v4l2_err(v4l2_dev, "failed to register v4l2 subdev: %d\n", ret); + goto error_media_entity; + } + + /* V4L2 Async */ + + v4l2_async_nf_init(notifier); + notifier->ops = &sun6i_isp_proc_notifier_ops; + + sun6i_isp_proc_source_setup(isp_dev, &proc->source_csi0, + SUN6I_ISP_PORT_CSI0); + sun6i_isp_proc_source_setup(isp_dev, &proc->source_csi1, + SUN6I_ISP_PORT_CSI1); + + ret = v4l2_async_nf_register(v4l2_dev, notifier); + if (ret) { + v4l2_err(v4l2_dev, + "failed to register v4l2 async notifier: %d\n", ret); + goto error_v4l2_async_notifier; + } + + return 0; + +error_v4l2_async_notifier: + v4l2_async_nf_cleanup(notifier); + + v4l2_device_unregister_subdev(subdev); + +error_media_entity: + media_entity_cleanup(&subdev->entity); + + return ret; +} + +void sun6i_isp_proc_cleanup(struct sun6i_isp_device *isp_dev) +{ + struct v4l2_async_notifier *notifier = &isp_dev->proc.notifier; + struct v4l2_subdev *subdev = &isp_dev->proc.subdev; + + v4l2_async_nf_unregister(notifier); + v4l2_async_nf_cleanup(notifier); + + v4l2_device_unregister_subdev(subdev); + media_entity_cleanup(&subdev->entity); +} diff --git a/drivers/staging/media/sunxi/sun6i-isp/sun6i_isp_proc.h b/drivers/staging/media/sunxi/sun6i-isp/sun6i_isp_proc.h new file mode 100644 index 000000000000..c5c274e21ad5 --- /dev/null +++ b/drivers/staging/media/sunxi/sun6i-isp/sun6i_isp_proc.h @@ -0,0 +1,66 @@ +/* SPDX-License-Identifier: GPL-2.0+ */ +/* + * Copyright 2021-2022 Bootlin + * Author: Paul Kocialkowski <[email protected]> + */ + +#ifndef _SUN6I_ISP_PROC_H_ +#define _SUN6I_ISP_PROC_H_ + +#include <media/v4l2-device.h> +#include <media/v4l2-fwnode.h> + +#define SUN6I_ISP_PROC_NAME "sun6i-isp-proc" + +enum sun6i_isp_proc_pad { + SUN6I_ISP_PROC_PAD_SINK_CSI = 0, + SUN6I_ISP_PROC_PAD_SINK_PARAMS = 1, + SUN6I_ISP_PROC_PAD_SOURCE = 2, + SUN6I_ISP_PROC_PAD_COUNT = 3, +}; + +struct sun6i_isp_device; + +struct sun6i_isp_proc_format { + u32 mbus_code; + u8 input_format; + u8 input_yuv_seq; +}; + +struct sun6i_isp_proc_source { + struct v4l2_subdev *subdev; + struct v4l2_fwnode_endpoint endpoint; + bool expected; +}; + +struct sun6i_isp_proc_async_subdev { + struct v4l2_async_subdev async_subdev; + struct sun6i_isp_proc_source *source; +}; + +struct sun6i_isp_proc { + struct v4l2_subdev subdev; + struct media_pad pads[3]; + struct v4l2_async_notifier notifier; + struct v4l2_mbus_framefmt mbus_format; + struct mutex lock; /* Mbus format lock. */ + + struct sun6i_isp_proc_source source_csi0; + struct sun6i_isp_proc_source source_csi1; +}; + +/* Helpers */ + +void sun6i_isp_proc_dimensions(struct sun6i_isp_device *isp_dev, + unsigned int *width, unsigned int *height); + +/* Format */ + +const struct sun6i_isp_proc_format *sun6i_isp_proc_format_find(u32 mbus_code); + +/* Proc */ + +int sun6i_isp_proc_setup(struct sun6i_isp_device *isp_dev); +void sun6i_isp_proc_cleanup(struct sun6i_isp_device *isp_dev); + +#endif diff --git a/drivers/staging/media/sunxi/sun6i-isp/sun6i_isp_reg.h b/drivers/staging/media/sunxi/sun6i-isp/sun6i_isp_reg.h new file mode 100644 index 000000000000..83b9cdab2134 --- /dev/null +++ b/drivers/staging/media/sunxi/sun6i-isp/sun6i_isp_reg.h @@ -0,0 +1,275 @@ +/* SPDX-License-Identifier: GPL-2.0+ */ +/* + * Copyright 2021-2022 Bootlin + * Author: Paul Kocialkowski <[email protected]> + */ + +#ifndef _SUN6I_ISP_REG_H_ +#define _SUN6I_ISP_REG_H_ + +#include <linux/kernel.h> + +#define SUN6I_ISP_ADDR_VALUE(a) ((a) >> 2) + +/* Frontend */ + +#define SUN6I_ISP_SRC_MODE_DRAM 0 +#define SUN6I_ISP_SRC_MODE_CSI(n) (1 + (n)) + +#define SUN6I_ISP_FE_CFG_REG 0x0 +#define SUN6I_ISP_FE_CFG_EN BIT(0) +#define SUN6I_ISP_FE_CFG_SRC0_MODE(v) (((v) << 8) & GENMASK(9, 8)) +#define SUN6I_ISP_FE_CFG_SRC1_MODE(v) (((v) << 16) & GENMASK(17, 16)) + +#define SUN6I_ISP_FE_CTRL_REG 0x4 +#define SUN6I_ISP_FE_CTRL_SCAP_EN BIT(0) +#define SUN6I_ISP_FE_CTRL_VCAP_EN BIT(1) +#define SUN6I_ISP_FE_CTRL_PARA_READY BIT(2) +#define SUN6I_ISP_FE_CTRL_LUT_UPDATE BIT(3) +#define SUN6I_ISP_FE_CTRL_LENS_UPDATE BIT(4) +#define SUN6I_ISP_FE_CTRL_GAMMA_UPDATE BIT(5) +#define SUN6I_ISP_FE_CTRL_DRC_UPDATE BIT(6) +#define SUN6I_ISP_FE_CTRL_DISC_UPDATE BIT(7) +#define SUN6I_ISP_FE_CTRL_OUTPUT_SPEED_CTRL(v) (((v) << 16) & GENMASK(17, 16)) +#define SUN6I_ISP_FE_CTRL_VCAP_READ_START BIT(31) + +#define SUN6I_ISP_FE_INT_EN_REG 0x8 +#define SUN6I_ISP_FE_INT_EN_FINISH BIT(0) +#define SUN6I_ISP_FE_INT_EN_START BIT(1) +#define SUN6I_ISP_FE_INT_EN_PARA_SAVE BIT(2) +#define SUN6I_ISP_FE_INT_EN_PARA_LOAD BIT(3) +#define SUN6I_ISP_FE_INT_EN_SRC0_FIFO BIT(4) +#define SUN6I_ISP_FE_INT_EN_SRC1_FIFO BIT(5) +#define SUN6I_ISP_FE_INT_EN_ROT_FINISH BIT(6) +#define SUN6I_ISP_FE_INT_EN_LINE_NUM_START BIT(7) + +#define SUN6I_ISP_FE_INT_STA_REG 0xc +#define SUN6I_ISP_FE_INT_STA_CLEAR 0xff +#define SUN6I_ISP_FE_INT_STA_FINISH BIT(0) +#define SUN6I_ISP_FE_INT_STA_START BIT(1) +#define SUN6I_ISP_FE_INT_STA_PARA_SAVE BIT(2) +#define SUN6I_ISP_FE_INT_STA_PARA_LOAD BIT(3) +#define SUN6I_ISP_FE_INT_STA_SRC0_FIFO BIT(4) +#define SUN6I_ISP_FE_INT_STA_SRC1_FIFO BIT(5) +#define SUN6I_ISP_FE_INT_STA_ROT_FINISH BIT(6) +#define SUN6I_ISP_FE_INT_STA_LINE_NUM_START BIT(7) + +/* Only since sun9i-a80-isp. */ +#define SUN6I_ISP_FE_INT_LINE_NUM_REG 0x18 +#define SUN6I_ISP_FE_ROT_OF_CFG_REG 0x1c + +/* Buffers/tables */ + +#define SUN6I_ISP_REG_LOAD_ADDR_REG 0x20 +#define SUN6I_ISP_REG_SAVE_ADDR_REG 0x24 + +#define SUN6I_ISP_LUT_TABLE_ADDR_REG 0x28 +#define SUN6I_ISP_DRC_TABLE_ADDR_REG 0x2c +#define SUN6I_ISP_STATS_ADDR_REG 0x30 + +/* SRAM */ + +#define SUN6I_ISP_SRAM_RW_OFFSET_REG 0x38 +#define SUN6I_ISP_SRAM_RW_DATA_REG 0x3c + +/* Global */ + +#define SUN6I_ISP_MODULE_EN_REG 0x40 +#define SUN6I_ISP_MODULE_EN_AE BIT(0) +#define SUN6I_ISP_MODULE_EN_OBC BIT(1) +#define SUN6I_ISP_MODULE_EN_DPC_LUT BIT(2) +#define SUN6I_ISP_MODULE_EN_DPC_OTF BIT(3) +#define SUN6I_ISP_MODULE_EN_BDNF BIT(4) +#define SUN6I_ISP_MODULE_EN_AWB BIT(6) +#define SUN6I_ISP_MODULE_EN_WB BIT(7) +#define SUN6I_ISP_MODULE_EN_LSC BIT(8) +#define SUN6I_ISP_MODULE_EN_BGC BIT(9) +#define SUN6I_ISP_MODULE_EN_SAP BIT(10) +#define SUN6I_ISP_MODULE_EN_AF BIT(11) +#define SUN6I_ISP_MODULE_EN_RGB2RGB BIT(12) +#define SUN6I_ISP_MODULE_EN_RGB_DRC BIT(13) +#define SUN6I_ISP_MODULE_EN_TDNF BIT(15) +#define SUN6I_ISP_MODULE_EN_AFS BIT(16) +#define SUN6I_ISP_MODULE_EN_HIST BIT(17) +#define SUN6I_ISP_MODULE_EN_YUV_GAIN_OFFSET BIT(18) +#define SUN6I_ISP_MODULE_EN_YUV_DRC BIT(19) +#define SUN6I_ISP_MODULE_EN_TG BIT(20) +#define SUN6I_ISP_MODULE_EN_ROT BIT(21) +#define SUN6I_ISP_MODULE_EN_CONTRAST BIT(22) +#define SUN6I_ISP_MODULE_EN_SATU BIT(24) +#define SUN6I_ISP_MODULE_EN_SRC1 BIT(30) +#define SUN6I_ISP_MODULE_EN_SRC0 BIT(31) + +#define SUN6I_ISP_MODE_REG 0x44 +#define SUN6I_ISP_MODE_INPUT_FMT(v) ((v) & GENMASK(2, 0)) +#define SUN6I_ISP_MODE_INPUT_YUV_SEQ(v) (((v) << 3) & GENMASK(4, 3)) +#define SUN6I_ISP_MODE_OTF_DPC(v) (((v) << 16) & BIT(16)) +#define SUN6I_ISP_MODE_SHARP(v) (((v) << 17) & BIT(17)) +#define SUN6I_ISP_MODE_HIST(v) (((v) << 20) & GENMASK(21, 20)) + +#define SUN6I_ISP_INPUT_FMT_YUV420 0 +#define SUN6I_ISP_INPUT_FMT_YUV422 1 +#define SUN6I_ISP_INPUT_FMT_RAW_BGGR 4 +#define SUN6I_ISP_INPUT_FMT_RAW_RGGB 5 +#define SUN6I_ISP_INPUT_FMT_RAW_GBRG 6 +#define SUN6I_ISP_INPUT_FMT_RAW_GRBG 7 + +#define SUN6I_ISP_INPUT_YUV_SEQ_YUYV 0 +#define SUN6I_ISP_INPUT_YUV_SEQ_YVYU 1 +#define SUN6I_ISP_INPUT_YUV_SEQ_UYVY 2 +#define SUN6I_ISP_INPUT_YUV_SEQ_VYUY 3 + +#define SUN6I_ISP_IN_CFG_REG 0x48 +#define SUN6I_ISP_IN_CFG_STRIDE_DIV16(v) ((v) & GENMASK(10, 0)) + +#define SUN6I_ISP_IN_LUMA_RGB_ADDR0_REG 0x4c +#define SUN6I_ISP_IN_CHROMA_ADDR0_REG 0x50 +#define SUN6I_ISP_IN_LUMA_RGB_ADDR1_REG 0x54 +#define SUN6I_ISP_IN_CHROMA_ADDR1_REG 0x58 + +/* AE */ + +#define SUN6I_ISP_AE_CFG_REG 0x60 +#define SUN6I_ISP_AE_CFG_LOW_BRI_TH(v) ((v) & GENMASK(11, 0)) +#define SUN6I_ISP_AE_CFG_HORZ_NUM(v) (((v) << 12) & GENMASK(15, 12)) +#define SUN6I_ISP_AE_CFG_HIGH_BRI_TH(v) (((v) << 16) & GENMASK(27, 16)) +#define SUN6I_ISP_AE_CFG_VERT_NUM(v) (((v) << 28) & GENMASK(31, 28)) + +#define SUN6I_ISP_AE_SIZE_REG 0x64 +#define SUN6I_ISP_AE_SIZE_WIDTH(v) ((v) & GENMASK(10, 0)) +#define SUN6I_ISP_AE_SIZE_HEIGHT(v) (((v) << 16) & GENMASK(26, 16)) + +#define SUN6I_ISP_AE_POS_REG 0x68 +#define SUN6I_ISP_AE_POS_HORZ_START(v) ((v) & GENMASK(10, 0)) +#define SUN6I_ISP_AE_POS_VERT_START(v) (((v) << 16) & GENMASK(26, 16)) + +/* OB */ + +#define SUN6I_ISP_OB_SIZE_REG 0x78 +#define SUN6I_ISP_OB_SIZE_WIDTH(v) ((v) & GENMASK(13, 0)) +#define SUN6I_ISP_OB_SIZE_HEIGHT(v) (((v) << 16) & GENMASK(29, 16)) + +#define SUN6I_ISP_OB_VALID_REG 0x7c +#define SUN6I_ISP_OB_VALID_WIDTH(v) ((v) & GENMASK(12, 0)) +#define SUN6I_ISP_OB_VALID_HEIGHT(v) (((v) << 16) & GENMASK(28, 16)) + +#define SUN6I_ISP_OB_SRC0_VALID_START_REG 0x80 +#define SUN6I_ISP_OB_SRC0_VALID_START_HORZ(v) ((v) & GENMASK(11, 0)) +#define SUN6I_ISP_OB_SRC0_VALID_START_VERT(v) (((v) << 16) & GENMASK(27, 16)) + +#define SUN6I_ISP_OB_SRC1_VALID_START_REG 0x84 +#define SUN6I_ISP_OB_SRC1_VALID_START_HORZ(v) ((v) & GENMASK(11, 0)) +#define SUN6I_ISP_OB_SRC1_VALID_START_VERT(v) (((v) << 16) & GENMASK(27, 16)) + +#define SUN6I_ISP_OB_SPRITE_REG 0x88 +#define SUN6I_ISP_OB_SPRITE_WIDTH(v) ((v) & GENMASK(12, 0)) +#define SUN6I_ISP_OB_SPRITE_HEIGHT(v) (((v) << 16) & GENMASK(28, 16)) + +#define SUN6I_ISP_OB_SPRITE_START_REG 0x8c +#define SUN6I_ISP_OB_SPRITE_START_HORZ(v) ((v) & GENMASK(11, 0)) +#define SUN6I_ISP_OB_SPRITE_START_VERT(v) (((v) << 16) & GENMASK(27, 16)) + +#define SUN6I_ISP_OB_CFG_REG 0x90 +#define SUN6I_ISP_OB_HORZ_POS_REG 0x94 +#define SUN6I_ISP_OB_VERT_PARA_REG 0x98 +#define SUN6I_ISP_OB_OFFSET_FIXED_REG 0x9c + +/* BDNF */ + +#define SUN6I_ISP_BDNF_CFG_REG 0xcc +#define SUN6I_ISP_BDNF_CFG_IN_DIS_MIN(v) ((v) & GENMASK(7, 0)) +#define SUN6I_ISP_BDNF_CFG_IN_DIS_MAX(v) (((v) << 16) & GENMASK(23, 16)) + +#define SUN6I_ISP_BDNF_COEF_RB_REG 0xd0 +#define SUN6I_ISP_BDNF_COEF_RB(i, v) (((v) << (4 * (i))) & \ + GENMASK(4 * (i) + 3, 4 * (i))) + +#define SUN6I_ISP_BDNF_COEF_G_REG 0xd4 +#define SUN6I_ISP_BDNF_COEF_G(i, v) (((v) << (4 * (i))) & \ + GENMASK(4 * (i) + 3, 4 * (i))) + +/* Bayer */ + +#define SUN6I_ISP_BAYER_OFFSET0_REG 0xe0 +#define SUN6I_ISP_BAYER_OFFSET0_R(v) ((v) & GENMASK(12, 0)) +#define SUN6I_ISP_BAYER_OFFSET0_GR(v) (((v) << 16) & GENMASK(28, 16)) + +#define SUN6I_ISP_BAYER_OFFSET1_REG 0xe4 +#define SUN6I_ISP_BAYER_OFFSET1_GB(v) ((v) & GENMASK(12, 0)) +#define SUN6I_ISP_BAYER_OFFSET1_B(v) (((v) << 16) & GENMASK(28, 16)) + +#define SUN6I_ISP_BAYER_GAIN0_REG 0xe8 +#define SUN6I_ISP_BAYER_GAIN0_R(v) ((v) & GENMASK(11, 0)) +#define SUN6I_ISP_BAYER_GAIN0_GR(v) (((v) << 16) & GENMASK(27, 16)) + +#define SUN6I_ISP_BAYER_GAIN1_REG 0xec +#define SUN6I_ISP_BAYER_GAIN1_GB(v) ((v) & GENMASK(11, 0)) +#define SUN6I_ISP_BAYER_GAIN1_B(v) (((v) << 16) & GENMASK(27, 16)) + +/* WB */ + +#define SUN6I_ISP_WB_GAIN0_REG 0x140 +#define SUN6I_ISP_WB_GAIN0_R(v) ((v) & GENMASK(11, 0)) +#define SUN6I_ISP_WB_GAIN0_GR(v) (((v) << 16) & GENMASK(27, 16)) + +#define SUN6I_ISP_WB_GAIN1_REG 0x144 +#define SUN6I_ISP_WB_GAIN1_GB(v) ((v) & GENMASK(11, 0)) +#define SUN6I_ISP_WB_GAIN1_B(v) (((v) << 16) & GENMASK(27, 16)) + +#define SUN6I_ISP_WB_CFG_REG 0x148 +#define SUN6I_ISP_WB_CFG_CLIP(v) ((v) & GENMASK(11, 0)) + +/* Global */ + +#define SUN6I_ISP_MCH_SIZE_CFG_REG 0x1e0 +#define SUN6I_ISP_MCH_SIZE_CFG_WIDTH(v) ((v) & GENMASK(12, 0)) +#define SUN6I_ISP_MCH_SIZE_CFG_HEIGHT(v) (((v) << 16) & GENMASK(28, 16)) + +#define SUN6I_ISP_MCH_SCALE_CFG_REG 0x1e4 +#define SUN6I_ISP_MCH_SCALE_CFG_X_RATIO(v) ((v) & GENMASK(11, 0)) +#define SUN6I_ISP_MCH_SCALE_CFG_Y_RATIO(v) (((v) << 16) & GENMASK(27, 16)) +#define SUN6I_ISP_MCH_SCALE_CFG_WEIGHT_SHIFT(v) (((v) << 28) & GENMASK(31, 28)) + +#define SUN6I_ISP_SCH_SIZE_CFG_REG 0x1e8 +#define SUN6I_ISP_SCH_SIZE_CFG_WIDTH(v) ((v) & GENMASK(12, 0)) +#define SUN6I_ISP_SCH_SIZE_CFG_HEIGHT(v) (((v) << 16) & GENMASK(28, 16)) + +#define SUN6I_ISP_SCH_SCALE_CFG_REG 0x1ec +#define SUN6I_ISP_SCH_SCALE_CFG_X_RATIO(v) ((v) & GENMASK(11, 0)) +#define SUN6I_ISP_SCH_SCALE_CFG_Y_RATIO(v) (((v) << 16) & GENMASK(27, 16)) +#define SUN6I_ISP_SCH_SCALE_CFG_WEIGHT_SHIFT(v) (((v) << 28) & GENMASK(31, 28)) + +#define SUN6I_ISP_MCH_CFG_REG 0x1f0 +#define SUN6I_ISP_MCH_CFG_EN BIT(0) +#define SUN6I_ISP_MCH_CFG_SCALE_EN BIT(1) +#define SUN6I_ISP_MCH_CFG_OUTPUT_FMT(v) (((v) << 2) & GENMASK(4, 2)) +#define SUN6I_ISP_MCH_CFG_MIRROR_EN BIT(5) +#define SUN6I_ISP_MCH_CFG_FLIP_EN BIT(6) +#define SUN6I_ISP_MCH_CFG_STRIDE_Y_DIV4(v) (((v) << 8) & GENMASK(18, 8)) +#define SUN6I_ISP_MCH_CFG_STRIDE_UV_DIV4(v) (((v) << 20) & GENMASK(30, 20)) + +#define SUN6I_ISP_OUTPUT_FMT_YUV420SP 0 +#define SUN6I_ISP_OUTPUT_FMT_YUV422SP 1 +#define SUN6I_ISP_OUTPUT_FMT_YVU420SP 2 +#define SUN6I_ISP_OUTPUT_FMT_YVU422SP 3 +#define SUN6I_ISP_OUTPUT_FMT_YUV420P 4 +#define SUN6I_ISP_OUTPUT_FMT_YUV422P 5 +#define SUN6I_ISP_OUTPUT_FMT_YVU420P 6 +#define SUN6I_ISP_OUTPUT_FMT_YVU422P 7 + +#define SUN6I_ISP_SCH_CFG_REG 0x1f4 + +#define SUN6I_ISP_MCH_Y_ADDR0_REG 0x1f8 +#define SUN6I_ISP_MCH_U_ADDR0_REG 0x1fc +#define SUN6I_ISP_MCH_V_ADDR0_REG 0x200 +#define SUN6I_ISP_MCH_Y_ADDR1_REG 0x204 +#define SUN6I_ISP_MCH_U_ADDR1_REG 0x208 +#define SUN6I_ISP_MCH_V_ADDR1_REG 0x20c +#define SUN6I_ISP_SCH_Y_ADDR0_REG 0x210 +#define SUN6I_ISP_SCH_U_ADDR0_REG 0x214 +#define SUN6I_ISP_SCH_V_ADDR0_REG 0x218 +#define SUN6I_ISP_SCH_Y_ADDR1_REG 0x21c +#define SUN6I_ISP_SCH_U_ADDR1_REG 0x220 +#define SUN6I_ISP_SCH_V_ADDR1_REG 0x224 + +#endif diff --git a/drivers/staging/media/sunxi/sun6i-isp/uapi/sun6i-isp-config.h b/drivers/staging/media/sunxi/sun6i-isp/uapi/sun6i-isp-config.h new file mode 100644 index 000000000000..19c24c5fd322 --- /dev/null +++ b/drivers/staging/media/sunxi/sun6i-isp/uapi/sun6i-isp-config.h @@ -0,0 +1,43 @@ +/* SPDX-License-Identifier: ((GPL-2.0+ WITH Linux-syscall-note) OR MIT) */ +/* + * Allwinner A31 ISP Configuration + */ + +#ifndef _UAPI_SUN6I_ISP_CONFIG_H +#define _UAPI_SUN6I_ISP_CONFIG_H + +#include <linux/types.h> + +#define V4L2_META_FMT_SUN6I_ISP_PARAMS v4l2_fourcc('S', '6', 'I', 'P') /* Allwinner A31 ISP Parameters */ + +#define SUN6I_ISP_MODULE_BAYER (1U << 0) +#define SUN6I_ISP_MODULE_BDNF (1U << 1) + +struct sun6i_isp_params_config_bayer { + __u16 offset_r; + __u16 offset_gr; + __u16 offset_gb; + __u16 offset_b; + + __u16 gain_r; + __u16 gain_gr; + __u16 gain_gb; + __u16 gain_b; +}; + +struct sun6i_isp_params_config_bdnf { + __u8 in_dis_min; + __u8 in_dis_max; + + __u8 coefficients_g[7]; + __u8 coefficients_rb[5]; +}; + +struct sun6i_isp_params_config { + __u32 modules_used; + + struct sun6i_isp_params_config_bayer bayer; + struct sun6i_isp_params_config_bdnf bdnf; +}; + +#endif /* _UAPI_SUN6I_ISP_CONFIG_H */ diff --git a/drivers/staging/media/tegra-video/vi.c b/drivers/staging/media/tegra-video/vi.c index 9d46a36cc014..11dd142c98c5 100644 --- a/drivers/staging/media/tegra-video/vi.c +++ b/drivers/staging/media/tegra-video/vi.c @@ -1811,7 +1811,7 @@ static int tegra_vi_graph_parse_one(struct tegra_vi_channel *chan, } /* skip entities that are already processed */ - if (remote == dev_fwnode(vi->dev) || + if (device_match_fwnode(vi->dev, remote) || tegra_vi_graph_find_entity(chan, remote)) { fwnode_handle_put(remote); continue; diff --git a/include/media/videobuf2-core.h b/include/media/videobuf2-core.h index 3253bd2f6fee..4b6a9d2ea372 100644 --- a/include/media/videobuf2-core.h +++ b/include/media/videobuf2-core.h @@ -386,6 +386,12 @@ struct vb2_buffer { * the buffer contents will be ignored anyway. * @buf_cleanup: called once before the buffer is freed; drivers may * perform any additional cleanup; optional. + * @prepare_streaming: called once to prepare for 'streaming' state; this is + * where validation can be done to verify everything is + * okay and streaming resources can be claimed. It is + * called when the VIDIOC_STREAMON ioctl is called. The + * actual streaming starts when @start_streaming is called. + * Optional. * @start_streaming: called once to enter 'streaming' state; the driver may * receive buffers with @buf_queue callback * before @start_streaming is called; the driver gets the @@ -405,6 +411,10 @@ struct vb2_buffer { * callback by calling vb2_buffer_done() with either * %VB2_BUF_STATE_DONE or %VB2_BUF_STATE_ERROR; may use * vb2_wait_for_all_buffers() function + * @unprepare_streaming:called as counterpart to @prepare_streaming; any claimed + * streaming resources can be released here. It is + * called when the VIDIOC_STREAMOFF ioctls is called or + * when the streaming filehandle is closed. Optional. * @buf_queue: passes buffer vb to the driver; driver may start * hardware operation on this buffer; driver should give * the buffer back by calling vb2_buffer_done() function; @@ -432,8 +442,10 @@ struct vb2_ops { void (*buf_finish)(struct vb2_buffer *vb); void (*buf_cleanup)(struct vb2_buffer *vb); + int (*prepare_streaming)(struct vb2_queue *q); int (*start_streaming)(struct vb2_queue *q, unsigned int count); void (*stop_streaming)(struct vb2_queue *q); + void (*unprepare_streaming)(struct vb2_queue *q); void (*buf_queue)(struct vb2_buffer *vb); @@ -641,8 +653,10 @@ struct vb2_queue { u32 cnt_queue_setup; u32 cnt_wait_prepare; u32 cnt_wait_finish; + u32 cnt_prepare_streaming; u32 cnt_start_streaming; u32 cnt_stop_streaming; + u32 cnt_unprepare_streaming; #endif }; diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h index f91260e79ddc..e8c621ad8439 100644 --- a/include/uapi/linux/videodev2.h +++ b/include/uapi/linux/videodev2.h @@ -1781,6 +1781,8 @@ struct v4l2_ext_control { __u8 __user *p_u8; __u16 __user *p_u16; __u32 __user *p_u32; + __u32 __user *p_s32; + __u32 __user *p_s64; struct v4l2_area __user *p_area; struct v4l2_ctrl_h264_sps __user *p_h264_sps; struct v4l2_ctrl_h264_pps *p_h264_pps; |